Motion comparison method

Abstract

The present invention relates to a method for comparing sequences of movements performed by two people. Image sequences corresponding to the sequences of movements are temporally aligned. Several types of offsets between the images are calculated and used to establish a movement execution score indicating a degree of similarity in the two sequences of movements performed. The invention also relates to a device for implementing the method and comprising in particular image capture devices for recording the sequences of images as well as a computer calculation module for executing certain steps of the method.

Description

Title of the invention: Method for comparing movement Technical field

[0001] The present invention relates to a method for comparing movement as well as a device suitable for implementing this method. State of the art

[0002] Although algorithms for analyzing and comparing movements based on video recordings are now widespread, particularly in the world of video games, most of them only allow a movement performed by two different people to be compared based on a very limited number of parameters. Indeed, in many sports disciplines, several other factors must be taken into account in order for the comparison to be relevant to the discipline in question. In the field of dance, for example, factors such as the delay or advance of the dancer in relation to the tempo of the music can prove to be just as important as the position of the limbs.

[0003] In order for the comparison to be used for the improvement of the dancer's technique, for example in the case where one of the two people is a teacher and the other is a student, as many of these factors as possible must be able to be taken into account when comparing a movement of the student with the movement of the teacher.

[0004] Document US9358456 describes in particular a method for comparing movements between two players, consisting of displaying instructions to the first player so that he creates and performs a dance movement, recording the dance movement of the first player, generating a representation in the form of an icon of the first player performing his dance movement and displaying this icon to the second player so that he also performs this dance movement, recording the dance movement of the second player and comparing the two dance movements.

[0005] This method is thus limited to comparing the positions of a member of the first player with respect to a member of the second player. Brief summary of the invention

[0006] An object of the present invention is to provide a motion comparison method free from the limitations known in the prior art.

[0007] Another aim of the invention is to propose a method for comparing movement making it possible to take into account several factors relevant to the comparison.

[0008] Another aim of the invention is to propose a method of comparing movement making it possible to detail the comparison on particular areas of the body.

[0009] According to the invention, these aims are achieved in particular by means of a movement comparison method comprising the steps of: a. recovering a model sequence of images, each image representing a model individual performing a model sequence of movements; b. production, by means of a computer calculation module such as a computer, a mobile phone / smartphone, or any other device equipped with a calculator, of a model skeleton comprising, i. for at least three images of the model sequence of images, at least one model joint position of at least one joint of the model individual represented in each of the at least three images, ii. a model index corresponding to the position of the image in the model sequence and iii. a model time associated with the image; c. retrieving a user sequence of images, each image representing a user performing a user sequence of movements; d. production, by means of a computer device such as a computer, a mobile phone / smartphone, or any other device equipped with a calculator, of a user skeleton comprising, i. for at least three images of the user sequence of images, at least one user joint position of at least one joint of the user depicted in each of the at least three images, ii. a user index corresponding to the position of the image in the user sequence, iii. and a user time associated with the image; e. association with each of the at least three images of the model sequence and the at least one articulation of the model individual represented on the image, at least one image of the user sequence by applying a dynamic temporal deformation algorithm (also called registration) based on the difference between a model articulation position and a user articulation position; f. for each of the at least three images of the model sequence and the at least one articulation of the model individual represented on the image, selection of at least one reference image of the at least one associated image maximizing a difference between the model index and the user index; g. for at least one image of the model sequence and for at least one articulation of the model individual represented on the image, calculation of an offset between the image and the corresponding reference image; h. calculating a user movement execution score based on at least one offset, the score corresponding to a similarity between the model movement sequence and the user movement sequence.

[0010] The production step may comprise producing a skeletonization model file comprising, for each image of the model sequence of images, a model joint position for each joint of the model individual represented in the image, a model index corresponding to the position of the image in the model sequence and a model time associated with the image.

[0011] The retrieving step may comprise retrieving a user sequence of images, each image representing a user performing a user sequence of movements.

[0012] The producing step may comprise producing a user skeletonization file comprising, for each image in the user sequence of images, a user joint position for each joint of the user represented in the image, a user index corresponding to the position of the image in the user sequence, and a user time associated with the image.

[0013] The association step may comprise associating with each image of the model sequence and for each articulation of the model individual represented on the image, a set of images of the user sequence by applying a dynamic time warping algorithm based on the difference between the model articulation position and the user articulation position.

[0014] The selection step may comprise, for each image of the model sequence and for each articulation of the model individual represented on the image, the selection of a reference image from the set of images maximizing a difference between the model index and the user index.

[0015] The calculation step may comprise, for each image of the model sequence and for each articulation of the model individual represented on the image, the calculation of an offset between the image and the corresponding reference image.

[0016] The step of recovering the model sequence of images may comprise: • recording the model sequence of images using a first image capture device; and / or wherein the step of retrieving the user sequence of images comprises • recording the user sequence of images using a second image capture device.

[0017] An offset may be a time offset determined during a step comprising: • for the at least three images of the model sequence and for the at least one articulation of the model individual represented on the image, calculation of the time shift on the basis of the model time and the user time of the reference image associated with the image of the model sequence, the movement execution score being calculated based on the time lags.

[0018] An offset may be a spatial offset determined during a step comprising: • For the at least three images of the model sequence of images and for the at least one joint of the model individual represented in the image, calculate the spatial offset between the model position and the user position, the movement execution score being calculated on the basis of the spatial offsets.

[0019] An offset may be a speed offset determined during a step comprising: • for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented in the image, calculating a speed offset between an articulation speed of the model individual and an articulation speed of the user, the movement execution score being calculated based on the velocity offsets.

[0020] An offset may be an acceleration offset determined during a step comprising: • for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented in the image, calculating an acceleration offset between an articulation acceleration of the model individual and an articulation acceleration of the user, the movement execution score being calculated based on the acceleration offsets.

[0021] The spatial, velocity and acceleration offsets can be calculated for each image of the model sequence and for each joint of the model individual.

[0022] Spatial, velocity, and acceleration offsets can be calculated between the raw data of the model and user skeletonizations, but can also be calculated between mathematical transformations of this data, such as low-pass, high-pass, or band-pass filters, moving averages, or any other filter.

[0023] The step of calculating the execution score may comprise sub-steps of: • for at least one articulation of the model individual, calculate an articulation execution score based on the temporal, spatial, velocity and / or acceleration offsets; • calculate the execution score based on the articulation execution score.

[0024] The execution score may be a weighted average of articulation execution scores of a plurality of articulations of the model individual.

[0025] A weighting of the weighted average may be performed based on a movement type, movement difficulty and / or user level.

[0026] An offset may be a torsion offset determined during a step comprising: • for each image of the model sequence of images, determining: • a model torsion angle of a portion of the model individual's body represented in the image; and • a user twist angle of a portion of the user's body represented in the image; • calculate a torsion offset based on the model torsion angle and the user torsion angle; the movement execution score being further calculated based on the torsion offsets.

[0027] The model torsion angle can be formed by a straight line passing through points of the skeletonization model file representing hips of the model individual and a reference straight line corresponding to a horizontal straight line of a terrestrial reference frame, and in which the user torsion angle is formed by a straight line passing through points of the user skeletonization file representing hips of the user and the reference straight line.

[0028] The model torsion angle may be formed by a straight line passing through points in the skeletonization model file representing hips of the model individual and a straight line passing through points in the skeletonization file representing shoulders of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing hips of the user and a straight line passing through points in the user skeletonization file representing shoulders of the user.

[0029] The model torsion angle may be formed by a straight line passing through points in the skeletonization model file representing shoulders of the model individual and a straight line passing through the points in the skeletonization model file representing the middle of the segment joining ears and a nose of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line passing through the points in the user skeletonization file representing the middle of the segment joining ears and a nose of the user.

[0030] The model torsion angle may be formed by a straight line passing through points in the skeletonization model file representing a thumb and a little finger of the model individual and a straight line passing through points in the skeletonization file representing an elbow and a wrist of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing a thumb and a little finger of the user and a line passing through points in the user skeletonization file representing an elbow and a wrist of the user.

[0031] The model torsion angle may be formed by a straight line passing through points in the skeletonization model file representing a heel and a big toe of the model individual and a straight line passing through points in the skeletonization file representing hips of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing a heel and a big toe of the user and a straight line passing through points in the user skeletonization file representing hips of the user.

[0032] An offset may be a bending offset determined during a step comprising: • for each image in the model sequence of images, determine: • a model flexion angle of a portion of the model individual's body represented in the image; and • a user flexion angle of a portion of the user's body represented in the image; • calculate a flexion offset based on the model flexion angle and the user flexion angle; the movement execution score being further calculated based on flexion offsets.

[0033] The model flexion angle can be obtained by calculating the angle formed by a straight line passing through points in the skeletonization model file representing shoulders of the model individual and a straight line passing through points in the skeletonization file representing hips of the model individual, in a plane containing the points corresponding to the hips of the model individual and the middle of the segment joining the shoulders of the model individual, and in which the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line passing through points in the user skeletonization file representing hips of the user, in a plane containing the points corresponding to the hips of the user and the middle of the segment joining the shoulders of the user.

[0034] The model flexion angle can be obtained by calculating the angle formed by a straight line passing through points in the skeletonization model file representing shoulders of the model individual and a straight line orthogonal to the plane containing points in the skeletonization model file corresponding to ears and a nose of the model individual, in a plane containing the points corresponding to the shoulders of the model individual and in the middle of the segment joining points corresponding to the ears of the individual model, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line orthogonal to the plane containing points in the user skeletonization file corresponding to ears and a nose of the user, in a plane containing the points corresponding to the shoulders of the user and the midpoint of the segment joining points corresponding to the ears of the user.

[0035] The model flexion angle can be obtained by calculating the angle formed by a straight line passing through points in the skeletonization model file representing the middle of the segment joining shoulders of the model individual and the middle of the segment joining hips of the model individual, and a straight line orthogonal to the plane formed by points in the skeletonization model file representing ears and a nose of the model individual, in a plane orthogonal to the straight line passing through points in the skeletonization model file representing the shoulders of the model individual, and in which the user flexion angle is obtained by calculating the angle formed by a straight line passing through points in the user skeletonization file representing the middle of the segment joining shoulders of the user and the middle of the segment joining hips of the user,and a straight line orthogonal to the plane formed by points in the user skeletonization file representing the user's ears and nose, in a plane orthogonal to the straight line passing through points in the user skeletonization file representing the user's shoulders.

[0036] Torsion offsets can be calculated between the raw data of the model and user skeletonizations, but can also be calculated between mathematical transformations of this data, such as low-pass, high-pass or band-pass filters, moving averages, or any other filter.

[0037] These aims are also achieved by means of a motion comparison device comprising: • a first image capture device configured to capture a model sequence of images of a model individual performing a model sequence of movements; • a second image capture device configured to capture a user sequence of images of a user performing a user sequence of movements; and • a computer calculation module allowing the implementation of the method described above. The first and second devices can be the same device or two separate devices. Brief description of the figures

[0038] Examples of implementation of the invention are indicated in the description illustrated by the appended figures in which: • [Fig.l] illustrates an image sequence alignment using a dynamic time warping algorithm. • [Fig.2] illustrates the determination of a torso torsion angle. • [Fig.3] illustrates the determination of a head twist angle. • [Fig.4] illustrates the determination of an arm torsion angle. • [Fig.5] illustrates the determination of a leg torsion angle. • [Fig.6] illustrates the determination of a torso flexion angle. • [Fig.7] illustrates the determination of a lateral head flexion angle. • [Fig.8] illustrates the determination of a front-back flexion angle (nod) of the head. Example(s) of embodiment of the inventionBrief description of the figures

[0039] The method of the present invention aims to compare a movement or a series of movements of a model individual (or leader), typically a teacher or a reference person to a movement or a series of movements of a user (or follower), typically a student or a person wishing to improve the execution of the movement.

[0040] The method is suitable for many sports or activities based on a movement technique in which the progression of a practitioner is based at least in part on the acquisition of certain movements or movement sequences. For example, the improvement or learning of dance movements, martial arts, but also certain sports such as golf, skateboarding or figure skating or gymnastics, in which the importance of posture or rhythm is preponderant and essentially based on the reproduction (and therefore the adequacy) with certain movements. A comparable interest will also be noted for the learning and improvement of movements during industrial handling and manipulation activities to increase the safety, well-being and efficiency of employees.

[0041] In the context of the present invention, the characteristics, attributes, quantities, etc., associated with the model individual (respectively with the user) are designated by the expression “model” (respectively “user”), e.g. a “model movement” (respectively a “user movement”), a model image (respectively a “user image”), a model position (respectively a “user position”), etc. Certain occurrences of the terms “model” and “user” may be omitted when the context is sufficiently clear to simplify the reading of this text.

[0042] The first step of the method comprises retrieving a model sequence of images, each image representing the model individual performing a model sequence of movements.

[0043] An image sequence typically refers to a video recording comprising several images arranged chronologically. The image sequence may also be the result of editing, for example in order to include a series of particular movements.

[0044] According to one embodiment, retrieving a model sequence of images involves uploading or downloading a digital video file. This file may be stored on a dedicated server or on the memory of a device of the model individual such as a smartphone, a tablet, a computer, etc.

[0045] Alternatively or additionally, the recovery of the model sequence of images may also comprise the actual recording of the sequence.

[0046] According to one embodiment, the model individual records using an image capture device (camera, smartphone, tablet, etc.) and transmits the recording to a dedicated server.

[0047] Based on this model sequence of images, a skeletonization file comprising for each image of the sequence, a model joint position for each joint of the model individual represented on the image, a model index corresponding to the position of the image in the model sequence and a model time associated with the image, is produced.

[0048] Many file formats can be used for the skeleton file such as "Comma Separated Value" .csv, "pickle" .pki or .pickle, "parquet" .parquet, "feather" .fea or .feather, "Avro", text ".txt", "ORC" or be stored in a database in another format. The file can be stored in a database.

[0049] The skeletonization file is produced by a skeletonization algorithm making it possible to record the three-dimensional position of joints of the model individual represented on each image of the sequence. The skeletonization can be carried out from images representing the joints in 2D or 3D.

[0050] In addition to the position of the joints, a model index designating the position of the image within the sequence is associated with each image. As detailed below, the possibility of being able to order the images of the sequence is essential to the implementation of the method.

[0051] Finally, a model time associated with the image corresponding to the time interval elapsed since the first image of the model sequence is also included for each image in the skeletonization file.

[0052] Steps of recovering a sequence of images and producing a skeletonization file are also carried out for the user.

[0053] More particularly, the method comprises a step including the recovery of a user sequence of images, each image representing the user performing a user sequence of movements. The user sequence of movements typically corresponding to a model sequence of movements to which the user wishes to compare his own.

[0054] As in the case of the model individual, the retrieval of the user sequence of images may include the uploading or downloading of a digital video file. This file may be stored on a dedicated server or on the memory of a user device such as a smartphone, a tablet, a computer, etc.

[0055] Alternatively or additionally, the recovery of the user sequence of images may also comprise the actual recording of the sequence.

[0056] According to one embodiment, the user records using an image capture device (camera, smartphone, tablet, etc.) and transmits the recording to a dedicated server.

[0057] A user file for skeletonizing the user sequence is then produced, the file comprising for each image of the sequence, a user joint position for each joint of the user represented on the image, a user index corresponding to the position of the image in the user sequence and a user time associated with the image, is produced.

[0058] The above considerations concerning the format of the skeletonization model file and the skeletonization algorithm are also applicable to the format of the skeletonization user file and to the skeletonization algorithm.

[0059] The two skeletonization files can then be compared by starting with a step comprising the association with each image of the model sequence and for each articulation of the model individual represented on the image, a set of images of the user sequence by applying a dynamic time warping or recalibration algorithm (“Dynamic Time Warping” or DTW) based on the difference between the model articulation position and the user articulation position.

[0060] Dynamic time warping algorithms aim to enable the comparison of two time series despite possible speed variations between the series.

[0061] According to an embodiment illustrated in [Fig. 1], this algorithm involves creating a matrix with the positions of a model joint on the abscissa, and the positions of the corresponding user joint on the ordinate. The interior of the matrix is ​​made up of the distances between each point of the sequences of positions with all the points of the other sample. The distance between the points is a formula based on the Euclidean distance combined with the distance from the diagonal.

[0062] As a non-limiting illustration, if we set L, the sequence of positions of the shoulder of the model individual, image after image up to a final image "m" lm and F, the sequence of positions of the shoulder of the user, image after image, until the final image "n": F = f? fn and if we set also d(L j) the Euclidean distance (eg measured in meters) between the shoulder of the model individual on image j and that of the user at image j, we then write the distance to be entered in the matrix between the shoulder of the leader at frame j and that of the follower at frame j:D(if j) -d(i, j) + min{D(i- 1, j)D(i- 1, j- V)D(i, j- 1)}.

[0063] The goal is to find a path to go from the top left point of the matrix to the bottom right while minimizing the sum of all the boxes we have passed through.

[0064] By indicating by W the path made up of all the boxes w: W — W], W^, Wk, then, all the paths respecting: • monotonicity: all the squares on the path always progress either downwards, or to the right, or both • continuity: the squares must touch, either by height, by width, or by the corner. • The boundary: it starts at the top left and ends at the bottom right, are eligible. The original algorithm consists of calculating all eligible paths and taking only the one that minimizes the sum of the boxes w, this path gives the agreement between the points of the sample. fw 1 DTW(L, F) = min [ —]

[0065] Calculating all the paths can quickly become very time-consuming and computationally intensive. It is therefore advisable to optimize the search for the diagonal to speed up the work. To do this, it is advantageous to use the principle of "divide and conquer": adding the calculation on small structures that make up the large structure often takes less time than a single calculation on a large structure.

[0066] This optimization consists of dividing the weight matrix in two and applying the same algorithm recursively to the two upper left and lower right matrices.

[0067] The first separation divides the matrix into two equal parts in the width direction. Then on the separation column, it finds, around the central point, the minimum. This minimum point must not be separated by more than the "permissible offset deviation", that is to say, the temporal difference that we estimate to be maximum between the movement of the model individual and that of the user. This minimum point therefore separates the matrix into four

[0068] The algorithm is then applied to the top left matrix as well as the bottom right one which will also be divided into four according to the same process and so on until arriving at a matrix whose width or height is less than or equal to 2. The paths are then found in this matrix. By going through it in its longest direction, we calculate the weight of each path which respects the classic rules of the DTW paths: • monotonicity: all the squares on the path always progress either downwards, or to the right, or both • continuity: the boxes must touch, either by height, by width, or by the corner. • The border: it starts at the top left and ends at the bottom right. The path with the minimum weight is retained for this matrix and will be concatenated to the paths of the other matrices.

[0069] Variants and / or optimizations of this dynamic time warping algorithm may be considered by those skilled in the art without departing from the scope of the present invention.

[0070] This step therefore makes it possible to associate with each image of the model sequence, a set (which may contain only one image in certain cases) of images of the user sequence which minimizes the difference (i.e. the error) in the position of the user joint relative to the position of the model joint. The association can be carried out using the model and user indices associated with each image of the model and user sequences. Indeed, it is thus possible to easily assign one or more user indices to a model index in order to associate the corresponding images.

[0071] This step is illustrated in the left graph of [Fig.l] in which: • some template images are associated with only one user image; • some template images are associated with multiple user images; • some user images are associated with multiple template images.

[0072] more precisely behind the images corresponding to model indices 4 and 5, then ahead of the images corresponding to model indices 7 to 10).

[0073] The method then comprises a step including the association with each image of the model sequence and for each articulation of the model individual, of a single user reference image among the images in the set determined in the previous step. This association is achieved by maximizing a difference between the template index and the user index. This means that among all the images in the user sequence of the set associated with an image in the template sequence, a reference image is selected based on a comparison of the user indices of the images in the set with the template index of the image in the template sequence.

[0074] The reference image is selected by keeping only the image of the set whose difference between the user index and the model index is the greatest. In other words, the selected reference image corresponds to the image of the set on which the user is the most advanced or the most delayed compared to the model individual, at least with regard to the articulation in question. If for an image of the model sequence, the user started early and finished late, if the delay was greater than the advance, it is the image corresponding to the delay that is selected, if it is the opposite, it is the image corresponding to the advance that will be kept. A new coordinated list is therefore created by keeping only the maximization of the differences between the images of the model sequence and the images of the user sequence.

[0075] This association of one and only one user image with each model image then makes it possible to calculate offsets which will serve as a basis for the comparison between the two sequences of movements. Several types of offsets can thus be determined in order to compare the two sequences of images according to several parameters.

[0076] As detailed below, the offset can be, for each joint, temporal, spatial, speed, acceleration, torsion, flexion, etc. The joints can also be grouped into limbs or body portions (eg arms, legs, torso and head, hips and shoulders, etc.) so that offsets can be calculated per limb or body portion.

[0077] Finally, the method comprises calculating a movement execution score taking into account the previously determined offsets. This score corresponds to a similarity between the model movement sequence and the user movement sequence.

[0078] Many types of scores can be calculated based on the data available as well as on the interests or instructions given by the model individual, the user or even an automatic learning program, etc. In addition, since the offsets are determined image by image as well as joint by joint, the execution score can also relate to a particular sub-sequence of images, to a particular joint, to a grouping of joints (e.g. a limb, the entire skeleton, etc.), to an aggregation or an average of several intermediate scores with possibly weightings introduced according to predetermined factors.

[0079] According to one embodiment, one or more steps of the method are adapted to be carried out consecutively via an API or a mobile application. Typically, the application can retrieve the model sequence by proposing to the model individual to upload a video recording to a dedicated server or directly to make a video recording via a camera integrated on a device (smartphone, tablet, computer equipped with a webcam, etc.), then execute a program allowing the creation of the corresponding skeletonization file. The application then stores the video recording and the skeletonization file on the server. A user of the application can then select the recording made available by the application and be offered to record themselves during the realization of their user movement sequence.The application then executes a program for comparing the model and user sequences as described above so as to generate an execution score and display it to the user on his device (smartphone, tablet, computer, etc.) so that the user can see the level of similarity of his movement sequence with that of the model individual. Improvement advice based on the execution score may for example also be offered by the application as well as suggestions for other recordings (e.g. more or less advanced courses, recordings corresponding to the user's interests, etc.).

[0080] As mentioned above, several types of offsets are possible per model image and per articulation.

[0081] According to one embodiment, a time shift is determined for each articulation between the images of the model sequence and their associated reference images. This time shift is calculated using the model time and the user time associated with each image of the two sequences in their respective skeletonization file. A time shift, for example in milliseconds, can thus be calculated by taking the difference between the two model and user times. The time shift can therefore correspond to the delay or advance of the position of a joint of the user represented on the reference image relative to the joint of the model individual represented on the image.

[0082] In the case where the movements must be carried out in rhythm with respect to music for example, knowledge of the tempo of the music can allow the measurement of the offset to be expressed as a percentage of bpm (bit per minute).

[0083] An average (to have the overall trend between delay and advance) and an absolute value average (so a delay and an advance do not cancel each other out) per image of the model sequence can be calculated for each articulation. The advantage of both types of averages is that the overall average can indicate a value close to zero: the user is generally on time, while he may have large delays at certain times and large advances at others. This disparity can thus be indicated by an average in absolute value. A large difference between the average and the absolute average therefore indicates this disparity. Conversely, the user could present a systematic delay, the average and the average in absolute value would therefore be relatively close. By articulation, an average and an average in absolute value can also be calculated on all the images of the model sequence.

[0084] The execution score can thus be calculated by averaging in absolute value all the time shifts on all the images of the model sequence and on all the articulations, optionally taking into account a possible weighting per image and / or per articulation. In cases where the execution score is determined solely on the basis of time shifts, the execution score can for example be expressed in milliseconds (or in seconds).

[0085] The execution score can also be given as a percentage. The value of the execution score, for example in milliseconds, can then be reported, depending on the available data: • to a known value of the tempo of a piece of music to which the movement sequence is to be performed. This value can, for example, be retrieved from a music sharing site, calculated on the fly by an algorithm or indicated by the model individual, • the overall duration of the model image sequence, • to the maximum deviation allowed when determining a path during the dynamic time warping algorithm as explained above.

[0086] Concrete examples of time shifts and execution scores calculated on the basis of these shifts are given below. Per image of the model sequence and per articulation, the time shift can be: • positive for an advance, negative for a delay in milliseconds; • positive for an advance, negative for a delay in percentage; Per image of the model sequence and for the entire skeleton, the time shift can be: • average giving the trend between advance and delay weighted according to the importance of the articulation in milliseconds; • in absolute average so that the delay and advance do not cancel each other out, this average also follows the weighting fixed by the choreography given in milliseconds; • average giving the trend between advance and delay weighted according to the importance of the articulation in percentage; • in absolute average so that the delays and advances do not cancel each other out, this average also follows the weighting fixed by the choreography given as a percentage. For the entire model sequence and per articulation, the time shift can be: • positive for an advance, negative for a delay in milliseconds; • positive for an advance, negative for a delay in percentage; For the entire model sequence and for the entire skeleton, the time lag can be: • average giving the trend between advance and delay weighted according to the importance of the articulation in milliseconds; • on absolute average so that delays and advances do not cancel each other out, this average may possibly follow a weighting fixed by a choreography.

[0087] For each model image - reference image pair it is therefore possible to calculate a time difference. However it is also advantageous to calculate a position difference. Indeed, it could be that the user is late, with a gesture consistent with that of the model individual, but it is also possible that the user is on time with a non-compliant gesture.

[0088] Thus, according to one embodiment, a spatial offset is calculated for each image of the model sequence of images and for each joint of the model individual represented on the image. This spatial offset corresponds to the distance between the model position and the user position of a joint. The movement execution score can thus be calculated on the basis of these spatial offsets.

[0089] For each image of the model sequence and for each joint, a spatial offset corresponding to a positioning error is calculated. This is a Euclidean distance in meters as well as a direction vector indicating the direction of the error.

[0090] For each image of the model sequence an average is calculated over all the joints, optionally taking into account a weighting by joint, by type of offset, etc. The Euclidean distances being unsigned, there is no need to calculate an average in absolute value.

[0091] For the entire model image sequence, an average can be given for each joint. An overall average can also be given based on a weighting by offset type.

[0092] A version of all these distances can also be calculated as percentages. In this case, the distances can be reduced to: • on one meter; • the distance of the joint from the center of the hip (i.e. from a point in the skeleton file corresponding to a hip); • the maximum distance between, for each joint, the point closest to the hips and the one furthest away. For example, for the hand, the closest point has a distance close to 0, the furthest from the hips has a distance of approximately lm20 (when the arm is stretched upwards, the maximum reference distance is then lm20). For the head, the minimum distance is around 50 cm (with the back bent to bring the head towards the stomach) to 80 cm, with the head stretched upwards, the maximum distance difference is therefore 0.3 m.

[0093] Concrete examples of spatial shifts and execution scores calculated on the basis of these shifts are given below. Per image of the model sequence and per articulation, the spatial shift may comprise: • the Euclidean distance in meters; • the distance in percentage; • the direction vector of the error; Per image of the model sequence and for the entire skeleton, the spatial shift can include: • a weighted average over all joints of the distances in meters; • a weighted average over all joints of the distances in percentage ; • the sum of all direction vectors and its normalized version; For the entire model and joint image sequence, the spatial shift may include: • an average of Euclidean distances in meters; • an average of the distances in percentage on all the images; • a sum of all direction vectors and its normalized version; For the entire model image sequence and for the entire skeleton the spatial shift can include: • a weighted average over all joints and all images of the model sequence of distances in meters; • a weighted average over all joints and all images of the model sequence of the percentage distances; • a sum of all direction vectors and its normalized version.

[0094] According to one embodiment, a speed offset is calculated for each image of the model sequence of images and for each joint of the model individual shown in the image. The movement execution score can thus be calculated based on these speed shifts.

[0095] Two versions of speed differences can be calculated: • the Euclidean distance for the same articulation between two images of the model sequence divided by the time between the two corresponding images. • a speed vector, composed of the vector difference between the position of the model individual's joint and that of the user divided by the time difference between successive images. This calculation also makes it possible to have the direction of the speed.

[0096] For the speed based on the difference in Euclidean distance, per image and per joint, a signed difference can be calculated in m / s, this is the difference in speed itself, without taking into account its orientation.

[0097] For the velocity based on the velocity vector difference, the direction vector can be calculated, as well as the magnitude of the vector difference, in m / s, hence the oriented velocity difference.

[0098] Per image of the model sequence, a sum of the direction vectors, an average of the magnitude of the weighted speed difference vector as well as the average and an average in absolute value of the difference of the weighted absolute speeds can be calculated. The sum of the direction vectors will make it possible to know if the whole body is moving in an erroneous way in the same direction, the average of the magnitude makes it possible to have the error of speed including the error of orientation of this one, the average of the absolute speed indicates if, overall, the speed is faster or slower, the average in absolute value of the absolute speed makes it possible to see if the distribution is homogeneous.

[0099] For video and per articulation the average over all images of the model sequence is proposed for the Euclidean speed, the magnitude of the speed difference vector as well as the sum of the unit vector of the speed error.

[0100] These differences can also be expressed as a percentage, reduced to the maximum speed per articulation of the model sequence.

[0101] Concrete examples of speed offsets and execution scores calculated based on these offsets are given below. Per frame of the model sequence and per joint, the speed offset may include: • a Euclidean speed difference in m / s; • a percentage difference in Euclidean speed; • a magnitude of the speed difference vector in m / s; • a magnitude of the velocity difference vector in percentage; • a direction vector of the offset (i.e. of the error); Per frame of the model sequence and for the entire skeleton, the speed shift can include: • a weighted average over all joints of the Euclidean speed differences in m / s; • a weighted average over all joints of the percentage differences in Euclidean speeds; • a weighted average over all joints of the magnitude of the velocity difference vector in m / s; • a weighted average across all joints of the magnitude of the velocity difference vector in percentage; • the sum of all direction vectors and its normalized version; For the entire model and joint image sequence, the velocity offset may include: • an average over all images of the Euclidean speed differences in m / s; • an average over all images of the percentage differences in Euclidean speeds; • an average over all images of the magnitude of the velocity difference vector in m / s; • an average over all images of the magnitude of the velocity difference vector in percentage; • the sum of all direction vectors and its normalized version; For the entire model image sequence and for the entire skeleton the speed shift can include: • a weighted average over all joints and all images of Euclidean speeds in m / s; • a weighted average across all joints and all images of the percentage Euclidean velocity differences; • a weighted average over all joints and all images of the magnitude of the velocity difference vector in m / s; • a weighted average over all joints and all images of the magnitude of the velocity difference vector in percentage; • the sum of all direction vectors and its normalized version.

[0102] According to one embodiment, an acceleration offset is calculated for each image of the model sequence of images and for each joint of the model individual represented in the image. The movement execution score can thus be calculated on the basis of these acceleration offsets.

[0103] Two versions of acceleration differences are calculated: • the Euclidean speed for the same articulation between two images of the model image sequence divided by the time between the two corresponding images. • the acceleration vector, composed of the vector difference between the vector velocity of the model individual's joint and that of the user divided by the time difference between successive images. This calculation also makes it possible to have the direction of the acceleration.

[0104] For the Euclidean acceleration based on the Euclidean velocity difference, per frame and per joint, a signed difference can be calculated in m / s2, this is the actual acceleration difference, without taking into account its orientation.

[0105] For the velocity based on the acceleration vector difference, the direction vector can be calculated, as well as the magnitude of the vector difference, in m / s2 hence the oriented acceleration difference.

[0106] Per image, a sum of the direction vectors, an average of the magnitude of the weighted acceleration difference vector as well as the average and an average in absolute value of the difference of the weighted absolute accelerations can be calculated and taken into account to calculate the acceleration offset. The sum of the direction vectors will make it possible to know if the whole body accelerates erroneously in the same direction, the average of the magnitude makes it possible to have the acceleration error including the error of orientation of this one, the average of the absolute acceleration indicates if, overall, this one is greater or smaller, the average in absolute value of the absolute acceleration makes it possible to see if the distribution is homogeneous.

[0107] For the entire model image sequence and per articulation the average over all images is proposed for the Euclidean acceleration, the magnitude of the acceleration difference vector as well as the sum of the unit vector of the acceleration error.

[0108] These differences can be expressed as a percentage, reduced to the maximum acceleration per articulation of the model sequence.

[0109] Concrete examples of acceleration offsets and execution scores calculated based on these offsets are given below. Per frame of the model sequence and per joint, the acceleration offset may include: • a difference in Euclidean acceleration in m / s2; • a percentage difference in Euclidean acceleration; • a magnitude of the acceleration difference vector in m / s2; • a magnitude of the acceleration difference vector in percentage; • a direction vector of the error; Per frame of the model sequence and for the entire skeleton, the acceleration offset can include: • a weighted average over all joints of the Euclidean accelerations in m / s2; • a weighted average over all joints of the differences in Euclidean accelerations in percentage; • a weighted average over all joints of the magnitude of the acceleration difference vector in m / s2: • a weighted average across all joints of the magnitude of the acceleration difference vector in percentage; • a sum of all direction vectors and its normalized version. For the entire model image sequence and per joint, the acceleration offset can include: • an average over all images of the differences in Euclidean accelerations in m / s2; • an average over all images of the Euclidean acceleration differences in percentage; • an average over all images of the magnitude of the acceleration difference vector in m / s2; • an average over all images of the magnitude of the acceleration difference vector in percentage; • the sum of all direction vectors and its normalized version. For the entire model image sequence and for the entire skeleton, the acceleration offset can include: • a weighted average across all joints and all images of the acceleration differences; • a weighted average over all joints and all images of the percentage differences in Euclidean accelerations; • a weighted average over all joints and all images of the magnitude of the acceleration difference vector in m / s2; • a weighted average over all joints and all images of the magnitude of the acceleration difference vector in percentage; • the sum of all direction vectors and its normalized version.

[0110] In addition to time, space, velocity and acceleration shifts, posture shifts (torsion, flexion, etc.) may also be calculated and factored into the movement execution score.

[0111] A torsion offset is determined based on angle differences between a position of the model individual and a position of the user. Thus, a torsion offset can be calculated for each image of the model image sequence by determining a model torsion angle of a body portion of the model individual represented in the image and a user torsion angle of a portion of the user's body represented in the image and corresponding to the body portion of the model individual. The calculation of the torsion offset is then based on the difference between the model torsion angle and the user torsion angle. The movement execution score can then include a torsion offset.

[0112] According to one embodiment, the torsion angle corresponds to a general orientation of the body. The model (resp. user) torsion angle is an orientation angle of the model individual (resp. user) relative to a fixed horizontal line, for example an axis of a terrestrial reference frame. The model and user torsion angles are compared to give an angular difference in degrees [°] or radians [rad]. They are also proposed as a percentage over 360°: • either 360° • either at the maximum amplitude of the general orientation of the model individual during the model image sequence.

[0113] The angular velocity of the orientation (in other words the rotation speed during a turn) of the model individual and the user are calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [7s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum torsion speed of the model individual during the model sequence.

[0114] The orientation angular acceleration of the model individual and the user is calculated by taking the difference of the orientation angular velocities divided by the time difference between the two images. The difference in angular acceleration is given in degrees per second squared [7s2] or radians per second squared [rand / s2]. They can also be given as a percentage, in which case they are reduced to the maximum rotation acceleration of the model individual during the model sequence.

[0115] According to an embodiment illustrated in [Fig.2], the torsion angle corresponds to a torsion of the bust of the model individual and of the user. The model (resp. user) torsion angle is typically the angle between the projection on a horizontal plane (i.e. on the ground) of a straight line dl passing through points of the model (resp. user) skeletonization file representing the hips of the model individual (resp. user) and the projection on a horizontal plane of a straight line d2 passing through points of the model (resp. user) skeletonization file representing the shoulders of the model individual (resp. user).

[0116] The model and user torsion angles are compared to give an angular difference in degrees [°] or radians [rad]. They can also be calculated as a percentage: • either at the maximum angle of torsion of the bust; • either at the maximum angle of torsion of the model individual during the sequence model.

[0117] The angular velocity of the model individual and the user is determined by calculating the difference in the model and user twist angles divided by the time difference between the two corresponding frames. The angular velocity difference is given in degrees per second [° / s] or radians per second [rand / s]. They can also be given as a percentage, in which case they are related to the maximum twist velocity of the model individual during the model sequence.

[0118] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum torsional acceleration of the model individual during the model sequence.

[0119] According to one embodiment, the torsion angle corresponds to a twisting of the head of the model individual and the user. The twisting of the head is typically measured relative to the shoulders. As illustrated in [Fig.3], the model (resp. user) torsion angle is the angle between: • a straight line d3 passing through the points of the model file (resp. user) of skeletonization representing the middle of the segment joining the two ears and the nose of the model individual (resp. of the user), and • a straight line d4 passing through points of the model file (resp. user) of skeletonization representing the shoulders of the model individual (resp. user); these two lines being projected into a plane orthogonal to a line d5 passing through the middle of the segment joining the shoulders and the middle of the segment joining the ears.

[0120] The torsion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum head torsion angle; • either at the maximum angle of head torsion of the model individual during the model sequence.

[0121] The angular velocity of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [° / s] or radians per second [rad / s]. They can also be given in percentage, in this case they are brought back to the maximum torsion speed of the model individual during the model sequence.

[0122] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum torsional acceleration of the model individual during the model sequence.

[0123] According to one embodiment, the torsion angle corresponds to an arm twist of the model individual and the user. The arm twist is typically measured by an invariant straight line when the arm enters into torsion. As illustrated in [Fig.4], the model (resp. user) torsion angle is the angle between: • a straight line d6 passing through points in the model file (resp. user) of skeletonization representing a thumb and a little finger of the model individual (resp. user); and • a straight line d7 passing through points of the model file (resp. user) of skeletonization representing a hip and a wrist of the model individual (resp. user).

[0124] The torsion angles (of the arm) of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of arm torsion • either at the maximum angle of arm torsion of the model individual during the model sequence.

[0125] The angular velocity of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [° / s] or radians per second [rand / s]. They can also be given as a percentage, in which case they are reduced to the maximum torsion velocity of the model individual during the model sequence.

[0126] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum torsional acceleration of the model individual during the model sequence.

[0127] According to one embodiment, the torsion angle corresponds to a leg twist of the model individual and the user. The leg twist is typically measured by an invariant straight line when the leg enters into torsion. As illustrated in [Fig.5], the model (resp. user) torsion angle is the angle between: • a straight line d8 passing through points of the model file (resp. user) of skeletonization representing a heel and a big toe of the model individual (resp. user); and • a straight line d9 passing through points in the model file (resp. user) of skeletonization representing the two hips of the model individual (resp. user), these two lines being projected into a horizontal plane (i.e. parallel to the ground or perpendicular to the vertical axis in a standard terrestrial reference frame).

[0128] The (leg) torsion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum leg twist angle • either at the maximum angle of leg torsion of the model individual during the model sequence.

[0129] The angular velocity of the model individual and the user is calculated by taking the difference of the torsion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [° / s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum torsion velocity of the model individual during the model sequence.

[0130] The torsional angular acceleration of the model individual and the user is calculated by taking the difference of the torsional angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum torsional acceleration of the model individual during the model sequence.

[0131] In addition to torsion offsets, flexion offsets between a body portion of the model individual and the corresponding body portion of the user may be calculated and taken into account in the movement execution score. Thus, a flexion offset may be calculated for each image of the model image sequence by determining a model flexion angle of a body portion of the model individual represented in the image and a user flexion angle of a body portion of the user represented in the image and corresponding to the body portion of the individual model. The calculation of the flexion offset is then based on the difference between the model flexion angle and the user flexion angle.

[0132] According to an embodiment illustrated in [Fig.6], the model (resp. user) flexion angle corresponds to a torso flexion angle. The model (resp. user) flexion angle is the angle between: • a straight line dlO passing through points of the model file (resp. user) of skeletonization representing the shoulders of the model individual (resp. user); and • a straight line dl 1 passing through points of the model skeleton file (resp. user) representing the hips of the model individual (resp. user), these lines being projected into the plane pl containing the points corresponding to the two hips and to the middle of the segment joining the shoulders of the model individual (resp. of the user).

[0133] The flexion angles (of the bust) of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also proposed as a percentage: • either at the maximum angle of torsion of the bust • either at the maximum angle of torsion of the model individual during the sequence of model images.

[0134] The angular velocity of the model individual and the user is calculated by taking the difference of the flexion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [° / s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum velocity of the model individual during the sequence of model images.

[0135] The flexion angular acceleration of the model individual and the user is calculated by taking the difference of the flexion angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum flexion acceleration of the model individual during the model image sequence.

[0136] With regard to the head, two types of flexion shift can also be calculated. The first corresponds to a lateral flexion of the head, and the second corresponds to head nods, i.e. a front-back flexion of the head.

[0137] According to an embodiment illustrated in [Fig.7], the model (resp. user) flexion angle corresponds to a lateral flexion angle of the head. The model (resp. user) flexion angle is the angle between: • a straight line dl2 passing through points of the model file (resp. user) of skeletonization representing the shoulders of the model individual (resp. user); and • a straight line dl3 orthogonal to the plane containing the points of the model skeletonization file (resp. user) representing the ears and nose of the model individual (resp. user), these lines being projected into the plane p2 containing the points corresponding to the two shoulders and to the middle of the segment joining the ears of the model individual (resp. of the user).

[0138] The head flexion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of flexion of the head; • either at the maximum angle of flexion of the model individual during the sequence of model images.

[0139] The angular velocity of the model individual and the user is calculated by taking the difference of the head flexion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [° / s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum velocity of the model individual during the sequence of model images.

[0140] The head flexion angular acceleration of the model individual and the user is calculated by taking the difference of the head flexion angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum head flexion acceleration of the model individual during the model image sequence.

[0141] According to an embodiment illustrated in [Fig.8], the model (resp. user) flexion angle corresponds to a front-back flexion angle of the head, i.e. a head nodding angle. The model (resp. user) flexion angle is then the angle between: • a straight line dl4 passing through points in the model (resp. user) skeleton file representing the middle of the segment joining the shoulders of the model individual (resp. of the user) and the middle of the segment joining the hips of the model individual (resp. of the user); and • a straight line dl5 orthogonal to the plane containing the points of the model file (resp. user) of skeletonization representing the ears and the nose of the model individual (resp. of the user), These lines being projected into the plane orthogonal to the line passing through the shoulders of the model individual (resp. of the user) and containing the midpoint of the segment joining the shoulders of the model individual (resp. of the user.

[0142] The head flexion angles of the model individual and the user are compared to give an angular difference in degrees [°] or radians [rad]. They are also offered as a percentage: • either at the maximum angle of head flexion (nodding); • either at the maximum angle of flexion (nodding) of the model individual during the model image sequence.

[0143] The angular velocity of the model individual and the user is calculated by taking the difference of the head flexion angles divided by the time difference between the two corresponding images. The difference in angular velocity is given in degrees per second [° / s] or radians per second [rad / s]. They can also be given as a percentage, in which case they are reduced to the maximum velocity of the model individual during the sequence of model images.

[0144] The head flexion angular acceleration of the model individual and the user is calculated by taking the difference of the head flexion angular velocities divided by the time difference between the two corresponding images. The difference in angular acceleration is given in degrees per second squared [° / s2] or radians per second squared [rad / s2]. They can also be given as a percentage, in which case they are reduced to the maximum head flexion acceleration of the model individual during the model image sequence.

[0145] The execution score may take into account one or more offsets of different types, per image, per total model image sequence, per joint, per group of joints or even per skeleton. Thus, the examples cited above may be taken independently or in combination when creating the execution score.

[0146] The calculation of the execution score may thus, for example, include intermediate calculations of articulation execution scores. Each articulation execution score depends on at least one type of offset (temporal, spatial, speed, acceleration, torsion, flexion, etc.). The articulation execution scores may then be aggregated to constitute the movement execution score.

[0147] In particular, weightings can be introduced according to particular parameters. These parameters can be defined, for example, in advance by the model individual, by the user, as part of a program for learning a sequence of movements or a discipline, etc. These parameters make it possible in particular to accentuate or reduce the importance of certain offsets or certain joints or portions of the body.

[0148] According to one embodiment, the execution score is weighted between the joints. The weights of the weighting depend on parameters such as for example the type of movement / sport / arts, the difficulty of the movement and / or the educational choices of the model individual.

[0149] A weighting by movement type can be determined for known genres of movement by a panel of specialists indicating which joints are most important for their disciplines. For example, Indian dance specialists might indicate that finger positions have a high weighting, which would be practically zero for running, or even another dance style like hip-hop. Conversely, these latter types of movements would have a higher weighting on the legs. There would therefore be a weight per joint, applied when the model individual chooses the movement type manually or when software (such as an AI) detects the movement type. When the genre is not known or not specified, the weights are all equal by default.

[0150] Within the same type of movement, there may be movements for beginners, advanced or experts. Or the same movements with beginner, advanced or expert expectations. In this case, per type of movement, the expert panel will designate a distribution of weightings according to the articulations. For example, in Latin dance, beginners focus at the beginning on their steps and the positioning of their feet, advanced dancers add arms, experts, rib cage movements. The weighting of all these articulations will evolve in the same way.

[0151] When a model individual wishes to propose a movement sequence focusing on a particular point or technique of his discipline, he can assign a particular weighting to the limbs he proposes to work on. For example, a traditional Chinese dance teacher may wish to make three videos: the first focusing on the feet and legs, the second on the arms, the third mixing the two previous ones. In this case the model individual will set a higher weighting on the legs and almost zero on the stockings for the first video, an inverse weighting for the second video and a weighting closer to his type of movement for the third. A model individual can also choose to make all his videos on his technical specialty, he will then choose a basic weighting for all his videos.

[0152] Alternatively or additionally, the weighting may also depend on the type of shift.

[0153] For example, in a tap dance choreography, the weighting of time would be more important, in martial arts disciplines, position would be the most important. For known genres of movement, a panel of specialists can indicate which are the most important time components in their discipline. For unknown genres, a generic weighting will be proposed.

[0154] According to one embodiment, a weighting is adapted according to the difficulty of movement. Thus, for a beginner user, the execution score can give more importance to the position offset and minimize the importance of the other offsets. For an advanced user, the speed offsets (the speed distribution) can constitute a significant part of the execution score, whereas for an expert user, the acceleration offsets will be predominant in the calculation of the execution score.

[0155] According to one embodiment, a weighting is adapted according to a type of sequence or the model individual. A sequence of images explaining the principle of a turn in classical dance would have every interest in increasing the importance of the orientation shifts, the orientation speed, or even the acceleration of the turn for the experts. Similarly, a model individual only presenting sequences of images on a specific technical point might wish for a particular weighting relevant to the technical point in question to be applied to all of its sequences.

[0156] As mentioned above, the calculation of the execution score can be carried out from intermediate execution scores (for example articulation execution scores). In order for these scores to be aggregated, it is advantageous to express them as a percentage of error in order to be able to mix the different errors according to their units of measurement.

[0157] The error averages (in time, position, speed, acceleration) weighted on the joints are done according to the method explained above for the limbs and for the skeleton.

[0158] The combinations of errors between • speed and acceleration • time and position • twists and orientations • time, position, velocity, acceleration, torsion and orientation can be performed according to one of the weightings described above. An execution score can thus be given as a percentage, it is equivalent to 100% if there is no no error and equal to score - 1QQ% _ e. where pi is the weight associated with the error ei.

[0159] As concrete examples, the following execution scores can be calculated (per image of the model sequence or for the entire model sequence, per joint, per limb and for the entire skeleton). • Per image of the model image sequence: • By joint: Time shift score, position score, velocity score, acceleration score, torsion / flexion score for affected limbs, torsion / flexion speed score for affected limbs, torsion / flexion acceleration score for affected limbs, combined velocity acceleration score, combined time / position / torsion score, overall combined score • Per limb: Time shift score across all weighted limb joints, position score across all weighted limb joints, velocity score across all weighted limb joints, acceleration score across all weighted limb joints, torsion / flexion score for affected limbs across all weighted limb joints, torsion / flexion velocity score for affected limbs across all weighted limb joints, torsion / flexion acceleration score for affected limbs across all weighted limb joints, combined velocity acceleration score across all weighted limb joints, combined time / position / torsion score across all weighted limb joints, overall combined score across all weighted limb joints • For the skeleton: Time shift score on all weighted joints, position score on all weighted joints, velocity score on all weighted joints, acceleration score on all weighted joints, orientation score, turn velocity score, turn acceleration score, torsion / flexion score for affected limbs on all weighted joints, torsion / flexion velocity score for affected limbs on all weighted joints, torsion / flexion acceleration score for affected limbs on all weighted joints, combined velocity acceleration score on all weighted limbs, combined time / position / torsion score on all weighted joints, overall combined score on all weighted joints. • For the entire model sequence:# • By joint: Time shift score, position score, velocity score, acceleration score, torsion / flexion score for affected limbs, torsion / flexion speed score for affected limbs, torsion / flexion acceleration score for affected limbs, combined velocity acceleration score, combined time / position / torsion score, overall combined score. • Per limb: Time shift score across all weighted limb joints, position score across all weighted limb joints, velocity score across all weighted limb joints, acceleration score across all weighted limb joints, torsion / flexion score for affected limbs across all weighted limb joints, torsion / flexion velocity score for affected limbs across all weighted limb joints, torsion / flexion acceleration score for affected limbs across all weighted limb joints, combined velocity acceleration score across all weighted limb joints, combined time / position / torsion score across all weighted limb joints, overall combined score across all weighted limb joints. • For the skeleton: Time shift score on all weighted joints, position score on all weighted joints, velocity score on all weighted joints, acceleration score on all weighted joints, orientation score, turn velocity score, turn acceleration score, torsion / flexion score for affected limbs on all weighted joints, torsion / flexion velocity score for affected limbs on all weighted joints, torsion / flexion acceleration score for affected limbs on all weighted joints, combined velocity acceleration score on all weighted limb joints, combined time / position / torsion score on all weighted joints, overall combined score on all weighted joints.

[0160] The present invention also relates to a movement comparison device comprising an image capture device configured to capture a model sequence of images of a model individual performing a model sequence of movements, an image capture device configured to capture a user sequence of images of a user performing a user sequence of movements, and a computer computing module for implementing the method described above.

[0161] As mentioned above, the first and second image capturing devices may include a smartphone, a tablet, a computer with a webcam, a camera, etc.

Claims

1. Claims A method of comparing motion comprising the steps of: • a) recovery of a model sequence of images, each image representing a model individual performing a model sequence of movements; • b) production, by means of a computer calculation module, of a model skeletonization comprising, • bl) for at least three images of the model sequence of images, at least one model articulation position of at least one articulation of the model individual represented on each of the at least three images, • b2) a model index corresponding to the position of each of the at least three images in the model sequence and • b3) a model time associated with each of the less images; • c) retrieving a user sequence of images, each image representing a user performing a user sequence of movements; • d) production, by means of a computer calculation module, of a user skeletonization comprising, • dl) for at least three images of the user sequence of images, at least one user articulation position of at least one articulation of the user represented on each of the at least three images, • d2) a user index corresponding to the position of each of the at least three images in the user sequence, • d3) and a user time associated with each of the at least three images; • e) associating to each of the at least three images of the model sequence and the at least one articulation of the model individual represented on the image, at least one image of the user sequence by applying a dynamic time warping algorithm based on the difference between a model joint position and a user joint position; • f) for each of the at least three images of the model sequence and the at least one joint of the model individual represented on the image, selecting at least one reference image of the at least one associated image maximizing a difference between the model index and the user index; • g) for at least one image of the model sequence and for at least one joint of the model individual represented on the image, calculating an offset between the image and the corresponding reference image, said offset being; a time offset, a spatial offset, a speed offset, an acceleration offset or a torsion offset;• h) calculating a user movement execution score based on the offset, the score corresponding to a similarity between the model movement sequence and the user movement sequence.;

2. The method of claim 1, wherein the step of retrieving the template sequence of images comprises: • recording the template sequence of images using an image capture device; and / or wherein the step of retrieving the user sequence of images comprises • recording the user sequence of images using an image capture device.

3. Method according to one of claims 1 to 2, an offset being a time offset determined during a step comprising: • for the at least three images of the model sequence and for the at least one articulation of the model individual represented on the image, calculation of the time offset on the basis of the model time and the user time of the reference image associated with the image of the model sequence, the movement execution score being calculated on the basis of the time offsets.

4. Method according to one of claims 1 to 3, an offset being a spatial offset determined during a step comprising: • for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented on the image, calculation of the spatial offset between the model position and the user position, the movement execution score being calculated on the basis of the spatial offsets.

5. Method according to one of claims 1 or 2, an offset being a speed offset determined during a step comprising: • for the at least three images of the model sequence of images and for the at least one articulation of the model individual represented on the image, calculating a speed offset between an articulation speed of the model individual and an articulation speed of the user, the movement execution score being further calculated on the basis of the speed offsets.

6. Method according to one of claims 1 or 2, an offset being an acceleration offset determined during a step comprising: • for the at least one image of the model sequence of images and for the at least one articulation of the model individual represented on the image, calculating an acceleration offset between an articulation acceleration of the model individual and an articulation acceleration of the user, the movement execution score being further calculated on the basis of the acceleration offsets.

7. Method according to one of claims 3, 4, 5 or 6, in which the step of calculating the execution score comprises sub-steps of: • for at least one articulation of the model individual, calculating an articulation execution score on the basis of the temporal, spatial, speed and / or acceleration offsets; calculating the execution score on the basis of the articulation execution score.

8. The method of claim 7, wherein the execution score is a weighted average of articulation execution scores of a plurality of articulations of the model individual.

9. A method according to claim 8, wherein a weighting of the weighted average is performed based on a movement type, movement difficulty and / or user level.

10. Method according to one of claims 1 or 2, an offset being a torsion offset determined during a step comprising: • for each image of the model sequence of images, determining: • a model torsion angle of a body portion of the model individual represented in the image; and • a user torsion angle of a body portion of the user represented in the image; • calculating a torsion offset on the basis of the model torsion angle and the user torsion angle; the movement execution score being further calculated on the basis of the torsion offsets.

11. Method according to claim 10, in which the model torsion angle is formed by a straight line passing through points of the skeletonization model file representing hips of the model individual and a reference straight line corresponding to a horizontal straight line of a terrestrial reference frame, and in which the user torsion angle is formed by a straight line passing through points of the user skeletonization file representing hips of the user and the reference straight line.

12. The method of claim 10, wherein the model torsion angle is formed by a straight line (dl) passing through points in the skeletonization model file representing hips of the model individual and a straight line (d2) passing through points in the skeletonization file representing shoulders of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing hips of the user and a straight line passing through points in the user skeletonization file representing shoulders of the user.

13. The method of claim 10, wherein the model twist angle is formed by a straight line (d4) passing through points in the skeletonization model file representing shoulders of the model individual and a straight line (d3) passing through the points in the skeletonization model file representing the middle of the segment joining ears and a nose of the model individual, and wherein the user twist angle is formed by a straight line passing through points in the user skeletonization file representing shoulders of the user and a straight line passing through the points in the user skeletonization file representing the middle of the segment joining ears and a nose of the user.

14. The method of claim 10, wherein the model torsion angle is formed by a straight line (d6) passing through points in the skeletonization model file representing a thumb and a little finger of the model individual and a straight line (d7) passing through points in the skeletonization file representing a hip and a wrist of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing a thumb and a little finger of the user and a straight line passing through points in the user skeletonization file representing a hip and a wrist of the user.

15. The method of claim 10, wherein the model torsion angle is formed by a straight line (d8) passing through points in the skeletonization model file representing a heel and a big toe of the model individual and a straight line (d9) passing through points in the skeletonization file representing hips of the model individual, and wherein the user torsion angle is formed by a straight line passing through points in the user skeletonization file representing a heel and a big toe of the user and a straight line passing through points in the user skeletonization file representing hips of the user.

16. Method according to one of claims 1 to 15, an offset being a flexion offset determined during a step comprising: • for each image of the model sequence of images, determining: • a model flexion angle of a portion of the body of the model individual represented in the image; and • a user flexion angle of a portion of the user's body represented in the image; • calculating a flexion offset based on the model flexion angle and the user flexion angle; The movement execution score being further calculated based on the flexion offsets.

17. The method of claim 16, wherein the model flexion angle is obtained by calculating the angle formed by a straight line (dl0) passing through points of the skeletonization model file representing shoulders of the model individual and a straight line (dl 1) passing through points of the skeletonization file representing hips of the model individual, in a plane (pl) containing the points corresponding to the hips of the model individual and the middle of the segment joining the shoulders of the model individual, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points of the user skeletonization file representing shoulders of the user and a straight line passing through points of the user skeletonization file representing hips of the user, in a plane containing the points corresponding to the hips of the user and the middle of the segment joining the shoulders of the user.

18. The method of claim 16, wherein the model flexion angle is obtained by calculating the angle formed by a straight line (dl2) passing through points of the skeletonization model file representing shoulders of the model individual and a straight line (dl3) orthogonal to the plane containing points of the skeletonization model file corresponding to ears to a nose of the model individual, in a plane (p2) containing the points corresponding to the shoulders of the model individual and to the middle of the segment joining points corresponding to the ears of the model individual, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points of the user skeletonization file representing shoulders of the user and a straight line orthogonal to the plane containing points of the user skeletonization file corresponding to ears to a nose of the user,in a plane containing the points corresponding to,

19.

20. user's shoulders and the middle of the segment joining points corresponding to the user's ears. The method of claim 16, wherein the model flexion angle is obtained by calculating the angle formed by a straight line (dl4) passing through points of the skeletonization model file representing the middle of the segment joining shoulders of the model individual and the middle of the segment joining hips of the model individual, and a straight line (dl5) orthogonal to the plane formed by points of the skeletonization model file representing ears and a nose of the model individual, in a plane orthogonal to the straight line passing through points of the skeletonization model file representing the shoulders of the model individual, and wherein the user flexion angle is obtained by calculating the angle formed by a straight line passing through points of the user skeletonization file representing the middle of the segment joining shoulders of the user and the middle of the segment joining hips of the user,and a straight line orthogonal to the plane formed by points of the user skeletonization file representing the user's ears and nose, in a plane orthogonal to the straight line passing through points of the user skeletonization file representing the user's shoulders. Motion comparison device comprising:, • an image capture device configured to capture a model sequence of images of a model individual performing a model sequence of movements; • an image capture device configured to capture a user sequence of images of a user performing a user sequence of movements: • a computer calculation module making it possible to implement the method according to claims 1 to 19.

Images