A method of assessing the movement of a human

Abstract

There is provided a method of assessing the movement of a human. The method comprises: obtaining a sequence of images of a human performing a movement; identifying in each image two or more primary biological features; assigning to each image primary data points indicative of the position of the two or more primary biological features within the image; calculating for each image a primary spatial measurement formed between the primary data points; calculating a primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and outputting a primary movement score that is dependent on the variance factor.

Description

[0001] Title - A method of assessing the movement of a human

[0002] The present invention relates to a method of assessing the movement of a human, and to a sensing apparatus for assessing the movement of a human.

[0003] Patients with musculoskeletal injuries or disorders are typically subject to an assessment by a healthcare professional, which enables the professional to understand the level of impairment of the patient, i.e. to determine how much their movement is restricted or affected by their injury or disorder.

[0004] Follow-up assessments are also conducted to determine whether rehabilitation and / or methods of treatment have reduced the level of impairment suffered by the patient over time.

[0005] A typical assessment may consist of the patient performing an exercise, and the healthcare professional assessing their performance of that exercise against a target level of performance. One example of such an exercise is the “sit to stand” exercise, whereby a patient starts in a seated position, stands up to a standing position, and then sits back down into the seated position.

[0006] Whilst assessing the performance of the patient, the healthcare professional will typically look at the balance, control, symmetry and speed shown by the patient during movement. These factors are compared with expected performance, and a level of impairment is determined accordingly.

[0007] Following rehabilitation and / or treatment, the patient will reperform the test, and the healthcare professional will reassess their performance (i.e. their balance, control, symmetry and speed shown during movement) to determine whether their movement has improved as a result of the rehabilitation and / or treatment. Further rehabilitation and / ore treatment can be prescribed accordingly.

[0008] However, this conventional method of assessment is qualitative in nature, and thus the significance of the patient’s impairment cannot be quantified, and nor can any improvement achieved as a result of the rehabilitation and / or treatment. This means that the assessment is entirely subjective, and depends largely on the healthcare professional doing the assessment. This also makes it difficult to effectively diagnose the effects of musculoskeletal injuries or conditions, and to prescribe effective treatments.

[0009] There has now been devised an improved method of assessing the movement of a human, and sensing apparatus for assessing the movement of a human, which overcome or substantially mitigate disadvantages associated with the prior art.

[0010] According to a first aspect of the invention, there is provided a method of assessing the movement of a human, the method comprising: obtaining a sequence of images of a human performing a movement; identifying in each image two or more primary biological features; assigning to each image primary data points indicative of the position of the two or more primary biological features within the image; calculating for each image a primary spatial measurement formed between the primary data points; calculating a primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and outputting a primary movement score that is dependent on the variance factor.

[0011] According to a further aspect of the invention there is provided a sensing apparatus configured to assess the movement of a human according to the method of the first aspect.

[0012] According to a further aspect of the invention there is provided a sensing apparatus for assessing the movement of a human, the sensing apparatus comprising: at least one sensor configured to capture a sequence of images of a human performing a movement; and at least one controller configured to: identify in each image two or more primary biological features; assign to each image primary data points indicative of the position of the two or more primary biological features within the image; calculate for each image a primary spatial measurement formed between the primary data points; calculate a primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and output a primary movement score that is dependent on the variance factor.

[0013] According to a further aspect of the invention there is provided a sensing apparatus for assessing the movement of a human, the sensing apparatus comprising at least one processor configured to perform the above-described method steps.

[0014] According to a further aspect of the invention there is provided a data carrier or data storage medium comprising machine readable instructions for the control of one or more processor to perform the above-described method steps.

[0015] According to further aspects of the invention there is provided one or more processors comprising machine readable instructions for performing or controlling any, any combination of, or all of the aforementioned method steps. According to further aspects of the invention there is provided a data carrier or data storage medium comprising machine readable instructions for controlling one or more processor to perform said step(s).

[0016] The method of assessing the movement of a human, and sensing apparatus for assessing the movement of a human, according to the aspects of the invention may be advantageous in that the assignment of data points to biological features within images, the subsequent calculation of a spatial measurement formed between those data points, and the calculation of a variance factor that is dependent on a change in the spatial measurement between at least two of the images, enables a quantitative movement score to be output. This in turn enables the significance of the patient’s impairment to be quantified, as well as any improvement achieved as a result of rehabilitation and / or treatment programs.

[0017] The primary spatial measurement may be a volume formed between the primary data points. The sequence of images may be a sequence of frames within a video.

[0018] The primary variance factor may be compared with at least one threshold value to determine the primary movement score. The primary variance factor may be compared with multiple threshold values.

[0019] The method may further comprise: calculating a further primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and outputting a further primary movement score that is dependent on the variance factor.

[0020] The further primary variance factor may be compared with at least one threshold value to determine the further primary movement score. The further primary variance factor may be compared with multiple threshold values.

[0021] The primary movement score may be indicative of a first aspect of the human’s movement and the further primary movement score may be indicative of a second different aspect of the human’s movement.

[0022] The primary variance factor or the further primary variance factor may be dependent on a change in the primary spatial measurement between two adjacent images.

[0023] The primary variance factor or the further primary variance factor may be dependent on a change in the primary spatial measurement across the full sequence of images. The primary variance factor or the further primary variance factor may be dependent on a net cumulative change in the primary spatial measurement across the full sequence of images.

[0024] The primary variance factor or the further primary variance factor may be dependent on a rate of change in the primary spatial measurement between at least two of the images. The primary variance factor or the further primary variance factor may be dependent on a rate of change in the primary spatial measurement between two adjacent images.

[0025] The primary variance factor or the further primary variance factor may be dependent on a rate of change in the primary spatial measurement across the full sequence of images.

[0026] The method may further comprise: outputting an overall primary movement score that is dependent on the primary movement score and the further primary movement score. Since the primary movement score may be indicative of a first aspect of the human’s movement and the further primary movement score may be indicative of a second different aspect of the human’s movement, the overall primary movement score may be indicative of various aspects of the human’s movement.

[0027] The method may further comprise: identifying in each image two or more secondary biological features; assigning to each image secondary data points indicative of the position of the two or more secondary biological features within the image; calculating for each image a secondary spatial measurement formed between the secondary data points; calculating a second variance factor that is dependent on a change in the second spatial measurement between at least two of the images; and outputting a second movement score that is dependent on the variance factor.

[0028] The secondary spatial measurement may be a volume formed between the secondary data points.

[0029] The secondary variance factor may be compared with at least one threshold value to determine the secondary movement score. The secondary variance factor may be compared with multiple threshold values. The primary biological features may be associated with a first portion of the human body and the secondary biological features are associated with a second portion of the human body. The first portion of the human body may be a left side of the human body, and the second portion of the human body may be a right side of the human body.

[0030] The method may further comprise: outputting an overall movement score that is dependent on the primary movement score and the secondary movement score. The overall movement score may be indicative of a level of balance and / or symmetry in the human’s movement.

[0031] Here, references to primary and secondary data points, biological features, spatial measurements, variance factors, and movement scores is not an indication of importance or order of calculation or determination, but is used only to distinguish between different data points, biological features, spatial measurements, variance factors, and movement scores.

[0032] Practicable embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, of which:

[0033] Figure 1 is a sensing apparatus configured to assess the movement of a human according to an embodiment of the invention;

[0034] Figure 2 is a flowchart illustrating a method of assessing the movement of a human according to an embodiment of the invention;

[0035] Figure 3 is a diagram illustrating the application of steps 130 and 140 of the method of Figure 2; and

[0036] Figure 4 is a diagram illustrating the application of steps 150 and 160 of the method of Figure 2. Figure 1 illustrates an apparatus 10 for capturing and processing a sequence of images of a patient performing a movement. The apparatus 10 comprises an image sensor 20, a controller 30, and a processor 40.

[0037] In use, controller 30 instructs the image sensor 20 to capture a sequence of images of the patient performing their movement. Most preferably, the sequence of images is a video. Once captured, the controller 30 transmits the sequence of images to the processor 40 for processing.

[0038] In the illustrated embodiment, the processor 40 forms part of the apparatus 10, however it is anticipated that the processor 40 may be a remote processor, and processing of the images may be conducted remotely, for example on a remote computing device. The apparatus may further comprise a memory (not illustrated), for storing the sequence of images either temporarily or permanently. The image sensor 20 may be a single sensor or a system of sensors, for example to capture different angles of the patient during movement.

[0039] Figure 2 illustrates a method of processing the sequence of images captured by the apparatus 10. In step 110, the patient is asked to begin performance of a movement. In step 120, the image sensor 20 captures a sequence of images during the movement of the patient. In step 130, biological features within the sequence of images are identified. In step 140, reference points are assigned to biological features of the patient across the sequence of images. At step 150, those reference points are linked together to define a spatial measurement, such as a length, an area, or a volume. Most preferably, the spatial measurement is a volume formed between the defined reference points. At step 160, the volume is monitored across the sequence of images. At step 170, the change in the volume is calculated between each of the images in the sequence of images.

[0040] At step 180, a movement score is determined that is dependent on the change in volume, and is indicative of how well the patient is deemed to have moved over the given movement, e.g. how much balance, control, symmetry and speed they have shown whilst moving. Figure 3 illustrates steps 130 and 140 of the method of Figure 2 in greater detail. In particular, Figure 3 illustrates a plurality of potential biological features 200 that have been identified in step 130. Figure 3 also illustrates that reference points 210A, 210B, 21 OC, 21 OD have been assigned to a plurality of those biological features. Reference point 210A has been assigned to the right shoulder of the patient (pictured on the left in Figure 3), reference point 21 OB has been assigned to the left shoulder of the patient (pictured on the right in Figure 3), reference point 21 OC has been assigned to the right hip of the patient (pictured on the left in Figure 3), and reference point 21 OD has been assigned to the left hip of the patient (pictured on the right in Figure 3). Hence, as per the method described in relation to Figure 2, it is these reference points 210A, 21 OB, 21 OC, 21 OD that are linked together to define a volume in step 150, and that volume is monitored and analysed in steps 160, 170 and 180.

[0041] Figure 4 illustrates steps 150 and 160 of the method of Figure 2 in greater detail. The top row of Figure 4 represents the patient movement in five stages. In this example, the patient is performing the sit-to-stand movement described in the background above, and five images 300A, 300B, 300C, 300D, 300E are obtained from a video recording of that movement.

[0042] In the first image 300A, the patient begins sat down. In the second image 300B, the patient is between sitting down and standing up. In the third image 300C, the patient is stood up. In the fourth image 300D, the patient is between standing up and sitting down. In the fifth image 300E, the patient is sitting down again. In practice, more than five images would likely be analysed to better understand the rate of change of volume across the movement, but only five are shown here for ease of explanation.

[0043] Since it is hard to illustrate the volumes formed between the reference points 210A, 210B, 210C, 21 OD here, due to the nature of the shapes formed, Figure 4 illustrates the volume in two-dimensional form from two different angles. In the second row of Figure 4, the movement of reference points 210A, 21 OB, 21 OC, 21 OD is shown from a front view of the patient during the movement. In the third row of Figure 4, the movement of reference points 21 OA, 21 OB, 21 OC, 21 OD is shown from a side view of the patient during the movement. In the front view, as shown in the second row of Figure 4, the reference points 21 OA, 21 OB, 201 C and 21 OD form a trapezium shape, due to the patient’s shoulders being broader than the hips. In the side view, as shown in the third row of Figure 4, the reference points 21 OA, 201 C and 21 OD form a triangle shape, due to the patient’s shoulders being slightly forward of the patient’s hips.

[0044] As shown in row 2 of Figure 4, across all of the images, the reference points 21 OA, 21 OB assigned to the shoulders generally maintain their separation in the x-axis, because the distance between the two shoulders stays substantially the same throughout the movement. As shown in row 2 of Figure 4, across all of the images, the reference points 21 OC, 21 OD assigned to the hips generally maintain their separation in the x-axis, because the distance between the two hips stays substantially the same throughout the movement.

[0045] As shown in row 3 of Figure 4, across all of the images, the reference points 21 OA, 21 OB assigned to the shoulders generally maintain their alignment in the z-axis, because the alignment between the two shoulders stays substantially the same throughout the movement. As shown in row 3 of Figure 4, across all of the images, the reference points 21 OC, 21 OD assigned to the hips generally maintain their alignment in the z-axis, because the alignment between the two hips stays substantially the same throughout the movement.

[0046] However, as can be seen in rows 2 and 3 of Figure 4, whilst the patient is moving between the sitting and standing positions (i.e. transitioning from image 300A to 300B to 300C), and between the standing and sitting positions (i.e. transitioning from image 300C to 300D to 300E), the separation in the y-axis between the reference points 21 OA, 21 OB assigned to the two shoulders and the reference points 21 OC, 21 OD assigned to the two hips changes. Indeed, because the patient hinges during the movement, the shoulders are lowered relative to the hips, and thus the separation in the y-axis between the reference points 210A, 21 OB assigned to the two shoulders and the reference points 21 OC, 21 OD assigned to the two hips becomes less in images 300B and 300D than it is in images 300A, 300C and 300E.

[0047] Similarly, as can be seen in row 3 of Figure 4, whilst the patient is moving between the sitting and standing positions (i.e. transitioning from image 300A to 300B to 300C), and between the standing and sitting positions (i.e. transitioning from image 300C to 300D to 300E), the separation in the z-axis between the reference points 210A, 21 OB assigned to the two shoulders and the reference points 21 OC, 21 OD assigned to the two hips changes. Indeed, because the patient hinges during the movement, the hips are sent backwards relative to the shoulders, and thus the separation in the z-axis between the reference points 210A, 21 OB assigned to the two shoulders and the reference points 21 OC, 21 OD assigned to the two hips becomes greater in images 300B and 300D than it is in images 300A, 300C and 300E.

[0048] Hence, as can be seen in Figure 4, the volume formed by the reference points 210A, 21 OB, 21 OC 21 OD changes throughout performance of the movement by the patient.

[0049] The example given is somewhat simplified for ease of explanation. In practice, the separation in the x-axis between reference point 210A and reference point 21 OB may vary throughout the movement, and the separation in the x-axis between reference point 21 OC and reference point 21 OD may vary throughout the movement. Similarly, the alignment in the z-axis between reference point 210A and reference point 21 OB may vary throughout the movement, and the alignment in the z-axis between reference point 21 OC and reference point 21 OD may vary throughout the movement. This may occur, for example, due to rotation of the shoulders and / or the hips during the movement.

[0050] In step 170, the change in volume can be calculated in a number of different ways.

[0051] In a first embodiment, the change in volume is calculated. The change in volume may be a total change in volume across the entire sequence of images. This calculation is useful because a high total change in volume is indicative of the patient’s form varying a lot throughout the movement, which may be a result of poor flexibility, mobility or strength. Alternatively, the change in volume may be a change in volume between two images within the sequence of images. This calculation is useful because a high change in volume between two images is indicative of the patient’s form varying a lot at a specific point of the movement, which may be a result of poor flexibility, mobility or strength in that specific position.

[0052] Where the change in volume is a total change in volume across the entire sequence of images, the total change in volume may be a total net cumulative change in volume, i.e. accounting for positive increases in volume and negative decreases in volume. For example, the volume may increase by 10cm3between images 300A and 300B, and by 11 cm3between images 300C and 300D in Figure 4, but decrease by 8cm3between images 300B and 300C, and by 9cm3between images 300D and 300E in Figure 4. Hence, the total net cumulative change in volume would be 10cm3- 8cm3+ 11 cm3- 9cm3, giving 4cm3.

[0053] A low change in volume, in each of the above alternatives, is indicative of the patient having good kinematics, i.e. displaying a good movement pattern. Hence, where the change in volume is low, a high movement score is output at step 180 of the method of Figure 2. Conversely, where the change in volume is high, a low movement score is output at step 180 of the method of Figure 2.

[0054] In a second embodiment, the rate of change in volume is calculated. The rate of change in volume may be an instantaneous rate of change in volume between two images within the sequence, i.e. an acceleration of change in volume. This calculation is useful because a high instantaneous rate of change in volume is indicative of the patient jolting, which is a result of a lack of control of their movement. Alternatively, the rate of change in volume may be calculated as the total change in volume over the sequence of images divided by the total time over which the sequence of images is taken. This calculation is useful because a high rate of change of volume is indicative of fast changes in the patient’s form, which is a result of a lack of control of their movement.

[0055] A low rate of change in volume, in each of the above alternatives, is indicative of the patient having good control of their kinematics, i.e. displaying a smooth and controlled movement pattern. Hence, where the rate of change in volume is low, a high movement score is output at step 180 of the method of Figure 2. Conversely, where the rate of change in volume is high, a low movement score is output at step 180 of the method of Figure 2.

[0056] In each of the embodiments described above, the movement score may be assigned based on a threshold or a series of thresholds. For example, the movement score may be indicative of “good” or “bad” kinematics or kinematics control, based on the change in volume being compared with a single threshold. Alternatively, the movement score may be a score out of 10, or a 100, based on comparison of the change in volume with a number of thresholds.

[0057] In some embodiments, an overall movement score may be given based on a plurality of individual movement scores assigned to each of the change in volume calculations. That is, the patient may be assigned a first movement score based on the change in volume, and a second movement score based on the rate of change in volume. The first movement score may be indicative of the patient’s kinematics, and the second movement score may be indicative of the patient’s control of their kinematics. The overall movement score may take the first and second movement scores into account, and is therefore indicative of the patient’s kinematics the patient’s control of their kinematics.

[0058] In further embodiments, it is envisaged that the method described in relation to Figure 2 could be repeated to monitor two different spatial measurements. For example, the method could be first implemented to monitor a spatial measurement relating to the left side of the body, and the method could then be implemented to monitor a spatial measurement relating to the right side of the body. A first movement score may therefore be given that is associated with the first spatial measurement, and a second movement score may be given that is associated with the second spatial measurement. This could be advantageous, for example, to determine imbalances in the patient’s body. In particular, any difference in the movement scores may indicate that one side of the body has responded better to rehabilitation or treatment than the other. In this embodiment, an overall movement score may be given based on the first and second movement scores.

[0059] In the embodiment illustrated in Figure 3, reference points 21 OA, 21 OB, 21 OC, 21 OD have been assigned to the right shoulder, the left shoulder, the right hip, and the left hip of the patient. However, it is envisaged that in other embodiments, reference points could be assigned to other biological features, for example (but not limited to) the other potential biological features 200 illustrated in Figure 3. In one example, assigning reference points to the patient’s left hip, right hip, right knee, and right ankle may be useful for assessing their right-sided knee kinematics, and assigning reference points to the patient’s right hip, left hip, left knee, and left ankle may be useful for assessing their left-sided knee kinematics.

Claims

Claims1 . A method of assessing the movement quality of a human, the method comprising: obtaining a sequence of images of a human performing a deliberate movement; identifying in each image two or more primary biological features; assigning to each image primary data points indicative of the position of the two or more primary biological features within the image; calculating for each image a primary spatial measurement formed between the primary data points; calculating a primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and outputting a primary movement score that is indicative of the quality of the human’s movement, the primary movement score being dependent on the variance factor.

2. The method according to Claim 1 , wherein the primary spatial measurement is a volume formed between the primary data points.

3. The method according to Claim 1 or Claim 2, wherein the sequence of images is a sequence of frames within a video.

4. The method according to any of Claims 1 -3, wherein the primary variance factor is compared with at least one threshold value to determine the primary movement score.

5. The method according to any of Claims 1 -4, further comprising: calculating a further primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and outputting a further primary movement score that is dependent on the further primary variance factor.

6. The method according to Claim 5, wherein the further primary variance factor is compared with at least one threshold value to determine the further primary movement score.

7. The method according to Claim 5 or Claim 6, wherein the primary movement score is indicative of a first aspect of the human’s quality of movement and the further primary movement score is indicative of a second different aspect of the human’s quality of movement.

8. The method according to any of Claims 1 -7, wherein the primary variance factor or the further primary variance factor is dependent on a change in the primary spatial measurement between two adjacent images.

9. The method according to any of Claims 1 -7, wherein the primary variance factor or the further primary variance factor is dependent on a change in the primary spatial measurement across the full sequence of images.

10. The method according to any of Claims 1 -7, wherein the primary variance factor or the further primary variance factor is dependent on a net cumulative change in the primary spatial measurement across the full sequence of images.11 .The method according to any of Claims 1 -7, wherein the primary variance factor or the further primary variance factor is dependent on a rate of change in the primary spatial measurement between at least two of the images.

12. The method according to any of Claims 1 -7, wherein the primary variance factor or the further primary variance factor is dependent on a rate of change in the primary spatial measurement between two adjacent images.

13. The method according to any of Claims 1 -7, wherein the primary variance factor or the further primary variance factor is dependent on a rate of change in the primary spatial measurement across the full sequence of images.

14. The method according to any of Claims 5-13, further comprising: outputting an overall primary movement score indicative of the quality of the human’s movement, the overall primary movement score being dependent on the primary movement score and the further primary movement score.

15. The method according to any of Claims 1 -14, further comprising: identifying in each image two or more secondary biological features; assigning to each image secondary data points indicative of the position of the two or more secondary biological features within the image; calculating for each image a secondary spatial measurement formed between the secondary data points; calculating a secondary variance factor that is dependent on a change in the secondary spatial measurement between at least two of the images; and outputting a secondary movement score that is dependent on the secondary variance factor.

16. The method according to Claim 15, wherein the secondary spatial measurement is a volume formed between the secondary data points.

17. The method according to Claim 15 or Claim 16, wherein the secondary variance factor is compared with at least one threshold value to determine the secondary movement score.

18. The method according to any of Claims 15-17, wherein the primary biological features are associated with a first portion of the human body and the secondary biological features are associated with a second portion of the human body.

19. The method according to Claim 18, wherein the first portion of the human body is a left side of the human body and the second portion of the human body is a right side of the human body.

20. The method according to any of Claims 15-19, further comprising: outputting an overall movement score that is indicative of the quality of thehuman’s movement, the overall primary movement score being dependent on the primary movement score and the secondary movement score.21 .A sensing apparatus configured to assess the movement of a human according to the method of any of Claims 1 -20.

22. A sensing apparatus according to Claim 21 , the sensing apparatus comprising at least one processor configured to perform the method of any of Claims 1 -20.

23. A data carrier or data storage medium comprising machine readable instructions for the control of one or more processor to perform a method of assessing the movement quality of a human, the method comprising: obtaining a sequence of images of a human performing a deliberate movement; identifying in each image two or more primary biological features; assigning to each image primary data points indicative of the position of the two or more primary biological features within the image; calculating for each image a primary spatial measurement formed between the primary data points; calculating a primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and outputting a primary movement score that is indicative of the quality of the human’s movement, the primary movement score being dependent on the variance factor.

24. A sensing apparatus for assessing the movement quality of a human, the sensing apparatus comprising: at least one sensor configured to capture a sequence of images of a human performing a deliberate movement; and at least one controller configured to: identify in each image two or more primary biological features; assign to each image primary data points indicative of the position of thetwo or more primary biological features within the image; calculate for each image a primary spatial measurement formed between the primary data points; calculate a primary variance factor that is dependent on a change in the primary spatial measurement between at least two of the images; and output a primary movement score that is indicative of the quality of the human’s movement, the primary movement score being dependent on the variance factor.

25. The sensing apparatus of Claim 24, wherein the at least one sensor comprises a plurality of sensors.

Images