Methods for detecting and correcting compensatory movement in an extended reality (XR) environment

Abstract

A method for detecting and correcting compensatory movement within an extended reality (XR) environment. The method comprises the steps of: collecting data regarding the body movement of a person; determining average values from the collected data for at least one movement parameter; identifying deviation from the average values for the at least one movement parameter; determining whether the deviation is in excess of a predetermined threshold; if the deviation is less than the predetermined threshold, continuing to collect data regarding the body movement of the person; and if the deviation is equal to or greater than the predetermined threshold, determining that the person is making a compensatory movement and initiating a corrective process.

Description

[0001] METHODS FOR DETECTING AND CORRECTING COMPENSATORY MOVEMENT IN AN EXTENDED REALITY (XR) ENVIRONMENT

[0002] TECHNICAL FIELD

[0003] The invention relates to methods for detecting and correcting compensatory movement in an extended reality (XR) environment.

[0004] BACKGROUND

[0005] Extended reality (XR) technologies, encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR), have emerged as promising tools for various applications, including rehabilitation and therapy. In recent years, there has been a growing interest in leveraging XR technologies to facilitate motor rehabilitation, particularly for individuals who have suffered brain injuries or other neurological disorders. These technologies offer immersive and engaging environments that can potentially enhance the effectiveness of rehabilitation exercises and promote better patient outcomes.

[0006] One of the key challenges in motor rehabilitation is the presence of compensatory movements, which are altered movement patterns that individuals adopt to compensate for the loss of motor function. Compensatory movements can be detrimental to the recovery process, as they may lead to non-optimal movement patterns and hinder further improvement of the affected limb. Moreover, excessive compensatory movements may exacerbate existing impairments and contribute to the development of secondary complications, such as musculoskeletal pain and joint instability.

[0007] Current approaches to address compensatory movements in motor rehabilitation primarily rely on therapist supervision and correction. However, these face-to-face interventions are labor-intensive, time-consuming, and not cost-efficient, making them unsuitable for large-scale implementation in healthcare systems. Furthermore, the effectiveness of therapist-guided interventions may be limited by the availability of skilled therapists and the variability in their expertise. To overcome these limitations, there has been a growing interest in developing automated methods for providing rehabilitation therapy within XR environments. Several XR systems have been proposed for motor rehabilitation, incorporating various tracking technologies, such as cameras, inertial measurement units (IMUs), and other external sensors, to monitor the user's movements and provide feedback on their performance.

[0008] However, existing XR systems for motor rehabilitation have several shortcomings. First, many of these systems focus primarily on the repetition of movements, rather than the quality of the movements themselves. This may lead to the reinforcement of compensatory movements and suboptimal recovery outcomes. Second, current XR systems do not adequately individualize therapy protocols to the specific needs of each patient, as a skilled therapist would. This may limit the effectiveness of the interventions and hinder the potential for personalized rehabilitation.

[0009] Moreover, existing XR systems often rely on external inputs for tracking user movements, which may not be sufficient for detecting and correcting compensatory movements. There is a need for more advanced algorithms and methods that can accurately identify compensatory movements and provide appropriate feedback to users in real-time. Such algorithms should be capable of analyzing the spatial position and orientation of the user's head-mounted display (HMD) and controllers, as well as other relevant data, to determine whether compensatory movements are being employed during the rehabilitation exercises.

[0010] Improvements are desired to overcome shortcomings of existing implementations.

[0011] SUMMARY

[0012] In general terms, the present disclosure is directed to a method for detecting and correcting compensatory movements in an extended reality environment. This method involves collecting data on a person's body movements, determining average values for certain movement parameters, identifying deviations from those averages, and initiating a corrective process if the deviation exceeds a predetermined threshold. Advantageously, this invention addresses the issue of compensatory movements during at-home self-directed rehabilitation exercises, which can hinder recovery and lead to nonoptimal movement patterns, by providing an automated system that detects and corrects these movements in immersive XR environments.

[0013] According to an aspect of the invention, there is provided a method for detecting and correcting compensatory movement within an extended reality (XR) environment. The method comprises the following steps: collecting data regarding the body movement of a person; determining average values from the collected data for at least one movement parameter; identifying deviation from the average values for the at least one movement parameter; determining whether the deviation is in excess of a predetermined threshold; if the deviation is less than the predetermined threshold, continuing to collect data regarding the body movement of the person; and if the deviation is equal to or greater than the predetermined threshold, determining that the person is making a compensatory movement and initiating a corrective process.

[0014] Optionally, the collected data relates to a spatial position and orientation of a head mounted display and / or handheld controllers of an extended reality system.

[0015] Optionally, the averages values relate to the body movement of a person when they are not performing a compensatory movement.

[0016] Optionally, the predetermined threshold relates to body movements that are identified to be representative of a compensatory movement.

[0017] Optionally, the method further comprises the step of setting a flag to indicate that the patient is performing a compensatory body movement when the deviation is greater than or equal to the predetermined threshold.

[0018] Optionally, the corrective action is initiated in response to the setting of the flag to alert that a compensatory movement has been detected.

[0019] Optionally, the person is implemented within an extended reality game or application. Optionally, if it is determined that the deviation is equal to or greater than the predetermined threshold, a compensating timer is updated.

[0020] Optionally, the timer is used to determine a time period over which the patient is performing compensatory body movements.

[0021] Optionally, a determination of a false compensation alert is made if the time period is less than a predetermined value.

[0022] Optionally, the compensatory movement is a shoulder movement.

[0023] Optionally, the compensatory movement is a torso movement.

[0024] Optionally, the corrective action is to prevent play of an extended reality based game.

[0025] Optionally, the corrective action is to display a visual and / or audio alert indicating that compensatory movement has been detected.

[0026] Optionally, the alert instructs the patient to stop making the compensatory movement.

[0027] Optionally, the corrective action is maintained until the person ceases to make the compensatory movement.

[0028] Optionally, the data regarding the compensatory movement is stored in either nonvolatile memory of an extended reality headset and / or transmitted to a remote computing device or server.

[0029] Optionally, the average values are based on the person's body movement from within a single session and at the end of the session, the data regarding the person's body movement is reset.

[0030] Optionally, the average values are based on the body movement and position of a person in a relaxed position prior to engaging with the extended reality environment. Optionally, the average values are dynamically updated in accordance with data recorded about the person's body movements.

[0031] BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other objects and advantages of the invention of the disclosures will be apparent upon consideration of the following detailed description, taken in conjunction with the accompanying drawings in which:

[0033] Figure 1 illustratively shows a flowchart for performing a method according to the present disclosure.

[0034] Figure 2 illustrates the difference between a person who is not compensating when performing a task v a person who is compensating when performing the same task.

[0035] Figure 3 illustratively shows a flowchart for performing another method according to the present disclosure.

[0036] DETAILED DESCRIPTION

[0037] Figure 1 illustratively shows a flowchart for performing a method 100 according to the present disclosure. The method 100 includes six steps: Step 101 - collecting data regarding the body movement of a person; Step 102 - determining average values from the collected data for at least one movement parameter; Step - 103 identifying deviation from the average values for the at least one movement parameter; Step 104 - determining whether the deviation is in excess of a predetermined threshold; Step 105 - if the deviation is less than the predetermined threshold, continuing to collect data regarding the body movement of the person; and Step 106 - if the deviation is equal to or greater than the predetermined threshold, determining that the person is making a compensatory movement and initiating a corrective process.

[0038] Advantageously, the method 100 provides a comprehensive and adaptable approach to detecting and correcting compensatory movements during physical rehabilitation exercises, specifically catering to individuals recovering from brain injuries. By continuously monitoring the body movement of a person through advanced sensors and data collection techniques, the method can capture intricate details of the individual's movements, including joint angles, muscle activation patterns, and biomechanical forces. This wealth of data allows for an accurate comparison to average values, which can be derived from a combination of the patient's own historical movement data, as well as normative data from healthy individuals performing similar tasks.

[0039] The method's ability to identify deviations that may indicate compensatory movements is particularly beneficial in cases where patients may have varying degrees of motor function loss, as it can accommodate a wide range of movement abnormalities and adapt to the patient's changing needs throughout the rehabilitation process. For example, the method can detect subtle compensatory movements, such as excessive trunk rotation during reaching tasks or abnormal gait patterns during walking exercises, which may otherwise go unnoticed by therapists or patients themselves.

[0040] By providing real-time feedback and initiating corrective processes, the method 100 can help ensure that patients perform the correct movements during their rehabilitation exercises, ultimately leading to better movement patterns and faster motor improvements. This can be achieved through various means, such as providing visual or auditory cues to guide the patient towards the desired movement pattern, adjusting the difficulty or complexity of the exercise to match the patient's current abilities, or even incorporating biofeedback techniques to enhance the patient's awareness of their own body movements.

[0041] Furthermore, the method 100 can be integrated with advanced rehabilitation technologies, such as robotic exoskeletons or functional electrical stimulation devices, to provide additional support and assistance to the patient during their exercises. This can help to further promote correct movement patterns and prevent the development of compensatory strategies that may hinder the patient's recovery.

[0042] In addition to its direct benefits for the patient, the method 100 also offers valuable insights for therapists and healthcare professionals involved in the patient's care. By continuously tracking the patient's progress and detecting compensatory movements in real-time, therapists can make more informed decisions regarding the patient's therapy protocols, such as adjusting the frequency or intensity of exercises, introducing new exercises to target specific movement deficits, or modifying the patient's goals and expectations based on their current performance.

[0043] In some embodiments the collected data relates to a spatial position and orientation of a head mounted display and / or handheld controllers of an extended reality (XR) environment.

[0044] Advantageously, by collecting data related to the spatial position and orientation of a head mounted display and / or handheld controllers, the method 100 can accurately monitor the physical body movement of a person translated into an extended reality environment. This allows for a more immersive and engaging rehabilitation experience, as patients can participate in virtual reality games or applications designed to facilitate their recovery. Furthermore, the use of head mounted displays and handheld controllers enables the method 100 to detect compensatory movements in real-time, allowing for immediate corrective action and feedback to the patient.

[0045] In some embodiments the averages values relate to the body movement of a person when they are not performing a compensatory movement.

[0046] Advantageously, by determining average values that relate to the body movement of a person when they are not performing a compensatory movement, the method 100 can establish a baseline for comparison when identifying deviations. This allows for a more accurate detection of compensatory movements, as the method can differentiate between normal and compensatory body movements based on the established average values.

[0047] In some embodiments the predetermined threshold relates to body movements that are identified to be representative of a compensatory movement.

[0048] Advantageously, by setting a predetermined threshold that relates to body movements indicative of compensatory movements, the method 100 can effectively determine when a person is making a compensatory movement during their rehabilitation exercises. This allows for the timely initiation of corrective processes, ensuring that patients perform the correct movements and avoid developing nonoptimal movement patterns that may hinder their recovery.

[0049] In some embodiments the method 100 further comprises the step of setting a flag to indicate that the patient is performing a compensatory body movement when the deviation is greater than or equal to the predetermined threshold.

[0050] Advantageously, by setting a flag when the deviation is greater than or equal to the predetermined threshold, the method 100 can effectively communicate to the extended reality environment or other connected systems that a compensatory movement has been detected. This enables the initiation of corrective actions, such as adjusting the virtual reality game or application to encourage the patient to perform the correct movements.

[0051] In some embodiments the corrective action is initiated in response to the setting of the flag to alert that a compensatory movement has been detected.

[0052] Advantageously, by initiating corrective action in response to the setting of the flag, the method can provide real-time feedback and guidance to the patient, helping them to correct their compensatory movements and improve their overall rehabilitation process. This can lead to better movement patterns, greater motor improvements, and a more efficient recovery.

[0053] In some embodiments the person is implemented within an extended reality game or application.

[0054] Advantageously, by implementing the person within an extended reality game or application, the method 100 can provide a more engaging and immersive rehabilitation experience for the patient. This can help to motivate patients to participate in their rehabilitation exercises and improve their adherence to therapy protocols, ultimately leading to better outcomes in their recovery process. In some embodiments, if it is determined that the deviation is equal to or greater than the predetermined threshold, a compensating timer is updated.

[0055] Advantageously, by updating the compensating timer when the deviation is equal to or greater than the predetermined threshold, the method 100 can effectively track the duration of compensatory movements during the rehabilitation exercises. This allows for a more accurate assessment of the patient's progress and the effectiveness of the corrective actions. Furthermore, the compensating timer can be used to determine a time period over which the patient is performing compensatory body movements, providing valuable information for adjusting therapy protocols and tailoring the rehabilitation process to the individual's needs.

[0056] In some embodiments the timer is used to determine a time period over which the patient is performing compensatory body movements.

[0057] Advantageously, by using the timer to determine the time period of compensatory body movements, the method 100 can provide a quantitative measure of the patient's compensatory movements during their rehabilitation exercises. This information can be used to evaluate the effectiveness of the corrective actions and to adjust the therapy protocols accordingly, ensuring that the patient receives the most appropriate and effective rehabilitation treatment.

[0058] In some embodiments a determination of a false compensation alert is made if the time period is less than a predetermined value.

[0059] Advantageously, by determining a false compensation alert when the time period is less than a predetermined value, the method 100 can prevent unnecessary corrective actions and avoid disrupting the patient's rehabilitation process. This ensures that only genuine compensatory movements are addressed, allowing for a more efficient and targeted rehabilitation experience.

[0060] As shown in Figure 3, a continuous detection loop is illustrated. Starting at step 301 , a detection loop is initiated. At step 302, the average values of a movement parameter are updated for continuous time intervals of a predetermined length. At step 303 any difference between the average values for adjacent time periods is calculated. At step 304 it is determined whether the difference, if any, is acceptable. At step 305 a timer is reset if the difference is acceptable and at step 306 it is determined that the person is not compensating. At step 307, as an alternative to step 305, the time is updated if the difference is not acceptable. At step 308 it is determined whether the time that the difference is observed is acceptable, i.e., does it fall within a predetermined threshold. At step 309 it is determined that the user is compensating. At step 310 data is recorded and the method reverts to step 301 .

[0061] In some embodiments the compensatory movement is a shoulder movement.

[0062] Advantageously, by detecting and addressing compensatory shoulder movements, the method 100 can help patients improve their upper limb function and reduce the reliance on compensatory movements during their rehabilitation exercises. This can lead to better movement patterns, greater motor improvements, and a more efficient recovery process.

[0063] In some embodiments the compensatory movement is a torso movement.

[0064] Advantageously, by detecting and addressing compensatory torso movements, the method 100 can help patients improve their overall body movement and posture during their rehabilitation exercises. This can lead to better movement patterns, greater motor improvements, and a more efficient recovery process. Figure 2 illustrates the difference between a person who is compensating when reaching for a mug v a person who is not compensating when performing the same task.

[0065] In some embodiments the corrective action is to prevent play of an extended reality game.

[0066] Advantageously, by preventing play of an extended reality game as a corrective action, the method 100 can effectively encourage the patient to perform the correct movements during their rehabilitation exercises. This can help to reinforce proper movement patterns and discourage compensatory movements, ultimately leading to better outcomes in their recovery process. In some embodiments the corrective action is to display a visual and / or audio alert indicating that compensatory movement has been detected.

[0067] Advantageously, by displaying a visual and / or audio alert as a corrective action, the method can provide immediate feedback to the patient, helping them to recognize and correct their compensatory movements in real-time. This can lead to better movement patterns, greater motor improvements, and a more efficient recovery process.

[0068] In some embodiments the alert instructs the patient to stop making the compensatory movement.

[0069] Advantageously, by instructing the patient to stop making the compensatory movement through the alert, the method 100 can effectively guide the patient towards performing the correct movements during their rehabilitation exercises. This can help to reinforce proper movement patterns and discourage compensatory movements, ultimately leading to better outcomes in their recovery process.

[0070] In some embodiments the corrective action is maintained until the person ceases to make the compensatory movement.

[0071] Advantageously, by maintaining the corrective action until the person ceases to make the compensatory movement, the method 100 can ensure that the patient consistently performs the correct movements during their rehabilitation exercises. This can lead to better movement patterns, greater motor improvements, and a more efficient recovery process.

[0072] In some embodiments the data regarding the compensatory movement is stored in either non-volatile memory of an extended reality (XR) environment 101 and / or transmitted to a remote computing device or server.

[0073] Advantageously, by storing the data regarding the compensatory movement in nonvolatile memory or transmitting it to a remote computing device or server, the method 100 can facilitate the tracking and analysis of the patient's progress during their rehabilitation exercises. This information can be used by therapists and healthcare professionals to adjust therapy protocols, monitor patient progress, and optimize the rehabilitation process.

[0074] In some embodiments the average values are based on the person's body movement from within a single session and at the end of a session data regarding the person's body movement is reset.

[0075] Advantageously, by basing the average values on the person's body movement within a single session and resetting the data at the end of the session, the method can provide a more accurate and up-to-date baseline for comparison when identifying deviations. This allows for a more precise detection of compensatory movements and ensures that the corrective actions are tailored to the patient's current movement patterns and needs.

[0076] In some embodiments the average values are based on the body movement and position of a person in a relaxed position prior to engaging with the extended reality (XR) environment.

[0077] Advantageously, by basing the average values on the body movement and position of a person in a relaxed position prior to engaging with the extended reality environment, the method 100 can establish a more accurate and representative baseline for comparison when identifying deviations. This allows for a more precise detection of compensatory movements and ensures that the corrective actions are tailored to the patient's current movement patterns and needs.

[0078] In some embodiments the average values are dynamically updated in accordance with data recorded about the person's body movements 103.

[0079] Advantageously, by dynamically updating the average values in accordance with the data recorded about the person's body movements, the method 100 can continuously adapt and refine the baseline for comparison when identifying deviations. This allows for a more accurate detection of compensatory movements over time and ensures that the corrective actions remain effective and relevant throughout the patient's rehabilitation process and within a single rehabilitation session as the person gets tired.

[0080] It will be understood that the term "extended reality (XR) environment" as used herein may refer to a digital environment that combines aspects of virtual reality (VR), augmented reality (AR), and mixed reality (MR), allowing users to interact with both real and virtual objects within a simulated space.

[0081] It will be understood that the terms "head mounted display" or “HMD” as used herein may refer to a wearable device, often in the form of a headset or glasses, that presents visual content to the user's eyes, typically used in virtual reality, augmented reality, or mixed reality applications.

[0082] It will be understood that the term "handheld controllers" as used herein may refer to portable input devices, typically incorporating buttons, joysticks, or motion sensors, that allow users to interact with and control various aspects of an extended reality environment, such as virtual or augmented reality systems.

[0083] It will be understood that the term "flag" as used herein may refer to a signal or indicator, such as a binary value or a specific data point, that is generated when a deviation in a person's body movement exceeds a predetermined threshold, thereby triggering the corrective process in the extended reality environment.

[0084] It will be understood that the term "extended reality game" as used herein may refer to an interactive digital experience that combines elements of virtual reality, augmented reality, and mixed reality, allowing users to engage with both real-world and virtual environments through various sensory inputs and devices.

[0085] The foregoing disclosure provides illustration and description but is not intended to be exhaustive or to limit the implementations to the precise form disclosed. Modifications may be made considering the above disclosure or may be acquired from practice of the implementations. As used herein, the term “component” is intended to be broadly construed as hardware, firmware, or a combination of hardware and software. It will be apparent that systems and / or methods described herein may be implemented in different forms of hardware, firmware, and / or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not limiting of the implementations. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code - it being understood that software and hardware can be used to implement the systems and / or methods based on the description herein. As used herein, satisfying a threshold may, depending on the context, refer to a value being greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, and / or the like, depending on the context. Although combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of various implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and / or disclosed in the specification.

[0086] Although each dependent claim listed below may directly depend on only one claim, the disclosure of various implementations includes each dependent claim in combination with every other claim in the claim set. No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Further, as used herein, the article “the” is intended to include one or more items referenced in connection with the article “the” and may be used interchangeably with “the one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and / or the like), and may be used interchangeably with “one or more.” Where only one item is intended, the phrase “only one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open- ended terms. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise. Also, as used herein, the term “or” is intended to be inclusive when used in a series and may be used interchangeably with “and / or,” unless explicitly stated otherwise (e.g., if used in combination with “either” or “only one of’).

Claims

CLAIMS1. A method for detecting and correcting compensatory movement within an extended reality (XR) environment, the method comprising the steps of: a. collecting data regarding the body movement of a person; b. determining average values from the collected data for at least one movement parameter; c. identifying deviation from the average values for the at least one movement parameter; d. determining whether the deviation is in excess of a predetermined threshold; e. if the deviation is less than the predetermined threshold, continuing to collect data regarding the body movement of the person; and f. if the deviation is equal to or greater than the predetermined threshold, determining that the person is making a compensatory movement and initiating a corrective process.

2. The method of claim 1 , wherein the collected data relates to a spatial position and orientation of a head mounted display and / or handheld controllers of an extended reality system.

3. The method of claim 1 or claim 2, wherein the averages values relate to the body movement of a person when they are not performing a compensatory movement.

4. The method of claim 3, wherein the predetermined threshold relates to body movements that are identified to be representative of a compensatory movement.

5. The method of claim 3 or claim 4 further comprising the step of setting a flag to indicate that the patient is performing a compensatory body movement when the deviation is greater than or equal to the predetermined threshold.

6. The method of claim 5, wherein the corrective action is initiated in response to the setting of the flag to alert that a compensatory movement has been detected.

7. The method of any preceding claim, wherein the person is implemented within an extended reality game or application.

8. The method of claim 1 , wherein if it is determined that the deviation is equal to or greater than the predetermined threshold, a compensating timer is updated.

9. The method of claim 8, wherein the timer is used to determine a time period over which the patient is performing compensatory body movements.

10. The method of claim 9, wherein a determination of a false compensation alert is made if the time period is less than a predetermined value.

11. The method of claim 1 , wherein the compensatory movement is a shoulder movement.

12. The method of claim 1, wherein the compensatory movement is a torso movement.

13. The method of claim 1 , wherein the corrective action is to prevent play of an extended reality based game.

14. The method of claim 1 , wherein the corrective action is to display a visual and / or audio alert indicating a that compensatory movement has been detected.

15. The method of claim 14, wherein the alert instructs the patient to stop making the compensatory movement.

16. The method of claim 13, 14, or 15, wherein the corrective action is maintained until the person ceases to make the compensatory movement.

17. The method of claim 16, wherein data regarding the compensatory movement is stored in either non-volatile memory of an extended reality headset and / or transmitted to a remote computing device or server.

18. The method of claim 1, wherein the average values are based on the person’s body movement from within a single session and at the end of a session data regarding the person’s body movement is reset.

19. The method of claim 1 , wherein the average values are based on the body movement and position of a person in a relaxed position prior to engaging with the extended reality environment.

20. The method of claim 19, wherein the average values are dynamically updated in accordance with data recorded about the person’s body movements.

Images