Domain-invariant gait state estimation

Abstract

A system, device, and method for generating gait state estimations for a user of a limb- assistive device. The method includes: training an encoder with a gait state decoder using annotated training data; training the encoder with a measurement decoder using target domain walking measurements; and determining a gait state estimation of the user through executing a neural network comprised of the encoder and the gait state decoder.

Description

DOMAIN-INVARIANT GAIT STATE ESTIMATIONTECHNICAL FIELDThis invention is related to estimating gait of an individual using sensor measurements, and related exoskeleton assistive walking devices and / or systems incorporating such gait estimation.BACKGROUNDRobotic exoskeletons may someday allow human users to overcome the limitations of our natural bodies. For example, emerging lower-limb exoskeletons can provide assistive joint torques to help a user walk and carry loads with promising outcomes, including reduced metabolic cost. Most research to date has focused on steady-state locomotion in a controlled laboratory setting where the task and phase rate (i.e., rate of continuous progression through the gait cycle) are nearly constant. This controlled environment makes it easier to design control strategies that deliver appropriate torque assistance in synchrony with the user’s gait because phase progression during steady-state locomotion can be reasonably predicted using time normalized by the stride period (e.g., the time between consecutive heel strike (HS) events). This approach is quite effective and widely used for controlling exoskeletons on treadmills. However, control strategies based on these assumptions perform poorly outside of the laboratory, where environments are uncertain and locomotion is highly non-steady and transitory. More recent research has focused on development of control strategies that detect changes in human gait throughout different walking conditions, such as changes in walking speed and / or changes in ground incline, which can then be used to adjust the torque assistance from the exoskeletons according to the task.The use of exoskeletons for human mobility has great potential for improving the quality of life for many individuals. However, the success of exoskeletons depends on the development of controllers that can operate outside of the laboratory and in the real world. Modem control strategies aim to learn the gait state, which is an encoding of human gait, using machine learning (ML) techniques. Past controllers have relied on supervised learning techniques, which require sets of pairs of data: input signals about walking that are available on the exoskeleton, typically kinematic measurements, which are paired with ground truth labels of the gait state; neural networks leam to predict the gait state label for kinematic measurements. While the input signals are easily obtainable, the ground truth labeling is usually done manually and can be tedious and time-consuming, limiting the scalability of these ML methods. Furthermore, open-source datasetsof human walking, which measure normative walking, are often used to train the ML networks, and this fails to take into account numerous considerations affecting gait for a particular individual and / or aspects of the individual, such as whether an exoskeleton assistive walking device is being used.SUMMARYIn accordance with an aspect of the invention, there is provided a method of generating gait state estimations for a user having a limb-assistive device. The method includes: training an encoder with a gait state decoder using annotated training data; training the encoder with a measurement decoder using target domain walking measurements; and determining a gait state estimation of the user through executing a neural network comprised of the encoder and the gait state decoder.The method may further include any of the following features or any technically-feasible combination of two or more of the following features:- the encoder is trained with the gait state decoder as a part of a first training phase, and wherein the encoder is trained with the measurement decoder as a part of a second training phase that is performed after the first training phase;- the first training phase uses supervised training in which the annotated training data includes walking measurements paired with gait state labels;- the second training phase uses unsupervised learning in which the target domain walking measurements are used as both input into the encoder and to determine a reconstruction loss by comparing the target domain walking measurements with output of the measurement decoder;- the first training phase and the second training phase are performed offline relative to a limb-assistive device worn by a user, and wherein the determining step is performed online by the limb-assistive device;- the limb-assistive device includes one or more sensors used to determine walking measurements that are input into the neural network for purposes of determining the gait state estimation; and / or- the walking measurements are determined through sensor data captured by the one or more sensors as the user walks while wearing the limb-assistive device.In accordance with another aspect of the invention, there is provided a non-transitory computer-readable memory comprising computer instructions that, when executed by one or moreprocessors, determine a gait state estimation of a user through executing a trained neural network, wherein the trained neural network is comprised of a trained encoder and a gait state decoder, and wherein the trained encoder is trained with a measurement decoder using target domain walking measurements. According to embodiments, the non-transitory computer-readable memory may 5 further include or be characterized in accordance with any of those features noted above in connection with the method above, including any technically-feasible combination of two or more of said features.In accordance with another aspect of the invention, there is provided a gait state estimation controller comprising: at least one processor; and non-transitory computer-readable memory 10 storing computer instructions that, when executed, cause the at least one processor to determine a gait state estimation through inputting sensor data into a trained neural network having an encoder and a gait state decoder. The neural network is trained using a training processing that includes: training the encoder with the gait state decoder using annotated training data; and training the encoder with a measurement decoder using target domain walking measurements.15 According to embodiments, the gait state estimation controller may further include or be characterized in accordance with any of those features noted above in connection with the method above, including any technically-feasible combination of two or more of said features. Furthermore, according to embodiments, the encoder is configured for generating latent encoding data, the gait state decoder is configured for determining the gait state estimation based on the 20 latent encoding data, and / or the gait state estimation is generated without use of the measurement decoder.According to yet another aspect of the invention, there is provided a limb-assistive device having the gait state estimation controller. And, according to an embodiment, the limb-assistive device further comprises one or more sensors for capturing sensor information and causing the 25 sensor data to be provided to the gait state estimation controller.BRIEF DESCRIPTION OF THE DRAWINGS

[0001] Illustrative embodiments will hereinafter be described in conjunction with the appended drawings, wherein:FIG. l is a diagrammatic illustration of a limb-assistive device having a neural network 30 trained for gait state estimation, according to one embodiment;FIG. 2 is a diagrammatic illustration of a training architecture for training the neural network of FIG. 1, according to one embodiment;FIG. 3 is a diagrammatic illustration of a first training phase of the training architecture of FIG. 2 in which a gait state decoder of the neural network of is trained, according to one embodiment; andFIG. 4 is a diagrammatic illustration of a second training phase of the training architecture of FIG. 2 in which an encoder of the neural network is trained, according to one embodiment.DETAILED DESCRIPTIONDescribed herein is a system, device, and method for generating gait state estimations for a user having a limb-assistive device, particularly through use of a trained neural network adapted to a target domain characterized by walking with use of a limb-assistive device, such as an exoskeleton-assisted walking device, at least according to embodiments. More particularly, embodiments described herein are directed to a system and device configured to determine a gait state estimation (phase, speed, ground incline, and / or stair locomotion) in real-time for a user wearing a limb-assistive device through inputting walking measurements into the neural network as the walking measurements are determined from sensor data. According to embodiments, the system and device are used for performing the method. In embodiments, the method includes training a neural network (to obtain a “trained neural network”) in two phases and according to two training tasks: (1) training an encoder with a gait state decoder using annotated training data; and (2) training the encoder with a measurement decoder using target domain walking measurements. Accordingly, at least in one embodiment, the trained neural network used in realtime is, thus, the encoder along with the gait state decoder as trained as a result of the first and second training phases.The first training phase is performed on the encoder and the gait state decoder in a source domain. The source domain is characterized by normative walking (without a limb-assistive device) and is then adapted as a result of the second training phase in which the encoder is further modified based on a reconstruction loss between inputted walking measurements and output of the measurement decoder. More particularly, the second training phase is performed in the target domain, which is characterized by use of a limb-assistive device, and this training results in the encoder being adapted to the target domain through learning salient nuances observable in the target domain that are not observable in the source domain.The neural network discussed herein is an artificial neural network and may be a “transformers” machine learning (ML) architecture that employs a feed forward, sequence-to-sequence (seq-2-seq) model and that features an attention mechanism, which is described by Vaswani et al. (“Attention is all you need.” Advances in neural information processing systems (NIPS 2017)). In the context of kinematics and gait state encoding and decoding, the transformers architecture is drawn to capture differences across locomotion types through embedding kinematic measurements into a high-dimensional space that reveals contextual relationships or associations (attention) amongst the kinematic measurements over time and / or across sensor types.As used herein, the term “limb-assistive device” encompasses both prosthetic and orthotic devices configured to provide or assist movement of a limb or portion of a limb about a natural or artificial j oint. “Powered” or “active” devices are distinguished from passive devices in that their behavior is changeable via application of non-user forces or energy to structural device components. The example disclosed below is categorized as a powered orthotic device in the form of an exoskeleton that assists the user with movement of the foot relative to the lower leg about an ankle joint. The disclosed control strategy is however applicable to prediction of the state of other human motions as used to control assistive device movement about a joint.FIG. 1 is a diagrammatic representation of a limb-assistive device 10. The illustrated example of the device 10 is a powered ankle exoskeleton 10 configured to assist movement about an ankle joint of a user U. The exoskeleton 10 includes first and second structural members 12, 14 coupled at a joint 16 that provides rotational movement of the structural members 12, 14 relative to each other at least about an axis A. In this case, the first structural member 12 is a leg brace configured for attachment to the lower leg between the knee and ankle of the user U, and the second structural member 14 is a foot plate configured to fit along and move with the user’s foot, which may be fitted along a shoe sole as shown. The device 10 additionally includes an actuator 18, such as an electric motor, that provides a torque at the joint to change the relative rotational position of the leg brace 12 and foot plate 14 or to impede changes in the relative rotational position of the structural members 12, 14. The device 10 additionally includes one or more sensors 20 and a controller 22. A sensor 20 is any feature of the device that produces or collects information pertinent to user movement, such as bodily movements during a walking gait cycle. The controller 22 receives information from the sensor(s) 20, processes the information, and controls the actuator 18 based on the processed information.In this example, the one or more sensors 20 includes a sensor that detects an angle defined between the leg brace 12 and foot plate 14 relative to a reference angle and may be embodied as an encoder associated with the actuator 18. Another type of sensor 20 that provides information pertinent to user gait is a heel strike sensor. Each sensor 20 may be affixed to one of the structural members 12, 14, the joint 16, or some other component of the device 10. In some cases, a sensor20 may be physically separate from the remainder of the device 10 and affixed to the user U elsewhere along their body, such as an accelerometer in communication with the controller 22 that detects transverse rocking movement of the user’s torso during walking. The device may include other unillustrated components, such as a transmission coupling the actuator with one of the structural members 12, 14, a spring and / or damping system, and a portable power source (e.g., a battery).FIG. 1 includes a simplified schematic of the architecture of the controller 22, which is configured to control output torque of the actuator 18 based on information received from the sensor(s) 20 and processed by the controller 22. The architecture of the controller 22 leverages machine learning for gait state estimations or predictions, which may then be further processed in order to determine a control output, such as a control output torque of the actuator 18. The illustrated architecture employs a neural network 24 that can learn the relationship between a gait state and kinematics using only labeled data, which eliminates the difficult task of attempting to fit an explicit mathematical model that describes that relationship.In terms of hardware, the controller 22 includes at least one processor and memory storing computer instructions that, when executed by the at least one processor, cause the controller 22 to determine a gait state estimate based on inputting kinematic measurements into the neural network 24, as discussed more below. The neural network 24 may use the Transformer architecture to learn and thereby encode kinematic measurements related to human gait cycles into a latent representation and, in use, decode that representation of the kinematic measurements into a succinct gait state vector including, for example, gait phase, walking speed, ground incline, stair locomotion, and / or transient motions, such as starting or stopping motions. The neural network 24 continually yields estimations or predictions of the current gait state based on the kinematic measurements provided by the sensor(s) 20. Additionally, the neural network 24, particularly the encoder 26, is fine-tunable with individualized data and can thus be personalized to individual users, which is often important given that people can vary significantly in their individual gait patterns. Each gait state estimation from the neural network 24 may be passed into other components of the controller 22, such as for determining a control output, such as a torque control output.The neural network 24 includes an encoder 26 and a gait state decoder 28 connected to the encoder 26 so that output of the encoder 26 is fed as input into the gait state decoder 28. The encoder 26 is used to encode walking measurements (also referred to as kinematic measurements) into a latent representation L of said measurement within a latent space. This latent space refers to a high dimensional space that encodes fundamental knowledge in the latent representation Labout locomotion, and is domain invariant to the source domain (normal walking) and target domain (exoskeleton walking). In general, the encoder 26 takes measurements of walking (e.g., foot angle) as input and then generates the latent representation L, a latent space embedding of those measurements. The neural network 24 as shown in FIG. 1 represents the neural network 24 as it is used for inference during real-time use of the limb-assistive device 10, continuously transforming walking measurements (as observed through captured sensor data) into gait state predictions.With reference to FIGS. 2-4, there is shown the neural network 24 as it exists during training, including a first training phase 100 (FIG. 3) and a second training phase 200 (FIG. 4). FIG. 2 depicts the neural network with its training components in its training form, which further includes a measurement decoder 30 that is connected to the encoder 26 so that output of the encoder 26 is fed as input into the measurement decoder 30. The measurement decoder 30 is used only for training in the second training phase 200 (FIG. 4) of the present embodiment and may, thus, be omitted from the neural network 24 when used for inference in its inference form, as shown in FIG. 1.Each of the encoder 26, the gait state decoder 28, and the measurement decoder 30 is implemented as a machine learning model, particularly a multi-layered or deep neural network, that functions as a feed-forward “black box” that has learned a relationship between its inputs and outputs, particularly for purposes of causing the neural network 24 to find correlations or mappings between the human gait state and kinematic measurements of the types provided by the sensor(s) 20. An example of a suitable neural network is the Transformer machine learning model which can learn the relationship between kinematic measurements and gait state via inputs of wide ranges of data representing the relationship over a variety of different gait states from a variety of different human subjects, including able-bodied human subjects. For example, some large, open- source gait datasets are available to teach the neural network 24 this relationship, and such datasets may be used for the first training phase 100. The relationship encoded by the neural network 24 can be further tuned to an individual user of the device 10 via further inputs of kinematic measurements taken during the user’s own gait at multiple known gait states. It is contemplated that other existing or future-developed machine learning models may be adapted for use as the primary neural network 24.The measurement decoder 30 is configured to attempt to reconstruct the walking measurements input into the encoder 26 based on the latent representation L output by the encoder 26 for said walking measurements. More particularly, the measurement decoder 30 generates anestimate of the walking measurements based on the latent representation L, and this estimate is then compared with the input so as to determine a reconstruction loss representing the difference between the output of the measurement decoder 30 and the input walking measurements fed into the encoder 26. This process is generally performed after having already trained the encoder 26 using a supervised training process (referred to as a first training task corresponding to a first training phase) utilizing the gait state decoder 28, and this process of the measurement decoder 30 is referred to as a second training task corresponding to a second training phase.With reference to FIGS. 3 and 4, the neural network 24 is trained in two phases: the first training phase 100 (FIG. 3) in which the encoder 26 is trained along with the gait state decoder 28 using supervised learning; and the second training phase 200 (FIG. 4) in which the encoder 26 is further trained using unsupervised learning. The bolder lines of FIG. 3 indicate the portions of the neural network 24 used during the first training phase 100, and the bolder lines of FIG. 4, likewise, indicate portions of the neural network 24 used during the second training phase 200.With reference to FIG. 3, the first training phase 100 results in the encoder 26 learning salient features between walking measurements and gait state in a source domain, particularly one characterized by normative or normal walking — that is, walking without a limb-assistive device. Accordingly, this first training phase 100 embeds or engrains correlations between walking measurements and gait state into parameters of the encoder 26. The first training phase 100 implements a supervised training technique whereby annotated training data is used to train the parameters through backpropagation. More particularly, as shown in FIG. 3, the annotated training data is represented by pairs of source domain walking measurements (shown as being input into the encoder 26) and source domain gait state labels. In the present embodiment, the annotated training data is obtained from an open-source training set, generally large in size and for embodying correlations between walking measurements and gait state in the source domain, as shown in FIG. 3. As a result of training the encoder 26 using this first training phase 100, the encoder 26 as its initial parameters modified through backpropagation into trained parameters, specifically referred to as source-trained parameters as said parameters embody the learned correlations between the walking measurements and the gait state in the source domain. The encoder 26 after being trained using the first training phase 100 may be referred to as a source- trained encoder.With reference to FIG. 4, the second training phase 200 results in adapting the encoder 26, as it exists after the first training phase 100, to the target domain, which is characterized by walking with a limb-assistive device, as shown in FIG. 4. The source-trained parameters of the source- trained encoder 26 of FIG. 4 are further modified based on backpropagation in the second trainingphase 200 based on a calculated reconstruction loss. This reconstruction loss is calculated by using a mean square error (MSE), mean absolute error (MAE), or another like loss function suitable for determining a mathematical representation of the difference between the output of the measurement decoder 30 and the walking measurements input into the encoder 26. For backpropagation, the gradient of the loss function with respect to the neural network’s parameters is calculated to update the parameters or weights of the encoder 26. This may be done using optimization algorithms like stochastic gradient descent (SGD) or other like optimization algorithms known in the art, like the Adam optimizer. The gradients of the reconstruction loss indicate the direction or manner in which the parameters / weights of the encoder 26 should be adjusted to minimize the loss. Similar techniques may be used for the first training phase 100 with respect to calculating the prediction loss using the output of the gait state decoder 28 and source domain gait state labels, as shown in FIG. 3.As a result of the training using both the first training phase 100 and the second training phase 200, the encoder 26 is adapted to generate latent representation of walking measurements in a domain invariant manner, at least with respect to the source and target domains. The fundamental relationship enshrined into the encoder 26 as a result of the first training phase 100 is further improved upon for use in the target domain through the unsupervised target measurement training process embodied in the second training phase 200. This imparts those nuanced relationships in the target domain that come to be as a result of use of a limb-assistive device.The neural network 24, particularly the encoder 26 and the gait state decoder 28, are trained prior to deploying the neural network 24 onto the limb-assistive device 10 or otherwise embodying the trained neural network 24 into a non-transitory computer-readable medium along with computer instructions for performing inference by inputting sensor data captured by the limb- assistive device 10 into the neural network 24.The neural network 24 may be trained offline, or may be trained using both offline and online techniques. As used herein, “offline”, when used in connection with training a model, refers to training that is performed prior to deploying the model (e.g., neural network) for its intended use. For example, the first training phase 100 is performed offline using an opensource training dataset. The first training phase 100 may be performed on a computer separate from the limb-assistive device, and this computer (referred to as an “offline training computer”) may have access to powerful computational resources for training, such as high-end graphics processing units (GPUs), tensor processing units (TPUs), and high-end central processing units (CPUs). The offline training computer may be provisioned using a cloud platform, such as Google™ Cloud orAmazon™ Web Services (AWS).The second training phase 200 may use offline training and / or online training. As used herein, “online”, when used in connection with training a model, refers to training that is performed after the model has been deployed. For example, offline training is used whereby an offline training computer performs the second training phase 200. The offline training computer used to perform the second training phase 200 may be the same or different as the offline training computer used to perform the first training phase 100. In some embodiments, the second training phase 200 is performed online whereby the limb-assistive device continuously or periodically uses observed walking measurements obtained from the sensor(s) 20 of the limb-assistive device 10 to train the encoder 26. Further, it has been recognized that people walk differently with an exoskeleton than without it. In this context, this online learning may be used to exoskeleton data without the need for ground truth labels.According to embodiments, the second training phase 200 is used to train the encoder 26 using training data pertaining to a particular user for which the encoder 26, once trained, will be used. For example, in the case of online training, walking measurements captured by the sensor(s) 20 are used to perform the unsupervised training of the second training phase 200, particularly where the walking measurements are used as both input into the encoder 26 and for comparison with the output of the measurement decoder 30. And, in some embodiments, offline training is used to perform the second training phase 200. This offline execution of the second training phase 200 is performed by recording walking measurements for a particular user using the limb-assistive device 10, transferring the walking measurements to an offline training computer, and then using those recorded measurements for training the encoder 26. This customized training enables the encoder 26 to be adapted for the particular user that is to use the limb-assistive device 10, further improving the accuracy of encoding performed by the encoder 26, at least with respect to the particular user.Any one or more of the processors discussed herein is an electronic processor implemented using any suitable electronic hardware capable of processing computer instructions and may be selected based on the application in which it is to be used. Examples of types of processors that may be used include central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), microprocessors, microcontrollers, etc. Any one or more of the memories discussed herein, unless stated otherwise, is a non-transitory, computer-readable memory that is implemented as any suitable type of memory that is capable of storing data or information in a non-volatile manner and in an electronic form so that the stored data orinformation is consumable by the processor. The memory may be any a variety of different electronic memory types and may be selected based on the application in which it is to be used. Examples of types of memory that may be used include including magnetic or optical disc drives, ROM (read-only memory), solid-state drives (SSDs) (including other solid-state storage such as solid state hybrid drives (SSHDs)), other types of flash memory, hard disk drives (HDDs), nonvolatile random access memory (NVRAM), etc. It should be appreciated that the computers may include other memory, such as volatile or transitory RAM that is used by the processor, and / or multiple processors.It is to be understood that the foregoing description is of one or more embodiments of the invention. The invention is not limited to the particular embodiment(s) disclosed herein, but rather is defined solely by the claims below. Furthermore, the statements contained in the foregoing description relate to the disclosed embodiments) and are not to be construed as limitations on the scope of the invention or on the definition of terms used in the claims, except where a term or phrase is expressly defined above. Various other embodiments and various changes and modifications to the disclosed embodiment(s) will become apparent to those skilled in the art.As used in this specification and claims, the terms “e.g.,” “for example,” “for instance,” “such as,” and “like,” and the verbs “comprising,” “having,” “including,” and their other verb forms, when used in conjunction with a listing of one or more components or other items, are each to be construed as open-ended, meaning that the listing is not to be considered as excluding other, additional components or items. Other terms are to be construed using their broadest reasonable meaning unless they are used in a context that requires a different interpretation. In addition, the term “and / or” is to be construed as an inclusive OR. Therefore, for example, the phrase “A, B, and / or C” is to be interpreted as covering all of the following: “A”; “B”; “C”; “A and B”; “A and C”; “B and C”; and “A, B, and C.”

Claims

CLAIMS1. A method of generating gait state estimations for a user having a limb-assistive device, comprising: training an encoder with a gait state decoder using annotated training data; training the encoder with a measurement decoder using target domain walking measurements; and determining a gait state estimation of the user through executing a neural network comprised of the encoder and the gait state decoder.

2. The method of claim 1, wherein the encoder is trained with the gait state decoder as a part of a first training phase, and wherein the encoder is trained with the measurement decoder as a part of a second training phase that is performed after the first training phase.

3. The method of claim 2, wherein the first training phase uses supervised training in which the annotated training data includes walking measurements paired with gait state labels.

4. The method of claim 3, wherein the second training phase uses unsupervised learning in which the target domain walking measurements are used as both input into the encoder and to determine a reconstruction loss by comparing the target domain walking measurements with output of the measurement decoder.

5. The method of claim 2, wherein the first training phase and the second training phase are performed offline relative to a limb-assistive device worn by a user, and wherein the determining step is performed online by the limb-assistive device.

6. The method of claim 5, wherein the limb-assistive device includes one or more sensors used to determine walking measurements that are input into the neural network for purposes of determining the gait state estimation.

7. The method of claim 6, wherein the walking measurements are determined through sensor data captured by the one or more sensors as the user walks while wearing the limb- assistive device8. A non-transitory computer-readable memory comprising computer instructions that, when executed by one or more processors, determine a gait state estimation of a user through executing a trained neural network, wherein the trained neural network is comprised of a trained encoder and a gait state decoder, and wherein the trained encoder is trained with a measurement decoder using target domain walking measurements.

9. A gait state estimation controller having the non-transitory computer-readable memory of claim 8.

10. A limb-assistive device having the non-transitory computer-readable memory of claim 8.

11. A gait state estimation controller, comprising: at least one processor; and non-transitory computer-readable memory storing computer instructions that, when executed, cause the at least one processor to determine a gait state estimation through inputting sensor data into a trained neural network having an encoder and a gait state decoder; wherein the trained neural network is trained using a training processing that includes: training the encoder with the gait state decoder using annotated training data; and training the encoder with a measurement decoder using target domain walking measurements.

12. The gait state estimation controller of claim 11, wherein the encoder is configured for generating latent encoding data, and wherein the gait state decoder is configured for determining the gait state estimation based on the latent encoding data.

13. The gait state estimation controller of claim 11, wherein the gait state estimation is generated without use of the measurement decoder.

14. A limb-assistive device having the gait state estimation controller of claim 11.

15. The limb-assistive device of claim 14, further comprising one or more sensors for capturing sensor information and causing the sensor data to be provided to the gait state estimation controller.

Images