Device and method for obtaining an optimized control signal for controlling a functional electrical stimulation (FES) system

Abstract

The present invention relates to a method and a device for obtaining a pre-trained deep reinforcement learning (DRL) model. The invention further relates to a method and a device for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system using said pre-trained DRL model.

Description

DEVICE AND METHOD FOR OBTAINING AN OPTIMIZED CONTROL SIGNAL FOR CONTROLLING A FUNCTIONAL ELECTRICAL STIMULATION (FES) SYSTEMFIELD OF INVENTION

[0001] The present invention relates to the field of neuro-rehabilitation systems. In particular, the invention relates to a method and a device for obtaining a pre-trained deep reinforcement learning (DRL) model. The invention further relates to a method and a device for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system using said pre-trained DRL model.BACKGROUND OF INVENTION

[0002] Functional Electrical Stimulation (FES) has emerged as a powerful tool in the rehabilitation and assistance of individuals with motor impairments. This technology utilizes electrical pulses to stimulate muscle contractions, thus facilitating movement and enhancing motor function. Beyond its immediate benefits in muscle activation, FES holds significant promise in leveraging brain plasticity — a key factor in rehabilitation. Brain plasticity, or neuroplasticity, refers to the brain's ability to reorganize itself by forming new neural connections when the old ones become non-functional. This capability is particularly vital for recovery following neurological injuries such as stroke or spinal cord injury.

[0003] The use of FES in rehabilitation is rooted in the principles of activity-dependent plasticity. By synchronizing motor output (muscle activation) with sensory feedback (proprioceptive signals), FES can drive the reorganization of neural circuits involved in motor control. This synchronized activity promotes Hebbian learning (“cells that fire together wire together"), where simultaneous activation of presynaptic and postsynaptic neurons strengthens synaptic connections, facilitating cortical reorganization and motor recovery. Repeatedly engaging motor and sensory pathways through coordinatedmovement and feedback can stimulate cortical areas responsible for motor control, promoting plasticity and recovery.

[0004] Current systems for gait correction using FES rely on a training of a deep learning model using databases of annotated gait recordings, followed by a personalized calibration of stimulation parameters. During setup, the deep learning model may undergo fine-tuning with the patient's specific gait data through transfer learning, ensuring tailored predictions for individual gait patterns. Real-time sensor data collection facilitates continuous gait phase prediction, guiding the FES device to administer electrical pulses at optimal phases within the gait cycle.

[0005] Although these types of systems give satisfying results, there is a need for improvements that address the following challenges. Indeed, current systems often rely on static models and don’t dynamically adjust to changes in the patient's gait over time without a manual intervention or a new personalization phase. Moreover, the systems typically operate in an open-loop manner, with limited adaptability to environmental conditions or evolving patient dynamics, potentially compromising stimulation efficacy and safety in varying contexts.

[0006] Therefore, there is a need for a device and method that addresses the aforementioned challenges.SUMMARY

[0007] This invention thus relates to a device for obtaining a pre-trained deep reinforcement learning (DRL) model, said pre-trained DRL model being configured to output an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said device comprising: at least one input configured to receive: o a training dataset comprising:■ synthetic data obtained by simulating at least one sequence of movements using at least one musculoskeletal model of a person, said at least one musculoskeletal model including a musculoskeletal model of a person having at least one neurological and / or musculoskeletal pathology,■ for each subject of a plurality of subjects having at least one neurological and / or musculoskeletal pathology:• multiple recordings obtained during multiple rehabilitation sessions undergone by said subject, said rehabilitation session being part of a rehabilitation process, wherein for each rehabilitation session, said multiple recordings include: o Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said subject, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording, o at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said subject and / or on at least one hand of said subject, o at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said subject,• a classification of said patient into a Functional Ambulation Category (FAC), an action space definition comprising a set of control actions available to said DRL model for obtaining said optimized control signal, said set of control actions being configured to stimulate at least one muscle and / or nerve, constraint parameters defining acceptable physiological ranges and muscle synergies for said optimized control signal,at least one processor configured to feed said training dataset to an untuned DRL model so as to obtain said pre-trained DRL model and use said action space definition and said constraint parameters to optimize said pre-trained DRL model, wherein said optimization is obtained using a reward function designed to optimize key performance indicators (KPIs) for gait performance, patient safety and rehabilitation efficiency, at least one output configured to provide said pre-trained DRL model.

[0008] Advantageously, the device of the invention allows to obtain a robust pre-trained DRL model specifically designed for FES systems control. Pre-training the DRL model allows to obtain a powerful foundation for personalization. Indeed, when further trained on a patient's specific data, the DRL model will leverage its pre-existing knowledge to refine the control strategy for optimal comfort and performance, leading to a more individualized FES experience for each patient. Moreover, pre-training the DRL model reinforces safety. By incorporating safety constraints and exploring a diverse range of stimulation patterns during this initial training phase, the DRL model is less likely to generate unsafe control strategies later. This broadens the applicability of the DRL model and ensures it operates within safe boundaries defined by the action space. Therefore, pretraining the DRL model facilitates the development of a faster, safer, and more adaptable functional electrical stimulation (FES) control system. This, in turn, contributes to improved efficiency and enhanced patient outcomes.

[0009] According to other advantageous aspects of the invention, the device comprises one or more of the features described in the following embodiments, taken alone or in any possible combination.

[0010] According to one embodiment, said multiple recordings further comprise at least one of: at least one Heart rate (HR) recording obtained using at least one HR sensor positioned on said subject, at least one temperature recording obtained using at least one temperature sensor positioned on said subject, at least one electroencephalogram (EEG) recording obtained using at least one EEG sensor positioned on said subject,at least one camera recording obtained using at least one camera positioned on said subject, said camera being configured to capture an environment of said subject, and at least one oxygen consumption recording obtained using at least one sensor configured to analyze an inhaled and exhaled air of said subject.

[0011] According to one embodiment, said dataset further comprises a classification of said patient into an Upper Extremity Functional Scale (UEFS).

[0012] According to one embodiment, said optimized control signal comprises optimized functional electrical stimulation parameters, said functional electrical stimulation parameters including at least one of a pulse amplitude, a pulse width, a pulse frequency, a pulse ramp, a pulse pattern and a spatial distribution.

[0013] Advantageously, tailoring the functional electrical stimulation (FES) parameters to the individual patient's needs can optimize the therapeutic effects, providing a personalized approach to motor rehabilitation.

[0014] Notably, the FES parameters define the shape of the control signal used to stimulate the patient’s nerves and / or muscles. The control signal comprises a succession of pulses, whose shape, intensity, frequency, width and spatial distribution (e.g. where the pulses are delivered, i.e. where the corresponding electrode is positioned on the patient’s body) can be tuned to patient’s specific needs.

[0015] For instance, the action potentials in the central nervous system (CNS) are frequency modulated, meaning that the intensity of the transmitted signal is proportional to the number of action potentials that occurs per unit time. Typical frequency of the nerve firing is around 4-12 Hz and the firing of the nerve fibers is asynchronous. Depending on the application, a variety of pulse frequencies can be used to generate contractions with FES. The most typical frequencies used in clinical applications range between 20 and 50 Hz. These higher frequencies are needed, because electrical stimulation activates muscle fibers synchronously and as such requires higher firing rates to generate tetanic contractions. Moreover, lower frequency stimulations (< 16 Hz) produce unfused contractions. They could also induce low-frequency fatigue, and they may not always be sufficient to elicit strong contractions. On the other hand, high frequency stimulation (50-80 Hz) can induce rapid onset of muscle fatigue, which is a significant limitation of electrical stimulation systems. However, higher frequencies of stimulation were reported to be more comfortable, because the response is smoothed down.

[0016] The pulse amplitude, or the intensity, by which the stimulation is delivered is related to the depolarizing effect, with higher amplitudes inducing a stronger depolarizing effect. Typical FES pulse amplitudes rarely exceed 100 mA, while the exact levels depend on muscle properties, including the size of the muscle as well as the size of the stimulating electrodes and the pulse width of the simulating waveform. Smaller upper-limb muscles typically require smaller electrodes and lower pulse amplitudes to be contracted (e.g., 10- 20 mA), while larger lower-limb and trunk muscles typically require larger amplitudes (e.g., 20-35 mA for contracting the soleus muscle). Increasing the stimulation amplitude results in additional recruitment of smaller fibers near the electrode and larger fibers farther from the electrode. With increasing amplitude, a threshold is reached beyond which no further fibers can be recruited, and no additional torque generated by the muscles. Moreover, very high intensities could lead to rapid muscle fatigue and discomfort during FES.

[0017] Pulse width, or pulse duration, is the time span of a stimulating pulse. To achieve adequate depolarization of the nerve cells and cause muscles to contract, sufficient pulse width is required. Typical FES pulse width in clinical applications is between 200 and 500 ps. Short pulse durations (10-50 ps) have been shown to be selective in activation of muscle nerves, which can generate larger torque with a small number of muscle fibers. However, very short pulse durations require larger pulse amplitudes to achieve adequate depolarization to contract the muscles. Larger pulse width was shown to produce stronger contractions, in addition to being able to penetrate deeper into subcutaneous tissue.

[0018] Pulses with a ramped shape (also called Ramped Functional Electrical Stimulation (FES)) is commonly used in various clinical and rehabilitation settings, particularly in situations where gradual and controlled muscle activation is desired. It can be used to gradually increase muscle activation levels over time, allowing for a smoother transition and reducing the risk of muscle fatigue. This approach is often beneficial in long-duration rehabilitation sessions or during repetitive tasks where maintaining muscle endurance is crucial. In individuals with hypersensitivity or discomfort, ramped FES canhelp mitigate pain by gradually introducing stimulation to the affected muscles. By slowly ramping up the intensity, the nervous system has more time to adapt, potentially reducing the perception of pain associated with FES. Ramped FES is commonly employed as a safety measure to prevent sudden or jarring muscle contractions, which could lead to falls or injury, especially in individuals with compromised motor control or balance. The gradual increase in stimulation intensity allows for smoother movements and reduces the risk of sudden muscle spasms. During gait training or walking rehabilitation, ramped FES can be used to facilitate smoother transitions between different phases of the gait cycle. By gradually activating specific muscles involved in walking, individuals can achieve a more natural and coordinated gait pattern, enhancing their overall walking ability. Ramped FES can help optimize muscle recruitment by gradually activating different motor units within a muscle. This approach allows for more efficient muscle contractions and may enhance overall movement quality and effectiveness of rehabilitation exercises.

[0019] The ideal FES pattern for muscle activation would be one that produces sufficiently high forces while minimizing fatigue. FES has traditionally consisted of constant-frequency trains (CFT): brief tetanic pulses of stimulation separated by constant interpulse intervals that produce a rapid rate of muscle tension but also rapid fatigue. It has been suggested that a stimulation pattern with variable frequency trains (VFT) (e.g. successive pulses) may limit fatigue development compared to the CFT pattern. VFT begins with 2 (doublet) or 3 (triplet) pulses separated by brief interpulse intervals, followed by regularly spaced pulses with longer interpulse intervals. This pattern is based on the “catchlike phenomenon”; tension is enhanced when an initial brief interpulse interval is added to the beginning of a subtetanic, less-fatiguing, train of pulses. In addition, the VFT pattern has been demonstrated to be better than the CFT pattern at generating force in fatigued muscles.

[0020] To minimize the premature fatigue associated with single electrode stimulation (SES), researchers have developed a technique involving “sequential” rotation of stimulation pulses between multiple active (cathode) electrodes positioned over a muscle belly or a group of muscle bellies. Sequential stimulation crudely mimics the asynchronous pattern and firing frequency range of motor unit recruitment associated with voluntary contractions. During sequential stimulation, each electrode is activated ata low frequency (e.g. 10-15 Hz), whereas maintaining a high composite frequency (e.g. 40-60 Hz when four cathodes are used) delivered to the muscle, or muscle group, as a whole. Spatially distributed sequential stimulation (SDSS) is a method to reduce muscle fatigue by distributing the center of an electrical field over a wider area within a single stimulation site and muscle belly, using an array of surface electrodes. SDSS is unique compared to other sequential stimulation methods in a sense that, whereas the stimulation is interleaved between electrodes, it is not applied to different muscle heads. Instead, SDSS is distributed between multiple active surface electrodes that are placed at the same muscle and over approximately the same area as during SES with a single active electrode. In this way, each SDSS electrode activates partially distinct motor unit populations theoretically reducing motor unit discharge rates of each motor unit population and, subsequently, muscle fatigue compared to SES.

[0021] To generate muscle contraction, the impedance under the electrodes, as well as the location, size and orientation of the electrodes are important for optimizing the current density. Having a smaller cathode electrode and placing it close to the target nerve with the larger anode placed a distance away from the cathode can be used to generate more specific / selective stimulation under the cathode while allowing a larger area of the skin under the anode to be used to close the electrical circuit and minimize discomfort under the cathode. Empirically, it is well-known that there are locations where muscles are most sensitive to electrical stimulation, i.e., motor points. Placement of electrodes on the motor point also plays an important role in generating strong muscle contractions.

[0022] According to one embodiment, said (KPIs) for gait performance include: an improvement in gait speed compared to a baseline, a change in joint angles during gait cycle, and a clinically meaningful difference in at least one gait parameter, said gait parameters including a step length and a cadence.

[0023] According to one embodiment, said (KPIs) for patient safety include: a number of falls or near-falls during one rehabilitation session, and a patient-reported discomfort or pain level.

[0024] According to one embodiment, said (KPIs) for rehabilitation efficiency include:a muscle fatigue level measured using at least one EMG sensor, a neuroplasticity induction, an energy consumption during one rehabilitation session, and a responsiveness and adaptation time of the DRL model to a gait deviation.

[0025] According to one embodiment, said DRL model is a Proximal Policy Optimization (PPO) model or a Deep Q-Network (DQN) model or a combination thereof.

[0026] The present invention further relates to a device for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal being obtained using a pre-trained deep reinforcement learning (DRL) model obtained from the device for obtaining a pre-trained DRL model described above, wherein said device comprises: at least one input configured to receive: o multiple recordings obtained during a rehabilitation session undergone by said patient, said rehabilitation session being part of a rehabilitation process, said multiple recordings including:■ Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said patient, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording,■ at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said patient and / or on at least one hand of said patient,■ at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said patient, at least one processor configured to provide as input to said pre-trained DRL model, said multiple recordings so as to obtain said optimized control signal, at least one output configured to provide said optimized control signal.

[0027] Advantageously, the device of the invention leverages a pre-trained DRL model that can be readily applied, receiving real-time sensor data from the patient and utilizing the pre-existing knowledge within the DRL model to generate optimal stimulation patterns. This translates to faster initiation of FES therapy and improved efficiency in clinical settings. The pre-trained model embodies a vast understanding of effective control strategies, accumulated through extensive training on diverse datasets. This broadens the applicability of the device and ensures consistent, high-quality control even for patients with unique needs or atypical responses to stimulation. Finally, the pre-trained model offers a degree of inherent safety due to the rigorous constraints and safety protocols incorporated during its training phase. This reduces the risk of adverse events and fosters user confidence in the FES therapy delivered by the device.

[0028] According to other advantageous aspects of the invention, the device comprises one or more of the features described in the following embodiments, taken alone or in any possible combination.

[0029] According to one embodiment, said multiple recordings further comprises at least one of: at least one Heart rate (HR) recording obtained using at least one HR sensor positioned on said patient, at least one temperature recording obtained using at least one temperature sensor positioned on said patient, at least one electroencephalogram (EEG) recording obtained using at least one EEG sensor positioned on said patient, at least one camera recording obtained using at least one camera positioned on said patient, said camera being configured to capture an environment of said patient, and at least one oxygen consumption recording obtained using at least one sensor configured to analyze an inhaled and exhaled air of said patient.

[0030] According to one embodiment, said at least one input is further configured to receive:an action space definition comprising a set of control actions available to said DRL model for obtaining said optimized control signal, said set of control actions being configured to stimulate at least one muscle and / or nerve of said patient, constraint parameters defining acceptable physiological ranges and muscle synergies for said optimized control signal, said at least one processor being further configured to re-train said pre-trained DRL model using said multiple recordings obtained during said rehabilitation session so as to obtain a personalized DRL model configured to output a personalized control signal and use said action space definition and said constraint parameters to optimize said personalized DRL model.

[0031] According to one embodiment, said re-training is performed at each rehabilitation session undergone by said patient so as to adapt to changes and improvements of said patient during said rehabilitation process.

[0032] Advantageously, the device of the invention offers continuous adaptation to various factors that can influence FES therapy effectiveness. Unlike conventional systems with fixed-value stimulation, this device dynamically adjusts based on real-time sensor data.

[0033] Notably, the device of the invention is particularly robust to variations in electrode placement, changes in the patient's gait pattern, and fluctuations in internal physiological and external environmental conditions. For example, slight electrode positioning changes during garment application can affect muscle response due to variations in the relationship between electrodes and underlying nerves. The device is capable of adapting the outputted control signal depending on these positioning changes.

[0034] Moreover, by analyzing real-time sensor data, the device identifies subtle changes and improvements in the patient's movement. This iterative learning process allows the system to refine stimulation patterns, ensuring that therapy remains aligned with the patient's current abilities and progress. This results in increasingly effective and personalized treatment, promoting faster and more sustainable rehabilitation outcomes.

[0035] Beyond user-related factors, the device also adapts to environmental changes such as terrain variations (flat ground, stairs, inclines, slippery surfaces) and different use modes (walking, cycling, rowing). Additionally, it can accommodate physiological changes like muscle fatigue (measured by EMG), energy expenditure (pulse oximetry), and cognitive state (EEG).

[0036] Furthermore, the device incorporates user intention through signals from various sensors (EEG, EMG, motion, pressure). This allows the system to recognize actions like sit-to-stand, gait initiation, stopping, turning, etc., and adjust stimulation accordingly.

[0037] Finally, the device prioritizes user comfort by monitoring movement smoothness and EMG reflex responses. By modifying stimulation parameters based on this data, the device achieves seamless and pleasant interaction with the body.

[0038] Overall, the device of the invention offers a versatile solution for FES therapy, applicable to diverse cases across various terrains, workspaces, and activities. This realtime control system can be used for both lower and upper limbs, providing a wider range of potential applications. In rehabilitation settings, the device can be integrated into therapies to enhance motor recovery by facilitating improved synchronization between motor output and sensory feedback. This has the potential to lead to more effective and faster rehabilitation outcomes compared to traditional methods that lack FES integration. Furthermore, the system can promote neuroplasticity by stimulating synaptic remodeling and neural reorganization. This can enhance the brain's natural ability to adapt and recover from injuries, potentially improving long-term neurological outcomes for patients. Beyond rehabilitation, the device's ability to personalize FES parameters allows for tailored treatment approaches. By adjusting stimulation intensity, timing, and frequency based on individual needs, therapists can optimize the therapeutic effects for each patient. Finally, the system can also provide valuable assistance with daily activities for individuals with motor impairments. This includes assistance with walking, reaching, and grasping objects, all of which are essential for independent living and improved quality of life.

[0039] The present invention further relates to a computer implemented method for obtaining a pre-trained deep reinforcement learning (DRL) model, said DRL model being configured to output an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said method comprising: receiving: o a training dataset comprising:■ synthetic data obtained by simulating at least one sequence of movements using at least one musculoskeletal model of a person, said at least one musculoskeletal model including a musculoskeletal model of a person having at least one neurological and / or musculoskeletal pathology,■ for each subject of a plurality of subjects having at least one neurological and / or musculoskeletal pathology:• multiple recordings obtained during multiple rehabilitation sessions undergone by said subject, said rehabilitation session being part of a rehabilitation process, wherein for each rehabilitation session, said multiple recordings include: o Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said subject, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording, o at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said subject and / or on at least one hand of said subject,o at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said subject,• a classification of said patient into a Functional Ambulation Category (FAC), o an action space definition comprising a set of control actions available to said DRL model for obtaining said optimized control signal, said set of control actions being configured to stimulate at least one muscle and / or nerve, o constraint parameters defining acceptable physiological ranges and muscle synergies for said optimized control signal, feeding said training dataset to an untuned DRL model so as to obtain said pretrained DRL model and use said action space definition and said constraint parameters to optimize said pre-trained DRL model, wherein said optimization is obtained using a reward function designed to optimize key performance indicators (KPIs) for gait performance, patient safety and rehabilitation efficiency, outputting said pre-trained DRL model.

[0040] The present invention further relates to a computer implemented method for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal being obtained using a pre-trained deep reinforcement learning (DRL) model obtained from the method according to claim 12, wherein said method comprises: receiving: o multiple recordings obtained during a rehabilitation session undergone by said patient, said rehabilitation session being part of a rehabilitation process, said multiple recordings including:■ Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said patient, said at least one recordingincluding at least one joint angle recording and at least one joint angle velocity recording,■ at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said patient and / or on at least one hand of said patient,■ at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said patient, providing as input to said pre-trained DRL model, said multiple recordings so as to obtain said optimized control signal, outputting said optimized control signal.

[0041] In addition, the disclosure relates to a computer program comprising software code adapted to perform a computer implemented method compliant with any of the above execution modes when the program is executed by a processor.

[0042] The present disclosure further pertains to a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the methods compliant with any of the above execution modes.

[0043] The present disclosure further relates to a non-transitory program storage device (i.e. computer-readable storage medium), readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the computer- implemented methods, compliant with the present disclosure.

[0044] Such a non-transitory program storage device can be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any suitable combination of the foregoing. It is to be appreciated that the following, while providing more specific examples, is merely an illustrative and not exhaustive listing as readily appreciated by one of ordinary skill in the art: a portable computer diskette, a hard disk, a ROM, an EPROM (Erasable Programmable ROM) or a Flash memory, a portable CD-ROM (Compact-Disc ROM).DEFINITIONS

[0045] In the present invention, the following terms have the following meanings:

[0046] The terms “adapted” and “configured” are used in the present disclosure as broadly encompassing initial configuration, later adaptation or complementation of the present device, or any combination thereof alike, whether effected through material or software means (including firmware).

[0047] The term “processor” should not be construed to be restricted to hardware capable of executing software, and refers in a general way to a processing device, which can for example include a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). The processor may also encompass one or more Graphics Processing Units (GPU), whether exploited for computer graphics and image processing or other functions. Additionally, the instructions and / or data enabling to perform associated and / or resulting functionalities may be stored on any processor- readable medium such as, e.g., an integrated circuit, a hard disk, a CD (Compact Disc), an optical disc such as a DVD (Digital Versatile Disc), a RAM (Random- Access Memory) or a ROM (Read-Only Memory). Instructions may be notably stored in hardware, software, firmware or in any combination thereof.

[0048] “Machine learning (ML)” designates in a traditional way computer algorithms improving automatically through experience, on the ground of training data enabling to adjust parameters of computer models through gap reductions between expected outputs extracted from the training data and evaluated outputs computed by the computer models.

[0049] A “hyper-parameter” presently means a parameter used to carry out an upstream control of a model construction, such as a remembering-forgetting balance in sample selection or a width of a time window, by contrast with a parameter of a model itself, which depends on specific situations. In ML applications, hyper-parameters are used to control the learning process.

[0050] “Datasets” are collections of data used to build an ML mathematical model, so as to make data-driven predictions or decisions. In “supervised learning” (i.e. inferring functions from known input-output examples in the form of labelled training data), threetypes of ML datasets (also designated as ML sets) are typically dedicated to three respective kinds of tasks: “training”, i.e. fitting the parameters, “validation”, i.e. tuning ML hyperparameters (which are parameters used to control the learning process), and “testing”, i.e. checking independently of a training dataset exploited for building a mathematical model that the latter model provides satisfying results.

[0051] A “neural network (NN)” designates a category of ML comprising nodes (called “neurons”), and connections between neurons modeled by “weights”. For each neuron, an output is given in function of an input or a set of inputs by an “activation function”. Neurons are generally organized into multiple “layers”, so that neurons of one layer connect only to neurons of the immediately preceding and immediately following layers.

[0052] The above ML definitions are compliant with their usual meaning, and can be completed with numerous associated features and properties, and definitions of related numerical objects, well known to a person skilled in the ML field. Additional terms will be defined, specified or commented wherever useful throughout the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description of particular and non- restrictive illustrative embodiments, the description making reference to the annexed drawings wherein:

[0054] Figure 1 is a block diagram representing schematically a particular mode of a device for obtaining a pre-trained deep reinforcement learning (DRL) model, compliant with the present disclosure;

[0055] Figure 2 is a flow chart showing successive steps executed with the device for obtaining a pre-trained deep reinforcement learning (DRL) model of figure 1 ;

[0056] Figure 3 is a block diagram representing schematically a particular mode of a device for obtaining an optimized control signal for controlling a Functional ElectricalStimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal being obtained using a pre-trained deep reinforcement learning (DRL) model obtained from the device of figure 1 ;

[0057] Figure 4 is a flow chart showing successive steps executed with the device for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system of figure 3.

[0058] Figure 5 shows an apparatus integrating the functions of the device for obtaining a pre-trained deep reinforcement learning (DRL) model of figure 1 and of the device for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system of figure 3.

[0059] On the figures, the drawings are not to scale, and identical or similar elements are designated by the same references.ILLUSTRATIVE EMBODIMENTS

[0060] The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.

[0061] All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.

[0062] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed inthe future, i.e., any elements developed that perform the same function, regardless of structure.

[0063] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein may represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0064] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0065] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input / output interfaces.

[0066] The present disclosure will be described in reference to a particular functional embodiment of a device 1 for obtaining a pre-trained deep reinforcement learning (DRL) model 30, said pre-trained DRL model 30 being configured to output an optimized control signal 51 for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal 51 being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, as illustrated on Figure 1.

[0067] The device 1 is adapted to receive as an input for the training: synthetic data 22 obtained by simulating at least one sequence of movements using at least one musculoskeletal model of a person, and for each subject of a plurality of subjects having at least one neurological and / or musculoskeletal pathology, multiple recordings 21 obtained during multiple rehabilitation sessions undergone by said subject, together witha classification 25 of said subject into a Functional Ambulation Category (FAC), optionally a classification 26 of said subject into an Upper Extremity Functional Scale (UEFS), an action space definition 23 comprising a set of control actions available to said DRL model for obtaining said optimized control signal 51, constraint parameters 24 defining acceptable physiological ranges and muscle synergies for said optimized control signal, and an untuned DRL model 20.

[0068] The device 1 is configured to output a pre-trained DRL model 30 using a training dataset and training parameters constructed (or received) with the above-mentioned data received as input.

[0069] The device 1 for obtaining a pre-trained deep reinforcement learning (DRL) model 30 is associated with a device 2 for obtaining an optimized control signal 51 for controlling a Eunctional Electrical Stimulation (EES) system comprising multiple electrodes worn by a patient, said optimized control signal 51 being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal 51 being obtained using the pretrained deep reinforcement learning (DRL) model 30 obtained with the device 1, as represented on Figure 3, which will be subsequently described.

[0070] Though the presently described devices 1 and 2 are versatile and provided with several functions that can be carried out alternatively or in any cumulative way, other implementations within the scope of the present disclosure include devices having only parts of the present functionalities.

[0071] Each of the devices 1 and 2 is advantageously an apparatus, or a physical part of an apparatus, designed, configured and / or adapted for performing the mentioned functions and produce the mentioned effects or results. In alternative implementations, any of the device 1 and the device 2 is embodied as a set of apparatus or physical parts of apparatus, whether grouped in a same machine or in different, possibly remote, machines. The device 1 and / or the device 2 may have functions distributed over a cloud infrastructure and be available to users as a cloud-based service, or have remote functions accessible through an API.

[0072] The device 1 and the device 2 may be integrated in a same apparatus or set of apparatus, and intended to same users. In other implementations, the structure of the device 2 may be completely independent of the structure of the device 1, and may be provided for other users. For example, the device 2 may have a pre-trained DRL model 30 available to operators for obtaining an optimized control signal 51 for controlling a Functional Electrical Stimulation (FES) system, wholly set from previous training effected upstream by other players with the device 1.

[0073] In what follows, the modules are to be understood as functional entities rather than material, physically distinct, components. They can consequently be embodied either as grouped together in a same tangible and concrete component, or distributed into several such components. Also, each of those modules is possibly itself shared between at least two physical components. In addition, the modules are implemented in hardware, software, firmware, or any mixed form thereof as well. They are preferably embodied within at least one processor of the device 1 or of the device 2.

[0074] The device 1 comprises a module 11 configured to receive as input (or construct) a training dataset comprising synthetic data 22, multiple recordings 21 obtained on a plurality of subjects, a classification of each subject into a Functional Ambulation Category (FAC) and optionally a classification of each subject into an Upper Extremity Functional Scale (UEFS).

[0075] The synthetic data 22 are obtained by simulating at least one sequence of movements using at least one musculoskeletal model of a person. The musculoskeletal model may be configured to simulate the biomechanics of the lower and / or upper limbs during activities such as walking or reaching or grasping. For instance, OpenSim may be used to obtain the musculoskeletal model. The at least one musculoskeletal model may comprise a musculoskeletal model of a person having at least one neurological and / or musculoskeletal pathology such as hemiplegia. In that case, the musculoskeletal model may be obtained by neutralizing or reducing control over some muscles to simulate the pathology. Additionally, the at least one musculoskeletal model may comprise a musculoskeletal model of a healthy person.

[0076] The multiple recordings 21 are obtained for each subject of a plurality of subjects having at least one neurological and / or musculoskeletal pathology. Typically, at least 100 subjects with diverse pathologies and varying FAC and / or UEFS classifications may be selected. The recordings may be collected during a plurality of successive rehabilitation sessions, as part of a rehabilitation process. The subjects may have their data collected during for instance at least two successive rehabilitation sessions, and between 50 rehabilitation sessions and 1000 rehabilitation sessions, thus enabling the analysis of progress, adaptation, and variability in response to therapy and environmental factors (i.e. temporal evolution). The elapsed time between two successive rehabilitation sessions may be consistently the same. Alternatively, the elapsed time between two successive rehabilitation sessions may vary. The time between two successive rehabilitation sessions may be comprised between a few days and a few years. During each rehabilitation session, the patient may be asked to perform at least one rehabilitation activity, with ongoing evolution over time. Such rehabilitation activity may involve physical therapy to improve strength, flexibility, and mobility; occupational therapy to help subjects regain independence in daily activities and cognitive therapy to enhance memory and problemsolving skills. The rehabilitation activities are preferably tailored to each subject's condition and rehabilitation goals. The subjects included in the dataset should preferably encompass a range of neurological and musculoskeletal pathologies, ensuring representation across different levels of mobility impairment and rehabilitation needs. For instance, such neurological and musculoskeletal pathologies encompass conditions such as multiple sclerosis, Parkinson's disease, osteoarthritis, rheumatoid arthritis, muscular dystrophy etc.

[0077] Notably, the recordings 21 may include at least one of the following recordings.

[0078] Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said subject. Notably, seven or more IMUs may provide real-time orientation, acceleration, and angular velocity data from key positions on the lower limbs (e.g. hip, thighs, shanks, and feet) or upper limbs (e.g. back, upper and lower arm, palm, and finger phalanges). This data is crucial for understanding the subject's movement patterns and posture.

[0079] At least one pressure sensor recording obtained using at least one pressure sensor. The pressure sensors may be Insole Pressure Sensors (e.g. pressure sensors comprised in an insole of a shoe) configured to detect pressure distribution and foot loading information. This input helps analyzing gait dynamics and foot pressure distribution during movement. The pressure sensor may further comprise Fingertip Pressure Sensors configured to detect pressure distribution information from a glove positioned on the hand of the subject or from the objects in the workspace. This input helps analyzing the strength of the grasp for manipulation with objects.

[0080] At least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said subject. Electromyography data capture electrical activity in muscles, providing insights into muscle activation patterns, fatigue levels during FES stimulation, and reflex reactions to nociceptive stimulation.

[0081] At least one Heart rate (HR) recording obtained using at least one HR sensor positioned on said subject. Heart rate may be captured from the electrical activity of the heart, providing insights into energy consumption during FES stimulation and safety during exercise.

[0082] At least one temperature recording obtained using at least one temperature sensor positioned on said subject. Temperature data on different body segments may provide insights into energy consumption during FES stimulation.

[0083] At least one electroencephalogram (EEG) recording obtained using at least one EEG sensor positioned on said subject. Real-time brain activity data may provide insights into cognitive and neurological states, aiding in the assessment of subject intent and mental readiness for movement tasks, as well as improving brain plasticity.

[0084] At least one camera recording obtained using at least one camera positioned on said subject, said camera being configured to capture an environment of said subject. Visual data from cameras enable environmental perception, including obstacle detection, terrain analysis, and gait observation for lower limbs, and / or detection of objects in the workspace, selection of the type of grasp and aperture of the hand, and detection of the start and end points of the hand trajectory for upper limbs.

[0085] At least one oxygen consumption recording obtained using at least one sensor configured to analyze an inhaled and exhaled air of said subject.

[0086] The classification 25 of said subject into a Functional Ambulation Category (FAC), allows to capture variations in functional walking ability, ranging from limited mobility to independent ambulation. For instance, the classification may be the following:Level 0: Non-functional ambulation (cannot walk or requires significant help).Level 1: Ambulates in parallel bars (needs help for balance or safety).Level 2: Ambulates on level surfaces but requires assistance for balance or coordination.Level 3: Ambulates on level surfaces with supervision or minimal assistance.Level 4: Ambulates independently on level surfaces but may need supervision or assistance on stairs or uneven surfaces.Level 5: Ambulates independently anywhere.

[0087] The classification 26 of said subject into an Upper Extremity Functional Scale (UEFS), allows to capture variations in functional reaching & grasping ability, ranging from limited mobility to independent performance of ADL.

[0088] The Upper Extremity Functional Scale (UEFS) may be a self-reported questionnaire designed to assess the functional status of an individual's upper extremities, including the arms, shoulders, and hands. It evaluates the difficulty an individual experiences while performing various activities that require upper limb function. The UEFS typically consists of multiple items where respondents rate their ability to perform specific tasks on a scale, often ranging from "no difficulty" to "unable to do".

[0089] Module 11 may be further configured to receive as input an action space definition 23 comprising a set of control actions available to the DRL model for obtaining said optimized control signal 51. In other words, the action space represents the FES stimulation patterns for different muscles and / or nerves involved in gait or reaching & grasping. Factors like muscle activation levels, timing of stimulation, comfort, and muscle fatigue reduction strategy may be considered.

[0090] Module 11 may be further configured to receive constraint parameters 24 defining acceptable physiological ranges and muscle synergies for said optimized controlsignal 51. The constraint parameters may be specific to the general population or to a specific sub-type of population (e.g. hemiplegic subjects). They allow to avoid convergence of the DRL model to local extrema outside physiological constraints (e.g., reducing the acceptable range of target angles for stroke subjects with contractures).

[0091] Module 11 may be further configured to receive the untuned DRL model 20.

[0092] The input data may be stored in one or more local or remote database(s) 10. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically-Erasable Programmable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk).

[0093] The device 1 further comprises optionally a module 12 for preprocessing the input data. For instance, module 12 may be configured to remove noise and artifacts from the recordings, synchronize the recordings, perform segmentation, labeling, features extraction, normalization, scaling etc.

[0094] The device may further comprise a module 13 configured to feed said training dataset to an untuned DRL model 20 so as to obtain said pre-trained DRL model 30 and use said action space definition 23 and said constraint parameters 24 to optimize said pretrained DRL model 30.

[0095] Advantageously, Deep Reinforcement Learning (DRL) is a branch of machine learning that combines deep neural networks with reinforcement learning principles. DRL is well-suited for the present invention as, once trained, it is capable of providing optimized control signals 51 through interaction with its environment (the subject's movement dynamics captured by the sensors) and internal body state (fatigue, cognitive state, energy consumption) to achieve defined objectives (maximizing therapeutic efficacy, daily movement assistance, or subject comfort).

[0096] Notably, a Deep Reinforcement Learning (DRL) architecture involves an agent interacting with an environment to maximize cumulative rewards (through a reward function), guided by the action space definition 23 and constrained by specific constraint parameters 24. The reward function quantifies the success of actions, providing feedback that the agent uses to learn and improve. The action space defines all possible actions theagent can take, whether discrete or continuous, while constraint parameters ensure the agent's actions remain within feasible and safe bounds. Advanced reinforcement learning algorithms like Proximal Policy Optimization (PPO) and Deep Q-Networks (DQN) may drive the learning process. PPO optimizes the policy directly using a clipped objective to maintain stability, while DQN employs a Q-value function approximation with experience replay and target networks to guide action selection. This structured approach enables the agent to iteratively refine its strategy, achieving optimal performance in complex environments.

[0097] According to the invention, the untuned DRL model 30 comprises a reward function designed to optimize key performance indicators (KPIs) for gait performance, subject safety and rehabilitation efficiency. Advantageously, the reward function is designed to incentivize correct gait or reaching and grasping patterns, considering factors like speed, stability, symmetry, energy efficiency, joint angle ranges, muscle fatigue, cognitive state, comfort, and clinical objectives.

[0098] The KPIs for gait performance include: an improvement in gait speed compared to a baseline, a change in joint angles during gait cycle, and a clinically meaningful difference in at least one gait parameter, said gait parameters including a step length and a cadence.

[0099] The KPIs for subject safety include: a number of falls or near-falls during one rehabilitation session, and a subject-reported discomfort or pain level.

[0100] The KPIs for rehabilitation efficiency include: a muscle fatigue level measured using at least one EMG sensor, a neuroplasticity induction, an energy consumption during one rehabilitation session, and a responsiveness and adaptation time of the DRL model to a gait deviation.

[0101] In the process of developing the DRL architecture, the implementation of the neural networks and the overall learning algorithms may be carried out using a specialized software library or framework such as TensorElow or PyTorch.

[0102] It may be observed that the operations by the modules 11, 12 and 13 are not necessarily successive in time, and may overlap, proceed in parallel or alternate, in any appropriate manner. For example, a new recording may be progressively received over time and preprocessed, while the module 13 is dealing with the previously obtained recording. In alternative examples, a batch of recordings 21 may be fully received and preprocessed before it is submitted to the module 13.

[0103] In its automatic actions, the device 1 may for example execute the following process (Figure 2): receiving a training dataset comprising synthetic data 22 and for each subject of a plurality of subjects, multiple recordings 21 obtained during multiple rehabilitation sessions undergone by said subject, together with a classification of said subject in a FAC classification 25 and optionally in a UEFS classification, in addition with constraint parameters 24, an action space definition 23 and an untuned DRL model 20 (step 41), optionally preprocessing the input data (step 42), feeding the training dataset training dataset to an untuned DRL model 20 so as to obtain said pre-trained DRL model 30 and use said action space definition 23 and said constraint parameters 24 to optimize said pre-trained DRL model 30, wherein said optimization is obtained using a reward function designed to optimize key performance indicators (KPIs) for gait performance, subject safety and rehabilitation efficiency (step 43).

[0104] The present invention also relates to the device 2 for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal being obtained using a pre-trained deep reinforcement learning (DRL) model obtained from the device 1. The device 2 will be described in reference to a particular function embodiment as illustrated in Figure 3.

[0105] The device 2 is adapted to receive as input multiple recordings obtained during a rehabilitation session undergone by said patient and to use the pre-trained DRL model to obtain the optimized control signal 51 for controlling a Functional Electrical Stimulation (FES) system.

[0106] The recordings 31 may be continuously collected, and the optimized control signal 51 may be adapted in real-time using closed-loop control to respond to changes in the patient's gait and environmental conditions.

[0107] To that end, the device 2 may comprise a module 61 configured to receive as input the multiple recordings 31 obtained on said patient and the pre-trained DRL model 30.

[0108] The multiple recordings 31 of said patient may notably be collected during a rehabilitation session as part of a rehabilitation process.

[0109] During the rehabilitation session, the patient may be asked to perform at least one rehabilitation activity, with ongoing evolution over time. Such rehabilitation activity may involve physical therapy to improve strength, flexibility, and mobility; occupational therapy to help patients regain independence in daily activities and cognitive therapy to enhance memory and problem- solving skills. The rehabilitation activities are preferably tailored to each patient's condition and rehabilitation goals. Notably, the recordings 31 may include at least one of the following recordings.

[0110] Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said patient. Notably, seven or more IMUs may provide real-time orientation, acceleration, and angular velocity data from key positions on the lower limbs (e.g. hip, thighs, shanks, and feet) or upper limbs (e.g. back, upper and lower arm, palm, and finger phalanges). This data is crucial for understanding the patient's movement patterns and posture.

[0111] At least one pressure sensor recording obtained using at least one pressure sensor. The pressure sensors may be Insole Pressure Sensors (e.g. pressure sensors comprised in an insole of a shoe) configured to detect pressure distribution and foot loading information. This input is essential for analyzing gait dynamics and foot pressure distribution during movement. The pressure sensor may further comprise FingertipPressure Sensors configured to detect pressure distribution information from a glove positioned on the hand of the patient or from the objects in the workspace. This input is essential for analyzing the strength of the grasp for manipulation with objects.

[0112] At least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said patient. Electromyography data will capture electrical activity in muscles, providing insights into muscle activation patterns, fatigue levels during FES stimulation, and reflex reactions to nociceptive stimulation.

[0113] At least one Heart rate (HR) recording obtained using at least one HR sensor positioned on said patient. Heart rate will be captured from the electrical activity of the heart, providing insights into energy consumption during FES stimulation and safety during exercise.

[0114] At least one temperature recording obtained using at least one temperature sensor positioned on said patient. Temperature data on different body segments will provide insights into energy consumption during FES stimulation.

[0115] At least one electroencephalogram (EEG) recording obtained using at least one EEG sensor positioned on said patient. Real-time brain activity data will provide insights into cognitive and neurological states, aiding in the assessment of patient intent and mental readiness for movement tasks, as well as improving brain plasticity.

[0116] At least one camera recording obtained using at least one camera positioned on said patient, said camera being configured to capture an environment of said patient. Visual data from cameras will enable environmental perception, including obstacle detection, terrain analysis, and gait observation for lower limbs, and / or detection of objects in the workspace, selection of the type of grasp and aperture of the hand, and detection of the start and end points of the hand trajectory for upper limbs.

[0117] At least one oxygen consumption recording obtained using at least one sensor configured to analyze an inhaled and exhaled air of said patient.

[0118] The input data may be stored in one or more local or remote database(s) 10. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically-ErasableProgrammable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk).

[0119] The input data may be stored in one or more local or remote database(s) 10. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically-Erasable Programmable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk). In advantageous embodiments, the pre-trained DRL model 30 and all its training parameters may have been previously generated by a system including the device 1. Alternatively, the pre-trained DRL model 30 and its training parameters may be received from a communication network.

[0120] Advantageously, before reception of the input data by module 11, baseline data from the patient may be collected to initialize the pre-trained DRL model 30, followed by a short calibration process where the pre-trained DRL model 30 learns initial parameters.

[0121] The device 2 further comprises optionally a module 62 for preprocessing the input data. Lor instance, module 62 may be configured to remove noise and artifacts from the recordings, synchronize the recordings, perform segmentation, labeling, features extraction, normalization, scaling etc.

[0122] The device 2 may further comprise a module 63 configured to provide as input to said pre-trained DRL model 31, said multiple recordings 31 so as to obtain said optimized control signal 51.

[0123] The optimized control signal 51 comprises optimized functional electrical stimulation parameters defining the optimized control signal 51. These optimized functional electrical stimulation parameters notably include the stimulation intensity, the pulse width, the frequency, the stimulation ramp (rate of increase / decrease in stimulation), the timing of muscle contractions, the use of doublets / triplets (multiple pulses in quick succession), the spatially distributed sequential stimulation (SDSS), the position and selection of electrodes among plurality of electrodes of the EES system, the position and selection of active recording electrodes (EMG, EEG, ECG) etc.

[0124] The device 2 will autonomously optimize the functional electrical stimulation parameters based on real-time patient assessments and environmental cues.

[0125] Additionally, mechanisms may be implemented for the system to learn from each rehabilitation session, adapting to daily variations and long-term progress of the patient, while ensuring safety by setting limits on parameter variations to prevent falls or injuries. This approach leverages principles from sim-to-real transfer in deep reinforcement learning, which addresses the challenge of transferring learning from simulated environments to real- world applications

[0126] To that end, module 11 may be further configured to receive a classification of said patient into the Functional Ambulation Category (FAC), optionally a classification of said patient into the Upper Extremity Functional Scale (UEFS), an action space definition and constraint parameters in order to re-train the pre-trained DRL model 30 with the patient’ s data.

[0127] The action space definition and the constraint parameters may be the same as for pre-training the DRL model (with device 1) or a new action space definition and new constraint parameters tailored to the patient’s needs.

[0128] For retraining the pre-trained DRL model 30, the patient may have its data (e.g. classification into FAC and optionally UEFS and recordings 31) collected during only one rehabilitation session or multiple rehabilitation sessions, such as for instance at least two successive rehabilitation sessions, and between 50 rehabilitation sessions and 1000 rehabilitation sessions. The elapsed time between two successive rehabilitation sessions may be consistently the same. Alternatively, the elapsed time between two successive rehabilitation sessions may vary. The time between two successive rehabilitation sessions may be comprised between a few days and a few years.

[0129] Re-training may notably be performed at each rehabilitation session undergone by said patient so as to adapt to changes and improvements of said patient during said rehabilitation process.

[0130] For retraining the pre-trained DRL model 30, the device 2 may further comprise training module (not represented) configured to feed a training dataset comprising the recordings 31 collected during at least one rehabilitation session on said patient and the classification into the FAC and optionally UEFS to the pre-trained DRL model 30 so asto obtain said re-trained DRL model and use said action space definition and said constraint parameters to optimize said re-trained DRL model.

[0131] The same reward function and KPIs as used in device 1 may be used for retraining. Alternatively, KPIs tailored to the patient’s needs may be used.

[0132] The device 2 may interact with a user interface 18, via which information can be entered and retrieved by a user. The user interface 18 includes any means appropriate for entering or retrieving data, information or instructions, notably visual, tactile and / or audio capacities that can encompass any or several of the following means as well known by a person skilled in the art: a screen, a keyboard, a trackball, a touchpad, a touchscreen, a loudspeaker, a voice recognition system.

[0133] For instance, an intuitive interface may be provided for clinicians and patients to monitor progress and make manual adjustments if necessary. Feedback from them may be used to continuously improve the system's algorithms and performance.

[0134] In its automatic actions, the device 2 may for example execute the following process (Figure 4): receiving multiple recordings 31 obtained during a rehabilitation session undergone by said patient (step 141), optionally preprocessing the input data (step 142), feeding said input data to said pre-trained DRL model 30 as to obtain said optimized control signal 51 (step 143).

[0135] An example of a FES system compliant with the invention may include the following elements.

[0136] A first element is a multi-channel electrostimulator configured to send electric pulses to the muscles of the patient to stimulate them.

[0137] The electro stimulator includes a portable electrical muscle stimulation (EMS) generator that may be housed within a housing such as a bag or a vest. The bag may be carried on the back or on the front or on one shoulder, or around the waist. Alternatively, the generator may be housed in a pocket of an article of clothing, such as trousers.

[0138] The generator may deliver a voltage comprised between 0 and 350 V and a current intensity comprised between 0 and 170 mA. The generator is associated with an intensity variator to deliver a pulse sequence with a pre-determined intensity, width and frequency.

[0139] The generator may have several channels that can be each configured to deliver a different pulse sequence.

[0140] The generator is in electrical connection with a set of electrodes configured to be positioned on the skin of the patient, either on the motor points of muscles, where nerve endings enter the muscle or on sensory receptors in the skin that are innervated or at sensory nerves endings.

[0141] The electrodes may be connected to the generator via conductive wires, electrical traces, conductive fibers, or a combination thereof. The conductive wires, electrical traces, conductive fibers, or a combination thereof can be embedded within the article of clothing, for instance in a layer of the article of clothing or interwoven with fibers used to make the article of clothing.

[0142] The electrodes may comprise several layers including a contact layer configured to attach to the patient’s skin, a connector layer configured to be connected to the generator and conductive layers positioned between the contact layer and connector layer. The contact layer can be made of a biocompatible polymeric layer. It may include an adhesive and / or a hydrogel. Alternatively, the contact layer may be dry and may require that a conductive gel or a hydrating lotion hydrates the skin surface of the patient.

[0143] The electrodes may be coupled to an inner surface of the article of clothing by adhesives, clips, straps, hook-and-loop fasteners, stitches or a combination thereof. The electrodes may be positioned such that when the patient puts the article of clothing on, the contact layer of the electrodes is automatically positioned in contact with the skin over the motor points, the sensory receptors or the sensory nerves endings that needs to be stimulated. Advantageously, the electrodes can be detached from the article of clothing to allow replacing a defective electrode.

[0144] A second element of the FES system is at least one IMU sensor and at least one pressure detection unit. The IMU sensor may be the as the one used to capture the recordings 31.

[0145] The IMU sensor includes at least one motion sensor such as an accelerometer, a gyroscope, a magnetometer and / or a combination thereof. Advantageously, the IMU sensor includes three motion sensors for each leg and one motion sensor for the hip. A first motion sensor may be positioned on the thigh, a second motion sensor may be positioned on the calf and a third motion sensor may be positioned on the foot. The motion sensors are configured to sense the three-dimensional movements of the leg throughout a gait cycle. The motion sensors may be embedded within the article of clothing, for instance in a layer of the article of clothing or interwoven with fibers used to make the article of clothing.

[0146] The pressure detection unit includes pressure sensors that may be positioned under the feet to sense the strength of a contact of the different parts of the patient’s feet with the ground. The expression “strength of a contact” refers to the pressure exerted by a at least one region of one foot on the ground, the pressure being defined as a force per unit of area. Advantageously, the pressure sensors may be included in an insole or a sole of a shoe. For instance, the pressure detection unit may include between 1 and 1000 sensors distributed on the sole to collect data on the pressure applied, the position and the pressure changes when the patient is walking, running, jumping, climbing, descending, sitting or standing. In a preferred embodiment, there are five pressure sensors per foot. Two pressure sensors may be positioned under the heel, on the medial and lateral side, and a third pressure sensor may be positioned under the big toe. The fourth and fifth sensors may be positioned under the metatarsal bones.

[0147] The pressure sensors may be capacitive sensors, resistive sensors, piezoelectric sensors, piezoresistive sensors or a combination thereof. The pressure sensors provide an electrical signal output, which is either a voltage or a current, that is proportional to the pressure exerted on said pressure sensors.

[0148] Advantageously, the FES system may include other sensors such as electromyography sensors to access muscle fatigue and movement intention or encoderspositioned at the legs joints to access the angular position of the different parts on the legs.

[0149] Additionally, as previously mentioned, the FES system may include other sensors such as, but not limited to a temperature sensor, EEG sensor, oxygen sensor, movement sensor (camera), heart rate sensor etc.

[0150] For instance, the patient motion data coming from the motion sensors and the patient foot plantar pressure data coming from the pressure sensors may be used to determine the foot strike pattern, the foot inclination angle, the tibia angle, the hip flexion and extension, the trunk lean, the ankle inversion and eversion, the foot progression angle, the pelvic drop, the knee flexion and extension, the stride length, or the displacement of the center of mass, the speed of gait, the cadence, etc.

[0151] A third element of the FES system is a device 2 for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system as described above. Such device 2 may be housed within the housing with the generator. Such device 2 may communicate with a control device such as a tablet, a PC or a smartphone, hosting the user interface. The communication may be wireless, using Bluetooth, WiFi or other protocols. The control device may be used to parameter the device for electrical neuromuscular stimulation and visualize information on the state of the device for electrical neuromuscular stimulation and on how the patient is faring.

[0152] A particular apparatus 9, visible on Figure 4, is embodying the device 1 as well as the device 2 described above. It corresponds for example to a workstation, a laptop, a tablet, a smartphone, or a head- mounted display (HMD).

[0153] That apparatus 9 is suited to IVF outcome predictions and to related ML training. It comprises the following elements, connected to each other by a bus 95 of addresses and data that also transports a clock signal:- a microprocessor 91 (or CPU);- a graphics card 92 comprising several Graphical Processing Units (or GPUs) 920 and a Graphical Random Access Memory (GRAM) 921;- a non-volatile memory of ROM type 96;- a RAM 97;- one or several I / O (Input / Output) devices 94 such as for example a keyboard, a mouse, a trackball, a webcam; other modes for introduction of commands such as for example vocal recognition are also possible;- a power source 98; and- a radiofrequency unit 99.

[0154] According to a variant, the power supply 98 is external to the apparatus 9.

[0155] The apparatus 9 also comprises a display device 93 of display screen type directly connected to the graphics card 92 to display synthesized images calculated and composed in the graphics card. The use of a dedicated bus to connect the display device 93 to the graphics card 92 offers the advantage of having much greater data transmission bitrates and thus reducing the latency time for the displaying of images composed by the graphics card. According to a variant, a display device is external to apparatus 9 and is connected thereto by a cable or wirelessly for transmitting the display signals. The apparatus 9, for example through the graphics card 92, comprises an interface for transmission or connection adapted to transmit a display signal to an external display means such as for example an LCD or plasma screen or a video-projector. In this respect, the RF unit 99 can be used for wireless transmissions.

[0156] It is noted that the word “register” used hereinafter in the description of memories 97 and 921 can designate in each of the memories mentioned, a memory zone of low capacity (some binary data) as well as a memory zone of large capacity (enabling a whole program to be stored or all or part of the data representative of data calculated or to be displayed). Also, the registers represented for the RAM 97 and the GRAM 921 can be arranged and constituted in any manner, and each of them does not necessarily correspond to adjacent memory locations and can be distributed otherwise (which covers notably the situation in which one register includes several smaller registers).

[0157] When switched-on, the microprocessor 91 loads and executes the instructions of the program contained in the RAM 97.

[0158] As will be understood by a skilled person, the presence of the graphics card 92 is not mandatory, and can be replaced with entire CPU processing and / or simpler visualization implementations.

[0159] In variant modes, the apparatus 9 may include only the functionalities of the device 1, and not those of the device 2. In addition, the device 1 and / or the device 2 may be implemented differently than a standalone software, and an apparatus or set of apparatus comprising only parts of the apparatus 9 may be exploited through an API call or via a cloud interface.

Claims

CLAIMS1. A device for obtaining a pre-trained deep reinforcement learning (DRL) model, said pre-trained DRL model being configured to output an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said device comprising: at least one input configured to receive: o a training dataset comprising:■ synthetic data obtained by simulating at least one sequence of movements using at least one musculoskeletal model of a person, said at least one musculoskeletal model including a musculoskeletal model of a person having at least one neurological and / or musculoskeletal pathology,■ for each subject of a plurality of subjects having at least one neurological and / or musculoskeletal pathology:• multiple recordings obtained during multiple rehabilitation sessions undergone by said subject, said rehabilitation session being part of a rehabilitation process, wherein for each rehabilitation session, said multiple recordings include: o Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said subject, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording, o at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said subject and / or on at least one hand of said subject,o at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said subject,• a classification of said patient into a Functional Ambulation Category (FAC), o an action space definition comprising a set of control actions available to said DRL model for obtaining said optimized control signal, said set of control actions being configured to stimulate at least one muscle and / or nerve, o constraint parameters defining acceptable physiological ranges and muscle synergies for said optimized control signal, at least one processor configured to feed said training dataset to an untuned DRL model so as to obtain said pre-trained DRL model and use said action space definition and said constraint parameters to optimize said pre-trained DRL model, wherein said optimization is obtained using a reward function designed to optimize key performance indicators (KPIs) for gait performance, patient safety and rehabilitation efficiency, at least one output configured to provide said pre-trained DRL model.

2. The device according to claim 1, wherein said multiple recordings further comprise at least one of: at least one Heart rate (HR) recording obtained using at least one HR sensor positioned on said subject, at least one temperature recording obtained using at least one temperature sensor positioned on said subject, at least one electroencephalogram (EEG) recording obtained using at least one EEG sensor positioned on said subject, at least one camera recording obtained using at least one camera positioned on said subject, said camera being configured to capture an environment of said subject, and at least one oxygen consumption recording obtained using at least one sensor configured to analyze an inhaled and exhaled air of said subject.

3. The device according to claim 1 or 2, said optimized control signal comprises optimized functional electrical stimulation parameters, said functional electrical stimulation parameters including at least one of a pulse amplitude, a pulse width, a pulse frequency, a pulse ramp, a pulse pattern and a spatial distribution.

4. The device according to any of claims 1 to 3, wherein said (KPIs) for gait performance include: an improvement in gait speed compared to a baseline, a change in joint angles during gait cycle, and a clinically meaningful difference in at least one gait parameter, said gait parameters including a step length and a cadence.

5. The device according to any of claims 1 to 4, wherein said (KPIs) for patient safety include: a number of falls or near-falls during one rehabilitation session, and a patient-reported discomfort or pain level.

6. The device according to any of claims 1 to 5, wherein said (KPIs) for rehabilitation efficiency include: a muscle fatigue level measured using at least one EMG sensor, a neuroplasticity induction, an energy consumption during one rehabilitation session, and a responsiveness and adaptation time of the DRL model to a gait deviation.

7. The device according to any of claims 1 to 6, wherein said DRL model is a Proximal Policy Optimization (PPO) model or a Deep Q-Network (DQN) model or a combination thereof.

8. A device for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal being obtained using a pre-trained deepreinforcement learning (DRL) model obtained from the device according to any of claims 1 to 7, wherein said device comprises: at least one input configured to receive: o multiple recordings obtained during a rehabilitation session undergone by said patient, said rehabilitation session being part of a rehabilitation process, said multiple recordings including:■ Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said patient, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording,■ at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said patient and / or on at least one hand of said patient,■ at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said patient, at least one processor configured to provide as input to said pre-trained DRL model, said multiple recordings so as to obtain said optimized control signal, at least one output configured to provide said optimized control signal.

9. The device according to claim 8, wherein said multiple recordings further comprises at least one of: at least one Heart rate (HR) recording obtained using at least one HR sensor positioned on said patient, at least one temperature recording obtained using at least one temperature sensor positioned on said patient, at least one electroencephalogram (EEG) recording obtained using at least one EEG sensor positioned on said patient, at least one camera recording obtained using at least one camera positioned on said patient, said camera being configured to capture an environment of said patient, and at least one oxygen consumption recording obtained using at least one sensor configured to analyze an inhaled and exhaled air of said patient.

10. The device according to claim 8 or 9, wherein said at least one input is further configured to receive: an action space definition comprising a set of control actions available to said DRL model for obtaining said optimized control signal, said set of control actions being configured to stimulate at least one muscle and / or nerve of said patient, constraint parameters defining acceptable physiological ranges and muscle synergies for said optimized control signal, said at least one processor being further configured to re-train said pre-trained DRL model using said multiple recordings obtained during said rehabilitation session so as to obtain a personalized DRL model configured to output a personalized control signal and use said action space definition and said constraint parameters to optimize said personalized DRL model.

11. The device according to claim 10, wherein said re-training is performed at each rehabilitation session undergone by said patient so as to adapt to changes and improvements of said patient during said rehabilitation process.

12. A computer implemented method for obtaining a pre-trained deep reinforcement learning (DRL) model, said DRL model being configured to output an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said method comprising: receiving: o a training dataset comprising:■ synthetic data obtained by simulating at least one sequence of movements using at least one musculoskeletal model of a person, said at least one musculoskeletal model including a musculoskeletal model of a person having at least one neurological and / or musculoskeletal pathology,■ for each subject of a plurality of subjects having at least one neurological and / or musculoskeletal pathology:• multiple recordings obtained during multiple rehabilitation sessions undergone by said subject, said rehabilitation session being part of a rehabilitation process, wherein for each rehabilitation session, said multiple recordings include: o Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said subject, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording, o at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said subject and / or on at least one hand of said subject, o at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said subject,• a classification of said patient into a Functional Ambulation Category (FAC), o an action space definition comprising a set of control actions available to said DRL model for obtaining said optimized control signal, said set of control actions being configured to stimulate at least one muscle and / or nerve, o constraint parameters defining acceptable physiological ranges and muscle synergies for said optimized control signal, feeding said training dataset to an untuned DRL model so as to obtain said pretrained DRL model and use said action space definition and said constraint parameters to optimize said pre-trained DRL model, wherein said optimization is obtained using a reward function designed to optimize key performanceindicators (KPIs) for gait performance, patient safety and rehabilitation efficiency, outputting said pre-trained DRL model.

13. A computer implemented method for obtaining an optimized control signal for controlling a Functional Electrical Stimulation (FES) system comprising multiple electrodes worn by a patient, said optimized control signal being configured to stimulate an ensemble of muscles and / or nerves of said patient through electrical signals sent by said electrodes, said optimized control signal being obtained using a pre-trained deep reinforcement learning (DRL) model obtained from the method according to claim 12, wherein said method comprises: receiving: o multiple recordings obtained during a rehabilitation session undergone by said patient, said rehabilitation session being part of a rehabilitation process, said multiple recordings including:■ Inertial Measurement Unit (IMU) recordings obtained using multiple IMU positioned on said patient, said at least one recording including at least one joint angle recording and at least one joint angle velocity recording,■ at least one pressure sensor recording obtained using at least one pressure sensor positioned under at least one foot of said patient and / or on at least one hand of said patient,■ at least one Electromyography (EMG) recording obtained using at least one EMG sensor positioned on said patient, providing as input to said pre-trained DRL model, said multiple recordings so as to obtain said optimized control signal, outputting said optimized control signal.

14. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the methods of any of claims 12 or 13.

15. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the methods of any of claims 12 or 13.

16. A non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a computer-implemented method of any of claims 12 or 13.

Images