System for neurological rehabilitation

Abstract

A method and system for the rehabilitation of patients with physical disabilities through the provision of music therapy is provided. [Solution] The method is implemented on a computer system and includes receiving biomechanical data of a patient's repetitive movements to determine baseline conditions, determining a constant baseline beat tempo based on the conditions, providing the patient with music of beat signals at the baseline beat tempo, providing the patient with cues for repetitive movements timed to the beat signals, receiving and analyzing time-stamped biomechanical data of the repetitive movements performed by the patient in time with the baseline beat signals, identifying an onset time for one or more cycles of the repetitive movements, determining entrainment parameters based on the onset times and the beat signals, modifying the baseline beat tempo based on the entrainment parameters, and determining whether a target beat tempo has been reached.

Description

[Technical field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Patent Application No. 62 / 322,504, entitled "SYSTEMS AND METHODS FOR NEUROLOGICAL REHABILITATION," filed April 14, 2016, which is incorporated by reference in its entirety.

[0002] [Technical field] The present disclosure relates generally to systems and methods for the rehabilitation of users with physical disabilities by providing music therapy.

[0003] [background] A number of controlled studies in the last decade have highlighted the clinical role of music in neurological rehabilitation. For example, supervised music therapy is known to directly enable cognitive, motor and language improvements. The process of listening to music enhances many forms of brain activity, igniting extensive bidirectional networks of brain regions involved in attention, semantic processing, memory, cognition, motor function and emotional processing.

[0004] Clinical data supports music therapy to improve memory, attention, executive function, and mood. PET scan studies of the neural mechanisms behind music have shown that enjoyable music can stimulate an extensive network between cortical and subcortical regions, including the ventral striatum, nucleus accumbens, amygdala, insula, hippocampus, hypothalamus, anterior tegmental area, anterior cingulate, orbitofrontal cortex, and ventromedial prefrontal cortex. The anterior tegmental area produces dopamine and has direct connections to the amygdala, hippocampus, anterior cingulate, and frontal cortex. This mesocorticolimbic system, which can be activated by music, plays an important role in mediating arousal, emotion, reward, memory attention, and executive function.

[0005] Neuroscience research has shown how the fundamental organizational processes for memory formation in music share mechanisms with non-musical memory processes. The fundamentals of phase grouping, hierarchical abstraction, and musical patterns have direct parallels in the temporal division principles for non-musical memory processes. This suggests that memory processes activated by music may translate and enhance non-musical processes.

[0006] Thus, there remains a need for improved devices, systems and methods for protecting the use of a user's identity and the security of providing personal information. [Summary]

[0007] In one aspect of the subject matter of the present disclosure, a method is provided for the rehabilitation of a patient with a physical disability by providing music therapy. The method is implemented on a computer system having one or more physical processors configured by machine-readable instructions that perform the method when the machine-readable instructions are executed. The machine-readable instructions include: receiving biomechanical data from a patient relating to the patient's repetitive motion to determine a baseline condition; determining a baseline beat tempo having a certain frequency based on the baseline condition; providing the patient with music having a beat signal at a baseline beat tempo; providing the patient with cues to perform each repetitive movement timed to an associated beat signal; receiving time-stamped biomechanical data of a patient relating to repetitive movements performed by the patient timed to a baseline beat signal; analyzing the biomechanical data to identify an onset time for one or more cycles of repetitive motion; determining an entrainment parameter based on the onset time and the associated beat signal; Varying a baseline beat tempo based on the entrainment parameters; determining whether a target beat tempo has been reached; Includes.

[0008] In some embodiments, the method includes measuring biomechanical data from the patient, the biomechanical data including motion, acceleration, and pressure associated with the patient's motion, in some embodiments, measuring the biomechanical data from the patient includes capturing video associated with the patient's motion.

[0009] In some embodiments, measuring biomechanical data from the patient includes transmitting data from a sensor associated with the patient to a computer system.

[0010] In some embodiments, measuring biomechanical data from the patient includes extracting data corresponding to stride length, cadence, speed, distance traveled, heel strike pressure or symmetry of the patient's gait.

[0011] In some embodiments, providing the patient with a cue for performing each repetitive motion includes providing the patient with a visual or audio cue. A handheld device may be provided to display the visual or audio cue.

[0012] In some embodiments, determining the entrainment parameter includes determining a delay between the beat signal and an onset time of each repetitive motion.

[0013] In some embodiments, if the delay between the beat signal and the onset time of the repetitive movement is substantially constant, the method includes increasing the baseline beat tempo towards the target tempo.

[0014] In some embodiments, providing music to the patient includes selecting music having a beat signal at a baseline beat tempo.

[0015] In some embodiments, providing music to the patient includes modifying existing music to have a beat signal at a baseline beat tempo.

[0016] In some embodiments, the repetitive motion comprises the patient walking, and the method further comprises: Analyzing the biomechanical data to classify the symmetry of the patient's gait; providing a first modification to the music while one of the patient's feet is moving and a second modification to the music while the other of the patient's feet is moving if the patient's gait is deemed asymmetric; In some embodiments the first modification and the second modification of the music are different from each other and include at least one of a chord change, a volume change, and a tempo change.

[0017] In some embodiments, the method further comprises: Analyzing biomechanical data to classify gait initiation defects; Providing haptic feedback to the patient to assist with initiation if the patient is having difficulty initiating Includes.

[0018] In some embodiments, the method further includes providing a user interface that enables a user to provide time-stamped notes regarding aspects of the patient's physical motion, which is analyzed based on the time-stamped notes and the biomechanical data.

[0019] In accordance with another aspect of the presently disclosed subject matter, there is provided a system for the rehabilitation of a patient having a physical impairment by providing music therapy, the system comprising: a handheld device having a display for interacting with a patient; a music providing device connected to the handheld device for providing the patient with music having a beat signal at a baseline beat tempo; one or more sensors associated with the patient, the sensors measuring biomechanical parameters related to repetitive physical movements of the patient; a transmitter for providing time-stamped biomechanical data and time-stamped beat signals from the patient to a handheld device; a computer system in at least intermittent communication with the handheld device, the computer system having one or more physical processors configured with machine-readable instructions; Including, The machine readable instructions are receiving biomechanical data of the patient relating to repetitive movements of the patient to determine a baseline condition; determining a baseline beat tempo having a certain frequency based on the baseline condition; providing the patient with music having a beat signal at a baseline beat tempo; providing instructions to the handheld device for providing cues to the patient to perform repetitive movements timed to the associated baseline beat signal; receiving time-stamped biomechanical data and time-stamped beat signals of a patient; Analyzing the time-stamped biomechanical data to identify an onset time of one or more cycles of repetitive motion; determining entrainment parameters based on the onset times and the associated beat signals; providing instructions to the handheld device for modifying the baseline beat tempo based on the entrainment parameters; Determine whether the target beat tempo has been reached This is for the purpose.

[0020] In some embodiments, the one or more sensors include sensors for measuring motion, acceleration and pressure associated with patient movement.

[0021] In some embodiments, the one or more sensors include an image capture device for capturing video associated with the patient's movements.

[0022] In some embodiments, a processor associated with the handheld or computer is configured to extract data corresponding to one or more of stride length, cadence, speed, distance traveled, heel strike pressure and symmetry of the patient's gait from one or more of the patient's motion, acceleration, pressure data and videos.

[0023] In some embodiments, the processor determines a delay between the beat signal and an onset time of the repetitive physical movement.

[0024] In some embodiments, The repetitive physical movement includes the patient walking; The computer system is configured with machine-readable instructions, The machine readable instructions are Biomechanical data was analyzed to classify the symmetry of the patient's gait; If the patient's gait is deemed asymmetrical, provide instructions to the handheld device to provide a first modification to the music while one of the patient's feet is moving and a second modification to the music while the other of the patient's feet is moving. This is for the purpose.

[0025] In some embodiments, The sensor includes a tactile feedback mechanism; The computer system is configured with machine-readable instructions, The machine readable instructions are Analyze biomechanical data to classify gait initiation defects; Providing instructions to the handheld device to provide tactile feedback from the sensor to the patient to assist initiation if the patient is having difficulty initiating. This is for the purpose.

[0026] These and other objects, features and advantages of the devices, systems and methods described herein will become apparent from the following description of specific embodiments, as illustrated in the accompanying drawings, in which the drawings are not necessarily to scale, emphasis instead being placed on illustrating the principles of the devices, systems and methods described herein. [Brief description of the drawings]

[0027] [Figure 1] FIG. 1 illustrates a system for treating a user by providing music therapy, according to an exemplary embodiment of the disclosed subject matter. [Diagram 2] FIG. 1 illustrates some of the components of a system for rehabilitation of a user by providing music therapy, according to an exemplary embodiment of the disclosed subject matter. [Diagram 3] 1 is a schematic diagram of a sensor for measuring biomechanical motion of a patient, in accordance with an exemplary embodiment of the disclosed subject matter; [Figure 4] FIG. 1 illustrates some components of a system according to an exemplary embodiment of the disclosed subject matter. [Diagram 5] FIG. 1 is a diagram of an example representation of components of a system for rehabilitation of a user by providing music therapy, according to an exemplary embodiment of the disclosed subject matter. [Figure 6] FIG. 13 is a flow diagram for one implementation of an analysis process according to an exemplary embodiment of the disclosed subject matter. [Figure 7] FIG. 1 is a flow diagram for one implementation of a process according to an exemplary embodiment of the disclosed subject matter. [Figure 8] FIG. 1 is a flow diagram for one implementation of a process according to an exemplary embodiment of the disclosed subject matter. [Figure 9] FIG. 1 is a flow diagram for one implementation of a process according to an exemplary embodiment of the disclosed subject matter. [Figure 10] FIG. 1 is a flow diagram for one implementation of a process according to an exemplary embodiment of the disclosed subject matter. [Figure 11]1 is a time plot illustrating music and a patient's physical movement, according to an exemplary embodiment of the disclosed subject matter; [Figure 12] FIG. 1 illustrates a patient response according to an exemplary embodiment of the disclosed subject matter. [Figure 13] FIG. 1 illustrates a patient response according to an exemplary embodiment of the disclosed subject matter. [Figure 14] FIG. 1 illustrates a patient response according to an exemplary embodiment of the disclosed subject matter. [Figure 15] FIG. 1 illustrates a patient response according to an exemplary embodiment of the disclosed subject matter. [Figure 16] FIG. 1 illustrates a patient response according to an exemplary embodiment of the disclosed subject matter. [Figure 17] FIG. 1 illustrates a patient response according to an exemplary embodiment of the disclosed subject matter. [Figure 18] FIG. 1 illustrates an implementation of techniques for gait training of a patient, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 19] FIG. 1 illustrates an implementation of techniques for patient neglect training, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 20] FIG. 1 illustrates an implementation of a technique for patient intonation training, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 21] FIG. 1 illustrates an implementation of techniques for musical stimulation training of a patient, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 22] FIG. 1 illustrates an implementation of techniques for gross motor training of a patient, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 23] 14A-14C illustrate implementations of techniques for patient grip strength training in accordance with exemplary embodiments of the disclosed subject matter. [Figure 24] FIG. 1 illustrates an implementation of a technique for patient speech cue training, in accordance with an exemplary embodiment of the disclosed subject matter. [Diagram 25] 1 illustrates an implementation of techniques for training a minimally conscious patient, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 26] FIG. 1 illustrates an implementation of a technique for patient attention training, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 27] FIG. 1 illustrates an implementation of a technique for patient attention training, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 28] FIG. 1 illustrates an implementation of a technique for patient attention training, in accordance with an exemplary embodiment of the disclosed subject matter. [Figure 29] FIG. 1 illustrates an implementation of techniques for patient agility training, in accordance with an exemplary embodiment of the disclosed subject matter. [Diagram 30] FIG. 1 illustrates an implementation of a technique for patient mouth motor training, in accordance with an exemplary embodiment of the disclosed subject matter. [Diagram 31] FIG. 1 illustrates an implementation of a technique for patient breathing training, in accordance with an exemplary embodiment of the disclosed subject matter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] [Detailed Description] The present invention relates generally to systems, methods and apparatus for implementing a dynamic closed-loop rehabilitation platform system that monitors and directs human behavioral and functional changes in language, movement and cognition that are temporally triggered by musical rhythm, harmony, melody and force cues.

[0029] In various embodiments of the present invention, a dynamic closed-loop rehabilitation platform music therapy system 100 is provided (illustrated in FIG. 1 ), which includes a sensor component and system 102, an edge processing component 104, a collection component 106, an analysis system 108, and a music therapy center 110. As described in more detail below, the sensor component, edge processing component, collection component machine learning process, and music therapy center may be provided on various hardware components. For example, in one embodiment, the sensor component and edge processing component may be placed or worn by the patient. In such an embodiment, the collection component and music therapy center may be provided on a handheld device. In such an embodiment, the analysis system may be located on a remote server. <Sensor system>

[0030] Throughout this description, the term "patient" is used to refer to an individual receiving music therapy treatment. The term "therapist" is used to refer to an individual providing music therapy treatment. In some embodiments, a patient can interact with the systems described herein without the presence of a therapist administering the treatment.

[0031] The sensor component 102 provides sensed biomechanical data about the patient. In some embodiments, the sensor component can include (1) a wearable wireless real-time motion sensing device or IMU (Inertial Measurement Unit), (2) a wearable wireless real-time combined multi-zone foot planter pressure / 6-dimensional motion capture (IMU) device (such as sensor 200), (3) a wearable wireless real-time electromyogram (EMG) device (such as sensor 208), and (4) a real-time wireless near-infrared (NIR) video capture device (such as imager 206) (see FIG. 4).

[0032] As illustrated in FIG. 2, the systems and methods described herein may be used in connection with treating a patient's gait disorder. Thus, an exemplary sensor 200 may be a combined multi-zone foot planter pressure / 6-DOF motion capture device. The sensor 200 records the pressure and 6-DOF motion profile of the patient's feet while the patient walks during a music therapy session. In some embodiments, the foot pressure / 6-DOF motion capture device has a variable recording time interval with a sampling rate of 100 Hz for foot pressure profiles with 1-4 zones, resulting in 100-400 pressure data points per foot-second.

[0033] The sensor 200 may include foot pressure pads 202 ranging from a heel pad (to measure one zone of pressure (such as heel strike pressure)) to a full insole pad (to measure four zones of pressure). Pressure measurements are made by sensing resistance changes in the transducer material as a result of compression due to the patient's weight transferred to the foot. These foot pressure maps are obtained at each sampling interval or at specific points during the music therapy session.

[0034] The sensor 200 is a 6D motion capture device 204 (3D (A x ,A y ,A z The method may include detecting changes in movement via a six degree of freedom micro-electromechanical system (MEMS) based sensor that determines linear acceleration and rotational motion (as pitch, yaw, and roll) in the foot. Sampling at 100 Hz produces 600 motion data points per second. These foot motion captures are taken at each sampling interval or at specific points during the music therapy session.

[0035] Multi-zone pressure sensing with a six degree of freedom motion capture device allows mappable spatial and temporal tracking of gait dynamics during walking. A schematic of the sensor 200 is illustrated in FIG.

[0036] From a system perspective, as illustrated in FIG. 4, patient P uses two foot sensors 200 for each foot (shown as right 200R and left 200L). In an exemplary embodiment, right foot sensor 200R transmits time-stamped internal measurement unit data and heel impact pressure data wirelessly on a first channel (e.g., channel 5 in the IEEE 802.15.4 direct sequence spread spectrum (DSSS) RF band). Left foot sensor 200L transmits time-stamped internal measurement unit data and heel impact pressure data wirelessly on a second channel (e.g., channel 6 in the IEEE 802.15.4 direct sequence spread spectrum (DSSS) RF band). To capture the right / left foot sensor RF data, a tablet or laptop 220 (which may be used by therapist T as described below) includes a wireless USB hub that includes two IEEE 802.15.4 DSSS RF transceivers tuned to the first and second channels (e.g., channels 5 and 6). A handheld wireless trigger 250 is used to start and stop the video and / or to annotate and index the time stream (described in more detail below).

[0037] Video analysis domain may be used to extract semantic and event information about the patient's therapy session. Patient actions and interactions are components of therapy that affect the therapy context and management. In some embodiments, one or more image capture devices 206 (such as video cameras) (see FIG. 4) are used with time-synchronized video feeds. Any suitable video can be incorporated into the system to capture the patient's movements. However, near-infrared (NIR) video capture is useful to protect patient privacy and reduce video data to be processed. The NIR video capture device captures NIR video images of the patient's body (such as the patient's torso and limb positions). In addition, the NIR video capture device captures the patient's real-time dynamic gait characteristics as a function of the music therapy session. In some embodiments, video is captured using a stationary camera and the background is subtracted to segment out foreground pixels.

[0038] 4, one or more video cameras 206 are triggered by a tablet or laptop application when a therapy session begins. The video cameras 206 can be stopped or started by the therapist by a handheld wireless trigger unit 250. This allows for the generation of labeled and time-stamped indexes of the captured biomechanical sensor data and video data streams.

[0039] In some embodiments, a wearable wireless real-time electromyography (EMG) device 208 may be worn by the patient. EMG sensors provide a full two-legged profile of the firing of the major muscles for a movement. Such sensors provide data on the exact time that the muscles fire. <Edge processing>

[0040] In some embodiments, edge processing is performed at the sensor 200, where sensor data is captured from the IMU and pressure sensors. This sensor data is filtered and grouped into various array sizes for further processing into frames reflecting extracted attributes and features, and these frames are transmitted (e.g., wirelessly) to a tablet or laptop collector 106. Alternatively, it will be appreciated that the raw biomechanical sensor data obtained from the sensor 200 may be forwarded to a remote processor for collection to perform edge processing functions.

[0041] The wearable sensors 200, 208 and video capture device 206 generate sensor data streams that are processed systemically to facilitate biomechanical feature extraction and classification. Sensor fusion (combining the output from multiple sensors capturing the same event) captures better results than any single component sensor input.

[0042] Capturing patient activities in the context of music therapy formalizes interactions applied in music therapy and develops patient-specific and generalized formal indicators of music therapy performance and effectiveness. By extracting and then analyzing video features, semantic, high-level information about patient behavior can be captured.

[0043] When processing the video, a learned background subtraction technique is used to create a background model that incorporates any changes in lighting conditions and occlusions in the physical area where the music therapy occurs. The result of background subtraction is a binary foreground map with an array of foreground chunks (which are 2D contours). The video is then sliced ​​into separate image frames for future image processing and sensor fusion. Video information is provided along with additional metadata by merging edge processed sensor data from the IMU, foot pressure pad(s) and EMG sensor. Sensor data can also be synchronized with other data using an RF trigger. Data can be sent directly to the collector, stored in internal board memory, or analyzed on the edge running the OpenCV library.

[0044] The edge processor 104 may be a microprocessor (such as a 32-bit microprocessor integrated into a foot pressure / 6 DOF motion capture device that enables high speed multi-zone scanning at a rate of 100-400 complete foot pressure / 6 DOF motion profiles per second).

[0045] The foot pressure / 6DOF motion capture device collects foot pressure / 6DOF motion profile data for real-time gait analysis, resulting in feature extraction and classification. In some embodiments, the foot pressure / 6DOF motion capture device initializes the microcontroller unit (MCU), continuous operator processing (COP), general purpose input / output (GPIO), serial peripheral interface (SPI), interrupt request (IRQ), and sets the desired RF transceiver clock frequency by calling routines including Microcontroller Unit Initialization (MCUInit), General Purpose Input / Output Initialization (GPIOInit), Serial Peripheral Interface Initialization (SPIInit), Interrupt Request Acknowledge Initialization (IRQInit), Interrupt Request Acknowledge (IRQACK), Serial Peripheral Interface Driver Read (SPIDrvRead), and IRQPinEnable. MCUInit is a master initialization routine that turns off the MCU watchdog and causes the timer module to use the bus clock (BUSCLK) as a reference with a pre-scaling of 32.

[0046] The state variable gu8RTxMode is set to SYSTEM_RESET_MODE and the routines GPIOLinit, SPIInit and IRQInit are called. The state variable gu8RTxMode is set to RF_TRANSCEIVER_RESET_MODE and IRQFLAG is checked to see if an IRQ is asserted. First the RF transceiver interrupt is cleared using SPIDrvRead and then the RF transceiver is checked for an ATTN IRQ interrupt. Finally, for MCUInit a call is made to PLMEPhyReset to reset the physical MAC layer, IRQACK (to ACK pending IRQ interrupts) and IRQPinEnable (for pin, Enable, IE and IRQ CLR, on the negative edge of the signal).

[0047] The foot pressure / 6 DOF motion sensor 200 waits (e.g., 250 ms) for a response from the foot pressure / 6 DOF motion collection node to determine whether a default full foot pressure scan will be performed or a mapped foot pressure scan will be initiated. In the case of a mapped foot pressure scan, the foot pressure / 6 DOF motion collection node sends foot pressure scan mapping configuration data to the appropriate electrodes.

[0048] One aspect of the analysis pipeline is a feature set processing process that defines the captured sensor values ​​and their resulting sensor fusion values ​​that are used to create feature vectors that define the input data structure for analysis. Representative values ​​are Ax(i), Ay(i), Az(i) and Ph(i), where i is the i-th sample, Ax(i) is the acceleration in the x-direction that is lateral to the foot sensor, Ay(i) is the acceleration in the y-direction that is forward to the foot sensor, Az(i) is the acceleration in the z-direction that is upward to the foot sensor, and Ph(i) is the heel impact pressure. The sensor values ​​are presented in Table 1.

[0049] [Table 1]

[0050] In some embodiments, the sensor fusion technique uses the heel impact pressure value Ph(i) to "gate" the analysis of the following exemplary feature values ​​to derive a window of data, as described below. For example, as presented in Table 2 below, "onset" (start) can be determined based on heel pressure exceeding a threshold indicative of heel impact, and "stop" can be determined based on heel pressure dropping below a threshold indicative of heel off. It is understood that heel impact pressure is one example of a parameter that can be used for "gating" analysis. In some embodiments, "gating" is determined by using IMU sensor data, video data, and / or EMG data.

[0051] [Table 2]

[0052] High-level feature values ​​are computed from the fused sensor values ​​(such as the example values ​​presented in Table 3).

[0053] [Table 3]

[0054] The system described herein provides the ability to "gate" or provide a "window" on a patient's biomechanical data. Gating biomechanical data is useful for repetitive patient motion (such as repetitive stepping while the patient is walking). Sensor data from one or more sources (pressure sensors, IMU sensors, video data, EMG data, etc.) are used to identify cycles of motion that repeat over time. For example, as a patient walks, foot pressure repeatedly increases and decreases as the patient's foot contacts and then leaves the ground. Similarly, foot velocity increases as the foot moves forward and decreases to zero while the foot is attached to the ground. As a further example, the Y position or height of the patient's foot cycles between a low position (touching the ground) and a high position (approximately the midpoint of the step). The "gating" technique identifies repeating cycles or "windows" in such data. In the case of a patient's walking, the cycle repeats with each step. Although there is variability between cycles (e.g., between steps), certain patterns are repeated with each cycle. Selecting the onset time (start time) of each cycle involves locating an identifiable point (maximum or minimum) of a biomechanical parameter. The parameter for the onset time is selected based on available data. Thus, in some embodiments, the moment when the heel impact pressure exceeds a threshold may be used to define the limit of the onset time of each cycle. (See, for example, FIG. 5, where pressures 316a and 316b include cyclical characteristics). "Onset" may be determined as the moment when the pressure exceeds a threshold.) Similarly, the limit of the onset time may be defined by when foot velocity decreases to zero.

[0055] In some embodiments, raw frame data is pre-processed to obtain and "gate" instantaneous data (e.g., identify a window), and then analyze the data within the window to identify outliers and analyze the data (e.g., exponential analysis, averaging data within multiple windows). Fusion of sensor data by including both IMU data and heel impact pressure data allows for more accurate identification of onset time for a step (or other unit of motion repetition) than using data from a single sensor. Sensor data captured within a step is considered a "window," and information extracted from this analysis includes, for example, stride length, step count, cadence, time the step occurred, distance traveled, stance phase / sway phase, both support times, velocity, symmetry analysis (e.g., between left and right feet), outward sway, shuffling, force vector, lateral acceleration, step width, variability in these dimensions, additional parameters derived from the information above, etc. Feature extraction can be processed on a microprocessor chip (e.g., a 32-bit chip). Wireless synchronous gated biomechanical sensor data capture and video data capture capabilities allow for the creation of time series templates.

[0056] Data can be indexed by the patient or therapist during a music therapy session. The "gating" feature described above is useful for linking exception conditions to specific steps or gaits. For example, a therapist may observe a specific exception condition or behavior (such as an anomaly or incident) in the patient's movement. The indexing feature allows the therapist to initiate the exception condition or behavior (such as capturing a "recording") via voice control or a user interface on a handheld tablet or laptop (such as the wireless trigger unit 250 illustrated in FIG. 4). Annotations may be created, including timestamps and comments (such as the occurrence of a patient's "stumbling" while walking). Such indexing facilitates the creation of time series templates. These time series templates are studied for review of therapy session events and development of time series templates to train machine learning algorithms (such as nonlinear multilayer perceptrons (NLMLPs), convolutional neural networks (CNNs), recurrent neural networks (RNNs) with long short-term memory (LSTMs), etc.).

[0057] In one embodiment, a communication protocol is provided to transfer sensor data from the edge processing 104 (e.g., sensor 200) to the collector 106. See Table 4 below. In some embodiments, if communication is idle for more than 100 ms, the RF times out.

[0058] [Table 4]

[0059] In one embodiment, a foot pressure sensor zone scan is performed by the FootScan routine while the FootDataBufferIndex is initialized and the foot pressure sensor zones are activated (by enabling the MCU instruction mode for output [PTCDD_PTCDDN=Output] and pulling the associated port line low [PTCD_PTCD6=0]). Once the foot pressure sensor zones are activated based on the foot pressure sensor zone scan map, each foot pressure sensor zone attached to each MCU analog signal port is sampled and then the current voltage readings are converted into digital form (time zone foot pressure).

[0060] Some variables (such as FootDataBufferIndex and IMUBufferIndex) are used to prepare the IEEE802.15.4 RF packet gsTxPacket.gau8TxDataBuffer[ ] for sending the data used in FootDataBuffer[ ] and IMUBuffer[ ]. The RF packet is sent using the RFSendRequest(&gsTxPacket) routine. This routine checks if gu8RTxMode is set to IDLE_MODE and uses gsTxPacket as a pointer to call the RAMDrvWriteTx routine, which then calls SPIDrvRead to read the RF transceiver's TX packet length register contents. These contents are used to update the mask length setting, then add 2 to the CRC and add 2 to the code byte.

[0061] SPISendChar is called to send a 0x7E byte, which is the second code byte. Then SPIWaitTransferDone is called again to verify that the transmission has occurred. Once these code bytes have been sent, the remainder of the packet is sent using a for loop, where psTxPkt->u8DataLength+1 is the number of iterations for the sequential sequence for SPISendChar, SPIWaitTransferDone, SPIClearRecieveDataReg. Once complete, the packet to be sent is loaded into the RF transceiver. The ANTENNA_SWITCH is set to transmit, LNA_ON mode is enabled, and finally, a RTXENAssert call is made to actually send the packet. <Collector>

[0062] The primary function of the collector 106 is to capture data from the edge processing 104, transfer the data to the analysis system 108 and receive processed data from the analysis system 108, and transfer the data to the music therapy center 110 (described below). In some embodiments, the collector 106 provides control functions (e.g., user interface for logging in, configuring the system, interacting with the user) and includes a display unit for visualizing / displaying the data. The collector 106 may include lightweight analysis or machine learning algorithms for classification (e.g., lateral tremor, asymmetry, instability, etc.).

[0063] The collector 106 receives body, movement, and localization data from the edge processor 104. The data received at the collector 106 may be raw or may have been processed at the edge 104 before being forwarded to the collector. For example, the collector 106 receives fused sensor data that is subjected to "windowing" processing and feature extraction. The forwarded data may include two levels of data: (1) RF packets sent from the right / left foot sensors as described in Table 1, and (2) RF packets from the right / left foot sensors that include higher level attributes and features as described in Tables 2 and 3. The collector 106 stores the data locally. In some embodiments, the collector 106 classifies the movement from the received data (e.g., comparing it to a locally stored model (pre-downloaded from the analysis system or sent to the analysis system for classification)). The collector may include a display unit to visualize / display the data.

[0064] In some embodiments, the collector 106 runs on a local computer including a memory, a processor, and a display. Examples of devices on which the collector can be installed include AR devices, VR devices, tablets, mobile devices, laptop computers, desktop computers, etc. FIG. 2 illustrates a handheld device 220 having a display 222 and performing the collector function. In some embodiments, the connection parameters for transferring data between the patient sensor and the collector include the use of Windows Device Manager (e.g., baud rate: 38400, data bits: 8, parity: none, stop bits: 1). In some embodiments, the collector 106 includes a processor that is held or worn by the music therapy patient. In some embodiments, the collector 106 includes a processor that is remote from the music therapy patient (carried by the therapist and connected to the music therapy patient wirelessly or via a wired connection).

[0065] In some embodiments, the foot pressure / 6 DOF motion collection node captures RF transmitted data packets (containing real-time foot pressure / 6 DOF motion profile data from the foot pressure / 6 DOF motion capture device) initiated by the foot pressure / 6 DOF motion collection node creating an RF packet receive queue driven by the callback function of the RF transceiver packet receive interrupt.

[0066] When an RF packet is received from a foot pressure / 6 DOF Mocap device 200, a check is first made to determine if it is from a new foot pressure / 6 DOF Mocap device or an existing one. If it is from an existing foot pressure / 6 DOF Mocap device, the RF packet sequence number is checked to determine continuous synchronization before further parsing the packet. If this is a foot pressure / 6 DOF Mocap device, a foot pressure / 6 DOF Mocap device context state block is created and initialized. The context state block contains information such as foot pressure profile, [additional information?], etc.

[0067] Above this RF packet session level process for node-to-node communication is the parsing of the RF packet data payload, which contains the foot pressure profile based on the current variable pressure according to the 6 degree of freedom motion. This is structured as follows: |0x10|Start|F1|F2|F3|F4|Ax|Ay|Az|Pi|Yi|Ri|XOR checksum|.

[0068] The IEEE 802.15.4 standard specifies a maximum packet size of 127 bytes, reserving 47 bytes for Time Synchronous Mesh Protocol (TSMP) operations, leaving 80 bytes for payload. IEEE 802.15.4 is compatible with 2.4 GHz Industrial, Scientific, and Medical (ISM) band radio frequency (RF) transceivers.

[0069] The RF module contains a complete 802.15.4 physical layer (PHY) modem designed for the IEEE 802.15.4 wireless standard (supporting peer-to-peer, star, and mesh networks). This is combined with the MCU to create the required wireless RF data links and networks. The IEEE 802.15.4 transceiver supports 250kbps O-QPSK data on a 5.0MHz channel and full spread spectrum encoding and decoding.

[0070] In some embodiments, control, status reading, data writing and data reading are done through the sensing system node device's RF transceiver interface port. The sensing system node device's MPU accesses the sensing system node device's RF transceiver via interface "transactions" (multiple byte-long bursts of data sent on the interface bus). Each transaction is three or more bursts long depending on the type of transaction. A transaction is always a read or write access to a register address. The data associated with any single register access is always 16 bits in length.

[0071] In some embodiments, control and data transfer for the RF transceiver of the foot pressure / 6 DOF motion collection node is achieved by a serial peripheral interface (SPI). While the normal SPI protocol is based on 8-bit transfers, the RF transceiver of the foot pressure / 6 DOF motion collection node imposes a higher level transaction protocol (based on multiple 8-bit transfers per transaction). A single SPI read or write transaction consists of an 8-bit header transfer followed by two 8-bit data transfers.

[0072] The header describes the type of access and the register address. Subsequent bytes are read or write data. SPI also supports recursive "data burst" transactions where additional data transfers may occur. The recursive mode is primarily for fast setup of the RF and packet RAM access of the foot pressure / 6 DOF motion collection node.

[0073] In some embodiments, all foot pressure sensor zones are scanned sequentially and the entire process is repeated until a reset condition or inactivity power down mode. The 6 DOF motion is captured by a serial UART interface from the MCU to the Inertial Measurement Unit (IMU). The sampling rate is 100-300 Hz for all sensing dimensions (Ax, Ay, Az, pitch, yaw, roll) and the sampled data is stored in the IMUBuffer[ ].

[0074] A call is made to SPIDryWrite to update the TX packet length field. Next, a call is made to SPIClearRecieveStatReg to clear the status register, followed by a call to SPIClearRecieveDataReg to clear the receive data register to put the SPI interface into a state ready for reading or writing. Once the SPI interface is ready, a call is made to SPISendChar to send a 0xFF character representing the first code byte, then SPIWaitTransferDone is called to verify that the transfer has occurred.

[0075] 5 is an example output 300 that may be provided on the display 222 of a handheld device. For example, when treatment is provided to a walking patient, the display output 300 may include a portion for the right foot 302 and a portion for the left foot 304. The display for the right foot may include the acceleration A x 310a, A y 321a and A z314a and foot pressure 316a. Similarly, the display for the left foot includes acceleration A x 310a, A y 312a and A z 314a as well as foot pressure 316a.

[0076] Classification is understood to be the correlation of data (e.g., sensor fusion data, feature data, or attribute data to real-world events (e.g., patient activity or disposition). Typically, classification is created and performed on the analysis system 108. In some embodiments, the collector 106 has a local copy of some "templates." In this way, incoming sensor data and feature extraction data can be classified at the collector or at the analysis system.

[0077] Context represents the environment or facts that form the setting for an event, statement, situation, or idea. Context-aware algorithms examine the who, what, when, and where of the environment and the time when the algorithm is performed on some data. Some context-aware actions include identity, location, time, and the action being performed. In using contextual information to form deterministic actions, a contextual interface occurs between the patient, the environment, and the music therapy session.

[0078] The patient's response context to the music therapy session may include one layer of algorithms that interpret the fused sensor data to infer higher level information. These algorithms distill the patient's response context. For example, the patient's biomechanical gait sequence is analyzed in relation to a particular part of the music therapy session. In one example, "lateral tremor" is a classifier of interest. Thus, it is determined that the patient's gait will become more fluid with less lateral tremor. <Analysis system>

[0079] The analysis system 108 (often referred to as the backend system) stores the large scale models / archives and includes machine learning / analysis processing using the models described herein. In some embodiments, a web interface is provided for login and a dashboard to view archived data. In some embodiments, the analysis system 108 is located on a remote server computer that receives data from the collector 106 running on a handheld unit (such as handheld device or tablet 220). It is contemplated that the processing power required to perform the analysis and machine learning functions of the analysis system 108 may be located on the handheld device 220.

[0080] Data is transferred from the collector 106 to the analysis system 108 for analysis processing. As illustrated in FIG. 6, the analysis process 400 includes a user interface 402 that receives data from the collector 106. A database storage 404 receives input data from the collector 106 for storage. Training data and output of the analysis process (e.g., ensemble machine learning system 410) may be stored in the storage 404 to facilitate the creation and refinement of predictive models and classifiers. A data bus 406 allows data to flow through the analysis process. A training process 408 is performed on the training data to derive one or more predictive models. The ensemble machine learning system 410 utilizes the predictive models. The output of the ensemble machine learning system 410 is an aggregation of these predictive models. This aggregate output is also used for classification requests using a template classifier 412 (tremor, symmetry, fluidity, or learned biomechanical parameters (entrainment, initiation, etc.)). An API 418 is connected to the collector and / or the music therapy center. The treatment algorithms 414 and the prediction algorithms 416 include multi-layer perceptron neural networks, hidden Markov models, radial basis function networks, Bayesian estimation models, and the like.

[0081] An exemplary application of the systems and methods described herein is the analysis of a patient's biomechanical gait. Gait sequences are feature extracted into a sequence of characteristic features. The presence of these and other features in the captured sensor fusion data informs a context detection algorithm whether the patient's biomechanical gait sequence is valid or not. Biomechanical gait sequence capture requires robust context detection, which is then abstracted across a representative population of music therapy patients.

[0082] One example of such an activity is the location of patients at a given time and their response to music therapy at that time. Recognizing and correlating patients' responses to music therapy allows for recognition-specific patterns of music therapy patient response. Each specific music therapy administration is then benchmarked and analyzed for performance and effectiveness by creating baselines of music therapy patient responses and correlating them to future music therapy patient responses.

[0083] Using distance measures with biomechanical capture of gait, combined with motion sensing, patient path trajectories are determined using temporal and spatial changes / displacements between two or more music therapy sessions. Features are extracted from this sensor fusion data capture and classified to label various key patient therapy responses. Further sensor fusion data analysis uses histograms to enable early music therapy response pattern detection.

[0084] To analyze the sensor-fused data of the music therapy sessions, a patient-specific Bayesian inference model using Markov chains is initially used. Each state of the chain represents a patient-specific response pattern captured from the music therapy baseline session. The inference is based on knowledge of the appearance of the patient's response pattern at each sample interval and a temporal link to the previous state.

[0085] The prediction routine (a multi-layer perceptron neural network (MLPNN)) uses a node-based model of a directed graph with a top-layer root node that predicts the requirements to reach subsequent nodes and obtains the patient's sensor fusion data feature vector, which includes time-series processed motion data, music signature data, and video image data of particular interest for further processing. In this case, the directed graph looks like an upside-down tree (with leaves at the bottom of the tree and the root at the root node). From each node, the routine can proceed left, where the left is the left node in the next layer below the top layer (where the root node is located), and select the left subnode as the next observed node, or it can proceed right, where the right is the right node in the next layer below the top layer (where the root node is located), based on the value of a particular variable whose index is stored in the observed node. If this value is less than a threshold, the routine proceeds to the left node, and if this value is greater than a threshold, the routine proceeds to the right node. These regions (here left and right) become the predictor space.

[0086] The model uses two types of input variables: ordered and categorical. Ordinal variables are values ​​that are compared to a threshold (also stored in the node). Categorical variables are discrete values ​​that test whether they belong to a certain limited subset of values, stored in the node. This is applicable to various classification processes. For example, "mild", "moderate" and "severe" can be used to describe tremors, which are examples of categorical variables. Conversely, finely-distinct ranges of values ​​or a numerical scale can be used to describe tremors in a similar way (but numerically).

[0087] If the categorical variable belongs to a limited set of values, the routine proceeds to the left node, otherwise the routine proceeds to the right node. To make this decision, at each node, a pair of entities (variable_index, decision_rule (threshold / subset)) is used. This pair is called a branch, and branches depending on the variable variable_index.

[0088] Once a node is reached, the value assigned to that node is used as the output of the prediction routine. A multi-layer perceptron neural network is built recursively starting from the root node. All training data, feature vectors, and responses are used to split the root node as described above, where entity(variable_index, decision_rule(threshold / subset)) partitions the prediction space. At each node, an optimal decision rule for the best principal split is found based on the Gini "purity" criterion for the sum of squared errors for classification and regression. The Gini coefficient is based on a measure of total variance across the aggregate classes. The Gini "purity" criterion refers to a small Gini coefficient value, indicating that the node contains mainly observations from a single class, which is the desired state.

[0089] After the multi-layer perceptron neural network is constructed, it may be pruned using a cross-validation routine. Some of the branches of the tree are pruned to avoid overfitting the model. This routine may also be applied to standalone decisions. One of the notable properties of this decision algorithm (MLPNN) (mentioned above) is its ability to calculate the relative decision power and importance of each variable.

[0090] The most frequent interaction types are determined for the patient's interaction feature vector using variable importance ratings. Pattern recognition begins with the definition of a decision space suitable for distinguishing between various categories of music therapy responses and music therapy events. The decision space can be represented by an N-dimensional graph, where N is the number of attributes or measurements considered to represent music therapy responses and music therapy events. The N attributes constitute a feature vector or signature that can be plotted on the graph. After sufficient samples are input, the decision space will show clusters of music therapy responses and music therapy events that belong to various categories, and new vectors can be associated with these clusters.

[0091] The dynamic closed-loop rehabilitation platform music therapy system utilizes several deep learning neural networks to learn and reproduce patterns. In one embodiment, a nonlinear decision space is constructed using an adaptive radial basis function (RBF) model generator. New vectors can be calculated using the RBF model and / or using a K-nearest neighbor classifier. Figure 6 illustrates the workflow of the machine learning subsystem of the dynamic closed-loop rehabilitation platform music therapy system.

[0092] FIG. 7 illustrates a supervised training process 408 that includes several training samples 502. For example, the inputs are features as described in Table 3 above, and example outputs are items such as tremor, asymmetry, force, the extent of these items, prediction of changes, classification of how well the patient is recovering, etc. It is understood that as part of this process, new outputs are learned. This provides a basis for higher level prediction and classification abstractions as this is applied to different use cases (e.g., various disease states, combinations with pharmaceuticals, provider notification, fitness, and fall prevention). These training samples 502 are run through learning algorithms A1 504a, A2 504b, A3 504c, ..., AN 504n to derive predictive models in M1 506a, M2 506b, M3 506c, ..., MN 506n. Exemplary algorithms include multi-layer perceptron neural networks, hidden Markov models, radial basis function networks, and Bayesian estimation models.

[0093] 8 illustrates an ensemble machine learning system 410 as an aggregation of predictive models M1 506a, M2 506b, M3 506c, ..., M N 506n on sample data 602 (e.g., feature extracted data) to provide multiple prediction result data 606a, 606b, 606b, ..., 606n. An aggregation layer 608 (including, e.g., decision rules and voting) is used to derive an output 610 given multiple predictive models.

[0094] The MR ConvNet system has two layers, the first layer is a convolutional layer with average pooling support. The second layer of the MR ConvNet system is a fully connected layer that supports polynomial logistic regression. Polynomial logistic regression (also called Softmax) is a generalization of logistic regression to handle multiple classes. In logistic regression, the labels are binary.

[0095] Softmax is a model used to predict the probability of various possible outputs. In what follows, we consider a multi-class classifier with m discrete classes, fed through a final Softmax output layer: Y1 = Softmax (W11*X1 + W12*X2 + W13*X3 + B1) [1] Y2 = Softmax (W21*X1 + W22*X2 + W23*X3 + B2) [2] Y3 = Softmax (W31*X1 + W32*X2 + W33*X3 + B3) [3] Ym = Softmax (Wm1*X1 + Wm2*X2 + Wm3*X3 + Bm) [4] In general: Y = softmax (W*X + B) [5] Softmax (X)i = the sum of exp (Xi) / exp (Xj), for j = 1 to N. [6] where Y = classifier output, X = sample input (all scaled (normalized) feature values), and W = weight matrix. Classification can be, for example, scoring asymmetry (such as "moderate asymmetry score, 6 out of 10 (10 for high level asymmetry to 0 for no asymmetry)") or scoring gait fluidity (such as "gait fluidity score 8 out of 10, normal"). The analysis pipeline is shown in Figure 9.

[0096] Softmax regression can handle a large number of classes, more than 2. For logistic regression, P(x) = 1 / (1 + exp(-Wx)), where W contains the model parameters trained to minimize the cost function, and x is the input feature vector. ((x(1), y(1)),…,(x(i), y(i))) [7] represents the training set. For multi-class classification, Softmax regression is used where y can take on N different values ​​representing each class, instead of 1 and 0 in the binary case. Thus, for a training set ((x(1), y(1)),...,(x(i), y(i))), y(n) can be any value in the range of 1 to N classes.

[0097] Then p(y = N|x;W) is the probability for each value of i = 1, ..., N. Below is a mathematical example of the Softmax regression process: Y(x) = (p(y = 1|x;W), p(y = 2|x;W), … p(y = N|x;W)) [8] where Y(x) is the answer to the hypothesis, which, given an input x, outputs the probability distribution of all classes whose normalized sum is 1.

[0098] The MR ConvNet system convolves each biomechanical template filter (as a vector) with the biomechanical data frame (as a vector) within each window to generate a response using an average pooling function that averages the feature responses. The convolution process adds an arbitrary bias to calculate Wx, which is then fed into a logistic regression (sigmoid) function.

[0099] Next, in the second layer of the MR ConvNet system, the subsampled biomechanical template filter responses are translated into a 2D matrix (where each column represents a biomechanical data frame within a window as a vector), at which point the Softmax recurrent activation process begins using: Y(x) = (1 / (exp(Wx) + exp(Wx) + …. + exp(Wx)) * (exp(Wx), exp(Wx), … , (exp(Wx)) [9]

[0100] The MR ConvNet system is trained using an optimization algorithm (gradient descent) in which a cost function J(W) is defined and minimized. J(W) = 1 / j*((H(t(j = 1), p(y = 1|x;W) + H(t(j = 2), p(y = 2|x;W) + … + H(t(j), p(y = N|x;W))

[10] where t(j) is the target class. This averages all the class entropies over the j training samples. The cross-entropy function is: H(t(j), p(y = N|x;W) = - t(j = 1)* log(p(y = 1|x;W)) + t(j = 2)* log(p(y = 2|x;W)) + … + t(j )* p(y = N|x;W)

[11]

[0101] In FIG. 10, the ensemble machine learning system 408 includes multiple predictive models (e.g., template sequence 1 (tremor) 706a, template sequence 2 (asymmetry) 706b, template sequence 3 (fluidity) 706c...additional templates (other learned biomechanical parameters, e.g., entrainment, initiation, etc.) 706n (applied to conditioned inputs 702) (which may be, for example: stride length of right and left features (x1, x2), stride length variance of right and left features (x3, x4), gait right and left features (x6, x7), gait right and left feature variance (x8, x9), etc.) (here the samples (x1, x2,..., xn) are referred to as vector X, which is input to 702 in the ensemble of ML algorithms). These are the conditioned reference normalized and / or scaled inputs. The aggregate classifier 708 outputs such information as tremor scale, symmetry scale, fluidity scale, etc. <Music Therapy Center>

[0102] The music therapy center 110 is a decision-making system running on a processor (e.g., a handheld device or laptop computer 220 of FIG. 2 ). The music therapy center 110 takes inputs from the characterized extracted sensor data in the collector 106 and compares them to a defined process to provide therapy, and then communicates the content of the auditory stimuli to be played via the music delivery system 230.

[0103] Embodiments of the present invention use contextual information to determine why a situation is occurring and then encode the observed behavior, thereby generating dynamic and coordinated changes to the system state (and therefore the music therapy session) in a closed-loop manner.

[0104] The interaction between the patient and the music therapy session provides real-time data for determining the context-awareness of the music therapy patient, including movement, posture, stepping, and gait responses. As input data is collected by the sensing nodes (at the sensors), the embedded nodes process the context-aware data (at the edge processing) and provide immediate dynamic actions and / or transmit the data to an analysis system 108 (e.g., a flexible network-based processing cloud environment for storage and further processing and analysis).

[0105] The program takes any existing song content and modifies cadence, major / minor chords, time signature and musical cues (e.g., melodic, harmonic, rhythmic and intensity cues) based on the input. The system may overlay a metronome onto an existing song. Song content may be beat-mapped (e.g., in the case of W in response to an AV or MP3 file) or in MIDI format so that entrainment potentials can be calculated with precise knowledge of when the beats occur. Sensors attached to the patient may be configured to provide tactile / vibration feedback pulses in the musical content. <Example>

[0106] An exemplary application of the method is described herein. Gait training analyzes the real-time relationship between the beat of music being played for the patient and each step the patient takes in response to each beat of the music. As described above, gating analysis is used to determine a repeated window of data (with some variability for each step or repetitive motion). In some embodiments, the start of the window is determined as the time when the heel impact pressure exceeds a threshold (or other sensor parameter). FIG. 11 is an exemplary time plot showing the beat of the music ("beat time") and the step taken by the patient ("step time"). Thus, the onset time in this case is associated with the "step time". In particular, the plot shows the beat time 1101 of the music as "beat time 1". After a period of time, the patient takes a step in response to the beat time 1001 at time "step time 1" (i.e., step time 1102). The entrainment potential 1103 represents the delay (if any) between beat time 1 and step time 1.

[0107] 12-13 show examples of entrainment of a patient's gait using the system described herein. FIG. 12 shows "perfect" entrainment (e.g., the entrainment potential is always zero). This occurs when there is no or negligible delay between the beat time and the associated step time that occurs in response to this beat time. FIG. 13 shows phase-shift entrainment (where the entrainment potential is not zero, but is constant or has minimal variation over time). This occurs when the delay between the beat time and the step time is constant over time and within an acceptable range.

[0108] With further reference to FIG. 11, the EP ratio is calculated as the ratio of the length of time between beat times to the length of time between step times:

number

number

[0109] 14 and 15 illustrate entrainment responses of patients over time using the techniques described herein. FIG. 14 (left Y-axis: EP ratio, right Y-axis: beats per minute, X-axis: time) shows a scatter of dots 1402, which represent the average EP ratio of the first patient's gait. The graph shows an upper limit 1404 of +0.1 and a lower limit 1406 of -0.1. A line segment 1408 shows tempo over time (starting at 60 beats per minute (bpm) and increasing in steps to 100 bpm). FIG. 14 shows that the EP ratio remains close to 1 (±0.1) as the tempo increases from 60 bpm to 100 bpm. FIG. 15 shows the gait of a second patient, again showing that the EP ratio remains close to 1 (±0.1) as the tempo increases from 60 bpm to 100 bpm.

[0110] 16 and 17 (Y-axis: entrainment potential, X-axis: time) illustrate two patients' responses to changes in beat time (e.g., changes in tempo) and / or chord changes, changes in tactile feedback, changes in foot cueing (e.g., left-right or left-right-cane cueing), etc. FIG. 16 shows a time-based plot where the patient's gait is equilibrated at "full entrainment" (constant, zero or negligible entrainment potential) or at a constant phase shift entrainment potential. As shown in this figure, a period of time (prime time 1602) is required before equilibrium occurs. FIG. 17 shows a time-based plot where the patient's gait is not equilibrated (e.g., does not reach full entrainment or constant phase shift entrainment potential after a change to the beat time). Prime time is useful because it represents a data set separate from measurements of entrainment accuracy. The prime time parameter can also be used to screen future songs for appropriateness. For example, if a patient exhibits a relatively long prime time when a particular piece of music is used, then that piece of music is not very effective in treatment.

[0111] FIG. 18 illustrates a technique useful for gait training where repetitive movements refer to steps taken by a patient while walking. Gait training is adapted to the patient population, diagnosis, and condition to provide an individualized and personalized music intervention. The program changes content, cadence, major / minor chords, time signature, and musical cues (e.g., melodic, harmonic, and intensity cues) when appropriate based on input. The program can select music using date of birth, listed musical preferences, and entraining tempo to provide a passive music playlist to be used on a steady basis. The key inputs for gait training are the cadence, symmetry, and stride length of the user performing a physical action (e.g., walking). The program uses connected hardware to provide haptic / vibration feedback at the BPM of the music. Appropriate populations for gait training include patients with traumatic brain injury (TBI), stroke, Parkinson's disease, MS, and aging.

[0112] The method begins at step 1802. At step 1804, biomechanical data is received at the collector 106 based on data from sensors (e.g., sensors 200, 206, 208). The biomechanical data includes initiation, stride length, cadence, symmetry, data on assistive devices, or a set of such patient features (stored and generated by the analysis system 108). Examples of biomechanical data parameters are listed in Tables 1, 2, and 3 above. From one or more data sources, a baseline condition is determined. First, the patient's walking without music playing is sensed. The sensor and feature data on the patient's initiation, stride length, cadence, symmetry, data on assistive devices, etc. constitute the patient's baseline biomechanical data for the treatment session. Second, the sensor data from previous sessions of the same patient and any higher level classification data from the analysis system 108 constitute the patient's historical data. Third, the sensor data and the higher level classification data (for other similarly situated patients) constitute the population data. Thus, the baseline conditions may include data from one or more of: (a) the patient's baseline biomechanical data for the treatment session; (b) data from the patient's previous sessions; and (c) population data. A baseline beat tempo is then selected from the baseline conditions. For example, the baseline beat tempo may be selected to match the patient's current cadence before playing the music. Alternatively, the baseline beat tempo may be selected as a fraction or multiple of the patient's current cadence. As another alternative, the baseline beat tempo may be selected to match the baseline beat tempo used in the patient's previous sessions. As yet another alternative, the baseline beat tempo may be selected based on baseline beat tempos used for other patients with similar physical conditions. Finally, the baseline beat tempo may be selected based on a combination of any of the above data. From this data, a target beat tempo may also be determined.For example, the target beat tempo may be selected as a percentage increment in the baseline beat tempo with reference to improvements shown by other similarly situated patients, Tempo being understood to refer to the frequency of beats in music.

[0113] In step 1806, music provided to the patient from the handheld device 220 on the music providing device 230 (e.g., earbuds or headphones or speakers) is initiated at a baseline tempo or a division of the baseline tempo. To provide music to the patient at the baseline tempo, music having a constant baseline tempo is selected from a database or existing music is modified (e.g., selectively sped up or slowed down to provide a beat signal at a constant tempo).

[0114] In step 1808, the patient is instructed to listen to the beat of the music. In step 1810, the patient is instructed to walk at a baseline tempo (may receive cues for the left and right foot). The patient is instructed to walk so that each step closely matches the beat of the music (e.g., "in time" with the beat tempo). Steps 1806, 1808, and 1810 may be initiated by the therapist or by audio or visual cues on the handheld device 220.

[0115] In step 1812, the sensors 200, 206, 208 attached to the patient are used to record patient data (heel strike pressure, six-dimensional motion, EMG activity, video recording of the patient's movements, etc.). All sensor data is time-stamped. Data analysis, including the "gated" analysis described herein, is performed on the time-stamped sensor data. For example, analysis of the sensor data (e.g., heel strike pressure) is performed to determine the onset time of each step. Additional data received includes the time associated with each beat signal of the music provided to the patient.

[0116] In step 1814, a connection is made to an entrainment model (e.g., the ensemble machine learning system 410 of the analysis system 108, or a model downloaded to the collector 106 and executed on the handheld device 220) for prediction and classification. (It is understood that such a connection may already exist or may be initiated at this point.) Such a connection is typically very fast or instantaneous.

[0117] In step 1816, an optional entrainment analysis performed by analysis system 108 is applied to the sensor data. The entrainment analysis includes determining the delay between the beat signal and the onset of each step taken by the patient. As an output from the entrainment analysis, a determination is made as to the accuracy of the entrainment (e.g., a measure of the instantaneous relationship between the baseline beat tempo and the patient's steps, as described above with respect to the entrainment potential and EP ratio). If the entrainment is not accurate (e.g., the entrainment potential is not constant within an acceptable range), an adjustment is made in step 1818 (e.g., speeding up or slowing down the beat tempo, increasing the volume, increasing the sensory input, overlaying a metronome or other relevant sound, etc.). If the entrainment is accurate (e.g., the entrainment potential is constant within an acceptable range), an incremental change is made to the tempo in step 1820. For example, the baseline tempo of the music played on the handheld device is increased (e.g., by 5%) toward the target tempo.

[0118] At step 1822, connections are made to an entrainment model for prediction and classification. (It is understood that such connections may already exist or may be initiated at this point.) At step 1824, an optional symmetry analysis is applied to the sensor data. As an output from the symmetry analysis, a determination is made as to the symmetry of the patient's gait (e.g., how well the movement of the patient's left foot matches the movement of the patient's right foot in terms of stride length, speed, stance phase, sway phase, etc.). If the steps are not symmetrical (e.g., below a threshold), at step 1826, an adjustment is made to the music broadcast to the patient by the handheld device. A first modification may be made to the music played while one of the patient's feet is moving, and a second modification may be made to the music played while the other of the patient's feet is moving. For example, a minor chord (or increased volume, sensory input, tempo change, or audio / metronome superimposition) is played on one side (e.g., the affected side) and a major chord is played on the other side (e.g., the unaffected side). The machine learning system 410 predicts when a symmetry problem will arrive based on the "fingerprint" of each scenario leading up to it (e.g., by analyzing movements that indicate asymmetry). Asymmetry can be determined by comparing normal gait parameters, as someone in the background can determine how affected their side is (and compared to the other).

[0119] At step 1828, connections are made to an entrainment model for prediction and classification. (It is understood that such connections may already exist or may be initiated at this point.) At step 1830, an optional center of balance analysis (e.g., whether the patient is leaning forward) is performed on the sensor data. This analysis may be performed by combining the outputs of the foot sensors and by combining the video outputs. As an output from the center of balance analysis, a determination is made as to whether the patient is leaning forward. If the patient is leaning forward, at step 1832, a cue is given to the patient (provided by the therapist or by audio or visual instructions on the handheld device) to "stand up straight."

[0120] At step 1834, a connection is made to an entrainment model for prediction and classification. (It is understood that such connections may already exist or may be initiated at this point.) At step 1836, an initiation analysis (e.g., the patient exhibits hesitation or difficulty in initiating walking) is performed on the sensor data. As an output from the initiation analysis, a determination is made as to whether the patient exhibits problems with initiation. If the patient exhibits problems with initiation (e.g., below a threshold), at step 1838, haptic feedback may be provided to the patient (which may include a countdown in beat tempo or before the start of a song).

[0121] At step 1840, it may be determined whether the patient is using an assistive device (e.g., a cane, crutches, walker, etc.). In some embodiments, the handheld device 220 provides a user interface for the patient or therapist to input information regarding the use of the assistive device. If a cane is present, at step 1842, the analysis changes to three meters (e.g., cane, right foot, left foot) with cues of "left foot," "right foot," and "cane" provided by the therapist or by audio or visual instructions on the handheld device.

[0122] At step 1844, a connection is made to an entrainment model for prediction and classification. (It is understood that such a connection may already exist or may be initiated at this point.) Optionally, an entrainment analysis 1846 (substantially as described above in step 1816, but with differences described below) may be applied to the sensor data. For example, the entrainment is compared to previous entrainment data from an earlier time point in the session, data from previous sessions of the patient, or data regarding entrainment of other patients. As an output from the entrainment analysis, a determination is made regarding the accuracy of the entrainment (e.g., how well the patient's gait is aligned to the baseline tempo). If the entrainment is not accurate, at step 1848, adjustments are made in a manner similar to that described in step 1818 above.

[0123] If the entrainment is accurate, then in step 1850 a determination is made whether the patient is walking at the target tempo. If the target tempo has not been reached, the method proceeds to step 1820 (described above) where an incremental change is made to the tempo. For example, the baseline tempo of the music played on the handheld device is increased or decreased (e.g., by 5%) toward the target tempo. If the target tempo has been reached, the patient may continue therapy for the remaining time in the session (step 1852). In step 1854, music at the desired tempo may be curated and left on the device 220 of FIG. 2 for use outside of therapy sessions. This music content is used by the patient as homework / practice between dedicated therapy sessions. The program ends in step S827.

[0124] 18 may be performed in an order different from that disclosed. For example, the assessments in steps 1816, 1824, 1830, 1836, 1840, 1846, and 1850 may be performed simultaneously. Furthermore, multiple connections to analysis system 108 (e.g., steps 1814, 1822, 1828, 1834, and 1844) may only be made once throughout the described treatment session.

[0125] FIG. 19 illustrates a technique useful for neglect training. For neglect training, the systems and methods described herein use connected hardware to provide tactile / vibration feedback when the patient correctly taps the target. The connected hardware includes a device, a video motion capture system, or a connected bell. All of these devices are connected within the above-mentioned system and have speakers to vibrate when tapped and play auditory feedback. For example, a connected bell provides data to the system in the same manner as the sensor 200 (e.g., data about the patient tapping the bell). A video motion capture system provides video data to the system in the same manner as the video camera 206. The key input for neglect training is information about tracking of movement to a specific location. The program uses connected hardware to provide tactile / vibration feedback when the patient correctly taps the target. A suitable population for neglect training includes patients with spatial neglect or visuospatial neglect conditions.

[0126] The flow diagram illustrated in FIG. 19 for neglect training is substantially identical to the flow diagram illustrated in FIG. 18 for locomotor training, with differences as described below. For example, the baseline test establishes the patient's condition and / or improvement from previous tests. In some embodiments, the baseline test includes showing four objects evenly spaced from left to right on a screen (e.g., the display 222 of the handheld device 220). The patient is instructed by a cue appearing on the display 222 or audibly by the therapist to strike the objects in time with the beat of the background music. As in locomotor training, the patient is instructed to strike a bell in time with the beat of the background music. Feedback is provided for each accurate strike. Once baseline information is collected, several objects evenly spaced from left to right are displayed on the screen. As described above, the patient is instructed to strike the objects in sequence from left to right in time with the beat of the background music. Feedback is provided for each accurate strike. As with gait training, analysis system 108 evaluates the patient's response, classifies the response, and provides instructions to increase or decrease the object, or to increase or decrease the tempo of the music to reach a target tempo.

[0127] FIG. 20 illustrates a technique useful for intonation training. For intonation training, the systems and methods described herein rely on speech processing algorithms. The phrases typically selected are common words that belong to the categories of bilabial, guttural, and vowels. Hardware is connected to the patient to provide beats-per-minute tactile feedback to one of the patient's hands. The key inputs for intonation training are tone of voice, spoken words, and speech rhythm. Suitable populations for intonation training include patients with Broca's aphasia, expressive aphasia, non-fluent aphasia, apraxia, autism spectrum disorder, and Down's syndrome.

[0128] The flow diagram for intonation training, illustrated in FIG. 20, is substantially identical to the flow diagram illustrated in FIG. 18 for gait training, with differences that are described below. For example, tactile feedback is provided to one of the patient's hands to assist with tapping. The patient is instructed to listen to music that is played by cues appearing on the display 222 or audibly by the therapist. The spoken phrase to be learned is played in two parts, the first of which is pitched high and the second of which is pitched low. The patient is then instructed to sing the phrase using the device with the two pitches played by cues appearing on the display 222 or audibly by the therapist. As with gait training, the analysis system 108 evaluates the patient's response, classifies the response in terms of pitch accuracy, words spoken, and ranking by the patient or assistant / therapist, provides instructions to provide alternative phrases, and compares the response to target speech parameters.

[0129] FIG. 21 illustrates a technique useful for musically stimulated training. For musically stimulated training, the systems and methods described herein rely on speech processing algorithms. Well-known songs are used, along with algorithms to filter out predictive sections (called expectant interference). The hardware includes a speaker that accepts and processes the singing by the patient, and in some embodiments, the therapist may manually provide input regarding the accuracy of the singing. Key inputs are information about tone of voice, words spoken, rhythm of speech, and musical preferences. Suitable populations include patients with Broca's aphasia, non-fluent aphasia, TBI, stroke, and primary progressive aphasia.

[0130] The flow diagram for musical stimulation training, illustrated in FIG. 21, is substantially identical to the flow diagram illustrated in FIG. 18 for gait training, with differences that are explained below. For example, music is played for the patient, and the patient is instructed to listen to a song by cues appearing on the display 222 or audibly by the therapist. Musical cues are added to the song. Then, at predictive spots, words or sounds are blanked out and gestural musical cues are played to prompt the patient to sing the missing words or sounds. As with gait training, the analysis system 108 evaluates the patient's responses, classifies them in terms of pitch accuracy, words spoken, and ranking by the patient or assistant / therapist, and provides instructions to play additional portions of the song to improve the speech to the target speech parameters.

[0131] FIG. 22 illustrates a technique useful for gross motor training of patients. For gross motor training, the systems and methods described herein are directed to assisting with ataxia, range of motion, or initiation. The more difficult parts of the exercise are musically "accented" (e.g., by using melodic, harmonic, rhythmic, and / or strength cues). The key input is information about the movement in X, Y, and Z capture via connected hardware or video camera systems. The appropriate populations are neurological, orthopedic, strength, endurance, balance, posture, range of motion, TBI, SCI, stroke, and cerebral palsy patients.

[0132] The flow diagram for gross motor training, illustrated in Figure 22, is substantially identical to the flow diagram illustrated in Figure 18 for gait training, with differences as described below. As with gait training, the patient is provided with cues to move in time to the baseline beat of a musical selection. Analysis system 108 evaluates the patient's response, classifies the response in terms of movement precision and entrainment as described above, and provides instructions to increase or decrease the tempo of the music being played.

[0133] FIG. 23 illustrates a technique useful for grip training. For grip training, the systems and methods described herein rely on sensors associated with a gripping device. The hardware includes a gripping device with connected speakers, pressure sensors associated with a handheld 220. The key input is pressure provided by the patient to the gripping device in a manner similar to the heel impact pressure measured by the sensor 200. Suitable populations are neurological, orthopedic, strength, endurance, balance, posture, range of motion, TBI, SCI, stroke, and cerebral palsy patients.

[0134] The flow diagram for grip strength training, illustrated in Fig. 23, is substantially identical to the flow diagram illustrated in Fig. 18 for gait training, with differences as described below. As with gait training, the patient is provided with cues to apply forces to the gripping device timed to the baseline beat of the musical selection. Analysis system 108 evaluates the patient's response, classifies the response in terms of movement precision and entrainment as described above, and provides instructions to increase or decrease the tempo of the music being played.

[0135] FIG. 24 illustrates a technique useful for speech cueing training. For speech cueing training, the systems and methods described herein rely on voice processing algorithms. The hardware may include a speaker that accepts and processes the singing by the patient, and in some embodiments, the therapist may manually provide input regarding speech accuracy. Key inputs are tone of voice, words spoken, speech rhythm, and musical preferences. Suitable populations include patients with robotic problems, word finding problems, and stuttering problems.

[0136] The flow diagram for speech cueing training, shown in Figure 24, is substantially identical to the flow diagram illustrated in Figure 18 for gait training, with differences that are explained below. As with gait training, the patient is cued by cues appearing on the display 222 or audibly by the therapist to speak sentences by saying one syllable timed to each beat of the musical selection. Analysis system 108 evaluates the patient's speech and classifies the response in terms of speech accuracy and entrainment as described above, and provides instructions to increase or decrease the tempo of the music being played.

[0137] Figure 25 illustrates a technique useful for training minimally conscious patients. The systems and methods described herein rely on an imaging system, such as a 3D camera, to measure whether the patient's eyes are open, the direction the patient is looking, and the resulting patient's pulse and heart rate. The program explores and optimizes heart rate, stimulation, respiratory rate, eye closure, posture, and anxiety. Suitable populations include patients with coma and impaired consciousness.

[0138] The flow diagram for training a minimally conscious patient, shown in FIG. 25, is substantially identical to the flow diagram illustrated in FIG. 18 for gait training, with the differences explained below. As in gait training, the patient is provided with increasing stimuli at the patient's breathing rate (PBR). For example, the patient is first provided with musical chord stimuli at PBR, and it is observed whether the patient's eyes are open or not. If the patient's eyes are not open, the stimuli are sequentially increased (or a recording of such sounds is played) from humming a simple melody at PBR, to singing "ahh" at PBR, to singing the patient's name at PBR, and at each input, it is checked whether the patient's eyes are open or not. The analysis system 108 evaluates the patient's eye tracking, classifies the response in terms of consciousness level, and provides instructions to change the stimuli.

[0139] 26-28 illustrate techniques useful for attention training. For attention training, the systems and methods described herein operate in a closed-loop fashion to help patients maintain, divide, shift, and select attention. No visual cues are allowed to signal which movement to perform. Appropriate populations include patients with brain tumors, multiple sclerosis, Parkinson's disease, and neurological diseases and disorders.

[0140] The flow diagram for attention maintenance training, shown in FIG. 26, is substantially identical to the flow diagram illustrated in FIG. 18 for gait training, with the differences described below. As with gait training, the patient is provided with an instrument (which may be any functional instrument, drumsticks, drums, keyboard, or wirelessly connected, respectively) and is instructed by cues appearing on the display 222 or audibly by the therapist to proceed along or perform tasks against audio cues defined by levels 1-9 illustrated in FIG. 26. The analysis system 108 evaluates the patient's ability to complete the task accurately, classifies the response, and varies the tempo or difficulty of the task. Similarly, FIG. 27 illustrates a flow diagram for attention shift training. In attention shift training, instructions are provided by cues appearing on the display 222 or audibly by the therapist to proceed along or perform tasks against audio cues alternating between the left and right ears. FIG. 28 illustrates a flow diagram for divided attention. Here, along with audio signals at both the left and right ears, instructions are provided to proceed along or perform a task relative to the audio cues.

[0141] The flow diagram for agility training, shown in Figure 29, is substantially identical to the flow diagram illustrated in Figure 18 for gait training, with differences that are explained below. For agility training, the patient is instructed to tap their fingers on piano keys to collect baseline movement and range of motion information. A song is started at a particular beats per minute, and the patient begins tapping the baseline number of fingers. The analysis system 108 evaluates the patient's ability to complete the task accurately, classifies the response, and varies the tempo or difficulty of the task.

[0142] The flow diagram for oral motor training, shown in Figure 30, is substantially identical to the flow diagram illustrated in Figure 18 for gait training, with differences that are explained below. For oral motor training, the patient is instructed to perform a task alternating two sounds (e.g., "o" and "ah"). Analysis system 108 assesses the patient's ability to complete the task accurately, classifies the response, and varies the tempo or difficulty of the task (e.g., by providing different target sounds).

[0143] The flow diagram for breathing exercises, shown in Fig. 31, is substantially identical to the flow diagram illustrated in Fig. 18 for walking exercises, with differences explained below. For breathing exercises, a baseline breathing rate and shallowness are determined. Music is provided at the patient's baseline tempo at their breathing rate, and the patient is instructed to perform the breathing tasks described at each level in Fig. 31. Analysis system 108 assesses the patient's ability to complete the task accurately, classifies the response, and varies the tempo or difficulty of the task (e.g., by providing different breathing patterns).

[0144] The above-mentioned systems, devices, methods, processes, etc. can be implemented in hardware, software, or any suitable combination thereof for an application. Hardware may include general-purpose computers and / or special-purpose computing devices. This includes implementations in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices or processing circuits (with internal and / or external storage). Additionally or alternatively, it may include one or more application specific integrated circuits, programmable gate arrays, programmable array logic components, or any other device(s) that can be configured to process electronic signals. It is further understood that implementations of the above-mentioned processes or devices may include computer executable code (created using a structured programming language (e.g., C), an object-oriented programming language (e.g., C++), or any other high-level or low-level programming language (including assembly language, hardware description language, database programming language and techniques)) that can be stored, compiled, or translated for execution on one or more of the above-mentioned devices, and combinations of heterogeneous processors, processor architectures, or combinations of various hardware and software. In another aspect, the method may be implemented as a system performing each of its steps, or may be distributed in some manner across multiple devices. At the same time, processing may be distributed across multiple devices (such as the various systems described above), or all functionality may be integrated into a dedicated standalone device or other hardware. In another aspect, the means for performing each step associated with each of the processes described above may include any of the hardware and / or software described above. All such permutations are considered to fall within the scope of the present disclosure.

[0145] Each of the embodiments described herein may include a computer program product that includes computer executable or computer usable code that, when executed on one or more computing devices, performs all and / or any of the steps. The code may be stored in a non-transitory manner in computer memory (from which the program may execute, such as a random access memory associated with a processor) or storage (such as a disk drive, flash memory, or any other optical, electromagnetic, magnetic, infrared, or other device or combination of devices). In another aspect, any of the above-described systems and methods may be implemented in any suitable transmission or propagation medium (one that carries the computer executable code) and / or any input / output thereof.

[0146] It is understood that the above-described apparatus, systems, and methods are described by way of example, and not by way of limitation. Except where otherwise expressly stated, each of the disclosed steps can be modified, supplemented, omitted, and / or rearranged without departing from the scope of the present disclosure. Numerous variations, additions, omissions, and other modifications will be apparent to those skilled in the art. In addition, the order and presentation of the method steps in the above description and drawings are not intended to require that each of the described steps be performed in the order in which they are described, unless that order is expressly required or otherwise clear from the context.

[0147] Each method step of the implementations described herein is intended to include any suitable manner of causing such method step to be performed (consistent with the patentability of the appended claims), unless a different meaning is expressly provided or otherwise apparent from the context. Thus, for example, performing step X includes any suitable manner of causing other participants (e.g., a remote user, a remote processing resource (such as a server or cloud computer), or a machine) to perform method X. Similarly, performing steps X, Y, and Z may include any manner of controlling or directing any combination of such other individuals or resources to perform steps X, Y, and Z to obtain the benefit of such steps. Thus, unless a different meaning is expressly provided or otherwise apparent from the context, each method step of the implementations described herein is intended to include any suitable manner of causing one or more other participants or entities to perform each step (consistent with the patentability of the appended claims). Such participants or entities need not be under the direction or control of any other participants or entities, nor need they be located in any particular jurisdiction.

[0148] Moreover, it should be understood that the methods described above are provided by way of example, and unless expressly stated otherwise, the disclosed steps can be modified, supplemented, omitted, and / or rearranged without departing from the scope of the present disclosure.

[0149] It is understood that the above methods and systems are described by way of example, and not by way of limitation. Numerous variations, additions, omissions, and other modifications will be apparent to those skilled in the art. In addition, the order or presentation of method steps in the above description and drawings is not intended to require that the described steps be performed in that order, unless that order is expressly required or otherwise clear from the context. Thus, while specific embodiments have been shown and described, it will be apparent to those skilled in the art that numerous changes and modifications in form and detail can be made without departing from the spirit and scope of the present disclosure.

Claims

1. 1. A system for the rehabilitation of patients with physical disabilities by providing music therapy during treatment sessions, comprising: During the treatment session, music having a plurality of beat signals at a beat tempo is provided to the patient, each of the beat signals being output at a respective output time; during the treatment session, the patient attempts to perform a plurality of repetitive movements timed to the plurality of beat signals; The system comprises: a music providing device that outputs the music having the plurality of beat signals at the beat tempo having a certain frequency to the patient; a computer system having one or more physical processors configured with machine-readable instructions; Equipped with When executed, the machine-readable instructions cause the one or more physical processors to: receiving time-stamped biomechanical data of the patient for a plurality of repetitive movements, each of the repetitive movements being performed by the patient in time with a corresponding one of the plurality of beat signals, the biomechanical data being measured using a sensor; analyzing the time-stamped biomechanical data to identify an onset time for each of the plurality of repetitive movements; measuring a delay between the onset time and the output time of the corresponding beat signal for each of the plurality of repetitive movements occurring over a period of time; measuring the measured delay corresponding to an entrainment parameter; modifying the music based on the entrainment parameters, the modifying including increasing the beat tempo towards the target beat tempo when the delay between the onset time and the output time of the corresponding beat signal, measured individually for each of a plurality of the repetitive movements occurring over a period of time, is substantially constant and when the beat tempo is less than a target beat tempo, the modified beat tempo having a constant frequency; continuing the treatment session with the modified music; and A system that executes the above.

2. and further comprising the sensor associated with a patient. the one or more physics processors are configured to measure the biomechanical data from the patient; The system of claim 1 , wherein measuring the biomechanical data from the patient comprises measuring, with the sensor, one or more of motion, acceleration, and pressure associated with movement of the patient.

3. The sensor includes: an inertial measurement unit (IMU) device configured to measure motion data; an image capture device configured to capture video associated with the patient's movements; The system of claim 2 , comprising one or more of:

4. The one or more physical processors include: modifying one or more of the plurality of beat signals; adjusting the chords of the music; adjusting the volume of the music; adjusting the beat of the music; adding one or more auditory cues to the music; adding rhythmic components to the music; and The system of claim 1 , configured to modify the music by at least one of:

5. the one or more physical processors are configured to provide cues to the patient for performing each repetitive movement in time with a corresponding beat signal; The system of claim 1 , wherein the cues to the patient include visual, auditory, or tactile cues provided to the patient in association with a corresponding beat signal.

6. the one or more physical processors are configured to determine the entrainment parameter by determining in real time a delay between the onset time of each of the plurality of repetitive movements and the output time of each of the corresponding beat signals; A specified onset time for a repetitive movement is a time at which a specified identifiable event occurs during the repetitive movement. The system of claim 1 .

7. The one or more physical processors include: selecting music having a plurality of beat signals at said beat tempo; and modifying existing music to have a plurality of beat signals at said beat tempo; The system of claim 1 , configured to perform one or more of the following:

8. the plurality of repetitive movements includes the patient walking; The one or more physical processors include: analyzing the biomechanical data to classify the patient's gait symmetry; modifying the music as a function of the classification of the symmetry of the patient's gait. The system of claim 1 , configured to:

9. The one or more physical processors include: analyzing the biomechanical data to classify a defect in the repetitive motion; modifying the music based on the classification of the defects; The system of claim 1 , configured to:

10. The one or more physical processors include: receiving an initial set of biomechanical data from the patient relating to the repetitive motion of the patient; determining a baseline condition for the patient based on the initial set of biomechanical data; determining the beat tempo based on the baseline condition; The system of claim 1 , configured to:

11. 1. A system for the rehabilitation of patients with physical disabilities by providing music therapy during treatment sessions, comprising: During the treatment session, music having a plurality of beat signals at a beat tempo is provided to the patient, each of the beat signals being output at a respective output time; During the treatment session, the patient performs a plurality of repetitive movements timed to the plurality of beat signals; The system includes a music providing device that outputs the music having the plurality of beat signals at the beat tempo having a certain frequency to the patient; The system is configured to: receiving time-stamped biomechanical data of said patient, wherein the time-stamped biomechanical data being associated with each repetitive motion performed by the patient in association with a corresponding one of the plurality of beat signals; the biomechanical data is measured using a sensor associated with the patient; - analyzing the time-stamped biomechanical data to identify an onset time for one or more of the plurality of repetitive movements; - determining, for each of one or more repetitive movements, a delay between the identified onset time and a corresponding output time of a corresponding one of the plurality of beat signals; - the measured delay corresponds to an entrainment parameter, wherein the delay is measured separately for each of the plurality of repetitive movements occurring over a period of time; - modifying the music output to the patient based on the entrainment parameters, thereby causing the music providing device to output modified music to the patient. a computer system having one or more physical processors, the computer system being configured to: The system, wherein modifying the music includes increasing the beat tempo toward the target beat tempo in response to determining that the delay measured individually for each of the plurality of repetitive movements occurring over the time period is substantially constant and in response to determining that the beat tempo is less than a target beat tempo, wherein the modified beat tempo has a constant frequency.

12. 12. The system of claim 11, further comprising the sensor associated with the patient and measuring the time-stamped biomechanical data.

13. The one or more physical processors include: modifying one or more of the plurality of beat signals; adjusting the chords of the music; adjusting the volume of the music; adjusting the beat of the music; adding one or more auditory cues to the music; adding rhythmic components to the music; and The system of claim 12 , further comprising:

14. the computer system is configured to determine the entrainment parameters in real time; A specified onset time for a repetitive movement is a time at which a specified identifiable event occurs during the repetitive movement. The system of claim 11.

15. The computer system is configured to: analyzing the biomechanical data to classify a defect in the repetitive motion; Modifying the music based on the classification of the defects. The system of claim 11 , configured as follows:

16. 1. A system for the rehabilitation of a patient with a physical disability by providing rhythmic auditory therapy during a treatment session, comprising: During the therapy session, the patient is provided with rhythmic auditory content having a plurality of beat signals at a beat tempo, each of the beat signals being output at a respective output time; During the treatment session, the patient repeatedly attempts to perform a repetitive movement in time with the plurality of beat signals; The system comprises: an auditory presentation device that outputs to a patient the rhythmic auditory content having the plurality of beat signals at the beat tempo having a certain frequency; a computer system having one or more physical processors configured with machine-readable instructions; Equipped with The machine-readable instructions, when executed, cause the one or more physical processors to: receiving time-stamped biomechanical data of the patient relating to the repetitive movements performed by the patient, each of the repetitive movements being performed in association with a corresponding one of the plurality of beat signals, the biomechanical data being measured using a sensor; determining a temporal relationship between each repetitive motion and its corresponding beat signal for each of the plurality of repetitive motions occurring over a time period based on the time-stamped biomechanical data and each of the output times of each of the corresponding beat signals; calculating an entrainment parameter based on the determined temporal relationship for each of the one or more of the repetitive movements; modifying the rhythmic auditory content output to the patient based on the determined entrainment parameters, the modifying including increasing the beat tempo toward the target beat tempo in response to determining that the temporal relationship determined individually for each of the plurality of repetitive movements occurring over the time period is substantially constant and in response to determining that the beat tempo is less than a target beat tempo, the modified beat tempo having a constant frequency; Configure the system to run.

17. the one or more physical processors are configured to repeat the receiving, determining, calculating, and modifying throughout the treatment session; whereby the rhythmic auditory content output to the patient is dynamically modified in real time based on the entrainment parameters.

17. The system of claim 16.

18. The sensor further comprises: The system of claim 16 , wherein the one or more physics processors measure the biomechanical data from the patient using the sensors associated with the patient.

19. The one or more physical processors include: modifying one or more of the plurality of beat signals; adjusting a chord of said rhythmic auditory content; and adjusting a volume of the rhythmic auditory content; and adjusting the meter of the rhythmic auditory content; and adding one or more auditory cues to the rhythmic auditory content; adding rhythmic components to said rhythmic auditory content; The system of claim 16 , configured to modify the rhythmic auditory content by at least one of:

20. the one or more physical processors are configured to provide cues to the patient for performing each repetitive movement in time with a corresponding beat signal; 17. The system of claim 16, wherein the cues include visual, auditory, or tactile cues provided to the patient in association with a corresponding beat signal.

21. The one or more physical processors further include: analyzing the biomechanical data to assess deficiencies in the repetitive movements; modifying the rhythmic auditory content based on the assessment of the deficiencies. The system of claim 16, configured to:

Images