Systems and methods for neurorehabilitation

Abstract

The present invention discloses a method and system for rehabilitating a patient with a limb disability by providing repetitive motion therapy. The system is configured to receive biomechanical data from the patient regarding repetitive movements performed using a first side and a second side of the body, respectively, and select an entrainment side based on the data to provide the repetitive motion therapy. The system further performs the repetitive motion therapy by: providing auditory stimulation to the patient, the auditory stimulation including a beat signal output at corresponding beat times; receiving time-stamped biomechanical data of repetitive movements performed by the patient using the entrainment side associated with the corresponding beat times of the beat signal; calculating an entrainment potential of the entrainment side by comparing the timing of the repetitive movements with the timing of the beat signal; and modifying the auditory stimulation based on the calculated entrainment potential.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a continuation-in-part of U.S. patent application Ser. No. 16 / 569,388, filed on September 12, 2019, entitled “Systems and Methods for Neurologic Rehabilitation,” and is further based upon and claims the benefit of U.S. Provisional Patent Application Ser. No. 62 / 934,457, filed on November 12, 2019, entitled “Systems and Methods for Neurologic Rehabilitation,” each of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to systems and methods for rehabilitating users with physical impairments by providing music therapy. Background Art

[0004] Over the past decade, numerous controlled studies have highlighted the clinical role of music in neurorehabilitation. For example, rigorously planned music therapy is known to directly lead to cognitive, motor, and language enhancements. The process of listening to music enhances brain activity in multiple ways, stimulating a broad bilateral network of brain regions associated with attention, semantic processing, memory, cognition, motor function, and emotional processing.

[0005] Clinical data supports the ability of music therapy to enhance memory, attention, executive function, and mood. PET scan studies investigating the neural mechanisms underlying music have shown that pleasurable music stimulates an extensive network of cortical and subcortical regions, including the ventral striatum, nucleus accumbens, amygdala, insula, hippocampus, hypothalamus, ventral tegmental area, anterior cingulate cortex, orbitofrontal cortex, and ventromedial prefrontal cortex. The ventral tegmental area produces dopamine and has direct connections to the amygdala, hippocampus, anterior cingulate cortex, and prefrontal cortex. This mesocorticolimbic system, activated by music, plays a key role in regulating arousal, mood, reward, memory attention, and executive function.

[0006] Neuroscience research reveals how the fundamental organizational processes of musical memory formation share mechanisms with non-musical memory processes. The underlying principles of phrase grouping, hierarchical abstraction, and musical patterning bear direct parallels to the temporal chunking principles underlying non-musical memory processes. This suggests that memory processes activated by music can translate and enhance non-musical processes.

[0007] Therefore, there remains a need for improved apparatuses, systems, and methods for protecting the use of user identities and securely providing personal information. Summary of the Invention

[0008] In one aspect of the disclosed subject matter, a method is provided for rehabilitating a patient with a limb impairment by providing repetitive motion therapy in which auditory stimulation with a metronomic signal is provided to the patient and the patient attempts to perform repetitive movements in time with the metronomic signal using a first side and an opposing second side of the patient's body. The method is implemented on a computer system having a processor configured with machine-readable instructions that, when executed, perform the method.

[0009] Specifically, the method includes the step of receiving, at the computer system, biomechanical data from the patient regarding repetitive movements performed by the patient using the first side and the second side of the body, respectively. The method also includes selecting, by the computer system, an entrainment side based on the received biomechanical data, wherein the entrainment side is one of the first side or the second side of the body.

[0010] Additionally, the method includes executing, by the computer system, a repetitive motion therapy. Executing the repetitive motion therapy includes outputting auditory stimulation to the patient, the auditory stimulation including a beat signal output at corresponding beat times, and receiving time-stamped biomechanical data from the patient associated with repetitive movements performed by the patient using at least the entrainment side associated with the beat signal. Executing the repetitive motion therapy further includes calculating an entrainment potential for the entrainment side, wherein the entrainment potential is calculated by comparing the timing of the repetitive movements performed by the patient using the entrainment side with the corresponding beat times of the beat signal. Executing the repetitive motion therapy also includes modifying the auditory stimulation based on the calculated entrainment potential.

[0011] According to another aspect, a system for rehabilitating a patient with a limb disability by providing repetitive motion therapy is provided, wherein the patient is provided with auditory stimulation having a metronomic signal and attempts to perform repetitive movements using a first side of the patient's body and an opposing second side of the body in time with the metronomic signal. The system includes a computer system having a processor configured by machine-readable instructions to receive biomechanical data from the patient regarding repetitive movements performed by the patient using the first and second sides of the body, respectively, wherein the biomechanical data is measured using sensors associated with the patient. In addition, the processor is configured to select an entrainment side based on the received biomechanical data, wherein the entrainment side is one of the first side or the second side of the body.

[0012] Furthermore, the processor is configured to perform repetitive motion therapy by outputting auditory stimulation to the patient, the auditory stimulation including a beat signal output at corresponding beat times, and receiving time-stamped biomechanical data from the patient associated with repetitive movements performed by the patient using at least the entrainment side associated with the beat signal. The biomechanical data is measured using a sensor associated with the patient. Furthermore, the processor is configured to perform repetitive motion therapy by calculating an entrainment potential of the entrainment side and by modifying the auditory stimulation based on the calculated entrainment potential. Specifically, the entrainment potential is calculated by comparing the timing of the repetitive movements performed by the patient using the entrainment side with the corresponding beat times of the beat signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Objects, features, and advantages of the devices, systems, and methods described herein will become apparent from the following description of specific embodiments thereof, as illustrated in the accompanying drawings, which are not necessarily drawn to scale, emphasis instead being placed upon illustrating the principles of the devices, systems, and methods described herein.

[0014] Figure 1 is a diagram illustrating a system for treating a user by providing music therapy according to an exemplary embodiment of the disclosed subject matter;

[0015] Figure 2 is a diagram illustrating several components of a system for rehabilitating a user by providing music therapy, according to an exemplary embodiment of the disclosed subject matter;

[0016] Figure 3 is a schematic diagram of a sensor for measuring biomechanical movement of a patient according to an exemplary embodiment of the disclosed subject matter;

[0017] Figure 4 is a diagram showing several components of a system according to an exemplary embodiment of the disclosed subject matter;

[0018] Figure 5 An exemplary display illustrating components of a system for rehabilitating a user by providing music therapy according to an exemplary embodiment of the disclosed subject matter;

[0019] Figure 6 is a flow chart of one implementation of an analysis process according to an exemplary embodiment of the disclosed subject matter;

[0020] Figure 7-10 is a flow chart of one implementation of a process according to an exemplary embodiment of the disclosed subject matter;

[0021] Figure 11is a time chart illustrating music and a patient's body movements according to an exemplary embodiment of the disclosed subject matter;

[0022] Figure 12-13 Patient responses according to exemplary embodiments of the disclosed subject matter are presented;

[0023] Figure 14-15 Patient responses according to exemplary embodiments of the disclosed subject matter are presented;

[0024] Figure 16-17 Patient responses according to exemplary embodiments of the disclosed subject matter are presented;

[0025] Figure 18 An embodiment of a technique for gait training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0026] Figure 19 An embodiment of a technique for neglect training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0027] Figure 20 An embodiment of a technique for voice intonation training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0028] Figure 21 An embodiment of a technique for music stimulation training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0029] Figure 22 An embodiment of a technique for gross motor training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0030] Figure 23 An embodiment of a technique for handgrip training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0031] Figure 24 An embodiment of a technique for verbal prompt training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0032] Figure 25 Implementations of techniques for training minimally conscious patients according to exemplary embodiments of the disclosed subject matter are presented;

[0033] Figures 26-28 An embodiment of a technique for attention training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0034] Figure 29 An embodiment of a technique for flexibility training of a patient according to an exemplary embodiment of the disclosed subject matter is presented;

[0035] Figure 30 An embodiment of a technique for oral motor training of a patient according to an exemplary embodiment of the disclosed subject matter is presented; and

[0036] Figure 31 An embodiment of a technique for respiratory training of a patient according to an exemplary embodiment of the disclosed subject matter is presented; and

[0037] Figure 32

[0014] Embodiments of techniques for side-specific repetitive motion training of a patient according to exemplary embodiments of the disclosed subject matter are presented. DETAILED DESCRIPTION

[0038] The present invention generally relates to systems, methods, and devices for implementing a dynamic closed-loop rehabilitation platform system that monitors and guides changes in human behavior and function, such as changes in speech, movement, and cognition, that are temporally triggered by musical rhythmic, harmonic, melodic, and dynamic cues.

[0039] In various embodiments of the present invention, there is provided Figure 1 The dynamic closed-loop rehabilitation platform music therapy system 100 shown in the figure includes a sensor component and system 102, an edge processing component 104, a collector component 106, an analysis system 108, and a music therapy center 110. As will be described in more detail below, the sensor component, edge processing component, collector component, machine learning process, and music therapy center can be set on various hardware components. For example, in one embodiment, the sensor component and edge processing component can be positioned or worn by the patient. In such embodiments, the collector component and music therapy center can be set on a handheld device. In such embodiments, the analysis system can be located on a remote server.

[0040] Sensor system

[0041] Throughout the description herein, the term "patient" is used to refer to an individual receiving music therapy treatment. The term "therapist" is used to refer to an individual providing music therapy treatment. In some embodiments, a patient can interact with the system described herein to administer treatment without the therapist being present.

[0042] The sensor assembly 102 provides sensed biomechanical data about the patient. In some embodiments, the sensor assembly may include (1) a wearable wireless real-time motion sensing device or IMU (Inertial Measurement Unit); (2) a wearable wireless real-time combined multi-zone plantar pressure / 6-dimensional motion capture (IMU) device, such as sensor 200; (3) a wearable wireless real-time electromyography (EMG) device, such as sensor 208; (4) a real-time wireless near-infrared (NIR) video capture device, such as imaging device 206 (see Figure 4 ).

[0043] like Figure 2 As shown in FIG, the systems and methods described herein are used in conjunction with treating a patient's walking condition. Thus, the exemplary sensor 200 can be a combined multi-zone foot pressure / 6-DOF motion capture device. As the patient walks during a music therapy session, the sensor 200 records the patient's foot pressure and 6-DOF motion profile. In some embodiments, the foot pressure / 6-DOF motion capture device has a variable recording duration, wherein the sampling rate for the foot pressure profile is 100 Hz, and the foot pressure profile includes 1 to 4 zones, resulting in 100 to 400 pressure data points per second per foot.

[0044] The sensor 200 may include a foot pressure pad 202, which may have a heel pad (for measuring one pressure zone, e.g., heel strike pressure) to a full insole pad (for measuring four pressure zones). Pressure measurements are made by sensing the change in resistance of the transducer material due to compression caused by the patient's weight shifting to the foot. These foot pressure maps are obtained for each sampling interval or at a specific moment during a music therapy session.

[0045] The sensor 200 may include a 6-dimensional motion capture device 204 that detects changes in motion using a 6-DOF micro-electromechanical system (MEMS) based sensor that determines the three dimensions A x 、A y 、A z The foot motion capture system measures linear acceleration and rotational motion, such as pitch, roll, and yaw. Sampling at 100 Hz yields 600 motion data points per second. These foot motion captures are acquired for each sampling interval, or at specific moments during a music therapy session.

[0046] Multi-zone pressure sensing with a 6-DOF motion capture device allows for mappable spatial and temporal gait dynamic tracking while walking. Figure 3 A schematic diagram of the sensor 200 is shown in FIG.

[0047] From a system perspective, Figure 4As shown, patient P uses two foot sensors 200, one for each foot, designated right 200R and left 200L. In an exemplary embodiment, right foot sensor 200R wirelessly transmits time-stamped internal measurement unit data and heel strike pressure data on a first channel (e.g., channel 5) in the IEEE 802.15.4 direct sequence spread spectrum (DSSS) RF band. Left foot sensor 200L wirelessly transmits time-stamped internal measurement unit data and heel strike pressure data on a second channel (e.g., channel 6) in the IEEE 802.15.4 direct sequence spread spectrum (DSSS) RF band. As described below, a tablet or laptop computer 220, optionally used by therapist T, includes a wireless USB hub containing two IEEE 802.15.4 DSSS RF transceivers tuned to the first and second channels (e.g., channels 5 and 6) to capture the right / left foot sensor RF data. The handheld wireless trigger 250 is used to start and stop the video and / or to create symbols and index the time stream, as discussed in more detail below.

[0048] The video analysis domain can be used to extract patient semantic and event information about the therapy session. Patient actions and interactions are components of therapy that influence the therapy environment and team. In some embodiments, one or more image capture devices 206, such as cameras (see Figure 4 ) is used with a time-synchronized video feed. Any suitable video can be incorporated into the system to capture patient movement; however, near-infrared (NIR) video capture helps protect the patient's privacy and reduces the amount of video data to be processed. The NIR video capture device captures NIR video images of the patient's body, such as the position of the patient's torso and limbs. Further, the NIR video capture device captures the patient's real-time dynamic gait characteristics according to the music therapy working phase. In some embodiments, the video is captured with a fixed camera, where the background is subtracted to segment out the foreground pixels.

[0049] like Figure 4 As shown, one or more cameras 206 are triggered by a tablet or laptop application at the start of a therapy session. The therapist can stop or start the cameras 206 using a handheld wireless trigger unit 250. This allows for the creation of a time-stamped index of the captured biomechanical sensor data and video data stream.

[0050] In some embodiments, a wearable wireless real-time electromyography (EMG) device 208 can be worn by the patient. The EMG sensor provides a full bipedal profile of the major muscles activated for movement. Such sensors provide data on the exact time when the muscles are activated.

[0051] Edge processing

[0052] In some embodiments, edge processing is performed at the sensor 200, where sensor data is captured from the IMU and pressure sensor. This sensor data is filtered, grouped into various array sizes for further processing into frames reflecting extracted attributes and features, and these frames are wirelessly transmitted to the collector 106 on, for example, a tablet or laptop computer. It should be understood that the raw biomechanical sensor data obtained from the sensor 200 can alternatively be transmitted to a remote processor for collection for edge processing functions.

[0053] The wearable sensors 200, 208 and the video capture device 206 generate sensor data streams that are processed holistically to facilitate biomechanical feature extraction and classification. Sensor fusion, which combines the outputs from multiple sensors capturing a common event, can capture results better than any single component sensor input.

[0054] Capturing patient activities in the context of music therapy formalizes interactions when applied to music therapy and in developing patient-specific and generalized formal metrics of music therapy performance and efficacy. Extracting video features and then analyzing them allows for capturing semantic, high-level information about patient behavior.

[0055] When processing the video, learned background subtraction techniques are used to create a background model that incorporates any changes in lighting conditions and occlusions in the physical area where the music therapy is taking place. The result of background subtraction is a binary foreground map with an array of foreground blobs, which are two-dimensional outlines. The video is then cut into individual image frames for future image processing and sensor fusion. By merging edge-processed sensor data from the IMU, foot pressure pads, and EMG sensors, the video information is provided along with additional metadata. The sensor data can be time-synchronized with other data using an RF trigger. The data can be sent directly to the collector, stored in internal board memory, or analyzed on the edge running the OpenCV library.

[0056] The edge processor 104 may be a microprocessor, such as a 32-bit microprocessor incorporated into a foot pressure / 6-DOF motion capture device, which enables rapid multi-zone scanning at a rate of 100 to 400 complete foot pressure / 6-DOF motion profiles per second.

[0057] The foot pressure / 6-DOF motion capture device collects foot pressure / 6-DOF motion curve data for real-time gait analysis, thereby performing feature extraction and classification. In some embodiments, the foot pressure / 6-DOF motion capture device initializes the microcontroller unit (MCU), continuous operation process (COP), general input and output (GPIO), serial peripheral interface (SPI), interrupt request (IRQ) and sets the desired RF transceiver clock frequency by calling routines including microcontroller unit initialization (MCUInit), general input and output initialization (GPIOInit), serial peripheral interface initialization (SPIInit), interrupt request acknowledgement initialization (IRQInit), interrupt request acknowledgement (IRQACK), serial peripheral interface driver read (SPIDrvRead), and IRQPinEnable. MCUInit is the main initialization routine. When pre-scaled to 32, the main initialization routine disables the MCU watchdog and sets the timer module to use the bus clock (BUSCLK) as a reference.

[0058] The state variable gu8RTxMode is set to SYSTEM_RESET_MODE and the routines GPIOBinit, SPIInit, and IRQInit are called. The state variable gu8RTxMode is set to RF_TRANSCEIVER_RESET_MODE and the IRQFLAG is checked to see if the IRQ is asserted. The RF transceiver interrupt is first cleared using SPIDrvRead, and then the RF transceiver is checked for an ATTN IRQ interrupt. Finally, for MCUInit, PLMEPhyReset is called to reset the physical MAC layer, IRQACK is called (to acknowledge the pending IRQ interrupt), and IRQPinEnable is called to set the pin, enable, IE, and IRQ CLR on the negative edge of the signal.

[0059] The foot pressure / 6DOF motion sensor 200 will wait for a response from the foot pressure / 6DOF motion collection node, for example, 250 milliseconds, to determine whether to perform a default full foot pressure scan or initiate a mapped foot pressure scan. In the case of a mapped foot pressure scan, the foot pressure / 6DOF motion collection node will send foot pressure scan mapping configuration data to the appropriate electrodes.

[0060] One aspect of the analysis pipeline is the feature set engineering process, which defines the captured sensor values ​​and the resulting sensor fusion values ​​that are used to create feature vectors to define the input data structure for the analysis. Representative values ​​are Ax(i), Ay(i), Az(i), and Ph(i), where i is the i-th sample, where Ax(i) is the acceleration in the x-direction, which is lateral relative to the foot sensor; Ay(i) is the acceleration in the y-direction, which is forward relative to the foot sensor; Az(i) is the acceleration in the z-direction, which is upward relative to the foot sensor; and Ph(i) is the heel strike pressure. The sensor values ​​are presented in Table 1:

[0061]

[0062] In some embodiments, sensor fusion techniques use heel strike pressure values ​​Ph(i) to "gate" the analysis of the following exemplary feature values ​​to derive a data window as described below. For example, a "start" (begin) can be determined based on the heel pressure exceeding a threshold indicating heel strike, and a "stop" can be determined based on the heel pressure falling below a threshold indicating heel lift-off as presented in Table 2 below. It should be understood that heel strike pressure is an example of a parameter that can be used for "gating" the analysis. In some embodiments, "gating" is determined using IMU sensor data, video data, and / or EMG data.

[0063]

[0064] Higher-level feature values ​​are calculated based on the fused sensor values, as shown in Table 3:

[0065]

[0066] The system described herein provides the ability to "gate" or provide "windows" into patient biomechanical data. Gating of biomechanical data is useful for repetitive patient movements, such as the repetitive stride of a patient walking. Sensor data from one or more sources, such as pressure sensors, IMU sensors, video data, and EMG data, is used to identify movement cycles that repeat over time. For example, as a patient walks, foot pressure repeatedly increases and decreases as the patient's foot contacts the ground and then lifts off the ground. Similarly, foot velocity increases as the foot moves forward and decreases to zero when the foot lands. As another example, the Y position or height of a patient's foot cycles between a low position (on the ground) and a high position (approximately in the middle of the stride). "Gating" techniques identify repetitive cycles or "windows" in such data. When a patient walks, the cycle repeats with each step. Although there may be variations between cycles (e.g., between steps), certain patterns repeat with each cycle. Selecting the start time (start time) for each cycle involves locating an identifiable point (maximum or minimum) for a biomechanical parameter. Parameter selection for selecting the start time is based on available data. Therefore, in some embodiments, the moment when the heel strike pressure exceeds the threshold can be used to divide the start time of each cycle. (See, for example, Figure 5 Pressures 316a and 316b contain cyclic characteristics. "Start" can be determined at the moment the pressure exceeds a threshold.) Similarly, the start time can be divided when the foot speed drops to zero.

[0067] In some embodiments, the raw frame data is pre-processed, and the instantaneous data is acquired and "gated", for example, a window is identified, and the data within the window is then analyzed to identify outliers and perform analysis on the data, such as index analysis, averaging the data in multiple windows. Compared to using data from a single sensor, sensor data fusion by including IMU data and heel strike pressure data allows more accurate identification of the start time of a single stride or other repetitive motion unit. The sensor data captured within a single stride is considered a "window", and the information extracted from this analysis includes, for example, stride length, number of steps, rhythm, time of footfall, distance traveled, stance phase / swing phase, double support time, speed, symmetry analysis (for example, between the left leg and the right leg), outward swing, dragging, power vector, lateral acceleration, step width, variability of each of these dimensions, other parameters derived from the information described above, etc. Feature extraction can be processed on a microprocessor chip, for example, a 32-bit chip. Capturing wireless synchronous gated biomechanical sensor data and video data capture functionality allows the creation of time series templates.

[0068] During a music therapy session, the patient or therapist can index the data. The "gating" function described above is useful for associating abnormal conditions with specific strides or steps. For example, the therapist can observe specific abnormal conditions or behaviors (such as anomalies or events) in the patient's movements. The indexing function allows the therapist to access the patient's movements through a user interface (such as a handheld tablet or laptop) on the patient's computer. Figure 4 Abnormal conditions or behaviors can be initiated (e.g., captured to "record") using the wireless trigger unit 250 shown in the figure) or voice control. Symbols containing timestamps and annotations can be created, such as "stumble" when the patient was walking. Such indexing facilitates time series template creation. These time series templates will be studied to examine therapy session events and develop time series templates for training machine learning algorithms such as nonlinear multilayer perceptrons (NLMLPs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs) with long short-term memory (LSTMs).

[0069] In one embodiment, a communication protocol is provided to transmit sensor data from edge processing 104 (eg, at sensor 200) to collector 106. See Table 4 below. In some embodiments, if the connection is idle for more than 100 milliseconds, the RF has timed out.

[0070]

[0071] In one embodiment, the foot pressure sensor zone scan is performed by the FootScan routine, in which the FootDataBufferIndex is initialized and the foot pressure sensor zone is activated by enabling the MCU direction mode of output [PTCDD_PTCDDN=Output] and lowering the associated port line [PTCD_PTCD6=0]. Since the foot pressure sensor zone is activated based on the foot pressure sensor zone scan map, the foot pressure sensor zone attached to the MCU analog signal port will be sampled, and then the current and voltage readings will be converted to digital form (i.e., time zone foot pressure).

[0072] Several variables, such as FootDataBufferIndex and IMUBufferIndex, are used to prepare the IEEE 802.15.4 RF packet gsTxPacket.gau8TxDataBuffer[] for sending the data used in FootDataBuffer[] and IMUBuffer[]. The RF packet is sent using the RFSendRequest(&gsTxPacket) routine. This routine checks if gu8RTxMode is set to IDLE_MODE and calls the RAMDrvWriteTx routine using gsTxPacket as a pointer. The RAMDrvWriteTx routine then calls SPIDrvRead to read the contents of the RF transceiver's TX packet length register. Using these contents, the mask length setting is updated, and then 2 is added for the CRC and 2 for the code byte.

[0073] Call SPISendChar to send the second code byte, 0x7E, and then call SPIWaitTransferDone again to verify that the transmission is complete. After sending these code bytes, use a for loop to send the rest of the packet, where psTxPkt→u8DataLength+1 is the number of iterations through the series of sequential SPISendChar, SPIWaitTransferDone, and SPIClearRecieveDataReg. Upon completion, the RF transceiver loads the packet to be sent. Set ANTENNA_SWITCH to transmit, enable LNA_ON mode, and finally call RTXENAssert to actually send the packet.

[0074] Collector

[0075] The primary function of the collector 106 is to capture data from the edge processing 104, transmit the data to the analysis system 108, receive processed data from the analysis system, and transmit the data to the music therapy center 110, as described below. In some embodiments, the collector 106 provides control functions, such as a user interface for logging in, configuring the system, and interacting with the user, and includes a display unit for visualizing / displaying the data. The collector 106 may include lightweight analysis or machine learning algorithms for classification (e.g., lateral tremor, asymmetry, instability, etc.).

[0076] The collector 106 receives body, motion, and positioning data from the edge processor 104. The data received at the collector 106 may be raw or processed at the edge 104 before being transmitted to the collector. For example, the collector 106 receives fused sensor data, subjected to "windowing" and feature extraction. The transmitted data may contain two levels of data: (1) RF packets sent from the right / left foot sensors as described in Table 1; (2) RF packets from the right / left foot sensors containing higher-level attributes and features as described in Tables 2 and 3. The collector 106 stores the data locally. In some embodiments, the collector 106 classifies the movement based on the received data, for example, by comparing the received data with a model stored locally (pre-downloaded from the analysis system) or sent to the analysis system for classification. The collector may include a display unit to visualize / display the data.

[0077] In some embodiments, the collector 106 operates on a local computer including a memory, a processor, and a display. Exemplary devices on which the collector is installed may include AR devices, VR devices, tablet computers, mobile devices, laptop computers, desktop computers, etc. Figure 2 A handheld device 220 is shown having a display 222 and performing collector functions. In some embodiments, the connection parameters for transmitting data between the patient sensors and the collector include using the device manager in Windows (e.g., baud rate: 38400, data bits: 8, parity: none, stop bits: 1). In some embodiments, the collector 106 comprises a processor held or worn by the music therapy patient. In some embodiments, the collector 106 comprises a processor that is remote from the music therapy patient and carried by the therapist and connected to the music therapy patient wirelessly or via a wired connection.

[0078] In one embodiment, the foot pressure / 6-DOF motion collection node captures RF-transmitted data packets containing real-time foot pressure / 6-DOF motion curve data from a foot pressure / 6-DOF motion capture device. This is initiated by the foot pressure / 6-DOF motion collection node, which creates an RF packet receive queue driven by a callback function of the RF transceiver packet receive interrupt.

[0079] When an RF packet is received from the foot pressure / 6DOF motion capture device 200, a check is first performed to determine if this is from a new foot pressure / 6DOF motion capture device or an existing foot pressure / 6DOF motion capture device. If this is from an existing foot pressure / 6DOF motion capture device, the RF packet sequence number is checked to determine continuous synchronization before further analysis of the packet. If this is from a foot pressure capture / 6DOF motion capture device, a foot pressure / 6DOF motion capture device context state block is created and initialized. The context state block contains information such as the foot pressure curve, [what additional information?]

[0080] Above this node-to-node communication RF packet session level process is the analysis of the RF packet data payload. This payload contains the foot pressure profile based on the current variable pressure after 6-DOF motion. This is structured as follows: |0×10|Start|F1|F2|F3|F4|Ax|Ay|Az|Pi|Yi|Ri|XOR Checksum|.

[0081] The IEEE 802.15.4 standard specifies a maximum packet size of 127 bytes, with the Time Synchronization Mesh Protocol (TSMP) reserving 47 bytes for operation, leaving 80 bytes for payload. IEEE 802.15.4 complies with 2.4 GHz Industrial, Scientific, and Medical (ISM) band radio frequency (RF) transceivers.

[0082] The RF module contains a complete 802.15.4 physical layer (PHY) modem designed to support the IEEE 802.15.4 wireless standard for peer-to-peer, star, and mesh networks. The modem combines with the MCU to create the required wireless RF data link and network. The IEEE 802.15.4 transceiver supports 250kbps O-QPSK data and full spread spectrum encoding and decoding in a 5.0MHz channel.

[0083] In some embodiments, control, status reads, data writes, and data reads are performed via the RF transceiver interface port of the sensing system node device. The sensing system node device's MPU accesses the sensing system node device's RF transceiver via interface "transactions," where multiple-byte bursts of data are transmitted on the interface bus. Each transaction is three or more bursts long, depending on the transaction type. A transaction is always a read or write access to a register address. The data associated with any single register access is always 16 bits long.

[0084] In some embodiments, control and data transfer of the RF transceiver of the foot pressure / 6-DOF motion collection node is implemented via a serial peripheral interface (SPI). While the normal SPI protocol is based on 8-bit transfers, the RF transceiver of the foot pressure / 6-DOF motion collection node uses a higher-level transaction protocol based on multiple 8-bit transfers per transaction. A single SPI read or write transaction consists of an 8-bit header transfer followed by two 8-bit data transfers.

[0085] The header indicates the access type and register address. The following bytes are the read or write data. SPI also supports recursive "data burst" transactions, in which additional data transfers may occur. This recursive mode is primarily used for packet RAM access and fast RF configuration of foot pressure / 6DOF motion collection nodes.

[0086] In some embodiments, all foot pressure sensor zones are scanned sequentially and the entire process is repeated until a reset condition or inactive power-down mode. 6-DOF motion is captured via a serial UART interface to the inertial measurement unit (IMU) from the MCU. The sampling rate for all sensing dimensions, i.e., Ax, Ay, Az, pitch, yaw, and roll, is 100-300 Hz, and the sampled data is stored in IMUBuffer[].

[0087] Call SPIDryWrite to update the TX packet length field. Next, call SPIClearRecieveStatReg to clear the status register, and then call SPIClearRecieveDataReg to clear the receive data register to prepare the SPI interface for reading or writing. After the SPI interface is ready, call SPISendChar to send the 0xFF character representing the first code byte, and then call SPIWaitTransferDone to verify that the transmission is complete.

[0088] Figure 5 is an exemplary output 300 that may be provided on the display 222 of the handheld device. For example, when providing therapy for a patient's gait, the display output 300 may include a portion 302 for the right foot and a portion 304 for the left foot. The display for the right foot may include acceleration A as a function of time. x 310a、A y 312a and A z 314a and foot pressure 316a. Similarly, the display for the left foot includes acceleration A x 310a、A y 312a and A z 314a and foot pressure 316a.

[0089] Classification is understood as the correlation of data, such as sensor fusion data, feature data, or attribute data, with real-world events, such as a patient's activity or temperament. Typically, classification is created and executed on the analysis system 108. In some embodiments, the collector 106 has a local copy of some "templates." Thus, incoming sensor data and feature extraction data can be classified at the collector or analysis system.

[0090] Context refers to the circumstances or facts that form the background of an event, statement, situation, or idea. Context-aware algorithms examine the "who," "what," "when," and "where" associated with the environment and timing of the algorithm's execution on certain data. Some context-aware actions incorporate identity, location, time, and the activity being performed. When contextual information is used to formulate deterministic actions, contextual interfacing occurs between the patient, the environment, and the music therapy session.

[0091] The context of a patient's response to a music therapy session can be incorporated into a layer of algorithms that interpret the fused sensor data to infer higher-level information. These algorithms extract the context of the patient's response. For example, the patient's biomechanical gait sequence is analyzed because it is associated with a specific part of the music therapy session. In one example, "lateral tremor" is the classifier of interest. Therefore, it is determined that the patient's gait has become smoother and the lateral tremor has decreased.

[0092] Analysis System

[0093] The analysis system 108, sometimes referred to as the backend system, stores the large model / archive and includes the machine learning / analytics processing that applies the models described herein. In some embodiments, a web interface and dashboard for logging in to view the archived data are also provided. In some embodiments, the analysis system 108 resides on a remote server computer that receives data from the collector 106 running on a handheld unit such as a handheld device or tablet computer 220. It is contemplated that the processing power required to perform the analysis and machine learning functions of the analysis system 108 can also be located on the handheld device 220.

[0094] The data is transmitted from the collector 106 to the analysis system 108 for analysis and processing. Figure 6As shown, analysis process 400 includes a user interface 402 for receiving data from collector 106. Database memory 404 receives input data from collector 106 for storage. Training data and outputs of the analysis process, such as the integrated machine learning system 410, can also be stored in memory 404 to facilitate the creation and refinement of predictive models and classifiers. A data bus 406 allows data to flow through the analysis process. A training process 408 is performed on the training data to derive one or more predictive models. The integrated machine learning system 410 utilizes the predictive models. The output of the integrated machine learning system 410 is an aggregate of these predictive models. This aggregate output is also used by template classifier 412 to classify requirements such as tremor, symmetry, mobility, or learned biomechanical parameters such as entrainment, activation, etc. An API 418 connects to the collector and / or music therapy center. Therapy algorithms 414 and prediction algorithms 416 include multilayer perceptron neural networks, hidden Markov models, Radial-based function networks, Bayesian inference models, and the like.

[0095] An exemplary application of the systems and methods described herein is the analysis of a patient's biomechanical gait. The gait sequence is feature-extracted into a series of characteristic features. The presence of these and other features in the captured sensor fusion data informs a context detection algorithm if the patient's biomechanical gait sequence is valid. Biomechanical gait sequence capture requires robust context detection, which is then abstracted across a representative population of music therapy patients.

[0096] Examples of such activity include the patient's location at a particular moment in time and the patient's response to music therapy at that moment. Identifying and correlating patient responses to music therapy allows for the identification of specific patterns in patient responses to music therapy. This allows for benchmarking and analysis of specific music therapy programs to understand their performance and efficacy by creating a baseline of patient responses to music therapy and correlating these responses with those of future music therapy patients.

[0097] In conjunction with motion sensing, distance metrics with gait biomechanical capture are used to determine patient path trajectories using temporal and spatial variations / deviations between two or more music therapy sessions. Features are extracted from this sensor-fused data capture and classified to identify key patient therapy responses. Additional sensor-fused data analysis uses histograms to enable initial music therapy response pattern detection.

[0098] For sensor fusion data analysis during therapy sessions, a patient-specific Bayesian inference model can be initially employed using a Markov chain. The chain's states represent patient-specific response patterns captured during the baseline music therapy session. Inference is based on knowledge of the patient's response pattern at each sample interval and its temporal linkage to previous states.

[0099] The prediction routine, a multilayer perceptron neural network (MLPNN), uses a directed graph node-based model with a top-level root node that predicts the requirements for reaching subsequent nodes and obtaining the patient's sensor-fused data feature vector. This sensor-fused data feature vector contains time-series processed motion data, music signature data, and video image data, which are particularly important for further processing. In this case, the directed graph looks like a tree drawn upside down, with the leaves at the bottom and the root node at the root. From each node, the routine can either go to the left, where the left node is the left node on the next level below the top level, where the root node is located, and select the left child node as the next observation node, or go to the right, where the right node is the right node on the next level below the top level, where the root node is located, based on the value of a variable whose index is stored in the observation node. If the value is less than a threshold, the routine goes to the left node, and if it is greater than the threshold, the routine goes to the right node. These regions, here, left and right, become the prediction space.

[0100] The model uses two types of input variables: ordinal variables and categorical variables. Ordinal variables are values ​​that are compared to thresholds, also stored in the nodes. Categorical variables are discrete values ​​that are tested to see if they belong to a finite subset of values ​​and stored in the nodes. This can be applied to various classifications. For example, mild, moderate, and severe can be used to describe tremors and are examples of categorical variables. Conversely, fine-grained value ranges or numerical scales can be used to similarly describe tremors, but numerically.

[0101] If the categorical variable belongs to a finite set of values, the routine goes to the left node, and if the categorical variable does not belong to a finite set of values, the routine goes to the right node. In each node, this decision is made using a pair of entities: variable_index, decision_rule (threshold / subset). This pair is called a split, and it splits on the variable variable_index.

[0102] Once a node is reached, the value assigned to it is used as the output of the prediction routine. Multilayer Perceptron neural networks are built recursively, starting at the root node. As previously mentioned, all training data, feature vectors, and responses are used to split the root node; the entities: variable_index, decision_rule (threshold / subset) split the prediction region. At each node, the best decision rule for the best primary split is found based on the Gini "purity" criterion for classification and the sum of squared errors for regression. The Gini index is based on a measurement of the total variance of a set of classes. A small Gini index value indicates that the node contains mostly observations from a single class, which is a desirable state.

[0103] Once the multilayer perceptron neural network is constructed, it can be pruned using a cross-validation routine. To prevent overfitting, some branches of the tree are cut off. This routine can be applied to independent decision-making. As described above, a significant property of the decision-making algorithm (MLPNN) is its ability to calculate the relative power and importance of each variable.

[0104] Variable importance rankings are used to identify the most frequent interaction types within a patient interaction feature vector. Pattern identification begins by defining a decision space suitable for distinguishing between different categories of music therapy responses and events. This decision space can be represented by a graph with N dimensions, where N is the number of attributes or measurements considered to represent music therapy responses and events. These N attributes form a feature vector, or signature, that can be plotted in a graph. After sufficient input samples, the decision space reveals clusters of music therapy responses and events belonging to different categories, which can be used to associate new vectors with these clusters.

[0105] The dynamic closed-loop rehabilitation platform music therapy system utilizes several deep learning neural networks to learn and recall patterns. In one embodiment, a nonlinear decision space is established using an adaptive radial basis function (RBF) model generator. New vectors can be calculated using the RBF model and / or a K-nearest neighbor classifier. Figure 6 The workflow of the machine learning subsystem of the music therapy system of the dynamic closed-loop rehabilitation platform is demonstrated.

[0106] Figure 7A supervised training process 408 is shown, which includes multiple training samples 502. For example, the input would be features such as those described in Table 3 above, and example outputs would be items such as tremor, asymmetry, and strength, the extent of these items, predictions of change, and classification of patient recovery. It should be understood that new outputs are learned as part of this process. This provides a basis for higher-level abstractions in prediction and classification as it applies to different use cases (e.g., different disease states, combinations with medications, notifications to providers, fitness, and fall prevention). These training samples 502 are run using learning algorithms A1 504a, A2 504b, A3 504c...AN 504n to derive predictive models in M1 506a, M2 506b, M3 506c...MN 506n. Exemplary algorithms include multilayer perceptron neural networks, hidden Markov models, Radial-based function networks, and Bayesian inference models.

[0107] Figure 8 An integrated machine learning system 410 is shown as an aggregation of prediction models M1 506a, M2 506b, M3 506c...MN506n on sample data 602 (e.g., feature extraction data) to provide a plurality of prediction result data 606a, 606b, 606b...606n. Given the plurality of prediction models, an aggregation layer 608, such as one including decision rules and voting, is used to derive an output 610.

[0108] The MR ConvNet system has two layers, the first of which is a convolutional layer with mean pooling support. The second layer of the MR ConvNet system is a fully connected layer that supports multinomial logistic regression. Multinomial logistic regression, also known as softmax, is a generalization of logistic regression to handle multiple classes. In the case of logistic regression, the labels are binary.

[0109] Softmax is a model for predicting the probability of different possible outputs. The following assumes a multi-class classifier with m discrete classes through the Softmax final output layer:

[0110] Y1=Softmax(W11*X1+W12*X2+W13*X3+B1) [1]

[0111] Y2=Softmax(W21*X1+W22*X2+W23*X3+B2) [2]

[0112] Y3=Softmax(W31*X1+W32*X2+W33*X3+B3) [3]

[0113] Ym=Softmax(Wm1*X1+Wm2*X2+Wm3*X3+Bm) [4]

[0114] Usually: Y=softmax(W*X+B) [5]

[0115] Softmax(X)i=exp(Xi) / j=the sum of exp(Xj) from 1 to N[6]

[0116] Where Y = classifier output; X = sample input (all scaled (normalized) eigenvalues); and W = weight matrix. The classification might, for example, score asymmetry as "moderate asymmetry score of 6 out of 10" (10 being highly asymmetric and 0 being no asymmetry) or gait fluidity as "gait fluidity score of 8 out of 10 being normal." Figure 9 The analysis pipeline is shown in .

[0117] Softmax regression allows handling more than two classes. For logistic regression: P(x) = 1 / (1+exp(-Wx)), where W contains the model parameters trained to minimize the cost function. In addition, x is the input feature vector, and

[0118] ((x(1),y(1)),...,(x(i),y(i))) [7]

[0119] Let y(n) represent the training set. For multi-class classification, Softmax regression is used, where y can take N different values ​​representing the class, rather than 1 and 0 as in the binary case. So for the training set ((x(1), y(1)),...,(x(i), y(i))), y(n) can be any value in the range of 1 to N classes.

[0120] Next, p(y=N|x;W) is the probability of each value of i=1,...,N. The following mathematically illustrates the Softmax regression process:

[0121] Y(x)=(p(y=1|x;W),p(y=2|x;W),...p(y=N|x;W)) [8]

[0122] Where Y(x) is the answer to the hypothesis, that is, given an input x, output the probability distribution of all classes so that its normalized sum is 1.

[0123] The MR ConvNet system convolves each windowed biomechanical data frame (as a vector) with each biomechanical template filter (as a vector) and then generates a response using a mean pooling function that averages the feature responses. The convolution process calculates Wx while adding any bias and then passes it to a logistic regression (sigmoid) function.

[0124] Next, in the second layer of the MR ConvNet system, the subsampled biomechanical template filter responses are moved into a two-dimensional matrix, where each column represents the windowed biomechanical data frame as a vector. The Softmax regression activation process is now initiated using:

[0125] Y(x)=(1 / (exp(Wx)+exp(Wx)+...+exp(Wx))*(exp(Wx),exp(Wx),...,(exp(Wx))[9]

[0126] The MR ConvNet system is trained using the optimization algorithm gradient descent, where the cost function J(W) is defined and is to be minimized:

[0127] J(W)=1 / j*((H(t(j=1),p(y=1|x;W)+H(t(j=2),p(y=2|x;W)+...+H(t(j),p(y=N|x;W)) 10]

[0128] Where t(j) is the target class. This averages all cross entropies over j training examples. The cross entropy function is:

[0129] H(t(j),p(y=N|x;W)=-t(j=1)*log(p(y=1|x;W))+t(j=2)*log(p(y=2|x;W))+...+t(j)*p(y=N|x;W)

[11]

[0130] exist Figure 10 In

[15] , the integrated machine learning system 408 comprises multiple predictive models, e.g., Template Series 1 (tremor) 706a, Template Series 2 (symmetry) 706b, Template Series 3 (fluidity) 706c, ... and additional templates (other learned biomechanical parameters, e.g., entrainment, activation, etc.) 706n, which are applied to conditional inputs 702, which can be, for example, stride length right and left features (x1, x2), variance of stride length right and left features (x3, x4), rhythm right and left features (x6, x7), variance of rhythm right and left features (x8, x9), etc. ... This is where the samples (x1, x2, ... xn) are referred to as vectors X, which are input to 702 in the ensemble of ML algorithms. These are the inputs that the conditions refer to, normalized and / or scaled. The aggregate classifier 708 outputs information such as a tremor scale, a symmetry scale, a fluidity scale, etc.

[0131] Music Therapy Center

[0132] The music therapy center 110 is a processor (such as Figure 2The music therapy center 110 takes input from the feature-extracted sensor data at the collector 106, compares the input to the defined therapy delivery process, and then delivers auditory stimulation content played through the music delivery system 230.

[0133] Embodiments of the present invention use contextual information to determine why a situation occurred and then encode the observed actions, which in a closed-loop manner results in dynamic and modulated changes in the system state, and therefore in dynamic and modulated changes in the music therapy session.

[0134] The interaction between the patient and the music therapy session provides real-time data for determining the patient's contextual awareness of the music therapy session, including movement, posture, stride length, and gait response. After the sensing nodes (at the sensors) collect the input data, the embedded nodes (at the edge processing) process the contextually aware data and provide immediate dynamic actions and / or transmit the data to the analysis system 108, such as an elastic network-based processing cloud environment for storage and further processing and analysis.

[0135] Based on the input, the program will take any existing song content and change the tempo, major / minor chords, rhythm, and musical cues (e.g., melodic cues, harmony cues, rhythmic cues, and dynamic cues). The system can overlay a metronome on an existing song. The song content can be beat-mapped (e.g., if W responds to an AV or MP3 file) or in MIDI format so that entrainment potentials can be calculated using precise knowledge of when the beats occur. Sensors on the patient can be configured to provide tactile / vibratory feedback on the music content.

[0136] Examples

[0137] This article describes an exemplary application of the method. Gait training analysis is a real-time relationship between the beats of music played to the patient and the individual steps taken by the patient in response to these specific musical beats. As discussed above, gated analysis is used to determine a data window that repeats with each step or repeated movement, but with some variations. In some embodiments, the start of the window is determined as the time when the heel strike pressure exceeds a threshold (or other sensor parameter). Figure 11 is an exemplary time diagram showing the corresponding output times (also referred to as beat times) of a musical beat ("time beat") and a patient's step ("time step"). Thus, the start time in this case is associated with a "time step." Specifically, the diagram shows a time beat 1101 of the music at time beat 1. Some time later, the patient takes a step at time step 1 in response to time beat 1001, i.e., time step 1102. The entrainment potential 1103 indicates the delay, if any, between time beat 1 and time step 1.

[0138] Figure 12-13 An example of entrainment of a patient's gait using the system described herein is demonstrated. Figure 12 "Perfect" entrainment is demonstrated, i.e., a constant entrainment potential of zero. This occurs when there is no or negligible delay between the time tick and the associated time step taken in response to the time tick. Figure 13 Phase-shifted entrainment is demonstrated, i.e., a condition where the entrainment potential is not zero but remains constant or has minimal variation over time. This occurs when there is a consistent delay between the time tick and the time step within an acceptable tolerance.

[0139] Continue to refer Figure 11 , the EP ratio is calculated as the ratio of the duration between time ticks to the duration between time steps:

[0140]

[0141] Wherein time beat 1 1101 corresponds to the time of the first music beat, and time step 1 1102 corresponds to the time of the patient's footstep in response to time beat 1. Time beat 2 1106 corresponds to the time of the second music beat, and time step 2 1108 corresponds to the time of the patient's footstep in response to time beat 2. The goal is EP ratio = 1 or EP ratio / factor = 1. The factor is determined as follows:

[0142]

[0143] This factor allows for subdividing the beat, or having someone take a step every 3rd or 3rd beat of every 4th beat. It provides flexibility for different scenarios.

[0144] Figure 14 and 15 The entrainment response of patients over time using the techniques described herein is shown. Figure 14 (Left Y-axis: EP ratio; Right Y-axis: beats per minute; X-axis: time) shows a scatter of points 1402 representing the average of the EP ratios for the first patient's gait. The graph shows an upper limit of +0.1 1404 and a lower limit of -0.1 1406. Line 1408 shows the tempo over time (starting at 60 beats per minute and increasing gradually to 100 bpm). Figure 14 It shows that the EP ratio remains near 1 (±0.1) as the tempo increases from 60 bpm to 100 bpm. Figure 15 The EP ratio of the second patient's gait is shown, where the EP ratio also remains near 1 (±0.1) as the cadence increases from 60 bpm to 100 bpm.

[0145] Figure 16 and 17 (Y-axis: entrainment potential; X-axis: time) Shows the responses of two patients to changes in tempo (e.g., rhythm) and / or chord, changes in tactile feedback, changes in foot cues (e.g., left-right, left-right cane cues), etc. Figure 16 A time-based graph is shown where the patient's gait is balanced with either "perfect entrainment" (constant zero or negligible entrainment potential) or a constant phase-shifted entrainment potential. As illustrated in the figure, a certain period of time, the golden time 1602, is required until equilibrium occurs. Figure 17 A time-based plot is presented where the patient's gait is unbalanced, for example, not achieving perfect entrainment or a constant phase-shifted entrainment potential after a tempo change. Golden time is useful because it represents a dataset separate from the measurement of entrainment accuracy. The golden time parameter can also be used to screen the suitability of future songs. For example, when a patient exhibits a long golden time value when using a piece of music, the piece is less suitable for therapy.

[0146] Figure 18 The invention discloses a technique useful for gait training, in which repetitive movement refers to the steps taken by the patient when walking. Gait training is adapted to individual patient populations, diagnoses and conditions to provide personalized and individualized music intervention. Based on the input, the program changes the content, rhythm, major / minor chords, rhythm and music prompts (e.g., melody prompts, harmony prompts and dynamic prompts) where applicable. The program can select music by using date of birth, listed music preferences and entrainment rhythm to provide a passive music playlist for regular use. The key input for gait training is the rhythm, symmetry and stride length of the user performing physical activity (e.g., walking). The program uses connected hardware to provide tactile / vibration feedback with the BPM of the music. Suitable groups for gait training include patients with traumatic brain injury (TBI), stroke, Parkinson's disease (PD), multiple sclerosis (MS), functional neurological disorders (FND) and aging patients.

[0147] The method begins at step 1802. At step 1804, biomechanical data is received at the collector 106 based on data from sensors, such as sensors 200, 206, and 208. The biomechanical data may include actuation, stride length, rhythm, symmetry, data regarding assistive devices, or other such patient feature sets stored and generated by the analysis system 108. Exemplary biomechanical data parameters are listed in Tables 1, 2, and 3 above. Baseline conditions are determined from one or more data sources. First, the patient's gait is sensed without any music playing. Sensor and feature data regarding the patient's actuation, stride length, rhythm, symmetry, data regarding assistive devices, and the like comprise baseline biomechanical data for the patient's therapy session. Second, sensor data from previous sessions of the same patient, along with any higher-level categorized data from the analysis system 108, comprise historical data for the patient. Third, sensor data and higher-level categorized data from other similarly situated patients comprise population data. Therefore, the baseline condition can include data from one or more of the following: (a) baseline biomechanical data of the patient's therapy work phase; (b) data from the patient's previous work phase; and (c) group data. A baseline beat rhythm is then selected from the baseline condition. For example, the baseline beat rhythm can be selected before playing the music to match the patient's current rhythm. Alternatively, the baseline beat rhythm can be selected as a fraction or multiple of the patient's current rhythm. As another alternative, the baseline beat rhythm can be selected to match the baseline beat rhythm used in the same patient's previous work phase. As yet another alternative, the baseline beat rhythm can be selected based on the baseline beat rhythm used for other patients with similar physical conditions. Finally, the baseline beat rhythm can be selected based on a combination of any of the data described above. The target beat rhythm can also be determined based on this data. For example, the target beat rhythm can be selected as a percentage increase in the baseline beat rhythm with reference to the improvements shown by other patients in similar situations. Rhythm is understood to refer to the frequency of beats in music.

[0148] At step 1806, music is provided to the patient at a baseline tempo or a subdivision of the baseline tempo from the handheld device 220 on the music delivery device 230 (e.g., earbuds or headphones, or speakers). To provide music to the patient at the baseline tempo, music with a constant baseline tempo is selected from a database, or existing music is modified, e.g., selectively sped up or slowed down, to provide a beat signal at a constant tempo.

[0149] At step 1808, the patient is instructed to listen to the beat of the music. At step 1810, the patient is instructed to walk at a baseline tempo, optionally receiving cues regarding left and right foot movement. The patient is instructed to walk so that each step closely matches the beat of the music, e.g., walking "in time" to the tempo. Steps 1806, 1808, and 1810 can be initiated by the therapist or by auditory or visual instructions on the handheld device 220.

[0150] At step 1812, sensors 200, 206, and 208 on the patient are used to record patient data, such as heel strike pressure, 6-dimensional movement, EMG activity, and a video recording of the patient's movement. All sensor data is time-stamped. Data analysis is performed on the time-stamped sensor data, including the "gated" analysis discussed herein. For example, sensor data, such as heel strike pressure, is analyzed to determine the start time of each footstep. Additional data received includes the time associated with each beat signal of the music provided to the patient.

[0151] At step 1814, a connection is made to the entrained model (e.g., the integrated machine learning system 410 of the analysis system 108 or a model downloaded on the collector 106 and running on the handheld device 220) for prediction and classification. (It should be understood that such a connection may be pre-existing or initiated at this time.) Such a connection is typically very fast or instantaneous.

[0152] At step 1816, an optional entrainment analysis performed at analysis system 108 is applied to the sensor data. Entrainment analysis involves determining the delay between the tempo signal and the onset of each step taken by the patient. As output from the entrainment analysis, a determination is made regarding the accuracy of entrainment, for example, a measurement of the instantaneous relationship between the baseline tempo and the patient's footsteps, as described above, regarding the entrainment potential and EP ratio. If entrainment is inaccurate, for example, the entrainment potential is not constant within tolerance, adjustments are made at step 1818, such as speeding up or slowing down the tempo, increasing the volume, increasing sensory input, superimposing a metronome or other relevant sound, etc. If entrainment is accurate, for example, the entrainment potential is constant within tolerance, an incremental change is made to the tempo at step 1820. For example, the baseline tempo of the music played on the handheld device is increased toward the target tempo, for example, by 5%.

[0153] At step 1822, a connection is made to the entrainment model for prediction and classification. (It should be understood that such a connection may be pre-existing or initiated at this time.) At step 1824, an optional symmetry analysis is applied to the sensor data. As an output of the symmetry analysis, a determination is made regarding the symmetry of the patient's gait, e.g., how closely the patient's left foot movement matches the patient's right foot movement in terms of stride length, velocity, stance phase, swing phase, etc. If the footsteps are asymmetrical, e.g., below a threshold, then at step 1826, the music played to the patient by the handheld device is adjusted. A first modification may be made to the music played during the movement of one of the patient's feet, and a second modification may be made to the music played during the movement of the other foot. For example, a minor chord (or increased volume, sensory input, tempo change, or overlay of sound / metronome) may be played on one side (e.g., the affected side), and a major chord may be played on the other side (e.g., the unaffected side). Based on a "fingerprint" of the scenario leading to the symmetry issue, e.g., by analyzing movements indicative of asymmetry, the machine learning system 410 can predict in advance when symmetry issues will arise. Asymmetry can be determined by comparing someone's normal gait parameters to their background, and how much one side is affected can be determined and compared to the other side.

[0154] At step 1828, a connection is made to the entrainment model for prediction and classification. (It should be understood that such a connection may be pre-existing or initiated at this time.) At step 1830, an optional center of balance analysis is performed on the sensor data, for example, to determine whether the patient is leaning forward. This analysis can be performed by combining the output of the foot sensors with the video output. As an output from the center of balance analysis, a determination is made as to whether the patient is leaning forward. If the patient is leaning forward, a prompt to the patient to "stand up straight" is provided at step 1832, either by the therapist or through an audible or visual instruction on a handheld device.

[0155] At step 1834, a connection is made to the entrainment model for prediction and classification. (It should be understood that such a connection may be pre-existing or initiated at this time.) At step 1836, a start-up analysis is applied to the sensor data, e.g., if the patient exhibits hesitation or difficulty initiating walking. As an output from the start-up analysis, a determination is made as to whether the patient exhibits start-up issues. If the patient exhibits start-up issues, e.g., below a threshold, tactile feedback may be provided to the patient, which may include a countdown to the beat rhythm or a countdown before the song begins, at step 1838.

[0156] At step 1840, it is optionally determined whether the patient is using an assistive device, such as a cane, crutches, walker, etc. In some embodiments, the handheld device 220 provides a user interface for the patient or therapist to input information regarding the use of the assistive device. If a cane is present, the analysis is changed to a three-beat pattern, such as cane, right foot, left foot, and prompts for "left foot," "right foot," and "cane" are provided at step 1842 by the therapist or via audible or visual instructions on the handheld device.

[0157] At step 1844, a connection is made to the entrainment model for prediction and classification. (It should be understood that such a connection may be pre-existing or initiated at this time.) An optional entrainment analysis 1846 is applied to the sensor data, substantially as described above in step 1816, with the differences noted herein. For example, entrainment may be compared to previous entrainment data from an earlier session, from a previous session of the patient, or compared to data related to entrainment for other patients. As an output from the entrainment analysis, a determination is made regarding the accuracy of the entrainment, e.g., how closely the patient's gait adheres to the baseline cadence. If the entrainment is inaccurate, adjustments are made at step 1848, substantially in the same manner as described above in step 1818.

[0158] If the entrainment is accurate, a determination is made at step 1850 as to whether the patient is walking at the target cadence. If the target cadence is not achieved, the method proceeds to step 1820 (as described above) so that an incremental change is made to the cadence. For example, the baseline cadence of the music played on the handheld device is increased or decreased, e.g., 5%, toward the target cadence. If the target cadence has been achieved, the patient can continue the therapy for the remainder of the session (step 1852). At step 1854, music at the desired cadence that is not used in the therapy session can be curated and retained. Figure 2 The music content is used as homework / exercise by the patient between specialized therapy sessions. At step 827, the program ends.

[0159] It should be understood that the above description and Figure 18 The steps shown in the figure may be performed in a different order than that disclosed. For example, the evaluations at steps 1816, 1824, 1830, 1836, 1840, 1846, and 1850 may be performed simultaneously. In addition, multiple connections to the analysis system 108 (e.g., steps 1814, 1822, 1828, 1834, and 1844) may be performed once during the entire therapy session described.

[0160] Figure 19Technology useful for neglect training is demonstrated. For neglect training, the systems and methods described herein use connected hardware to provide tactile / vibration feedback when the patient correctly hits the target. The connected hardware includes a device, a video motion capture system, or a connected bell. All of these devices are connected to the described system, vibrate when tapped, and have a speaker to play auditory feedback. For example, the connected bell provides data to the system in the same way as sensor 200, for example, data about the patient knocking the bell. The video motion capture system provides video data to the system in the same way as camera 206. The key input for neglect training is information related to tracking movement to a specific location. The program uses connected hardware to provide tactile / vibration feedback when the patient correctly hits the target. Suitable groups for neglect training include patients with spatial neglect or unilateral visual neglect symptoms.

[0161] Figure 19 The flowchart of neglect training shown in Figure 18 The flowchart for gait training shown in FIG is substantially the same, with the differences noted herein. For example, a baseline test determines the patient's status and / or improvement from a previous test. In some embodiments, the baseline test includes displaying four objects evenly spaced from left to right on a screen (e.g., display 222 of handheld device 220). The patient is instructed, either through a prompt appearing on display 222 or verbally by the therapist, to tap the objects in time to the beat of background music. As with gait training, the patient is instructed to tap a bell in time to the beat of the background music. Each accurate tap provides feedback. After collecting baseline information, a plurality of objects evenly spaced from left to right are displayed on the screen. As described above, the patient is instructed to tap the objects in sequence from left to right to the beat of the background music. Each accurate tap provides feedback. As with gait training, the analysis system 108 evaluates the patient's response, categorizes the response, and provides instructions to add or remove objects, or to increase or decrease the tempo of the music to achieve the target tempo.

[0162] Figure 20 The invention discloses a technique useful for intonation training. For intonation training, the system and method described herein rely on speech processing algorithms. The phrases typically selected are common words in the following categories: bilabial sounds, glottal sounds, and vowels. The hardware is connected to the patient and provides tactile feedback to one of the patient's hands in beats per minute. The key inputs for intonation training are the intonation of spoken speech and words and the rhythm of speech. Suitable populations for intonation training include patients with Broca's aphasia, expressive aphasia, non-fluent aphasia, apraxia, autism spectrum disorder, and Down's syndrome.

[0163] Figure 20 The flowchart of intonation training shown in Figure 18The flowchart of gait training shown in is essentially the same, with the differences noted herein. For example, tactile feedback is provided to one of the patient's hands to encourage tapping. The patient is then instructed to listen to the music being played by a prompt appearing on the display 222 or a verbal prompt from the therapist. The spoken phrase to be learned is played by dividing it into two parts, wherein the first of the two parts is high-pitched and the second of the two parts is low-pitched. The patient is then instructed to sing the phrase using the device using the two pitches being played by a prompt appearing on the display 222 or a verbal prompt from the therapist. As with gait training, the analysis system 108 evaluates the patient's response and classifies the response based on pitch accuracy, spoken words, and ranking by the patient or assistant / therapist, and provides instructions to provide alternative phrases and compare the response with the target speech parameters.

[0164] Figure 21 The invention discloses a technique useful for music stimulation training. For music stimulation training, the system and method described herein rely on a speech processing algorithm. Familiar songs are used together with the algorithm to separate the expected part (called the expected violation). The hardware includes a speaker for receiving and processing the patient's singing, and in some embodiments, the therapist can manually provide input about the singing accuracy. The key input is information related to the intonation of the voice and words spoken and the rhythm of the speech and the music preference. Appropriate groups include patients suffering from Broca's aphasia, non-fluent aphasia, TBI, stroke and primary progressive aphasia.

[0165] Figure 21 The flowchart of music stimulation training shown in Figure 18 The flowchart for gait training shown in is essentially the same as that shown in FIG, with the differences noted herein. For example, a song is played for the patient and the patient is instructed to listen to the song by a prompt appearing on the display 222 or verbally prompted by the therapist. Musical cues are added to the song. Subsequently, at the expected point, a word or note is omitted and a hand gesture musical cue is played to prompt the patient to sing the missing word or note. As with gait training, the analysis system 108 evaluates the patient's response and categorizes the response based on pitch accuracy, spoken words, and ranking by the patient or assistant / therapist, and provides instructions for playing other parts of the song to improve the speech to the target speech parameters.

[0166] Figure 22Techniques useful for gross motor training are demonstrated. For gross motor training, the systems and methods described herein are intended to help with ataxia, range of motion, or activation. The more challenging aspects of the exercises are musical "accents," for example, by using melodic cues, harmonic cues, rhythmic cues, and / or dynamic cues. The key input is information related to movement captured in X, Y, and Z via connected hardware or a camera system. Suitable populations include neurological, orthopedic, strength, endurance, balance, posture, range of motion, TBI, SCI, stroke, and cerebral palsy patients.

[0167] Figure 22 The flowchart of gross motor training shown in Figure 18 The flowchart for the gait training presented in FIG is substantially the same, with the differences noted herein. As with the gait training, the patient is prompted to move to the baseline tempo of the music selection. The analysis system 108 evaluates the patient's response and categorizes the response based on the accuracy of the movement and entrainment as described above, and provides instructions to increase or decrease the tempo of the music being played.

[0168] Figure 23 Techniques useful for grip training are presented. For grip training, the systems and methods described herein rely on sensors associated with a gripper mechanism. The hardware includes a gripper mechanism with a pressure sensor and a connected speaker associated with a handheld device 220. The key input is the pressure provided by the patient to the gripper mechanism in a manner similar to the heel strike pressure measured by sensor 200. Suitable populations include patients with neurological, orthopedic, strength, endurance, balance, posture, range of motion, TBI, SCI, stroke, and cerebral palsy.

[0169] Figure 23 The flowchart of grip strength training shown in Figure 18 The flowchart for gait training is substantially the same as that shown in FIG, with the differences noted herein. As with gait training, the patient is prompted to apply force to the gripping device to the baseline tempo of the selected music. Analysis system 108 evaluates the patient's response and categorizes it based on the accuracy of movement and entrainment as described above, and provides instructions to increase or decrease the tempo of the music being played.

[0170] Figure 24 Techniques useful for speech prompting training are presented. For speech prompting training, the systems and methods described herein rely on speech processing algorithms. The hardware may include speakers for receiving and processing the patient's singing, and in some embodiments, the therapist may manually provide input regarding speech accuracy. Key inputs are the intonation of spoken voice and words, the rhythm of speech, and musical preferences. Suitable populations include patients with robotic problems, word-calling problems, and stuttering problems.

[0171] Figure 24 The flowchart of verbal prompt training shown in Figure 18 The flowchart for gait training is substantially the same as that shown in FIG, with the differences noted herein. As with gait training, the patient is prompted to speak a sentence by speaking one syllable in time with each beat of the music selection, either through prompts appearing on display 222 or verbal prompts from the therapist. Analysis system 108 evaluates the patient's speech and categorizes responses based on speech and entrainment accuracy as described above, and provides instructions to increase or decrease the tempo of the music being played.

[0172] Figure 25 Techniques useful for training minimally conscious patients are demonstrated. The systems and methods described herein rely on imaging systems, such as 3-D cameras, to measure whether the patient's eyes are open, the direction the patient is looking, and the patient's pulse or heart rate. The program searches and optimizes heart rate, stimulation, respiratory rate, eye closure, posture, and restlessness. Suitable populations include patients with coma and impaired consciousness.

[0173] Figure 25 The flowchart for the training of minimally conscious patients is shown in Figure 18 The flowchart for gait training shown in is essentially the same, with the differences noted herein. As with gait training, the patient is provided with increasing stimulation at the patient's respiratory rate (PBR). For example, stimulation is first provided to the patient with a PBR of a musical chord and the patient's eyes are observed to see if they are open. If the patient's eyes are not open, the stimulation is increased sequentially, from humming a simple melody with PBR, to singing "ah" with PBR, to singing the patient's name with PBR (or playing a recording of such a sound), and checking at each input whether the patient's eyes are open. The analysis system 108 evaluates the patient's eye tracking and classifies the response according to the level of consciousness and provides instructions to change the stimulation.

[0174] Figures 26-28 Techniques useful for attention training are demonstrated. For attention training, the systems and methods described herein operate in a closed-loop manner to help patients maintain, divide, alternate, and select their attention. No visual cues are allowed to indicate which movements to perform. Suitable populations include patients with brain tumors, multiple sclerosis, Parkinson's disease, and other neurological diseases and injuries.

[0175] Figure 26 The flowchart of sustained attention training shown in Figure 18 The flowchart for gait training presented in is essentially the same as that presented in

[15] , with the differences noted herein. As with gait training, the patient is provided with an instrument (e.g., any working instrument such as drumsticks, drums, keyboard, or wirelessly connected versions of each) and is instructed to perform the following steps: Figure 26As shown, the patient is instructed by prompts appearing on the display 222 or verbal prompts from the therapist to follow or perform audio prompts defined by levels 1 through 9. The analysis system 108 assesses the patient's ability to accurately complete the task and categorizes the response to change the pace or difficulty of the task. Similarly, Figure 27 A flow chart for alternating attention training is shown, in which instructions are provided by prompts appearing on the display 222 or verbally prompted by the therapist to follow or perform a task of audio prompts that alternate between the left and right ears. Figure 28 A flow chart for distraction is presented in which instructions are provided to follow or perform a task with an audio prompt having audio signals in both the left and right ears.

[0176] Figure 29 The flowchart of flexibility training shown in Figure 18 The flowchart for gait training presented in [ 15 ] is essentially the same as that shown in [ 15 ], with the differences noted herein. For dexterity training, the patient is instructed to tap a piano keyboard with their fingers to collect baseline movement and range of motion information. The song begins at a specific beats per minute, and the patient begins tapping with a baseline number of fingers. Analysis system 108 assesses the patient's ability to accurately complete the task and categorizes the response to alter the tempo or difficulty of the task.

[0177] Figure 30 The flowchart of oral motor training shown in Figure 18 The flowchart for gait training shown in FIG is substantially the same, with the differences noted herein. For oral motor training, the patient is instructed to perform a task that alternates between two sounds (e.g., "oh" and "ah"). The analysis system 108 assesses the patient's ability to accurately complete the task and classifies the response to change the tempo or difficulty of the task, for example, by providing different target sounds.

[0178] Figure 31 The flowchart of breathing exercises shown in Figure 18 The flowchart for gait training presented in is essentially the same as that presented in the present article, with the differences noted herein. For breathing training, a baseline breathing rate and shallowness of breathing are determined. The music provides a baseline rhythm at the patient's breathing rate, and the patient is instructed to perform Figure 31 The analysis system 108 assesses the patient's ability to accurately complete the task and classifies responses to change the pace or difficulty of the task, for example, by providing different breathing patterns.

[0179] According to further aspects of the disclosed embodiments, methods, systems, and apparatus are described herein for determining optimal biomechanical signals (including but not limited to motion signals) to support next-generation medical / therapy systems as they relate to improving or maintaining motor function. This includes a method for calculating gait-specific parameters for patients with walking impairments due to neurological injury or disease or any brain-related degeneration. The present invention also includes the ability to utilize direct stimulation such as rhythmic auditory stimulation (RAS) and auditory motor entrainment (AME) to fuse biometric signals, patients, and audio content into a synchronized state to improve motor function.

[0180] As discussed in conjunction with the exemplary gait training systems and techniques described above (see, e.g., Figure 18 ), an exemplary process implemented by the music therapy center 110 for determining whether a person is entrained while walking involves calculating how closely the person's footsteps are aligned with the beat or tempo pulse in a piece of music. An "entrainment parameter" (EP) and / or an EP ratio parameter may provide a measure of how closely the timing of the user's footsteps is in phase with the timing of the beats of the music. A range of EP values ​​may be suitable for indicating whether a person is "entrained" by the music and may be evaluated, for example, using an average over a rolling time window and filtering out natural fluctuations.

[0181] For healthy walkers, they are able to take steps of the same length and spacing on both sides of their body, with consistent variability. However, this becomes more complicated for walkers with unhealthy gait. People with abnormal gait (e.g., due to stroke, MS, PD, osteoarthritis, etc.) may actually have difficulty entraining with both feet. This is because, for example, the patient lacks control and / or confidence on one side of their body. Therefore, test results may reveal good entrainment on the patient's more controlled step side, but relatively poor entrainment for the less controlled step side (e.g., high variability in step timing or step initiation time of the other foot).

[0182] This side-to-side variability phenomenon is not limited to gait and may occur and be applicable to other repetitive motion activities, such as upper or lower limb movements. Therefore, although the exemplary embodiments are described in the context of systems and methods for gait training, these systems and methods can be similarly implemented to handle other repetitive motion activities.

[0183] As described above with respect to step 1822, symmetry analysis can be applied to sensor data collected from each foot or other body location, and a determination can be made regarding the symmetry of the patient's gait or other repetitive motion. Furthermore, according to one or more embodiments, the systems and methods disclosed herein are specifically configured to identify, account for, and overcome (i.e., reduce) variability in entrainment results between the two sides of the body (including the upper and lower extremities). This is achieved by the system assessing and training the two sides of the body differently.

[0184] To overcome variability caused by one foot, according to one prominent aspect, the system can be configured to select only one of the patient's feet for entrainment calculation. This selected side is referred to as the "entrainment side." In some cases, selecting one side to focus entrainment calculations on can be beneficial, as monitoring entrainment measurements on both sides may not be as useful for some patients with unhealthy gait. In the exemplary embodiments described herein, the patient's relatively better-performing side can be selected as the entrainment side. However, it should be understood that a less-performing side (e.g., the more affected side of the patient's body) can be selected as the entrainment side, as this approach may be more beneficial for some ailments and related therapies. The process implemented by the system for selecting a particular foot over another foot can involve various strategies or considerations, including, for example: which side of the patient's body achieves the most consistent footsteps; the side with the least amount of footstep variability (e.g., determined based on the standard deviation and / or mean of gait-related parameters); the side with the best entrainment in a head-to-head comparison between the sides, etc. The process for selecting a foot can also be a combination of the above options. Additionally or alternatively, adaptive or self-learning strategies can be applied. The system can use any of these strategies while someone is walking with the sensors, with or without the help of pacing cues.

[0185] In addition to or alternatively to the above-described method for selecting an entrainment side, the system can also be configured to assess an EP specific to a selected foot. In such embodiments, the EP and related equations, as previously shown and discussed, are calculated by the system using the time difference between the repetitive motion of the selected side compared to the period between alternating rhythmic components or beats in the audio content (e.g., an auditory stimulus cue corresponding to a step with the selected foot). As can be appreciated, the rhythmic components (e.g., beats) of the auditory stimulus (e.g., music) can be arranged to cause the patient to step with a first foot on a first beat and with the other foot on a subsequent beat. Thus, the timing of steps performed using the entrainment side is assessed taking into account the timing of the alternating rhythmic components.

[0186] For example, in one embodiment, the system can be configured to perform a baseline calculation during a scenario where someone is walking with a sensor and there is no music. Based on the received biomechanical feedback data captured by the system from the sensor, a comparative calculation can be performed to determine which side is closest to the actual number of steps taken. The system then selects this side as the "entrainment side." Similarly, during the same setting, calculations can be performed on the side with the lowest coefficient of variance (CV), variability, or standard deviation. This side is then selected as the entrainment side. In another embodiment, the system can be configured to select the "entrainment side" when the patient is using a cane or other device. In this case, the entrainment side can be the left side, the right side, the side of the body holding the cane, or the side of the body opposite the side holding the cane.

[0187] Additionally or alternatively, the system is configured to collect biomechanical data while a person walks and listens to audio content, such as music or a metronome. The system can be configured to calculate both right and left foot EP and compare the separately calculated EPs to select an entrainment side. For example, the side with the best entrainment parameters (e.g., producing more consistent entrainment results) over a particular time period can be selected as the entrainment side and used for the remainder of the gait training session.

[0188] Additionally or alternatively, in some embodiments, the system can be configured to determine and compare the variability of the gait cycle time (GCT) for each foot. For example, as background to the discussion of windowing / gating techniques above, various types of repetitive motion cycles are described herein. The system can use an analysis that calculates the lowest GCT variability (e.g., the most stable interval) to inform entrainment side selection. This determination can be important because the side with the most consistent periodicity will be more likely to match the beat intervals of the music. It should be further noted that this technique can also enable the system to operate effectively when using only a single sensor at a body location other than the foot (such as a motion-sensing wearable device or smartphone). In this case, an alternating pattern of lower variability values ​​will indicate the entrainment side.

[0189] The various phases of gait fall into two main categories: "swing" and "stance." Within these phases are additional phases: heel strike, foot flat, mid-stance, heel-off, toe-off, mid-swing, and end-swing. When gait phases are assessed from an IMU or sensor and someone's gait is impaired, errors will often occur in certain phases from step to step. Even if errors are present, there may be one or more gait portions that are more consistent. In addition to selecting the entrainment side, according to another salient aspect, the system can be configured to select a specific motion gait phase for use in step, step onset detection, and related entrainment calculations as mentioned in the gait training example above. This extends the method described above because it involves detecting the start timestamp of the step ("gated analysis"). In some embodiments, the selected motion gait phase can generally be the gait phase with the most consistency and / or highest amplitude. The consistency of gait features evaluated for entrainment or step onset time detection is very important for reliably determining whether a patient is walking with beat (e.g., if inconsistent gait phases are selected, the patient may be walking with beat, but gait fluctuations may affect the accuracy of the system's data-based calculations, thereby providing false negative results). As described above, the gait cycle has multiple phases, which can include heel strike, mid-stance, toe-off, and many different segments between these points. These can all be part of the gait cycle, and the time to fully complete the gait cycle can be referred to as the gait cycle time.

[0190] When deciding to use step timing as part of the EP entrainment calculation, the system preferably uses a strategy to select a portion of the gait cycle. This becomes particularly important as the quality of movement (e.g., gait) degrades. In this case, the system can be configured to implement a method to ensure robustness to degraded gaits, which can include measuring, by a voting system, which detected gait phase appears strongest (e.g., occurs most frequently). An example of a voting system is to record each reported gait phase within a one-minute period. In an ideal gait, each gait phase recorded would be identical, e.g., each phase detected by the system would be reported with the same number. In impaired gaits typical of rehabilitation patients, the number of each reported gait phase is influenced by the patient's biomechanical movement pattern (e.g., spatial circulation). In this case, the gait phase reported with the highest frequency will be associated with the gait phase with the highest reliability. To calculate entrainment accuracy, the voting system will select this highest-ranked phase when marking boundaries of step timing (e.g., gait phase "gated analysis" boundaries). In the event of a tie, other parameters such as variability, precision, or accuracy can be used to select the phase. In some embodiments, the system may be configured to first check variability or other parameters, followed by counting methods. In some embodiments, the entrainment accuracy may be calculated with each stage, thereby selecting the one with the strongest entrainment accuracy as the preferred stage.

[0191] Additionally or alternatively, the system can implement a method that involves monitoring incoming gait phases and using an algorithmic approach to "jump" to a different gait phase if it is more reliably present at a later stage. This can occur when the patient increases or decreases their speed, or undergoes other significant changes in their gait. Many such "signal strength" strategies are used for RF communications, and the system can implement similar concepts here to help extract the best biomechanical signal for patients whose gait patterns are affected by disease or injury.

[0192] In some cases, a user may place a sensor on the wrong foot, for example, a left sensor on the right foot. This can lead to errors in calculating symmetry and entrainment. Therefore, the system can be configured to implement a method that compares spatial movement patterns to determine if a sensor is placed on the wrong foot and, accordingly, prompts the system to "auto-correct." More specifically, upon detecting such an error condition, the system can be configured to swap the data input from each foot via software to avoid adversely affecting therapy or offline processing of gait data.

[0193] Additionally or alternatively, in some embodiments, an identification of the user's side as the "affected" side (e.g., after a stroke) can be provided as input to the system. The affected side typically has a physical impairment, such as hemiplegia or a circumferential gait pattern, that hinders the user's step timing on that side. The system can be configured to use any such input to confirm any empirically derived determination, including or in combination with the methods discussed above.

[0194] When selecting an entrainment side and then monitoring specific parameters (e.g., gait phase) for the purpose of calculating EP, the system can be configured to provide side-specific repetitive motion therapy using the exemplary systems and methods described above. Figure 18 The described gait training systems and techniques can be adapted to provide side-specific repetitive motion therapy. More specifically, based on biomechanical sensor data monitored from the selected entrainment side, the patient's EP can be assessed, and the EP can be utilized to dynamically generate rhythmic auditory stimulation cues for the patient, such as in combination with Figure 18 Furthermore, while in this exemplary embodiment, EP may be calculated based on biomechanical data captured from a selected side of entrainment, and is therefore described as "side specific," it will be appreciated that rhythmic auditory stimulation cues may be output to assist in entrainment of both sides of the patient's body.

[0195] In view of the foregoing, it can be understood that what is disclosed is a method for rehabilitating a patient with a limb impairment by providing music therapy. The method is implemented on a computer system having one or more physical processors configured by machine-readable instructions that, when executed, perform the method. In one or more embodiments, as Figure 32 As shown in FIG, method 3200 includes the step of receiving 3205 biomechanical data from a patient regarding repetitive movements performed by the patient using both sides of the body. The method also includes analyzing 3210 the biomechanical data to assess side-specific characteristics of the patient's movements, such as the symmetry of the patient's movements or otherwise determining the entrainment potential for each side of the patient's body performing the repetitive movement activity. The method also includes identifying 3215 an entrained side based on the analysis of the biomechanical data. The entrained side is a specific side of the body that is analyzed to define therapy that can be administered to the patient to train the entrained side, the contralateral side, or both sides. It is important to note that after the analysis of the selected entrained side is complete, it may not be necessary to consider a single side as the entrained side due to sufficient temporal symmetry in its physical movements. In these cases, the onset times of both the left and right sides can be used to calculate the EP. In addition to selecting the entrained side at step 3215, according to another aspect, the method includes selecting a specific movement phase that is monitored and used to measure the patient's entrainment potential. After selecting the entrainment side and the specific movement phase to be monitored for calculating the EP, the method includes providing the patient with side-specific repetitive movement therapy 3225. For example, the method includes evaluating the entrainment potential specific to the selected side of the body, for example, while providing the side-specific repetitive movement therapy. More specifically, the entrainment potential can be calculated based on the time difference between the repetitive movement of the selected side and the period between alternating rhythmic components or beats in the audio content.

[0196] According to another aspect, providing repetitive motion therapy to a patient using a single side, i.e., a selected entrained side, to calculate an EP comprises the steps of: receiving biomechanical data from the patient regarding repetitive movements of at least the patient's selected entrained side and determining a baseline condition for the patient's entrained side; and determining a baseline tempo rhythm having a constant frequency based on the baseline condition. The step of providing therapy further comprises: providing music (or more generally, auditory stimulation) to the patient, wherein the music has a tempo signal (or more generally, a rhythmic component) output at corresponding tempo times. Outputting the music may further comprise providing cues to the patient to perform each repetitive movement in time with the associated baseline tempo signal. It should be understood that the cues need not be specific cues provided in addition to the music. The cues may be the tempo of the music, which serves to consciously or subconsciously cue the patient to perform the movement in time with the tempo. Furthermore, tempo cues may be provided for each repetitive movement performed using either side of the patient's body.

[0197] The step of providing therapy further includes the step of receiving time-stamped biomechanical data from the patient related to the repetitive movements performed by the patient using at least the entrained side in time with the beat signal. Biomechanical data for repetitive movements performed using either side of the patient's body can be received. The step of providing therapy further includes the step of analyzing the time-stamped biomechanical data to identify the start times of specific movement phases of one or more cycles of repetitive movements performed using the entrained side. Additionally, the method includes calculating an entrainment potential based on the start times and corresponding output times of the associated beat signal. More specifically, calculating the entrainment potential may include determining a delay between the start time of the specific movement phase of each repetitive movement and the corresponding output time of the associated beat signal. The step of providing therapy further includes the step of modifying auditory stimulation based on the calculated entrainment potential. For example, modifying the auditory stimulation may include modifying a baseline beat tempo based on the calculated entrainment potential and determining whether a target beat tempo has been achieved. While the exemplary embodiments of the system and method for providing side-specific repetitive motion therapy have been discussed in relation to gait training, it should be understood that the system and method are similarly applicable to training other parts of the body.

[0198] The above-mentioned systems, devices, methods, processes, etc. can be implemented in hardware, software, or any combination thereof suitable for application. Hardware can include general-purpose computers and / or special-purpose computing devices. This is included in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices or processing circuit systems and internal and / or external memories. On the contrary, this can also include one or more application-specific integrated circuits, programmable gate arrays, programmable array logic components, or any one or more other devices that can be configured to process electronic signals. It should be further understood that the implementation of the above-mentioned process or device can include computer executable code that can be stored, compiled, or interpreted to run on one of the above-mentioned devices using a structured programming language (such as C), an object-oriented programming language (such as C++), or any other high-level or low-level programming language (including assembly language, hardware description language, and database programming language and technology), as well as a heterogeneous combination of processors, processor architectures, or different hardware and software combinations. On the other hand, the method can be embodied in a system that performs its steps and can be distributed on the device in several ways. At the same time, the processing can be distributed on devices such as the various systems described above, or all functions can be integrated into a dedicated standalone device or other hardware. On the other hand, the means for performing the steps associated with the processes described above may include any of the hardware and / or software described above.All such permutations and combinations are intended to fall within the scope of the present disclosure.

[0199] The embodiments disclosed herein may comprise a computer program product comprising computer executable code or computer usable code that, when executed on one or more computing devices, performs any and / or all of its steps. The code may be stored in a non-transient form in a computer memory, which may be a memory for executing the program (such as a random access memory associated with a processor) or a storage device such as a disk drive, flash memory, or any other optical, electromagnetic, magnetic, infrared, or other device or combination of devices. On the other hand, any of the systems and methods described above may be embodied in any suitable transmission or propagation medium that carries the computer executable code and / or any input or output from the computer executable code.

[0200] It should be understood that the apparatus, system, and method described above are illustrated by way of example and not limitation. In the absence of explicit indications to the contrary, the disclosed steps may be modified, supplemented, omitted, and / or rearranged without departing from the scope of the present disclosure. Many variations, additions, omissions, and other modifications will be apparent to those of ordinary skill in the art. In addition, the order or presentation of the method steps in the foregoing description and accompanying drawings is not intended to require an order in which the steps are performed, unless the context clearly requires an order or is otherwise clarified.

[0201] The method steps of the implementation described herein are intended to include any suitable method consistent with the patentability of the following claims that these method steps are performed, unless the context clearly provides different meanings or is otherwise clarified. Therefore, for example, performing step X includes any suitable method for making another party (such as a remote user, a remote processing resource (such as, a server or a cloud computer) or a machine) perform step X. Similarly, performing steps X, Y and Z can include any method for guiding or controlling any combination of these other individuals or resources to perform steps X, Y and Z to obtain the benefits of these steps. Therefore, the method steps of the embodiment described herein are intended to include any suitable method consistent with the patentability of the following claims that one or more other parties or entities are performed, unless the context clearly provides different meanings or is otherwise clarified. These parties or entities do not need to be under the guidance or control of any other party or entity, and do not need to be located in a specific jurisdiction.

[0202] It should also be understood that the above methods are provided by way of example. In the absence of explicit indications to the contrary, the disclosed steps may be modified, supplemented, omitted and / or rearranged without departing from the scope of the present disclosure.

[0203] It will be understood that the methods and systems described above are illustrated by way of example and not limitation. Many variations, additions, omissions, and other modifications will be apparent to those skilled in the art. In addition, the order or presentation of the method steps in the foregoing description and accompanying drawings is not intended to require the order in which the steps are performed, unless the context clearly requires a particular order or is otherwise clarified. Therefore, although specific embodiments have been shown and described, it will be apparent to those skilled in the art that various changes and modifications in form and detail may be made therein without departing from the spirit and scope of the present disclosure.

Claims

1. A system for rehabilitating a patient with a limb disability by providing repetitive movement therapy, wherein auditory stimulation with a metronomic signal is provided to the patient and the patient attempts to perform repetitive movements in time with the metronomic signal using a first side and an opposite second side of the patient's body, the system comprising: A computer system having a processor configured by machine-readable instructions to: receiving biomechanical data from the patient regarding repetitive movements performed by the patient using the first side and the second side of the body, respectively, wherein the biomechanical data is measured using sensors associated with the patient; selecting an entrainment side based on the received biomechanical data, wherein the entrainment side is one of the first side or the second side of the body; The processor performs repetitive motion therapy by: Outputting auditory stimulation to the patient, wherein the auditory stimulation includes a beat signal output at a corresponding beat time; receiving time-stamped biomechanical data of the patient associated with the repetitive movements performed by the patient using at least the entrainment side associated with the beat signal, wherein the biomechanical data is measured using sensors associated with the patient; calculating an entrainment potential of the entrainment side, wherein the entrainment potential is calculated by comparing the timing of the repetitive movement performed by the patient using the entrainment side with the corresponding beat time of the beat signal; and The auditory stimulus is modified according to the calculated entrainment potential.

2. The system of claim 1 , further comprising: an auditory output device, the auditory output device being configured to provide the auditory stimulation to the patient; as well as The sensor is associated with the patient and measures the time-stamped biomechanical data.

3. The system of claim 1 , wherein the repetitive motion is a cyclic motion comprising a plurality of phases, and wherein the processor is further configured to select a particular phase among the phases of the repetitive motion, and wherein the entrainment potential is calculated based on the timing of the particular phase of each cycle of the repetitive movement performed by the patient using the entrainment side.

4. The system of claim 3, wherein the processor is configured to provide repetitive motion therapy by: analyzing the time-stamped biomechanical data to identify a respective start time of the particular phase for each of a plurality of cycles of the repetitive movement performed using the entrained side; and The entrainment potential is calculated based on the corresponding start time and the corresponding beat time of a corresponding beat signal in the beat signals.

5. The system of claim 4, wherein the entrainment potential is calculated by comparing a time period between the corresponding start times of the specific phases of each of a plurality of cycles with a time period between alternate beats in the beat signal. 6 . The system of claim 4 , wherein calculating the entrainment potential comprises measuring a delay between the corresponding start time of the specific phase of each of a plurality of cycles and the corresponding beat time of a corresponding beat signal in the beat signals.

7. The system of claim 1 , wherein the processor is configured to select the entrainment side by determining a symmetry of the repetitive movement performed using the first side of the body relative to the repetitive movement performed using the second side of the body based on the biomechanical data, and wherein the entrainment side is selected based on the determined symmetry.

8. The system of claim 1 , wherein the processor is configured to select the entrainment side by measuring one or more of the following from the biomechanical data: consistency of said repetitive motion for each side of said body, respectively; variability of the repetitive motion for each side of the body separately; and an entrainment potential, separately for each side of the body; and The entrainment side is selected based on the results of the measuring step.

9. The system of claim 8, wherein the variability measured is gait cycle time (GCT) variability measured separately for each side of the body.

10. The system of claim 1, wherein the processor is configured to modify the auditory stimulus by adjusting a timing of the beat signal based on the calculated entrainment potential and based on a target beat tempo.

11. The system of claim 1 , wherein the processor is configured, prior to performing the step of performing the repetitive motion therapy: determining a baseline condition of the patient's entrained side based on the biomechanical data; and A baseline beat rhythm is defined for the auditory stimulus according to the baseline condition, wherein the baseline beat rhythm has a constant frequency.

Images