Systems and methods for controlling devices using detected changes in neural-related signals
By detecting and analyzing the intensity changes of nerve-related signals in subjects, and by combining intravascular devices and external processors, the problem of complexity and high error rate in controlling peripheral devices in patients with locked-in syndrome of existing BCI systems has been solved, achieving more efficient device control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SYNCHRON AUSTRALIA PTY LTD
- Filing Date
- 2021-04-01
- Publication Date
- 2026-07-21
AI Technical Summary
Existing brain-computer interface (BCI) systems have difficulty effectively controlling peripheral devices, especially for patients with locked-in syndrome. In particular, control via a single virtual switch is complex, has a high error rate, and generates false positive signals.
By detecting changes in the intensity of neural signals in subjects, including decreases and increases in the power of neural oscillations, the system uses a processor combining an intravascular device and an external or implanted device for filtering and machine learning classification to automatically detect changes in the intensity of neural signals and transmit input commands to the device based on these changes.
This enables patients with locked-in syndrome to control peripheral devices simply and accurately, reducing operational complexity and error rates, and improving control independence and efficiency.
Smart Images

Figure CN122431522A_ABST
Abstract
Description
[0001] This application is a divisional application of the application filed on April 1, 2021, with application number 202180027286.0 and entitled "System and method for controlling a device using changes in detected neural-related signals".
[0002] Cross-reference to related applications This application claims the benefit of U.S. Provisional Application No. 63 / 003,480, filed April 1, 2020, the contents of which are incorporated herein by reference in their entirety. Technical Field
[0003] This disclosure generally relates to brain-computer interfaces, and more specifically, to systems and methods for controlling devices by utilizing detected changes in neural signals of a subject. background Previously, it has been demonstrated that people with limited mobility can use brain-computer interfaces (BCIs) to control peripheral devices, such as personal electronic devices, Internet of Things (IoT) devices, software, and mobility vehicles. An effective BCI should allow people with all ranges of limited mobility to effectively control these peripheral devices, including those with severe limitations, such as locked-in patients who can only control a single switch or virtual switch via BCI.
[0004] However, current BCIs allow patients with such locked-in syndromes to access peripheral devices via a single switch, typically using automated switch scanning for this control. Automated switch scanning involves serially scanning a number of interactive items on a given control panel, where the user selects a target item by engaging a switch as the target item is highlighted. This approach is tedious and intolerant when an incorrect selection is made, as the user must wait for the serial scan to complete before restarting the entire process to correct the selection.
[0005] Furthermore, because current BCI systems often rely on neurally related signals that are difficult to detect or may produce false positives, patients with locked-in syndrome may even have difficulty engaging with a single virtual switch.
[0006] Therefore, a solution is needed that allows patients with severely limited mobility to maintain or retain their independence, even when such patients control only a single virtual switch. This solution should not be overly complex and should address the shortcomings of current BCI systems and methods. Overview Systems and methods for controlling a device using detected changes in neurally related signals from a subject are disclosed. In one embodiment, a method for controlling a device or software application is disclosed. This method may include detecting a decrease in the intensity of a subject's neurally related signal below a measured baseline level, detecting an increase in the intensity of the neurally related signal above the baseline level after the decrease, and transmitting an input command to the device when or after the increase in the intensity of the neurally related signal is detected.
[0007] In some embodiments, the subject's neural-related signals may be neural oscillations or electroencephalograms (EEGs). Neural oscillations may include oscillations in one or more frequency bands. In some embodiments, neural oscillations include beta-band oscillations with frequencies between about 12 Hz and 30 Hz.
[0008] In some embodiments, an intravascular device implanted in the subject can be used to measure or monitor neurally related signals. In these and other embodiments, the steps of detecting a decrease or increase in the intensity of neurally related signals and transmitting input commands can be performed using one or more processors of a device separate from the intravascular device or one or more processors of a device embedded in or coupled to the intravascular device.
[0009] In some embodiments, the device may be configured to be located outside the subject's body. In other embodiments, the device may be configured to be implanted inside the subject's body (e.g., in the subject's chest area or arm).
[0010] The device may refer to a telemetry unit and / or a host device. In other embodiments, the device may refer to a computing device or a controller / control unit of an implantable or non-implantable device.
[0011] Neurological signals can be detected using electrodes implanted in the blood vessels of a subject. For example, neurological signals can be detected using electrodes implanted in the blood vessels of a subject's brain.
[0012] The method may further include using one or more processors and one or more software filters to filter the raw neurally relevant signals obtained from the intravascular device. The method may also include feeding the filtered signal into a classification layer of software running on the device or another device. The classification layer is configured to use a machine learning classifier to automatically detect decreases and increases in the intensity of the neurally relevant signals.
[0013] In some embodiments, detecting a decrease in the intensity of a neurally relevant signal may include detecting a decrease in the power of a neural oscillation below a baseline oscillation power level. For example, a decrease in the intensity of a neurally relevant signal below a measured baseline level may be referred to as desynchronization of the neurally relevant signal. This decrease in the intensity of the neurally relevant signal may be caused by the subject evoking and maintaining task-related or task-irrelevant thoughts.
[0014] In these and other embodiments, detecting an increase in the intensity of a neurally relevant signal may include detecting an increase in the power of a neural oscillation exceeding a baseline oscillation power level. For example, an increase in the intensity of a neurally relevant signal exceeding a measured baseline level may be referred to as a rebound of the neurally relevant signal. The increase in the intensity of the neurally relevant signal may be caused by the subject mentally releasing task-related or task-irrelevant thoughts.
[0015] Task-irrelevant thoughts can be thoughts related to the subject's physical function, such as the subject maintaining the contraction of the subject's hamstring muscle.
[0016] The method may further include providing the subject with at least one of visual feedback, auditory feedback, tactile feedback, and neural stimulation feedback after the input command is transmitted to the device.
[0017] Transmitting input commands may include transmitting input commands to one or more terminal applications running on the device. Input commands may be commands to the device to perform at least a portion of a task associated with a task-related idea.
[0018] The device may be at least one of a personal computing device (e.g., a laptop, mobile phone, and / or tablet) or an Internet of Things (IoT) device (e.g., a smart light switch, refrigerator, oven, and / or washing machine). In some embodiments, the device may be a means of transportation, such as a wheelchair.
[0019] Another method for controlling a device or software application is disclosed. This method may include detecting a decrease in the intensity of a subject's neurally related signal below a measured baseline level, detecting an increase in the intensity of the neurally related signal above the baseline level after the decrease, determining the duration of the decrease in the intensity of the neurally related signal, selecting an input command from a plurality of conditional input commands based on the duration, and transmitting the selected input command to the device.
[0020] In some embodiments, selecting an input command based on duration may include comparing the duration with one or more time thresholds associated with the conditional input command, and selecting an input command from multiple conditional input commands based on whether the duration exceeds or fails to reach one or more time thresholds. The duration of the decrease in the intensity of the neurally related signal may be the amount of time the subject holds the thought.
[0021] In some embodiments, the subject's neural-related signals may be neural oscillations or electroencephalograms (EEGs). Neural oscillations may include oscillations across one or more frequency bands. In some embodiments, neural oscillations include beta-band oscillations with frequencies between about 12 Hz and 30 Hz.
[0022] In some embodiments, an intravascular device implanted in the subject can be used to measure or monitor neurally related signals. In these and other embodiments, the steps of detecting a decrease or increase in the intensity of the neurally related signal, determining the duration of the decrease in the intensity of the neurally related signal, selecting an input command from a plurality of conditional input commands based on the duration, and transmitting the selected input command to the device can be performed using one or more processors of a device separate from the intravascular device or one or more processors of a device embedded in or coupled to the intravascular device.
[0023] In some embodiments, the device may be configured to be located outside the subject's body. In other embodiments, the device may be configured to be implanted inside the subject's body (e.g., in the subject's chest area or arm).
[0024] The device may refer to a telemetry unit and / or a host device. In other embodiments, the device may refer to a computing device or a controller / control unit of an implantable or non-implantable device.
[0025] Neurological signals can be detected using electrodes implanted in the blood vessels of a subject. For example, neurons can be detected using electrodes implanted in the brain of a subject.
[0026] The method may further include using one or more processors and one or more software filters to filter the raw neurally relevant signals obtained from the intravascular device. The method may also include feeding the filtered signal into a classification layer of software running on the device or another device. The classification layer is configured to use a machine learning classifier to automatically detect decreases and increases in the intensity of the neurally relevant signals.
[0027] In some embodiments, detecting a decrease in the intensity of a neurally relevant signal may include detecting a decrease in the power of a neural oscillation below a baseline oscillation power level. For example, a decrease in the intensity of a neurally relevant signal below a measured baseline level may be referred to as desynchronization of the neurally relevant signal. The decrease in the intensity of the neurally relevant signal may be caused by the subject evoking and maintaining task-related or task-irrelevant thoughts.
[0028] In these and other embodiments, detecting an increase in the intensity of a neurally relevant signal may include detecting an increase in the power of a neural oscillation exceeding a baseline oscillation power level. For example, an increase in the intensity of a neurally relevant signal exceeding a measured baseline level may be referred to as a bounce of the neurally relevant signal. The increase in the intensity of the neurally relevant signal may be caused by the subject mentally releasing task-related or task-independent thoughts. Task-independent thoughts may be related to the subject's physical function, such as the subject maintaining a contraction of their hamstrings.
[0029] The method may further include providing the subject with at least one of visual feedback, auditory feedback, tactile feedback, and neural stimulation in relation to the selected input command before transmitting the input command to the device.
[0030] Transmitting input commands may include transmitting input commands to one or more terminal applications running on the device. Input commands may be commands to the device to perform at least a portion of a task associated with a task-related idea.
[0031] The device may be at least one of a personal computing device (e.g., a laptop, mobile phone, and / or tablet) or an Internet of Things (IoT) device (e.g., a smart light switch, refrigerator, oven, and / or washing machine). In some embodiments, the device may be a means of transportation, such as a wheelchair.
[0032] Another method for controlling a device or software application is disclosed. This method may include detecting a first change in a subject's neurally related signal, detecting a second change in the neurally related signal, and transmitting an input command to the device when or after the second change in the neurally related signal is detected.
[0033] The method may further include determining the duration of a first change in a neurally related signal and using that duration to select an input command from a plurality of conditional input commands. The method may also include providing the subject with at least one of visual, auditory, tactile, and neural stimulation feedback regarding the selected input command.
[0034] In some embodiments, the first change may be a decrease in the intensity of the neurally related signal to below the baseline signal level. In these embodiments, the second change may be an increase in the intensity of the neurally related signal to above the baseline signal level. Furthermore, in these embodiments, the first change may occur when the subject generates and holds a thought, and the second change may occur when the subject generates and holds a second thought.
[0035] In other embodiments, the first change may be an increase in the intensity of the neurally related signal above the baseline signal level. In these embodiments, the second change may be a decrease in the intensity of the neurally related signal below the baseline signal level. Furthermore, in these embodiments, the first change may occur when the subject generates and holds a thought, and the second change may occur when the subject generates and holds a second thought. Alternatively, the first change may occur when the subject mentally releases the first thought, and the second change may occur when the subject generates and holds a second thought.
[0036] In some embodiments, the idea can be a task-related idea. In other embodiments, the idea can be a task-independent idea. The idea can be related to the subject's physical function.
[0037] Changes in neurally related signals can be detected using an intravascular device implanted in the subject. In these and other embodiments, the steps of detecting a first change in the neurally related signal, detecting a second change in the neurally related signal, determining the duration of the change in the neurally related signal, selecting an input command from a plurality of conditional input commands based on the duration, and transmitting the selected input command to the device can be performed using one or more processors of a device separate from the intravascular device or one or more processors of a device embedded in or coupled to the intravascular device.
[0038] In some embodiments, the device may be configured to be located outside the subject's body. In other embodiments, the device may be configured to be implanted inside the subject's body (e.g., in the subject's chest area or arm).
[0039] The device may refer to a telemetry unit and / or a host device. In other embodiments, the device may refer to a computing device or a controller / control unit of an implantable or non-implantable device.
[0040] Neurological signals can be detected using electrodes implanted in the blood vessels of a subject. For example, neurons can be detected using electrodes implanted in the brain of a subject.
[0041] The method may further include using one or more processors and one or more software filters to filter the raw neurally relevant signals obtained from the intravascular device. The method may also include feeding the filtered signal into a classification layer of software running on the device or another device. The classification layer is configured to use a machine learning classifier to automatically detect decreases and increases in the intensity of the neurally relevant signals.
[0042] A system for controlling a device is also disclosed. The system may include an intravascular device configured to measure or monitor neurally related signals of a subject and a means including one or more processors. One or more processors may be programmed to detect when the intensity of the subject's neurally related signals decreases below a measured baseline level, detect when the intensity of the neurally related signals increases above the baseline level after a decrease, and transmit input commands to the device when or after detecting an increase in the intensity of the neurally related signals.
[0043] In some embodiments, one or more processors may be programmed to detect when the intensity of a subject’s neural-related signal decreases below a measured baseline level, detect when the intensity of the neural-related signal increases above the baseline level after the decrease, determine the duration of the decrease in the intensity of the neural-related signal, select an input command from a plurality of conditional input commands based on the duration, and transmit the selected input command to the device.
[0044] One or more processors may also be programmed to compare the duration with one or more time thresholds associated with the conditional input command, and to select an input command from multiple conditional input commands based on whether the duration exceeds or fails to reach one or more time thresholds.
[0045] The subject's neural-related signal can be a neural oscillation. For example, one or more processors can be programmed to detect a decrease in the intensity of a neural-related signal by detecting a decrease in the power of a neural oscillation below the baseline oscillation power level. One or more processors can also be programmed to detect an increase in the intensity of a neural-related signal by detecting an increase in the power of a neural oscillation above the baseline oscillation power level.
[0046] Neural oscillations can include oscillations across one or more frequency bands. For example, neural oscillations can include beta-band oscillations with frequencies between approximately 12 Hz and 30 Hz.
[0047] In some embodiments, a decrease in the intensity of neurally related signals may be caused by the subject recalling and holding task-related thoughts. In these embodiments, an increase in the intensity of neurally related signals may be caused by the subject mentally releasing task-related thoughts. The input command may be a command to the device to perform at least a portion of a task associated with the task-related thoughts.
[0048] In other embodiments, a decrease in the intensity of neurally relevant signals may be caused by the subject recalling and holding task-irrelevant thoughts. An increase in the intensity of neurally relevant signals may be caused by the subject mentally releasing task-irrelevant thoughts. The input command may be a command to the device to perform at least a portion of a task unrelated to task-irrelevant thoughts. For example, task-irrelevant thoughts may be thoughts related to the subject's physical function.
[0049] In some embodiments, a decrease in the intensity of a neurally related signal below a measured baseline level can be considered desynchronization of the neurally related signal. In these embodiments, an increase in the intensity of a neurally related signal above a measured baseline level can be considered a bounce of the neurally related signal.
[0050] Intravascular devices can be configured to be implanted inside a subject's brain. For example, an intravascular device can be configured to be implanted in a vein or sinus of the subject. Electrodes from the intravascular device implanted in the subject can be used to measure or monitor neural signals.
[0051] In some embodiments, the device may be configured to be located outside the subject's body. In other embodiments, the device may be configured to be implanted inside the subject's body.
[0052] One or more processors may also be programmed to use one or more software filters to filter the raw neural signals obtained from the intravascular device. One or more processors may be further programmed to feed the filtered signals into a classification layer, thereby using a machine learning classifier to automatically detect decreases and increases in the intensity of the neural signals.
[0053] One or more processors may also be programmed to provide the subject with at least one of visual, auditory, and tactile feedback regarding the input command. In these and other embodiments, the intravascular device may be configured to provide the subject with feedback in the form of neural stimulation regarding the input command.
[0054] One or more processors may also be programmed to transmit input commands to one or more terminal applications running on the device. In some embodiments, the device may be at least one of a personal computing device (e.g., a laptop, mobile phone, and / or tablet) or an Internet of Things (IoT) device (e.g., a smart light switch, refrigerator, oven, and / or washing machine). In other embodiments, the device may be a mobile vehicle, such as a wheelchair.
[0055] Systems for controlling the device may include intravascular devices configured to measure or monitor neurological signals in a subject and devices configured to detect changes in neurological signals.
[0056] In some embodiments, the device may include one or more processors programmed to detect a first change in a subject’s neural-related signal, detect a second change in the neural-related signal, and transmit an input command to the device when or after the second change in the neural-related signal is detected.
[0057] One or more processors may also be programmed to determine the duration of a first change in a neurally related signal and, based on that duration, select an input command from a plurality of conditional input commands.
[0058] In some embodiments, the first change may be a decrease in the intensity of the neurally related signal to below the baseline signal level. In these embodiments, the second change may be an increase in the intensity of the neurally related signal to above the baseline signal level.
[0059] In other embodiments, the first change may be an increase in the intensity of the neurally related signal above the baseline signal level. In these embodiments, the second change may be a decrease in the intensity of the neurally related signal below the baseline signal level.
[0060] For example, a first change occurs when a subject generates and holds a thought. A second change occurs when the subject mentally releases the thought.
[0061] Alternatively, the first change may occur when the subject mentally releases the first thought, and the second change may occur when the subject generates and retains the second thought. Brief description of the attached diagram The accompanying drawings shown and described are exemplary embodiments and are not limiting. The same reference numerals always indicate the same or functionally equivalent features.
[0062] Figure 1A A variation of the general-purpose switch module is shown.
[0063] Figure 1B This shows that when a patient is thinking of an idea, Figure 1A A variant of the general-purpose switch module.
[0064] Figure 1C It shows Figure 1A and Figure 1B A variation of the user interface of the host device for a general-purpose switch module.
[0065] Figure 2 A- Figure 2 D illustrates a variation of the general switch model for communicating with terminal applications.
[0066] Figure 3 A variation of the wireless universal switch module for communicating with terminal applications is shown.
[0067] Figure 4 A variation of a general-purpose switch module for recording neurological signals from a patient is shown.
[0068] Figure 5 It shows the result of Figures 1A-1C A variation of the method used in the general-purpose switch module.
[0069] Figure 6 It shows the result of Figures 1A-1C A variation of the method used in the general-purpose switch module.
[0070] Figure 7 It shows the result of Figures 1A-1C A variation of the method used in the general-purpose switch module.
[0071] Figure 8 A variation of a method is shown that uses detected changes in neurally related signals of a subject to control a device or software.
[0072] Figure 9 An example spectrogram is shown, which illustrates changes in detected neurally related signals in the subject.
[0073] Figure 10 This illustrates another variation of a method that uses detected changes in neural signals in a subject to control a device or software.
[0074] Figure 11 This illustrates another variation of a module that uses detected changes in neural signals in a subject to control a device or software.
[0075] Figure 12A An example spectrogram illustrating how a subject can retain their thoughts for a short period of time is shown.
[0076] Figure 12B An example spectrogram illustrating how a subject can retain their thoughts over a longer period of time is shown.
[0077] Figure 13 This illustrates yet another variation of a method that uses detected changes in neural signals in a subject to control a device or software.
[0078] Figure 14 The power variations of the beta band (e.g., 12 Hz to 30 Hz) and gamma band (e.g., 60 Hz to 80 Hz) frequencies are shown when the subject evokes / holds and then releases thoughts about movement of the subject's left and right ankles. Detailed description A universal switch module, a universal switch, and their usage methods are disclosed. For example, Figures 1A-1CA variation of the universal switch module 10 is illustrated, in which a patient 8 (e.g., a BCI user) can control one or more terminal applications 12 by thinking of an idea 9. Module 10 may include a neural interface 14 and a host device 16. Module 10 (e.g., host device 16) can communicate with one or more terminal applications 12 via wired and / or wireless communication. The neural interface 14 may be a bio-media signal detector (e.g., an electrical conductor, a biochemical sensor), the host device 16 may be a computer (e.g., a laptop, a smartphone), and the terminal application 12 may be any electronic device or software. The neural interface 14 may monitor neurally related signals 17 of the bio-media via one or more sensors. A processor of module 10 may analyze the detected neurally related signals 17 to determine whether the detected neurally related signals 17 are associated with an idea 9 assigned to an input command 18 of the terminal application 12. When the idea 9 assigned to the input command 18 is detected by the neural interface 14 and associated with the input command 18 by the processor, the input command 18 may be sent (e.g., via a processor, controller, or transceiver) to the terminal application 12 associated with that input command 18. Idea 9 can be assigned to input commands 18 of multiple terminal applications 12. Module 10 thus advantageously enables the patient 8 to independently control multiple terminal applications 12 (e.g., a first terminal application and a second terminal application) with a single idea (e.g., idea 9), wherein idea 9 can be used to control the first and second applications at different times and / or simultaneously. In this way, module 10 can be used as a universal switch module, capable of using the same idea 9 to control multiple terminal applications 12 (e.g., software and devices). Idea 9 can be a universal switch, assignable to any input command 18 of any terminal application 12 (e.g., input command 18 of the first terminal application and input command 18 of the second terminal application). The first terminal application can be a first device or first software. The second terminal application can be a second device or second software.
[0079] When patient 8 thinks of idea 9, the input command 18 associated with idea 9 can be sent by module 10 (e.g., via a processor, controller, or transceiver) to their respective terminal application 12. For example, if idea 9 is assigned as input command 18 to a first terminal application, then when patient 8 thinks of idea 9, input command 18 of the first terminal application can be sent to the first terminal application, and if idea 9 is assigned as input command 18 to a second terminal application, then when patient 8 thinks of idea 9, input command 18 of the second terminal application can be sent to the second terminal application. Idea 9 can thus interface with or control multiple terminal applications 12, allowing idea 9 to function like a universal button (e.g., idea 9) on a universal controller (e.g., the patient's brain). Any number of ideas 9 can be used as switches. The number of ideas 9 used as switches can correspond to, for example, the number of controls (e.g., input commands 18) required or desired to control the terminal application 12.
[0080] Taking a video game controller as an example, the patient's idea 9 can be assigned to any input command 18 associated with any single button, any combination of buttons, and any directional movement (e.g., any directional movement of a joystick or control panel, such as a steering wheel), allowing the patient 8 to use their idea 9 to play any game on any video game system, whether or not a conventional physical controller is present. A video game system is just one example of terminal application 12. Module 10 enables the idea 9 to be assigned to any input command 18 of terminal application 12, allowing the patient's idea 9 to be mapped to controls of any software or device. Module 10 can thus organize the patient's idea 9 into a set of assignable switches that are essentially generic but specific in execution once assigned to an input command 18. Other exemplary examples of terminal application 12 include mobile devices (e.g., vehicles, wheelchairs, wheelchair lifts), prosthetics (e.g., prosthetic arms, prosthetic legs), telephones (e.g., smartphones), smart home appliances, and smart home systems.
[0081] The neural interface 14 can detect neurally relevant signals 17, including signals associated with thought 9 and signals not associated with thought 9. For example, the neural interface 14 may have one or more sensors that can detect (also referred to as acquire, sense, record, and measure) neurally relevant signals 17, including signals generated by the patient 8's biological mediators when the patient 8 thinks of thought 9, and signals generated by the patient 8's biological mediators that are not associated with thought 9 (e.g., forming a patient's response to stimuli not associated with thought 9). The sensors of the neural interface 14 can record signals from and / or stimulate the patient 8's biological mediators. Biological mediators can be, for example, neural tissue, vascular tissue, blood, bone, muscle, cerebrospinal fluid, or any combination thereof. Sensors can be, for example, electrodes, wherein electrodes can be any electrical conductor used to sense the electrical activity of the biological mediators. Sensors can be, for example, biochemical sensors. The neural interface 14 may have a single type of sensor (e.g., electrodes only) or multiple types of sensors (e.g., one or more electrodes and one or more biochemical sensors).
[0082] Neurally relevant signals can be any signal detectable from biological media (e.g., electrical signals, biochemical signals), any feature or multiple features extracted from detected neurally relevant signals (e.g., via a computer processor), or both, wherein the extracted features can be or may include characteristic information about the patient's thought 9, such that different thoughts 9 can be distinguished from each other. As another example, neurally relevant signals can be electrical signals, any signal caused by electrical signals (e.g., biochemical signals), any feature or multiple features extracted from detected neurally relevant signals (e.g., via a computer processor), or any combination thereof. Neurally relevant signals can be neural signals such as electroencephalograms (EEGs). When biological media are within the patient's skull, neurally relevant signals can be, for example, brain signals (e.g., detected from brain tissue) generated or caused by the patient's thought 9. In this way, neurally relevant signals can be brain-related signals, such as electrical signals from any part or multiple parts of the patient's brain (e.g., the motor cortex, sensory cortex). When the biological mediator is outside the patient's skull, the neurally relevant signal can be, for example, an electrical signal associated with muscle contraction (e.g., muscle contraction of body parts such as eyelids, eyes, nose, ears, fingers, arms, toes, and legs), generated or caused by the patient 8 thinking of thought 9. The thought 9 that the patient 8 thinks when the neurally relevant signal is detected from the patient 8's brain tissue (e.g., movement of a body part, memory, task) can be the same as or different from the thought 9 that the patient 8 thinks when the neurally relevant signal is detected from non-brain tissue. The neural interface 14 can be located inside the patient's brain, outside the patient's brain, or both.
[0083] Module 10 may include one or more neural interfaces 14, such as 1 to 10 or more neural interfaces 14, including each neural interface increment within this range (e.g., 1 neural interface, 2 neural interfaces, 10 neural interfaces), wherein each neural interface 14 may have one or more sensors (e.g., electrodes) configured to detect neurally related signals (e.g., neural signals). The location of the neural interface 14 may be selected to optimize the recording of neurally related signals, such as selecting the location with the strongest signal, the location with the least interference from noise, the location with the least trauma to the patient 8 caused by implantation or attachment of the neural interface 14 to the patient 8 (e.g., via surgery), or any combination thereof. For example, the neural interface 14 may be a brain-computer interface (such as an intravascular device (e.g., a stent)) having one or more electrodes for detecting brain electrical activity. In the case of using multiple neural interfaces 14, the neural interfaces 14 may be the same as or different from each other. For example, when using two neural interfaces 14, both neural interfaces 14 can be intravascular devices with electrodes (e.g., expandable and retractable stents with electrodes), or one of the neural interfaces 14 can be an intravascular device with electrodes, while the other of the two neural interfaces 14 can be a device with sensors, different from an intravascular device with electrodes.
[0084] Figure 1A and Figure 1B Further illustrating, module 10 may include a telemetry unit 22 adapted to communicate with neural interface 14, and a communication conduit 24 (e.g., a wire) facilitating communication between neural interface 14 and telemetry unit 22. Host device 16 may be adapted to perform wired and / or wireless communication with telemetry unit 22.
[0085] Figure 1A and Figure 1B The telemetry unit 22 may further include an internal telemetry unit 22a and an external telemetry unit 22b. The internal telemetry unit 22a can communicate with the external telemetry unit 22b wirelessly, either via wired or wireless communication. For example, the external telemetry unit 22b can be wirelessly connected to the internal telemetry unit 22a through the patient's skin. The internal telemetry unit 22a can communicate wirelessly or via wired communication with the neural interface 14, and the neural interface 14 can be electrically connected to the internal telemetry unit 22a via a communication conduit 24. The communication conduit 24 can be, for example, a wire, such as a stent lead.
[0086] Module 10 may have a processor (also called a processing unit) that can analyze and decode the neural signals detected by neural interface 14. The processor may be a computer processor (e.g., a microprocessor). The processor may apply mathematical algorithms or models to detect neural signals corresponding to when the patient 8 generates thought 9. For example, once neural interface 14 senses neural signals 17, the processor may apply mathematical algorithms or models to detect, decode, and / or classify the sensed neural signals 17. As another example, once neural interface 14 senses neural signals 17, the processor may apply mathematical algorithms or models to detect, decode, and / or classify information in the sensed neural signals 17. Once the neural signals 17 detected by neural interface 14 are processed by the processor, the processor may associate the processed information (e.g., the detected, decoded, and / or classified neural signals 17 and / or the information on the detection, decoding, and / or classification of sensed neural signals 17) with input commands 18 of terminal application 12.
[0087] Neural interface 14, host device 16, and / or telemetry unit 22 may have processors. As another example, neural interface 14, host device 16, and / or telemetry unit 22 may have processors (e.g., processors as described above). For example, host device 16 may analyze and decode neurally related signals 17 detected by neural interface 14 via the processor. Neural interface 14 may communicate with host device 16 via wired or wireless communication, and host device 16 may communicate with terminal application 12 via wired or wireless communication. As another example, neural interface 14 may communicate with telemetry unit 22 via wired or wireless communication, telemetry unit 22 may communicate with host device 16 via wired or wireless communication, and host device 16 may communicate with terminal application 12 via wired or wireless communication. For example, data may be transferred from neural interface 14 to telemetry unit 22, from telemetry unit 22 to host device 16, from host device 16 to one or more terminal applications 12, or any combination thereof, to detect idea 9 and trigger input command 18. As another example, data can be transmitted in reverse order, for example, from one or more terminal applications 12 to host device 16, from host device 16 to telemetry unit 22, from telemetry unit 22 to neural interface 14, or any combination thereof, for example, to stimulate biological mediators via one or more sensors. The data can be data collected or processed by a processor, including, for example, neurally related signals and / or features extracted therefrom. When data flows, for example, from the processor to the sensors, the data can include stimulation instructions such that when neural interface 14 processes the stimulation instructions, the sensors of the neural interface can stimulate the biological mediators.
[0088] Figure 1A and Figure 1BFurther illustrating, when patient 8 conceives thought 9, the patient 8's biological media (e.g., intracranial, extracranial, or both) can generate neurally related signals 17 detectable by neural interface 14. When patient 8 conceives thought 9, sensors of neural interface 14 can detect the neurally related signals 17 associated with thought 9. The neurally related signals 17 associated with thought 9, features extracted from these neurally related signals 17, or both, can be assigned to or associated with any input command 18 for any terminal application 12 controllable by the universal switch module 10. Therefore, each of the detectable neurally related signals 17 and / or their extracted features can advantageously be used as a universal switch, assignable to any input command 18 of any terminal application 12. In this way, when neural interface 14 detects thought 9, the input command 18 associated with thought 9 can be triggered and sent to the terminal application 12 associated with the triggered input command 18.
[0089] For example, when neural interface 14 detects idea 9 (e.g., through sensed neural correlation signals 17), the processor can analyze (e.g., detect, decode, classify, or any combination thereof) the sensed neural correlation signals 17 and associate the sensed neural correlation signals 17 and / or features extracted from them with a correspondingly assigned input command 18. The processor can thus determine whether idea 9 (e.g., the sensed neural correlation signals 17 and / or features extracted from them) is associated with any input command 18. When it is determined that idea 9 is associated with input command 18, the processor or controller can activate (also called trigger) input command 18. Once input command 18 is triggered by module 10 (e.g., by the processor or controller of host device 16), the triggered input command 18 can be sent to its corresponding terminal application 12, such that terminal application 12 (e.g., a wheelchair, prosthetic arm, smart home appliance such as a coffee machine) can be controlled using the triggered input command 18. Once the terminal application 12 receives the triggered input command 18, it can execute one or more instructions of the input command 18 (e.g., moving the wheelchair forward at a speed of 1 meter per second, pinching the thumb and forefinger of the prosthetic arm together, or turning on the smart coffee machine). Therefore, when it is determined that the idea 9 (e.g., the sensed neural signals 17 and / or features extracted from them) is associated with the input command 18, the input command 18 can be sent to its corresponding terminal application 12.
[0090] The extracted features can be components of the sensed neural signal 17, including, for example, voltage fluctuation patterns in the sensed neural signal 17, power fluctuations embedded in a specific frequency band within the sensed neural signal 17, or both. For example, the neural signal 17 can have different ranges of oscillation frequencies corresponding to the time when the patient 8 thinks of thought 9. The frequencies of a specific frequency band can contain specific information. For example, high-frequency bands (e.g., 65Hz-150Hz) can contain information associated with motor-related thoughts, and therefore, features in that high-frequency band range (e.g., extracted or identified from the sensed neural signal 17) can be used to classify and / or decode neural events (e.g., thought 9).
[0091] Idea 9 can be a universal switch. Idea 9 can act as (e.g., be used as) a universal switch, where idea 9 can be assigned to any input command 18 and vice versa. Idea 9 can be assigned to or associated with any input command 18 of any terminal application 12 controlled by the universal switch module 10 through detectable neural correlation signals 17 and / or features that can be extracted from neural correlation signals 17. Patient 8 can activate a desired input command 18 by thinking of idea 9 associated with the input command 18 desired by patient 8. For example, when neural interface 14 detects idea 9 (e.g., memory of the patient's 9th birthday party) assigned to a specific input command 18 (e.g., moving the wheelchair forward), a processor (e.g., of host device 16) can associate the neural correlation signals 17 associated with the idea (e.g., memory of the 9th birthday party) and / or features extracted from it with the corresponding assigned input command 18 (e.g., moving the wheelchair forward). When a detected neural-related signal (e.g., and / or its associated extracted features) is associated with an assigned input command 18, the host device 16 can send the input command 18 to the terminal application 12 associated with the input command 18 via a processor or controller to control the terminal application 12 using the input command 18, which is triggered by the patient 8 by thinking of a thought 9.
[0092] When idea 9 is assigned to multiple terminal applications 12 and only one of the terminal applications 12 is active (e.g., powered on and / or running), host device 16 can send a triggered input command 18 to the active terminal application 12. As another example, when idea 9 is assigned to multiple terminal applications 12 and some of the terminal applications 12 are active (e.g., powered on or running) and some of the terminal applications 12 are inactive (e.g., powered off or in standby mode), host device 16 can send triggered input commands 18 to both the active and inactive terminal applications 12. When input command 18 is received by an active terminal application 12, the active terminal application 12 can execute input command 18. Inactive terminal applications 12 can execute input command 18 when the inactive application 12 becomes active (e.g., powered on or started running), or input command 18 can be placed in a queue (e.g., by module 10 or by terminal application 12) to be executed when the inactive application 12 becomes active. As yet another example, in the case where idea 9 is assigned to multiple terminal applications 12 and more than one of the terminal applications 12 is active (e.g., powered on and / or running) (e.g., the first terminal application and the second terminal application are active), host device 16 may send a triggered input command 18 associated with the first terminal application to the first terminal application, and may send a triggered input command 18 associated with the second terminal application to the second terminal application, or module 10 may give patient 8 the option of which of the triggered input commands 18 patient 8 wants to send (e.g., send only the triggered input command 18 associated with the first terminal application, send only the triggered input command 18 associated with the second terminal application, or send both triggered input commands 18).
[0093] Idea 9 can be any idea or combination of ideas. For example, idea 9 conceived by patient 8 can be a single idea, multiple ideas, consecutive multiple ideas, simultaneous multiple ideas, ideas with different durations, ideas with different frequencies, one or more ideas in sequence, one or more combined ideas, or any combination thereof. Idea 9 can be a task-related idea, a task-independent idea, or both, wherein a task-related idea is related to patient 8's expected task, while a task-independent idea is unrelated to patient 8's expected task. For example, idea 9 can be of the first task, and patient 8 can conceive of the first task to accomplish the second task (also referred to as the expected task and the target task), for example, by using module 10. The first task can be the same as or different from the second task. When the first task is the same as the second task, idea 9 can be a task-related idea. When the first task is different from the second task, idea 9 can be a task-independent idea. For example, if the first task conceived by patient 8 is to move a limb (e.g., arm, leg) and the second task is the same as the first task, i.e., moving a limb (e.g., arm, leg) using a prosthetic limb, idea 9 (e.g., of the first task) can be a task-related idea. The prosthetic limb can be, for example, a terminal application 12 controlled by patient 8 using idea 9. For example, for a task-related thought, when the target task is to move the cursor, patient 8 might think of moving the cursor. In contrast, for a task-independent thought 9, where patient 8 thinks of moving a limb (e.g., the arm) as the first task, the second task can be any task different from the first task of moving the limb (e.g., the arm), such that the second task can be any task of the terminal application 12 different from the first task. For example, for a task-independent thought, when the target task is to move the cursor to the right, patient 8 might think of moving a body part (e.g., their hand) to the right. Patient 8 can therefore think of a first task (e.g., thought 9) to accomplish any second task, where the second task can be the same as or different from the first task. The second task can be any task of any terminal application 12. For example, the second task can be any input command 18 of any terminal application 12. Thought 9 (e.g., the first task) can be assignable to any second task. Thought 9 (e.g., the first task) can be assigned to any second task. Patient 8 can thus think of a first task that triggers any input command 18 of any terminal application 12 (e.g., any second task). Therefore, the first task can advantageously be used as a universal switch. Each thought 9 can generate a repeatable neurally relevant signal (e.g., a detectable neurally relevant signal 17) detectable by neural interface 14. Each detectable neurally relevant signal and / or feature that can be extracted from it can be a switch. For example, a switch can be activated (also referred to as being triggered) when the patient 8 thinks of thought 9 and the sensor detects that the switch is activated and / or the processor determines that one or more features extracted from the detected neurally relevant signal are present.The switch can be a general-purpose switch, capable of being assigned and reassigned to any input command 18, such as any set of input commands. Input commands 18 can be added to, removed from, and / or modified from any set of input commands. For example, each terminal application 12 can have a set of input commands 18 associated with it, and the neural-related signal 17 of idea 9 can be assigned to that set of input commands 18.
[0094] Some ideas 9 can be task-independent ideas (e.g., patient 8 tries to move their hand to move the cursor to the right), some ideas 9 can be task-related ideas (e.g., patient 8 tries to move the cursor when the target task is to move the cursor), some ideas 9 can be both task-independent and task-related ideas, or any combination thereof. In the case where idea 9 is both a task-independent and task-related idea, idea 9 can be used both as a task-independent idea (e.g., patient 8 tries to move their hand to move the cursor to the right) and as a task-related idea (e.g., patient tries to move the cursor when the target task is to move the cursor), such that idea 9 can be associated with a plurality of input commands 18, wherein one or more of these input commands 18 can be task-related to idea 9, and wherein one or more of these input commands 18 can be task-independent of idea 9.
[0095] In this way, idea 9 can be a universal switch that can be assigned to any input command 18 of any terminal application 12, wherein each idea 9 can be assigned to one or more terminal applications 12. Module 10 advantageously enables each patient 8 to use their ideas 9 to control any terminal application 12 that the patient 8 wants to control, just like a button on a controller (e.g., a video game controller, any control interface). For example, idea 9 can be assigned to each input command 18 of the terminal application 12, and the assigned input commands 18 can be used in any combination, such as a button on a controller, to control the terminal application 12. For example, in the case where terminal application 12 has four input commands 18 (e.g., like four buttons on a controller - a first input command, a second input command, a third input command, and a fourth input command), different ideas 9 can be assigned to each of the four input commands 18 (e.g., a first idea 9 can be assigned to a first input command, a second idea 9 can be assigned to a second input command, a third idea 9 can be assigned to a third input command, and a fourth idea 9 can be assigned to a fourth input command), so that patient 8 can use these four ideas 9 to activate the four input commands 18 and their combinations (e.g., any order, number, frequency, and duration of the four input commands 18) to control terminal application 12. For example, for terminal application 12 with four input commands 18, the four input commands 18 can be used to control terminal application 12 using any combination of the four ideas 9 assigned to the first, second, third, and fourth input commands, including, for example, a single activation of each input command itself, multiple activations of each input command itself (e.g., two activations of less than 5 seconds, three activations of less than 10 seconds), combinations of multiple input commands 18 (e.g., simultaneous or consecutive first and second input commands), or any combination thereof. Like each individual idea 9, each combination of ideas 9 can act as a universal switch. Patient 8 can control multiple terminal applications 12 using first, second, third, and fourth ideas 9. For example, first idea 9 can be assigned to a first input command of first terminal application 12, first idea 9 can be assigned to a first input command of second terminal application 12, second idea 9 can be assigned to a second input command of first terminal application 12, second idea 9 can be assigned to a second input command of second terminal application 12, third idea 9 can be assigned to a third input command of first terminal application 12, third idea 9 can be assigned to a third input command of second terminal application 12, fourth idea 9 can be assigned to a fourth input command of first terminal application 12, fourth idea 9 can be assigned to a fourth input command of second terminal application 12, or any combination thereof.For example, a first idea 9 can be assigned to a first input command of a first terminal application 12 and a first input command of a second terminal application 12; a second idea 9 can be assigned to a second input command of a first terminal application 12 and a second input command of a second terminal application 12; a third idea 9 can be assigned to a third input command of a first terminal application 12 and a third input command of a second terminal application 12; a fourth idea 9 can be assigned to a fourth input command of a first terminal application 12 and a fourth input command of a second terminal application 12, or any combination thereof. The first, second, third, and fourth ideas 9 can be assigned to any application 12 (e.g., the first and second terminal applications). Some ideas can be assigned to only a single application 12, while some ideas can be assigned to multiple applications 12. Even in the case where idea 9 is assigned to only a single application 12, an idea assigned to only one application 12 can be assigned to multiple applications 12, allowing the patient 8 to utilize the general applicability of idea 9 as needed or desired (e.g., an idea assigned to only one terminal application 12). As another example, all ideas 9 can be assigned to multiple terminal applications 12.
[0096] The functionality of each input command 18 or combination of input commands for terminal application 12 can be defined by patient 8. As another example, the functionality of each input command 18 or combination of input commands for terminal application 12 can be defined by terminal application 12, allowing third parties to insert and make their terminal application's input commands 18 assignable (also mappable) to a set or subset of the patient's repeatable ideas 9. This can advantageously allow third-party programs to be more easily accessed and customized according to the different expectations, needs, and abilities of different patients 8. Module 10 can advantageously be an application programming interface (API) with which third parties can interface and allow patient 8's ideas 9 to be assigned and reassigned to various input commands 18, wherein, as described herein, each input command 18 can be activated by the idea 9 that patient 8 wants to assign to the input command 18 that patient 8 wishes to activate.
[0097] The patient's thought 9 can be assigned to the input command 18 of the terminal application 12 via a person (e.g., the patient or another person), a computer, or both. For example, the patient's thought 9 (e.g., detectable neural signals and / or extractable features associated with thought 9) can be assigned to the input command 18 by the patient, can be assigned by a computer algorithm (e.g., based on the signal strength of the detectable neural signals associated with thought 9), can be altered by the patient (e.g., reassigned), can be altered by an algorithm (e.g., based on the relative signal strength of a switch or the availability of a new, repeatable thought 9), or any combination thereof. The input command 18 and / or the function associated with the input command 18 can be, but does not necessarily have to be, unrelated to the thought 9 associated with activating the input command 18. For example, Figures 1A-1CAn exemplary variant of a non-specific or generic mode-switching procedure (e.g., an application programming interface (API)) that can be plugged in by a third party is shown, which allows idea 9 (e.g., detectable neurally related signals and / or extractable features associated with idea 9) to be assigned and reassigned to various input commands 18. By assigning input commands 18, idea 9 is assigned, and vice versa, patient 8 can use the same idea 9 with various input commands 18 in the same or different terminal applications 12. Similarly, by reassigning input commands 18, idea 9 is assigned, and vice versa, patient 8 can use the same idea 9 with various input commands 18 in the same or different terminal applications 12. For example, idea 9 assigned to an input command 18 that causes a prosthetic hand (e.g., a first terminal application) to open can be assigned to a different input command 18 that causes a cursor (e.g., a second terminal application) to do something on a computer (e.g., any function associated with a cursor associated with a computer mouse or touchpad, including, for example, cursor movement and selection using the cursor, such as left-click and right-click).
[0098] Figures 1A-1C It is further illustrated that the patient's thought 9 can be assigned to multiple terminal applications 12, allowing the patient to switch between multiple terminal applications 12 without having to reassign the input command 18 each time the patient uses a different terminal application 12. For example, the thought 9 can be assigned to multiple terminal applications 12 simultaneously (e.g., to a first terminal application and a second terminal application, wherein the process of assigning the thought 9 to both the first and second terminal applications can, but does not need to, occur simultaneously). Thus, the patient's thought 9 can advantageously control any terminal application 12, including, for example, external gaming devices or various household appliances and devices (e.g., light switches, appliances, locks, thermostats, security systems, garage doors, windows, curtains, including any smart devices or systems, etc.). The neural interface 14 can thus detect functionally task-independent neural signals 17 (e.g., brain signals) associated with the input command 18 of the terminal application 12, wherein the terminal application 12 can be any electronic device or software, including devices inside and / or outside the patient's body. As another example, neural interface 14 can thus detect neurally relevant signals 17 (e.g., brain signals) associated with a functional task-related input command 18 of terminal application 12, wherein terminal application 12 can be any electronic device or software, including devices inside and / or outside the patient's body. As yet another example, neural interface 14 can thus detect neurally relevant signals 17 (e.g., brain signals) associated with task-related thoughts, task-independent thoughts, or both task-related and task-independent thoughts.
[0099] Some ideas 9 can be task-independent ideas (e.g., patient 8 tries to move their hand to move the cursor to the right), some ideas 9 can be task-related ideas (e.g., patient 8 tries to move the cursor when the target task is to move the cursor), some ideas 9 can be both task-independent and task-related ideas, or any combination thereof. Where an idea 9 is both a task-independent and task-related idea, it can function as both a task-independent idea (e.g., patient 8 tries to move their hand to move the cursor to the right) and a task-related idea (e.g., patient tries to move the cursor when the target task is to move the cursor), such that idea 9 can be associated with a plurality of input commands 18, wherein one or more of these input commands 18 can be task-related to idea 9, and wherein one or more of these input commands 18 can be task-independent of idea 9. As another example, all ideas 9 can be task-independent ideas. The task-independent idea 9 and / or the idea 9 used by the patient 8 as a task-independent idea (e.g., the idea 9 assigned to an input command 18 that is independent of idea 9) enable the patient 8 (e.g., a BCI user) to independently control various terminal applications 12, including software and devices, using the given task-independent idea (e.g., idea 9).
[0100] Figures 1A-1CThe illustration shows, for example, a patient 8 who can think of idea 9 (e.g., with or without being asked to think of idea 9) and then rest. This task of thinking of idea 9 can generate detectable neurally related signals corresponding to the idea 9 that the patient is thinking of. The task of thinking of idea 9 and then resting can be performed once, for example, when patient 8 thinks of idea 9 to control terminal application 12. As another example, the task of thinking of idea 9 can be repeated multiple times, for example, when patient 8 controls terminal application 12 by thinking of idea 9, or when the patient is training how to use idea 9 to control terminal application 12. When recording neurally related signals (e.g., brain-related signals) (such as neural signals), features can be extracted (e.g., spectral power / time-frequency domain) or identified from the signals themselves (e.g., time-domain signals). These features can contain characteristic information about idea 9 and can be used to identify idea 9, distinguish multiple ideas 9 from one another, or both. As another example, these features can be used to develop or train mathematical models or algorithms that can use machine learning methods and other methods to predict the types of ideas that generate neural signals. Using the algorithm and / or model, what patient 8 is thinking can be predicted in real time, and this prediction can be associated with any desired input command 18. The process of patient 8 thinking the same thought 9 can be repeated, for example, until the prediction provided by the algorithm and / or model matches patient 8's thought 9. In this way, patient 8 can calibrate each of their thoughts 9 that they will use to control the terminal application 12, such that each thought 9 assigned to the input command 18 generates a repeatable neurally relevant signal detectable by the neural interface 14. The algorithm can provide patient 8 with feedback 19 regarding whether the prediction matches the actual thought 9 they should be thinking, where this feedback can be visual, auditory, and / or tactile, which can induce patient 8 to learn through trial and error. Feedback 19 can also be feedback in the form of neural stimulation. Machine learning methods and mathematical algorithms can be used to classify thoughts 9 based on features extracted and / or identified from the sensed neurally relevant signals 17. For example, a training dataset can be recorded in which patient 8 rests and thinks multiple times. The processor can extract relevant features from the sensed neural signals 17, and the parameters and hyperparameters of a mathematical model or algorithm based on this data to distinguish between rest and thinking can be optimized to predict real-time signals. The same mathematical model or algorithm, already tuned to predict real-time signals, then advantageously allows module 10 to convert idea 9 into a real-time universal switch.
[0101] Figure 1A The neural interface 14 is further shown to monitor biological mediators (e.g., the brain), such as electrical signals from the monitored tissue (e.g., neural tissue). Figure 1AIt is further illustrated that the neurally related signal 17 can be a brain-related signal. A brain-related signal can be, for example, an electrical signal originating from any or more parts of the patient's brain (e.g., the motor cortex, sensory cortex). As another example, a brain-related signal can be any signal detectable in the skull (e.g., an electrical signal, a biochemical signal), any feature or features extracted (e.g., via a computer processor) from a detected brain-related signal, or both. As yet another example, a brain-related signal can be an electrical signal, any signal caused by an electrical signal (e.g., a biochemical signal), any feature or features extracted (e.g., via a computer processor) from a detected brain-related signal, or any combination thereof.
[0102] Figure 1A The diagram further illustrates that the terminal application 12 can be detached from module 10, but can be connected to module 10 via wired or wireless communication. As another example, module 10 (e.g., host device 16) can be permanently or removably attached to or attachable to the terminal application 12. For example, host device 16 can be removably docked with application 12 (e.g., a device with software that module 10 can communicate with). Host device 16 can have a port that can be docked with application 12, or vice versa. The port can be a charging port, a data port, or both. For example, in the case where the host device is a smartphone, the port could be a lighting port. As yet another example, host device 16 can have a tethered connection to application 12, such as a tethered connection using a cable. The cable can be a power cable, a data transmission cable, or both.
[0103] Figure 1B Furthermore, it is shown that when patient 8 thinks of thought 9, the neural signal 17 can be a brain-related signal corresponding to thought 9. Figure 1B The host device 16 may further illustrate that it has a processor (e.g., a microprocessor) that analyzes (e.g., detects, decodes, classifies, or any combination thereof) the neural-related signals 17 received from the neural interface 14, associates the neural-related signals 17 received from the neural interface 14 with their corresponding input commands 18, associates features extracted from (e.g., in the spectral power / time-frequency domain) or identified in the neural-related signals 17 themselves (e.g., in the time domain) received from the neural interface 14 with their corresponding input commands 18, saves the neural-related signals 17 received from the neural interface 14, saves the signal analysis (e.g., features extracted or identified from the neural-related signals 17), saves the association between the neural-related signals 17 and the input commands 18, saves the association between the features extracted or identified from the neural-related signals 17 and the input commands 18, or any combination thereof.
[0104] Figure 1BThe host device 16 is further shown to have memory. Data saved by the processor can be stored locally in memory, stored on a server (e.g., in the cloud), or both. Idea 9 and the resulting data (e.g., detected neural-related signals 17, extracted features, or both) can be used as a reference library. For example, once idea 9 is calibrated, neural-related signals 17 associated with the calibrated idea and / or its signature (also referred to as extracted) features can be saved. For example, when neural interface 14 detects neural-related signals 17, idea 9 can be considered calibrated when neural-related signals 17 and / or features extracted from them have a repeatable signature or feature recognizable by the processor. Neural-related signals monitored and detected in real time can then be compared with this real-time stored calibration data. Whenever one of the detected signals 17 and / or its extracted features matches a calibration signal, a corresponding input command 18 associated with the calibration signal can be sent to the corresponding terminal application 12. For example, Figure 1A and Figure 1B The module 10 demonstrates how patient 8 can be trained using the neural signals 17 associated with their idea 9 and these calibrations stored in a reference library. The training can provide feedback to patient 8 19.
[0105] Figure 1C An exemplary user interface 20 of the host device 16 is further shown. The user interface 20 may be a computer screen (e.g., a touch screen, a non-touch screen). Figure 1CAn exemplary display of user interface 20 is shown, including selectable systems 13, selectable input commands 18, and selectable terminal applications 12. Systems 13 may be groups of one or more terminal applications 12. Systems 13 can be added to and removed from host device 16. Terminal applications 12 can be added to and removed from host device 16. Terminal applications 12 can be added to and removed from system 13. Each system 13 may have a corresponding set of input commands 18, which can be assigned to a corresponding set of terminal applications 12. As another example, user interface 20 may display input commands 18 for each of the active terminal applications 12 (e.g., a remote control). As yet another example, user interface 20 may display input commands 18 for active terminal applications (e.g., a remote control) and / or for deactivated terminal applications 12 (e.g., a stimulation cannula, a telephone, a smart home device, a wheelchair). This advantageously allows module 10 to control any terminal application 12. User interface 20 allows ideas 9 to be easily assigned to various input commands 18 of multiple terminal applications 12. The system grouping of terminal applications (e.g., system 1 and system 2) advantageously allows patient 8 to organize terminal applications 12 together using user interface 20. Existing systems 13 can be uploaded to the module and / or patient 8 can create their own systems 13. For example, a first system could have all terminal applications 12 used by patient 8 related to mobility (e.g., wheelchair, wheelchair lift). As another example, a second system could have all terminal applications 12 used by patient 8 related to prosthetics. As yet another example, a third system could have all terminal applications 12 used by patient 8 related to smart home appliances. As yet another example, a fourth system could have all terminal applications 12 used by patient 8 associated with software or devices used by the patient for their occupation. Terminal applications 12 can be in one or more systems 13. For example, terminal application 12 (e.g., wheelchair) can be in system 1 and / or system 2. This organizational efficiency makes it easier for patient 8 to manage their terminal applications 12. Module 10 may have one or more systems 13, for example, 1 to 1000 or more systems 13, including every single increment of systems 13 within this range (e.g., 1 system, 2 systems, 10 systems, 100 systems, 500 systems, 1000 systems, 1005 systems, 2000 systems). For example, Figure 1C It is shown that module 10 may have a first system 13a (e.g., system 1) and a second system 13b (e.g., system 2). Furthermore, although... Figure 1C The diagram shows that terminal applications 12 can be grouped into various systems 13, with each system having one or more terminal applications 12. However, as another example, user interface 20 may not group terminal applications into systems 13.
[0106] Figure 1C The host device 16 is further shown to be able to assign idea 9 to input command 18. For example, idea 9, neural-related signal 17 associated with idea 9, extracted features of neural-related signal 17 associated with idea 9, or any combination thereof, can be assigned to input command 18 of system 13, for example by selecting input command 18 (e.g., left arrow) and from a drop-down menu displaying idea 9 and / or data associated with it (e.g., neural-related signal 17 associated with idea 9, extracted features of neural-related signal 17 associated with idea 9, or both) that can be assigned to the selected input command 18. Figure 1C It further illustrates that when input command 18 is triggered by thought 9 or associated data, feedback (e.g., visual, auditory, tactile and / or neurostimulation feedback) can be provided to patient 8. Figure 1C Further illustration shows that one or more terminal applications 12 can be activated and deactivated in system 13. Activated terminal application 12 can be in a powered-on, powered-off, or standby state. Activated terminal application 12 can receive triggered input commands 18. Deactivated terminal application 12 can be in a powered-on, powered-off, or standby state. In one example, unless terminal application 12 is activated, deactivated terminal application 12 may not be controllable by the patient 8's thoughts 9. Activating terminal application 12 using user interface 20 can power on terminal application 12. Deactivating terminal application 12 using user interface 20 can power off deactivated terminal application 12, or otherwise disengage module 10 from deactivated terminal application 12 such that the processor does not associate neurally related signals 17 with thoughts 9 assigned to deactivated terminal application 12. For example, Figure 1C An exemplary system 1 with five terminal applications 12 is shown, wherein the five terminal applications include five devices (e.g., remote control, stimulation cannula, telephone, smart home device, wheelchair), one of which (e.g., remote control) is activated while the others are deactivated. Once “Start” is selected (e.g., via icon 20a), the patient 8 can control the terminal application 12 of the system (e.g., system 1), which is activated using input commands 18 associated with the terminal application 12 of system 1 (e.g., remote control). Figure 1C It is further shown that any changes made using the user interface 20 can be saved using the save icon 20b, and any changes made using the user interface 20 can be canceled using the cancel icon 20c. Figure 1C This further illustrates that the terminal application 12 can be an electronic device.
[0107] Figures 1A-1CThe same specific set of ideas 9 is shown to be used to control multiple terminal applications 12 (e.g., multiple terminal devices), thus making module 10 a universal switch module. Module 10 advantageously allows a patient 8 (e.g., a BCI user) to independently control various terminal applications 12, including, for example, multiple software and devices, using a given task-independent idea (e.g., idea 9). Module 10 can (e.g., via neural interface 14) acquire neurally relevant signals, can (e.g., via a processor) decode the acquired neurally relevant signals, can (e.g., via a processor) associate the acquired neurally relevant signals 17 and / or features extracted from these signals with corresponding input commands 18 of one or more terminal applications 12, and can (e.g., via module 10) control multiple terminal applications 12. Using module 10, idea 9 can be advantageously used to control multiple terminal applications 12. For example, module 10 can be used to control multiple terminal applications 12, wherein a single terminal application 12 can be controlled at a time. As another example, module 10 can be used to control multiple terminal applications simultaneously. Each idea 9 can be assigned to input commands 18 of multiple applications 12. In this way, idea 9 can act as a universal digital switch, where module 10 can effectively reorganize the patient's motor cortex to represent digital switches, wherein each idea 9 can be a digital switch. These digital switches can be universal switches used by patient 8 to control multiple terminal applications 12, because each switch (e.g., via module 10) can be assigned to any input command 18 of multiple terminal applications 12 (e.g., input command of the first terminal application and input command of the second terminal application). Module 10 can distinguish different ideas 9 (e.g., different switches) via a processor.
[0108] Module 10 can interface with, for example, 1 to 1000 or more terminal applications 12, including each increment of terminal applications 12 within this range (e.g., 1 terminal application, 2 terminal applications, 10 terminal applications, 100 terminal applications, 500 terminal applications, 1000 terminal applications, 1005 terminal applications, 2000 terminal applications). For example, Figure 1C The first system 13a is shown to have a first terminal application 12a (e.g., a remote control), a second terminal application 12b (e.g., a stimulation cannula), a third terminal application 12c (e.g., a telephone), a fourth terminal application 12d (e.g., a smart home device) and a fifth terminal application 12e (e.g., a wheelchair).
[0109] Each terminal application may have, for example, 1 to 1000 or more input commands 18 that can be associated with the thought 9 of patient 8, or, as another example, 1 to 500 or more input commands 18 that can be associated with the thought 9 of patient 8, or, as yet another example, 1 to 100 or more input commands 18 that can be associated with the thought 9 of patient 8, including every single input command 18 within these ranges (e.g., 1 input command, 2 input commands, 10 input commands, 100 input commands, 500 input commands, 1000 input commands, 1005 input commands, 2000 input commands), and including any subranges within these ranges (e.g., 1 to 25 or fewer input commands 18, 1 to 100 or fewer input commands 18, 25 to 1000 or fewer input commands 18), such that any number of input commands 18 can be triggered by the thought 9 of patient 8, where any number can be, for example, the number of input commands 18 assigned to the thought 9 of patient 8. For example, Figure 1C An exemplary set of input commands 18 associated with an active terminal application 12 (e.g., a first terminal application 12a) is shown, including a first terminal application first input command 18a (e.g., left arrow), a first terminal application second input command 18b (e.g., right arrow), and a first terminal application third input command 18c (e.g., Enter). As another example, Figure 1C An exemplary set of input commands 18 associated with a deactivated terminal application 12 (e.g., a second terminal application 12b) is shown, including a fourth input command 18d of the second terminal application (e.g., selecting an output), wherein the fourth input command 18d of the second terminal application has not yet been selected, but can be any input command 18 of the second terminal application 12b. A first input command 18a of the first terminal application is also referred to as a first input command 18a of the first terminal application 12a. A second input command 18b of the first terminal application is also referred to as a second input command 18b of the first terminal application 12a. A third input command 18c of the first terminal application is also referred to as a third input command 18c of the first terminal application 12a. A fourth input command 18d of the second terminal application is also referred to as a fourth input command 18d of the second terminal application 12b.
[0110] When patient 8 thinks of idea 9, module 10 (e.g., via a processor) can associate the neurally related signals 17 associated with idea 9 and / or features extracted from them with the input command 18 assigned to idea 9, and the input command 18 associated with idea 9 can be sent by module 10 (e.g., via a processor, controller, or transceiver) to their respective terminal applications 12. For example, if idea 9 is assigned as a first input command 18a to a first terminal application 12a, then when patient 8 thinks of idea 9, the first input command 18a of the first terminal application 12a can be sent to the first terminal application 12a, and if idea 9 is assigned as a fourth input command 18d to a second terminal application 12b, then when patient 8 thinks of idea 9, the fourth input command 18d of the second terminal application 12b can be sent to the second terminal application 12b. Therefore, a single idea (e.g., idea 9) can interface with or control multiple terminal applications 12 (first terminal application 12a and second terminal application 12b). Any number of ideas 9 can be used as switches. The number of ideas 9 used as switches can correspond to, for example, the number of controls (e.g., input commands 18) required or desired by the control terminal application 12. Ideas 9 can be assigned to multiple terminal applications 12. For example, a neural-related signal 17 associated with a first idea and / or features extracted from it can be assigned to a first input command 18a of a first terminal application, and can be assigned to a fourth input command 18d of a second terminal application. As another example, a neural-related signal 17 associated with a second idea and / or features extracted from it can be assigned to a second input command 18b of a first terminal application, and can be assigned to a first input command of a third terminal application. The first idea can be different from the second idea. Multiple terminal applications 12 (e.g., first terminal application 12a and second terminal application 12b) can operate independently of each other. In the case where module 10 is used to control a single terminal application (e.g., first terminal application 12a), the first idea can be assigned to multiple input commands 18. For example, a single first idea can activate a first input command, and the first idea together with a second idea can activate a second input command different from the first input command. Idea 9 can therefore act as a universal switch, even when only a single terminal application 12 is controlled by module 10, because a single idea can be combined with other ideas to form an additional switch. As another example, a single idea can be combined with other ideas to form an additional universal switch that can be assigned to any input command 18, where multiple terminal applications 12 can be controlled by module 10 via idea 9.
[0111] Figure 2 A- Figure 2 D shows that the neural interface 14 can be a scaffold 101. The scaffold 101 can have struts 108 and sensors 131 (e.g., electrodes). The scaffold 101 can be retractable and expandable.
[0112] Figure 2 A- Figure 2 D further illustrates that the stent 101 can be implanted into the vascular system of a subject, such as a blood vessel passing through the subject's sinus or vein. As a more specific example, the stent 101 can be implanted into the superior sagittal sinus of a subject. Figure 2 A illustrates an exemplary module 10, and Figure 2 B- Figure 2 D shows Figure 2 Three magnified views of module 10 of A. The scaffold 101 can be implanted, for example via the jugular vein, into the superior sagittal sinus (SSS) covering the primary motor cortex to passively record brain signals and / or stimulate tissue. The scaffold 101 can detect, for example, neurally related signals 17 associated with thought 9 via sensor 131, enabling paralyzed individuals due to nerve damage or disease to communicate, improve mobility, and potentially achieve independence through direct brain control of assistive technologies such as terminal application 12. Figure 2 C illustrates a communication conduit 24 (e.g., a stent lead) that can extend from the stent 101, pass through the jugular vein wall, and tunnel under the skin to the subclavian bursa. In this way, the communication conduit 24 facilitates communication between the stent 101 and the telemetry unit 22.
[0113] Figure 2 A- Figure 2 D further illustrates that terminal application 12 can be a wheelchair.
[0114] Figure 3 The neural interface 14 (e.g., scaffold 101) is shown to be a wireless sensor system 30 that can communicate wirelessly with the host device 16 (e.g., without telemetry unit 22). Figure 3 A stent 101 is shown within a blood vessel 104, which covers the motor cortex of the patient 8. The motor cortex picks up neural signals and relays this information to a wireless transmitter 32 located on the stent 101. The neural signals recorded by the stent 101 can be wirelessly transmitted through the patient's skull to a wireless transceiver 34 (e.g., placed on the head). The wireless transceiver 34 then decodes the acquired neural signals and transmits them to a host device 16. As another example, the wireless transceiver 34 may be part of the host device 16.
[0115] Figure 3 This further illustrates that terminal application 12 can be a prosthetic arm.
[0116] Figure 4A neural interface 14 (e.g., scaffold 101) is shown to be used to record neurally related signals 17 from the brain, such as neurally related signals 17 from neurons in the superior sagittal sinus (SSS) or branch cortical veins, including the steps of: (a) implanting the neural interface 14 into a blood vessel 104 in the brain (e.g., superior sagittal sinus, branch cortical vein); (b) recording neurally related signals; (c) generating data representing the recorded neurally related signals; and (d) transmitting the data to a host device 16 (e.g., with or without telemetry unit 22).
[0117] All contents of U.S. Patent No. 10,512,555 are incorporated herein by reference in their entirety for all purposes, including all systems, devices, and methods disclosed therein, and any combination of features and operations disclosed therein. For example, neural interface 14 (e.g., scaffold 101) can be any scaffold (e.g., scaffold 101) disclosed in, for example, U.S. Patent No. 10,512,555.
[0118] Furthermore, the neural interface, stent, or scaffold disclosed herein can be any scaffold, stent, stent-electrode, or stent-electrode array disclosed in the following: U.S. Patent Publication No. 2020 / 0363869; U.S. Patent Publication No. 2020 / 0078195; U.S. Patent Publication No. 2020 / 0016396; U.S. Patent Publication No. 2019 / 0336748; U.S. Patent Publication No. 2014 / 0288667; U.S. Patent No. 10,575,783; U.S. Patent No. 10,485,968; U.S. Patent No. 10,729,530; and International Patent Application No. PCT / US filed November 6, 2020. U.S. Patent Application No. 62 / 927,574, filed October 29, 2019; U.S. Patent Application No. 62 / 932,906, filed November 8, 2019; U.S. Patent Application No. 62 / 932,935, filed November 8, 2019; U.S. Patent Application No. 62 / 935,901, filed November 15, 2019; U.S. Patent Application No. 2020 / 059509, filed November 27, 2019; U.S. Patent Application No. 62 / 941,317; U.S. Patent Application No. 62 / 950,629, filed December 19, 2019; U.S. Patent Application No. 63 / 003,480, filed April 1, 2020; U.S. Patent Application No. 63 / 057,379, filed July 28, 2020; and U.S. Patent Application No. 63 / 062,633, filed August 7, 2020, the contents of which are incorporated herein by reference in their entirety.
[0119] Using module 10, patient 8 can be prepared to interface with multiple terminal applications 12. Using module 10, patient 8 can perform multiple tasks using a type of electronic command based on a specific task-independent thought (e.g., thought 9). For example, using module 10, patient 8 can perform multiple tasks with a single task-independent thought (e.g., thought 9).
[0120] For example, Figure 5 A variation of method 50 for preparing an individual to interface with an electronic device or software (e.g., terminal application 12) is shown, having operations 52, 54, 56 and 58. Figure 5 Method 50 may include, in operation 52, measuring the individual's neural-related signals to obtain a first sensory neural signal when the individual generates a first task-independent thought. Method 50 may include, in operation 54, transmitting the first sensory neural signal to a processing unit. Method 50 may include, in operation 56, associating the first task-independent thought and the first sensory neural signal with a first input command. Method 50 may include, in operation 58, compiling the first task-independent thought, the first sensory neural signal, and the first input command into an electronic database.
[0121] As another example, Figure 6 A variation of method 60 for controlling a first device and a second device (e.g., a first terminal application 12a and a second terminal application 12b) is shown, having operations 62, 64, 66 and 68. Figure 6 Method 60 may include, in operation 62, measuring neural-related signals of the individual to obtain sensed neural signals when the individual generates task-irrelevant thoughts. Method 60 may include, in operation 64, transmitting the sensed neural signals to a processor. The method may include, in operation 66, associating the sensed neural signals with a first device input command and a second device input command via the processor. The method may include, in operation 68, electrically transmitting the first device input command to a first device or the second device input command to a second device when associating the sensed neural signals with the first device input command and the second device input command.
[0122] As another example, Figure 7 A variation of method 70 for preparing an individual to interface with a first device and a second device (e.g., a first terminal application 12a and a second terminal application 12b) is shown, having operations 72, 74, 76, 78 and 80. Figure 7Method 70 may include, in operation 72, measuring brain-related signals of an individual to obtain sensed brain-related signals when the individual generates a task-specific thought by thinking of a first task. The method may include, in operation 74, transmitting the sensed brain-related signals to a processing unit. The method may include, in operation 76, associating the sensed brain-related signals with a first device input command associated with a first device task via the processing unit. The first device task may be different from the first task. The method may include, in operation 78, associating the sensed brain-related signals with a second device input command associated with a second device task via the processing unit. The second device task may be different from both the first device task and the first task. The method may include, in operation 80, when associating the sensed brain-related signals with the first device input command and the second device input command, electrically transmitting the first device input command to a first device to perform the first device task associated with the first device input command, or electrically transmitting the second device input command to a second device to perform the second device task associated with the second device input command.
[0123] As another example, Figures 5-7 A variation of the method of controlling multiple terminal applications 12 using a general-purpose switch (e.g., idea 9) is shown.
[0124] As another example, Figures 5-7 The operations shown can be performed and repeated in any order and in any combination. Figures 5-7 This disclosure is not in any way limited to the methods shown or the specific order of operations listed. For example, the operations listed in methods 50, 60, and 70 may be performed in any order, or one or more operations may be omitted or added.
[0125] As another example, a variation of the method using module 10 may include measuring an individual's brain-related signals to obtain a first sensed brain-related signal when the individual generates a task-independent thought (e.g., thought 9). The method may include transmitting the first sensed brain-related signal to a processing unit. The method may include the processing unit applying a mathematical algorithm or model to detect a brain-related signal corresponding to when the individual generated thought 9. The method may include associating the task-independent thought and the first sensed brain-related signal with one or more input commands 18 of a quantity of N. The method may include compiling the task-independent thought (e.g., thought 9), the first sensed brain-related signal, and the N input commands 18 into an electronic database. The method may include monitoring the individual's first sensed brain-related signal (e.g., using a neural interface) and, upon detection of the first sensed brain-related signal, electrically transmitting at least one of the N input commands 18 to a control system. The control system may be a control system of terminal application 12. The N input commands 18 may be, for example, 1 to 100 input commands 18, including every single input command 18 within that range. N input commands can be assigned to Y terminal applications 12, where the Y terminal applications can be, for example, 1 to 100 terminal applications 12, including each increment of terminal applications 12 within that range. As another example, the Y terminal applications 12 can be, for example, 2 to 100 terminal applications 12, including each increment of terminal applications 12 within that range. The Y terminal applications 12 can include at least one of, for example, controlling a mouse cursor, controlling a wheelchair, and controlling a speller. The N input commands 18 can be at least one of binary input associated with a task-independent idea, hierarchical input associated with a task-independent idea, and continuous trajectory input associated with a task-independent idea. The method can include associating M detections of a first sensed brain-related signal with the N input commands 18, where M is 1 to 10 or fewer detections. For example, when M is a single detection, a task-independent idea (e.g., idea 9) and a first sensed brain-related signal can be associated with a first input command (e.g., first input command 18a). As another example, when M consists of two detections, a task-independent idea (e.g., idea 9) and a first sensed brain-related signal can be associated with a second input command (e.g., second input command 18b). As yet another example, when M consists of three detections, a task-independent idea (e.g., idea 9) and a first sensed brain-related signal can be associated with a third input command (e.g., third input command 18c). The first, second, and third input commands can be associated with one or more terminal applications 12. For example, the first input command could be an input command for a first terminal application, the second input command could be an input command for a second terminal application, and the third input command could be an input command for a third application, such that a single idea 9 can control multiple terminal applications 12.Each of the M detections in Idea 9 can be assigned to multiple terminal applications, such that the number of terminals with M detections (e.g., 1, 2, or 3 detections) can act as a universal switch that can be assigned to any input command 18. The first, second, and third input commands can be associated with different functions. The first, second, and third input commands can also be associated with the same function, such that the first input command is associated with a first parameter of the function, the second input command is associated with a second parameter of the function, and the third input command is associated with a third parameter of the function. The first, second, and third parameters of the function can be, for example, speed, volume, or a progressive level of both. For example, a progressive level of speed can be associated with the movement of a wheelchair, the movement of a mouse cursor on the screen, or both. A progressive level of volume can be associated, for example, with the sound volume of a car, computer, telephone, or any combination thereof. At least one of the N input commands 18 can be a click-and-hold command associated with a computer mouse. The method may include associating a combination of task-independent ideas (e.g., Idea 9) with the N input commands 18. The method may include associating a combination of Z task-independent ideas with N input commands 18, wherein the Z task-independent ideas can be 2 to 10 or more task-independent ideas, or more broadly, 1 to 1000 or more task-independent ideas, including each unit increment within these ranges. At least one of the Z task-independent ideas may be a task-independent idea, wherein the task-independent idea may be a first task-independent idea, such that the method may include measuring a brain-related signal of an individual when the individual generates a second task-independent idea to obtain a second sensed brain-related signal, transmitting the second sensed brain-related signal to a processing unit, and associating the second task-independent idea and the second sensed brain-related signal with N2 input commands, wherein when the combination of the first and second sensed brain-related signals is obtained sequentially or simultaneously, the combination may be associated with N3 input commands. The task-independent idea may be an idea of moving a limb. The first sensed brain-related signal may be at least one of electrical activity and functional activity of brain tissue. Any operation in this exemplary method may be performed in any combination and in any order.
[0126] As another example, a variation of the method using module 10 may include measuring an individual's brain-related signals to obtain a first sensed brain-related signal when the individual generates a first task-specific thought by thinking of a first task (e.g., by thinking of idea 9). The method may include transmitting the first sensed brain-related signal to a processing unit. The method may include the processing unit applying a mathematical algorithm or model to detect a brain-related signal corresponding to when the individual generated the idea. The method may include associating the first sensed brain-related signal with a first task-specific input command (e.g., input command 18) associated with a second task, wherein the second task is different from the first task (e.g., such that idea 9 relates to a task different from the task configured to be performed by input command 18). The first task-specific thought may be independent of the associating step. The method may include assigning the second task to the first task-specific command instruction, regardless of the first task. The method may include reassigning a third task to the first task-specific command instruction, regardless of the first and second tasks. The method may include compiling the first task-specific thought, the first sensed brain-related signal, and the first task-specific input command into an electronic database. The method may include monitoring the individual's first sensed brain-related signal and, upon detection of the first sensed brain-related signal, electrically transmitting the first task-specific input command to a control system. The first task-specific idea can be, for example, about a physical task, a non-physical task, or both. The generated idea can be, for example, a single idea or a composite idea. A composite idea can be two or more non-simultaneous ideas, two or more simultaneous ideas, and / or a series of two or more simultaneous ideas. Any operation in this exemplary method can be performed in any combination and in any order.
[0127] As another example, a variation of the method using module 10 may include measuring an individual's brain-related signals to obtain a first sensed brain-related signal when the individual is thinking a first thought. The method may include transmitting the first sensed brain-related signal to a processing unit. The method may include the processing unit applying a mathematical algorithm or model to detect brain-related signals corresponding to when the individual generated the thought. The method may include generating a first command signal based on the first sensed brain-related signal. The method may include assigning a first task to the first command signal, regardless of the first thought. The method may include separating the first thought from the first sensed brain activity. The method may include reassigning a second task to the first command signal, regardless of the first thought and the first task. The method may include compiling the first thought, the first sensed brain-related signal, and the first command signal into an electronic database. The method may include monitoring the individual's first sensed brain-related signal and, upon detecting the first sensed brain-related signal, transmitting a first input command to a control system. The first thought may include, for example, a thought about a real or imagined muscle contraction, a real or imagined memory, or both, or any abstract thought. The first thought may be, for example, a single thought or a compound thought. Any operation in this exemplary method may be performed in any combination and in any order.
[0128] As another example, a variation of the method using module 10 may include measuring the electrical activity of an individual's brain tissue to obtain a first sensed EEG activity when the individual is thinking a first thought. The method may include transmitting the first sensed EEG activity to a processing unit. The method may include the processing unit applying a mathematical algorithm or model to detect brain-related signals corresponding to when the individual generated the thought. The method may include generating a first command signal based on the first sensed EEG activity. The method may include assigning a first task and a second task to the first command signal. The first task may be associated with a first device, and the second task may be associated with a second device. The first task may be associated with a first application of the first device, and the second task may be associated with a second application of the first device. The method may include assigning the first task to the first command signal regardless of the first thought. The method may include assigning the second task to the first command signal regardless of the first thought. The method may include compiling the first thought, the first sensed EEG activity, and the first command signal into an electronic database. The method may include monitoring the individual's first sensed EEG activity and, upon detecting the first sensed EEG activity, electrically transmitting the first command signal to a control system. Any operation in this exemplary method may be performed in any combination and in any order.
[0129] As another example, a variation of the method using module 10 may include measuring the individual's neurally relevant signals to obtain a first sense neural signal when the individual generates a task-free thought. The method may include transmitting the first sense neural signal to a processing unit. The method may include the processing unit applying a mathematical algorithm or model to detect brain-related signals corresponding to when the individual generated the task-free thought. The method may include associating the task-free thought and the first sense neural signal with a first input command. The method may include compiling the task-free thought, the first sense neural signal, and the first input command into an electronic database. The method may include monitoring the individual's first sense neural signal and, upon detection of the first sense neural signal, electrically transmitting the first input command to a control system. The neurally relevant signal may be a brain-related signal. The neurally relevant signal can be measured from neural tissue in the individual's brain. Any operation in this exemplary method may be performed in any combination and in any order.
[0130] As another example, a variation of the method using module 10 may include measuring an individual's neural-related signals to obtain a first sensed neural-related signal when the individual generates a first task-specific thought by thinking of a first task. The method may include transmitting the first sensed neural-related signal to a processing unit. The method may include the processing unit applying a mathematical algorithm or model to detect brain-related signals corresponding to when the individual generated the thought. The method may include associating the first sensed neural-related signal with a first task-specific input command associated with a second task, which is different from the first task, thereby providing a mechanism for the user to control multiple tasks with different task-specific inputs using a single user-generated thought. The method may include compiling the task-independent thought, the first sensed neural signal, the first input command, and the corresponding task into an electronic database. The method may include using the memory of the electronic database to automatically group combinations of task-independent thoughts, sensed brain-related signals, and one or more inputs of number N based on the task, brain-related signal, or thought, to automatically map control functions for automatic system setup for use. The neural-related signal may be a neural-related signal of brain tissue. Any operation in this exemplary method may be performed in any combination and in any order.
[0131] Module 10 can perform any combination of any methods and can perform any operation of any method disclosed herein.
[0132] Figure 8 Embodiments of a method 100 for controlling a device (e.g., a personal electronic device, an IoT device, a mobile vehicle, etc.), a software application (e.g., terminal application 12), or a combination thereof using detected changes in neurally related signals of a subject are illustrated. In some embodiments, the neurally related signals may be the subject's brain waves or other types of synchronized brain electrical activity.
[0133] The neurally relevant signals may include one or more neural oscillations of the subject, including neural oscillations in the beta frequency range or beta band (approximately 12 Hz to 30 Hz), the alpha frequency range or alpha band (approximately 7 Hz to 12 Hz), the gamma frequency range or gamma band (approximately 30 Hz to 140 Hz, more specifically, 60 Hz to 80 Hz), the delta frequency range or delta band (approximately 0.1 Hz to 3 Hz), the theta frequency range or the theta band (approximately 4 Hz to 7 Hz), or combinations thereof. The neurally relevant signals may also include neural oscillations in the mu band (approximately 7.5 Hz to 12.5 Hz), the sensorimotor rhythm (SMR) band (approximately 12.5 Hz to 15.5 Hz), or combinations thereof.
[0134] As described above, module 10 or its components can be used to monitor or measure neural signals of a subject. For example, neural interface 14, telemetry unit 22, host device 16, or combinations thereof can be used to monitor or measure neural signals.
[0135] In some embodiments, the neural interface 14 may be an intravascular device (e.g., an expandable and retractable stent) implanted in the subject. In some embodiments, electrodes of the neural interface 14 implanted in the subject may be used to monitor or measure neural-related signals. For example, electrodes of an intravascular device (e.g., electrodes coupled to a stent) may be used to monitor or measure neural-related signals.
[0136] As discussed earlier, endovascular devices can be implanted inside a subject's brain. For example, endovascular devices can be implanted in at least one of the subject's frontal cortex, motor cortex, and sensory cortex. Endovascular devices can also be implanted in other parts of the subject's brain.
[0137] Method 100 may include detecting, in operation 802, a decrease in the intensity of a subject's neurally relevant signal below a baseline level. For example, detecting a decrease in the intensity of a neurally relevant signal may include detecting a decrease in the power of at least one neural oscillation of the subject (e.g., a neural oscillation at a β-band frequency) (e.g., in microvolts squared per Hz (µV)). 2 It is measured by parameters such as Hz, decibels (dB), average t-fraction, and average z-fraction.
[0138] In some embodiments, the baseline level may be defined as the median or average intensity over a specific time period (e.g., the last few seconds or minutes). In these and other embodiments, the baseline level may be varied or continuously adjusted and set. In other embodiments, the baseline level may be a predefined or predetermined level. For example, the baseline level may be determined based on the time of day, the activities or actions performed by the subject, or a combination thereof.
[0139] A decrease in the intensity of a neural signal can refer to a statistically significant (e.g., more than 2 standard deviations (SD)) decrease in the intensity of the neural signal relative to the baseline level. This statistically significant decrease in the intensity of a neural signal can also be referred to as desynchronization or desynchronization of the neural signal.
[0140] For example, when the monitored or measured neural signal is a beta-band oscillation, method 100 may include detecting a statistically significant decrease or reduction in beta-band oscillation power relative to a baseline beta-band power level. More specifically, such a statistically significant reduction in beta-band oscillation power may be referred to as beta desynchronization.
[0141] Method 100 may also include, in operation 804, detecting a subsequent increase in the intensity of the neurally related signal beyond a baseline level after a decrease. For example, method 100 may include detecting a statistically significant (e.g., greater than 2 SD) increase in the intensity of the neurally related signal beyond a baseline level. In some embodiments, such a statistically significant increase in the intensity of the neurally related signal may be referred to as a bounce of the neurally related signal.
[0142] For example, when the monitored or measured neural signal is a beta band oscillation, method 100 may include detecting a statistically significant increase or rise in the power of the beta band oscillation relative to a baseline beta band power level. More specifically, such a statistically significant increase in the power of the beta band oscillation may be referred to as a beta bounce.
[0143] In some embodiments, the power of a selected number of neural bands (e.g., beta band, gamma band, etc.) can be continuously monitored on selected channels using electrodes coupled to neural interface 14, and the power readings can be filtered and fed into a machine learning classifier at predetermined intervals (e.g., every 100 milliseconds or 100 ms). The machine learning classifier can then classify the power into discrete states or events, such as (1) desynchronization (“desynchronization”) events, (2) bounce events, or (3) rest or non-events, by comparing the power to a baseline level for a particular neural band.
[0144] Intensity changes can be detected using one or more processors, such as neural interface 14 (e.g., via electrodes of an intravascular device implanted in the brain), telemetry unit 22, host device 16, or a combination thereof.
[0145] Method 100 may further include transmitting an input command 18 to the device or software when or after an increase in the intensity of a neurally related signal is detected in operation 806. In some embodiments, the input command 18 may be transmitted when or after an increase in the intensity of a neurally related signal is detected, but before the signal bounce is complete.
[0146] Input commands 18 can be transmitted using one or more processors, such as telemetry unit 22, host device 16, or combinations thereof. As previously described, input commands 18 can be transmitted to one or more terminal applications (e.g., application software) running on a peripheral or personal computing device, such as a laptop, desktop computer, smartphone, or tablet. When input commands 18 are transmitted to software programs, they can be transmitted via one or more application programming interfaces (APIs). In these and other embodiments, input commands 18 can be transmitted to IoT devices, mobile vehicles (e.g., electric wheelchairs), or other types of peripheral or personal computing devices to control these devices or vehicles. Furthermore, input commands 18 can also be transmitted to one or more terminal applications (e.g., software) running on such peripheral or personal computing devices or vehicles.
[0147] In some embodiments, a decrease in the intensity of the neurally related signal may be caused by the subject evoking or generating a task-related thought and maintaining that thought for a period of time. In these embodiments, a subsequent increase in the intensity of the neurally related signal (e.g., signal bounce) may be caused by the subject mentally releasing the task-related thought. Furthermore, in these embodiments, the input command 18 may be a command transmitted to the device or software to complete at least a portion of the task associated with the task-related thought.
[0148] For example, a method for controlling an electric wheelchair may include a subject generating and maintaining the idea of moving the electric wheelchair forward. While the subject maintains the idea of moving the electric wheelchair forward, module 10 may detect a decrease in the intensity of the subject's neurally relevant signals (e.g., beta oscillation desynchronization). Module 10 may then detect a subsequent increase in the intensity of the subject's neurally relevant signals (e.g., beta oscillation bounce) when the subject releases the idea of moving the electric wheelchair forward. Module 10 may then transmit an input command 18 to the electric wheelchair to move the electric wheelchair forward upon detecting an increase in the intensity of the subject's neurally relevant signals. As will be appreciated by those skilled in the art, this method can be extended to cover any number of task-related ideas and to the control of other devices, vehicles, or software not specifically mentioned in the preceding examples.
[0149] In other embodiments, a decrease in the intensity of the neurally relevant signal may be caused by the subject evoking or generating task-irrelevant thoughts and maintaining those thoughts for a period of time. In these embodiments, a subsequent increase in the intensity of the neurally relevant signal may be caused by the subject mentally releasing the task-irrelevant thoughts. Furthermore, in these embodiments, the input command 18 may be a command transmitted to the device or software to complete at least a portion of a task unrelated to the task-irrelevant thoughts.
[0150] For example, another method of controlling an electric wheelchair may include the subject generating and maintaining an idea related to the subject's bodily function, such as contracting the subject's muscles. An idea related to the subject's bodily function can be considered a task-independent idea because it is unrelated to the task of controlling the subject's electric wheelchair. While the subject maintains the idea of contracting the subject's muscles, module 10 may detect a decrease in the intensity of the subject's neurally relevant signal (e.g., β-oscillation desynchronization). Module 10 may then detect a subsequent increase in the intensity of the subject's neurally relevant signal (e.g., β-oscillation bounce) when the subject releases the idea of contracting the subject's muscles. Module 10 may then transmit an input command 18 to the electric wheelchair to move the electric wheelchair forward upon detecting an increase in the intensity of the subject's neurally relevant signal. As will be appreciated by those skilled in the art, this method can be extended to cover any number of task-independent ideas and to the control of other devices, vehicles, or software not specifically mentioned in the preceding examples.
[0151] Method 100 may further include additional operation 808, providing the subject with visual feedback, auditory feedback, tactile feedback, neural stimulation feedback, or a combination thereof, after the input command 18 has been transmitted to the device or software. This feedback may notify the subject that the input command 18 has been successfully transmitted or that the input command 18 is being implemented. In other embodiments, the feedback may notify the subject that signal desynchronization or signal bounce has been detected.
[0152] In some embodiments, visual feedback may include written text displayed on a display of host device 16. In other embodiments, visual feedback may include one or more lights illuminated on telemetry unit 22 or host device 16, or a combination thereof. Auditory feedback may include one or more sounds or auditory alarms generated by telemetry unit 22 or host device 16. In other embodiments, auditory feedback may include computer-generated or pre-recorded audio messages played by telemetry unit 22 or host device 16. Tactile feedback may include one or more sensors or electronic components configured to provide physically perceptible feedback to a subject in the form of vibrations, motions, or other forces applied to the subject's body or appendage. In some embodiments, tactile feedback may be applied to a subject via telemetry unit 22, additional wearable units, a seat supporting the subject, a structure or platform, or a combination thereof. Feedback in the form of neural stimulation may include the transmission of electrical impulses via electrodes implanted in the subject's body. For example, neural stimulation feedback may include the transmission of electrical impulses to the subject's brain via electrodes of neural interface 14 implanted in the subject's brain. In other embodiments, neural stimulation feedback may include non-invasive stimulation, such as stimulation of the subject via transcranial direct current stimulation (tDCS), transcranial magnetic stimulation (TMS), transcranial alternating current stimulation (tACS), transcranial pulsed current stimulation (tPCS), transcranial random noise stimulation (tRNS), or a combination thereof.
[0153] Figure 9 The diagram shows a spectrum illustrating the desynchronization (or reduction) of neural oscillations in the β band or β frequency of a subject, followed by a rebound (or increase) of the β band neural oscillations. More specifically, the spectrum shows the power of the β band oscillations decreasing to below baseline power levels, followed by an increase to above baseline power levels. In this particular spectrum, the power is expressed as the average t-fraction. Power can also be expressed in decibels (dB), z-fractions, or µV. 2 It is expressed in / Hz.
[0154] Desynchronization of neural signals can be caused by a subject recalling or generating thoughts (e.g., task-related or task-independent thoughts) and holding those thoughts for a period of time. Rebound of neural signals can be caused by a subject mentally releasing those thoughts.
[0155] Figure 9 It is also shown that input command 18 can be transmitted after or upon detection of a signal bounce. Input command 18 is transmitted before the signal bounce is complete. As will be discussed in more detail in the following sections, the duration of desynchronization can play a role in determining which input command(s) 18 are transmitted to the device or software program.
[0156] Figure 10Another method 200 is illustrated for controlling a device (e.g., a personal electronic device, IoT device, mobile vehicle, etc.) or software program (e.g., terminal application 12) using detected changes in neurally related signals of a subject. Method 200 may include detecting a decrease in the intensity of a neurally related signal of the subject to below a baseline level in operation 202. For example, detecting a decrease in the intensity of a neurally related signal may include detecting a decrease in the power of at least one neural oscillation of the subject.
[0157] The baseline level can be defined as the median or average intensity over a certain time period. The baseline level can be varied or continuously adjusted and set. In other embodiments, the baseline level can be a predefined or predetermined level. A decrease in the intensity of the neurally related signal can refer to a statistically significant decrease in the intensity of the neurally related signal relative to the baseline level. In some embodiments, such a statistically significant decrease in the power of the β-band oscillation can be referred to as desynchronization or desynchronization of the neurally related signal.
[0158] Method 200 may further include, in step 204, detecting an increase in the intensity of a neurally related signal beyond a baseline level after a decrease. For example, method 200 may include detecting a statistically significant increase in the intensity of the neurally related signal beyond a baseline level. In some embodiments, the increase in the intensity of the neurally related signal may be referred to as a bounce of the neurally related signal.
[0159] Similar to the previous section, neurally relevant signals can be the subject's brain waves or other types of synchronized brain electrical activity. Neurally relevant signals can include one or more neural oscillations of the subject, including oscillations in the β frequency range or β band, α frequency range or α band, γ frequency range or γ band, δ frequency range or δ band, θ frequency range or θ band, or combinations thereof. Neurally relevant signals can also include neural oscillations in the μ band, SMR band, or combinations thereof.
[0160] The neural signals of a subject can be monitored or measured using module 10 or its components. For example, neural interface 14, telemetry unit 22, host device 16, or a combination thereof can be used to monitor or measure neural signals.
[0161] The neural interface 14 may be an intravascular device (e.g., an expandable and retractable stent) implanted in the subject's brain. In some embodiments, electrodes of the neural interface 14 implanted in the subject's brain may be used to monitor or measure neurally related signals. For example, electrodes of a stent implanted in the subject's brain may be used to monitor or measure neurally related signals.
[0162] Method 200 may further include determining the duration of the decrease in intensity of the neurally related signal in operation 206. For example, when the neurally related signal is a beta-band oscillation, operation 206 may include determining the duration of desynchronization of the beta-band oscillation.
[0163] As will be discussed in more detail below, in some embodiments, the power of a selected number of neural bands (e.g., beta band, gamma band, etc.) can be continuously monitored on selected channels using electrodes coupled to neural interface 14, and the power readings can be filtered and fed into a machine learning classifier at predetermined intervals (e.g., every 100 milliseconds). The machine learning classifier can then classify the power into discrete states or events, such as (1) desynchronization (“desynchronization”) events, (2) bounce events, or (3) rest or non-events, by comparing the power to a baseline level for a particular neural band. In these embodiments, determining the duration of signal desynchronization may include counting the number of consecutive desynchronization events preceding a bounce event.
[0164] In other embodiments, determining the duration of the decrease in the intensity of the neurally related signal may include: transmitting a first signal to telemetry unit 22, host device 16, or other device used as part of module 10 when a decrease in the intensity of the neurally related signal is first detected; and transmitting a second signal to telemetry unit 22, host device 16, or other device used as part of module 10 when the decrease in the intensity of the neurally related signal stops or a signal bounce is detected. Telemetry unit 22, host device 16, or other device used as part of module 10 can then determine the duration by calculating the time elapsed between the two signals.
[0165] Method 200 may also include selecting an input command 18 from a plurality of conditional input commands based on duration in operation 208. Selecting an input command 18 from a plurality of conditional input commands may further include comparing the duration with one or more time thresholds associated with the conditional input command, and selecting the input command 18 based on whether the duration exceeds or fails to reach one or more time thresholds.
[0166] For example, two conditional input commands can be associated with the following time thresholds: Command 1: 300 ms ≤ duration < 1000 ms (i.e., between 3 and 9 consecutive desynchronization events) Command 2: 1000 ms ≤ duration (i.e., 10 or more consecutive desynchronization events) The time threshold can range from 100 ms to 30,000 ms (or 30 seconds). While a range is provided, this disclosure considers this range, and those skilled in the art should understand that any subrange of the disclosed range is acceptable. For example, the range of 100 ms to 30,000 ms may include 100 ms to 500 ms, 100 ms to 1000 ms, 100 ms to 10,000 ms, 1000 ms to 10,000 ms, or any other subrange within this range. In alternative embodiments, the time threshold can range from 100 ms to greater than 30,000 ms, such as 50,000 ms, 60,000 ms, 100,000 ms, etc.
[0167] Furthermore, multiple conditional input commands can include two to ten or more conditional input commands. Each conditional input command can be associated with one or more time thresholds. For example, in some cases where two conditional input commands exist, these two conditional input commands can be associated with the same time threshold. In this example, the input command can be selected based on whether the duration reaches / exceeds the time threshold or fails to reach the time threshold.
[0168] As another example, three conditional input commands can be associated with the following time thresholds: Command 1 (e.g., open software application #1): 100 ms ≤ duration < 1000 ms Command 2 (e.g., open software application #2): 1000 ms ≤ duration < 2000 ms Command 3 (e.g., open software application #3): 2000 ms ≤ duration In some embodiments, a decrease in the intensity of the neurally relevant signal may be caused by the subject recalling and holding a thought (e.g., a task-related thought or a task-irrelevant thought). Furthermore, an increase in the intensity of the neurally relevant signal may be caused by the same subject mentally releasing the thought. The duration of the decrease in the intensity of the neurally relevant signal can then be correlated with the amount of time the subject held the thought (e.g., a task-related thought or a task-irrelevant thought) before mentally releasing it.
[0169] As previously described, the idea can be a task-related idea or a task-independent idea. A subject can recall and hold a task-related idea to transmit input command 18 to the device, thereby completing at least a portion of the task associated with the task-related idea. In some embodiments, the length of time the subject holds the task-related idea can indicate which input command to select from a plurality of conditional input commands, but all such conditional input commands may be designed to complete at least a portion of the task associated with the task-related idea.
[0170] Subjects may also evoke and maintain task-independent thoughts to transmit input commands to the device, thereby completing at least a portion of a task unrelated to the task-independent thoughts. In some embodiments, the length of time a subject maintains a task-independent thought may determine which input command to select from a plurality of conditional input commands, but all such conditional input commands may be designed to complete at least a portion of a task unrelated to the task-independent thoughts.
[0171] Method 200 may also include additional or optional operations 210, providing the subject with visual, auditory, tactile, neural stimulation, or combinations thereof feedback regarding the selected input command 18. This feedback may inform the subject of the selected input command 18 and also provide the subject with the option to correct the input command 18 by selecting a different input command 18 or canceling the selected input command 18.
[0172] Method 200 may also include transmitting the selected input command 18 to the device or software in operation 212. The input command 18 may be transmitted when or after the subject confirms the selected input command 18, or the input command 18 may be transmitted without receiving such confirmation.
[0173] Input commands 18 can be transmitted using one or more processors, such as telemetry unit 22, host device 16, or combinations thereof. As previously described, input commands 18 can be transmitted to one or more terminal applications (e.g., application software) running on a peripheral or personal computing device, such as a laptop, desktop computer, smartphone, or tablet. When input commands 18 are transmitted to software programs, they can be transmitted via one or more application programming interfaces (APIs). In these and other embodiments, input commands 18 can be transmitted to IoT devices, mobile vehicles (e.g., electric wheelchairs), or other types of peripheral or personal computing devices to control these devices or vehicles. Furthermore, input commands 18 can also be transmitted to one or more terminal applications (e.g., software) running on such peripheral or personal computing devices or vehicles.
[0174] When a subject generates and holds the thought of moving the electric wheelchair forward for approximately 2 seconds, the subject can control the electric wheelchair using method 200 or a variant thereof. Module 10 can detect a decrease in the intensity of the subject's neurally relevant signal lasting for approximately 2 seconds. Module 10 can then detect a subsequent increase in the intensity of the subject's neurally relevant signal (e.g., β-oscillation bounce) when the subject releases the thought of moving the electric wheelchair forward. Module 10 can then select an input command 18 from a plurality of conditional input commands based on a duration of approximately 2 seconds. In this case, input command 18 could be a command to the electric wheelchair to move the electric wheelchair forward two meters. Other conditional input commands could include commands to move the electric wheelchair forward one meter (desynchronization duration of 1 second or less) or three meters (desynchronization duration of 3 seconds or longer). Module 10 can then transmit input command 18 to the electric wheelchair to move the electric wheelchair forward two meters. As will be appreciated by those skilled in the art, this method can be extended to cover any number of task-related thoughts and to cover the control of other devices, vehicles, or software not specifically mentioned in the preceding examples.
[0175] While the subject generates and maintains a thought related to their bodily function (e.g., contracting their muscles) for approximately 2 seconds, the subject may also use method 200 or a variation thereof to control their motorized wheelchair. A thought related to the subject's bodily function can be considered a task-independent thought because it is unrelated to the task of controlling the subject's motorized wheelchair. Module 10 can detect a decrease in the intensity of the subject's neurally relevant signals lasting approximately 2 seconds. Module 10 can then select an input command 18 from a plurality of conditional input commands based on a duration of approximately 2 seconds. In this case, input command 18 could be a command to the motorized wheelchair to move it forward two meters. Other conditional input commands could include commands to move the motorized wheelchair forward one meter (with a desynchronization duration of 1 second or less, e.g., caused by the subject generating and maintaining a thought of contracting muscles for 1 second or less) or three meters (with a desynchronization duration of three seconds or longer, e.g., caused by the subject generating and maintaining a thought of contracting muscles for 3 seconds or longer). Module 10 can then transmit input command 18 to the motorized wheelchair to move it forward two meters.
[0176] As will be understood by those skilled in the art, this method can be extended to cover any number of task-agnostic ideas and to cover the control of other devices, vehicles, or software not specifically mentioned in the foregoing examples.
[0177] Figure 11A system or another embodiment of module 10 is shown, comprising a neural interface 14 and at least one device 300 or apparatus running various software layers configured to process and classify neurally related signals and select input commands 18 based on the processed and classified data. In some embodiments, device 300 may be any one of a telemetry unit 22, a host device 16, or a combination thereof. In these and other embodiments, device 300 (e.g., telemetry unit 22) may be implanted in a subject, for example, in the subject's chest region or arm.
[0178] In other embodiments, device 300 may refer to a processing unit or controller embedded within or coupled to neural interface 14 implanted in or within the brain of a subject. One or more processors of device 300 may be programmed to execute software instructions that constitute various software layers.
[0179] like Figure 11 As shown, the software layer may include a preprocessing layer 302, a classification layer 304, and a temporal click logic layer 306. The preprocessing layer 302, classification layer 304, and temporal click logic layer 306 may be part of a multi-layer software architecture.
[0180] The preprocessing layer 302 may include multiple software filters configured to filter and smooth the raw signal obtained from the neural interface 14. Electrodes of the neural interface 14 (e.g., a scaffold implanted in the subject's brain) can be used to continuously monitor the subject's neural-related signals on selected channels. The neural-related signals may be sampled every 100 ms, or 100 ms "chunks" or bins of the raw signal may be passed to the preprocessing layer 302 for processing and smoothing. As previously described, the monitored neural-related signals may be one or more neural bands of the subject (e.g., beta band oscillations, gamma band oscillations, etc.), and the intensity of the neural-related signals may be the power of such neural bands.
[0181] For example, data corresponding to a 100 ms bin of raw neural-related signals obtained from three independent channels of neural interface 14 can first be passed to preprocessing layer 302. Preprocessing layer 302 can then apply: (1) a threshold filter for performing threshold-based disconnection and acceptance rejection; (2) a notch filter for performing 50 Hz notch filtering; (3) a bandpass filter for performing 4–30 Hz Butterworth bandpass filtering; (4) a wavelet artifact removal filter for performing wavelet-based artifact rejection; (5) a multi-taper spectral decomposition filter for performing multi-taper spectral decomposition; and (6) a boxcar smoothing filter for performing time-based boxcar smoothing. The filtered data is then fed to classification layer 304.
[0182] Classification layer 304 includes a machine learning classifier configured to classify the resulting data segments or bins into: desynchronization events or states (also known as "key-down" classified events), bounce events or states (also known as "key-up" classified events), or rest events or states. The machine learning classifier may be a pre-trained classifier.
[0183] In some embodiments, the machine learning classifier may utilize a supervised learning model, such as a support vector machine (SVM). As a more specific example, the machine learning classifier may be a pre-trained SVM. In other embodiments, the machine learning classifier may be a Gaussian mixture model classifier, a Naive Bayes classifier, or another type of machine learning classifier.
[0184] The categorized events or states can then be fed to the time-click logic layer 306 to select an input command 18 based on the number of events / states and conditions or thresholds stored as part of the time-click logic layer 306. For example, when the time-click logic layer 306 detects a bounce event following three to nine consecutive desynchronization events, it can select an input command to open a first software application (e.g., the subject's Gmail® application). Alternatively, when the time-click logic layer 306 detects a bounce event following ten or more consecutive desynchronization events, it can select another input command to open a second software application (e.g., the subject's WhatsApp® application).
[0185] Figure 12A An example spectrogram 400 shows a subject maintaining a thought for a short period of time. This thought can be task-related (e.g., pressing a mouse cursor to select a software application) or task-independent (e.g., contracting the subject's hamstring tendon). In this example, the neurally relevant signal could be the subject's beta band oscillation. Figure 12AAs shown, when a subject recalls and maintains the thought, the power of the subject's β-band neural oscillations can decrease to below the baseline β-band power level.
[0186] As long as the subjects maintain this idea, classification layer 304 (see Figure 11 The classification layer 304 will then classify the beta band oscillation as being in a desynchronized state. More specifically, as long as the subject holds this thought, the classification layer 304 will classify time segments of the beta band signal (e.g., a 100 ms “cage”) as consecutive desynchronization events. When the subject holds a thought for approximately 400 ms, this roughly corresponds to four consecutive desynchronization events, each of which is set to approximately 100 ms.
[0187] Figure 12A It was also shown that when subjects mentally release their thoughts, the power of their beta band oscillations can increase to levels exceeding baseline beta band power. When this release occurs, classification layer 304 classifies this time segment of the beta band signal as a bounce event.
[0188] Once the time-click logic layer 306 detects a bounce event, it will count the number of consecutive desynchronization events preceding the bounce event and compare this total with one or more time thresholds associated with a specific condition input command.
[0189] In this example, four consecutive desynchronization events (corresponding to the subject holding the thought for approximately 400 ms) fall within a short duration range between three consecutive desynchronization events (approximately 300 ms) and nine consecutive desynchronization events (approximately 900 ms). As a result, a first input command can be selected associated with this short duration range. The first input command could be a command to open a first software application running on a device communicating with module 10, such as the subject's Gmail® application. Figure 12A As shown, after a bounce event is detected, the input command can be transmitted to the device immediately.
[0190] Figure 12B Another example spectrum 402 shows subjects maintaining their thoughts for an extended period of time. (e.g.) Figure 12B As shown, subjects can maintain this thought (e.g., task-related or task-unrelated) for approximately 1000 ms or 1 second. Similar to... Figure 12A When a subject arouses and maintains a thought, the power of the subject's beta band oscillations can decrease to below baseline beta band power levels. Figure 12B As shown, maintaining this idea for approximately 1000 ms roughly corresponds to ten consecutive desynchronization events, where each desynchronization event is set to approximately 100 ms.
[0191] When a subject mentally releases a thought, the power of the subject's beta band oscillations can increase to above the baseline beta band power level. When this release occurs, classification layer 304 will classify this time segment of the beta band signal as a bounce event.
[0192] In this example, ten consecutive desynchronization events (corresponding to the subject holding the thought for approximately 1000 ms) fall within a long duration range of ten or more consecutive desynchronization events (x ≥ 1000 ms). As a result, a second input command associated with this long duration range can be selected. The second input command could be a command to open a second software application running on a device communicating with module 10, such as the subject's WhatsApp® application. Figure 12B As shown, after a bounce event is detected, the input command can be transmitted to the device immediately.
[0193] Although continuous desynchronization events have been discussed in the foregoing examples, this disclosure contemplates that, for neural bands other than β-band frequencies, continuous events other than synchronization events (e.g., an increase in the intensity of a neurally related signal) can also be used to select input commands.
[0194] Figure 13 Another method 500 is illustrated for controlling a device (e.g., a personal electronic device, an IoT device, a mobile vehicle, etc.), a software application (e.g., terminal application 12), or a combination thereof using detected changes in the subject's neurally related signals. Method 500 may include detecting a first change in the subject's neurally related signals in operation 502. Method 500 may also include detecting a second change in the subject's neurally related signals following the first change in operation 504.
[0195] In one embodiment, a first change in the neurally related signal may be a decrease in the intensity of the neurally related signal to below the baseline signal level, while a second change in the neurally related signal may be an increase in the intensity of the neurally related signal to above the baseline signal level. For example, the first change in the neurally related signal may be a decrease in the power of the subject's neural oscillations, while the second change in the neurally related signal may be an increase in the power of the neural oscillations.
[0196] In this embodiment, a first change in neural signals may occur when the subject generates and holds a thought (such as a task-related thought or a task-independent thought). A second change in neural signals may occur when the subject mentally releases the thought.
[0197] In an alternative embodiment, the first change in the neurally related signal can be an increase in the intensity of the neurally related signal above the baseline signal level, while the second change can be a decrease in the intensity of the neurally related signal below the baseline signal level. For example, the first change in the neurally related signal can be an increase in the power of the subject's neural oscillations, while the second change can be a decrease in the power of the neural oscillations. More specifically, the first change in the neurally related signal can be an increase in the power of the gamma band oscillations above the baseline gamma band power level, while the second change can be a decrease in the power of the gamma band oscillations below the baseline gamma band power level. In this embodiment, the first change in the neurally related signal (i.e., an increase in the power of the gamma band oscillations) can occur when the subject generates and holds thoughts such as task-related thoughts or task-independent thoughts. The second change in the neurally related signal (i.e., a decrease in the power of the gamma band oscillations) can occur when the subject mentally releases the thoughts.
[0198] In another embodiment, the first change in the neurally related signal may be an increase in the intensity of the subject's neurally related signal caused by the subject mentally releasing a first thought (e.g., a task-related thought or a task-independent thought). In this embodiment, the second change in the neurally related signal may be a decrease in the intensity of the neurally related signal to below the baseline signal level caused by the subject generating and holding a second or subsequent thought (e.g., another task-related thought or a task-independent thought). The first thought, the second thought, or a combination thereof may be thoughts related to the subject's bodily functions, such as the control of the subject's muscle groups. The input command 18 may be transmitted when or after the subject generates the second thought.
[0199] Intensity changes can be detected using one or more processors, such as neural interface 14 (e.g., via electrodes of an intravascular device implanted in the brain), telemetry unit 22, host device 16, or a combination thereof.
[0200] Method 500 may further include determining the duration of a first change in the neurally related signal in operation 506. For example, when the neurally related signal is a neural oscillation of the subject, operation 506 may include determining the duration of a power change in the neural oscillation.
[0201] In some embodiments, the duration of the first change can be determined by calculating the elapsed time between the onset of the first change and the onset of the second change in the neurally related signal. For example, the duration can be prolonged when the subject holds the thought (e.g., a task-related thought or a task-irrelevant thought) for a longer period of time.
[0202] In other embodiments, samples or time segments of the subject's neurally relevant signals can be fed into a machine learning classifier (e.g., Figure 11The machine learning classifier 308 determines the duration of the first change, and the machine learning classifier can classify the signal sample or signal bin into one of several predefined states (e.g., desynchronization state, bounce state, or rest state).
[0203] Method 500 may also include selecting an input command 18 from a plurality of conditional input commands based on duration in operation 508. Selecting an input command 18 from a plurality of conditional input commands may further include comparing the duration with one or more time thresholds associated with the conditional input command, and selecting the input command 18 based on whether the duration exceeds or fails to reach one or more time thresholds.
[0204] Method 500 may also include optional operation 510, providing the subject with visual, auditory, tactile, neural stimulation, or a combination thereof feedback regarding the selected input command 18. For example, providing visual, auditory, tactile, neural stimulation, or a combination thereof may allow the subject an opportunity to confirm the selected input command 18. In other embodiments, feedback may be provided after the input command 18 has been transmitted to the device or software to notify the subject that the input command 18 has been transmitted or is being transmitted or is in the process of being executed.
[0205] Method 500 may further include transmitting a selected input command 18 to a device or software in operation 512. In some embodiments, the input command 18 may be transmitted shortly after a second change in the subject's neurally related signals is detected.
[0206] Input commands 18 can be transmitted using one or more processors, such as telemetry unit 22, host device 16, or combinations thereof. As previously described, input commands 18 can be transmitted to one or more terminal applications (e.g., application software) running on a peripheral or personal computing device, such as a laptop, desktop computer, smartphone, or tablet. When input commands 18 are transmitted to software programs, they can be transmitted via one or more application programming interfaces (APIs). In these and other embodiments, input commands 18 can be transmitted to IoT devices, mobile vehicles (e.g., electric wheelchairs), or other types of peripheral or personal computing devices to control these devices or vehicles. Furthermore, input commands 18 can also be transmitted to one or more terminal applications (e.g., software) running on such peripheral or personal computing devices or vehicles.
[0207] One technical challenge faced by the applicant is how to allow patients with mobility impairments (including those with severely limited mobility, such as those with locked-in syndrome) to control devices or software applications, where such patients may only be able to control their thoughts and / or certain muscle groups. One solution discovered by the applicant involves detecting a first change (e.g., a decrease) in the intensity of a subject's neurally relevant signals (e.g., neural oscillations or brainwaves) below baseline levels, and detecting a second change (e.g., an increase) in the intensity of the neurally relevant signals after the first change above baseline levels, and transmitting input commands to the device upon or after the detection of the second change in the neurally relevant signals. These changes can be detected using a neural interface implanted in the subject's brain. This allows the subject to control the device or software application by generating thoughts (e.g., task-related thoughts or task-independent thoughts) and mentally releasing those thoughts.
[0208] Another technical challenge faced by the applicant is determining which of the numerous neurally relevant signals from the subject should be monitored when a reliable and repeatable signal is required to achieve such control. One solution found by the applicant is to use neural oscillations from the subject, including those in the beta band (approximately 12 Hz to 30 Hz), the gamma frequency range or gamma band (approximately 30 Hz to 140 Hz, more specifically, 60 Hz to 80 Hz), the alpha frequency range or alpha band (approximately 7 Hz to 12 Hz), the delta frequency range or delta band (approximately 0.1 Hz to 3 Hz), the theta frequency range or the theta band (approximately 4 Hz to 7 Hz), or combinations thereof. Furthermore, neurally relevant signals may also include neural oscillations in the μ band (approximately 7.5 Hz to 12.5 Hz), the sensorimotor rhythm (SMR) band (approximately 12.5 Hz to 15.5 Hz), or combinations thereof.
[0209] Another technical challenge faced by the applicant is how to allow patients with mobility impairments, including those with severely limited mobility such as locked-in syndrome, to quickly and accurately control multiple devices or software applications. One solution discovered by the applicant is based on the duration of changes in the subject's neurally relevant signals (e.g., a decrease in the intensity of neurally relevant signals). For example, the duration of changes in neurally relevant signals can be calculated by using a machine learning classifier to categorize neurally relevant events (e.g., desynchronization events or bounce events) and comparing the number of such events to a predetermined threshold associated with the conditional input command. This allows the subject to choose among different commands by holding a thought (e.g., a task-related thought or a task-irrelevant thought) for a period of time and mentally releasing the thought when the threshold duration is reached.
[0210] Figure 14The diagram illustrates the power variations in the beta band (e.g., 12 Hz to 30 Hz) and gamma band (e.g., 60 Hz to 80 Hz) frequencies as the subject arouses, holds, and subsequently releases thoughts about movement of the subject's left and right ankles. Figure 14 As shown, when a subject generates and holds a thought, the power or intensity of the neural oscillation in the β band decreases and then increases when the subject releases the thought. Figure 14 It was also shown that the power or intensity of neural oscillations in the gamma band can increase when a subject generates and holds a thought, and subsequently decrease when the subject releases the thought. For example, the thought could be associated with the subject attempting to move their left and right ankles.
[0211] Several embodiments have been described. However, those skilled in the art will understand that various changes and modifications can be made to this disclosure without departing from the spirit and scope of the embodiments. Elements of the systems, devices, apparatuses, and methods shown in any embodiment are exemplary for a particular embodiment and can be combined or otherwise used in other embodiments within this disclosure. For example, the steps of any method depicted in the figures or described in this disclosure do not require a specific or sequential order to achieve the desired result. Furthermore, other steps may be provided, or steps or operations may be eliminated or omitted from the described methods or processes to achieve the desired result. Additionally, any component or part of any apparatus or system described in this disclosure or depicted in the figures may be removed, eliminated, or omitted to achieve the desired result. Furthermore, for the sake of brevity and clarity, certain components or parts of the systems, devices, or apparatuses shown or described herein have been omitted.
[0212] Therefore, other embodiments are within the scope of the appended claims, and the description and / or drawings may be regarded as illustrative rather than restrictive.
[0213] Each of the various variations or embodiments described and illustrated herein has discrete components and features that can be readily separated from or combined with features of any other variation or embodiment. Modifications can be made to adapt the specific circumstances, materials, composition of substances, processes, actions of processes, or steps to the objectives, spirit, or scope of the invention.
[0214] The methods described in this article can be implemented in any logically possible order of the listed events, as well as in the order in which the events are listed. Furthermore, additional steps or operations can be provided, or steps or operations can be eliminated to achieve the desired result.
[0215] Furthermore, where a range of values is provided, every intermediate value between the upper and lower limits of that range, as well as any other specified value or intermediate value within that range, is included within the scope of this invention. Additionally, any optional features of the described variations of the invention may be presented and claimed independently or in combination with any one or more of the features described herein. For example, a description of a range from 1 to 5 should be considered to have disclosed subranges (e.g., from 1 to 3, from 1 to 4, from 2 to 4, from 2 to 5, from 3 to 5, etc.) and individual numbers within that range (e.g., 1.5, 2.5, etc.) and any wholly or partially increments therebetween.
[0216] All existing subjects mentioned herein (e.g., publications, patents, patent applications) are incorporated herein by reference in their entirety, unless such subject matter may conflict with the subject matter of this invention (in which case the content shown herein shall prevail). The referenced items are provided only for their disclosure prior to the filing date of this application. Nothing herein should be construed as an admission that this invention is not entitled to precede such material by virtue of a prior invention.
[0217] References to a singular item include the possibility of the existence of a plural of the same item. More specifically, unless the context clearly specifies otherwise, the singular forms “a,” “an,” “the,” and “the” as used herein and in the appended claims include the indicated plural. It should also be noted that claims may be drafted to exclude any optional elements. Therefore, this statement is intended as a priori basis for the use of such exclusive terms such as “solely,” “only,” etc., or the use of “negative” limitations in relation to referenced claim elements. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0218] When the phrase “at least one of…” modifies multiple items or components (or a list of items or components), a reference to the phrase refers to any combination of one or more of these items or components. For example, the phrase “at least one of A, B, and C” means (i) A; (ii) B; (iii) C; (iv) A, B, and C; (v) A and B; (vi) B and C; or (vii) A and C.
[0219] In understanding the scope of this disclosure, the term "comprising" and its derivatives as used herein are intended to be open-ended terms that specify the presence of said features, elements, components, groups, integers, and / or steps, but do not exclude the presence of other unstated features, elements, components, groups, integers, and / or steps. The foregoing also applies to words with similar meanings, such as the terms "including," "having," and their derivatives. Furthermore, the terms "part," "segment," "section," "component," "element," or "assembly" when used in the singular can have a dual meaning of a single part or multiple parts. As used herein, the following directional terms "forward, backward, above, downward, vertical, horizontal, below, lateral, sideways, and vertically," and any other similar directional terms, refer to those positions of the equipment or apparatus or those directions in which the equipment or apparatus is translated or moved.
[0220] Finally, the degree terms used in this paper, such as “substantially,” “approximately,” and “approximately,” refer to a specified value or a specified value plus a reasonable amount of deviation from the specified value (e.g., a deviation of up to ±0.1%, ±1%, ±5%, or ±10%, as such variation is appropriate) such that the final result is not significantly or substantially altered. For example, “approximately 1.0 cm” can be interpreted as meaning “1.0 cm” or “between 0.9 cm and 1.1 cm.” When degree terms such as “approximately” or “approximately” are used to refer to numbers or values that are part of a range, the term can be used to modify the minimum and maximum numbers or values.
[0221] This disclosure is not intended to be limited to the specific forms set forth herein, but rather to cover alternatives, modifications, and equivalents to the variations or embodiments described herein. Furthermore, the scope of this disclosure fully encompasses other variations or embodiments that may become apparent to those skilled in the art based on this disclosure.
[0222] This application also relates to the following: 1) A method for controlling a device, comprising: The intensity of neural signals in the subjects was detected to be lower than the measured baseline level; The intensity of the detected neural-related signal increases to above the baseline level after the initial decrease; and When or after an increase in the intensity of the neural-related signal is detected, an input command is transmitted to the device.
[0223] 2) According to the method of 1), wherein the subject’s neural-related signal is the subject’s neural oscillation.
[0224] 3) According to the method of 2), wherein detecting a decrease in the intensity of the neural-related signal includes detecting a decrease in the power of the neural oscillation to below the baseline oscillation power level.
[0225] 4) According to the method of 2), wherein detecting an increase in the intensity of the neural-related signal includes detecting an increase in the power of the neural oscillation to exceed the baseline oscillation power level.
[0226] 5) The method according to 2), wherein the neural oscillation comprises oscillations in one or more frequency bands.
[0227] 6) The method according to 5), wherein the neural oscillation comprises a β-band oscillation with a frequency between about 12 Hz and 30 Hz.
[0228] 7) The method according to 1), wherein an intravascular device implanted in the subject is used to measure or monitor the neural-related signal, and wherein one or more processors are used to perform the steps of detecting a decrease or increase in the intensity of the neural-related signal and transmitting the input command.
[0229] 8) The method according to 7), wherein the intravascular device is implanted in the brain of the subject.
[0230] 9) The method according to 7), wherein the nerve-related signals are measured or monitored using electrodes of an intravascular device implanted in the subject.
[0231] 10) The method according to 7), wherein the one or more processors are one or more processors of a device located outside the subject.
[0232] 11) The method according to 7), wherein the one or more processors are one or more processors of a device implanted in the subject.
[0233] 12) The method according to 7) further includes using the one or more processors to filter the raw neural-related signals obtained from the intravascular device using one or more software filters.
[0234] 13) The method according to 12) further includes feeding the filtered signal into a classification layer to automatically detect decreases and increases in the intensity of the neural-related signal using a machine learning classifier.
[0235] 14) The method according to 1) further includes providing the subject with at least one of visual feedback, auditory feedback, tactile feedback and neural stimulation feedback regarding the input command.
[0236] 15) The method according to 1), wherein transmitting the input command includes transmitting the input command to one or more terminal applications running on the device.
[0237] 16) The method according to 1), wherein the decrease in the intensity of the neurally related signal is caused by the subject arousing and maintaining a task-related thought, wherein the increase in the intensity of the neurally related signal is caused by the subject mentally releasing the task-related thought, and wherein the input command is a command to the device to perform at least a portion of a task associated with the task-related thought.
[0238] 17) The method according to 1), wherein the decrease in the intensity of the neurally related signal is caused by the subject evoking and maintaining a task-irrelevant thought, wherein the increase in the intensity of the neurally related signal is caused by the subject mentally releasing the task-irrelevant thought, and wherein the input command is a command to the device to perform at least a portion of a task unrelated to the task-irrelevant thought.
[0239] 18) The method according to 17), wherein the task-independent idea is an idea related to the subject's physical function.
[0240] 19) According to the method of 1), wherein the decrease in the intensity of the neural-related signal below the measured baseline level is the desynchronization of the neural-related signal, and wherein the increase in the intensity of the neural-related signal above the measured baseline level is the bounce of the neural-related signal.
[0241] 20) The method according to 1), wherein the device is at least one of a personal computing device, an Internet of Things (IoT) device, and a mobile vehicle.
[0242] 21) A method for controlling a device, comprising: The intensity of neural signals in the subjects was detected to be lower than the measured baseline level; The intensity of the detected neural-related signal increases to above the baseline level after the initial decrease; Determine the duration of the decrease in the intensity of the neural-related signal; Select an input command from multiple conditional input commands based on the duration; and The selected input command is transmitted to the device.
[0243] 22) According to the method of 21), wherein selecting the input command based on the duration includes: The duration is compared with one or more time thresholds associated with the plurality of conditional input commands; and The input command is selected from the plurality of conditional input commands based on whether the duration exceeds or fails to reach one or more time thresholds.
[0244] 23) According to the method of 21), wherein the subject’s neural-related signal is the subject’s neural oscillation.
[0245] 24) According to the method of 23), wherein detecting a decrease in the intensity of the neural-related signal includes detecting a decrease in the power of the neural oscillation to below the baseline oscillation power level.
[0246] 25) According to the method of 23), wherein detecting an increase in the intensity of the neural-related signal includes detecting an increase in the power of the neural oscillation to exceed the baseline oscillation power level.
[0247] 26) The method according to 23), wherein the neural oscillation comprises oscillations in one or more frequency bands.
[0248] 27) The method according to 26), wherein the neural oscillation comprises a β-band oscillation with a frequency between about 12 Hz and 30 Hz.
[0249] 28) The method according to 21), wherein an intravascular device implanted in the subject is used to measure or monitor the nerve-related signal, and wherein one or more processors are used to perform the steps of detecting a decrease or increase in the intensity of the nerve-related signal, determining the duration of the decrease in the intensity of the nerve-related signal, selecting the input command, and transmitting the input command.
[0250] 29) The method according to 28), wherein the intravascular device is implanted in the brain of the subject.
[0251] 30) The method according to 28), wherein electrodes of an intravascular device implanted in the subject are used to measure or monitor the neural-related signals.
[0252] 31) The method according to 28), wherein the one or more processors are one or more processors of a device located outside the subject.
[0253] 32) The method according to 28), wherein the one or more processors are one or more processors of a device implanted in the subject.
[0254] 33) The method according to 28) further includes using the one or more processors to filter the raw neural-related signals obtained from the intravascular device using one or more software filters.
[0255] 34) The method according to 33) further includes feeding the filtered signal into a classification layer to automatically detect decreases and increases in the intensity of the neural-related signal using a machine learning classifier.
[0256] 35) The method according to 21) further includes providing the subject with at least one of visual feedback, auditory feedback, tactile feedback and neural stimulation feedback regarding the selected input command before transmitting the input command to the device.
[0257] 36) The method according to 21), wherein transmitting the input command includes transmitting the input command to one or more terminal applications running on the device.
[0258] 37) The method according to 21), wherein the decrease in the intensity of the neural-related signal is caused by the subject arousing and maintaining a task-related thought, wherein the increase in the intensity of the neural-related signal is caused by the subject mentally releasing the task-related thought, wherein the duration of the decrease in the intensity of the neural-related signal is the amount of time the subject maintains the task-related thought, and wherein the input command is a command to the device to perform at least a portion of a task associated with the task-related thought.
[0259] 38) The method according to 21), wherein the decrease in the intensity of the neurally related signal is caused by the subject evoking and maintaining a task-irrelevant thought, wherein the increase in the intensity of the neurally related signal is caused by the subject mentally releasing the task-irrelevant thought, wherein the duration of the decrease in the intensity of the neurally related signal is the amount of time the subject maintains the task-irrelevant thought, and wherein the input command is a command to the device to perform at least a portion of a task unrelated to the task-irrelevant thought.
[0260] 39) According to the method of 38), wherein the task-independent idea is an idea related to the subject's physical function.
[0261] 40) According to the method of 21), wherein the decrease in the intensity of the neural-related signal below the measured baseline level is the desynchronization of the neural-related signal, and wherein the increase in the intensity of the neural-related signal above the measured baseline level is the bounce of the neural-related signal.
[0262] 41) A method for controlling a device, the method comprising: The first change in the subject's neural signals was detected; Detecting a second change in the neurally related signal; and Upon or after the detection of a second change in the neural-related signal, an input command is transmitted to the device.
[0263] 42) The method according to 41) further includes determining the duration of a first change in the neural-related signal and using the duration to select an input command from a plurality of conditional input commands based on the duration.
[0264] 43) According to the method of 41), wherein the first change is that the intensity of the neural-related signal is reduced to below the baseline signal level.
[0265] 44) According to the method of 43), wherein the second change is an increase in the intensity of the neural-related signal to exceed the baseline signal level.
[0266] 45) The method according to 41), wherein the first change is an increase in the intensity of the neurally related signal to exceed the baseline signal level, and wherein the neurally related signal is a neural oscillation of the subject, and wherein the neural oscillation is a gamma band oscillation with a frequency between about 30 Hz and 140 Hz.
[0267] 46) According to the method of 45), wherein the second change is that the intensity of the neural-related signal is reduced to below the baseline signal level.
[0268] 47) The method according to 41), wherein the first change occurs when the subject generates and maintains an idea.
[0269] 48) The method according to 47), wherein the second change occurs when the subject mentally releases the thought.
[0270] 49) The method according to 41), wherein the first change occurs when the subject mentally releases the first thought.
[0271] 50) The method according to 49), wherein the second change occurs when the subject generates and maintains the second thought.
[0272] 51) A system for controlling equipment, comprising: Intravascular devices configured to measure or monitor neurally related signals in a subject; and An apparatus including one or more processors, wherein the one or more processors are programmed to: The intensity of neural signals in the subjects decreased to below the measured baseline level. The intensity of the detected neural-related signal increases to above the baseline level after the initial decrease, and When or after an increase in the intensity of the neural-related signal is detected, an input command is transmitted to the device.
[0273] 52) According to the system described in 51), wherein the subject’s neural-related signal is the subject’s neural oscillation.
[0274] 53) The system according to 52), wherein one or more processors are programmed to detect a decrease in the intensity of the neural-related signal by detecting a decrease in the power of the neural oscillation to below a baseline oscillation power level.
[0275] 54) The system according to 52), wherein the one or more processors are programmed to detect an increase in the intensity of the neural-related signal by detecting an increase in the power of the neural oscillation to exceed a baseline oscillation power level.
[0276] 55) The system according to 52) wherein the neural oscillation comprises oscillations of one or more frequency bands.
[0277] 56) The system according to 55) wherein the neural oscillation comprises a β-band oscillation with a frequency between about 12 Hz and 30 Hz.
[0278] 57) The system according to 51), wherein the intravascular device is configured to be implanted in the brain of the subject.
[0279] 58) The system according to 51), wherein the intravascular device is configured to be implanted in the vein or sinus of the subject.
[0280] 59) The system according to 51) wherein electrodes of an intravascular device implanted in the subject are used to measure or monitor the neural-related signals.
[0281] 60) The system according to 51), wherein the device is configured to be located outside the body of the subject.
[0282] 61) The system according to 51), wherein the device is configured to be implanted in the subject.
[0283] 62) The system according to 51), wherein the one or more processors are further programmed to filter the raw neural-related signals obtained from the intravascular device using one or more software filters.
[0284] 63) According to the system of 62), wherein the one or more processors are further programmed to feed the filtered signal into a classification layer to automatically detect decreases and increases in the intensity of the neural-related signal using a machine learning classifier.
[0285] 64) The system according to 51), wherein the one or more processors are further programmed to provide the subject with at least one of visual feedback, auditory feedback, tactile feedback and neural stimulation in the form of feedback regarding the input command via the intravascular device.
[0286] 65) The system according to 51), wherein the one or more processors are further programmed to transmit the input commands to one or more terminal applications running on the device.
[0287] 66) The system according to 51), wherein the decrease in the intensity of the neurally related signal is caused by the subject arousing and maintaining a task-related thought, wherein the increase in the intensity of the neurally related signal is caused by the subject mentally releasing the task-related thought, and wherein the input command is a command to the device to perform at least a portion of a task related to the task-related thought.
[0288] 67) The system according to 51), wherein the decrease in the intensity of the neurally related signal is caused by the subject evoking and maintaining a task-irrelevant thought, wherein the increase in the intensity of the neurally related signal is caused by the subject mentally releasing the task-irrelevant thought, and wherein the input command is a command to the device to perform at least a portion of a task unrelated to the task-irrelevant thought.
[0289] 68) According to the system described in 67), the task-independent idea is an idea related to the subject's physical function.
[0290] 69) According to the system of 51), wherein a decrease in the intensity of the neural-related signal below a measured baseline level is desynchronization of the neural-related signal, and an increase in the intensity of the neural-related signal above a measured baseline level is a bounce of the neural-related signal.
[0291] 70) The system according to 51), wherein the device is at least one of a personal computing device, an Internet of Things (IoT) device, and a mobile vehicle.
[0292] 71) A system for controlling equipment, comprising: Intravascular devices configured to measure or monitor neurally related signals in a subject; and An apparatus including one or more processors, wherein the one or more processors are programmed to: The intensity of neural signals in the subjects decreased to below the measured baseline level. The intensity of the detected neural-related signal increases to above the baseline level after the initial decrease. Determine the duration of the decrease in the intensity of the neurally related signal. Selecting an input command from multiple conditional input commands based on the duration, and The selected input command is transmitted to the device.
[0293] 72) The system according to 71), wherein the one or more processors are further programmed to: The duration is compared with one or more time thresholds associated with the plurality of conditional input commands; and The input command is selected from the plurality of conditional input commands based on whether the duration exceeds or fails to reach one or more time thresholds.
[0294] 73) According to the system of 71), wherein the subject’s neural-related signal is the subject’s neural oscillation.
[0295] 74) The system according to 73), wherein one or more processors are programmed to detect when the power of the neural oscillation decreases to below the baseline oscillation power level.
[0296] 75) The system according to 73), wherein one or more processors are programmed to detect when the power of the neural oscillation increases to exceed the baseline oscillation power level.
[0297] 76) The system according to 73) wherein the neural oscillation comprises oscillations of one or more frequency bands.
[0298] 77) The system according to 76) wherein the neural oscillation comprises a β-band oscillation with a frequency between about 12 Hz and 30 Hz.
[0299] 78) The system according to 71), wherein the intravascular device is configured to be implanted in the brain of the subject.
[0300] 79) The system according to 71), wherein the intravascular device is configured to be implanted in the vein or sinus of the subject.
[0301] 80) The system according to 71) wherein electrodes of an intravascular device implanted in the subject are used to measure or monitor the neural-related signals.
[0302] 81) The system according to 71), wherein the device is configured to be located outside the body of the subject.
[0303] 82) The system according to 71), wherein the device is configured to be implanted in the subject.
[0304] 83) The system according to 71), wherein the one or more processors are further programmed to filter the raw neural-related signals obtained from the intravascular device using one or more software filters.
[0305] 84) The system according to 83) wherein the one or more processors are further programmed to feed the filtered signal into a classification layer to automatically detect decreases and increases in the intensity of the neural-related signal using a machine learning classifier.
[0306] 85) The system according to 71), wherein the one or more processors are further programmed to provide the subject with at least one of visual, auditory, tactile, and neural stimulation feedback regarding the input command via the intravascular device.
[0307] 86) The system according to 71) wherein the one or more processors are further programmed to transmit the input commands to one or more terminal applications running on the device.
[0308] 87) The system according to 71), wherein the decrease in the intensity of the neural-related signal is caused by the subject arousing and maintaining a task-related thought, wherein the increase in the intensity of the neural-related signal is caused by the subject mentally releasing the task-related thought, wherein the duration of the decrease in the intensity of the neural-related signal is the amount of time the subject maintains the task-related thought, and wherein the input command is a command to the device to perform at least a portion of a task associated with the task-related thought.
[0309] 88) The system according to 71), wherein the decrease in the intensity of the neurally related signal is caused by the subject waking up and holding a task-irrelevant thought, wherein the increase in the intensity of the neurally related signal is caused by the subject mentally releasing the task-irrelevant thought, wherein the duration of the decrease in the intensity of the neurally related signal is the amount of time the subject holds the task-irrelevant thought, and wherein the input command is a command to the device to perform at least a portion of a task unrelated to the task-irrelevant thought.
[0310] 89) According to the system described in 88), the task-independent idea is an idea related to the subject's physical function.
[0311] 90) According to the system of 71), wherein a decrease in the intensity of the neural-related signal below a measured baseline level is desynchronization of the neural-related signal, and wherein an increase in the intensity of the neural-related signal above a measured baseline level is a bounce of the neural-related signal.
[0312] 91) A system for controlling equipment, comprising: Intravascular devices configured to measure or monitor neurally related signals in a subject; and An apparatus including one or more processors, wherein the one or more processors are programmed to: Detect the first change in the subject's neural-related signals; Detecting a second change in the neurally related signal; and Upon or after the detection of a second change in the neural-related signal, an input command is transmitted to the device.
[0313] 92) The system according to 91), wherein the one or more processors are further programmed to determine the duration of a first change in the neurally related signal and to use the duration to select an input command from a plurality of conditional input commands based on the duration.
[0314] 93) The system according to 91), wherein the first change is that the intensity of the neural-related signal is reduced to below the baseline signal level.
[0315] 94) The system according to 93), wherein the second change is that the intensity of the neural-related signal increases to exceed the baseline signal level.
[0316] 95) The system according to 94), wherein the neurally related signal is a neural oscillation of the subject, and wherein the neural oscillation is a β-band oscillation with a frequency between about 12 Hz and 30 Hz.
[0317] 96) The system according to 91), wherein the first change is that the intensity of the neural-related signal increases to exceed the baseline signal level.
[0318] 97) The system according to 96), wherein the second change is that the intensity of the neural-related signal is reduced to below the baseline signal level.
[0319] 98) The system according to 97) wherein the neural-related signal is a neural oscillation of the subject, and wherein the neural oscillation is a gamma band oscillation with a frequency between about 30 Hz and 140 Hz.
[0320] 99) According to the system described in 91), the first change occurs when the subject generates and maintains an idea.
[0321] 100) The system according to 99) wherein the second change occurs when the subject mentally releases the thought.
[0322] 101) The system according to 91) wherein the first change occurs when the subject mentally releases the first thought.
[0323] 102) The system according to 101) wherein the second change occurs when the subject generates and maintains the second thought.
Claims
1. A method for controlling a device, comprising: The intensity of neural signals detected in the subjects increased to levels exceeding the measured baseline. The intensity of the detected neural-related signal decreases below the measured baseline level after the increase; and When the intensity of the neural signal decreases after an increase in the detected intensity, an input command is transmitted to the device.
2. The method according to claim 1, wherein, The neural-related signal is the subject's neural oscillation.
3. The method according to claim 2, wherein, Detecting an increase in the intensity of the neural-related signal includes detecting an increase in the power of the neural oscillation to exceed the baseline oscillation power level.
4. The method according to claim 3, wherein, The power referred to is the power spectral density.
5. The method according to claim 3, wherein, Detecting a decrease in the intensity of the neural-related signal includes detecting a decrease in the power of the neural oscillation.
6. The method according to claim 1, wherein, The nerve-related signals are measured or monitored using an intravascular device implanted in the subject, wherein one or more processors are used to perform the steps of detecting an increase or decrease in the intensity of the nerve-related signals and transmitting the input commands.
7. The method according to claim 1, wherein, The increase in the intensity of the neurally related signal is caused by the subject mentally releasing a first thought, and the decrease in the intensity of the neurally related signal is caused by the subject arousing and holding a second thought.
8. A system for controlling equipment, comprising: An intravascular device configured to measure or monitor neurally related signals of a subject, wherein the neurally related signals of the subject are neural oscillations of the subject; and An apparatus comprising one or more processors, wherein said one or more processors are programmed to: The intensity of the detected neural signal increases to exceed a measured baseline level, wherein detecting the increase in the intensity of the detected neural signal includes detecting the power of the detected neural oscillation increases to exceed a baseline oscillation power level; The intensity of the detected neural-related signal decreases below the measured baseline level after the increase; and When the intensity of the neural signal decreases after an increase in the detected intensity, an input command is transmitted to the device.
9. The system according to claim 8, wherein, The one or more processors are programmed to detect an increase in the intensity of the neural-related signal by detecting an increase in the power of the neural oscillation to exceed a baseline oscillation power level.
10. The system according to claim 9, wherein, The power referred to is the power spectral density.
11. The system according to claim 8, wherein, The intravascular device is configured to be implanted into a vein or sinus in the subject's brain.
12. A method for controlling a device, comprising: Using neural interfaces to measure or monitor neural signals in subjects; The measured neural-related signals are fed into a machine learning classifier; The neural-related signals are classified into one or more events using the machine learning classifier, wherein the one or more events include at least one of desynchronization events, bounce events, or rest events; as well as Based on the selection of one or more events classified by the machine learning classifier, an input command will be transmitted to the device.
13. The method according to claim 12, wherein, The input command to be transmitted to the device is selected based on the events classified by the machine learning classifier, including selecting the input command based on at least one of a series of events and the number of events.
14. The method according to claim 13, wherein, The series of events is one or more desynchronization events followed by bounce events.
15. The method according to claim 13, wherein, The series of events is a bounce event followed by one or more desynchronization events.
16. The method according to claim 12, wherein, The desynchronization event is when the intensity of neurally related signals decreases to below the baseline level.
17. The method according to claim 16, wherein, The decrease in the intensity of the neural-related signal is a decrease in the power of the subject's neural oscillations.
18. The method according to claim 17, wherein, The power of the neural oscillation is the power spectral density.
19. The method according to claim 12, wherein, The rebound event is defined as an increase in the intensity of neurally related signals above the baseline level following a desynchronization event.
20. The method of claim 12, further comprising filtering the neurally related signal measured prior to feeding the neurally related signal into the machine learning classifier.