Control of computer processing through the conversion of biosignals and prediction of traumatic brain injury based on sleep state.

Abstract

The present invention provides a method and system for transforming biosignals to perform various operations associated with a computing device. The method may include accessing biosignal data collected by a biosignal data acquisition assembly, which includes a housing having one or more clusters of electrodes. Each cluster of one or more electrodes may include at least one active electrode. The method may also include identifying a first signal based on the biosignal data, which represents a first intention to move a first part of a subject's body. The first signal is generated before a second signal, which represents a second intention to move a second part of the subject's body. The method may also include transforming the first signal to identify a first operation to be performed by a computing device.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims the benefits of U.S. Provisional Application No. 63 / 452,268, titled "CONTROL OF COMPUTER OPERATIONS VIA TRANSLATION OF BIOLOGICAL SIGNALS," filed on 15 March 2023, and U.S. Provisional Application No. 63 / 452,275, titled "TRAUMATIC BRAIN INJURY PREDICTION BASED ON SLEEP STATES," both of which are incorporated herein by reference in their entirety.

[0002] This disclosure generally relates to converting biosignals from a subject and identifying operations performed by a computing device. Specifically, this disclosure relates to methods and systems for analyzing activation sequences of biosignals to identify one or more operations performed by a computing device. Furthermore, this disclosure generally relates to analyzing physiological data, and more specifically (though not necessarily exclusive), to predicting the presence of traumatic brain injury based on indicators associated with sleep state. [Background technology]

[0003] Control of computer processing through the conversion of biosignals Within the brain, various nerve cells work together to generate a rich and continuous set of neural electrical signals. Such signals have a powerful influence on bodily control. For example, signals can initiate physical movement and facilitate cognitive thought. Furthermore, neural signals can wake a person during sleep. A deeper understanding of the biological pathways of signal-action could offer the possibility of using biosignals to perform actions previously impossible for humans (e.g., using thought to move a mouse cursor).

[0004] Brain-computer interfaces (BCIs) can be configured to convert the electrical activity of the brain to determine operations performed by external devices. For example, they can analyze biosignals from the brain to control a cursor or operate an orthotic device. Therefore, BCIs are often used to study, map, support, enhance, or repair human cognitive or sensorimotor functions. BCI implementations range from non-invasive procedures (EEG, MEG, EOG, MRI) to partially invasive procedures (ECoG and intravascular) and invasive procedures (microelectrode arrays), with the invasiveness of the procedure based on how closely the electrodes are positioned relative to the brain tissue.

[0005] Despite decades of intensive research, the direct conversion of brain signals to various human actions remains challenging due to the complexity of the signals. Furthermore, directly converting brain signals to identify specific tasks can be inefficient because it requires several signal preprocessing steps before the signals are actually converted. For example, converting brain signals to write English sentences might require removing noise from brain signals unrelated to the ongoing task. Similar difficulties can arise when converting brain signals to human actions related to controlling other devices. Therefore, there is a need for technologies that can efficiently convert biosignals to perform various computing device processes.

[0006] Prediction of traumatic brain injury based on sleep state An electroencephalogram (EEG) is a tool used to measure the electrical activity generated by the brain. The brain's functional activity is collected by electrodes placed on the subject's scalp. Conventional monitoring and diagnostic devices include multiple electrodes attached to the subject, which tap brain signals and transmit those signals via cables to an amplifier unit. The resulting EEG signals can be used to diagnose and monitor various conditions affecting the brain.

[0007] For example, traumatic brain injury (TBI) can occur when the normal function of the brain is damaged by an external force experienced by the subject (e.g., impact to the head, sudden acceleration or deceleration, or piercing head injury). TBI can be classified as mild, moderate, or severe based on the severity of the damage to normal brain function when the subject experiences the external force. Symptoms of mild TBI (i.e., concussion) include headache, dizziness, visual disturbances, sensitivity to light, and behavioral changes, while symptoms of moderate to severe TBI include the above symptoms plus slurred speech, nausea, seizures, and loss of consciousness.

[0008] However, detecting and diagnosing TBI can be difficult. For example, diagnosing TBI may involve performing a neurological examination on the subject, which can assess not only the symptoms mentioned above but also thinking, motor function, coordination, sensory function, reflexes, etc. Since normal or average motor function, coordination, etc., vary from subject to subject and are unique, it can be difficult to determine from a neurological examination whether a subject has TBI. Therefore, neurological examination may not be an effective method for diagnosing TBI because it may not be an effective way to determine whether a subject is experiencing changes in thinking, motor function, coordination, etc. Neurological examination may be particularly ineffective when symptoms are subtle (e.g., mild TBI) and / or when the subject's baseline information (i.e., normal thinking, motor function, coordination, etc.) is unknown. Furthermore, there are currently no Food and Drug Administration (FDA) approved medical devices intended for standalone use in diagnosing TBI. For example, imaging techniques (e.g., CT scans, MRI scans, etc.) often fail to show signs of traumatic brain injury. In particular, while imaging techniques may detect bleeding or other appropriate signs of moderate or severe TBI, they cannot detect signs of mild TBI. Therefore, more reliable techniques for the detection and diagnosis of TBI may be needed. [Overview of the Initiative]

[0009] In some embodiments, a method is provided for translating biosignals to perform various operations associated with a computing device. The method may include accessing biosignal data collected by a biosignal data acquisition assembly, which includes a housing having one or more clusters of electrodes. Each cluster of one or more electrodes may include at least one active electrode. The method may also include identifying a first signal based on the biosignal data, representing a first intention to move a first part of a subject's body. The first signal is generated before a second signal, which represents a second intention to move a second part of the subject's body. The method may also include translating the first signal to identify a first operation to be performed by the computing device. The method may also include outputting a first instruction to perform the first operation.

[0010] In some embodiments, the biosignal data includes electroencephalography (EEG) data, where the first signal is generated from the left hemisphere of the subject's brain and the second signal is generated from the right hemisphere of the brain. In some embodiments, the biosignal data includes electromyography (EMG) data, where the first part is from the subject's left limb and the second part is from the subject's right limb.

[0011] The first operation may include performing one or more functions associated with the graphical user interface of the computing device. For example, the first operation may include moving a cursor displayed on the graphical user interface from a first position to a second position. In another embodiment, the first operation may include entering text on the graphical user interface. After the text is entered on the graphical user interface, one or more machine learning models can be applied to the entered text to predict additional text to be entered on the graphical user interface. In yet another embodiment, the first operation may include entering one or more images or icons on the graphical user interface. In some embodiments, the first operation includes launching an application stored on the computing device or executing one or more commands associated with the application.

[0012] In some embodiments, the first operation includes accessing one or more interface elements of the intent communication interface to identify one or more operations to be performed by the computing device. In some embodiments, the intent communication interface is a tree including a root interface element connected to the first interface element and the second interface element.

[0013] Accessing interface elements may involve selecting a first interface element of an intent communication interface in preference to a second interface element. The first interface element is associated with first interface operation data, and the second interface element is associated with second interface operation data. The second operation performed by the computing device is identified based on the first interface operation data. A second instruction for performing the second operation may be output.

[0014] Other interface elements of the intent communication interface may be accessed based on biosignals collected at different points in time. Additional biosignal data collected by the biosignal data acquisition assembly may be accessed at a later point in time. Based on the additional biosignal data, a third signal representing a third intent to move a second part of the subject's body may be identified. In some embodiments, the third signal is generated before a fourth signal, which represents a fourth intent to move a first part of the subject's body. The third signal can be transformed to identify a third operation to be performed by the computing device. Based on the third operation, a third interface element of the intent communication interface may be selected in preference to a fourth interface element, with the third interface element associated with third interface operation data and the fourth interface element associated with fourth interface operation data. The third and fourth interface elements are connected to the first interface element. The fourth operation to be performed by the computing device may be identified by accessing the third interface operation data of the selected third interface element. In some embodiments, the fourth operation includes entering one or more alphanumeric characters on the computing device's graphical user interface. A third instruction for performing the fourth operation may be output.

[0015] Additionally or alternatively, the first operation can be used to control various devices. For example, the computing device may be an augmented reality device or a virtual reality device, and the first operation may include performing one or more operations associated with the augmented reality device or a virtual reality device. In another embodiment, the computing device may include one or more robotic components, and the first operation may include controlling one or more robotic components.

[0016] Some embodiments of the present disclosure include a system that includes one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium that includes instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more of the methods disclosed herein and / or some or all of one or more of the processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium that includes instructions configured to cause one or more data processors to perform some or all of one or more of the methods disclosed herein and / or some or all of one or more of the processes disclosed herein.

[0017] Some embodiments relate to a computer-implemented method. The method includes accessing neuro-signal data indicative of electrical activity from a part of a subject's brain over one or more sleep time periods, predicting, for each of one or more time intervals in the one or more sleep time periods, an interval-specific indicator associated with a sleep stage, generating an aggregate indicator based on the interval-specific indicator, generating a risk level indicator for the subject based on the aggregate indicator, and outputting a result based on or representing the aggregate indicator.

[0018] In some embodiments, the aggregate indicator corresponds to an estimated absolute or relative time that the subject was in the sleep stage of stage 2. In some embodiments, the risk level indicator represents the likelihood that the subject has suffered a traumatic brain injury.

[0019] In some embodiments, predicting the interval-specific indicator includes performing at least one Fourier transform on the neuro-signal data within the interval. In some embodiments, the method includes determining that an alert condition is satisfied based on the aggregate indicator. In some embodiments, the result is output in response to determining that the alert condition is satisfied.

[0020] In some embodiments, outputting the results includes transmitting an alert communication to a third-party system associated with the monitoring of the subject. In some embodiments, the neural signal data includes electroencephalogram data. In some embodiments, the segment-specific indicator identifies the predicted sleep stage. In some embodiments, the segment-specific indicator identifies the predicted probability that the subject is in the sleep stage of stage 2.

[0021] Some embodiments relate to a system. The system includes one or more data processors and a non-transitory computer-readable storage medium containing instructions. When executed on the one or more data processors, the instructions cause the one or more data processors to access neural signal data indicative of electrical activity from a part of the subject's brain over one or more sleep time periods, predict a segment-specific indicator associated with a sleep stage for each of one or more time segments within the one or more sleep time periods, generate an aggregate indicator based on the segment-specific indicator, generate a risk level indicator for the subject based on the aggregate indicator, and output a result based on or representing the aggregate indicator.

[0022] In some embodiments, the aggregate indicator corresponds to the estimated absolute or relative time that the subject was in the sleep stage of stage 2. In some embodiments, the risk level indicator represents the likelihood that the subject has suffered a traumatic brain injury.

[0023] In some embodiments, predicting the segment-specific indicator includes performing at least one Fourier transform on the neural signal data within the segment. In some embodiments, the instructions cause the one or more data processors to further determine that an alert condition is satisfied based on the aggregate indicator when executed on the one or more data processors. In some embodiments, the result is output in response to determining that the alert condition is satisfied.

[0024] In some embodiments, outputting results includes sending an alert communication to a third-party system associated with monitoring the subject. In some embodiments, neural signal data includes electroencephalogram (EEG) data. In some embodiments, a segment-specific index identifies the predicted sleep stage. In some embodiments, a segment-specific index identifies the predicted probability that the subject is in stage 2 sleep.

[0025] Some embodiments relate to a computer program product tangibly embodied in a non-temporary machine-readable storage medium, including instructions. The instructions cause one or more data processors to access neural signal data representing electrical activity from a portion of a subject's brain over one or more sleep periods, to predict a segment-specific index associated with a sleep stage for each of one or more time segments within one or more sleep periods, to generate a cumulative index based on the segment-specific index, to generate a risk level index for the subject based on the cumulative index, and to output results based on or representing the cumulative index.

[0026] In some embodiments, the cumulative index corresponds to the estimated absolute or relative time the subject was in stage 2 sleep. In some embodiments, the risk level index represents the likelihood that the subject has suffered traumatic brain injury.

[0027] In some embodiments, predicting a segment-specific metric involves performing at least one Fourier transform on the neural signal data within the segment. In some embodiments, the instruction causes one or more data processors to further determine that an alert condition has been met based on the cumulative metric. In some embodiments, a result is output in response to the determination that the alert condition has been met.

[0028] In some embodiments, outputting results includes sending an alert communication to a third-party system associated with monitoring the subject. In some embodiments, neural signal data includes electroencephalogram (EEG) data. In some embodiments, a segment-specific index identifies the predicted sleep stage. In some embodiments, a segment-specific index identifies the predicted probability that the subject is in stage 2 sleep.

[0029] The terms and expressions used are for illustrative purposes only, not limitation, and in using such terms and expressions there is no intention to exclude equivalents of the illustrated and described features or parts thereof, but it should be recognized that various modifications are possible within the scope of this disclosure. Accordingly, while the invention as described in the claims is specifically disclosed by some embodiments and any features, modifications and variations of the concepts disclosed herein may be made by those skilled in the art, and it should be understood that such modifications and variations are deemed to be within the scope defined by the appended claims. [Brief explanation of the drawing]

[0030] [Figure 1] This figure shows a user wearing a multi-electrode miniature device that communicates wirelessly with other electronic devices. [Figure 2] This figure shows one embodiment of a network-connected device designed to facilitate the coordinated evaluation and use of electrical recordings of living organisms. [Figure 3] This figure shows one embodiment of a multi-electrode device that communicates wirelessly with other electronic devices. [Figure 4] This is a simplified block diagram of one embodiment of a multi-electrode device. [Figure 5] This is a simplified block diagram of one embodiment of an electronic device that communicates with a multi-electrode device. [Figure 6] This is a flowchart of one embodiment of a process for collecting biological electrode data channels using a multi-electrode device. [Figure 7]This is a flowchart illustrating one embodiment of a process for analyzing channel biodata to identify frequency signatures at various biological stages. [Figure 8] This is a flowchart illustrating one embodiment of a process for analyzing channel biodata to identify frequency signatures at various biological stages. [Figure 9] This is a flowchart of one embodiment of a process for normalizing a spectrogram and classifying biometric data using group discrimination frequency signatures. [Figure 10] This is a schematic diagram showing an example of determining the activation sequence of a biological signal according to some embodiments. [Figure 11] This figure shows an example of an intent communication interface used to convert biosignal data into processing for one or more computing devices, according to some embodiments. [Figure 12] This figure shows a process, according to some embodiments, for converting biosignal data into processing for one or more computing devices. [Figure 13] This is an exemplary schematic diagram illustrating the use of an intent communication interface for inputting text and images, according to some embodiments. [Figure 14] This figure shows an example of an intent communication interface for inputting images, according to some embodiments. [Figure 15] This figure shows another example of an intent communication interface for inputting text in another language, according to some embodiments. [Figure 16] This figure shows an example of an intent communication interface for processing computer applications. [Figure 17] This is a schematic diagram illustrating the use of machine learning techniques to enhance the intent communication interface, as demonstrated in some embodiments. [Figure 18] This figure shows an exemplary process of a recurrent neural network for generating predicted words based on text data, according to one embodiment. [Figure 19]This figure shows another example of recurrent neural network processing for generating predicted words based on text data, according to some embodiments. [Figure 20] This is an illustrative schematic diagram of a long- and short-term memory network for generating predicted words based on text data, according to one embodiment. [Figure 21] This is an exemplary schematic diagram for implementing forget gates and input gates in a long- and short-term memory network according to some embodiments. [Figure 22] This figure shows an exemplary processing of the output gate of a long- and short-term memory network according to one embodiment. [Figure 23] This is an exemplary schematic diagram illustrating the use of an intent communication interface for translating biosignal data into one or more processes associated with a virtual reality device, according to some embodiments. [Figure 24] This is an exemplary schematic diagram illustrating the use of an intent communication interface for converting biosignal data into one or more processes associated with a computing device having one or more robotic components, according to some embodiments. [Figure 25] This is an exemplary schematic diagram illustrating the use of an intent communication interface for converting biosignal data into one or more processes associated with an accessory device, according to some embodiments. [Figure 26] This figure shows a computing system capable of implementing either of the computing systems or environments described above. [Figure 27] This is a block diagram of an example of a system for acquiring physiological data according to one embodiment of the present disclosure. [Figure 28] This figure shows an example of a graph predicting stage 2 sleep according to one embodiment of the present disclosure. [Figure 29] This is a block diagram of an example of a system that predicts the presence of traumatic brain injury based on indicators associated with sleep state, according to one embodiment of the present disclosure. [Figure 30]This is a block diagram of an example of a computing system that predicts the presence of traumatic brain injury based on indicators associated with sleep state, according to one embodiment of the present disclosure. [Figure 31] This is a flowchart of a process for predicting the presence of traumatic brain injury based on indicators associated with sleep state, according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0031] Certain embodiments disclosed herein facilitate the conversion of biosignals (e.g., electroencephalography (EEG) data, electromyography (EMG) data) and enable the identification of various operations associated with a computing device. In some embodiments, a signal processing application accesses the subject's biosignal data. In some cases, the biosignal data is collected by a biosignal data acquisition assembly. A biosignal data acquisition assembly (e.g., the multi-electrode device 110 in Figure 1) may include a housing having one or more electrode clusters, where each cluster of one or more electrode clusters includes at least one active electrode. The biosignal data collected by the biosignal data acquisition assembly may include different types of biosignals. For example, the biosignal data may include EEG data collected from electrodes placed on the subject's forehead. In another embodiment, the biosignal data may include EMG data collected from electrodes placed on the subject's limbs. In some cases, the biosignal data is accessed via a wireless communication network (e.g., a short-range communication network).

[0032] Biosignals from a subject can be analyzed by a signal processing application to detect signal activation sequences. For example, detecting signal activation sequences may involve processing biosignal data to identify a first signal representing a first intention to move a first part of the subject's body, where the first signal is generated before a second signal. In some cases, the second signal represents a second intention to move a second part of the subject's body. For example, if the biosignal data includes EEG data, the first signal is generated from the left hemisphere of the brain and it is generated before a second signal generated from the right hemisphere of the subject's brain. In another embodiment, if the biosignal data includes EMG data, it is possible to analyze the biosignal data to detect that a first signal representing an intention to move a first muscle (e.g., the left arm) is generated before a second signal representing another intention to move a second muscle (e.g., the right arm). Additionally or alternatively, both EEG and EMG data can be used to determine that the first signal was generated before the second signal.

[0033] Based on a first signal generated before a second signal, the signal processing application identifies a specific operation to be performed by a computing device. This operation may include entering one or more alphanumeric characters on the computing device's graphical user interface. In another embodiment, this operation may include moving a cursor displayed by the graphical user interface. This operation may also include operations performed by different types of computing devices, including controlling one or more robotic components or an augmented reality or virtual reality device. The signal processing application may then output a command for the computing device to perform the identified operation. In some cases, the signal processing application resides inside the computing device, where the computing device can directly access the command and perform the operation. In some embodiments, the signal processing application resides outside the computing device. For example, the signal processing application may be part of an interface system (e.g., a BCI system), where the signal processing application can send commands to the computing device via a communication network and perform the operation. Additionally or alternatively, a signal processing application may send commands to one or more accessory devices (e.g., smartwatches) that are communicatively coupled to a computing device, enabling them to perform identified operations.

[0034] In some embodiments, an identified operation includes accessing interface operation data from one or more intent communication interfaces, where the interface operation data is used to determine other operations performed by a computing device. In some cases, the intent communication interface includes a set of interface elements, at least one of which includes corresponding interface operation data. In an exemplary embodiment, a tree containing multiple nodes is accessible, where each node of the tree is connected to one or more child nodes. Each interface element may include interface operation data that identifies a particular operation, which may be accessed when biosignal data indicates that the left and right parts of the body are activated simultaneously (e.g., both parts are activated within a given time interval). The interface operation data may be used by the same or other computing devices to perform a particular operation. For example, an interface element may include interface operation data corresponding to the alphabetical letter "z", where the identified operation performed by the computing device includes entering the letter "z" into a graphical user interface associated with this computing device.

[0035] In some cases, a sequence of biosignal activations is used over multiple instances to traverse one or more interface elements of the intent communication interface until a specific interface element is accessed and the associated operation is accessed. In an exemplary embodiment, a user interface operation may begin from the root interface element of the intent communication interface. At first time point, it is possible to process the biosignals detected from the subject to determine that a first signal representing an intention to move a first part of the body (e.g., an intention to grasp the left hand) was generated before a second signal representing another intention to move a second part of the body (e.g., an intention to grasp the right hand). Based on such a determination, the left child interface element connected to the root interface element can be accessed. If it is determined that the left child interface element contains two child interface elements, the traversal of the intent communication interface can continue at the left child interface element. Next, at second time point, it is possible to analyze the biosignals detected from the subject to determine that a third signal representing a third intention to move a second part of the body was generated before a fourth signal representing a fourth intention to move a first part of the body. Accordingly, the right child interface element connected to the previous interface element can be accessed. If it is determined that the rightmost child interface element contains two of its own child interface elements, the traversal of the intent communication interface continues. As a result, the traversal of the intent communication interface can be performed over subsequent points in time until a specific interface element is reached. From a specific interface element, the interface operation data associated with that interface element can be accessed based on the detection of other biosignal data representing the intention to move both the left and right parts of the body simultaneously. Next, a specific operation to be performed by a computing device (e.g., entering the number "1") can be identified from the interface operation data. After the operation is performed, the intent communication interface traversal process can be iterated from the root interface element until the desired result (e.g., entering a complete sentence) is reached.

[0036] Intentional communication interfaces for converting activation sequences of biosignals are applicable and can enhance various operations associated with computing devices. In some cases, interface elements of an intention communication interface identify one or more words or phrases predicted by a machine learning model. For example, suppose previously entered text data on a graphical user interface includes "the teacher typed into his computer...". Based on the previous text data, it is possible to update one or more interface elements of the intention communication interface to include predicted words or phrases that logically follow the existing text. In a continuation of this embodiment, the interface element may include one of the predicted words or phrases, such as "keyboard," "screen," or "device," where these words and phrases are predicted by processing the previous text data using a machine learning model (e.g., a long-short-term memory neural network). In some cases, other interface elements of the intention communication interface include a default set of alphanumeric characters to allow the user to enter text different from the predicted words or phrases. Machine learning-based word prediction can further increase the efficiency of performing complex tasks on a graphical user interface.

[0037] In some cases, the interface elements of an intent communication interface identify operations associated with a specific type of computing device, including augmented or virtual reality devices. For example, augmented reality (AR) glasses can display a set of virtual screens. The intent communication interface may traverse across different points in time using biosignals to select the first virtual screen from a set of virtual screens. Once the first virtual screen is selected, the interface elements of the intent communication interface may automatically update to identify a set of operations (e.g., delete, create a new virtual screen, move to a different location, increase or decrease screen size, correct screen orientation), and then the intent communication interface may traverse again to identify a specific operation (e.g., increase screen size) from the set of operations. The intent communication interface may be automatically updated again so that the interface elements identify a subset of operations related to increasing screen size (e.g., 1x, 2x, 3x). As a result, multiple traversals of the intent communication interface can be performed to efficiently execute tasks specifically associated with AR glasses. The technology using activation sequences of biosignals can be extended to other types of devices, such as computing devices with robotic components (e.g., drone devices).

[0038] Therefore, the specific embodiments described herein improve existing BCIs by implementing techniques that can efficiently convert a subject's biosignals to perform complex tasks. For example, biosignal activation sequences can be used to determine various operations to be performed by a computing device. Rather than relying on directly converting biosignals to specific operations, using activation sequences and corresponding intent communication interfaces can reduce potential errors and lead to the efficient execution of computer processing. Furthermore, the intent communication interface can be configured to perform different operations across various computing platforms (e.g., robotics, augmented reality devices). Finally, the use of biosignal activation sequences can be further enhanced using machine learning techniques to increase the efficiency and effectiveness of performing computing device processing. Thus, the embodiments described herein reflect improvements in the functionality of neural interface systems and graphical user interface technologies.

[0039] I. Systems for analyzing biological signals A. Multi-electrode devices Figure 1 shows a user 105 using a multi-electrode device 110. The device is shown as being attached to the user's forehead 115 (for example, via a positioning adhesive between the device and the user). The device may include multiple electrodes for detecting and recording nerve signals. Following signal recording, the device can transmit (for example, wirelessly) the data (or a processed version thereof) to another electronic device 120, such as a smartphone. The other electronic device 120 can then further process the data and / or respond to the data, as will be further described herein. Thus, Figure 1 illustrates that the multi-electrode device 105 can be miniaturized and easily positioned. Although only one device is shown in this embodiment, it will be understood that multiple devices may be used in some embodiments.

[0040] Furthermore, while Figure 1 shows the device 110 being attached to the user 105 by adhesive, other mounting means may also be used. For example, a head harness or band can be positioned around the user and the device. Also, while housing all electrodes for the channel in a single small unit is often advantageous in terms of ease of use, it will be understood that in another embodiment, the electrodes may be located outside the main device housing or positioned apart from each other. In one example, a device such as that described in PCT application PCT / US2010 / 054346 is used. PCT / US2010 / 054346 is incorporated herein by reference in its entirety for all purposes.

[0041] Devices 115a and 115b can communicate directly (e.g., via Bluetooth or BTLE connection) or indirectly. For example, each device can communicate (e.g., via Bluetooth or BTLE connection) with a server 120 which may be located near the tennis court 110.

[0042] The biosignal data collected by the multi-electrode device 110 may include different types of biosignals. For example, the biosignal data may include EEG data collected from electrodes placed on the subject's forehead. In another embodiment, the biosignal data may include EMG data collected from electrodes of the multi-electrode device 110 placed on the subject's limbs. In some cases, the biosignal data may include: (i) instructions of intent to move the corresponding part of the body; and (ii) the time when the biosignal was generated.

[0043] It is possible to analyze the biosignals collected by the multi-electrode device 110 to detect signal activation sequences. For example, detecting signal activation sequences may involve processing the biosignal data to identify a first signal and a second signal. The first signal represents an intention to move a first part of the subject's body. In some cases, the first signal is generated before the second signal. The second signal may represent another intention to move a second part of the subject's body. Therefore, detecting signal activation sequences may involve determining that the first signal, representing an intention to move a first part of the subject's body, was generated before the second signal, representing another intention to move a second part of the subject's body.

[0044] The multi-electrode device 1110 can transmit a signal activation sequence of biosignals to the electronic device 120 via a short-range connection. The electronic device 120 can process the signal activation sequence to identify a specific operation. This operation may include entering one or more alphanumeric characters on the graphical user interface of a computing device. In another embodiment, this operation may include moving a cursor displayed by the graphical user interface. This operation may also include operations performed by different types of computing devices, including the control of one or more robotic components or the control of an augmented reality or virtual reality device. The electronic device 120 can then perform the identified operation. Thus, various operations can be performed by the electronic device 120 based on the biosignal activation sequence. Furthermore, various embodiments for processing biosignal data collected from the multi-electrode device 110 are described in Sections III to VI of this disclosure.

[0045] B. Exemplary computing environments Figure 2 shows an example of networked devices to facilitate the coordinated evaluation and use of bioelectrical recordings. One or more multi-electrode devices 205 can collect channel data derived from bioelectrical data recorded by a user. The biosignal data can then be presented and processed by one or more other electronic devices, such as a mobile device 210a (e.g., a smartphone), a tablet 210b, a laptop, or a desktop computer 201c. One or more devices 201, 205, and / or 210 can analyze the biosignal data to determine a signal activation sequence. For example, detecting a signal activation sequence may involve processing the biosignal data to identify a first signal and a second signal. The first signal represents an intention to move a first part of the subject's body. In some cases, the first signal is generated before the second signal. The second signal may represent another intention to move a second part of the subject's body. One or more devices 201, 205, and / or 210 can identify specific actions that can be performed based on the signal activation sequence. For example, a particular operation may involve entering one or more alphanumeric characters on the graphical user interface of a computing device.

[0046] Inter-device communication is possible via short-range connections 215 (e.g., Bluetooth, BTLE, or ultra-wideband connections) or via Wi-Fi networks 220 such as the Internet.

[0047] One or more devices 205 and / or 210 can further access a data management system 225, which can receive and evaluate data from (for example) a collection of multi-electrode devices. For example, a healthcare provider or pharmaceutical company (e.g., conducting a clinical trial) can use the data from the multi-electrode devices to measure a patient's health. Thus, for example, the data management system 225 can store data associated with a specific user and / or generate demographics.

[0048] Figure 3 shows a multi-electrode device 300 communicating with another electronic device 302 (for example, wirelessly or via cable). This communication can enhance the functionality of the multi-electrode device by leveraging the resources of the other electronic device (e.g., faster processing speed, larger memory, larger display screen, more advanced input receiving capabilities). In one example, electronic device 302 includes an interface function that allows a user (e.g., the same person whose signal is recorded, or a different person) to view information (e.g., a summary of the recorded data and / or operation options) and / or control operations (e.g., control the functions of the multi-electrode device 300, or control another operation such as voice construction). Communication between device 300 and device 302 may occur intermittently when device 300 collects and / or processes data, or after a data collection period. Data may be pushed from device 300 to and / or pulled from other device 302. For example, the multi-electrode device 300 may push biosignal data to an electronic device 302 via a wireless communication network (e.g., a short-range communication network). The electronic device 302 can process the biosignal data to determine a signal activation sequence (e.g., whether the signal indicates an intention to clench the left hand), which can then be used to identify a specific operation performed by the electronic device 302 and / or other computing devices. Furthermore, various embodiments for processing biosignal data collected from the multi-electrode device 205 or 300 are described in Sections III-VI of this disclosure.

[0049] C. System Architecture Figure 4 is a simplified block diagram of a multi-electrode device 400 (for example, one that implements the multi-electrode device 300) according to one embodiment. The multi-electrode device 400 may include a processing subsystem 402, a storage subsystem 404, an RF interface 408, a connector interface 410, a power subsystem 412, an environmental sensor 414, and electrodes 416. The multi-electrode device 400 does not have to include each of the illustrated components and / or may include other components (not explicitly shown).

[0050] The storage subsystem 404 can be implemented using, for example, a magnetic storage medium, flash memory, other semiconductor memory (e.g., DRAM, SRAM), or any other non-temporary storage medium, or a combination of media, and may include volatile and / or non-volatile media. In some embodiments, the storage subsystem 404 can store biometric data (e.g., biosignal data), user information (e.g., identification information and / or medical history information), and / or analytical variables (e.g., previously defined strong frequencies, or frequencies for distinguishing between signal groups). In some embodiments, the storage subsystem 404 can also store one or more application programs (or apps) 434 executed by the processing subsystem 410 (e.g., to initiate and / or control data acquisition, data analysis, and / or transmission).

[0051] The processing subsystem 402 can be implemented as one or more integrated circuits, for example, one or more single-core or multi-core microprocessors or microcontrollers, examples of which are known in the art. During processing, the processing system 402 can control the operation of the multi-electrode device 400. In various embodiments, the processing subsystem 404 can execute various programs depending on the program code and can maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed can be made to reside in a storage medium such as the processing subsystem 404 and / or the storage subsystem 404.

[0052] Through appropriate programming, the processing subsystem 402 can provide various functionalities to the multi-electrode device 400. For example, in some embodiments, the processing subsystem 402 can execute code that controls the collection, analysis, application, and / or transmission of biometric data. In some embodiments, some or all of this code can interact with an interface device (e.g., the other device 302 in Figure 3) by, for example, generating messages to be sent to an interface device and / or receiving and interpreting messages from an interface device. For example, processing of biosignal data may include a processing subsystem 402 that provides biosignal data to an interface device, where the interface device (e.g., the other device 302 in Figure 3) can transform the biosignal data to identify operations associated with a computing device. In some embodiments, some or all of the code can operate locally with respect to the multi-electrode device 400. For example, a storage subsystem 404 can store signal processing applications for transforming biosignal data. The processing subsystem 402 of the multi-electrode device 400 can execute signal processing applications to identify various operations associated with a computing device, details of which are further described in Sections III-VI of this disclosure.

[0053] The processing subsystem 402 is also capable of executing a data acquisition code 436 that can record and store data detected by the electrodes 416. In some cases, the signals can be differentially amplified and filtered. The signals, along with recording details (e.g., recording time and / or user identifier), can be stored in a biometric data store 437. The data can be further analyzed to detect physiological correspondences. In one embodiment, processing the spectrogram of the recorded signals can reveal frequency characteristics corresponding to specific sleep stages. In another embodiment, an arousal detection code 438 can analyze the gradient of the spectrogram to identify and evaluate sleep disorder indicators and detect arousal. In yet another embodiment, a signal actuator code 439 can convert specific biometric signal features into the movement of an external object (e.g., a cursor). For example, the signal actuator code 439 can be used to identify biometric signal data corresponding to an intention to move a specific part of a subject's body (e.g., the left hand), which can then be converted into a specific operation performed by a computing device. Such techniques and codes are further described herein.

[0054] The radio frequency (RF) interface 408 enables the multi-electrode device 400 to communicate wirelessly with various interface devices. The RF interface 408 may include RF transceiver components, such as antennas and support circuits, enabling data communication over a wireless medium using protocols such as Wi-Fi (IEEE 802.11 family of standards), Bluetooth® (a family of standards published by Bluetooth SIG, Inc.), or other protocols for wireless data communication. In some embodiments, the RF interface 408 can implement a short-range sensor (e.g., Bluetooth, BLTE, ultra-wideband wireless) proximity sensor 409 that supports proximity detection through another protocol for estimating signal strength and / or determining proximity to other electronic devices. In some embodiments, the RF interface 408 can provide near-field communication ("NFC") capabilities, such as implementing the ISO / IEC 18092 standard. NFC can support wireless data exchange between devices over very short distances (e.g., 20 centimeters or less). The RF interface 408 can be implemented using a combination of hardware (e.g., driver circuits, antennas, modulators / demodulators, encoders / decoders, and other analog and / or digital signal processing circuits) and software components. Multiple different radio communication protocols and associated hardware can be incorporated into the RF interface 408.

[0055] The connector interface 410 allows the multi-electrode device 400 to communicate with various interface devices via a wired communication path, for example, using a Universal Serial Bus (USB), a Universal Asynchronous Receiver / Transmitter (UART), or other protocols for wired data communication. In some embodiments, the connector interface 410 may provide a power port, thereby allowing the multi-electrode device 400 to receive power, for example, to charge an internal battery. For example, the connector interface 410 may include support circuitry in addition to a connector, such as a mini-USB connector or a custom connector. In some embodiments, the connector may be a custom connector that provides dedicated power and ground contacts, as well as digital data contacts that can be used to implement different communication technologies in parallel. For example, two pins may be assigned as USB data pins (D+ and D-), and two other pins may be assigned as serial transmit / receive pins (e.g., implementing a UART interface). Assigning pins to specific communication technologies can be done by hardwired or negotiated while the connection is established. In some embodiments, the connector may also provide a connection for transmitting and / or receiving bioelectrical signals, which can be transmitted in analog and / or digital format to or from other devices (e.g., device 302 or other multi-electrode devices).

[0056] The environmental sensor 414 may include a variety of electronic, mechanical, electromechanical, optical, or other devices that provide information about the external conditions surrounding the multi-electrode device 400. In some embodiments, the sensor 414 can provide a digital signal to the processing subsystem 402 as desired, for example, in streaming format or in response to polling by the processing subsystem 402. Any type and combination of environmental sensors are available. An accelerometer 442 is illustrated as an example. The acceleration detected by the accelerometer 442 can be used to estimate whether the user is sleeping or about to sleep, and / or to estimate their activity level.

[0057] The electrode 416 may include, for example, a circular surface electrode and may be made of gold, tin, silver, and / or silver / silver chloride. The electrode 416 may have a diameter greater than 1 / 8 inch and less than 1 inch. The electrode 416 may include an active electrode 450, a reference electrode 452, and (optionally) a ground electrode 454. The electrodes may be distinguishable from each other or not. The position of the electrodes may be fixed within the device and / or movable (e.g., tethered to the device). In some embodiments, some electrodes 416 are configured to collect EEG data. Additionally or alternatively, other electrodes may be configured to collect EMG data.

[0058] The power subsystem 412 can provide power and power management functions to the multi-electrode device 400. For example, the power subsystem 414 may include a battery 440 (e.g., a rechargeable battery) and associated circuitry that distributes power from the battery 440 to other components of the multi-electrode device 400 that require power. In some embodiments, the power subsystem 412 may also include circuitry that can operate to charge the battery 440 when, for example, the connector interface 410 is connected to a power source. In some embodiments, the power subsystem 412 may include a “wireless” charger, such as an inductive charger, to charge the battery 440 independently of the connector interface 410. In some embodiments, the power subsystem 412 may further include, in addition to or instead of, the battery 440, other power sources such as a solar cell.

[0059] It will be understood that the multi-electrode device 400 is illustrative and can be modified and altered. For example, the multi-electrode device 400 may include a user interface that allows the user to interact directly with the device. In another embodiment, the multi-electrode device may have a mounting indicator that shows (e.g., by light color or sound) whether the contact between the device and the user's skin is appropriate and / or whether the recorded signal is of acceptable quality.

[0060] Furthermore, while the multi-electrode device is described with reference to specific blocks, it should be understood that these blocks are defined for explanatory purposes only and do not imply a specific physical arrangement of components. Moreover, the blocks do not need to correspond to physically separate components. Blocks can be configured to perform various operations, for example, by programming a processor or by providing appropriate control circuits, and the various blocks may or may not be reconfigurable depending on how the initial configuration is obtained. Embodiments can be realized in various devices, including electronic devices implemented using any combination of circuitry and software. Also, not all blocks in Figure 4 need to be implemented in a given embodiment of the multi-electrode device.

[0061] Interface devices such as device 302 in Figure 3 may be implemented as electronic devices using blocks similar to those described above (e.g., processor, storage medium, RF interface, etc.) and / or other blocks or components. Figure 5 is a simplified block diagram of an interface device 500 according to one embodiment (e.g., implementing device 302 in Figure 3). The interface device 500 may include a processing subsystem 502, a storage subsystem 504, a user interface 506, an RF interface 508, a connector interface 510, and a power subsystem 512. The interface device 500 may also include other components (not explicitly shown). Many of the components of the interface device 500 may be similar to, or identical to, the components of the multi-electrode device 300 in Figure 3.

[0062] For example, the storage subsystem 504 may be largely similar to the storage subsystem 404 and may include, for example, the use of magnetic storage media, flash memory, other semiconductor memory (e.g., DRAM, SRAM), or any other non-temporary storage media, or combinations of media, and may include volatile and / or non-volatile media. Similar to the storage subsystem 504, the storage subsystem 504 can be used to store data and / or program code executed by the processing subsystem 502. For example, the storage subsystem 504 may store signal processing applications for transforming biosignal data to enable the identification of various operations associated with the computing device.

[0063] The user interface 506 may include any combination of input and output devices. The user can operate the input devices of the user interface 506 to invoke the functions of the interface device 500, and can view, listen to, and / or experience the output from the interface device 500 via the output devices of the user interface 506. Examples of output devices include a display 520 and a speaker 522. Examples of input devices include a microphone 526 and a touch sensor 528.

[0064] The display 520 can be implemented using small-form 5-factor speaker technology, such as a liquid crystal display (LCD), light-emitting diode (LED), or organic light-emitting diode (OLED). In some embodiments, the display 520 may incorporate a flexible display element or a curved glass display element, thereby allowing the interface device 500 to conform to a desired shape. One or more speakers 522 may be provided using small-form 5-factor speaker technology, including any technology capable of converting electronic signals into audible sound waves. The speakers 522 can be used to generate tones (e.g., beeps or ringtones) and / or voice. In some cases, the display 520 displays an intent communication interface. Biosignal data may be converted to access interface operation data from the intent communication interface, where the interface operation data is used by a signal processing application to identify specific operations performed by a computing device. Various embodiments for implementing the intent communication interface are described in Sections III-VI of this disclosure.

[0065] Examples of input devices include a microphone 526 and a touch sensor 528. The microphone 526 may include any device that converts sound waves into electronic signals. In some embodiments, the microphone 526 may be sensitive enough to provide a representation of a specific word uttered by the user. In other embodiments, the microphone 426 may be used to provide a general ambient sound level indication without necessarily providing a high-quality electronic representation of a specific sound.

[0066] The touch sensor 528 may include, for example, a capacitive sensor array having the ability to limit the location of contact to a specific point or area on the surface of the sensor, and possibly the ability to distinguish between multiple simultaneous contacts. In some embodiments, the touch sensor 428 can be superimposed on the display 520 to provide a touchscreen interface, and the processing subsystem 504 can translate touch events into specific user inputs depending on what is currently displayed on the display 520.

[0067] The processing subsystem 502 can be implemented as one or more integrated circuits, for example, one or more single-core or multi-core microprocessors or microcontrollers, examples of which are known in the art. During processing, the processing system 502 can control the operation of the interface device 500. In various embodiments, the processing subsystem 502 can execute various programs depending on the program code and can maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed can reside in a storage medium such as the processing subsystem 502 and / or the storage subsystem 504. For example, the processing subsystem 502 can access biosignal data provided by a multi-electrode device (e.g., multi-electrode device 400), execute a signal processing application to transform the biosignal data, and enable the identification of various operations associated with a computing device. Transforming the biosignal data may include determining a signal activation sequence, such as processing the biosignal data to identify a first signal and a second signal. The first signal represents an intention to move a first part of the subject's body. In some cases, the first signal is generated before the second signal. The second signal may represent another intention to move a second part of the subject's body. A signal processing application can identify specific actions that can be performed based on the signal activation sequence. For example, a specific action may involve entering one or more alphanumeric characters on the graphical user interface of a computing device.

[0068] Through appropriate programming, the processing subsystem 502 can provide various functionalities to the interface device 500. For example, in some embodiments, the processing subsystem 502 can execute an operating system (OS) 532 and various applications 534. In some embodiments, some or all of this application program can interact with the multi-electrode device, for example, by generating messages to be sent to the multi-electrode device and / or by receiving and interpreting messages from the multi-electrode device. In some embodiments, some or all of the application program can be processed locally at the interface device 500.

[0069] The processing subsystem 502 is capable of executing a data acquisition code 536 (which may be part of the OS 532, part of an application, or separate as needed). The data acquisition code 536 may be at least partially complementary to the data acquisition code 436 in Figure 4. In some cases, the data acquisition code 536 is configured, upon execution of the code, to cause device 500 to receive raw or processed bioelectrical signal data (e.g., EEG or EMG signals) from a multi-electrode device (e.g., multi-electrode device 300 in Figure 3), where the bioelectrical signal may indicate an intention to move a specific part of the subject's body. The data acquisition code 536 can further define the processing to be performed on the received data (e.g., applying filters, generating metadata indicating the source multi-electrode device or reception time, and / or compressing the data). Furthermore, the data acquisition code 536 can, at runtime, cause the raw or processed bioelectrical signals to be stored in a biodata store 537.

[0070] In some cases, executing data acquisition code 536 may cause device 500 to collect data that may include other biometric data (e.g., patient's body temperature or pulse) or external data (e.g., light intensity or geographic location). This information may be stored together with biosignal data (e.g., metadata for EEG or EMG recordings may include patient's body temperature and / or location) and / or separately (e.g., with a timestamp to enable future time-synchronized data matching). In these examples, it will be understood that interface device 500 may include appropriate sensors (e.g., cameras, thermometers, GPS receivers) that collect this additional data, or may communicate with other devices having such sensors (e.g., via RF interface 508).

[0071] The processing subsystem 502 is also capable of executing one or more codes that can analyze raw or processed bioelectrical signals (i.e., biosignal data) in real time or retrospectively to detect events of interest. For example, the execution of the arousal detection code 538 can evaluate changes in a spectrogram (constructed using EEG data) corresponding to the patient's sleep period to determine whether and / or when an arousal occurred. In one example, this evaluation may involve determining a change variable corresponding to the amount by which the power (e.g., normalized power) at one or more frequencies of the time increment has changed relative to one or more other time increments. In one example, this evaluation may involve assigning each time increment to a sleep stage and detecting the time intervals in which the assignments changed. The sleep stage classification can (potentially) further describe the arousal occurring (e.g., by indicating which stage the arousal occurs in and / or by identifying how many sleep stages the arousal traversed).

[0072] In another embodiment, the execution of signal actuator code 539 evaluates and transforms EEG and / or EMG data representing an intention to move a part of a subject's body (e.g., the left hand) to enable the identification of various operations associated with a computing device. First, a mapping can be constructed to associate a specific EEG and / or EMG signature with a specific action. That action may be an external action, such as the movement of a cursor on a screen. For example, that action may include controlling robotic components of another device or inputting data on a graphical user interface. The mapping can be performed using clustering and / or component analysis, and raw or processed signals recorded from one or more active electrodes (e.g., from one or more multi-electrode devices, each positioned on a different muscle) can be utilized.

[0073] In one example, the execution of signal actuator code 539 causes an interactive visualization to be presented on display 520. The cursor position on the screen is controllable based on real-time analysis of EEG and / or EMG data using mapping. Thus, the person being recorded can interact with the interface hands-free. In an exemplary example, the visualization may include voice-assisted visualization that allows the person to select letters, sequences of letters, words, or phrases. Through sequential selection, the person can construct a sentence, paragraph, or conversation. The text can be used electronically (e.g., to generate an email or letter) or verbalized to communicate with another person nearby (e.g., using the voice component of signal actuator 539 to send an audio output to speaker 522).

[0074] The RF (radio frequency) interface 508 and / or connector interface 510 enable the interface device 500 to communicate wirelessly with various other devices (e.g., the multi-electrode device 400 in Figure 4) and networks. The RF interface 508 may correspond to the RF interface 408 in Figure 4 (e.g., including its described features), and / or the connector interface 510 may correspond to the connector interface 410 (e.g., including its described features). The power subsystem 512 can provide power and power management functions to the interface device 512. The power subsystem 512 may correspond to the power subsystem 41 (e.g., including its described features).

[0075] It will be understood that the interface device 500 is illustrative and subject to modification and alteration. In various embodiments, other controls or components may be provided in addition to or instead of those described above. Any device capable of interacting with other devices (e.g., multi-electrode devices) to store, process, and / or use recorded bioelectrical signals may be the interface device.

[0076] Furthermore, while interface devices are described with reference to specific blocks, it should be understood that these blocks are defined for illustrative purposes only and do not imply a specific physical arrangement of components. Moreover, blocks do not need to correspond to physically separate components. Blocks can be configured to perform various operations, for example, by programming a processor or by providing appropriate control circuits, and the various blocks may or may not be reconfigurable depending on how the initial configuration is obtained. Embodiments can be realized in various devices, including electronic devices implemented using any combination of circuitry and software. Also, not all blocks in Figure 5 need to be implemented in a given embodiment of a mobile device.

[0077] Communication between one or more multi-electrode devices, one or more mobile devices, and an interface device may be implemented according to any communication protocol (or combination of protocols) that both devices are programmed to or configured to use. In some cases, standard protocols such as the Bluetooth protocol or the ultra-wideband protocol may be used. In some cases, custom message formats and syntax (including, for example, a set of rules for interpreting specific bytes or sequences of bytes in digital data transmission) may be defined, and messages may be transmitted using standard serial protocols, such as virtual serial ports defined in a particular Bluetooth standard. Embodiments are not limited to specific protocols, and those skilled in the art who obtain this teaching will recognize that a number of protocols are available.

[0078] According to certain embodiments, one or more multi-electrode devices can be conveniently used to collect electrical biometric data from a patient. The data can be processed to identify physiologically important signals. The detection itself can be useful because it can inform the user or a third party about the patient's health and / or the effectiveness of the current treatment. In some cases, the signals can be used to automatically control another object, such as a computer cursor. Such capabilities can extend the user's physical capabilities (e.g., those that may be impaired due to illness) and / or improve ease of operation.

[0079] II. Methods for identifying intentions to move body parts by analyzing biosignals To facilitate the conversion of biosignals (e.g., electroencephalography (EEG) data, electromyography (EMG) data) for identifying various computational processes, machine learning or statistical analysis techniques can be used to identify biosignal data that represents an intention to move a specific part of a subject's body (e.g., left hand, right hand). For example, one or more signal processing analyses (e.g., Independent-Component Analysis (ICA)) can be used to identify reference signatures of biosignal data that can be used to determine whether biosignals obtained from different subjects correspond to an intention to move a specific part of the body.

[0080] As an exemplary embodiment, a reference dataset can be collected that includes a set of biosignal data (e.g., EEG data) representing imagined movements of the left and right hands. For example, each biosignal data in the reference dataset may include 32-channel EEG signals recorded by a multi-electrode device (e.g., multi-electrode device 110 in Figure 1), where biosignal data can be recorded for a corresponding subject performing an intended movement of the left or right hand (e.g., moving the cursor to the left interface element of the intention communication tree and grasping with the left hand). In some cases, the biosignal data in this set may also include a non-motor state that is the baseline for the corresponding subject. Next, each biosignal data in the reference dataset can be decomposed into one or more independent components (ICs) representing the biosignal data.

[0081] In some cases, the biosignal data in this set can be initially projected into a 15-dimensional subspace using Principal Component Analysis (PCA), where the PCA components can be further processed to generate ICA. PCA can be used to reduce the dimensionality of biosignal data in a reference dataset. Since PCA can significantly reduce computation time and the need for large amounts of computer memory, implementing ICA of biosignals with PCA can be advantageous.

[0082] Next, one or more biosignal signatures representing an intention to move a specific part of a subject's body may be identified from ICs in a reference dataset. In some cases, one or more biosignal signatures are selected from ICs that best represent the intention to move a specific part of the body. For example, a biosignal signature is identified at least in part on a spatial pattern of ICs that correlates with activation of the sensorimotor cortex of the corresponding cerebral hemisphere. A biosignal signature can be used as a reference signature to classify whether biosignals collected from other subjects represent an intention to move the left or right part of the body. In addition to using ICAs, other types of signal analysis can be used to identify biosignals that represent an intended movement of a part of the body, as those skilled in the art would assume.

[0083] III. How to identify the intention behind moving a body part by analyzing spectrogram data A. Collection of biosignal data Figure 6 is a flowchart of a process 600 for acquiring a channel of electrode data from a living organism using a multi-electrode device, according to one embodiment. All parts of process 600 can be implemented in a multi-electrode device (e.g., multi-electrode device 400). In some cases, parts of process 600 (e.g., one or more of blocks 610-635) can be implemented in an electronic device separate from the multi-electrode device, and these blocks can be executed immediately after receiving a signal from the multi-electrode device (e.g., immediately after acquisition), before storing data related to the recording, on demand depending on the acquired data, and / or before using the acquired data.

[0084] In block 605, the active and reference signals can be acquired using each electrode. In some cases, a ground signal is further acquired from the ground electrode. The active and reference electrodes and / or the active and ground electrodes can be mounted on a single device (e.g., a multi-electrode device) at a certain distance from each other and / or in close proximity to each other (e.g., so that the centers of the electrodes are located less than 12, 6, or 4 inches from each other and / or so that the electrodes are positioned to increase the likelihood of recording signals from the same muscle or the same brain region).

[0085] In some cases, a reference electrode is positioned close to the active electrode to increase the likelihood that both electrodes will sense electrical activity from the same brain region or muscle. For example, a first active electrode positioned near a first reference electrode can be used to collect a first biosignal (e.g., EEG) generated from the left hemisphere of the subject's brain, where the first biosignal represents an intention to move the subject's right limb. Similarly, a second active electrode positioned near a second reference electrode can be used to collect a second biosignal generated from the right hemisphere of the subject's brain, where the second biosignal represents an intention to move the subject's left limb. The sequence in which the first and second biosignals are detected can be used to identify a specific operation associated with a computing device. In other examples, the reference electrode is positioned far away from the active electrode (e.g., in an electrically relatively neutral region, which may include areas not on the brain or prominent muscles) to reduce overlap of the signals being studied.

[0086] Prior to acquisition, electrodes can be attached to a person's skin. This may include, for example, attaching a single device that completely houses one or more electrodes, and / or attaching one or more individual electrodes (e.g., flexibly extending beyond the device housing). In one example, such attachment is performed using adhesive (e.g., applying an adhesive substance to at least a portion of the underside of the device, applying adhesive patches on and around the device, and / or applying double-sided adhesive patches to the underside of at least a portion of the device) to attach a multi-electrode device including an active electrode and a reference electrode to a person. For EEG recording, the device can be attached, for example, near the frontal lobe of a person (e.g., on the forehead). For EMG recording, the device can be attached over muscles (e.g., over the jaw muscles or neck muscles).

[0087] In some cases, only one activation signal is recorded at a time. In other examples, each of the set of activation electrodes records an activation signal. In this scenario, the activation electrodes may be positioned at different locations on the body (e.g., different sides of the body, different muscle types, or different brain regions). For example, for EMG recording, the device's activation electrodes are attached across the left and right limbs of the subject's body to determine the signal activation sequence and enable the identification of various operations associated with the computing device. Each activation electrode may be associated with a reference electrode, or fewer references may be collected relative to the number of activation signals collected. Each activation electrode may reside within a separate multi-electrode device.

[0088] In block 610, a reference signal can be subtracted from the active electrode. This reduces noise in the active signal, such as recording noise or noise caused by the patient's breathing or movement. While proximity between the reference and active electrodes has traditionally been avoided, such a position can improve the portion of the active electrode noise (e.g., patient movement noise) that is shared by the reference electrode noise. For example, when a patient turns over in bed, the movement experienced by an active electrode positioned over the central F7 region of the brain will be quite different from the movement experienced by a reference electrode positioned on the opposite ear. On the other hand, if both electrodes are positioned over the same F7 region, they are likely to experience similar movement artifacts. The signal difference may cause a loss of representation of the electrical activity of some cells from the underlying physiological structure, but the majority of the remaining signal can be attributed to such activity (by noise reduction).

[0089] In block 615, the signal difference can be amplified. The amplification gain is, for example, between 100 and 100,000. In block 620, the amplified signal difference can be filtered. The filter applied may include, for example, an analog high-pass filter or a band-pass filter. Filtering can reduce the signal contribution from fluid potentials such as breathing. The filter may include a low cutoff frequency of about 0.1 to 1 Hz. In some cases, the filter may also include a high cutoff frequency, which may be set to a frequency below a determined Nyquist frequency given based on the sampling rate.

[0090] Block 625 allows the filtered analog signal to be converted to a digital signal. Block 630 allows a digital filter to be applied to the digital signal. The digital filter can reduce the DC signal component. Digital filtering can be performed using linear or nonlinear filters. The filters may include, for example, finite or infinite impulse response filters or window functions (e.g., Hanning function, Hamming function, Blackman function, or rectangular function). The filter features may be defined to reduce the DC signal contribution while preserving the high-frequency signal component.

[0091] The filtered signals may be analyzed in block 635. As will be described in more detail herein, the analysis may include micro-analyses such as classification of individual segments of the signals (e.g., sleep stage, wakefulness or non-wakefulness, and / or intention to move). The analysis may also include, alternatively or additionally, macro-analyses such as characterization of overall sleep quality or muscle activity.

[0092] As described above, in some cases, multiple devices collaborate to execute process 600. For example, the multi-electrode device 400 in Figure 4 can execute blocks 605-625, and a remote device (e.g., a server, computer, smartphone, or interface device 405) can execute blocks 630-635. To facilitate such shared process processing, it will be understood that the devices can communicate with each other and share appropriate information. For example, after block 625, the multi-electrode device 400 can transmit a digital signal (e.g., using a short-range network or WiFi network) to another electronic device, such as the interface device 500 in Figure 5. The other electronic device receives the signal and can then execute blocks 630-635.

[0093] Process 600, although not explicitly illustrated, may store raw and / or processed data. The data can be stored in a multi-electrode device, a remote device, and / or the cloud. In some cases, both the raw data and its processed version may be stored (for example, identifying classifications associated with portions of the data).

[0094] Furthermore, it will be understood that process 600 can be an ongoing process. For example, the activation signal and reference signal may be collected continuously or periodically over a long period of time until all operations are performed and the target result (e.g., inputting text into a graphical user interface) is reached. Some or all of process 600 may be executable in real time as signals are collected, and / or the data may be processed entirely or partially in batches. For example, during a recording session, blocks 605-635 may be executable in real time at each point in a series of time points to facilitate the input of each character of text into a graphical user interface.

[0095] B. Identifying frequency signatures from biosignal data. Figure 7 is a flowchart of a process 700, according to one embodiment, for analyzing channel biodata to identify frequency signatures at various biological stages. All parts of process 700 can be implemented in a multi-electrode device (e.g., the multi-electrode device 400 in Figure 4) and / or an electronic device located away from the multi-electrode device (e.g., the interface device 500 in Figure 5).

[0096] In block 705, signals can be converted into spectrograms. Signals may include those based on recordings from human-positioned electrodes, such as differentially amplified and filtered signals. Spectrograms can be generated by analyzing the signals into time bins and calculating the spectrum for each time bin (e.g., using the Fourier transform). Therefore, spectrograms may include multidimensional power matrices with dimensions corresponding to time and frequency.

[0097] Selected portions of the spectrogram can be optionally removed in block 710. These portions may include those associated with a particular time bin that may be determined to have insufficient signal quality and / or lack or be unsuitable reference data. For example, reference data (e.g., corresponding to human evaluations of the data) can be used to determine signatures for various physiological events in order to develop a conversion or mapping from signals to physiological events (e.g., an intention to move a specific part of the body). Therefore, data portions for which reference data is unavailable can be ignored while determining signatures.

[0098] In block 715, the spectrogram may be divided into time blocks or sets of epochs. Each time block may have the same duration (e.g., 30 seconds) and may (occasionally) contain multiple (and a fixed number, etc.) time increments, where each time increment corresponds to a recording time. In some cases, a time block is defined as a single time increment in the spectrogram. In some cases, a time block is defined as multiple time increments. The duration of a time block may be determined, for example, based on the timescale of the physiological event in question (e.g., a 2-second time block to identify a signal representing an intention to move a part of the body), the temporal precision or duration of the corresponding reference data, and / or the desired precision, accuracy, and / or speed of signal classification.

[0099] Each time bin within each time block can be assigned to a group in block 720 based on reference data. For example, human scoring of EEG data can identify an intention to move a corresponding part of the body (e.g., an intention to clench the left hand) for each time block. A time bin within a given time block can be associated with a corresponding part of the body. A time bin within a time block can be assigned to the "left part" group (if the intention to move the left part of the body occurred during the block) or to the "right part" group (if the intention to move the right part of the body occurred). Similarly, for a given EMG recording, a patient can indicate an intention to move a specific part of their body. For example, after moving the fingers of their right hand, the patient may indicate an intention to move the cursor associated with the intention communication interface from the root interface element to the right child interface element. The time bin associated with jaw contraction can then be assigned to the "right part" group.

[0100] In block 725, spectrogram features can be compared across groups. In one example, one or more spectral features can be initially determined for each time bin, and these sets of features can be compared in block 725. For example, strong frequencies or fragmentation values ​​can be determined, as will be described in more detail herein. In another embodiment, for each individual time bin, the power (or normalized power) at each of one or more frequencies can be compared. In yet another embodiment, a collective spectrum may be determined based on the spectrum associated with the time bin assigned to a given group, and then features can be determined based on the collective spectrum. For example, the collective spectrum may include the mean spectrum or the median spectrum, and the features may include strong frequencies, fragmentation values, or power (at one or more frequencies). In yet another embodiment, the collective spectrum may include, for each time bin, the n1% power (power below n1% of the power at that frequency) and the n2% power (power below n2% of the power at that frequency).

[0101] Using features, one or more group discrimination frequency signatures may be identified in block 730. A frequency signature may include the identification of a variable to be identified or determined based on a given spectrum used for group assignment. The variable can then be used as part of reference data to improve the detection of biosignals representing (for example) an intention to move a particular part of the body. For example, a group discrimination frequency signature may include a specific frequency, thereby the power of that frequency being used for group assignment. In another embodiment, the group discrimination frequency may include weights associated with each of one or more frequencies, thereby the weighted sum of the frequencies' powers being used for group assignment.

[0102] A frequency signature may include a subset of frequencies and / or weights of one or more frequencies. For example, it may be possible to determine overlap between the power distributions of two or more groups, and the group discrimination frequency may be identified as a frequency with overlap below a threshold, or as a frequency with relatively small (or minimal) overlap. In one example, the model may be used to determine which multiple frequency (or single frequency) features are reliably available to discriminate between groups. In one example, the group discrimination signature may be identified as a frequency associated with an information value (e.g., based on the entropy derivative) that exceeds an absolute or relative value (e.g., relative to the value of another frequency).

[0103] For example, block 730 may include assigning weights to each of two or more frequencies. Then, a variable which is a weighted sum of (normalized or denormalized) powers can be calculated to determine which group the spectrum should be assigned to. For example, block 725 may include using component analysis (e.g., principal component analysis or independent component analysis), and block 730 may include identifying one or more components.

[0104] Figure 8 is a flowchart of a process 800, according to one embodiment, which analyzes channel biodata to identify the frequency signature of an intended movement. All parts of process 800 can be implemented in a multi-electrode device (e.g., the multi-electrode device 400 in Figure 4) and / or an electronic device located away from the multi-electrode device (e.g., the interface device 500 in Figure 5).

[0105] Block 805 allows for the collection of spectrogram samples corresponding to various physiological states. In some cases, at least some states correspond to an intention to move a corresponding part of the body having specific attributes. For example, samples can be collected from both periods when the muscles of the left arm were activated and another period when the muscles of the right arm were activated, so that the sample contains data representing the intention to move the corresponding muscles of the body. In some cases, the collected samples are based on records from one person. On the other hand, they are based on records from multiple people.

[0106] In some cases, at least some states correspond to states of intent. For example, samples (e.g., based on EMG data) can be collected such that some data corresponds to an intent that triggers a specific action (e.g., clenching the right hand), while other data does not correspond to such an intent.

[0107] Spectrogram data may include the spectrogram of raw data, the spectrogram of filtered data, a once-normalized spectrogram (for example, normalizing the power at each frequency based on the power across time bins of the same frequency, or based on the power across frequencies in the same time bin), or a multiple-normalized spectrogram (for example, normalizing the power at each frequency at least once based on the power normalized or unnormalized power across time bins of the same frequency, and at least once based on the power normalized or unnormalized power across frequencies in the same time bin).

[0108] In block 810, spectrogram data from a base state (e.g., non-action phase) can be compared with spectrogram data from one or more non-base states (e.g., intention to move a specific part of the body, action state) to identify significant values. For example, for a comparison between a base state and a single non-base state, frequency-specific significant values ​​may include p-values, which can be determined frequency by frequency based on a statistical examination of the power distribution in the two states.

[0109] Next, blocks 815-820 are executed for each comparison between pairs of non-base states (e.g., operating states) and base states (e.g., non-operating states). In block 815, a threshold significance number can be set. The threshold may be determined based on the distribution of a set of frequency-specific significance values ​​and a defined percentage (n%). For example, the threshold significance number can be defined as the value at which n% (e.g., 60%) of the frequency-specific significance values ​​fall below the threshold significance number.

[0110] A set of frequencies having frequency-specific significance values ​​below a threshold can be identified in block 820. These frequencies may therefore include frequencies that sufficiently distinguish between the base state and the non-base state (based on the threshold significance number).

[0111] Next, blocks 815 and 820 are repeated for each additional comparison between the base state and another non-base state. The results then include a set of the highest n% significant frequencies associated with each non-base state.

[0112] In block 825, frequencies present in all sets (or sets of a threshold number) are identified. Therefore, the identified overlapping frequencies may include frequencies among the top n% of significant frequencies when determining each of the multiple non-base states from the base state.

[0113] In block 830, a determination can be made as to whether the overlap rate is greater than the overlap threshold. If not, process 800 can return to block 815, where a new (e.g., higher) threshold significance number may be set. For example, the threshold percentage (n%) used to define the threshold significance number may be incremented (e.g., by 1%) to include more frequencies in the set identified in block 820.

[0114] If it is determined that the overlap exceeds the overlap threshold, process 800 can proceed to block 835, where one or more group discrimination frequency signatures can be defined using the frequencies in the overlap between sets. The signature may include the identification of a subset of frequencies in the spectrogram and / or weights for each of the one or more frequencies. The weights may be based, for example, on the frequency-specific significance of the frequencies for each comparison of one or more base states to non-base states, or on a subset of frequencies including a given frequency (if the overlap evaluation does not require the identified frequencies to be present in the entire set of frequencies). In some cases, the signature may include one or more components defined by assigning weight frequencies in the overlap. For example, component analysis may be performed using state assignment and power in the frequencies within the overlap to identify one or more components.

[0115] Subsequent analysis (e.g., of different data) can focus on the group-defining frequency signature. In some cases, the spectrogram (e.g., normalized or denormalized spectrogram) can be cropped to exclude frequencies not defined as group-defining frequencies. For example, process 800 can be performed first to identify the group-defining frequencies, and process 700 (e.g., subsequently analyzing different data) can crop the spectrogram of the signals using the group-defining frequencies before comparison.

[0116] C. Normalization of Spectrogram Data Figure 9 is a flowchart of a process 900, according to one embodiment, for normalizing a spectrogram and classifying biodata using a group discrimination frequency signature. All parts of process 900 can be implemented in a multi-electrode device (e.g., the multi-electrode device 400 in Figure 4) and / or an electronic device located away from the multi-electrode device (e.g., the interface device 500 in Figure 5).

[0117] In blocks 905 and 910, the spectrogram constructed from the recorded bioelectrical signals (e.g., EEG or EMG data) is normalized (e.g., once, multiple times, or iteratively). In some embodiments, the spectrogram is constructed from channel data of one or more channels, each generated based on signals recorded using a device that fixes multiple electrodes to each other or tethers multiple electrodes to each other.

[0118] The first normalization performed in block 905 can be done by first determining the Z-score of the power associated with each frequency in the spectrogram (i.e., across all time bins) for that frequency. Then, the power at that frequency can be normalized using this Z-score value.

[0119] The (optional) second normalization performed in block 910 can be done by first determining a Z-score for each time bin in the spectrogram, based on the power associated with that time bin (i.e., across all time bins). Then, the power in that time bin can be normalized using this Z-score value.

[0120] These normalizations can be performed iteratively (alternating) for a set number of times, or until the normalization coefficient (or change in the normalization coefficient) falls below a threshold. In some cases, normalization is performed only once, which results in either block 905 or block 910 being omitted from process 900. In some cases, the spectrogram is not normalized.

[0121] For each time bin in the spectrogram, the corresponding spectrum can be collected in block 915. In block 920, one or more variables can be determined for a time bin based on the spectrum and one or more group-discriminatory frequency signatures. For example, the variables may include power at a selected frequency identified by the signature. In another embodiment, the variables may include the values ​​of components defined by the signature (determined, for example, by calculating a weighted sum of the power values ​​in the spectrum). Thus, in some cases, block 920 includes projecting the spectrum onto a new basis. Blocks 915 and 920 are executable for each time bin.

[0122] In block 925, group assignments are made based on associated variables. In some cases, individual time bins are assigned. In some cases, a set of time bins (e.g., individual epochs) is assigned to a group. Assignments can be made, for example, by comparing a variable to a threshold (e.g., if the variable is below a threshold, it is assigned to one group, otherwise it is assigned to another), or by using clustering or modeling techniques (e.g., a Gaussian, Naive, or Bayes classifier). In some cases, assignments are limited so that a given feature (e.g., a time bin or time epoch) cannot be assigned to more than a certain number of groups. This number may be the same as, or (depending on the embodiment) different from, the number of groups or states (both base and non-base states) used to determine one or more group discrimination frequency signatures. Assignments can be general or state-specific (e.g., so that clustering analysis generates assignments to one of five groups without any group being linked to any particular physiological significance).

[0123] Furthermore, fragmentation values ​​can be defined for each time interval. Fragmentation values ​​may include temporal fragmentation values ​​or spectral fragmentation values. For temporal fragmentation values, the temporal gradient of the spectrogram can be determined and divided into segments. The spectrogram may include the raw spectrogram and / or a spectrogram that has been normalized once, twice, or more times across time bins and / or across frequencies (e.g., a spectrogram that is first normalized across time bins and then normalized across frequencies). A given segment may include a set of time bins, each of which can be associated with a vector of partial bias power values ​​(extending a range of frequencies). For each frequency, a gradient frequency eigenvariate may be defined for any time bin within the time block and based on the partial bias power values ​​defined for the frequency. For example, the variable may be defined as the average of the absolute values ​​of the frequency partial bias power values. Fragmentation values ​​may be defined as the frequencies at which the value of the frequency eigenvariate is high or highest. Spectral fragmentation values ​​can be defined similarly, but can be based on the spectral gradient of the spectrogram.

[0124] IV. Conversion of biosignals for controlling computer processing In some embodiments, the subject's biosignals are used to identify various operations associated with a computing device. For example, a biosignal activation sequence (e.g., biosignals activated from the left hemisphere of the brain) can be used to determine the various operations performed by the computing device. Rather than relying on directly translating complex biosignals into specific operations, using activation sequences and corresponding intent communication interfaces (e.g., intent communication interface 1100) can reduce potential errors and lead to the efficient execution of computer processing.

[0125] A. Activation sequence Figure 10 is a schematic diagram 1000 showing an example of determining the activation sequence of a biosignal according to some embodiments. In some embodiments, a multi-electrode device 1002 accesses biosignal data from a subject. The multi-electrode device 1002 (e.g., multi-electrode device 110 in Figure 1) may include software and hardware components for detecting and converting biosignals generated to move different parts of the subject's body. For example, the multi-electrode device 1002 may include a housing having one or more electrode clusters. The biosignal data collected by the multi-electrode device 1002 may include different types of biosignals. For example, the biosignal data may include EEG data collected from electrodes placed on the subject's forehead. In another embodiment, the biosignal data may include EMG data collected from electrodes placed on the subject's limbs. In some cases, the biosignal data is accessed by other computing devices (e.g., electronic device 120 in Figure 1) via a wireless communication network (e.g., a short-range communication network). The biosignal data may include: (i) instructions of intent to move the corresponding part of the body; (ii) the time when the biosignal was generated.

[0126] Biosignals from a subject may be analyzed to detect signal activation sequences. For example, detecting signal activation sequences may involve processing biosignal data to identify a first signal and a second signal. The first signal represents an intention to move a first part of the subject's body. In some cases, the first signal is generated before the second signal. The second signal may represent another intention to move a second part of the subject's body. Therefore, detecting signal activation sequences may involve determining that the first signal, representing an intention to move a first part of the subject's body, was generated before the second signal, representing another intention to move a second part of the subject's body. For example, EEG data may show that a biosignal detected from the right hemisphere of the subject's brain 1008A, representing an intention to move its left hand, was generated before a biosignal detected from the left hemisphere of brain 1008B (e.g., the left hemisphere of the brain), representing another intention to move the right hand. In another embodiment, EMG data may indicate that a biosignal representing an intention to move body part 1010B (e.g., the right hand) was generated before a biosignal representing a different intention to move another body part 1010B (e.g., the left arm). In some cases, different types of biosignal data (e.g., EEG and EMG) may be used together to determine or improve the accuracy of determining the activation sequence of biosignals.

[0127] The multi-electrode device 1002 can transmit a signal activation sequence of a biosignal via a short-range connection 1004 and identify a specific operation to be performed by the computing device 1006. This operation may include entering one or more alphanumeric characters on the computing device's graphical user interface. In another embodiment, this operation may include moving a cursor displayed by the graphical user interface. This operation may also include operations performed by different types of computing devices, including the control of one or more robotic components or the control of an augmented reality or virtual reality device. The signal processing application can then output instructions for the computing device to perform the identified operation. Continuing from the above embodiment, the computing device 1006 identifies an operation to enter the phrase "Lorem ipsum" 1012, where each alphanumeric character can be determined and entered based on the signal activation sequence of the biosignal at the corresponding point in time. In this way, various operations can be performed and the computing device 1006 can be controlled based on the biosignal activation sequence.

[0128] B. Intentional Communication Interface In some embodiments, biosignal data is converted to access interface operation data from one or more intent communication interfaces, where the interface operation data is used by a signal processing application to identify a specific operation performed by a computing device. In some cases, the intent communication interface includes a set of interface elements, at least one of which includes corresponding interface operation data. Figure 11 shows an example of an intent communication interface 1100 used to convert biosignal data into processing for one or more computing devices, according to some embodiments. For example, the intent communication interface 1100 may include a plurality of interface elements, where each of the plurality of interface elements of the intent communication interface is connected to one or more child interface elements. Each interface element may include interface operation data that identifies a specific operation, which can be accessed when the biosignal data indicates an intention to move both the left and right parts of the body simultaneously (e.g., an intention to grasp both hands together within a given time interval). For example, a subject can access the interface operation data of a particular interface element of the intent communication interface 1100 based on their intention to grasp both their left and right hands. Interface operation data may be used by the same or other computing devices to perform specific operations.

[0129] In some cases, a biosignal activation sequence is used multiple times to traverse one or more interface elements of the intent communication interface until a specific interface element is accessed and the associated operation is accessed. Traversal of the intent communication interface 1100 can begin from the root interface element 1102 of the intent communication interface 1100. For example, a cursor can be used to identify that the root interface element 1102 has been selected. The root interface element 1102 may be connected to one or more interface elements, at which point in time biosignal data can be processed. In Figure 11, the root interface element 1102 is connected to four interface elements, including the "t" interface element, the "e" interface element, the "the" interface element, and the "maybe" interface element. In some cases, the root interface element 1102 contains interface operation data indicating the direction in which the intent communication interface 1100 is traversed. For example, the root interface element 1102 identifies a downward arrow so that the intent communication interface traverses downward to access interface elements 1104, 1106, and 1108.

[0130] A subject can traverse the intent communication interface 1100 based on an activation sequence of biosignal data across multiple time points. For example, a multi-electrode device (e.g., multi-electrode device 1002) can access biosignal data from the subject at a first time point. By analyzing the biosignal data, it can be determined that a first signal has been generated representing an intention to move a first part of the subject's body, where the first signal was generated before a second signal representing another intention to move a second part of the subject's body was generated. The first signal can then be transformed to traverse the root interface element of the intent communication interface to another interface element of the intent communication interface. For example, a subject might imagine clenching their left hand, and as a result, biosignal data will be generated from the right hemisphere of the subject's brain. By analyzing the biosignal data generated from the right hemisphere of the brain, it can be determined that the intent communication interface 1100 should traverse from the root interface element 1102 to the "t" interface element 1104 (i.e., the left child node). In some cases, the cursor identifies the selection of interface element 1104. Next, the subject can access interface operation data associated with interface element 1104 based on the intention to grasp both hands (for example, by entering the letter "t" into the graphical user interface), or traverse the intention communication interface 1100 based on the intention to grasp with the left or right hand.

[0131] In the continuation of the embodiment, the subject can continue traversing the intent communication interface based on the intention to grasp the right hand at a second time point. The intention to grasp the right hand can be associated with biosignals generated from the left hemisphere of the subject's brain. By analyzing the biosignal data generated from the left hemisphere of the brain, it can be determined that the intent communication interface 1100 should be traversed from the "t" interface element 1104 to the "i" interface element 1106. After the second time point, the cursor can identify the selection of interface element 1106. Similar to the previous example, the subject can then access interface operation data associated with interface element 1106 based on the subject's intention to grasp both hands (e.g., inputting the letter "i" into the graphical user interface), or, instead, further traverse the intent communication interface 1100 based on the intention to grasp the left or right hand. The above step of traversing the intent communication interface 1100 can be repeated over subsequent time points until an interface element with the desired interface operation data is reached. In a continuation of this embodiment, the traversal of the intent communication interface 1100 can continue until the "c" interface element 1108 reaches a third time point, where the subject can access interface operation data associated with the interface element 1108 based on the intention to grasp both hands (for example, by entering the letter "c" into the graphical user interface). Various embodiments of inputting text and images using biosignal data are also described in Section IV of this disclosure.

[0132] The intent communication interface 1100 is applicable and can enhance various operations associated with computing devices. In some embodiments, various data and operations are identified from the intent communication interface 1110. As shown in Figure 11, alphanumeric characters are accessible from interface elements 1104, 1106, and 1108. Different words and phrases are also accessible from the intent communication interface 1110. For example, the word "the" is accessible from interface element 1110, and the phrase "I want" is accessible from interface element 1112 of the intent communication interface 1110. In some cases, if biosignal data representing an intention to move the left or right part of the body is detected at a leaf interface element (e.g., a node in a tree with zero child nodes), the traversal of the intent communication interface 1100 returns to the root interface element 1102 because there are no further interface elements to traverse from the leaf interface element. For example, if a signal processing application receives biosignal data at the leaf interface element 1112 indicating an intention to move the left side of the body, the cursor associated with the intention communication interface can return to the root interface element 1102. By returning to the root interface element, the subject can re-navigate the intention communication interface 1102.

[0133] In some cases, one or more words or phrases are assigned to one or more interface elements of the intent communication interface 1110. The words or phrases can be determined based on previous user data, at which point one or more words or phrases can be assigned to each interface element of the intent communication interface 1110. For example, by processing previous user data, it can be determined that the word "maybe" is a word frequently used in a given word processing application. Based on this determination, the intent communication interface 1110 can be updated so that interface element 1114 includes the word "maybe". In some cases, the previous user data includes user-specific data, such as document files created and edited by the subject. Additionally or alternatively, the previous user data may include user group-specific data (e.g., similar geographical locations, similar occupations) and / or general user data. Additionally or alternatively, one or more words or phrases may be configured by the user to be included in the default layout of the intent communication interface 1110. For example, the word "please" is a frequently used term that can be configured by the user to be assigned to one of the interface elements of the intent communication interface 1110, so that the word "please" will be displayed each time the intent communication interface 1110 is presented to the user.

[0134] Additionally or alternatively, interface elements of the intent communication interface 1110 identify one or more words or phrases predicted by a machine learning model. For example, suppose previously entered text data on a graphical user interface includes "the teacher typed into his computer...." Based on the entered text data, one or more interface elements of the intent communication interface 1110 can be updated to include predicted words or phrases that logically follow the existing text. In the remainder of this embodiment, the interface element may include one of the predicted words or phrases, such as "keyboard," "screen," or "device," where these words and phrases are predicted by processing the previous text data using a machine learning model (e.g., a long-short-term memory neural network). Various embodiments of using machine learning techniques to identify one or more words or phrases are described in Section V of this disclosure.

[0135] In some embodiments, the intent communication interface 1100 includes interface elements that identify operations that control the intent communication interface 1110. For example, a subject can access interface operation data of the root interface element 1102 to trigger a change in the direction in which the intent communication interface 1100 is traversed. For example, at the initial point in time, based on the intention to grasp both hands, the subject can access interface operation data associated with the root interface element 1102 (e.g., a downward arrow). The interface operation data accessed from the root interface element 1102 can trigger a change from a downward arrow to an upward arrow. As a result, the intent communication interface 1100 may be traversed upward, thereby enabling access to different characters or words associated with interface element 1110 ("the") and interface element 1112 (phrase "I want"). In another embodiment, a subject can access interface operation data from a specific interface element to access different data from the intent communication interface 1110, including different languages ​​(e.g., German, Spanish) or alphanumeric characters associated with different sets of frequently used words or phrases. In some cases, accessing different data from the intent communication interface 1110 includes access to the option of assigning one or more words / phrases to the corresponding interface elements so that the corresponding interface elements become part of the default layout of the intent communication interface 1110. Therefore, by utilizing different configurations for the intent communication interface 1110, convenient access to various types of information from the intent communication interface 1110 can be facilitated.

[0136] In some embodiments, the intent communication interface 1110 includes interface elements that identify functions associated with a particular application. These functions can be used to launch an application stored on the computing device or to execute one or more commands associated with the application. For example, interface element 1116 identifies the word “Settings” which is used as an application function to open the settings menu of a word processing application. In another embodiment, interface element 1118 identifies the “->” symbol which is used as an application function to move the insertion point of a word processing application to a different location in the document. In some cases, some application functions are assigned to corresponding interface elements based on previous user data.

[0137] C. Methods for controlling computer processing by converting biological signals. Figure 12 shows a process 1200 that converts biosignal data into processing for one or more computing devices, according to some embodiments. For illustrative purposes, the process 1200 is described with reference to the components shown in Figures 1-5, but other implementations are possible. For example, program code stored in a non-temporary computer-readable medium is executed by one or more processing devices (e.g., the multi-electrode device 300 in Figure 3, the electronic device 302 in Figure 3) to cause one or more processing devices to perform one or more operations as described herein.

[0138] In step 1205, the signal processing application accesses the biosignal data collected by the biosignal data acquisition assembly. The biosignal data acquisition assembly may include a housing having one or more electrode clusters, where each cluster of the one or more electrode clusters includes at least one active electrode. In some cases, the biosignal data includes EEG data and / or EMG data.

[0139] In step 1210, the signal processing application identifies a first signal representing an intention to move a first part of the subject's body based on biosignal data. In some cases, the first signal is generated before a second signal representing another intention to move a second part of the subject's body. In some cases, if the biosignal data includes EEG data, the first signal is detected from the left hemisphere of the subject's brain and the second signal is detected from the right hemisphere of the brain. The first part may correspond to the subject's left limb, and the second part may correspond to the subject's right limb. The movement may include any type of action associated with the corresponding part of the body (e.g., gripping, holding, shaking). In some cases, both EEG and EMG data are used together to determine that the first signal was generated before the second signal.

[0140] In step 1215, the signal processing application transforms a first signal to identify a first operation to be performed by the computing device. The first operation may include performing one or more functions associated with the computing device's graphical user interface. These functions may include (i) moving a cursor displayed on the graphical user interface from a first position to a second position, (ii) entering text on the graphical user interface, and (iii) entering one or more images or icons on the graphical user interface. Alternatively, one or more machine learning models may be applied to the entered text to predict additional text to be entered on the graphical user interface.

[0141] In some cases, the first operation includes launching an application stored on the computing device or executing one or more commands associated with the application. Additionally or alternatively, the first operation can be used to control various devices. For example, the computing device may be an augmented reality device or a virtual reality device, and the first operation may include executing one or more operations associated with the augmented reality device or a virtual reality device. In another embodiment, the computing device may include one or more robotic components, and the first operation may include controlling one or more robotic components.

[0142] In some embodiments, a first operation includes accessing interface operation data from one intent communication interface, which is used to determine other operations to be performed by the computing device. In some cases, the intent communication interface includes a set of interface elements. At least one interface element in that set may include corresponding interface operation data. For example, the intent communication interface may be a tree including a root interface element connected to a first interface element and a second interface element. The first operation may include selecting the first interface element of the intent communication interface from the root interface element, in preference to the second interface element. In some cases, the first interface element is associated with first interface operation data, and the second interface element is associated with second interface operation data. A second operation to be performed by the computing device may then be identified by accessing the first interface operation data of the selected first interface element. In some cases, the second operation is identified and selected when biosignal data at a subsequent time point indicates an intention to move both the left and right parts of the body simultaneously (e.g., clenching both hands within a given time interval). A second instruction to perform the second operation may then be output.

[0143] Additionally or alternatively, the intent communication interface may be traversed to access other interface elements based on additional biosignal data collected from the subject at a subsequent point in time. For example, additional biosignal data collected by a biosignal data acquisition assembly may be accessed at another point in time. Based on the additional biosignal data, a third signal representing a third intent to move a second part of the subject's body (e.g., a biosignal representing an intent to move the left arm, detected from the right hemisphere of the brain) may be identified. In some cases, the third signal is generated before a fourth signal representing a fourth intent to move a first part of the subject's body (e.g., a biosignal representing an intent to move the right arm, detected from the left hemisphere of the brain). The third signal can then be transformed to identify a third that is performed by a computing device.

[0144] Based on the third operation, the third interface element of the intent communication interface may be selected in preference to the fourth interface element, where the third and fourth interface elements are connected to the first interface element. In some cases, the third interface element is associated with third interface operation data, and the fourth interface element is associated with fourth interface data. For example, the third interface element may be an interface element containing the letter "c", and the fourth interface element may be another interface element containing the letter "l". The fourth operation to be performed by the computing device can be identified by accessing the third interface operation data of the selected third interface element. In the continuation of this embodiment, the fourth operation may include entering the letter "c" on the graphical user interface. After the fourth operation is identified, a third instruction for performing the fourth operation may be output.

[0145] In step 1220, the signal processing application outputs a first instruction to perform a first operation. In some cases, the signal processing application is inside a computing device, where the computing device can directly access the instruction and perform the operation. In some embodiments, the signal processing application is outside the computing device. For example, the signal processing application may be part of an interface system (e.g., a multi-electrode device), where the signal processing application can send instructions to a computing device via a communication network and perform the operation. Additionally or alternatively, the signal processing application may send instructions to one or more accessory devices (e.g., a smartwatch) that are communicatively coupled to the computing device, so that they can perform identified operations. Additionally or alternatively, steps 1205 to 1220 can be repeated to perform multiple operations over multiple points in time. Process 1200 then terminates.

[0146] V. Input of text and images using the subject's biosignals As described in Section III of this disclosure, biosignal data is used by signal processing applications to enable the identification of various operations performed by computing devices. An intent communication interface can be used to facilitate a subject's brain-based communication by translating the subject's biosignals into one or more operations, such as inputting words or phrases into a word processing application.

[0147] To navigate the intent communication interface, biosignals can be used to identify the intention to move a part of the subject's body (e.g., left hand, right hand), regardless of whether actual physical movement occurs. For example, if a subject wants to traverse the intent communication interface toward the left interface element, they can imagine grasping their left hand. To access interface operation data from the interface element, the subject can imagine grasping both hands simultaneously. The configuration of the intent communication interface allows the subject to perform various computer operations without being limited by cursor speed. In some cases, alphanumeric characters are positioned across various interface elements of the intent communication interface such that frequently used characters are located near the root interface element of the intent communication interface.

[0148] Figure 13 is an exemplary schematic diagram 1300 of an intent communication interface for inputting text and images, according to one embodiment. In Figure 13, the intent communication interface 1302 includes a plurality of interface elements, each containing interface operation data that identifies a specific operation to be performed by the computing device. For example, the intent communication interface 1302 includes an interface element 1304 that identifies the letter "h", and interface elements that identify the letters "t", "e", "n", "i", "o", and "a", respectively. In addition, the intent communication interface 1302 also includes an interface element 1306 that identifies words or phrases such as "the", "i am", "i want", "no", "maybe", and "yes", respectively. In some cases, the words or phrases on the intent communication interface 1302 include predictable suggested words or phrases based on previous user data, in which case one or more words or phrases can be assigned to each interface element of the intent communication interface. In some cases, suggested words or phrases may include one or more phrases that complement previously entered text on the graphical user interface to form a complete sentence. For example, if previously entered text data includes "Please do not hesitate to...", the suggested phrase might include "contact us if you have any questions or comments." Suggested phrases can be assigned to corresponding interface elements of the intent communication interface. In some cases, previous user data may include user-specific data, including document files created and edited by the subject. Additionally or alternatively, previous user data may include user group-specific data (e.g., similar geographical location, similar occupation) and / or general user data.

[0149] Additionally or alternatively, one or more words or phrases may be configured by the user to be included in the default layout of the intent communication interface. For example, the word "please" is a frequently used term that may be configured by the user to be assigned to one of the interface elements of the intent communication interface, so that the word "please" will be displayed each time the intent communication interface is presented to the user.

[0150] The signal processing application can transform the subject's biosignal data across different time points to traverse the intent communication interface 1302 to a specific interface element (e.g., the "h" interface element 1304). In some cases, it analyzes the activation sequence of biosignals to determine which interface element of the intent communication interface 1302 should be traversed. For example, the traversal may begin at the root interface element 1308, at which point the cursor can identify the selection of the root interface element 1308. The signal processing application can detect first biosignal data generated at the first time point, where the first biosignal data represents an intention to move a first part of the body (e.g., an intention to clench the right hand). The signal processing application can then traverse from the root interface element 1308 to the "e" interface element, where the cursor can identify that the "e" interface element has been selected. The signal processing application can detect second biosignal data generated at the second time point, where the second biosignal data represents another intention to move a first part of the body, thereby traversing from the "e" interface element to the "a" interface element. Finally, the signal processing application can detect the third biosignal data generated at the third time point, where the third biosignal data represents a third intention to move a second part of the body (for example, the intention to clench the left hand). The signal processing application can then traverse the intention communication interface 1302 from the "a" interface element to the "h" interface element 1304. After the third time point, the cursor can identify that the "h" interface element 1304 has been selected.

[0151] In the "h" interface element 1302, if fourth biosignal data generated at the fourth time point is detected, the signal processing application can access the "h" character and input it into a graphical user interface (e.g., a word processing application), where the fourth biosignal data represents a fourth intention to move both the first and second parts of the body simultaneously. For example, a subject can access the interface operation data of the "h" interface element 1304 of the intention communication interface 1302 based on the intention to grasp both the left and right hands simultaneously.

[0152] In some cases, if biosignal data indicating an intention to move the left or right part of the body is detected at a leaf interface element (e.g., a node in a tree with zero child nodes), the traversal of the intention communication interface 1302 returns to the root interface element 1308 because there are no further interface elements to traverse from the leaf interface element. For example, if a signal processing application receives biosignal data indicating an intention to move a first part of the body at a given leaf interface element (e.g., the "g" interface element), the cursor associated with the intention communication interface can return to the root interface element 1308. By returning to the root interface element, the subject can re-navigate the intention communication interface 1302.

[0153] As shown in interface element 1306, different words and phrases can be accessed from the intent communication interface 1302. In some embodiments, one or more of the interface elements 1306 are updated based on words or characters previously entered on the graphical user interface. Continuing this embodiment, when the character "h" is entered on the word processing application, the signal processing application can modify the layout of the intent communication interface 1302 to generate an updated intent communication interface 1310. The layout of the updated intent communication interface 1310 may include identical interface operation data directed to an interface element that identifies a single alphanumeric character. However, because the character "h" has been entered, the updated intent communication interface 1310 includes interface elements that identify words such as "have," "home," and "has," respectively. The subject can then traverse the updated intent communication interface 1310 to enter complete words beginning with the character "h," thereby improving the efficiency of entering text or images into the graphical user interface.

[0154] As an exemplary implementation of inputting the word “have” into a word processing application, the signal processing application can initiate a traversal process at the root interface element 1314. The downward arrow at the root interface element 1314 can be changed to an upward arrow (not shown) based on the detection of biosignal data representing an intention to move the first and second parts of the body simultaneously (e.g., an intention to grasp both hands at the same time). The upward arrow may indicate that the traversal of the updated intention communication interface 1302 will be performed upward. At a later point, the signal processing application can detect biosignal data generated from the left hemisphere of the subject’s brain representing another intention to move the first part of the body (e.g., an intention to grasp the right hand). The signal processing application can then traverse from the root interface element 1314 to the “have” interface element 1316. In interface element 1316, the subject can input the word "have" into a word processing application based on the detection of further biosignal data representing an intention to move the first and second parts of the body simultaneously.

[0155] Additionally or alternatively, the intent communication interface may be configured to provide other types of input, including images, emojis, and / or characters in other languages ​​(e.g., Arabic). In some cases, various keyboard layouts are accessed from the intent communication interface. In some cases, accessing other types of input from the intent communication interface includes access to the option of assigning one or more words / phrases to corresponding interface elements so that the corresponding interface elements become part of the default layout of the intent communication interface. For example, Figure 14 shows an example of an intent communication interface 1400 for inputting images, according to some embodiments. For example, the image input by the intent communication interface 1400 may be an emoji. The emoji layout can be accessed instead of the English layout by accessing an interface element that identifies an operation to switch from the English layout to the emoji layout (e.g., the “Settings” interface element 1116 in Figure 11). Furthermore, the emoji layout of the intent communication interface 1400 can be returned to the English layout by accessing the interface operation data of interface element 1402, which identifies another operation to switch back to English. In another embodiment, Figure 15 shows another example of an intent communication interface 1500 for inputting text in other languages, according to some embodiments. In Figure 15, the intent communication interface 1500 shows a layout that identifies Arabic characters. Similar to the emoji layout of the intent communication interface 1400, the Arabic layout can be switched back to the English layout by accessing the interface operation data of interface element 1502 and identifying another operation to switch to English.

[0156] In some embodiments, the intent communication interface is used to perform one or more operations associated with a particular type of application. These operations can be used to launch an application stored on a computing device or to execute one or more commands associated with an application. For example, Figure 16 shows an example of an intent communication interface 1600 for operating a computer application, according to some embodiments. The intent communication interface 1600 may be used to perform one or more operations associated with a chess game application 1602. For example, the subject's biosignal data can be converted over a first set of time points to select option 1604 to play a chess game with a friend. Then, additional biosignal data from the subject can be converted over a second set of time points to select option 1606 to start a chess game. Next, the layout of the intent communication interface 1600 can be updated to select and move pieces in the chess game application 1602, thereby allowing the subject to play the game without physical movement.

[0157] VI. Enhancing intent communication interfaces using machine learning techniques In addition, intent communication interfaces can be further enhanced using machine learning techniques. Figure 17 is a schematic diagram 1700 of the use of machine learning techniques to enhance an intent communication interface in some embodiments. In Figure 17, the intent communication interface 1702 (e.g., intent communication interface 1302 in Figure 13) is used to input words and phrases into a word processing application 1704. The intent communication interface 1702 may include one or more interface elements that identify words or phrases predicted by a machine learning model. By feeding predicted words and phrases into the interface elements based on the context of existing text, machine learning techniques can improve the efficiency of performing complex tasks on a graphical user interface. Additionally or alternatively, various operations corresponding to a particular type of application may be predicted and then input into the intent communication interface 1702 as a person skilled in the art would assume. For example, a machine learning model may process an existing paragraph in a word processing application to produce output that predicts text formatting options such as "bold," "italic," and "underline."

[0158] As an exemplary embodiment, a word processing application 1704 displays text data 1706 entered by a subject, such as "the teacher typed into his computer...". A text prediction application (not shown) can apply a machine learning model to the text data 1706, which has been trained using a training dataset containing text data previously entered by the subject and / or other users. The machine learning model can produce an output containing one or more predicted words that would follow the text data 1706. For example, the predicted words might include "keyboard," "screen," or "device." In some cases, the predicted words might include one or more phrases that complement the text data to form a complete sentence. For example, if the previously entered text data includes "Please do not hesitate to...", the predicted phrase might include "contact us if you have any questions or comments." The predicted phrases can be assigned to corresponding interface elements of the intent communication interface 1702. The layout of the intent communication interface 1702 can be updated so that at least some interface elements include predicted words. In particular, one or more interface elements of the intent communication interface 1702 may include predicted words or phrases, such as the “screen” interface element 1708, the “keyboard” interface element 1710, and the “device” interface element 1712. In some cases, other interface elements 1714 of the intent communication interface 1702 may continue to include a default set of alphanumeric characters to allow the user to enter text that will differ from the predicted words or phrases.

[0159] Figures 18-22 illustrate example configurations of machine learning models that predict one or more words based on text data. To generate predicted words, the text prediction application can receive text data (e.g., text data 1706) containing multiple tokens (e.g., words, punctuation). The text prediction application can preprocess the text data by encoding each token into an input embedding (e.g., a vector represented by multiple values) based on its semantic features. In some cases, the text prediction application is configured to generate input embeddings having a predetermined number of dimensions. Each input embedding may contain a set of values ​​that identify one or more semantic features of the text data. In some cases, the text prediction application uses a pre-trained model (e.g., word2vec, fastText) to encode each token into an input embedding.

[0160] Text prediction applications can apply machine learning models to input embeddings representing text data. For example, the machine learning model may be a Recurrent Neural Network (RNN). Additionally or alternatively, the sequence prediction layer may include a Long Short-Term Memory (LSTM) network, which is a type of RNN. The LSTM network may be a bidirectional LSTM network. In some embodiments, the input embeddings are processed using one or more network layers of the machine learning model to generate a set of output features. The set of output features can be processed using a fully connected layer of the machine learning model to generate an output that identifies one or more predicted words following the text data. As a result, the machine learning model can generate predicted words based on the contextual relationship between the words and the text data.

[0161] FIG. 18 is a diagram illustrating an exemplary process of a recurrent neural network 1800 for generating words predicted based on text data, according to some embodiments. As shown in the RNN, it includes a chain of iterative modules (“cells”) of a neural network. Specifically, the process of the RNN includes an iteration of a single cell indexed by the position of a text token (t) within the text tokens of the text data. To provide that iterative behavior, the RNN maintains a hidden state s ,

[0163] , , t , t , t which is provided as an input to the next iteration of the network. As referred to herein, the variables s t and h t are used interchangeably to represent the hidden state of the RNN. As shown in the left portion of FIG. 18, the RNN receives a feature representation of a text token x t and a hidden state value s t-1 determined using a set of input features of previous text tokens. The following equation provides how the hidden state s t is determined. TIFF2026513516000002.tif14170 Here, U and W are weight values applied to x t and s t-1 respectively, and φ is a non-linear function such as tanh or ReLU.

[0162] The output of the recurrent neural network is expressed as follows. TIFF2026513516000003.tif16170 Here, V is a weight value applied to the hidden state value s t respectively.

[0163] Thus, the hidden state s<000001B> can be called the memory of the network. In other words, the hidden state s t depends on information associated with inputs and / or outputs used or derived from one or more previous text tokens. Step o tThe output in this case is a set of values ​​used to generate one or more predicted words following the text data, and it is computed based at least partially on memory at the text token position t.

[0164] Figure 19 shows another example of a recurrent neural network processing 1900 for generating predicted words based on text data, according to one embodiment. For clarity, Figure 19 shows the network as an expanded RNN. In Figure 19, φ is specifically illustrated as a tanh function, and the linear weights U, V, and W are not explicitly illustrated. Unlike conventional deep neural networks that use different parameters in each layer, the RNN shares identical parameters (U, V, W above) across all text tokens. This reflects the fact that the same task is performed at each text section location with different inputs. This significantly reduces the total number of parameters to be learned.

[0165] Figure 20 is an exemplary schematic diagram of a long-short-term memory network 2000 for generating predicted words based on text data, according to one embodiment. An LSTM network is a type of RNN, where the LSTM network learns the long-term dependencies between tokens in the text data. In some cases, the LSTM network is a bidirectional LSTM network. A bidirectional LSTM network applies two LSTM network layers to the text token input features, namely (i) a first LSTM network layer trained to process the input features of text tokens according to a forward sequence of text tokens in the text data (e.g., from the first text token to the last text token) and (ii) a second LSTM network layer trained to process the input features of text tokens according to a reverse sequence of text tokens in the text data (e.g., from the last text token to the first text token).

[0166] As shown in Figure 20, an LSTM network, like the RNNs shown in Figures 18 and 19, can contain a series of cells. Similar to RNNs, each cell in the LSTM network acts to compute a new hidden state for the next time step.

[0167] Hidden state s t In addition to maintaining and updating the cell state C, the LSTM network also maintains and updates the cell state C. t Maintain the following. As used herein, the cell state encodes information of the input observed up to that step (step by step). In some embodiments, instead of using a single layer in a standard RNN such as the tanh layer shown in Figure 19, the LSTM network includes a second layer for adding and removing information from the cell via a set of gates. The gates include a point-specific sigmoid function coupled to an Hadamard product multiplication function, where the sigmoid function is as follows: TIFF2026513516000004.tif9170

[0168] TIFF2026513516000005.tif3170 symbol or The symbol TIFF2026513516000006.tif2170 represents the Hadamard product. Gates can allow or deny the flow of information through a cell. Since the sigmoid function takes values ​​between 0 and 1, the function value affects how much of each feature of the previous text token should be allowed to pass through the gate. Referring again to Figure 20, the LSTM network cell contains three gates: a forget gate, an input gate, and an output gate.

[0169] Figure 21 is an exemplary schematic diagram 2100 for implementing a forget gate and input gate of a long-short-term memory network according to some embodiments. For example, Figure 21 shows a forget gate 2102 of an LSTM network. The LSTM network uses the forget gate to recall previous hidden states h t-1 and the current input x tBased on this, it is determined which information to discard in the cell state (long-term memory). The LSTM network uses the sigmoid function of the hidden gate to determine h t-1 Information from and x t Information is passed from there. The output of the forget gate contains values ​​between 0 and 1. The LSTM network determines that outputs close to 0 should be forgotten. Conversely, the LSTM network determines that outputs close to 1 should be retained. Forget gate f t The output value can also be expressed as follows: TIFF2026513516000007.tif18170 Here, W f b is a scalar constant, f The 'b' term is the bias term, and the parentheses indicate the combination of input values.

[0170] Figure 21 also shows the input gate processing of a long-short-term memory network according to some embodiments. The LSTM network performs input gate processing across two phases, shown in phases 2104 and 2106, respectively. For example, the first phase 2104 of the LSTM network includes passing the previous hidden state and the current input to a sigmoid function. The sigmoid function takes the input value (h t-1 , x t The tanh function converts the input values ​​to values ​​between 0 and 1 to determine whether the cell state values ​​should be updated. In some cases, 0 indicates a low importance value and 1 indicates a high importance value. Furthermore, the LSTM network passes the hidden state and current input to the tanh function, which squishes the input values ​​between -1 and 1 to facilitate network tuning. Thus, the tanh function gives new candidate values A vector for TIFF2026513516000008.tif5170 may be created and added to the cell state. Sigmoid function i t The output value can also be expressed by the following formula. TIFF2026513516000009.tif17170

[0171] Also, the tanh function The output value of TIFF2026513516000010.tif5170 can also be expressed by the following formula. TIFF2026513516000011.tif13170

[0172] The second phase 2106 facilitates the forgetting of information corresponding to the input value to the forget gate, and the old state C t-1 Oblivion Gate f t This may include multiplying the output value by the cell state. The new candidate value for TIFF2026513516000012.tif14170 is obtained via point-by-point addition from the previous cell state C t-1 It is added to. This can also be expressed by the following relationship. TIFF2026513516000013.tif12170

[0173] Figure 22 shows an exemplary processing of the output gate 2200 of a long-short-term memory network according to one embodiment. The LSTM network uses the output gate to process the cell state C t The output is generated by applying the corresponding value. The output gate can be used to determine what the next hidden state should be. As mentioned above, the hidden state may contain information about the previous input. The hidden state can also be used for prediction. First, the previous hidden state and the current input can be passed to the sigmoid function. Next, the newly modified cell state can be passed to the tanh function. The tanh output can be multiplied by the sigmoid output to determine what information the hidden state should contain. Therefore, the output may be the hidden state. The new cell state and new hidden state may be carried over to the next time step.

[0174] For example, an LSTM network uses the input value h t-1 , x t The sigmoid function can pass this to the cell state C. tThe tanh function is applied to this, which is then modified by a forget gate and an input gate. The LSTM network can then multiply the output of the tanh function (e.g., a value between -1 and 1 representing the cell state) by the output of the sigmoid function. The LSTM network can retrieve the hidden state determined from the output gate (e.g., return_sequence=true) and assign the hidden state as a set of output features used to generate the predicted word. For example, a fully coupled neural network can be used to process a given set of output features and generate the predicted word following the text data. The LSTM network may continue such a retrieval process so that the set of output features is determined toward the text token. In some cases, the output of the output gate is a new hidden state used for the subsequent text token in the text data. The processing of the output gate can be expressed by the following equation: TIFF2026513516000014.tif19170TIFF2026513516000015.tif16170

[0175] The LSTM networks shown in Figures 20-22 are just one example of a machine learning model that generates predicted words or phrases using text data. In some cases, gated recurrent units ("GRUs") or other variations of RNNs are used. Furthermore, those skilled in the art will recognize that the internal structure shown in Figures 20-22 can be modified in numerous ways, for example, including peephole connections.

[0176] VII. Controlling various devices based on the subject's biosignals In some embodiments, an intent communication interface is used to perform operations associated with a specific type of computing device, including augmented or virtual reality devices, robotic components, and accessory devices. For example, augmented reality (AR) glasses can display a set of virtual screens. The intent communication interface may be traversed using biosignals across different points in time to select a first virtual screen from a set of virtual screens. Once the first virtual screen is selected, interface elements of the intent communication interface (e.g., modifying the layout of the intent communication interface) may be automatically updated to include a set of operations (e.g., deletion, creation of a new virtual screen, moving to a different location, increasing or decreasing screen size, correcting screen orientation). The intent communication interface can then be traversed again to identify a specific operation (e.g., increasing screen size) from the set of operations. The intent communication interface may again be automatically updated so that its interface elements identify a subset of operations related to increasing screen size (e.g., 1x, 2x, 3x). As a result, multiple traversals of the intent communication interface can be performed to efficiently execute tasks specifically associated with AR glasses. The technology using biosignal activation sequences can be extended to other types of devices, such as computing devices with robotic components (e.g., drone devices).

[0177] A. Augmented reality devices or virtual reality devices In some embodiments, biosignal data is converted to access interface operation data from one or more intent communication interfaces, where the interface operation data is used by a signal processing application to identify one or more operations performed by an augmented reality or virtual reality device. Figure 23 is an exemplary schematic diagram 2300 of an intent communication interface 2302 for converting biosignal data into one or more operations associated with a virtual reality device 2304, according to some embodiments. For example, the intent communication interface 2302 may include multiple interface elements. Each interface element may include interface operation data that identifies a particular operation, which can be accessed by detecting biosignal data representing an intention to move the left and right parts of the body simultaneously. For example, a subject can access the interface operation data of a particular interface element of the intent communication interface 2300 based on the intention to grasp both their left and right hands.

[0178] As described above, a sequence of biosignal activations can be used multiple times to traverse one or more interface elements of the intent communication interface until a specific interface element is accessed and the associated operation is accessed. For example, a multi-electrode device (e.g., multi-electrode device 1002) can access biosignal data from a subject at a first time point. By analyzing the biosignal data, a first signal representing an intention to move a first part of the subject's body can be detected, where the first signal is generated before a second signal representing another intention to move a second part of the subject's body. The first signal can then be transformed to traverse the root interface element of the intent communication interface 2302 to another interface element of the intent communication interface.

[0179] For example, a subject might imagine clenching their left hand, and as a result, biosignal data generated from the right hemisphere of the subject's brain would be detected. By analyzing the biosignal data generated from the right hemisphere of the brain, the intent communication interface 2302 can determine that it should traverse from the root interface element to the “menu” interface element 2306. In some cases, the cursor identifies the selection of the “menu” interface element 2306. The subject can then access the interface operation data associated with interface element 2306 based on their intention to clench both hands. The “menu” operation can then be performed by the virtual reality device 2304, and as a result, another virtual screen with different menu options is displayed on the virtual reality device 2304. In some cases, the layout of the intent communication interface 2302 may be modified to include a set of suboperations that can be performed by the virtual reality device 2304, where the set of suboperations includes one or more operations that can be performed within the “menu” (e.g., opening a game or chat application, configuring wireless network settings).

[0180] Alternatively, instead of accessing the "menu" interface element 2306, the subject can further traverse the intent communication interface 2302 based on the intention to grasp the right hand, and as a result reach the "volume" interface element 2308. The subject may also access the interface operation data associated with the "volume" interface element 2308, which causes a modification of the layout of the intent communication interface 2302 to include a "+" interface element for increasing the volume of the virtual reality device 2304 and a "-" interface element for decreasing the volume of the virtual reality device 2304. The subject can then traverse the modified intent communication interface 2302 to increase or decrease the volume of the virtual reality device 2304.

[0181] Various operations associated with the virtual reality device 2304 can trigger interface elements of the intent communication interface 2302. For example, it is possible to access the "keyboard" interface element 2310 and display a modified intent communication interface (e.g., intent communication interface 1100 in Figure 11) having a layout including alphanumeric characters and predicted words. The subject can further access the interface operation data of the root interface element of the intent communication interface 2302 to cause a change in the direction in which the intent communication interface 2302 is traversed. For example, the intent communication interface 2300 can be traversed upward, thereby enabling access to the interface operation data of the "zoom" interface element 2312. The interface operation data of the "zoom" interface element 2312 can be used to change the zoom level of one or more image objects displayed in the interface operation data accessed by the "zoom" interface element 2312. Those skilled in the art will be able to trigger interface elements of the intent communication interface 2302 with other types of operations associated with the virtual reality device 2304, which facilitates efficient control of the virtual reality device 2304 based on the subject's biosignals (and without physical movement).

[0182] B. Robotic Devices In some embodiments, biosignal data is converted to access interface operation data from one or more intent communication interfaces, where the interface operation data is used by a signal processing application to identify one or more operations performed by a computing device having one or more robotic components. Figure 24 is an exemplary schematic diagram 2400, according to some embodiments, using an intent communication interface 2402 to convert biosignal data into one or more operations associated with a computing device having one or more robotic components. The robotic components can be associated with any type of robot (e.g., humanoid robot, assembly line robot). For example, the computing device could be a drone device 2404, which includes components for flying through the air. For example, the intent communication interface 2402 may include multiple interface elements. Each interface element may include interface operation data that identifies a particular operation, which can be accessed when biosignal data representing an intention to move the left and right parts of the body simultaneously is detected.

[0183] Therefore, by using a biosignal activation sequence over multiple iterations, one or more interface elements of the intent communication interface can be traversed until a specific interface element is accessed and the associated operation is accessed. For example, a multi-electrode device (e.g., multi-electrode device 1002) can access biosignal data from a subject at a first time point. By analyzing the biosignal data, a first signal representing an intention to move a first part of the subject's body can be detected, where the first signal is generated before a second signal representing another intention to move a second part of the subject's body. The first signal can then be transformed to traverse the root interface element of the intent communication interface 2402 to another interface element of the intent communication interface.

[0184] For example, a subject might imagine clenching their left hand, and as a result, biosignal data generated from the right hemisphere of the subject's brain would be detected. By analyzing the biosignal data generated from the right hemisphere of the brain, the intent communication interface 2402 can determine that it should traverse from the root interface element to the "forward" interface element 2406. The subject can then access the interface operation data associated with interface element 2406 based on the intention to clench both hands. The "forward" operation can then be performed by the drone device 2404, and as a result, the drone device 2404 may move in the forward direction. In some cases, the cursor does not return to the root interface element but remains within the "forward" interface element 2406 so that the drone device 2404 can continue to move in the forward direction. To return to the root interface element, the subject can traverse to a leaf interface element (e.g., a node in a tree with zero child nodes). In the case of biosignal data representing an intention to move the left or right part of the body at a leaf interface element, traversal of the intent communication interface 2402 can return to the root interface element.

[0185] The subject can further traverse the intent communication interface 2402 to access other types of operations, including “turn left,” “menu,” “ascend,” and “descend.” In some cases, the camera component of the drone device 2404 is activated based on accessing interface operation data associated with the “camera” interface element 2408. Those skilled in the art will be able to activate the interface elements of the intent communication interface 2402 with other types of operations associated with the drone device 2404, thereby facilitating efficient control of the drone device 2404 based on the subject's biosignals (and without requiring physical movement).

[0186] C. Accessory Devices In some embodiments, biosignal data is converted to access interface operation data from one or more intent communication interfaces, where the interface operation data is used by a signal processing application to identify one or more operations performed by an accessory device. Figure 25 is an exemplary schematic diagram 2500, according to some embodiments, using an intent communication interface 2502 to convert biosignal data into one or more operations associated with an accessory device. The accessory device may include various devices (e.g., wireless headphones, heart monitor, smartwatch). For example, the accessory device may be a smartwatch device 2504. For example, the intent communication interface 2502 may include multiple interface elements. Each interface element may include interface operation data that identifies a particular operation, which can be accessed when biosignal data indicates that the left and right parts of the body are activated simultaneously (e.g., both parts are activated within a given time interval).

[0187] A sequence of biosignal activations can be used multiple times to traverse one or more interface elements of an intent communication interface until a specific interface element is accessed and the associated operation is accessed. For example, a multi-electrode device (e.g., multi-electrode device 1002) can access biosignal data from a subject at a first time point. By analyzing the biosignal data, a first signal representing an intention to move a first part of the subject's body can be detected, where the first signal is generated before a second signal representing another intention to move a second part of the subject's body. The first signal can then be transformed to traverse the root interface element of the intent communication interface 2502 to another interface element of the intent communication interface.

[0188] For example, a subject might imagine clenching their left hand, and as a result, biosignal data generated from the right hemisphere of the subject's brain would be detected. By analyzing the biosignal data generated from the right hemisphere of the brain, it can be determined that the intent communication interface 2502 should traverse from the root interface element to the “menu” interface element 2506. The subject can then access the interface operation data associated with interface element 2506 based on their intention to clench both hands. The “menu” operation can then be performed by the smartwatch device 2504, and as a result, different menu options may be displayed on the smartwatch device 2504. In some cases, the layout of the intent communication interface 2502 may be modified to include a set of suboperations that can be performed by the smartwatch device 2504, where the set of suboperations includes one or more operations that can be performed within the “menu” (e.g., opening a smartwatch application, configuring wireless network settings).

[0189] The subject can further traverse the intent communication interface 2502 to access other types of operations, including “select object,” “scroll left,” “record heart rate,” and “increase volume.” Those skilled in the art can introduce interface elements of the intent communication interface 2502 with other types of operations associated with the smartwatch device 2504, thereby facilitating efficient control of the smartwatch device 2504 based on the subject's biosignals (and without physical movement).

[0190] VIII. Examples of Computing Environments Any suitable computing system or any group of computing systems can be used to perform the processing described herein. For example, Figure 26 shows a computing system 2600 that can implement any of the computing systems or environments described above. In some embodiments, the computing system 2600 includes a processing device 2602 that runs a signal processing application 2615 for translating biological signals into computer device processing, a memory for storing various data calculated or used by the signal processing application 2615, an input device 2614 (e.g., a mouse, stylus, touchpad, touchscreen), and an output device 2616 that presents the output to the user (e.g., a display device that displays graphical content generated by the signal processing application 2615). For illustrative purposes, Figure 26 shows a single computing system in which the signal processing application 2615 is run and the input device 2614 and output device 2616 are present. However, these applications, datasets, and devices can be stored or included across different computing systems having devices similar to those shown in Figure 26.

[0191] The embodiment in Figure 26 includes a processing device 2602 communicatively coupled to one or more memory devices 2604. The processing device 2602 executes computer-executable program code stored in the memory devices 2604, accesses information stored in the memory devices 2604, or both. Examples of the processing device 2602 include a microprocessor, an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or any other suitable processing device. The processing device 2602 may include any number of processing devices, including a single processing device.

[0192] The memory device 2604 includes any suitable non-temporary computer-readable medium for storing data, program code, or both. The computer-readable medium may include any electronic, optical, magnetic, or other storage device capable of providing the processor with computer-readable instructions or other program code. Non-exclusive examples of the computer-readable medium include magnetic disks, memory chips, ROM, RAM, ASICs, optical storage devices, magnetic tapes, or other magnetic storage devices, or any other medium from which a processing device can read instructions. Instructions may include processor-specific instructions generated by a compiler or interpreter from code written in any suitable computer programming language, including, for example, C, C++, C#, Visual Basic, Java, Python, Perl, JavaScript, and ActionScript.

[0193] The computing system 2600 may further include several external or internal devices, such as a display device 2610 or other input or output devices. For example, the computing system 2600 is shown with one or more input / output ("I / O") interfaces 2608. The I / O interfaces 2608 can receive input from input devices or provide output to output devices. One or more buses 2606 are also included in the computing system 2600. Each bus 2606 connects one or more components of the computing system 2600 to each other or to external components in a communicative manner.

[0194] The computing system 2600 executes program code that configures the processing device 2602 to perform one or more of the processes described herein. The program code includes, for example, code that implements the signal processing application 2615, or other suitable applications that perform one or more of the processes described herein. The program code may reside in the memory device 2604 or any suitable computer-readable medium, or it may be executed by the processing device 2602 or any other suitable processor. In some embodiments, all modules within the signal processing application 2615 are stored in the memory device 2604, as shown in Figure 26. In additional or alternative embodiments, one or more of these modules from the signal processing application 2615 are stored in different memory devices of different computing systems.

[0195] In some embodiments, the computing system 2600 also includes a network interface device 2612. The network interface device 2612 includes any device or any group of devices suitable for establishing wired or wireless data connections to one or more data networks. Non-limiting examples of the network interface device 2612 include Ethernet network adapters, modems, and / or such. The computing system 2600 can use the network interface device 2612 to communicate with one or more other computing devices over the data network (for example, computing devices that receive inputs to or display outputs from the signal processing application 2615).

[0196] The input device 2614 may include any device or any group of devices suitable for receiving visual, auditory, or other appropriate inputs that control or influence the operation of the processing device 2602. Non-limiting examples of the input device 2614 include touchscreens, styluses, mice, keyboards, microphones, and other mobile computing devices. The output device 2616 may include any device or any group of devices suitable for providing visual, auditory, or other appropriate sensory outputs. Non-limiting examples of the output device 2616 include touchscreens, monitors, and other mobile computing devices.

[0197] Figure 26 shows the input device 2614 and output device 2616 as local to a computing device that performs an application to convert biosignals, but other implementations are possible. For example, in some embodiments, one or more of the input device 2614 and output device 2616 include a remote client computing device that communicates with the computing system 2600 via a network interface device 2612 using one or more data networks described herein.

[0198] IX. Prediction of traumatic brain injury based on sleep state Specific aspects and embodiments of this disclosure relate to systems and methods for predicting the presence of traumatic brain injury (TBI) based on neural signaling data associated with one or more sleep states. The neural signaling data may be acquired over one or more sleep duration periods of a subject via a physiological data acquisition assembly. The physiological data acquisition assembly includes at least one single channel of neural signaling data, in which at least one reference electrode and at least one active electrode are in close proximity. The assembly may be worn by a subject. For example, the assembly may include a patch configurable to be positioned (e.g., adhered) on the subject's forehead. Furthermore, the patch may have an adhesive film to which electrodes can be attached to collect neural signaling data.

[0199] In some embodiments of this disclosure, neural signal data can be used to predict, characterize, and / or analyze one or more sleep states. Sleep states may be any identifiable sleep or wakefulness state that represents behavioral, physical, or signal characteristics. In some cases, neural signal data is processed to estimate categories indicating predictions about whether a subject is awake or asleep for each of a plurality of time intervals, and potentially to estimate a specific type or stage of sleep if the subject is estimated to be asleep. Estimation may be based on converting time-domain electrical signals to frequency-domain intensity or power values ​​for each of a plurality of time intervals. Feature quantities may be defined as cumulative or maximum intensity or power values ​​in various frequency bands. Sleep states may then be estimated based on absolute or relative values ​​of one or more feature quantities. States may include Stage 1 sleep states, Stage 2 sleep states, Stage 3 sleep states, and REM sleep states.

[0200] Furthermore, in some embodiments, artificial intelligence (AI) techniques can be used to predict whether a subject will have a given condition, predict the severity of a given condition, or predict the effectiveness of treating a given condition. The AI ​​techniques may include inserting signal processing (e.g., applying one or more signal transformations) and using one or more models or rules to generate epoch-specific, night-specific, or subject-specific predictions. For example, nerve signals may be collected over a sleep period (e.g., nighttime). The nerve signals may be separated into epochs corresponding to absolute or relative time increments over a time period (e.g., time intervals of 1 minute, 5 minutes, or 10 minutes), and a spectrum may be generated for each epoch such that the power or intensity of each of the various frequency bands may be identified for each time increment.

[0201] The presence of TBI can be associated with a decrease in Stage 2 sleep. The presence of TBI may be further associated with an increase in Slow Wave Sleep (SWS) (i.e., Stage 3 sleep). Therefore, artificial intelligence rules can be defined to predict the likelihood of Stage 2 sleep deprivation and / or TBI based on features. For example, clustering techniques, support vector machine (SVM) techniques, principal component techniques, independent component techniques, logistic regression techniques, etc., may be used to predict, for each time epoch, whether the subject is in Stage 2 sleep (vs. Stage 1, Stage 3, REM, or wakefulness). In some cases, for each epoch, a likelihood that the subject is in Stage 2 sleep may be generated and then compared to a predetermined threshold or a learned threshold to predict whether the subject is in or was in Stage 2 sleep.

[0202] Furthermore, rules can be defined to predict whether a subject is suffering from TBI based on Stage 2 sleep predictions. For example, a rule may indicate that a subject is suffering from TBI if an epoch is predicted to be Stage 2 sleep below a threshold percentage (e.g., 10%, 15%, 20%, 25%, 30%, or 35%). In another embodiment, a rule may indicate that a subject is suffering from TBI by identifying that the length of Stage 2 sleep for one or more epochs (e.g., relative or absolute length) is below a predetermined threshold for healthy or normal sleep, or for a threshold learned by the subject. Similarly, a rule may indicate that a subject is suffering from TBI if an epoch is predicted to be Stage 3 sleep above a threshold percentage. Furthermore, a rule may indicate that a subject is suffering from TBI by identifying that the length of Stage 3 sleep exceeds a predetermined threshold for healthy or normal sleep, or for a threshold learned by the subject.

[0203] In certain embodiments, a subject may be suspected of having TBI after a head injury. Accordingly, neural signaling data may be collected and processed for one night's sleep after the injury. The neural signaling data can be divided into time segments, and a detection algorithm can be used to predict a subset of time segments associated with stage 2 sleep. Furthermore, segment-specific indices can be determined for each subset of time segments. The segment-specific indices may be the length of time spent in stage 2 sleep. The segment-specific indices can then be combined to generate a cumulative index representing the estimated absolute amount of time during which the subject is predicted to have been in stage 2 sleep throughout the night. Furthermore, AI technology can be implemented to generate a risk level index based on the cumulative index. The risk level index may be the likelihood that the subject has TBI. The AI ​​technology may output a risk level index based on predetermined rules that indicate a risk level index based on a cumulative index that is less than one or more threshold lengths. For example, the AI ​​technology can learn one or more thresholds from data indicating the normal or average length of stage 2 sleep for one night's sleep of a healthy subject or a subject before TBI was suspected. Next, the AI ​​technology can be trained to output a percentage as a risk level index representing the likelihood that a subject has traumatic brain injury, based on cumulative indicators and learned thresholds.

[0204] In this way, the detection and diagnosis of TBI can be improved. In particular, by implementing a detection algorithm that analyzes neural signal data and predicts sleep states, indicators (e.g., the absolute or relative amount of time a subject spent in a particular sleep state) can be derived with improved accuracy. Furthermore, by implementing AI technology that predicts risk level indicators based on the accumulation of indicators, for example, the accuracy of diagnosing TBI can be improved. In particular, the risk level indicator can provide a more accurate representation of the likelihood that a subject has TBI than neurological examinations, because the AI ​​technology is trained to make predictions using previous sleep data for the subject or associated subjects (e.g., healthy subjects of similar age to the subject, subjects of similar age with TBI, etc.). The risk level indicator may also be more accurate than current imaging diagnostics for TBI due to changes in sleep patterns used to predict risk level indicators associated with all levels of TBI (i.e., mild, moderate, and severe TBI). Furthermore, monitoring subjects to generate risk level indicators and outputting the results (e.g., percentages representing the risk level indicators) can facilitate the efficient treatment of TBI.

[0205] X. Predicting sleep states Neural signal data collected over one or more time periods via a physiological data acquisition assembly can be divided into time segments. Typically, neural signals can be examined temporally in a series of increments called epochs. For example, if neural signals are used for sleep analysis, sleep may be segmented into one or more epochs for use in analysis. Epochs can be segmented into different sections using a scanning window, where the scanning window defines different sections of the time series increment. Code can move the scanning window (incrementally or via shift) via a sliding window or a shift window, where the sections of the sliding window have overlapping or non-overlapping time series sequences. Alternatively, epochs may extend, for example, across the entire time series. In some embodiments, each epoch may be classified to correspond to a predicted sleep state represented. In some cases, prior to classification, epochs are normalized or double-normalized based on (e.g.) frequency information, amplitude information, power, intensity, or other appropriate features of EEG data that can correlate with sleep states. U.S. Patent Application No. 11 / 431,425, filed on 9 May 2006, is incorporated herein by reference for all purposes and discloses exemplary techniques for normalizing biometric data.

[0206] Furthermore, to predict sleep states, the detection algorithm may be configured in the time or frequency domain to detect signatures (e.g., frequency domain features, time domain features, time-frequency domain features, etc.) corresponding to predictions about whether the subject is asleep, and if sleep is detected, the detection algorithm may further respond to predictions of sleep states. The detection algorithm is executable for one or more epochs and can predict sleep states for one or more epochs. For example, a wakeful-sleep state can be predicted by detecting signals in one or more specific frequency bands (e.g., a band extending between about 13 and about 60 Hz with an amplitude of at least about 30 microvolts (μV)) (i.e., beta waves). The frequency band and amplitude can be determined by converting time-domain electrical signals to the frequency domain via mathematical transformations (e.g., Fourier transforms) or other suitable techniques.

[0207] Furthermore, the sleep states that the detection algorithm can predict may include Stage 1 sleep states, Stage 2 sleep states, Stage 3 sleep states, and Rapid Eye Movement (REM) sleep states. In one embodiment, the frequency band for detecting Stage 1 sleep from EEG data can be defined to correspond to a specific type of wave and / or sleep stage. For example, the frequency band corresponding to Stage 1 sleep states may be defined to span between 3 and 8 Hz. Thus, if the amplitude of the 3-8 Hz band is between 50 and 100 μV (i.e., theta waves), it can be inferred that the subject was in Stage 1 sleep.

[0208] Additional sleep state features, such as sleep spindles and K-complex waves, can be identified through sleep state prediction detection algorithms. For example, in the time domain, high-frequency bands lasting less than 2 seconds (e.g., a frequency band of approximately 15 Hz) may be detected as sleep spindles. Similarly, low-frequency bands lasting approximately 1 second in the time domain (e.g., a frequency band between 1 and 4 Hz and an amplitude between 100 and 200 μV) (i.e., delta waves) may be detected as K-complex waves. Therefore, if one or more portions of EEG data are detected as sleep spindles, and one or more portions of EEG data detected as K-complex waves are detected subsequently or nearby, it may be inferred that the subject was in stage 2 sleep.

[0209] In another embodiment, a frequency band extending between 1–4 Hz can be detected for a significantly longer period than 2 seconds (e.g., 20 minutes), which may indicate that the subject was in stage 3 sleep. Stage 3 sleep may also be called slow-wave or delta sleep. Furthermore, a frequency band extending between approximately 13–60 Hz and an amplitude of at least approximately 30 μV (i.e., beta waves) may indicate that the subject was in REM sleep. However, beta waves may also be detected during waking sleep. Therefore, additional physiological data, physical or biological indicators, or other appropriate data can be acquired and identified within the detection algorithm to distinguish between REM sleep and waking sleep. For example, EMG data may be acquired, and the detection algorithm may detect phase events (e.g., rapid eye movements and limb contractions) or tonic events (e.g., loss of tension in antigravity muscles) from the EMG data, both of which may indicate REM sleep. The detection of phase events or tonic events can be compared with or combined with neural signal data to distinguish REM sleep from waking sleep or other sleep states.

[0210] The exemplary embodiments are provided to introduce the reader to the general subject matter discussed herein and are not intended to limit the scope of the disclosed concepts. The following sections describe various additional features and examples with reference to the drawings, where similar reference numerals indicate similar elements and direction descriptions are used to illustrate exemplary embodiments, but like the exemplary embodiments, they should not be used to limit this disclosure.

[0211] Figure 27 is a block diagram of an example of a system 2700 for acquiring physiological data according to one embodiment of the present disclosure. The system 2700 may include a multi-electrode device 2704 which may have one or more active electrodes 2706a for collecting active signals and one or more reference electrodes 2706b for collecting corresponding reference signals. Furthermore, the multi-electrode device 2704 may include a ground electrode 2706c. The electrodes 2706a-c may be fixed in position within the device (e.g., patch 2702) or movable (e.g., tethered to the device). The system 2700 may further include a processing subsystem 2716, a storage subsystem 2718, an RF transceiver 2714, a connector interface 2712, a power subsystem 2708, and an environmental sensor 2720, each of which may be communicatively coupled to or to part thereof the multi-electrode device 2704.

[0212] The processing subsystem 2716 can be implemented as one or more integrated circuits, for example, one or more single-core or multi-core microprocessors or microcontrollers, examples of which are known in the art. The processing subsystem 2716 can control the operation of the multi-electrode device 2704 by executing various programs according to the program code, and may maintain multiple concurrently running programs or processes. For example, the processing subsystem 2716 may execute code that can control the collection, analysis, application and / or transmission of physiological data (e.g., electroencephalogram (EEG) data, electromyography (EMG) data, etc.). Some or all of the program code can be stored in the processing subsystem 2716, or the program code can be stored in a storage medium such as the storage subsystem 2718. Furthermore, the processing subsystem 2716 may amplify, filter, or combine the signals detected by electrodes 2706a-c of the multi-electrode device 2704, and further store the signals along with recording details (e.g., recording time or user identifier). In some embodiments, the processing subsystem 2716 can analyze physiological data or signals to detect physiological correspondences. For example, the recorded signals can reveal frequency characteristics corresponding to sleep stages.

[0213] Furthermore, the storage subsystem 2718 can be implemented using, for example, magnetic storage media, flash memory, other semiconductor memory (e.g., DRAM, SRAM), or any other non-temporary storage media, or a combination of media, and may include volatile and / or non-volatile media. In some embodiments, the storage subsystem 2718 can store physiological data, information about a subject (e.g., identification information or medical history information), or analytical variables derived from physiological data (e.g., frequency, amplitude, etc.). The storage subsystem 2718 can also store one or more programs that can be executed by the processing subsystem 2716. One or more programs may initiate or control the collection, analysis, or transmission of physiological data.

[0214] The RF transceiver 2714 enables the multi-electrode device 2704 to communicate wirelessly with various interface devices such as telephones, tablets, and laptops. The RF transceiver 2714 may include a combination of hardware components, such as driver circuits, antennas, modulators, demodulators, encoders, decoders, and other suitable analog and / or digital signal processing circuits, and may also include software components. Various wireless communication protocols can be implemented via the RF transceiver 2714 using the hardware associated with the software components. The RF transceiver components of the RF transceiver 2714 may include antennas and support circuits that enable data communication over wireless media such as Wi-Fi, Bluetooth®, or other suitable media for wireless data communication.

[0215] The connector interface 2712 allows the multi-electrode device 2704 to communicate with various interface devices via a wired communication path, for example, using a Universal Serial Bus (USB), a Universal Asynchronous Receiver / Transmitter (UART), or other protocols for wired data communication. In some embodiments, the connector interface 2712 may provide a power port that allows the multi-electrode device 2704 to receive power. The connector interface 2712 may also provide a connection for transmitting or receiving physiological data. For example, physiological data may be transmitted in analog or digital format to or from other devices, such as another multi-electrode device.

[0216] The environmental sensor 2720 may include a variety of electronic, mechanical, electromechanical, optical, or other devices that provide information about external conditions surrounding the multi-electrode device 2704 or about the subject. Any type and combination of environmental sensors 2720 may be used. For example, an accelerometer can be used to estimate whether the user is sleeping or about to sleep, or to estimate their activity level. In another embodiment, an electrooculogram sensor is used to detect eye movements to assist in identifying the Rapid Eye Movement (REM) sleep stage.

[0217] Furthermore, the power subsystem 2708 can provide power and power management functions to the multi-electrode device 2704. For example, the power subsystem 2708 may include a battery 2710 and associated circuitry that distributes power from the battery 440 to other components of the system 2700 that may require power.

[0218] System 2700 is illustrative and should be understood to be modifiable and modifiable. For example, the processing subsystem 2716 may execute code from the memory subsystem 2718 to analyze sleep states based on EEG data and predict a risk level index based on that analysis, where the risk level index may be the likelihood that the subject has TBI. Thus, System 2700 may further include a user interface that allows a user to interact directly with System 2700, for example, to receive the risk level index. The risk level index may be displayed in the user interface as a percentage or in another suitable format. For example, the risk level index may be output as a color corresponding to the severity (i.e., likelihood) of the risk. Thus, for example, the severity of the risk may be high, medium, or low, and the corresponding color output may be red, yellow, or green. Furthermore, although System 2700 is described with reference to specific blocks, it should be understood that these blocks are defined for the convenience of explanation and do not imply a specific physical arrangement of components.

[0219] Figure 28 shows an example of a graph 2800 predicting stage 2 sleep according to one embodiment of the present disclosure. Graph 2800 may include typical EEG signals of a subject predicted to be in a stage 2 sleep state. Graph 2800 may include the amplitude of the EEG signal in microvolts 2802 on the y-axis and the time in seconds 2808 on the x-axis. Thus, graph 2800 may be a visual representation of the electrical activity of a portion of the subject's brain over a 30-second time frame during a stage 2 sleep state. Graph 2800 can be predicted to represent a stage 2 sleep stage based on the presence of sleep spindles (e.g., sleep spindles 2804) and K-complex waves (e.g., K-complex waves 2806).

[0220] K-complexes and sleep spindles can occur in any non-REM sleep stage (i.e., stage 1, stage 2, and stage 3), but are most common in stage 2. For example, during stage 2 sleep, there may be 1 to 3 K-complexes per minute, and each K-complex may be associated with a preceding sleep spindle. Both K-complexes and sleep spindles tend to have a duration of 0.5 to 2 seconds. Furthermore, as illustrated, a K-complex may have a first positive voltage peak, followed by a large negative complex wave, and finally a second positive voltage peak. K-complexes can be defined as biphasic waves with a low frequency band (e.g., a frequency band ranging from 1 to 4 Hz). In contrast, sleep spindles such as 2804 can be defined as short, intense bursts of high-frequency (e.g., 11 to 15 Hz) activity.

[0221] In some embodiments, predicting stage 2 sleep based on the EEG signal shown in Graph 2800 may involve implementing a detection algorithm. The detection algorithm may include deriving feature quantities from the EEG signal from Graph 2800 and detecting sleep spindles 2804, K-compound waves 2806, or combinations thereof based on these feature quantities. For example, the EEG signal can be segmented using a sliding window of the first time quantity (e.g., 0.25, 0.5, or 1 second) with overlaps of the second time quantity (e.g., 0.1, 0.4, or 0.6 seconds). Time-frequency information for each segment of the EEG signal can then be obtained by applying a Short-Time Fourier Transform (STFT) or another suitable mathematical method. Furthermore, feature quantities (e.g., energy, power, etc.) can be derived based on the time-frequency information for each segment using fractional dimension (FD) techniques or another suitable technique. Finally, a classification algorithm or another suitable type of machine learning algorithm can be trained to classify the segments based on features, for example, as sleep spindles, K-complexes, or neither. In this way, portions of the EEG signal associated with sleep spindles and K-complexes can be detected. Detection of sleep spindles and K-complexes may indicate that the EEG signal is associated with stage 2 sleep.

[0222] Figure 29 is a block diagram of an example of a system 2900 according to one embodiment of the present disclosure, which predicts the presence of traumatic brain injury (TBI) based on indicators associated with sleep state. System 2900 may include a computing device 2901 that can be communicatively coupled to a display device 2904 and a multi-electrode device 2906. The computing device 2901 may communicate with the display device 2904 and the multi-electrode device 2906 via a network 2930 such as a local area network (LAN) or the internet. Furthermore, system 2900 can collect physiological data via the multi-electrode device 2906. The multi-electrode device 2906 may correspond to the multi-electrode device 2704 in Figure 27. Physiological data may include neural signal data2908 (i.e., electroencephalogram (EEG) data), electromyogram (EMG) data, electrooculogram (ECG) data, electrooculogram (EOG) data, or other appropriate physiological data.

[0223] In some embodiments, the computing device 2901 can access physiological data. For example, the computing device 2901 may access neural signal data 2908 collected via a multi-electrode device 2906. The neural signal data 2908 may show electrical activity 2922 from a portion of the subject's brain over any number of sleep durations.

[0224] In a particular embodiment, neural signal data 2908 may show electrical activity 2922 from a portion of the subject's brain over a sleep duration 2912, which may be a 20-minute segment of a night's sleep. The sleep duration 2912 can be further divided by a computing device 2901 into time segments 2914a to 2914d (i.e., epochs). Each of the time segments 2914a to 2914d may be of a predetermined length (e.g., 1, 5, or 10 minutes), and the computing device 2901 can predict segment-specific indices 2916a to 2916b for each of the time segments 2914a to 2914d. To support the prediction of segment-specific indices 2916a to 2916d, the detection algorithm can be configured in the time domain, frequency domain, or time-frequency domain to derive features (e.g., frequency band, amplitude, intensity, time duration, etc.) of the neural signal data 2908 for each of the time segments 2914a to 2914d. Then, these features can be used to predict the segment-specific indices 2916a to 2916d.

[0225] In some embodiments, segment-specific indices 2916a–2916d may be the predicted probabilities of specific sleep stages, i.e., time segments 2914a–2914d. In certain embodiments, the first segment-specific indice 2916a may have a 90% likelihood that the first time segment 2914a represents stage 2 sleep. The second segment-specific indice 2916b may have an 85% likelihood that the second time segment 2914b represents stage 2 sleep. The third segment-specific indice 2916c may have a 50% likelihood that the third time segment 2914c represents stage 2 sleep. Finally, the fourth segment-specific indice 2916d may have a 20% likelihood that the fourth time segment 2914d represents stage 2 sleep.

[0226] In some embodiments, the third time segment 2914c may be further analyzed in smaller time segments to predict whether any portion of the third time segment 2914c is associated with Stage 2. Furthermore, in some embodiments, the sleep duration 2912 may be one of many sleep durations, spanning one or more sleep cycles of a subject, or one or more nights of sleep. The sleep duration can be of any length and can be divided into any number of time segments.

[0227] Furthermore, the computing device 2901 can generate a cumulative index 2902 based on segment-specific indices 2916a to 2916b. For example, the cumulative index 2902 can be generated by summing the time segments associated with the predicted probability of stage 2 sleep exceeding a threshold. In a particular embodiment, the estimated absolute time of stage 2 sleep may be determined based on segment-specific indices 2916a to 2916d of the sleep duration 2912. The cumulative index 2902 may then be generated by summing the estimated absolute time for the sleep duration 2912 with the additional estimated absolute time of stage 2 sleep determined for an additional sleep duration. The sleep duration 2912 and the additional sleep duration may extend over the subject's entire night of sleep. Thus, the cumulative index 2902 may be the estimated absolute time over which the subject is predicted to have been in stage 2 sleep throughout the night. For example, the sleep duration may be 6 hours, and the cumulative index 2902 may be 90 minutes. In another embodiment, the cumulative index 2902 may be converted to relative time. Therefore, the cumulative index of 2902 can be considered as 25%.

[0228] The computing device 2901 can further generate a risk level index 2918 based on the cumulative index 2902. The risk level index 2918 may be the likelihood that the subject has TBI. In some embodiments, artificial intelligence techniques can be implemented to generate the risk level index 2918. The artificial intelligence techniques may include the use of one or more models or rules to generate subject-specific predictions based on the cumulative index 2902. In some embodiments, the presence of TBI may be associated with a decrease in stage 2 sleep, an increase in stage 3 sleep, a combination thereof, or other appropriate changes in a typical sleep pattern. Thus, rules may be defined to predict stage 2 sleep deficiency, excess stage 3 sleep, and / or TBI likelihood based on the cumulative index 2902.

[0229] In some embodiments, the computing device 2901 may train a machine learning algorithm to predict a risk level index 2918 by inputting historical neural signal data, along with instructions on whether the data is associated with a healthy subject or a subject suffering from TBI. For example, the historical neural signal data may include previous neural signal data associated with the subject's sleep, neural signal data collected from a healthy population, neural signal data collected from a population of subjects diagnosed with TBI, or another appropriate population from which the neural signal data can be analyzed and compared to the subject's neural signal data 2908.

[0230] Furthermore, in some embodiments, the threshold time quantity or other appropriate value associated with Stage 2 sleep may be predefined based on age group or another appropriate feature of the subject. The threshold time quantity for Stage 2 sleep may be a gradient, with each of the multiple threshold quantities for Stage 2 sleep corresponding to a different risk level. Moreover, machine learning algorithms or other appropriate AI techniques may be implemented to predict threshold quantities based on the sleep data of subjects of a certain age group, and can also be used to predict the risk level index corresponding to the threshold quantity. For example, for the age group of 30 to 50 years, the threshold quantities for Stage 2 sleep may be relative times of 40%, 30%, and 20%. The relative times may correspond to risk level indexes of approximately 70%, 80%, and 90%. Thus, in a particular embodiment, if the subject is 35 years old and the cumulative index 2902 for Stage 2 sleep over a sleep duration period 2912 is 25%, the computing device 2801 may predict an 80% risk level index. Therefore, the likelihood that the subject has traumatic brain injury may be 80%.

[0231] In response to the generation of the cumulative index 2902 and / or the risk level index 2918, the computing device 2901 may output result 2924 to the display device 2904. Result 2924 may be the value of the risk level index 2918, or it may represent the risk level index 2918. In some embodiments, result 2924 may be output to the display device 2904 in response to the computing device 2901 determining that an alert condition 2926 has been met. For example, the alert condition 2926 may be a threshold likelihood. Therefore, result 2924 may be output in response to the risk level index 2918 exceeding the threshold likelihood. Furthermore, outputting result 2924 may include sending an alert communication 2928 to a third-party system associated with the monitoring of the subject.

[0232] In this way, the detection and diagnosis of TBI can be improved. In particular, the accuracy of diagnosing TBI can be improved by predicting indicators associated with sleep stages and predicting a risk level indicator 2918 based on the accumulation of these indicators. Furthermore, monitoring subjects to generate the risk level indicator 2918 and outputting the result 2924 can increase the efficiency of diagnosis, thereby facilitating the efficient treatment of TBI.

[0233] Figure 30 is a block diagram of an example of a computing system 3000 according to one embodiment of the present disclosure, which predicts the presence of traumatic brain injury (TBI) based on indicators associated with sleep state. The computing system 3000 includes a processor 3003 that is communicatively coupled to a memory device 3005. In some embodiments, the processor 3003 and the memory device 3005 may be part of the same computing device, such as a server 3010. In other embodiments, the processor 3003 and the memory device 3005 may be distributed from each other (e.g., located at different locations).

[0234] The processor 3003 may include a single processor or multiple processors. Non-limiting examples of the processor 3003 include a Field-Programmable Gate Array (FPGA), an Application-Specific Integrated Circuit (ASIC), or a microprocessor. The processor 3003 can perform processing by executing instructions 3007 stored in the memory device 3005. Instructions 3007 may include processor-specific instructions generated by a compiler or interpreter from code written in any suitable computer programming language such as C, C++, C#, Java, or Python.

[0235] The memory device 3005 may include a single memory or multiple memories. The memory device 3005 may be volatile or non-volatile. Non-volatile memory includes any type of memory that retains stored information when the power is turned off. Examples of the memory device 3005 include electrically erasable and programmable read-only memory (EEPROM) or flash memory. At least a portion of the memory device 3005 may include a non-temporary computer-readable medium from which the processor 3003 can read instructions 3007. The non-temporary computer-readable medium may include electronic, optical, magnetic, or other storage devices that can provide the processor 3003 with computer-readable instructions or other program code. Examples of non-temporary computer-readable media may include magnetic disks, memory chips, ROMs, random-access memory (RAM), ASICs, configured processors, and optical storage devices.

[0236] The processor 3003 can perform processing by executing instructions 3007. For example, the processor 3003 can access neural signal data 3008 that shows electrical activity from a portion of the subject's brain over one or more sleep duration periods 3012. The processor 3003 can also predict a segment-specific index 3016 associated with the sleep stage 3020 for each of one or more time segments 3014 within one or more sleep duration periods 3012. The processor 3003 can further generate a cumulative index 3002 based on the segment-specific index 3016. The cumulative index 3002 corresponds to the estimated absolute or relative time the subject was in stage 2 sleep state 3006. Furthermore, the processor 3003 can generate a subject risk level index 3018 based on the cumulative index 3002. The risk level index 3018 may represent the likelihood that the subject has TBI 3022. Furthermore, the processor 3003 can output a result 3024 based on or representing the cumulative index 3002. For example, the processor 3003 can output the result 3024 to the display device 3004.

[0237] Figure 31 is a flowchart of a process 3100 for predicting the presence of traumatic brain injury based on sleep state-related indicators, according to one embodiment of the present disclosure. In some embodiments, the processor 3003 can implement some or all of the steps shown in Figure 31. Other embodiments may include more steps, fewer steps, different steps, or steps in a different order than those shown in Figure 31. The steps in Figure 31 are discussed below with reference to the components described above in relation to Figures 29 and 30.

[0238] In block 3102, the processor 3003 can access neural signal data 2908 that shows electrical activity 2922 from a portion of the subject's brain over one or more sleep duration periods 2912. The neural signal data 2908 may be received from or accessed by a multi-electrode device 2906. The neural signal data 2908 may be electroencephalography (EEG) data. Furthermore, the sleep duration period 2912 may correspond to one night of sleep (e.g., 6 hours of sleep), multiple nights of sleep, or a portion of one night of sleep (e.g., a sleep cycle).

[0239] In block 3104, the processor 3003 can predict segment-specific indices 2916a to 2916d associated with a subject's sleep stage for each of one or more time segments 2914a to 2914d within one or more sleep duration periods 2912. The sleep stages may be Stage 1, Stage 2, Stage 3, or REM. To predict the segment-specific indices 2916a to 2916d, the processor 3003 may perform at least one Fourier transform on the neural signal data 2908 for the time segments 2914a to 2914d. In this way, the neural signal data 2908 can be analyzed in the frequency domain to determine whether the frequency band, amplitude, or other appropriate frequency domain features of the neural signal data 2908 for each of the time segments 2914a to 2914d match a particular sleep stage. Thus, in some embodiments, the segment-specific indices 2916a to 2916d can identify whether a time segment is associated with a particular sleep stage. For example, segment-specific indices 2916a-2916d could represent the predicted probability that time segments 2914a-2914d are associated with, for example, stage 3 sleep.

[0240] In block 3106, the processor 3003 can generate a cumulative index 2902 based on segment-specific indices 2916a to 2916d. For example, the cumulative index 2902 can be generated by summing a subset of time segments 2914a to 2914d identified by segment-specific indices 2916a to 2916d as stage 3 sleep. In particular, if segment-specific indices 2916a to 2916d are predictive probabilities, the cumulative index 2902 can be generated by summing a subset of time segments 2914a to 2914d that have predictive probabilities above a probability threshold. Thus, the cumulative index 2902 may be an estimated absolute time (i.e., 90 minutes, 120 minutes, etc.) or relative time (40%, 45%, etc.) during which the subject is estimated to have been in a stage 3 sleep state for, for example, a sleep duration of 2912.

[0241] In block 3108, processor 3003 can generate a subject risk level index 2918 based on a cumulative index. The risk level index 2918 may represent the likelihood that the subject experienced TBI. In some embodiments, artificial intelligence techniques can be implemented to generate the risk level index 2918. The artificial intelligence techniques may include the use of models or rules to generate subject-specific predictions based on the cumulative index 2902. In some embodiments, the presence of TBI may be associated with a decrease in stage 2 sleep, an increase in stage 3 sleep, a combination thereof, or other appropriate changes in a typical sleep pattern. Thus, rules may be defined to predict stage 2 sleep deficiency, excessive stage 3 sleep, and / or TBI likelihood based on the cumulative index 2902.

[0242] In block 3110, the processor 3003 can output a result based on or representing the cumulative index 2902. Result 2924 may represent the value of the cumulative index 2902, the value of the risk level index 2918, or both the cumulative index 2902 and the risk level index 2918. In some embodiments, the processor 3003 may determine that an alert condition 2926 is met, and accordingly, result 2924 may be output to the display device 2904. For example, the alert condition 2926 may be a threshold, such as the threshold estimated absolute time for stage 3 sleep. Therefore, result 2924 may be output in response to the cumulative index 2902 exceeding the threshold. Furthermore, outputting result 2924 may include sending an alert communication 2928 to a third-party system associated with the monitoring of the subject.

[0243] XI. General Considerations Numerous specific details are described herein in order to provide a complete understanding of the claimed subject matter. However, a person skilled in the art will understand that the claimed subject matter can be put into practice even without these specific details. In other instances, methods, apparatus, or systems that would be known to a person skilled in the art are not described in detail so as not to obscure the claimed subject matter.

[0244] Unless otherwise specified, discussions throughout this specification using terms such as “processing,” “computing,” “calculating,” “determining,” and “identifying” are understood to refer to the operation or processing of computing devices, such as one or more computers or similar electronic computing devices, that process or transform data represented as physical electronic or magnetic quantities within memory, registers, or other information storage devices, transmission devices, or display devices of a computing platform.

[0245] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide a conditioned result for one or more inputs. Suitable computing devices include multipurpose microprocessor-based computer systems that access stored software to program or configure computing systems, ranging from general-purpose computing devices to dedicated computing devices implementing one or more embodiments of the subject herein. The teachings contained herein may be implemented in software used to program or configure computing devices using any suitable programming, scripting, or other type of language or combination of languages.

[0246] Embodiments of the methods disclosed herein may be performed in the processing of such computing devices. The order of the blocks shown in the above embodiments is changeable; for example, the blocks can be rearranged, combined, and / or divided into subblocks. Certain blocks or processes can be executed in parallel.

[0247] The use of “adapted” or “configured” in this specification is meant as an open and inclusive language that does not exclude devices adapted or configured to perform additional tasks or steps. Furthermore, the use of “based on” is meant to be open and inclusive in that a process, step, calculation, or other operation “based” on one or more enumerated conditions or values ​​may actually be based on additional conditions or values ​​beyond those enumerated. The headings, lists, and numbering included herein are for illustrative purposes only and are not intended to limit the scope.

[0248] While the subject matter has been described in detail with respect to its specific embodiments, those skilled in the art will understand that, upon understanding the foregoing, modifications, variations, and equivalents of such embodiments may readily be made. Therefore, it should be understood that this disclosure is presented for illustrative purposes only, not limitation, and does not preclude modifications, variations, and / or additional inclusions of the subject matter that would be readily apparent to those skilled in the art.

Claims

1. Accessing biosignal data collected by a biosignal data acquisition assembly including a housing having one or more electrode clusters, wherein each cluster of the one or more electrode clusters includes at least one active electrode. Based on the aforementioned biosignal data, the identification of a first signal representing a first intention to move a first part of the subject's body, wherein the first signal is generated before a second signal, and the second signal represents a second intention to move a second part of the subject's body. Converting the first signal to identify a first operation performed by a computing device, Outputting a first instruction to perform the first operation, Methods that include...

2. The method according to claim 1, wherein the biosignal data includes electroencephalography (EEG) data, the first signal is generated from the left hemisphere of the subject's brain, and the second signal is generated from the right hemisphere of the brain.

3. The method according to claim 1, wherein the biosignal data includes electromyography (EMG) data, the first portion being the left limb of the subject, and the second portion being the right limb of the subject.

4. The first operation includes performing one or more functions associated with the graphical user interface of the computing device, Moving the cursor displayed on the aforementioned graphical user interface from the first position to the second position. The method according to any one of claims 1 to 3, including

5. The first operation includes performing one or more functions associated with the graphical user interface of the computing device, Entering text on the aforementioned graphical user interface The method according to any one of claims 1 to 3, including

6. The method according to claim 5, further comprising applying one or more machine learning models to the input text to predict additional text to be entered on the graphical user interface.

7. The method according to any one of claims 1 to 3, wherein the first operation includes performing one or more functions associated with the graphical user interface of the computing device, and the first operation includes inputting one or more images or icons on the graphical user interface.

8. The method according to any one of claims 1 to 7, wherein the first operation includes launching an application stored in the computing device or executing one or more commands associated with the application.

9. The first operation is, The selection of a first interface element of an intent communication interface with priority over a second interface element, wherein the first interface element is associated with first interface operation data, and the second interface element is associated with second interface operation data. Identifying a second operation performed by the computing device by accessing the first interface operation data of the selected first interface element, Outputting a second instruction for executing the second described above The method according to any one of claims 1 to 8, including

10. The method according to claim 9, wherein the intent communication interface is a tree including a root interface element connected to the first interface element and the second interface element.

11. Accessing additional biosignal data collected by the biosignal data acquisition assembly at a different time, Based on the additional biosignal data, to identify a third signal representing a third intention to move the second part of the subject's body, wherein the third signal is generated before the fourth signal, and the fourth signal represents a fourth intention to move the first part of the subject's body. Convert the third signal to identify the third operation performed by the computing device, Based on the third operation, the selection of the third interface element of the intent communication interface with respect to the fourth interface element, wherein the third and fourth interface elements are connected to the first interface element, the third interface element is associated with third interface operation data, and the fourth interface element is associated with fourth interface operation data. Identifying a fourth operation performed by the computing device by accessing the third interface operation data of the selected third interface element, Outputting a third instruction to perform the fourth operation described above. The method according to claim 9, further comprising:

12. The method according to claim 11, wherein the fourth operation includes entering one or more alphanumeric characters on the graphical user interface of the computing device.

13. The method according to any one of claims 1 to 12, wherein the computing device is an augmented reality device or a virtual reality device, and the first operation comprises performing one or more operations associated with the augmented reality device or the virtual reality device.

14. The method according to any one of claims 1 to 12, wherein the computing device includes one or more robotic components, and the first operation includes controlling the one or more robotic components.

15. One or more data processors, A non-temporary computer-readable storage medium containing instructions, wherein, when executed on the one or more data processors, the instructions cause the one or more data processors to execute some or all of the methods of one or more described in claims 1 to 14. A system that includes this.

16. A non-temporary computer-readable storage medium for storing computer-executable instructions, wherein, when the instructions are executed by one or more processing devices, the non-temporary computer-readable storage medium causes one or more processing devices to execute some or all of the methods of one or more described in claims 1 to 14.

17. Accessing neural signal data showing electrical activity from a portion of the subject's brain over one or more sleep periods, For each of the one or more time segments within the aforementioned one or more sleep duration periods, predict segment-specific indicators associated with the sleep stage. The method involves generating a cumulative index based on the aforementioned category-specific index, wherein the cumulative index corresponds to the estimated absolute or relative time during which the subject was in stage 2 sleep. To generate a risk level index for the subject based on the cumulative index, wherein the risk level index represents the likelihood that the subject has suffered traumatic brain injury. Outputting results based on or representing the aforementioned cumulative indicators. Computer implementation methods, including those mentioned above.

18. The computer implementation method according to claim 17, wherein predicting the category-specific index includes performing at least one Fourier transform on the neural signal data within the category.

19. The computer implementation method according to claim 17 or 18, further comprising determining that an alert condition has been met based on the cumulative indicator, and outputting the result in response to the determination that the alert condition has been met.

20. The computer implementation method according to any one of claims 17 to 19, wherein outputting the results includes sending an alert communication to a third-party system associated with monitoring the subject.

21. The computer implementation method according to any one of claims 17 to 20, wherein the nerve signal data includes electroencephalogram (EEG) data.

22. The computer implementation method according to any one of claims 17 to 21, wherein the category-specific index identifies a predicted sleep stage.

23. The computer implementation method according to any one of claims 17 to 21, wherein the category-specific index identifies the predicted probability that the subject is in the Stage 2 sleep stage.

24. One or more data processors, A non-temporary computer-readable storage medium containing instructions, wherein, when the instructions are executed on the one or more data processors, the one or more data processors, By accessing neural signal data showing electrical activity from a portion of the subject's brain over one or more sleep periods, For each of the one or more time segments within the aforementioned one or more sleep duration periods, predict a segment-specific index associated with the sleep stage. A cumulative index is generated based on the index specific to the aforementioned category, and the cumulative index corresponds to the estimated absolute or relative time during which the subject was in stage 2 sleep. Based on the cumulative index, a risk level index is generated for the subject, and the risk level index represents the likelihood that the subject has suffered traumatic brain injury. Outputting results based on or representing the cumulative indicator, Non-temporary computer-readable storage medium and A system that includes this.

25. The system according to claim 24, wherein predicting the category-specific index includes performing at least one Fourier transform on the neural signal data within the category.

26. The system according to claim 24 or 25, wherein when the instruction is executed on one or more data processors, the one or more data processors further determine that an alert condition has been met based on the cumulative indicator, and the result is output in accordance with the determination that the alert condition has been met.

27. The system according to any one of claims 24 to 26, wherein outputting the results includes sending an alert communication to a third-party system associated with monitoring the subject.

28. The system according to any one of claims 24 to 27, wherein the nerve signal data includes electroencephalogram (EEG) data.

29. The system according to any one of claims 24 to 28, wherein the category-specific index identifies a predicted sleep stage.

30. The system according to any one of claims 24 to 28, wherein the category-specific index identifies the predicted probability that the subject is in the Stage 2 sleep stage.

31. A computer program product tangibly embodied in a non-temporary machine-readable storage medium, comprising instructions, wherein the instructions are transmitted to one or more data processors. By accessing neural signal data showing electrical activity from a portion of the subject's brain over one or more sleep periods, For each of the one or more time segments within the aforementioned one or more sleep duration periods, predict a segment-specific index associated with the sleep stage. A cumulative index is generated based on the index specific to the aforementioned category, and the cumulative index corresponds to the estimated absolute or relative time during which the subject was in stage 2 sleep. Based on the cumulative index, a risk level index is generated for the subject, and the risk level index represents the likelihood that the subject has suffered traumatic brain injury. Outputs results based on or representing the aforementioned cumulative indicator. A computer program product that is tangibly embodied in a non-temporary, machine-readable storage medium.

32. A computer program product tangibly embodied in a non-temporary machine-readable storage medium according to claim 31, wherein predicting the category-specific index comprises performing at least one Fourier transform on the neural signal data within the category.

33. The computer program product tangibly embodied in a non-temporary machine-readable storage medium according to claim 31 or 32, wherein the instruction causes one or more data processors to further determine that an alert condition has been met based on the cumulative indicator, and the result is output in response to the determination that the alert condition has been met.

34. A computer program product tangibly embodied in a non-temporary machine-readable storage medium according to any one of claims 31 to 33, wherein outputting the results includes sending an alert communication to a third-party system associated with monitoring the subject.

35. The computer program product is tangibly embodied in a non-temporary machine-readable storage medium according to any one of claims 31 to 34, wherein the nerve signal data includes electroencephalogram (EEG) data.

36. The aforementioned category-specific index is a computer program product tangibly embodied in a non-temporary machine-readable storage medium according to any one of claims 31 to 35, which identifies a predicted sleep stage.

37. The category-specific index identifies the predicted probability that the subject is in the Stage 2 sleep stage, and is a computer program product tangibly embodied in a non-temporary machine-readable memory medium according to any one of claims 31 to 35.

Images