Analysis of interictal epileptiform discharge (IED) effects on people with epilepsy

Abstract

Systems and methods are provided for detecting the presence of lEDs and contemporaneously assessing the effect of an IED on behavior and cognition of an epileptic subject. The system includes an electroencephalogram (EEG) acquisition station, as well as a computer with access to one or more processors and one or more memory stores. The memory stores include instructions for execution by the one or more processors for monitoring the electrical activity of a subject's brain using the EEG acquisition station, detecting an IED while the EEG is recorded of the subject, triggering, in response to the detection of an IED, a prompt to be delivered to the subject requiring a response from the subject, and then assessing the response from the subject. Detection of EEDs can be performed by MTF mapping of windows of an EEG into 2-dimensional predicted images for classification as either normal activity or IED event(s).

Description

Attorney Docket No.: 105010-201 SYSTEMS AND METHODS FOR ANALYSIS OF INTERICTAL EPILEPTIFORM DISCHARGE (IED) EFFECTS ON BEHAVIOR AND COGNITION IN PEOPLE WITH EPILEPSY CROSS REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to US Patent Application Serial No. 63 / 647,239,filed May 14, 2024 and U.S. Patent Application Serial No.63 / 805,026, filed May 13, 2025 under relevant portions of 35 USC §119 and 35 USC §120. The entire contents of each noted document is herein incorporated by reference. STATEMENT REGARDING FEDERALLY SPONORED RESEARCH AND DEVELOPMENT

[0002] The project leading to this application has received funding from the European Union’sHorizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 799791. BACKGROUND

[0003] People with epilepsy (PWE) are not only affected by seizures, but also by epileptiformphenomena that occur between seizures, known as interictal epileptiform discharges (IEDs). IEDs are typically not perceived by patients nor recognizable by routine clinical observation, but can have serious consequences given their association with transitory cognitive impairment and their higher prevalence compared to seizures. IEDs can be visualized using an electroencephalogram (EEG), and their effects can be measured using tests. IEDs can occur as single spikes or in series, which are referred to as spike trains or IED-bursts.

[0004] IED(-burst) associated missed responses by patients have been translated to obstaclespresented in a realistic driving simulator by comparing them with drowsiness and alcohol while driving on real roads. Using a very conservative estimate, the frequency rates of IED(-burst) associated missed responses in the simulator were comparable with the odds ratios for a nonfatal crash by self-reported drowsiness and with elevated blood alcohol concentration of 0.035% (0.35 per mile). 1 31345765.1Attorney Docket No.: 105010-201

[0005] Established neuropsychological tests used to assess the general level of cognitivefunction in people with epilepsy are not timed to coincide with IED appearance. Moreover, IEDs in the EEG do not automatically trigger neuropsychological tasks. In addition, many neuropsychological tasks are complex and require some time to be completed. IEDs of short duration and particular location may not increase the error rate, but increase only the time required to complete the task. On the other hand, the effects of IEDs on cognition and behavior, including (visuomotor) reaction speed, have been scientifically described since 1939. Many tests relied on the temporal coincidence of IEDs with the task to measure IED-effects, which greatly reduced the number of measurable outcomes. A few studies triggered a task as soon as an IED became visible to the experimenter, but the stimulus frequently appeared at the end of or even after an IED.

[0006] Thus, an integrated system is needed that not only accurately detects the presence ofepileptiform phenomena (including IEDs), but also contemporaneously and automatically measures the effects of such phenomena on behavior, reactivity, and cognition and makes these measurements available to healthcare professionals for counseling people with epilepsy who are seizure-free at the time of testing. BRIEF DESCRIPTION

[0007] Accordingly, there is herein provided a system and related methods that can detectinterictal phenomena for subjects with epilepsy wherein the detection takes place in real time from an EEG recording. According to at least some embodiments, systems and methods are provided that integrate Artificial Intelligence (AI) and neuropsychological and / or reaction tests into a closed-loop system for detecting and analyzing the effects of epileptic discharges in the EEG, including IEDs, on reactivity and cognition of a subject in real-time. As used herein, epileptic discharges include IEDs and other epileptic episodes. In some embodiments, a Markov Transition Fields (MTF) method, in which time series are mapped onto two dimensional images, is used to classify images, which were transformed from EEG to transition probabilities, using a deep convolutional neural network for purposes of detecting the occurrence of IEDs and then contemporaneously assessing their impact upon PWE. Thus, embodiments of the present 2 31345765.1Attorney Docket No.: 105010-201 technology can assist in recognizing IEDs with high accuracy in real time in order to test their possible consequences on daily social functioning using simple tasks for people with epilepsy.

[0008] In some embodiments, EEG analysis can be combined with a sliding window techniqueand Markov Transition Fields, which apply first order Markov chains and a visualization technique to map EEG window by window onto images and classify them by a residual convolutional neural network. In at least one embodiment, the system can be implemented as a closed-loop system, made up of a commercial EEG acquisition station connected to an external device, such as, for example, a laptop; a Transistor-Transistor-Logic (TTL) chip; and further electronic circuits. For purposes of assessment and upon detection of IEDs, a prompt or prompts are created, eliciting a response from the subject, also in real time and without digital latency of the closed-loop circuitry. In one embodiment, for example, automatic IED-detection triggered obstacles are provided to the subject. In at least one version, the obstacles can be presented in a simulated driving game In at least one other embodiment, triggered neuropsychological tasks are prompted in videos of a question-and-answer game for the subject to test their cognition performance. For each IED-detection and according to at least some embodiments, digital latency of the herein described closed-loop system can be avoided in the driving game, in which responses are measured in millisecond resolution, using a photo-voltaic sensor that detects the prompt (e.g., the on-screen appearance of the obstacle in the driving game) and initiates a timer, such as a stop watch, only when the patient is able to perceive the prompt, and which is stopped by the patient’s response to the prompt. Reaction times (RTs) and missed responses can be effectively and accurately measured.

[0009] As to the detection of IEDs and according to at least one embodiment, an IED detectionalgorithm is utilized that combines the statistical method of Markov chains with a visualization technique of transition probabilities, the so-called Markov Transition Fields, to map a running EEG recording portion by portion onto images and analyze the images with a deep convolutional neural network. The statistical method of Markov chains is well-suited for IED-burst detection because single interictal spikes and IED-bursts are a different stochastic entity than seizures and do not require temporal memory for their classification. Transforming adjacent voltage values of 3 31345765.1Attorney Docket No.: 105010-201 an EEG window into transition probabilities of the voltage values, which are represented as 2D images, is advantageous because deep neural networks are particularly efficient at classifying images. In one embodiment, the algorithm can be trained and optimized using obtained EEG data set from a plurality of subjects and mathematical methods used to optimize the algorithm including a focal, balanced cross entropy loss function, adding an unsupervised pre-training component using daily life images from the public (dataset ImageNet) to the supervised training of the deep neural network with images of transformed EEG, and fine-tuning, which is another transfer learning method.

[0010] According to at least one embodiment, a closed-loop system can include a Transistor-Transistor-Logic (TTL) chip, which gates the continuous EEG classification by the IED detection algorithm and activates downstream circuitry of the system, such as one or more electronic circuits on printed circuit boards (PCBs), only when the algorithm has classified an EEG window as an IED. In one embodiment, the downstream circuitry can be configured to perform all functions to initiate prompts and measure responses provided by the subject, for example, initiating an obstacle in the driving game and measuring IED-associated reaction time prolongations or initiating videos with neuropsychological questions or tasks, each requiring a response from the subject and determining incorrect or missed responses by the subject, as well as synchronizing the triggering of a prompt with the ongoing EEG recording, such as, for example, by means of a square-wave signal in an additional EEG channel. In one embodiment, additional circuitry can be provided and configured to perform all functions to emulate keyboard commands and according to at least one embodiment, activate videos of the assessment with neuropsychological tasks.

[0011] In at least one embodiment, a type of digital latency correction can be provided for thedriving game since only reaction times are measured with millisecond resolution. Digital latencies of electronic devices or circuits cannot be calculated or estimated in advance because they are asymmetrically distributed around a median due to, for example, signal transformation and unpredictable behavior of an operating system. Thus, it would be advantageous to measure digital latencies for each activation of the closed-loop circuity resulting from the classification of an IED by the algorithm. In at least one embodiment, a photo-voltaic sensor can be used and 4 31345765.1Attorney Docket No.: 105010-201 configured in the system to register each appearance of a triggered obstacle and initiate a timer, (such as a stopwatch), which can be provided, for example, in the electronics (on the circuit board) of the driving game. The timer is automatically stopped when the subject provides a response to the prompt (obstacle presented) or by a failure of the subject to properly react. In the latter instance, this can occur, for example, in a driving test if the obstacle is not overcome and the car crashes into the obstacle.

[0012] As noted and according to at least one embodiment, the reaction test can utilize asimplified driving game to determine reaction time by the subject. By using a simplified driving game, it is more likely that attention will not suppress IED prevalence and enough IEDs will occur during a temporally limited EEG recording and their effects on driving performance can be effectively measured and assessed. It will be understood that other forms of test could be utilized in lieu of a driving game.

[0013] Other forms of prompts can be provided to the subject upon IED detection. For example,and in at least one embodiment, a neuropsychological assessment has been designed to ultra- rapidly analyze the effects of brief IEDs on orientation (time, person, place), language comprehension, word recall, word repetition, knowledge of body parts, left-right discrimination, apraxia, memory (e.g., time), number comprehension, and executive functions of the brain of a subject. To this end and in at least one embodiment, videos are played contemporaneously with the IEDs. It will be understood that other forms of prompts in the form of games and / or tests could be envisioned using the inventive concepts that are described herein.

[0014] Advantageously, some embodiments of the present technology include systems thatintegrate IED-burst detection, digital latency correction, and reaction and cognition testing. Some embodiments are standardized, user-friendly, portable, and can assess even subtle effects of IEDs in real time. Some embodiments can also be used in private practice, overcoming physical distances to a tertiary epilepsy center. Patients can benefit from a better understanding of their epilepsy, leading to improved compliance and safety, individualized and appropriate adjustment of their treatment, and more accurate and objective testing to alleviate secondary problems such as depression and unemployment. In the future, the herein described system having an integrated 5 31345765.1Attorney Docket No.: 105010-201 neuropsychological test, as described herein, could help determine whether IEDs contribute to cognitive decline in people with Alzheimer's disease and whether antiseizure medications improve cognition, as the herein described neuropsychological test is designed to detect IED- associated deficits, whereas conventional neuropsychological tests do not have this temporal correlation of their tasks with the IEDs and may therefore be less sensitive.

[0015] Various embodiments of this technology will now be described with reference to theaccompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings show embodiments of the disclosed subject matter for the purpose ofillustrating the technology. However, it should be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, wherein:

[0017] FIGS. 1A – 1C depict schematically the inclusion of Markov Transition Fields (MTFs) ascombined with a residual convolutional neural network in order to classify images generated from an EEG recording in real time for IED detection in accordance with aspects of this disclosure;

[0018] FIG. 2A depicts an example of an architecture of a closed-loop test system that is made inaccordance with aspects of this disclosure;

[0019] FIG. 2B depicts a variation of the closed loop test system of FIG. 2A; and

[0020] FIGS. 3A- 3C depict an example of prompts being conveyed to a subject in the form ofmultiple neuropsychological tasks contemporaneous with the detection of IEDs and in accordance with aspects of this disclosure. DETAILED DESCRIPTION

[0021] In brief, the following relates to a system and related methods for detecting an IED of asubject having epilepsy in which electrical activity of an EEG recording is monitored as a series of voltage values, in which continuous series of the voltage values are extracted in windows from the monitored electrical activity and streamed to a processing device using a customized packet- 6 31345765.1Attorney Docket No.: 105010-201 switching technology. Once all data in a window have arrived at the processing device, all voltage values in a window are arranged into a set of quantiles, wherein for each quantile, the probability that each voltage value will remain in the quantile is calculated, the calculated probabilities are converted into a two-dimensional image and then using a deep convolutional network, the image is classified as either corresponding to normal brain activity or an epileptic discharge ( such as an IED event). The averaging, assessing, and calculating steps noted above are carried out in accordance with embodiments by transforming each window of voltage values using Markov Transition Fields (MTF) into MTF images for subsequent classification. The impact of detected IED(s) by the foregoing system is then assessed on a subject having epilepsy through the use of different prompts that are conveyed to the subject contemporaneously with the detected IEDs.

[0022] According to embodiments, accurately classified IED bursts as detected by the hereindescribed system, which can be trained using AI, can be integrated prompts that are embodied, for example, within a simplified driving test and / or a neuropsychological assessment, respectively. These prompts, as described herein, measure IED-burst induced deficits including missed responses to visual stimuli or to neuropsychological tasks that might be relevant not only in a hypothetical or test situation, but also in real life. Some embodiments of systems of the present technology can be used to test the EEG compatibility with the fitness-to-drive, or to screen for relevant transitory cognitive deficits caused by IED-bursts between seizures in routine EEG, for example, transient impairment of attention, social interaction, or memory recall to improve disease management and educational and professional integration of PWE.

[0023] Markov Transition Fields (MTFs) are used, in some embodiments, as amapping / transformation technique to create images from EEG for classification. MTFs retain much of the information contained in the original EEG, which is advantageous over classical methods of EEG analysis that use compact metrics to predict epileptiform potentials while disregarding much of the EEG information. In contrast to methods that use all (raw) microvolt values of an EEG for feature extraction and classification using deep neural networks, some embodiments of the MTF mapping technique lose some of the EEG information (such as the absolute amplitudes due to the normalization of the transition probabilities), but can also 7 31345765.1Attorney Docket No.: 105010-201 compress large time series and thus advantageously reduce the computational effort compared to raw EEG analysis using deep neural networks. Accordingly, the MTF technique can be used for faster classification with lower energy consumption. A further advantage over raw EEG analysis using deep neural networks is that the MTF mapping technique is a surjective function, which means that the original time series can be reconstructed from the network or the Markov transition matrix without major loss of information. By combining MTF visualization technology with a deep neural network as herein described, EEG analysis is linked to computer vision, combining the best of both worlds as deep neural networks are particularly efficient at classifying images.

[0024] A (first order) Markov chain or Markov process is a stochastic model that describes asequence of possible events in which the probability of each future event depends only on the current state that has been reached. Thus, no additional information from the past is required for a prediction. Markov chains have many applications as statistical models for real-world processes such as predicting stock market prices, currency rates, or in gambling. Markov processes form the basis for general stochastic simulation methods, for example in Markov chain Monte Carlo methods and reinforcement learning. Markov Transition Fields (MTFs) visualize the relationships of the data points of a time series. Data points are first discretized into quantiles based on their microvolt value or amplitude. By discretizing data points, the time series is also compressed, which can be advantageous for large time series and to reduce the computational effort. A Python Package for Time Series Classification using Markov Transition Fields is publicly available: https: / / github.com / johannfaouzi / pyts. The EEG is used herein as a continuous time series of technique to transform the EEGwindow-by- All microvolt values from each EEG window were assigned to a first matrix, namely, a Markov transition matrix. According to this embodiment, assignment was based on the absolute size or amplitude of each microvolt value into 32 evenly spaced quantiles (quantiles ^^^^on the column side, and quantiles ^^^^on therow side (^^, ^^ ∈ ℕ, [1, 32])). The size of the quantiles was recalculated for each window toaccount for EEG amplitude fluctuations from one window to the next. The transition frequencies between the quantiles were obtained by dividing the number of microvolt values in each of the 32 quantiles by the number of microvolt values in each row of the matrix. The transition 8 31345765.1Attorney Docket No.: 105010-201 ^^ −^^ frequencies or weights ^^1^^^^ ^^^^^^^^^^were normalized using the equations ^^0= ^^^^^^^^−^^^^^^^^and∑ ^^^^^^ = 1. The transition probabilities (normalized ^^^^^^) from thematrixassigned to each consecutive pair of the 200 microvolt values from each window, resulting in a Markov Transition Field matrix with size 200 x 200. The transition probability for each pair of the 200 microvolt values is plotted in an image referred to herein as a Markov-Transition-Field image or MTF-image (size 200 x 200 pixels). The brightness of a pixel indicates the (inverse) probability of a transition from quantile (y1,x1) to quantile (y2,x2) (1 and 2 are points in a time series at the time x1 and x2). In other words, pixels of particular color or brightness indicate certain probabilities for pairs of microvolt values to have different amplitudes. An MTF-image is symmetrical along the diagonal and represented the self-transition probability.

[0025] For purposes of this specific embodiment, and using between 200 and 800 microvoltvalues, the sliding window size is set to 200 data points at 256 Hz sampling frequency. The corresponding duration of one window is 0.781 s (200 (^^^^^^^^ ^^^^^^^^^^^^)2561 ^^(^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^) = 0.781 ^^). Thewindow moved forward with the ongoing50 datapoints corresponded to 1 256= 0.195 ^^ at a sampling frequency of 256 Hz. Thus, the step^^size is about 200 ms and each window overlaps with the next window by about 150 datapoints or 600 ms. The reasons for this window size were, first, the minimum duration of an intended target structure, the IED (or IED-burst), which was set at 0.4 s. Epileptiform EEG changes were only marked (by hand or otherwise) if they had this minimum length. This duration corresponded to0.4 ^^ × 2561 ^^= 102 microvolt values, half the number of data points in the window defined bythis embodiment. The second reason was to ensure that the EEG data were transmitted and processed ultra-fast. This included experimenting with the data transfer rate of the EEG communication protocol and taking into account the GPU of the processing device (laptop in this embodiment), which calculated the mathematical operations of the AI. A default MTF library (Python package) color-coded the pixels, but grayscale image arrays can also be used in order to reduce computational effort. It will be understood that the foregoing refers to a specific example and other modifications could be utilized. 9 31345765.1Attorney Docket No.: 105010-201

[0026] Analyzing 200 data points with a duration of 0.781 s at a sampling frequency of 256 Hzenables the analysis of EEG signals with a bandwidth of 0–128 Hz, i.e., up to the Nyquist frequency of 128 Hz (half the sampling frequency). Since IEDs (bursts) can have a much lower bandwidth (for example, 0.5–7 Hz), reducing the sampling frequency to 128 Hz by the AI would still allow the correct representation of EEG signals with a bandwidth of 0–64 Hz and thus the analysis of IEDs (bursts). Only 100 data points could be used while maintaining the window duration of 0.781 s (i.e., representation of normal EEG or an IED (bursts) with fewer data points / signals). This would speed up the process of transforming and classifying EEG windows and make the entire AI faster or compensate for additional calculations, such as the analysis of neighboring EEG channels.

[0027] With reference to Figure 1A, the combination of Markov Transition Fields (MTFs) asdescribed above with a residual convolutional neural network is shown schematically for the purposes of classifying images generated from an EEG recording in real time. In this figure, a section 102 of an EEG recording having multiple channels in longitudinal bipolar montage is shown. A sliding window (shown as a box 104) in channel Fp1-F7 moves forward with theongoing EEG recording. Two enlargements of the sliding window 104 each show a series ofvoltage (microvolt) values, one being for normal brain activity 108 and one for an IED 110, which are streamed in packets as part of an overall EEG-window from an EEG acquisition station 204, FIG.2A, to an external device 210, FIG.2A, such as a laptop (as shown by arrows112, 114). As shown, each sliding window (box 104) contained 200 microvolt values of EEGaccording to this embodiment (at 256 Hz sampling frequency) that were transformed into images using Markov Transition Fields, as previously described, and then classified using a residual convolutional neural network (CNN) 120. The CNN 120 selected for this embodiment is called ResNet 34 and includes 34 convolutional layers. MTF image examples of normal brain activity (FIG.1B) and IEDs (FIG.1C) are shown by way of example.

[0028] The use of first-order Markov chains and a sliding window technique, as herein describedonly for segmenting and streaming EEG, instead of having a second window follow the current window to learn EEG characteristics over time, or using one dimension of a recurrent neuronal network or CNN to share parameters over time, is very well-suited for IED-burst detection. A 10 31345765.1Attorney Docket No.: 105010-201 reason is because single interictal spikes and IED-bursts are a different stochastic entity than seizures and do not require temporal memory for their classification. Therefore, one embodiment of the model did not learn the temporal course of the EEG. This latter embodiment only uses a single EEG channel for purposes of diagnosis. This can be extended in straightforward fashion to all channels by introducing an MTF-image pooling layer before the neural network backbone. MTF-images are images resulting from conversion of the EEG into images using the MTF- visualization technique. Introduction of class weights and focal loss, that is introduction of a focal, balanced cross entropy loss function into the neural network clearly improved EEG classification. While class weights created a balance between the importance (here prevalence) of MTF-images with normal EEG and MTF-images with IEDs, focal loss allowed the model to focus its training on images that are difficult to learn. Since the focal, balanced cross entropy loss function was already implemented when the class ratio could be increased during training from an equal number to five times more MTF-images with normal EEG than MTF-images with IED- bursts, improving the classification performance.

[0029] Summarily, Markov Transition Fields (MTFs) compress and visualize time series asimages. MTFs were used to continuously visualize time series of voltage (microvolt) values from a sliding window that moved with an EEG-recording. More specifically, two-hundred (200) microvolt values in a sliding window were streamed as an EEG-window from an EEG- acquisition PC to an external processing device using a packet-switching technology. These microvolt values were discretized into a predetermined number of quantiles based on their amplitudes. In this embodiment, 32 quantiles were discretized. The quantiles were recalculated for each streamed EEG-packet by dividing the range between the minimum and maximum voltage (microvolt) values by the number of quantiles. Transition probabilities were then calculated from the number of microvolt values in each quantile in comparison to all microvolt values in the EEG-packet. They were normalized and arranged in a Markov Transition Field matrix of size [200x200]. After rearranging the positions of neighboring transition probabilities in the matrix, with the highest probabilities along the diagonal of the matrix, Python or similar software can be used to visualize the transition probabilities as a two-dimensional image, which is either greyscale or color-coded, wherein the created image is saved in a suitable format, such as.png. As noted, these predictive two-dimensional images are referred to as “MTF-images.” 11 31345765.1Attorney Docket No.: 105010-201

[0030] Each MTF-image was not saved by default, but immediately classified by the CNN 120,FIG.1A. In this specific embodiment, a freely available residual convolutional neural network ResNet34 was used, this CNN 120 having 34 neural layers and skip connections that provide deep neural networks with the flexibility to learn representations more efficiently, especially in image classification tasks. For purposes of this embodiment, the ResNet34 output layer with 1000 output units and softmax activation was replaced by a two-unit dense layer with sigmoid activation function for binary classification, that is, classification of MTF-images representing IEDs or normal EEG. Randomly initialized weights can be used to train ResNet34 with EEGs from various sources. To compare the classification performance, a freely available ResNet34, which had been pre-trained in a supervised manner with ImageNet, can be utilized, in which the pre-trained ResNet was subsequently retrained with the EEGs from multiple sources. Further details and examples relating to the coding environment, training, validation, and optimization steps of the neural network and classification of EEG segments from various patient groups are described in greater detail in .U.S. Patent Application No.63 / 647,239, which has been incorporated by reference in its entirety.

[0031] A closed loop system in accordance with an embodiment is herein described withreference to Figure 2A, the system including an EEG acquisition station 204 having a processing device 208. An EEG voltage signal can be recorded using a plurality of scalp electrodes 202 attached by known means to the head of a subject 201, wherein the resulting EEG voltage signal is transmitted via cables to an EEG amplifier 205. There, the EEG voltage signal can be converted into a digital value using a certain number of bits by an Analog to Digital (A / D) Converter (not shown). The signal can then be amplified and transmitted, such as via a Local Area Network (LAN) cable or similar link, to the EEG acquisition PC 208, where the EEG recording can be displayed and recorded. A TCP / IP protocol can be used to transmit EEG in packets to an external device 210, such as a laptop or other suitable processing device, using another LAN cable or similar link. WiFi can alternatively be used for data transmission, but the transmission may be less stable, meaning less data that can be transmitted per second. On the external device 210, the EEG-packets are collected until 200 EEG voltage signals of a window have been received. These are then displayed on the monitor of the external device and analyzed 12 31345765.1Attorney Docket No.: 105010-201 by the IED-detection algorithm. According to this system embodiment, the classification of an IED-burst activates a Transistor-Transistor Logic (TTL) chip 214 via an application programming interface (API), which in turn switches on a printed circuit board 218. According to this specific embodiment, one PCB 218a can be configured to control a driving test, and a second PCB 218b can be configured to control a neuropsychological assessment, the latter being a neuropsychological question-and-answer game developed for integration with the herein described system.

[0032] With reference to FIG. 2B, there is provided a variant of the closed loop system, which issimilar to that shown schematically in FIG.2A. Similar components are labeled with the same reference numerals for the sake of clarity. The system includes an external processing device 210, such as a laptop, a TTL-chip 214 to turn on the PCBs 218a, 218b. Though only PCB 218a is shown in FIG.2A, each of the PCBs include a pair of cables leaving the PCBs 218a, 218b via a jack plug 220. In this instance, the jack plug 220 having two pins on the other end (a 3.5 mm jack plug to two (2) pin touch-proof DIN connector with resistor bridge), is coupled into the EEG amplifier 205, FIG.2A, in order to create a differential channel so that the activation of the PCBs 218a, 218b can be synchronized with the ongoing EEG recording; a push button provided on the keyboard 207 of the external device 210 to terminate reaction time measurement in the driving test; and a photo-voltaic sensor 240 used to correct digital latencies of the reaction time measurements in the driving test, wherein the photo-voltaic sensor 240 can be disposed on the display screen 209 above the black horizontal bar of the external processing device 210.

[0033] The herein described closed loop system is configured according to this embodiment toassess IED-effects on behavior and cognition on a subject with epilepsy. An afferent arm is utilized as an interface that is used to stream the EEG in packets from the EEG acquisition PC 208 to the external processing device 210, which in this instance is a laptop computer. Once the external processing device received enough packets with microvolt values, each window according to this specific embodiment can be suitably filtered with a bandpass between 1 and 30 Hz and a 50 Hz notch filter (60 Hz notch filter in the U.S.), downsampled to 256 Hz (or any other frequency), and classified. The IED-detection algorithm communicates with an application programming interface (API) in the herein described system. In this embodiment, the API used 13 31345765.1Attorney Docket No.: 105010-201 was pylibftdi, an API obtained from a library that came with the transistor-transistor-logic chip(TTL) 214 (Future Technology Devices International), that was installed on the laptop. Morespecifically and when an IED-burst has been classified, the API activates via USB-C to USB-C cable the Transistor-Transistor-Logic (TTL) chip 214. The activated TTL-chip 214 is configured to send a DC signal of 100ms duration that switches on several functions of the downstream printed circuit boards 218a, 218b. In accordance with this particular embodiment, PCB 218a is designed for implementation of a simplified driving test and PCB 218b for the neuropsychological assessment. Each PCB performs several functions, including the synchronization of the activated PCB with the ongoing EEG-recording. PCB 218a additionally records patient responses with a pushbutton on the local hard disk of the external processing device 210 and performs digital latency correction of the reaction time (RT) measurement using the photo-voltaic sensor 240.

[0034] The photovoltaic sensor 240 can be attached to the external device 210 of the hereindescribed closed loop system and more specifically to the display screen. In one example, the photovoltaic sensor 240 can be used to trigger that the prompt has been received by the subject. Using the driving test as an example, a grey road may appear as displayed to the subject. An obstacle can be created by the test (for example, placing a cow in the road in the lane in which the car is moving) having a color contrast that can be sensed by the photovoltaic sensor 240, as a perceived voltage change based on the created light / dark contrast. The moment of this sensor detection was also the time from which the test participant / subject should ordinarily perceive the obstacle, independent of all electronic devices and their digital latencies.

[0035] Prior automated responsiveness tests were either too slow (i.e., the time required for thealgorithm to trigger a task with high sensitivity and specificity after the onset of the seizure was too long), or very prone to artifacts (i.e., they have a low specificity for detecting IEDs, such that the patients or the epilepsy or discharge types must be carefully considered, which precludes the use of these amplitude threshold devices in daily clinical routine), or suffer from other significant drawbacks. Some embodiments of the present technology, use algorithms that use AI / deep learning, to significantly improve the specificity of detecting IEDs with variable appearance and thus accommodate the intra- and inter-individual variability of IEDs. Some embodiments can be 14 31345765.1Attorney Docket No.: 105010-201 considered real-time systems as they are able to classify a running EEG, trigger a task, and record the patient's response.

[0036] Details relating to each of the prompts used in the closed loop system of FIGS. 2A-2B arenow discussed in greater detail. First, a driving test can be used according to at least one embodiment to measure IED-burst associated reaction time (RT)-prolongations and missed responses to a visual stimulus. The driving test can be integrated into the closed-loop system and defines a reaction time from the moment the visual stimulus is perceived by both the photo- voltaic sensor 240 and the subject, wherein the trigger time is the time between the onset of the IED burst and the perception of the visual stimulus by the sensor and the subject. Second, an ultra-rapid neuropsychological assessment can also be integrated, wherein this latter assessment detects transitory cognitive impairment during IEDs of short duration (an example being demonstrated in FIG.3), wherein each of these games / assessments can be integrated into the system together or separately.

[0037] The simplified driving test, which can be written in Java or any other suitableprogramming language, displays a road having a car piloted by the test subject, wherein an excerpt of the test is displayed in the display screen 209, FIG.2B, of the external device 210, FIG.2B. In brief, obstacles are created by the test that would cause the car to crash unless the test subject observes and then avoids the obstacle in time. The application of the driving test in the closed-loop system is herein described. First and when an MTF-image was classified as normal, no obstacle is triggered in the car test. When the system automatically detected an IED- burst, a trigger is sent to cause an obstacle to appear in the same lane of a car to the test subject. As defined by the programming of the driving test, the car would not swerve to avoid the obstacle, but would crash into the obstacle unless the patient switched lanes by means of a push button provided on the external device or other suitable means. The IED-onset to stimulus- perception time is the time from the onset of each IED-burst to the time when the visual stimulus was detected by the photo-voltaic sensor 240 and became perceptible to the patient. This IED- onset to stimulus-perception time is composed of the time required by the integrated test system to sample 200 microvolt values (with a window duration of 781ms at 256 Hz sampling frequency) window by window, which serve as the basis for EEG-classification, waiting for the 15 31345765.1Attorney Docket No.: 105010-201 arrival of 200 microvolt values (≈100ms) while the EEG is continuously streamed in small data packets, transforming an EEG window and classifying the resulting image ≈100ms), and sending a trigger that contains the variable digital latency of the closed-loop system (≈80ms). The first time is a variable of the concept of IED-detection, and the training and optimization of the model embodying the algorithm.

[0038] More specifically and using the closed-loop system, the driving test is integrated to workas follows: As soon as the PCB 218a was activated by a signal from the TTL-chip 214, the PCB 218a triggers an obstacle on the road of the driving test. The photo-voltaic sensor 240 attached to the monitor of the external device 210 registers the obstacle when the obstacle appears on the screen. This initiates a timer (a stopwatch) in the circuitry 218a. The stopwatch is stopped when the patient pressed a pushbutton on the keyboard 207 of the external device 210 (or other suitable response feature) that is connected to PCB 218a. If the patient fails to press the pushbutton, a crash of the car into the obstacle occurs. The PCB 218a dedicated for the driving test, according to this specific embodiment, contains a microcontroller running a suitable operating system, such as Linux. A Java application running on Linux is configured according to this embodiment to automatically calculate the reaction times, registers the missed reactions, that is, the crashes, and save the log file with reaction times and crashes in the root directory of the driving test on the hard disk of the external device. According to this embodiment, PCB 218a has several inputs and outputs. According to this embodiment, one input communicates with the USB-C to TTL-chip 214 using a jack-plug connection. A second input via a jack plug is connected to the pushbutton used by the patient to terminate the reaction time measurement. A third input using a jack plug extends from the photo-voltaic sensor 240, which initiates the timer in the PCB 218a. One DC output of PCB 218a is coupled to the amplifier / headbox of the EEG-acquisition station 205 using a cable with a jack plug on one end and two pins on the other end (3.5mm jack to 2-pin touch- proof DIN connector with resistor bridge). It will be noted that other suitable connection techniques can be utilized. The latter connection synchronizes obstacle triggering and patient response with the ongoing EEG recording according to this embodiment by displaying a rising and falling flank of a square wave signal in an empty channel of the ongoing EEG recording. 16 31345765.1Attorney Docket No.: 105010-201

[0039] Since the reaction time was measured with a resolution of milliseconds, the digitallatency of the closed-loop system had to be corrected. The photo-voltaic sensor 240 detects the change in light-dark contrast caused by the appearance of the visual stimulus on the screen and generates a voltage change. This voltage change is read and processed by an analog-to-digital converter (not shown). When the voltage exceeds a preset threshold, the timer in PCB218a is initiated to measure the time with millisecond resolution and is stopped when the subject presses the pushbutton to respond to the prompt. This process allowed measurement of the so-called “effective reaction” times from the very moment the obstacle could be perceived by the patient and not from the moment the IED-burst was predicted.

[0040] Details are now provided concerning the neuropsychological assessment test of thisembodiment. The printed circuit board 218b for this test contains a microcontroller that was galvanically isolated from the DC input for the signals from the TTL-chip 214, FIG.2A. As on the circuit board 218a of the driving test, an input signal from the TTL-chip 214 is received via a USB-C to jack connection and activates this electronic circuit. The input signal is emulated as a USB keyboard command by the microcontroller and sent to the external processing device (laptop 210, FIG.2A) as a TTL trigger through a USB interface. This activated video playback on the external processing device 210, FIG.2A.

[0040] The activated electronic circuit of the neuropsychological test also sends a signal via a DC output to the amplifier / headbox of the EEG-acquisition station 204, FIG.2A, to synchronize the triggering of a video with the ongoing EEG recording. A cable with a jack connection on one end and two pins on the other end was used for this connection, analogous to that of the driving test, as previously described. The DC signal is displayed as a square wave in an additional channel of the EEG recording, and the rise of the square wave corresponds to the activation of the electronic circuit. It should be noted that the digital latency of the closed-loop circuit was not corrected for this neuropsychological test because the patient responses to the videos that assess neuropsychological (cognition) functions are not measured with millisecond resolution.

[0041] The neuropsychological test according to this embodiment used the freely available PsychoPy software (https: / / www.psychopy.org / ) to implement an ultra-rapid assessment of neuropsychological functionsscript in Python programming language on the external processing device 210, FIG, 2A, initiates the video playback when activated by the 17 31345765.1Attorney Docket No.: 105010-201 TTL-trigger from the circuit board 218b (i.e., by the detection of an IED). The programming further includes a random number generator that arbitrarily picked one of the multiple (in this instance 38) videos that had been assigned a specific number. Only the videos that tested memory were played in a specific order. The videos contain brief instructions on video, acoustic and written information (subtitles) that last up to 5 seconds in order to test orientation (time, person, place), language comprehension, word recall, word repetition, knowledge of body parts, left-right discrimination, apraxia, memory (e.g., time), number comprehension, and executive functions of a subject. The instructions can be recorded in a number of different languages: (e.g., Arabic, Chinese, Croatian, Czech, English, French, German, Greek, Italian, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, and Turkish, among others). These instructions can be recorded using native speakers wherein the audio files can be saved in mp4 format and professionally edited. The test battery can be easily expanded by recording new videos with new tasks and in new languages.

[0042] An example of a neuropsychological test as administered to a subject 300 having epilepsy is depicted in FIGS.3A-3C using the system of FIGS.2A and 2B. The bracket in each panel indicates the start, duration, and end of each IED-burst (see EEG channels C3(-Ref) for Portion A and F3-C3 for Portions 2 and 3) between seizures. Per FIG.3A and once an IED-burst 310 is automatically detected by the system, a video is contemporaneously triggered for display to the patient asking for the name of the subject. This IED does not affect cognition and the patient can respond correctly, as shown. Per FIG.3B, the detected IED-burst 320 is slightly longer than that of the burst detected per FIG.3A, but more heterogeneous than the IED-burst detected in FIG. 3A. Cognition is not impaired, and the patient is responsive to the question posed according to the test, which is the recitation of a number (zero). However and per FIG.3C, the detected IED- burst 330 is over 3 seconds in duration, is not well organized, and contains some sharp potentials. In this case, the patient cannot answer the prompt, which in this case is a neuropsychological task requesting that the subject touch their nose. Additional details relating to each of the foregoing tests and assessment thereof is provided in previously incorporated priority document U.S. Patent Application No.63 / 647,239. 18 31345765.1Attorney Docket No.: 105010-201

[0043] Although the technology has been described and illustrated with respect to exemplary embodiments thereof, it should be understood by those skilled in the art that the foregoing and various other changes, omissions and additions may be made therein and thereto, without departing from the spirit and scope of the present invention. 19 31345765.1

Claims

Attorney Docket No.: 105010-201 CLAIMS 1. A method for detecting epileptic discharges in an electroencephalogram (EEG) of asubject in real time, the method comprising the steps of: monitoring the electrical activity of a subject’s brain as a series of voltage values; extracting, on a continuous basis, a series of voltage values grouped in windows from the monitored electrical activity; and for each window of voltage values: arranging the voltage values into a set of amplitude quantiles; calculating, for each quantile, the probability that each value in each quantile will remain in that quantile; converting the calculated probabilities to an image; and classifying, using a deep convolutional network, the image as corresponding to an epileptic discharge or as corresponding to normal brain activity.

2. The method of claim 1, wherein the image comprises a two-dimensional image in whicha particular color or brightness of a pixel in the image corresponds to a high probability that the next voltage value in time will remain in the amplitude quantile of its predecessor value of an ongoing EEG recording, and a pixel of different color or brightness in the image corresponds to a low probability that the value in time remains in the quantile of its predecessor value.

3. The method of claim 1, wherein the classified image is used for detecting the presence ofa single epileptic discharge or epileptic discharges in series that can be observed in the EEG, which occur between seizures, of a person with epilepsy.

4. The method of claim 1, wherein the arranging, calculating, and converting steps areperformed by mapping each window of voltage values to images using Markov Transition Fields (MTFs). 20 31345765.1Attorney Docket No.: 105010-201 5. The method of claim 1, wherein the deep convolutional network includes skipconnections to prevent overfitting when being trained with images of normal EEG or epileptic discharges.

6. A method for assessing the effect of an interictal epileptiform discharge (IED) onbehavior and cognition of a subject, comprising the steps of: monitoring electrical activity in the subject’s brain; detecting the presence of an IED based on the monitored electrical activity; triggering, in response to an indication of the presence of an IED, a prompt to be delivered to the subject requiring a response from the subject; and assessing the response from the subject.

7. The method of claim 6, wherein the triggering step is performed using a TTL chip thatgates the transmission of the prompt to the subject only upon indication of the presence of an IED.

8. The method of claim 6, wherein the prompt requests that the subject responds either byproviding input to a computer system or by executing or answering a command that is presented contemporaneously with the indicated presence of an IED.

9. The method of claim 6, wherein the monitoring step comprises monitoring an EEG of thesubject.

10. The method of claim 6, wherein the step of detecting the presence of an IED comprises:extracting, on a continuous basis, windows of voltage values from the monitored electrical activity; and for each window of voltage values: assigning the voltage values into a set of amplitude quantiles; 21 31345765.1Attorney Docket No.: 105010-201 calculating, for each quantile, the probability that each value in the quantile will remain in that quantile; converting the calculated probabilities to an image; and classifying, using a deep convolutional network, the image as corresponding to normal brain activity or corresponding to the presence of an IED.

11. The method of claim 9, wherein each detecting step comprises the use of MarkovTransition Fields (MTFs) on a continuous basis on windows of the EEG to generate two- dimensional images.

12. The method of claim 8, wherein the prompt comprises a task to be performed by thesubject for assessment.

13. The method of claim 8, wherein the prompt is in the form of either an obstacle, or a taskfor assessment of cognition, wherein each prompt requires a response.

14. The method of claim 13, wherein the obstacle is in a form of a simplified driving gamethat presents obstacles in order to assess reaction time by the subject.

15. The method of claim 8, wherein the prompt is in the form of a neuropsychologicalassessment presented as a game or test to test cognition of the subject.

16. The method of claim 15, wherein the neuropsychological test or game is presented as oneor more short videos of various situations or queries created by the game or test.

17. The method of claim 8, wherein the step of assessing comprises measuring the timebetween the delivery of a prompt to the subject (digital latency of the system) and the time the subject provides the response (effective reaction time). 22 31345765.1Attorney Docket No.: 105010-201 18. A system for assessing the effect of an IED on behavior and cognition of a subject,comprising: an electroencephalogram (EEG) acquisition station; a computer, with access to one or more processors and one or more memory stores, the memory stores comprising instructions for execution by the one or more processors for: monitoring the electrical activity of a subject’s brain using the EEG acquisition station; and detecting an IED while the EEG is recorded of the subject;’ triggering, in response to the detection of an IED, a prompt to be delivered to the subject requiring a response from the subject; and assessing the response from the subject.

19. The system of claim 18, wherein the detecting comprises:extracting, on a continuous basis, windows containing a series of voltage values from the monitored electrical activity from the EEG acquisition station; and for each window: arranging the voltage values into a set of quantiles; calculating, for each quantile, the probability that each voltage value in each quantile will remain in that quantile; converting the calculated probabilities to an image; and classifying, using a deep convolutional network the image as corresponding to an IED or as corresponding to normal brain activity.

20. The system of claim 18, wherein the detecting comprises mapping or transforming thewindows of voltage values from the EEG recording into images using Markov Transition Fields (MTFs). 23 31345765.1Attorney Docket No.: 105010-201 21. The system of claim 19, wherein the image comprises a two-dimensional image in whicha particular color or brightness of the image corresponds to a high probability that the next voltage value in time will remain in the quantile of its predecessor value of an ongoing EEG recording and a pixel of different color or brightness corresponds to a low probability that the next value in time remains in the quantile of its predecessor value.

22. The system of claim 19, wherein the deep convolutional network includes skipconnections.

23. The system of claim 18, further comprising a photovoltaic sensor arranged to identify theprompt substantially at the moment the prompt is delivered to the subject; and wherein the memory stores further comprising instructions for: activating a timer upon identification of the prompt by the sensor, as delivered to the subject; and stopping the timer upon receipt of the response from the subject to the prompt.

24. The system of claim 23, wherein the memory stores further comprise instructions formeasuring the time elapsed by the timer.

25. The system of claim 18, wherein the prompt is delivered contemporaneously with thedetection of an IED in the form of an obstacle or task.

26. The system of claim 25, wherein the prompt is delivered in form of a simplified drivinggame in which obstacles are presented to determine reaction times of the subject.

27. The system of claim 18, further comprising a TTL chip configured to gate delivery of theprompt only upon the time locked detection of an IED.

28. The system of claim 18, wherein the prompt is delivered to the subject as a series ofneuropsychological tasks, each task requiring a response from the subject. 24 31345765.1Attorney Docket No.: 105010-201 29. The system of claim 28, wherein the neuropsychological tasks are delivered in the formor a test or game configured to assess cognition of the subject. 25 31345765.1

Images