Integrated circuit system-on-a-chip for a neural interface

Abstract

In one aspect, a system-on-a-chip (SoC) includes a plurality of programmable channels. The SoC includes an analog front end in communication with the plurality of programmable channels. The SoC includes a processing element array configured to receive output signals from the analog front end. The SoC includes an instruction memory supporting an instruction set architecture for supervising computing task of the processing element array, wherein the instruction memory comprises infinite impulse response instructions, discrete Fourier transform instructions, convolutional layer instructions, and fully connected layer instructions, wherein the processing element array is configured to execute instructions in the instruction memory, which, when executed, cause the processing element array to perform functions of a confusion matrix based teach-student convolutional neural network for low power and low latency, and perform functions of a sparsity controller for lower power.

Description

INTEGRATED CIRCUIT SYSTEM-ON-A-CHIP FOR A NEURAL INTERFACEINVENTORS:Jie Gu Yijie Wei Zhiwei ZhongCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims the benefit of priority under 35 U.S.C. §119 from U.S. Provisional Patent Application Serial No. 63 / 550,308 entitled “Integrated Circuit System-On-Chip for a Neural Interface,” filed on February 6, 2024, the disclosure of which is hereby incorporated by reference in its entirety for all purposes.STATEMENT OF FEDERALLY FUNDED RESEARCH OR SPONSORSHIP

[0002] This invention was made with government support under grant numbers CNS-1816870 and CCF-2208573 awarded by the National Science Foundation. The government has certain rights in the invention.TECHNICAL FIELD

[0003] The present disclosure generally relates to integrated circuit system-on-a-chip, and more specifically relates to integrated circuit system-on-chip for a neural interface.BACKGROUND

[0004] Various industries have shown recent interest in incorporating brain-machine interfaces (BMIs) into consumer and clinical applications such as for virtual reality (VR) systems, mixed reality (MR) systems, consumer wearable devices, and biomedical applications, to name a few. Traditional VR headsets, for example, typically rely on conventional joysticks or basic gestures as user input, but miss a critical dimension, e.g., brain activities. Some existing VR / MR systems areintegrated with electroencephalogram (EEG) channels and typically include a VR headset, a 16 / 32- channel EEG cap, a neural recording analog frontend, and a PC for signal classification. Major drawbacks in such systems include (1) significant cumbersome wearing with poor user appearance, (2) lack of in-situ computing support for low-latency operation, (3) incapability of real-time mind imagery control and feedback based on brain activities, and (4) high power consumption from Al classification.

[0005] The description provided in the background section should not be assumed to be prior art merely because it is mentioned in or associated with the background section. The background section may include information that describes one or more aspects of the subject technology.BRIEF DESCRIPTION OF DRAWINGS

[0006] The disclosure is better understood with reference to the following drawings and description. The elements in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. Moreover, in the figures, like-referenced numerals may designate to corresponding parts throughout the different views.

[0007] FIG. 1 schematically illustrates a VR / MR headset mapped to head positioning of a user, according to certain aspects of the present disclosure.

[0008] FIG. 2 schematically illustrates an EEG channel selection of the VR / MR headset illustrated in FIG. 1 for VR / MR mind imagery and control, according to certain aspects of the present disclosure.

[0009] FIG. 3 illustrates a block diagram schematic of a fully-integrated SoC, according to certain aspects of the present disclosure.

[0010] FIG. 4 is a detailed schematic of a bio-Z measurement circuitry, including a bio-Zamplifier, of an analog core of the fully-integrated SoC illustrated in FIG. 3, according to certain aspects of the present disclosure.

[0011] FIG. 5 is a detailed schematic of a two-stage chopper amplifier of each channel of the 16- channel analog front end (AFE) of the fully-integrated SoC illustrated in FIG. 3, according to certain aspects of the present disclosure.

[0012] FIG. 6 illustrates graphs depicting accuracy improvements based on EEG channel selection, according to certain aspects of the present disclosure.

[0013] FIG. 7 illustrates a schematic depicting an ISA for general-purpose EEG classification, according to certain aspects of the present disclosure.

[0014] FIG. 8 illustrates a schematic depicting an example flow and configuration of CNN model for affection detection, according to certain aspects of the present disclosure.

[0015] FIG. 9 illustrates a schematic depicting a detailed neural processor architecture and configuration, according to certain aspects of the present disclosure.

[0016] FIG. 10 illustrates a schematic depicting exemplarily PE array task modes (e.g., Conv Mode, IIR Mode, and FC / DFT Mode), according to certain aspects of the present disclosure.

[0017] FIG. 11 illustrates a schematic depicting a confusion matrix guided teach-student CNN scheme, according to certain aspects of the present disclosure.

[0018] FIG. 12 illustrates a graph depicting normalized energy / class versus a teacher CNN model, a student CNN model, and a teacher-student CNN model, according to certain aspects of the present disclosure.

[0019] FIG. 13 illustrates a graph depicting accuracy percentage versus a teacher CNN model, a student CNN model, and a teacher- student CNN model, according to certain aspects of the presentdisclosure.

[0020] FIG. 14 illustrates a schematic flow diagram for teacher-student CNN control flow, according to certain aspects of the present disclosure.

[0021] FIG. 15 illustrates a schematic depicting sparsity enhancement techniques, according to certain aspects of the present disclosure.

[0022] FIG. 16 illustrates a schematic depicting on-chip inferencing, according to certain aspects of the present disclosure.

[0023] FIG. 17 illustrates a graph depicting normalized power versus sparsity, according to certain aspects of the present disclosure.

[0024] FIG. 18 illustrates a graph depicting overall accuracy percentage versus SSVEP dataset an motor imagery dataset, according to certain aspects of the present disclosure.

[0025] FIG. 19 illustrates a chart depicting average power breakdown (pW) in affect track, according to certain aspects of the present disclosure.

[0026] FIG. 20 illustrates a graph depicting teacher-student CNN model energy efficiency, according to certain aspects of the present disclosure.

[0027] FIG. 21 illustrates a table depicting measurement results, according to certain aspects of the present disclosure.

[0028] FIG. 22 illustrates a micrograph of the SoC, according to certain aspects of the present disclosure.

[0029] FIG. 23 illustrates an augmented photograph depicting the SoC of an add-on integrated with a headset configured with a subset of T3 / T5 / CZ / T4 / T6 channels of the plurality of programmable EEG channels for mental imagery, according to certain aspects of the presentdisclosure.

[0030] FIG. 24 illustrates an augmented photograph depicting the SoC of the add-on integrated with the headset of FIG. 23, according to certain aspects of the present disclosure.

[0031] FIG. 25 illustrates an augmented photograph depicting the SoC of the add-on integrated with the headset of FIG. 23, according to certain aspects of the present disclosure.

[0032] FIG. 26 illustrates an augmented photograph from a top view depicting the add-on with the SoC of FIG. 23, according to certain aspects of the present disclosure.

[0033] FIG. 27 illustrates an augmented photograph from a side view depicting the add-on with the SoC of FIG. 23, according to certain aspects of the present disclosure.

[0034] FIG. 28 illustrates an augmented photograph depicting the SoC of an add-on integrated with a headset configured with a subset of O1 / O2 / PZ channels of the plurality of programmable EEG channels for steady-state visual evoked potential (SSVEP), according to certain aspects of the present disclosure.

[0035] FIG. 29 an augmented photograph depicting the SoC of the add-on integrated with the headset of FIG. 28, according to certain aspects of the present disclosure.

[0036] FIG. 30 illustrates a graph depicting an SSVEP arrow waveform, according to certain aspects of the present disclosure.

[0037] FIG. 31 illustrates a chart depicting SSVEP control accuracy, according to certain aspects of the present disclosure.

[0038] FIG. 32 illustrates a graph depicting motor imagery control, according to certain aspects of the present disclosure.

[0039] FIG. 33 illustrates a graph depicting affect tracking of video content, according to certainaspects of the present disclosure.

[0040] FIG. 34 illustrates a graph depicting accuracy versus type of video content, according to certain aspects of the present disclosure.

[0041] In one or more implementations, not all of the depicted components in each figure may be required, and one or more implementations may include additional components not shown in a figure. Variations in the arrangement and type of the components may be made without departing from the scope of the subject disclosure. Additional components, different components, or fewer components may be utilized within the scope of the subject disclosure.DETAILED DESCRIPTION

[0042] The detailed description set forth below is intended as a description of various implementations and is not intended to represent the only implementations in which the subject technology may be practiced. As those skilled in the art would realize, the described implementations may be modified in various different ways, all without departing from the scope of the present disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0043] In certain aspects, the disclosed technology provides an integrated circuit (IC) system-on- a-chip (SoC) 10 architecture to perform biological signal sensing and deep learning based classifications. For example, the SoC 10 system supports both biological signal analog sensing, such as, but not limited to, electromyography (EMG), electroencephalogram (EEG), electrocardiogram (ECG), bioimpedance, and other appropriate biological signal analog sensing, and digital signal processing, such as, but not limited to, infinite impulse response (HR) filter, finite impulse response (FIR) filter, deep neural network classifier, and other appropriate digitalsignal processing. The digital signal processing tasks includes, but is not limited to, convolutional neural network (CNN), multilayer perception (MLP), discrete Fourier transform (DFT), HR, FIR, other appropriate tasks. Such tasks, for example, can be integrated on the SoC using highly efficient processing element array (PE-array) 12 based reconfigurable application-specific integrated circuit (ASIC) module. The sensed signals are further digitized and passed into a digital core 14, which is configured to perform, for example, at least filtering, spectrum analysis, and deep learning based classification.

[0044] The SoC 10 is fully integrated including all required analog and digital modules such as, but not limited to, low noise amplifier (LNA), analog-to-digital converter (ADC), digital processor, wireless transmitters, and other appropriate modules. The SoC 10 is configured to allow real-time sensing and classification of biological signals with low power and low latency. For example, the SoC 10 supports a confusion matrix based teacher-student convolutional neural network (CNN) 22 to reduce power consumption and latency. The SoC 10 includes sparsity enhancement technique, via a sparsity controller 24, to improve computing efficiency. The digital core 14 of the SoC 10 includes an instruction set architecture (ISA) 26 that is configured to allow flexible dataflow and programming support of the digital signal processing tasks.

[0045] As a non-limiting example, the SoC 10 can be utilized with a neural interface 28 or brainmachine interface (BMI) in virtual reality (VR), augmented reality (AR), and mixed reality (MR) applications. In such applications, the SoC 10 can be utilized in an add-on 30 for a headset 32, for example, and provides optimal channel selection and channel placement when integrated in the headset 32, for example, a VR headset, an AR headset, or a MR headset. The SoC 10 also provides real-time online support of four applications in VR operations including, but not limited to, (a)using attention and picture imagination for control of menu, e g. pop-up of menu and make selection on menu choices, (b) emotion (also called affect) classification during gaming, video watching, shopping, etc. and use of emotion to control game scene, e.g. game speed, (c) motor imagery where a user thinks their leg or arm movement and thought is classified by EEG signals, and (d) steady state visually evoked potential (SSVEP).

[0046] As another non-limiting example, the SoC 10 can be utilized with a neural interface or BMI in biomedical applications such as, but not limited to, cardiac monitoring and stimulation, EMG based prosthetic limb control, EEG detection or study, implantable biomedical device with realtime classification or control, and other appropriate biomedical applications. For example, the SoC 10 can detect EMG-based motion, motion intent, and pose / gait classification for rehabilitation; detect cardiac disease based on ECG signals; detect brain disease (e.g., seizure detection) based on EEG signals; utilize any analog sensor that requires deep learning digital classifier to, for example, detect human activity classifications, and detect voice or keyword activation via microphone based on inertial measurement unit (IMU) signals.

[0047] In another non-limiting example, the SoC 10 can be utilized with a neural interface or BMI, such as, in wearable devices supporting real user activity or health monitoring.

[0048] Continuing with the exemplarily VR and MR applications, with reference to FIGS. 1 -5, the disclosed technology provides a complete mind imagery system integrated into the headset 32 without extra wearing burden for mind-controlled BMI for VR / MR. The disclosed technology advantageously provides: (1) the SoC 10 chip supporting in-situ tasks of mind imagery control for consumer and clinical systems such as, but not limited to, VR systems, MR systems, AR systems, biomedical systems, and other appropriate systems; (2) seamless integration with the headset 32(e g., VR headsets, AR headsets, and MR headsets) and optimized selection of a plurality of programmable EEG channels 34 to enhance user acceptance and experience; (3) the general- purpose instruction set architecture (ISA) 26 with flexible dataflow, supporting a broad range of mind imagery operations; (4) the novel confusion matrix guided teacher-student CNN 22 scheme for major power saving of Al operations; (5) enhanced sparsity, via the sparsity controller 24, on EEG signals from the plurality of programmable channels 34 for energy reduction. In certain aspects of the disclosed technology, a 65nm SoC chip (e g., the SoC 10) is fabricated with in-situ demonstrations on various mind imagery-based VR controls. The disclosed technology achieves the lowest energy consumption, below 1 pJ / class for the compute-intensive CNN operations, due to the exceptional low power features and system-level optimizations.

[0049] With particular reference to FIGS. 1-2 and 6, EEG channel selection of the plurality of programmable channels 34 and integration is exemplarily illustrated with the headset 32 (e.g., a VR headset), which balances accuracy and user convenience. While 19 channels of the plurality of programmable channels 34 are exemplarily illustrated in FIG. 2, it should be understood that various channel selection combinations can be selectively configured with respect to any number of channels associated with a specific product, such as a particular VR headset brand or other type of product brand. For example, while the example EEG channel selection of the plurality of programmable channels 34 combination in FIG. 2 is exemplarily, it is understood that biological channels, such as EMG and electrooculogram (EoG) channels, may include other channels (32 or more channels, for example), which can be selectively configured according to various aspects of the disclosed technology. The plurality of programmable channels 34 are configurable to receiveEMG signals, EEG signals, ECG signals, bioimpedance signals, and other appropriate biologicalsignals. To support a variety of mind imagery tasks, 8 EEG channels of the plurality of programmable channels 34, T3, T5, 01, 02, T6, T4, PZ, and CZ, are selected and subtly incorporated into the head strap 36 of the headset 32 to maintain user aesthetics. Different mind tasks activate a subset of the eight selected EEG channels of the plurality of programmable channels 34, such as, but not limited to, a mental imagery subset 118 of T3 / T5 / CZ / T4 / T6 for mental imagery, a motor imagery control and affect monitoring subset 120 of T5 / CZ for motor imagery control and affect monitoring, and a steady-state visual evoked potential (SSVEP) subset 122 of O1 / O2 / PZ for SSVEP. The reduction of the plurality of programmable channels 34 from 19 to 8 significantly improves the user experience and usability of the technique. Using a saline solution, each of the eight selected EEG channels of the plurality of programmable channels 34 is in communication with a respective electrode 38 to capture EEG signals via pre-cut holes in the headband or head strap 36 of the headset 32. With particular reference to FIG. 3, a top-level schematic diagram illustrates the fully integrated SoC 10. In certain aspects, the SoC 10 includes an analog front end (AFE) 40 in communication with the plurality of programmable channels 34, which are utilized for signal acquisition and digitalization. In certain aspects, the analog front end (AFE) 40 includes sixteen (16) programmable channels. It should be understood that other numbers of programmable channels are within the scope of the disclosure. Each channel of the AFE 40 includes a two-stage chopper amplifier 44 (e.g., a low noise amplifier (LNA) 46 and a programmable gain amplifier (PGA) 48) with 45-72 dB gain and 0.05-400 Hz bandwidth, a low- pass filter (LPF) 50 with a corner frequency at 60 Hz, and an 8-bit successive-approximationregister (SAR) analog-to-digital converter (ADC) 52 operating from 128 Hz to 10 kHz. The AFE40 is in communication with a bio-Z amplifier 53 configured to measure bio-Z measurements of auser’s heart and blood vessels. The digital core 14 of the SoC 10 for integrated Al operations includes, for example, an 8x10 Processing Element (PE) array (e.g., the PE-array 12), control logic 54, and associated memory banks (e.g., a weight memory 56, an output memory 58, and an input memory 60). The SoC 10 includes an instruction memory 62 with specially developed ISA (e.g., the ISA 26) that provides global control and sequencing to the SoC operation for supporting a range of mind imagery tasks. In certain aspects, the real-time classified brain state and mindcontrol commands are transmitted to the VR headset 32 via an external Bluetooth module via inputs / outputs 63 (see FIG. 26) on the add-on 30 to control the VR environments and operations. FIG. 6 illustrates graphs 16 depicting accuracy improvements based on EEG channel selection.

[0050] A highly flexible computing architecture is required to support a variety of mind imagery tasks. Referring to FIGS. 7-10, the specially developed general-purpose ISA 26 is depicted for global data flow control, model configuration, channel selections, and other appropriate instructions. The SoC 10 includes an ultra-wide ISA command of 128 bits to supervise various computing tasks including, but not limited to, HR filter via HR instructions 64, discrete Fourier transform (DFT) via DFT instructions 66, Convolutional (Conv) layer (CNN operation) via Conv instructions 68, and fully connected (FC) layer with high hardware efficiency via FC instructions 70. To support the ever-changing Al models, the configuration of each sub-task, such as, but not limited to, the number of kernels, number of layers, branch target address (BTA) with weapontarget assignment (WTA) via WTA instructions 74, data movement via data mov instructions 72, and, sparsity settings, and other appropriate sub-tasks, are also integrated into the ISA for efficient scheduling and execution of different tasks. A detailed architecture of the digital neural processor(e.g., the digital core 14) of the SoC 10 is depicted in FIG. 9. A 10 x 8 PE array (e.g., the PE-array12) can be flexibly turned on or gated off by rows or columns. CNN, FC, DFT, and HR filtering operations can be specially performed by reusing the same PE-array (e.g., the PE-array 12) through different data flows with weight-stationary (e.g., the weight memory 56 stationary) for Conv layers and output- station ary (e.g., the output memory 58 stationary) for FC layers and DFT. Instead of the conventional systolic array, which engages significant pipelined flip-flops, the disclosed technology purposely removes most of the pipeline stages for power saving while still meeting the classification latency target at 5~10ms. Low power features, such as, but not limited to, sparsity enhancement, fine clock gating, and teacher-student CNN scheme are also implemented as described next.

[0051] Given the slow pacing of mental states, a teacher-student CNN scheme (via the conv instructions 68) is utilized to strike a balance between sensitivity, computational power, and accuracy, as depicted in FIGS. 11-17. Offline-trained teacher-student models are downloaded into the SoC 10 for brain activity monitoring. The small-size student CNN model 76 is about 3X faster, with 14% lower accuracy but 70% less energy consumption than the teacher CNN model 78. Confusion matrices 80 are used to decide which model (e.g., student CNN model 76 vs. teacher CNN model 78) to be activated for classifications judiciously. Referring to FIGS. 11-14, for the example of affect monitoring, while the initial classification was performed by the teacher CNN model 78 for high accuracy, as user's mental state lasts, the small student CNN model 76 is turned on for power saving. When a state transition is detected, the confusion matrix 80 is checked to evaluate the possibility of true transition or false alarm. A rej ection is issued if the confusion matrix 80 shows a high possibility of false detection. The rejection is followed by the engagement of the teacher CNN model 78 for confirmation. Essentially, the confusion matrix 80 is used to reduce thefalse classification from the student CNN model 76 leading to the overall enhanced accuracy for the student CNN model 76. In certain aspects, 55% energy / class saving can be achieved through the teacher- student CNN scheme with an accuracy drop of only 2.3% in the affect detection case. FIGS. 15-17 illustrate a sparsity enhancement technique 82 where small noisy signals are zero-ed using comparators 84 with a preset threshold. Direct sparse enhancement leads to a significant accuracy drop of over 15%. A special sparsity-aware training process that adds sparsity operation into the training process reduces the accuracy impacts. With the training flow, a total CNN power saving of 12% is achieved with an accuracy impact of up to 4.6%.

[0052] In certain aspects, the SoC 10 is fabricated in a 65nm CMOS with a total area of 7.5 mm2, which supports four (4) mind imagery / affect monitoring and control tasks, including (1) mental imagery-based VR interface control, i.e., users control the GUI operation through the imagination of pictures; (2) real-time affect state tracking and feedback control during VR gaming; (3) motor imagery, i.e., user's imagination of hand or leg motions; (4) SSVEP, i.e. user focuses on pictures flashing with various frequency as input to the system. In the mental imagery control task, users use pre-trained "focus" action to pop up a selection menu and imagine photos. For example, but not limited to, a user can use the pre-trained “focus” action to pop up a selection menu and imagine rotating stars or photos of rainy days or surfing in VR to select the menu items. In-situ measured accuracy showed 79-83% on mental imagery and control with mind commands successfully injected into the VR scenes. In affect tracking and control, a customized Endless Running game is built to dynamically adjust gaming difficulty based on the CNN-classified arousal levels to enhance user engagement, e.g., increasing moving speed when the user's arousal level is low.Measured results with the gamer's affect stages showed an overall accuracy of 93% in tracking thegamer's affect state and successful mind-based pace adjustment.

[0053] FIGS. 18-21 depict charts (e.g., first chart 86, second chart 88, and third chart 90) and a table 92 illustrating measurement results of the SoC 10. As illustrated in the first chart 86 in FIG. 18, public SSVEP and motor imagery datasets are evaluated by on-chip CNN, achieving 86% and 80% accuracy, with 2% drop from a baseline model in the SSVEP dataset. As illustrated in the third chart 90 in FIG. 20, the teacher- student CNN scheme achieves 1.97pJ / class for teacher CNN model 78, 0.6pJ / class for student CNN model 76, and 0.89pJ / class for combined operation. The table 92 in FIG. 21 compares results of the SoC 10 with results of traditional systems focusing on biomedical applications with integrated digital cores. Due to the simplified pipeline design, teacher-student CNN operation, and sparsity enhancement of the disclosed SoC 10, the digital core 14 consumes only sub-lpJ / class, the lowest energy reported among the existing demonstrations of biomedical CNN processors. The disclosed SoC 10 extends brain-interface technology to consumer electronics in the burgeoning VR / MR landscape, fostering the human-machine interfaces in the metaverse, as well as to biomedical systems. A micrograph 94 of the SoC 10 is depicted in FIG. 22.

[0054] FIGS. 23-27 illustrate augmented photographs depicting the SoC 10 of the add-on 30 integrated with the headset 32, according to certain aspects of the present disclosure. The add-on 30 can be powered by a battery 96. In certain aspects, the battery 96 is, but is not limited to, a Li- ion battery. FIGS. 23-25 are augmented photographs depicting the subset of T3 / T5 / CZ / T4 / T6 channels of the plurality of programmable channels 34 for mental imagery.

[0055] FIGS. 28-32 illustrate an SSVEP control scheme and motor imagery control scheme of theSoC 10 of the add-on 30 integrated with the headset 32 illustrated in FIGS. 23-27. FIG. 28 is anaugmented photograph depicting the subset of 01 / 02 / PZ channels of the plurality of programmable channels 34 for steady-state visual evoked potential (SSVEP). FIG. 29 is an augmented photograph depicting the subset of T5 / CZ channels of the plurality of programmable channels 34 for motor imagery control and affect monitoring. FIG. 30 is a graph 98 depicting an SSVEP arrow waveform. FIG. 31 is a chart 110 depicting SSVEP control accuracy. FIG. 32 is a graph 112 depicting motor imagery control.

[0056] FIG. 33 is a graph 114 depicting affect tracking of video content. FIG. 34 is a graph 116 depicting accuracy versus type of video content.

[0057] In one aspect, a method may be an operation, an instruction, or a function and vice versa. In one aspect, a clause or a claim may be amended to include some or all of the words (e.g., instructions, operations, functions, or components) recited in either one or more clauses, one or more words, one or more sentences, one or more phrases, one or more paragraphs, and / or one or more claims.

[0058] To illustrate the interchangeability of hardware and software, items such as the various illustrative blocks, modules, components, methods, operations, instructions, and algorithms have been described generally in terms of their functionality. Whether such functionality is implemented as hardware, software or a combination of hardware and software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application.

[0059] As used herein, the phrase “at least one of’ preceding a series of items, with the terms “and” or “or” to separate any of the items, modifies the list as a whole, rather than each member of the list (e.g., each item). The phrase “at least one of’ does not require selection of at least oneitem; rather, the phrase allows a meaning that includes at least one of any one of the items, and / or at least one of any combination of the items, and / or at least one of each of the items. By way of example, the phrases “at least one of A, B, and C” or “at least one of A, B, or C” each refer to only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

Claims

What is claimed is:

1. A system-on-a-chip, comprising: a plurality of programmable channels; an analog front end in communication with the plurality of programmable channels; a processing element array configured to receive output signals from the analog front end; and an instruction memory supporting an instruction set architecture for supervising computing task of the processing element array, wherein the instruction memory comprises infinite impulse response instructions, discrete Fourier transform instructions, convolutional layer instructions, and fully connected layer instructions, wherein the processing element array is configured to execute instructions in the instruction memory, which, when executed, cause the processing element array to perform functions of a confusion matrix based teach-student convolutional neural network for low power and low latency, and perform functions of a sparsity controller for lower power.

2. The system-on-a-chip of claim 1 , wherein the plurality of programmable channels are configured to receive electroencephalogram signals for supporting mind imagery tasks, wherein the plurality of programmable channels comprise a mental imagery subset for mentalimagery, a motor imagery control and affect monitoring subset for motor imagery control and affect monitoring, and a steady-state visual evoked potential (SSVEP) subset for SSVEP.

3. The system-on-a-chip of claim 2, wherein the plurality of programmable channels comprise eight channels.

4. The system-on-a-chip of claim 1, wherein the analog front end is a sixteen channel analog front end.

5. The system-on-a-chip of claim 4, wherein each channel of the analog front comprises a two-stage chopper amplifier, a low-pass filter, and an 8-bit successive- approximation-register analog-to-digital converter.

6. The system-on-a-chip of claim 5, wherein the two-stage chopper amplifier comprises a low noise amplifier and a programmable gain amplifier.

7. The system-on-a-chip of claim 5, wherein the two-stage chopper amplifier comprises 45-72 dB gain and 0.05-400 Hz bandwidth, the low-pass filter comprises a corner frequency at 60 Hz, and the 8-bit successive-approximation-register analog-to-digital converter operates from 128 Hz to 10 kHz.

8. The system-on-a-chip of claim 1, wherein the processing element array is an 8x10 processing element array.

9. The system-on-a-chip of claim 1, wherein the confusion matrix based teach- student convolutional neural network utilizes confusion matrices decides whether to activate oneof a student convolutional neural network model and a teacher convolutional neural network model.

10. The system-on-a-chip of claim 1, wherein the plurality of programmable channels are configured to receive one of electromyography signals, electrocardiogram signals, and bioimpedance signals.

11. An add-on for a headset, comprising: a system-on-a-chip comprising, a plurality of programmable channels; an analog front end in communication with the plurality of programmable channels; a processing element array configured to receive output signals from the analog front end; and an instruction memory supporting an instruction set architecture for supervising computing task of the processing element array, wherein the instruction memory comprises infinite impulse response instructions, discrete Fourier transform instructions, convolutional layer instructions, and fully connected layer instructions, wherein the processing element array is configured to execute instructions in the instruction memory, which, when executed, cause theprocessing element array to perform functions of a confusion matrix based teach- student convolutional neural network for low power and low latency, and perform functions of a sparsity controller for lower power.

12. The add-on of claim 11, wherein the plurality of programmable channels are configured to receive electroencephalogram signals for supporting mind imagery tasks, wherein the plurality of programmable channels comprise a mental imagery subset for mental imagery, a motor imagery control and affect monitoring subset for motor imagery control and affect monitoring, and a steady-state visual evoked potential (SSVEP) subset for SSVEP.

13. The add-on of claim 12, wherein the plurality of programmable channels comprise eight channels.

14. The add-on of claim 11, wherein the analog front end is a sixteen channel analog front end.

15. The add-on of claim 14, wherein each channel of the analog front comprises a two-stage chopper amplifier, a low-pass filter, and an 8-bit successive-approximation-register analog-to-digital converter.

16. The add-on of claim 15, wherein the two-stage chopper amplifier comprises a low noise amplifier and a programmable gain amplifier.

17. The add-on of claim 15, wherein the two-stage chopper amplifier comprises 45-72 dB gain and 0.05-400 Hz bandwidth, the low-pass filter comprises a corner frequency at 60 Hz, and the 8-bit successive-approximation-register analog-to-digital converter operates from 128 Hz to 10 kHz.

18. The add-on of claim 11, wherein the processing element array is an 8x10 processing element array.

19. The add-on of claim 11, wherein the confusion matrix based teach-student convolutional neural network utilizes confusion matrices decides whether to activate one of a student convolutional neural network model and a teacher convolutional neural network model.

20. The add-on of claim 11 , wherein the plurality of programmable channels are configured to receive one of electromyography signals, electrocardiogram signals, and bioimpedance signals.

Images