Field-enabled EEG systems, architectures, and methods
Patent Information
- Application Number
- JP2023552374
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-02-28
- Filing Date
- 2022-02-28
- Publication Date
- 2025-04-21
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing EEG systems are not suitable for use outside clinical settings due to their bulkiness, requirement for indoor operation, and inability to provide clinical-grade signal quality while minimizing motion artifacts, making them impractical for remote or field applications.
A portable EEG system with flexible, wireless electrodes that can be easily replaced, integrated into wearable accessories like headbands, and connected to mobile devices for real-time data collection and analysis, capable of synchronizing data from multiple users for group monitoring and analysis.
Enables high-quality EEG data collection in various environments, allowing for real-time monitoring and analysis of individual and group neural activity, facilitating applications in mental health assessment, media evaluation, and operational safety without the need for extensive setup or infrastructure.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] (Related Applications) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 154,751, filed February 28, 2021, which is incorporated by reference in its entirety.
[0002] (Technical field) The present invention relates generally to electroencephalogram ("EEG") systems, architectures, and methods relating to measuring and monitoring of subjects, and more particularly to field-enabled EEG systems, architectures, and methods for measuring and monitoring individuals, such as military, law enforcement, patients, consumers, and the like, outside of typical clinical environments. [Background technology]
[0003] (background) An electroencephalograph is an electrophysiological monitoring device that can record the electrical activity of a subject's brain. Scientists have been recording electrical activity in humans and animals since at least the late 1800s. Electroencephalography ("EEG") typically involves a number of electrodes placed on the subject (usually on the head) to record voltage fluctuations / changes resulting from ionic currents within the firing neurons of the subject's brain.
[0004] It was soon discovered that the brain's voltage fluctuations had many applications. These included using EEG as a diagnostic or clinical tool to diagnose diseases such as epilepsy, sleep disorders, states of consciousness, and even brain death. While advances in medical technology continued, such as the invention of MRI, the EEG's ability to monitor spontaneous changes over time solidified its importance in medicine.
[0005] Electroencephalograph electrodes typically include an adhesive or paste that secures the electrodes to the subject's head. The electrodes are also traditionally attached or bonded to a holder or substrate, such as a headband or head stocking. The electrodes typically include wires that are connected to the electroencephalograph, which detects changes in voltage and prints the results or findings on a screen or paper for analysis by a medical professional. Traditional electrodes also require the use of gel or other media between the subject's head and the electrodes to improve signal transmission.
[0006] An EEG machine typically consists of electronic circuitry, including amplifiers and a controller, to process the electrical signals received by the electrodes. Traditionally, the machine also included an output device, such as an oscillograph or, more recently, a liquid crystal display, to convert the data into a readable form. All of these devices have traditionally been large and heavy, and typically required permanent installation in a room.
[0007] Various attempts have been made to provide EEG systems, architectures, and methods that are comfortable for subjects to wear, and while advances have been made to improve subject comfort, they have not provided the EEG systems, architectures, and methods required in the modern era.
[0008] What is needed and what is provided by the present invention includes having an EEG system, architecture, and method that is easily portable and can be easily used by subjects in large areas or remote locations, and provides clinical grade signal quality with minimal or no motion artifacts. The present invention also provides an EEG system, architecture, and method that has electrodes that are easily replaced or swapped by subjects in remote locations. Another advantage of the present invention is that it can operate within a remote network that collects subject data in real time. Yet another advantage of the present invention is that it can collect individual subject data while the subject is in the field, and transmit, upload, or download the subject's data when the subject is in a secure area or location.
[0009] The above is not intended to limit the scope of the present invention or to describe each embodiment, aspect, implementation, feature, or advantage of the present invention. Detailed techniques and preferred embodiments of the present invention are described in the following paragraphs together with the accompanying drawings to enable those skilled in the art to fully understand the features of the claimed invention. It is understood that the features described in this specification and the features explained below can be used not only in the specific combination, but also in other combinations or separately without departing from the scope of the present invention.
[0010] What is needed is an EEG system that easily allows for synchrony between groups of monitored individuals. Typically, researchers use multiple EEG systems and manually reconcile the data. Alternatively, traditional systems use open source components with IR blasters combined with infrared sensors to synchronize the clocks of multiple devices. Data in these situations is collected wirelessly but may still require manual reconciliation.
[0011] In the past, screenings of movies and other media were held in movie studios or screening rooms, and viewers recorded their reactions at the end of the video / media clip by filling out a survey or adjusting a dial indicating the amount of positive or negative emotion they associated with a particular clip. Later systems used physiological sensors such as heart rate and galvanic skin response. EEG has also been shown to be useful, but most of these systems required large EEG systems or significant user preparation, making them inconvenient for large-scale prescreening studies.
[0012] Additionally, the 2020 COVID-19 pandemic has shifted the distribution of new video and film content from theaters to being viewed for the first time through digital streaming in people's homes. The trend toward in-home streaming of content was already growing before the pandemic and may reflect a permanent shift to in-home streaming as the primary method of receiving video entertainment. The pandemic's social distancing requirements have limited theaters' ability to screen or test movies and media. Furthermore, as the vast majority of video entertainment content is now received in the home through digital streaming, the location of video testing or screening in theaters or other locations represents a departure from the environment in which the average user views entertainment, and may result in deviations. What is needed is an EEG system that can be used remotely to provide users with relevant EEG data that can be used in marketing products and media. Summary of the Invention [Means for solving the problem]
[0013] (overview) One of the many advantages of the present invention is the ability to monitor an individual or a group of individuals, or a subject-user. An example includes monitoring a particular state of mind or body of a group. A variety of states can be monitored, including but not limited to mental and mental health, behavioral and health states. State monitoring is important to be able to monitor how a team performs in various situations and settings, as will be described. The system and method of the present invention can simultaneously monitor, collect and synchronize EEG and / or non-EEG data from a group of individuals. One advantage of the present invention is that the system includes the ability to analyze synchrony of a group of individuals. Synchrony is based on the idea that by looking at the EEG responses of multiple people, users or subjects simultaneously, group dynamics, teamwork and human responses can be better communicated and measured.
[0014] The systems and methods of the invention can use group EEG data to determine and understand the group's response to a stimulus, which could be an image such as a photograph or video, or it could be an event the group is experiencing together, including but not limited to a concert, movie, rally, or an altercation or engagement such as those experienced by law enforcement or the military. The systems and methods of the invention can also use individual user data and group data to monitor group state or group dynamics, such as the degree of teamwork, group and individual fatigue, and level of group aggression. [Brief description of the drawings]
[0015] BRIEF DESCRIPTION OF THE DRAWINGS In the accompanying drawings: [Figure 1A] FIG. 1A is a functional diagram of a field-enabled EEG system according to an embodiment of the present invention. [Figure 1B] FIG. 1B is a functional diagram of a field-enabled EEG system according to an embodiment of the present invention. [Figure 2A] FIG. 2A is a perspective view of a sensor assembly slab of a field-enabled EEG system according to an embodiment of the present invention. [Figure 2B] FIG. 2B is a perspective view showing a sensor assembly divided into individual sensors according to an embodiment of the present invention. [Figure 2C] FIG. 2C is a cross-sectional view of an exemplary sensor according to an embodiment of the present invention. [Figure 2D] FIG. 2D is a top view of an exemplary sensor according to an embodiment of the present invention. [Figure 2E] FIG. 2E is an end view of a sensor having a peelable conductive surface according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a top view of a user's head showing exemplary sensor locations according to an embodiment of the present invention. [Figure 4]FIG. 4 is a perspective view of a sensor coupler or housing according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a side view of an exemplary sensor coupler and sensor according to an embodiment of the present invention. [Figure 6A] FIG. 6A is a top view of an exemplary sensor substrate in accordance with an embodiment of the present invention. [Figure 6B] FIG. 6B is a bottom view of an exemplary sensor substrate in accordance with an embodiment of the present invention. [Figure 6C] FIG. 6C is an end view of an exemplary sensor substrate in accordance with an embodiment of the present invention. [Figure 7A] FIG. 7A is a cross-sectional view of an exemplary sensor according to an embodiment of the present invention. [Figure 7B] FIG. 7B is a cross-sectional view of an exemplary sensor held to a head accessory by a magnet, according to an embodiment of the present invention. [Figure 7C] FIG. 7C is a side view of an exemplary sensor assembly according to an embodiment of the present invention. [Figure 8] FIG. 8 is a top view of one example of an exemplary sensor board with plug-and-play capabilities in accordance with an embodiment of the present invention. [Figure 9] 9A and 9B are exemplary head accessories according to embodiments of the present invention, and FIGs. 9C-9E are exemplary electrode assemblies arranged in a headband along lines 9A-9E and 9A-9E according to embodiments of the present invention. [Figure 10] FIG. 10 is a flow chart illustrating the data selection process of the field-enabled EEG system of the present invention. [Figure 11] FIG. 11 is a front view of a head accessory on a user's head tethered to a smart device according to an embodiment of the present invention. [Figure 12A] FIG. 12A is a functional diagram illustrating an example of media usage of a system according to an embodiment of the present invention. [Figure 12B] FIG. 12B is a flow chart illustrating an example of media usage of the system according to an embodiment of the present invention. [Figure 12C] FIG. 12C is a flowchart showing an example of product placement by the system according to an embodiment of the present invention. [Figure 13A] FIG. 13A is a functional diagram of operation / employee monitoring according to an embodiment of the present invention. [Figure 13B] FIG. 13B is a flowchart of a fatigue score according to an embodiment of the present invention. [Figure 13C] FIG. 13C illustrates a functional diagram and a fatigue diagram of an operator monitoring using a system according to an embodiment of the present invention. [Figure 14] FIG. 14 is a functional diagram of user interface monitoring using a system according to an embodiment of the present invention. [Figure 15] FIG. 15 is a flowchart of user interface monitoring according to an embodiment of the present invention. [Figure 16A] FIG. 16A is a perspective view of a head accessory having a status indicator according to an embodiment of the present invention. [Figure 16B] FIG. 16B is a partial cross-sectional view of the head accessory of FIG. 16A and a remote device according to an embodiment of the present invention. [Figure 16C] FIG. 16C is a schematic diagram of an electrode testing process according to an embodiment of the present invention. [Figure 17A] FIG. 17A is a flowchart of electrode-off detection processing according to an embodiment of the present invention. [Figure 17B] FIG. 17B is a flowchart of an electrode verification process according to an embodiment of the present invention. [Figure 18] FIG. 18 is a perspective view of a head accessory having different sensors, padding, and adjusters according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0016] (Detailed Description) In the following description, the present invention will be described with reference to various exemplary embodiments, which are not intended to limit the present invention to the specific examples, environments, applications, or specific embodiments described herein. Thus, the description of these exemplary embodiments is provided for illustrative purposes only, and not to limit the present invention.
[0017] The sizes and relative proportions of the components are merely exemplary embodiments and may be varied unless specifically limited in the claims, and thus may be varied in size without departing from the scope of the invention.
[0018] The present invention describes devices, systems, and methods for monitoring, analyzing, and reporting neural activity by detecting, collecting, and analyzing electroencephalography ("EEG") measurements from an individual or group of individuals for many purposes discussed herein. The present invention can use the monitored EEG measurements alone or in combination with non-EEG data from other sources, including, but not limited to, anatomical data of the user, such as vital signs (e.g., blood pressure, temperature, pulse rate, respiratory rate, heart rhythm), and anatomical changes (e.g., eye movement, muscle twitches, facial movements, sweating). Other non-EEG data that can be used includes environmental stimuli (such as photographs, movies, commercials or advertisements, concerts, large gatherings, or encounters with police or military forces). The present invention can collect any observable stimulus, combine it with collected EEG data, analyze it, and provide an output that can be used by users, clinicians, marketing companies, businesses with employees, and military or police forces.
[0019] System Architecture In its simplest form, the system 10 of the present invention comprises different components or parts. As illustrated in Figures 1A and 1B, the system 10 comprises at least one EEG electrode support or applicator 12 (e.g., a headband head accessory that can be secured to or around a user's head A). The EEG electrode support 12 can take any form, including, but not limited to, a soft headband, a helmet, or a clip. The EEG electrode support 12 can comprise any material or combination of materials. For example, foam or rubber materials can be used alone or in combination with a generally more rigid shell.
[0020] The EEG electrode support, applicator, or assembly 12 includes one or more sensors or sensor assemblies 14 capable of reading at least EEG signals. The sensors 14 are spaced along an inner surface 16 of the EEG electrode support or applicator 12. One of the sensors 14 can be positioned adjacent to the user's mastoid process, while the other is positioned adjacent to the user's forehead. The placement of the electrodes 14 is limited only by the needs of the portion of the user's brain that needs to be monitored.
[0021] FIG. 3 shows an example of the location of the sensor 14 on the user's head. One or more central electrodes may be placed on the subject's or user's forehead. The central electrode, commonly referred to as Fpz, is connected to and operatively communicates with the active bias of the biosignal amplifier to provide balanced noise rejection on either side of the subject's or user's skull. The mastoid electrodes, commonly identified as A1 or A2, provide a far-field reference for the central forehead electrodes and also allow monitoring of the EEG signal across both sides of the subject's or user's head. This configuration allows for high ease of use, high signal integrity, and enables many EEG applications. Other locations include the temporal (commonly designated T#), occipital (commonly designated O#), and parietal (commonly designated P#).
[0022] The system 10 of the present invention also includes a remote (e.g., mobile) application device 20 that provides an alternative, but important, function to assist in the collection of EEG and / or non-EEG data. The remote application device 20 includes any type of smart device (e.g., smartphone, smartwatch, tablet, etc.). The remote application device 20 includes wireless and wired communication assemblies (e.g., Bluetooth and WIFI) capable of communicating with at least the EEG assembly 12. The remote application device 20 also includes storage capable of storing the EEG and non-EEG data. In an exemplary embodiment of the present invention, the remote application device 20 includes a program or application capable of analyzing the EEG and non-EEG data to determine the psychological and / or medical state of one or more users being monitored. The remote application device 20 can also provide periodic trends over time.
[0023] Continuing with FIG. 1A, an exemplary embodiment of the system 10 includes a data platform 30 (e.g., one or more servers (fixed or cloud-based)) capable of communicating with the remote application device 20 wirelessly or via a physical connection. The data platform 30 comprises at least one storage medium and random access memory ("RAM") capable of receiving, storing, and analyzing EEG and non-EEG data. The data platform 30 comprises a processor capable of performing analysis of the EEG and / or non-EEG data and preparing an output or readout of data points that can be correlated to a user's state or condition or correspond to the state or condition of the user. The data platform 30 includes one or more algorithms and future prediction processes that are HIPPA compliant.
[0024] System 10 can detect a variety of user states or situations, including, but not limited to, a medical state, a mental state (e.g., anxiety or depression), a physical state (e.g., alertness, fatigue, or illness), or an emotional state (happy, sad, joyful, like, or dislike). Although examples of user states are described herein, the invention applies to any user state, and the described user states should not be considered limiting.
[0025] The data points or data readout can be transmitted to a clinician portal 40, such as a hospital workstation, tablet, smart device, etc. Clinician B can review the data points or data readout and determine a course of action or treatment. Clinician portal 40 can store the results of the longitudinal data so that Clinician B can identify trends, triage based on the data points / trends received, and adjust the treatment plan based on the longitudinal data and forecasted trends.
[0026] In other exemplary embodiments of the invention, as described in more detail below, a third party, such as a logistics expert, a coach, a sergeant or general, a police chief, etc., may be able to use the data points or retrieved data to determine the actions of one or more monitored users. For example, the third party may use the data points or retrieved data to strategize the movement of one or more monitored users A, withdraw one or more monitored users A from the scene, etc.
[0027] The EEG head accessory 12 can take any number of configurations. In one exemplary embodiment of the present invention, the EEG head accessory 12 comprises a flexible substrate, such as a headband or stocking, capable of supporting polymer electrodes. The EEG head accessory 12 can be fabricated from a material that mimics the mechanical properties of human skin. The EEG head accessory 12 can also be operatively coupled to or integrated into other devices, such as a helmet. The electrodes 14 of the present invention can also be used separately from the EEG head accessory 12, either placed on the subject or incorporated into other clothing or equipment worn by the subject or user A (e.g., the subject's or user's hat, stocking cap, helmet, headphones, glasses, headscarf, etc.).
[0028] An important aspect of the present invention is a novel sensor or sensor assembly 14. Figures 2A-2C show an exemplary method or process for making / fabricating the sensor 14. As illustrated in Figure 2A, one or more sensors 14 can be fabricated by creating a sensor bed 15 comprising a support layer 16 and a conductive layer 18. The conductive layer 18 is deposited or formed including an EEG conductive material, such as a composition of silver nanowires embedded in polydimethylsiloxane ("PDMS"). A support layer 16 of PDMS is then deposited on the conductive layer 18 to form a large single novel sensor electrode bed 15. This unique configuration and process can be quickly set up for use in nearly any environment (e.g., sitting, walking, running, arguing or fighting), has high signal quality, and includes high motion artifact rejection. The support layer comprises a first surface 27a and a second surface 27b.
[0029] 2B, the large sensor or floor 15 can be divided or separated into individual sensors 14 using any cutting or dividing method known to one of ordinary skill in the art. The sensors 14 can be divided into any size, shape or configuration, depending only on the needs of the clinician, third party, and / or user being monitored.
[0030] 2C, the sensor 14 can be folded over another material, such as a compression member 22 (e.g., foam, rubber, etc.). In this embodiment, the PDMS layer 16 is positioned against the compression member 22 such that the conductive layer 18 faces outward on at least two sides. One of the sides, labeled C, can be positioned against the user's head, and the side labeled D can be positioned against the head accessory 12 that includes any circuitry.
[0031] In another exemplary embodiment, as shown in the top view of Figure 2D, the electrodes 14 are fixedly or removably coupled to the compression member 22, with the electrodes 14 extending beyond the compression member 22 so that they can contact the user's head. The compression member 22 can be coupled (fixedly or removably) to the head accessory 12.
[0032] An advantage of these sensors 14 and their manufacturing methods is that there are generally no wires or connections bonded to the opposing conductive layer 18 and extending through the PDMs layer 16. This greatly improves manufacturing efficiency while reducing areas of potential defects.
[0033] In another exemplary embodiment of the present invention, in addition to or instead of the support layer 16, a circuit board 17 can be adhered or bonded to the head accessory 12 to support the various electrode 14 configurations of the present invention. The circuit board 17 can be generally planar and generally flexible to conform to the head accessory 12. Although the circuit board is described as generally flexible, it can also be generally rigid or a combination thereof.
[0034] 1A, in an exemplary embodiment, the head accessory or assembly 12 includes storage 24 capable of storing local user EEG and non-EEG data. Storage 24 may comprise any known storage device, such as a hard drive, flash drive, RAM, etc. The present invention also includes an EEG head accessory or assembly 12 capable of providing real-time streaming of EEG and / or non-EEG data via Wi-Fi, Bluetooth, or cellular tower 50 (see FIG. 1B). Communication may be direct to the Internet or to a remote application device 20, such as the user's mobile phone.
[0035] In another exemplary embodiment, the support layer 16 may include or be equipped with electronic circuitry for storing EEG or non-EEG data generated by the electrodes 14 or by other sensors (e.g., cameras, thermometers, pulse or heart rate sensors) incorporated either within the support layer 16 or within a shell or covering such as a helmet that can communicate with the EEG head accessory or assembly 12.
[0036] To transmit EEG and / or non-EEG data, the support layer 16 may comprise, incorporate, hold or store a wireless transmitter E capable of wirelessly transmitting data to the remote application device 20, which may then transmit the data or results to a data platform 30, or any other data storage device, generally indicated in the drawings as the letter F. The data platform 30 may comprise a hard drive disk, stick, module or chip operatively coupled to or in operative communication with the support layer 16 and / or the remote application device 20. The data storage device E comprises any electronic circuitry capable of storing data. In this embodiment, the storage device F may be required to be directly connected to the support layer 16 or within a very short distance to the support layer 16 or assembly 12. Distance requirements or limitations may be implemented in the system to reduce the possibility of unauthorized capture of EEG or non-EEG data.
[0037] The system of the present invention may also include an interactive module or application 31, such as a mobile phone software application, to facilitate interaction between user A and the EEG head accessory 12. Through the interactive module or application 31, user A may power on / off the device 20, monitor the control status of each component (e.g., impedance and / or connection of each sensor electrode (lead-off sensing) (described below)), and stream or control the distribution of EEG and / or non-EEG data from the EEG head accessory 12 (e.g., from the headband substrate layer 16 to the mobile device 20).
[0038] Monitor user B may be any person, including, but not limited to, a physician, a strategic analyst, a psychiatrist, a marketing analyst, a clinician, a third party, etc. There may also be multiple monitor users B, each of which may control one or more functions of system 10. For example, subject user A wearing EEG head accessory, equipment, or assembly 12 may be able to turn on or repair one or more components of system 10. Monitor user B may also include a clinician who may receive either raw EEG and / or non-EEG data, or output data points, for purposes of treating subject user A wearing EEG head accessory or equipment 12. Other uses and users are described in more detail below.
[0039] As will become apparent, the present invention can perform real-time data processing of EEG and / or non-EEG data on the interactive module or application 31 or on any device capable of receiving and enabling data analysis and / or control or manipulation. The interactive module or application 31 can be static or dynamic. For example, the interactive module 31 can comprise an application on a smartphone (such as the remote application device 20) if low latency is required. The smartphone or remote application device 20 can then act as a gateway to transmit the EEG or non-EEG data to the data platform 30. The remote application device or smartphone 20 can also provide feedback or notifications to the subject user A or monitor user B on EEG trends or results calculated on the remote application device / smartphone 20 or on the data platform 30. The interactive module or application 31 can alert any EEG monitor parameter or output data point of the subject user A or monitor user B, including but not limited to sleep / wake state, fatigue state, consciousness state, anxiety level, brain injury, balance state, or brain disorder or abnormality.
[0040] As described above, the data platform 30 may be a server-based, secure system for managing EEG or non-EEG data collected from one or more subject-users A, and any determined or formulated data points (collectively, "sensitive data"). The sensitive data may be segregated into one or more accounts for each subject-user A. This improves security of the sensitive data while also ensuring compliance with medical requirements such as HIPAA.
[0041] The EEG data may be collected as raw EEG data stored in the data platform 30. The data platform 30 includes one or more algorithms that allow for many analyses or processing, including time-frequency spectrum analysis, and provides the analysis to the subject-user A or a caregiver. The subject-user A may access the EEG data, non-EEG data, or output via the remote application device 20. Alternatively, or in addition, the EEG data, non-EEG data, or output may be accessed by a caregiver, researcher, analyst, or other third party monitor user B via a third party or provider portal 40. A third party user may obtain and inspect multiple users' EEG data, non-EEG data, and / or output through the provider portal 40, which may comprise a laptop computer, desktop computer, tablet, smartphone, smartwatch, or the like. Additionally, the EEG data, non-EEG data, and / or output may be de-identified and processed with a larger data set, machine learning, or AI algorithms to find alternative patterns or relationships. The EEG data, non-EEG data, and / or output may be synchronized at a later time or in real time using the methods or steps set forth below.
[0042] Shielded Polymer Electrodes and Clamshell Connectors The present invention also has the novel advantage of allowing subject-user A to easily and quickly replace electrodes 14 on EEG head accessory 12. Electrodes 14 of the present invention can be quickly replaced while in the field, allowing monitor user B to continue receiving real-time EEG and non-EEG data from all subject-users A.
[0043] In one exemplary embodiment, multiple alternating layers of PDMS 16+silver nanowires 18 are employed, which are detachable from one another to allow a user in the field to remove the outer layers which may be damaged or contaminated. The different layers may be connected, bonded, or adhered by any fastening means or mechanism. For example, an adhesive may be placed between the alternating electrode layers.
[0044] As illustrated in FIG. 2C, when the electrodes 14 are wrapped around the foam or compressive member 22 of the headband 12, the silver nanowires or conductive layer 18 of the electrodes 14 have at least two surfaces: a contact surface C of the subject user A and a contact surface D of the headgear or headband 12. The contact surface C of the subject user A contacts the skin of the user A to receive the EEG signal. The contact surface D of the headgear 12 can contact a conductive member or strip 32, such as a metal piece connected to or embedded in a portion of the headgear or headband 12. The conductive member or strip 32 can extend around the inner surface of the helmet or other headgear 12. In this manner, the conductive member or strip 32 interconnects at least each electrode 14 of the system 10, each wrapped around the foam or compressive member 22. This eliminates the need for interconnecting wires between the electrodes 14.
[0045] As illustrated in FIG. 2D, in another exemplary embodiment of the present invention, each electrode 14 has its own backing or other support member 23 that provides some structure to the electrode 14. The backing or support member 23 can include any material, such as a compressible material 22, such as a foam or rubber material, of the headband 12, the support layer 16, or the circuit board 17. In some embodiments, the electrodes 14 can be folded around the backing or support member 23 in a manner similar to that described above. In this manner, the backing or support member 23 can be connected to the headband or headgear 12. Any type of connection mechanism or device can be used, such as adhesives, snaps, hook-and-loop fasteners, magnets, etc.
[0046] Electrode Coupler In another exemplary embodiment of the invention, as described above and illustrated in FIG. 2E, the electrode 14 has one or more conductive layers 18a that can be separated by a cover or support 25 to expose new electrode conductive layers 18b, and the ability to replace or repair the system in the field is an important aspect of the invention. Electrodes have a finite lifespan, so in the field, it is necessary for novice users to replace electrodes. Thus, the invention allows novice users to replace defective or fouled electrodes 14.
[0047] In one exemplary embodiment, as illustrated in FIGS. 4 and 5, the system 10 includes one or more electrode couplers or housings 34 coupled or attached to the headband member 12. The electrode couplers 34 may include a clamshell configuration having a lid portion 35a hinged to a base portion 35b. The lid portion 35a is configured to close over the electrode 14 disposed between the lid portion 35a and the base portion 35b. The lid portion 35a may be configured to compress the electrode 14 into an electrical connector, or electrode sensor contact 36, disposed in the base portion 35b of the electrode coupler 34, thereby maintaining high electrical conductivity to the circuit and also providing a secure mechanical fit. FIG. 4 illustrates a diagram of such an electrode coupler or connector 34. As can be seen, this exemplary embodiment includes multiple sensor contacts 36 in one connector 34. The sensor contacts 36 on the base portion 35b of the electrode coupler 34 may be in operative communication with a wire or other conductive material coupled to or disposed on a portion of the headband member or headgear 12. As shown particularly in FIG. 5, a support layer 16 or compressible material 22 is not required but may be included to provide support and comfort.
[0048] As shown in FIG. 5, the sensor contacts 36 of the electrode coupler 34 can be located slightly above the surface of the base portion 35b of the electrode coupler 34. The electrode coupler 34 and the sensor contacts 36 can be fabricated to have a bias or spring action such that the sensor contacts 36 can move at least partially in and out of the base portion 35b of the electrode coupler 34. In one embodiment, at least a portion of the polymer electrode 14 is provided or disposed between the lid portion 35a and the base portion 35b of the electrode coupler 34. The lid or cover 35a closes or latches over at least a portion of the electrode 14. As shown particularly in FIG. 5, application of pressure to the lid or cover 35a compresses the polymer soft electrode 14 and maintains electrical connection with the sensor contacts 36.
[0049] In another exemplary embodiment of the present invention, as illustrated in Figures 6A-6C, a one-piece or generally one-piece electrode coupler 32 is provided that allows for easy replacement of all of the electrodes 14 at the same time. The one-piece electrode coupler 32 includes a generally planar support member that may include a support layer 16, connecting members 23, etc. The electrodes 14 are bonded to or embedded on or within the coupler 32, such that all of the electrodes 14 can be easily replaced by replacing the coupler 32.
[0050] In another exemplary embodiment, the electrode 14 can be coupled, rather than fixed, to the coupler 34. As illustrated in FIG. 6A.1 , the electrode 14 can include a fastener 38, such as a conductive adhesive layer (with or without a support or cover 25) on at least one surface of the electrode 14 to enable it to be removably adhered to a contact sensor 36 of the coupler 32, which can be coupled to a headband member or portion of the headgear 12.
[0051] In another exemplary embodiment, the coupler 32 can contact a contact sensor 36 or other type of conductive member of the headband 12. When the coupler 32 is coupled to the headband member or headgear 12, the conductive members on the electrodes 14 and the sensor contacts 36 can transmit signals through the headgear 12, thereby transmitting EEG signals from the electrodes 14 to the electrode coupler 32. The electrode coupler 32 can include other electronic circuitry that enables the functionality of the headband or head accessory 12. The electrode coupler 32 can be fabricated by lithography or a similar manufacturing process.
[0052] In another embodiment, as illustrated in Figures 7A and 7B, either or both of the electrode 14 and the headband member or headgear 12 have magnet members 41a and 41b that allow for magnetic coupling to the headband member or headgear 12. As with the other embodiments, when the electrode 14 is placed on or magnetically coupled to the headband member or headgear 12, conductive material on or within the electrode 14 couples or mates with sensor contacts 36 of the headband member or headgear 12 or another portion thereof. The magnet members 41a or 41b may be encapsulated or fabricated to include conductive material to act as conductors for EEG and / or non-EEG data. A conductive or non-conductive magnet member 41a or 41b may also be disposed within the folded electrode 14.
[0053] Shielded Polymer Electrodes The system 10 of the present invention also includes a fully shielded electrode 14 for use under certain circumstances. As illustrated in FIG. 7C, the shielded electrode 14 includes a secondary ground layer or member 42 on the backside of the electrode 14. The secondary ground layer or member 42 is configured to shield at least a portion of the electrode 14, or along the entire exterior or outer surface, shielding all of the electrode 14 and traces from external radio frequency ("RF") energy. When connected to ground or an actively driven bias, it provides the highest possible level of shielding for biosensing systems (EEG, ECG, etc.), beyond what is currently available with commercially available electrodes. In one exemplary embodiment, PDMs or other support layers 16 may be disposed between the conductive layer 18 of the electrode 14 and the ground layer 42.
[0054] 6B and 6C show diagrams of a multi-channel electrode 14 of the system 10 with a multi-layer electrode including a shielding layer or member that may include a conductive layer 18 on the back side. As mentioned above, another shielding layer or member 18 may be connected or bonded to the layer of nanowires or the top surface of the conductive layer 18 with a ground connection or via 42 connecting the two conductive layers 18. This configuration resulted in a fully shielded electrode 14. Alternatively, if an electrode coupler 32 is utilized, the top of the electrode coupler 32 may be made conductive and connected to a ground layer or member, eliminating the need for the coupler 32 or simplifying the manufacturing process. Alternatively, the ground layer or member may be covered with PDMS to electrically insulate it from the surroundings. The ground layer may include a layer of the conductive layer 18. Other materials may also be used.
[0055] As illustrated in FIG. 7C, another shield electrode 14 of the present invention is shown. This shield electrode 14 does not include a coupler 32 or a ground connection 42. Instead, a novel electrode coupler 34 with a conductive layer 19 on either the lid or cover portion 35a or the base portion 35b contacts the shield layer or member 18. This configuration minimizes and / or simplifies the manufacturing process of the electrode 14 while providing high electrical conductivity and mechanical robustness within the electrode coupler or connector 34. Alternatively, an electrical sensor contact 36 or a series of electrical sensor contacts 36 can be employed on the lid or cover portion 35a of the coupler or connector 34. The electrical contacts can extend in any direction and have any configuration.
[0056] As illustrated in FIG. 8, the system 10 of the present invention also includes a removable or replaceable plug-in electrode 14. The body of the plug-in electrode 14 can have any of the configurations described above, including having an electrode coupler housing 34 that is generally planar or clam-shaped. The electrode coupler housing 34 is configured to have a connector 56 that includes either a male plug or a female socket. The headband member or headgear accessory 12 can include a corresponding male plug or female socket configured to mate with the male or female portion of the electrode coupler housing 34. The electrode coupler housing 34 can also be shielded or selectively openable and closable. This configuration of the electrode 14 allows for a more rigid, more traditional connector form factor, similar to, for example, Apple's Lightning connector or micro USB connector, to be integrated into the electrode 14 and system 10. This configuration allows user A to repair a defective or worn electrode 14 by simply unplugging the faulty electrode 14 and plugging in a new electrode 14. In this manner, permanent terminations are built into the electrodes 14 and the complete assembly is discarded when replacing the electrodes 14 on the headband or headgear 12.
[0057] It should be noted that the electrode coupler 34 can be configured in a variety of ways, including a flexible tail from the electrode 14 that can be folded back to minimize the height of the electrode 14. Other additional features, such as shielded 3D electrode 14 sensor contacts 36, can also be included in this configuration.
[0058] As illustrated in Figures 9A and 9B, the head accessory 12 generally comprises a soft but protective headband. As particularly illustrated in Figure 9B, the headband 12 may include a padding member 21 disposed on a rear inner surface of the headband 12. The padding member 21 may be disposed on the rear of the head of the user A. An adjuster 23 is operably coupled to the padding member 21 and extends through the headband 12 to allow movement of the padding member 21 relative to the headband 12. In this manner, the user A may adjust the headband 12 to fit his or her head.
[0059] 9C-9E, exemplary electrode 14 embodiments are shown. Each figure is a cross-section taken along the line of FIG. 9B, but showing a different electrode configuration. FIG. 9C shows two separate conductive layers 18 with a PDMs layer 16 (optional) held together by a magnet member 41, all disposed within an opening in the compressible material 22 of the headband 12. The magnet member 41 holds the electrode 14 within the opening, opposite the sensor contact 36 of the connector member 37.
[0060] FIG. 9D shows a similar electrode 14-headband 12 configuration as FIG. 9C, but includes a conductive bridge 33 coupled to each of the conductive layers 18 and extending through the PDMs layer 16. The purpose of the conductive bridge 33 is to allow the conductive layer 18 facing the user's head to transmit EEG signals to the inner conductive layer 18 proximate the sensor contacts 36. FIG. 9E is similar to the folded electrode 14 configuration described above. FIG. 18 shows an additional non-EEG sensor 15 incorporated into the headband 12.
[0061] Example Applications Personal Health Monitoring The system and method 10 of the present invention can be used to monitor the health of an individual or group (as described below). Any aspect of health can be monitored, including but not limited to mental health, behavioral health, and physical health. For example, it can be used in monitoring and / or treating a patient-user or subject-user A, particularly to identify precise biomarkers based on neural circuit computations that can measure and trend mental health conditions such as anxiety or depression at an individual or group level. The system and method 10 provides the mental health clinician B and the patient-user B with EEG data, non-EEG data, or output data points that can be quantified and trended over time to indicate progress or regression of treatment, similar to how body temperature is used as a monitoring measure for patients with influenza, or blood pressure is used as a monitoring biomarker for patients with heart failure.
[0062] In one exemplary embodiment of the present invention, mental health conditions such as anxiety and depression can be measured and trended utilizing a gamified algorithm or application 31 and accompanying EEG measurements. The gamified algorithm 31 entails a simple decision-making game, application, or process to detect, measure, or determine the neural correlates of anxiety, depression, or any mental health condition one wishes to measure or monitor.
[0063] The system and method 10 of the present invention can incorporate a gamified algorithm or decision game 31 into a remote application device 20 or any device with a displayable monitor or screen, such as a computer with a monitor, a television, a smartphone, etc. The gamified algorithm or decision game software 31 can take many forms, including having an avatar controlled by a patient-user A. Patient-user A controls the avatar to forage for food, for example berries in a berry patch. To maximize the overall reward in the form of total berries collected, patient-user A must make a "patch-leave" decision to stay in the current patch to gather the few remaining berries, weighing the "potential benefit" against the cost of moving to another berry patch as the supply dwindles. While the individuals' foraging strategies were near optimal, individual differences in patch-leave decisions have been found to be indicative of underlying brain states, as defined by normative models such as the limit value theorem. What was found was that individuals whose anxiety levels temporarily increased due to a stressor stayed in the berry patch for a shorter period of time than prescribed. However, individuals with chronically high levels of anxiety (operationalized as trait anxiety, a precursor to clinical anxiety and depression) spent more time than prescribed. Thus, gaming behavior and gamified algorithms or decision-making games31 can distinguish between episodic anxiety (which is generally short-lived and commonly caused by adverse events or situations such as PTSD) and trait anxiety (which is generally thought to be a characteristic of a person's personality).
[0064] A conventional gamified algorithm session without the system and method 10 of the present invention can identify individuals with trait anxiety with approximately 60% accuracy. However, when the gamified algorithm or decision game 31 is combined with the system and method 10 of the present invention, which provides high quality source-localized EEG signals, the accuracy improves to approximately 85%. The system and method 10 of the present invention can use EEG frequency content and signal power to measure the patient user A during the gamified algorithm or decision game software 31 and compare the measurements to known templates from individuals with subclinical and clinical levels of anxiety or depression. Although anxiety and depression have been mentioned, it should be understood that any mental health condition is within the spirit and scope of the present invention.
[0065] Another novel approach is to use the system and method 10 of the present invention to monitor subject-user A over a period of time to detect neural changes associated with Alzheimer's disease. In this way, clinician-user B can customize a treatment and treatment plan that will be most beneficial for subject-user A.
[0066] In use, the algorithms can be implemented in the above architecture by implementing a foraging / limit optimization game on a remote application device 20, such as a mobile phone or smartphone. To provide a measure of any mental health condition, such as anxiety, the system can be configured to be utilized in the following manner: 1. Subject-user A places the EEG head accessory 12 (e.g., a headband with electrodes) on his head and starts a program on the remote application device 20 that activates the EEG head accessory 12. Alternatively, or additionally, the EEG head accessory 12 may include a power switch or control device. 2. In the program, subject user A can begin measuring their mental health or anxiety by initiating a gamified algorithm or decision-making game 31. 3. The remote application device 20 can timestamp the EEG data at the start of the game 31 to indicate the EEG data collected while the game 31 is being played by subject user A. The timestamp can also include data from other sensors such as heart rate sensors, temperature sensors, respiration sensors, etc. 4. While subject-user A is playing the foraging game 31 on the remote application device 20 , the EEG electrode head accessory 12 measures and transmits EEG data of subject-user A to the remote application device 20 . 5. The remote application device 20 can simultaneously or temporally send or transmit the raw EEG data to the data platform 30. 6. The remote application device 20 can also send or transmit the results of the foraging game 31 or task, which can be considered non-EEG data. The raw EEG data and the results of the game 31 or non-EEG data can be transmitted separately or combined and transmitted together to the data platform 30. 7. In one embodiment, the remote application device 20 can also compare the raw EEG data and the game 31 results or non-EEG data and generate a report (which can be in the form of a graphical display on a mobile device) that can be sent to a third party user B, such as a doctor, and / or displayed to user B controlling the remote application device 20. 8. The data received by the data platform 30 (raw EEG data and / or non-EEG data) may be post-processed. For example, the raw EEG data may be filtered and manipulated to measure the frequency power spectrum of the anterior cingulate cortex and other relevant brain regions. This data may then be combined with the timing results of the foraging game to produce a measure of the user's anxiety. 9. If this is the first time the measurement is performed, or the first time within a period of time, the data platform 30 classifies the data as a baseline measurement. 10. If the user's EEG data matches the anxiety template and the game results indicate that the user stayed longer than average in each foraging field, a result is generated designating User A as having high trait anxiety. A score can be generated that combines the correlation results with the anxiety template and the magnitude of deviation from the population mean time in the foraging field. 11. If the user's EEG data is highly consistent with the anxiety template and the game results indicate that the user spent less time in each foraging field than the population average, a result is generated that designates the user as having state anxiety or stress anxiety (e.g., an anxious response to a particular event). As described above, a score can be generated that combines the correlation with the anxiety template and the magnitude of deviation from the population average time in the foraging field. 12. Subsequent measurements are compared to the baseline measurements to show a trend in User A's anxiety over time. 13. The analysis results and any trend information may be stored on the remote application device 20 and / or transmitted from the data platform 30 back to the remote application device 20 and application or program 31 for display to the subject user A. 14. The results may be made available in real time to a third party user B, such as a clinician, or may be accessed by third party user B when subject user A and third party user B meet. Third party user B may access the results or output data points on any device that can connect to the data platform 30, such as a mobile device or computer that has access to a provider portal 40 that is connected to or in communication with the data platform 30.
[0067] Group Monitoring - Synchronicity One of the many advantages of the present invention is the ability to monitor an individual or a group of individuals, or subject-users. One example includes monitoring a particular state of mind or body of a group. A variety of states can be monitored, including but not limited to mental and mental health, behavioral and physical states. Monitoring states is important to be able to monitor how a team performs in a variety of situations and settings, as will be described. The system and method of the present invention can simultaneously monitor, collect and synchronize EEG and / or non-EEG data from a group of individuals. One advantage of the present invention is that the system 10 includes the ability to analyze the synchrony of a group of individuals. Synchrony is based on the idea that by viewing the EEG responses of multiple people, users or subjects simultaneously, group dynamics, teamwork and human responses can be better communicated and measured. The system and method 10 of the present invention can use the group's EEG data to determine and understand the group's response to a stimulus, which could be an image, such as a photograph or video, or could be an event the group is experiencing together, including but not limited to a concert, a movie, a rally, or an altercation or engagement such as experienced by law enforcement or the military. The systems and methods of the present invention can use individual user data and group data to monitor group status or group dynamics, such as the degree of teamwork, group and individual fatigue levels, and group aggression levels.
[0068] The system and method 10 of the present invention can collect and use one or more synchrony data points in combination with EEG and / or non-EEG data of a group of users to monitor, analyze, and control or advise a group of users A. Synchrony data points include time, movement, geography or location. The present invention can use the synchrony data points to synchronize EEG and non-EEG data from participants to understand individual and group responses to specific stimuli, such as watching a video, participating in a game, watching a speech or other event, arguing, etc.
[0069] Geographic synchrony data points, when combined with EEG and / or non-EEG data, enable the system of the present invention to monitor and analyze the state of individual and group users as they move through stimuli, for example, while moving through a manufacturing plant, or at a conference or exhibition, or while driving through traffic jams, or while moving through a battlefield. Geographic synchrony data points, when combined with EEG and non-EEG data, help automatically show the responses of multiple users A to stimuli in specific locations. Examples include, for example: · To measure fatigue responses to specific manufacturing processes or activities; Observing driver distraction on public roads and indicating areas with high accident risk; or · Observing an individual's participation in a marketing exhibit or conference booth.
[0070] 1B, the systems and methods 10 of the present invention include one or more positioning systems 50, such as cell towers or Global Positioning Systems "GPS," that may be incorporated into or in operative communication with any component of the system 10, such as the EEG head accessory 12, the remote application device 20, or a third party GPS device such as a smart watch or other geolocation device. Any device or component of the system 10 may communicate with the positioning system or network 50.
[0071] temporal synchrony Continuing with FIG. 1B, the system and method 10 of the present invention can synchronize multiple EEGs via a remote application device 20, a cell phone, or other wireless beacon that broadcasts a standard time. In this exemplary embodiment, temporally synchronized data points can be collected and synchronized among multiple users A by synchronizing with each user A's broadcast time and clock. The time-synchronized data points can be transmitted via the remote application device 20 or directly from the EEG head accessory 12.
[0072] The steps for establishing or collecting temporally synchronized data points using cellular based transport are as follows: 1. The remote application device 20 receives the time from the internet, cell phone or gps network 50. 2. Optionally, the clock of the EEG head accessory 12 (e.g., a headband) is synchronized to the clock of the remote application device 20. 3. EEG data is collected from the EEG head accessory 12 and stored in the remote application device 20. The data can be time-stamped based on an internet or broadcast time signal. Digital time can be interwoven into the EEG data stream that is periodically transmitted along with the EEG data. If the latency of EEG data transfer between the EEG head accessory 12 and the remote application device 20 is too large or variable, time synchronizing data points and / or time-stamping the data and / or data points can be done directly in the EEG head accessory 12. 4. The time-stamped data and temporally synchronized data points may be transmitted over the Internet to a data server or data platform 30. 5. In the data platform 30, data from multiple users A is collected asynchronously or synchronously. The time information allows one or more users or a third party user to select individuals, groups, or subgroups of EEG data signals for synchronization. The remote application device 20 or the data platform 30 coordinates the EEG data signals and any non-EEG signals (e.g., the response of any data processing algorithms) and combines them in time based on the received temporal synchronization data points. 6. In this way, data synchronization from multiple users A can be obtained simply and automatically.
[0073] geographic synchrony The above-described systems and methods can be augmented by alternatively using time, GPS (Global Positioning System), Wi-Fi location data, or any type of location service or network 50 that is utilized and interwoven with EEG and non-EEG data, thereby location and / or time stamping the data and allowing automatic synchronization by location.
[0074] Geographic and temporal synchrony The system and method 10 of the present invention can also combine geographic and temporally synchronized data points to obtain additional information. In this manner, individual and group EEG data responses at one or more times and / or one or more locations can be established.
[0075] 10, the system and method 10 can obtain selected segments of EEG and / or non-EEG data 60 (e.g., responses from an application interacted with on the remote application device 20) from a single user at multiple time points when stimulation would occur. Monitor user B can select a subset of the data and one or more locations 61. Monitor user B can then time align the data with or without the system 10 relative to an arrival or data timestamp 62.
[0076] In use, the geographic and temporally synchronized data points can be combined with EEG and non-EEG data to be implemented in the following example situations: Arrival of an individual to a specific location, such as an employee arriving at the start of a manufacturing shift to perform a specific process: By combining geographic and time-synchronized data points with EEG data, fatigue or other cognitive or emotional responses can be automatically trended over time. An individual or group at an exhibition viewing a presentation at a pre-arranged time and place: the system 10 of the present invention allows for automatic selection of the EEG data response to the presentation. · Duration of fatigue or distraction at work: This allows for trending of fatigue or distraction over time in a particular or ship or aircraft job. Training: EEG and non-EEG data responses of students (e.g. pilots in a training simulator) can be observed to observe their reactions to emergency situations. Temporal synchronized data points can be used to combine EEG and non-EEG data to observe changes due to training. Driving: Driver distraction on roads in a specific geographic location. Timestamps further allow for determining distraction based on time of day or night.
[0077] Temporal synchronization with media In another exemplary embodiment of the present invention, as illustrated in FIG. 22A, the EEG and non-EEG data may be synchronized in time or geography with one of the media 70 displayed on the device or in the theater. In this manner, the EEG and / or non-EEG data responses of the audience or group A may be synchronized with the time or geography synchronization data points to monitor the appeal or popularity of the movie 70 and provide an indication of box office performance. The system and method 10 may also be used with user group A to understand the appeal of a particular commercial, product, media, personality, etc. It may also be used to determine how successful a particular commercial or product is in a particular geography, at a particular time of day, or for a particular television program or sporting event.
[0078] The system and method 10 of the present invention can utilize the runtime of the media 70 to establish time-stamped data points. This capability allows the system 10 to be used remotely in the user A's home or at a remote movie theater. User A can be sent or mailed an EEG head accessory 12 that can read the user A's EEG responses while watching the media 70. All of the EEG and non-EEG data can be collected from user A around the world and sent to the data platform 30 for processing. The data platform 30 can then generate reports that can be used collaboratively by film and commercial producers, product manufacturers, marketing companies, and the like. The system and method 10 of the present invention allows these third party users B to gain a physiological view of the potential success of a product (e.g., a film, commercial, or product) before investing in large-scale manufacturing or distribution.
[0079] The present invention can use data platform 30 to combine or interweave user A's EEG and non-EEG data with media 70 viewed while the EEG and non-EEG data was being collected. A third party user B can associate the EEG data responses and non-EEG data with specific points in time in the media 70 or with specific events that occurred within the media 70.
[0080] Media Screening and Analysis As illustrated in FIG. 12B, the EEG platform or system 10 of the present invention can be used in a variety of applications including healthcare, industrial safety, and business marketing. The EEG data collected by the system 10 can directly measure cognitive information of the user A, such as excitement, engagement, distraction, and positive and emotional states such as happiness and sadness. Thus, the EEG data of the system A can provide a more accurate and rapid way of measuring the impact of a person experiencing media such as video and audio, as well as print. The system A can provide valuable data related to preview testing, such as movie screenings, to determine the impact of a movie or a series of movie sequences prior to release.
[0081] Additionally, system 10 can be utilized in the development of user interfaces for a variety of applications, from computer software to aircraft pilot interfaces to automobile driver dashboards. By measuring the EEG signals and resultant data of system 10 over time and in the presence of various stimuli or tasks, system designers can optimize screens and interfaces to minimize fatigue and distraction.
[0082] In all these applications, the EEG signals of the present invention can be integrated with data from other sensors, such as heart rate or galvanic skin response, to increase the accuracy of the physiological cognitive response. Furthermore, when combined with an eye-tracking device 72 (see FIG. 12A), either in the form of eye-tracking glasses or a camera system embedded near a computer, mobile device, or television, eye position data can be used to determine which attributes of the media 70 provide a cognitive response. For example, eye-tracking data can be used to understand whether user or viewer A is looking at a particular product placement in a movie, and EEG can be used to determine whether this creates a positive or negative emotion associated with the product. In the case of user interface testing, eye-tracking can determine how long a user scans an interface before selecting a particular feature, or whether a particular alert or feature creates a distraction.
[0083] As described above, the architecture of system 10 includes an EEG head accessory or headband 12, a remote application device 20 having a mobile application 31, and a data platform 30 including servers and other monitoring devices. For media presentations, system 10 includes a film screen, television, or other monitor 70 for viewing video and / or audio playback. The monitor 70 displaying the media may be the same as the mobile device that interfaces with EEG head accessory 12, and a camera for eye tracking may also be integrated with that device 70.
[0084] As illustrated in FIG. 12A, data transfer for the media screening application involves all sensors integrated within the remote application device 20 before being sent over the Internet to the data platform 30, although the concept would similarly be employed if all sensors were to send information individually to the data platform 30.
[0085] The system 10 of the present invention is designed to be used at remote traditional cinema screening locations or to be run remotely in a user's home without significant setup or equipment burden. When a video 70 is started and played by user A, physiological data is collected and transmitted to the data platform 30 for aggregation and analysis. Furthermore, data across multiple users A at multiple locations can be aggregated to provide large volume data analysis without holding a single large in-person screening.
[0086] FIG. 12B shows a flow chart of the remote video screening process. A first step 74 involves the producer loading the media into the data platform 74 where it can be downloaded by user A. A next step 75 involves user A putting the head accessory 12 on his head and connecting it to the remote application device 20. If an eye-tracking device 72 is used, the user can put it on his head in step 76. In a next step 77, user A can select the media to watch. In step 78, the remote application device 20 synchronizes the media and streaming data. Optionally, in step 79, the remote application device 20 can synchronize the eye-tracking data with other data. In step 80, all data collected is time-stamped with respect to the media being watched and sent to the algorithm platform 30 for analysis. In step 81, the algorithm platform 30 performs an analysis of the data, which in step 82 can be synchronized with data from other individuals being monitored. In step 83, trends and related data points are collected in a report and passed to the monitored user B in step 84.
[0087] Examples of insights that can be gained from video and media screening by looking at deviations in specific EEG signals include: Overall cognitive / emotional response at the end of and throughout the media experience A moment that causes great stress Audience engagement during the media. This can be trended and shown to the content creator as a time plot in sync with the media or as an analytical plot. This way the content creator can know at what point during the media / movie / video the user has lost engagement or not. The degree of synchronicity between all users. This average response indicates the strength of the cognitive response at different points in time. The degree of EEG synchrony between users watching the same video at the same time and place. Synchrony between users has been shown to be a useful predictor of future box office revenue. · Normalization of cognitive data between users / viewers due to differences in cognitive and emotional states from the beginning to the end of the video. Separation and analysis of user / audience responses by demographics such as gender, age, location etc. This can be automatically selected or highlighted by the artificial intelligence algorithms employed in the data platform. Isolating and analyzing user / viewer reactions to various media modifications, such as alternate endings, alternate audio tracks, deleted scenes, etc.
[0088] It is noted that while the above processes and algorithms can be performed on pre-recorded videos such as movies, the processes and algorithms can be performed equally well on live streaming events such as political debates or athletic meets.
[0089] Product Placement Analysis In the film industry and television programming, the revenue source of a production is often revenue from product placements: examples include use of branded beverages and foods, select automotive and transportation brands, fashion products, retail stores, etc. To better monetize these placements, the system 10 can be utilized to demonstrate the value of various placements.
[0090] FIG. 12C shows a product placement flow chart for measuring the value of a product using the system 10 of the present invention. The first step 85 involves a producer loading media containing the product into a data platform where it can be downloaded by user A. The next step 86 involves user A wearing the head accessory 12 on his head and connecting it to the remote application device 20. If an eye-tracking device 72 is used, the user can wear it on his head in step 87. In the next step 88, user A can select the media with the product to watch. In step 89, the remote application device 20 synchronizes the media and streaming data. Optionally, in step 90, the remote application device 20 can synchronize the eye-tracking data with other data. In step 91, all data collected is time-stamped with respect to the media with the product being watched and sent to the algorithm platform 30 for analysis. In step 92, the algorithm platform 30 performs an analysis of the data, which in step 93 can be synchronized with data from other individuals being monitored. In step 94 , eye tracking and other data about the product is collected in a report and passed to monitor user B in step 95 .
[0091] Examples of insights that can be gained about product placement by analyzing EEG and eye-tracking data include: The percentage of viewers or users who viewed a particular product placement The length of time the viewer's or user's eye was fixed on the product The viewer’s or user’s cognitive and emotional response when their eye is fixed on the product (e.g., happiness, sadness, distraction, etc.). The degree of synchronicity among all viewers or users when viewing a product placement. This average response indicates the strength of the cognitive response to each placement. The degree of EEG synchrony between viewers or users watching the same video at the same time and place. Subset analysis of user / audience responses by demographics such as gender, age, location etc. This can be automatically selected or highlighted by the artificial intelligence algorithms employed in the data platform.
[0092] All of the above analyses are combined to generate a product placement "score" that indicates the length of time viewers noticed the product and the degree to which they felt positive about it. These scores can then be used to monetize the value of the product placement within the film or media content.
[0093] Operation Monitoring 13A and 13B, system 10 is utilized to monitor a subject user A performing a task. In this section, subject user A will be referred to as operator A, which may refer to an operator of any process (e.g., a production operator in a manufacturing or industrial plant, a sailor on a civilian or military ship, or any other activity that requires a human to complete a task).
[0094] Studies have shown that the majority of workplace accidents are caused by fatigue, distraction, and inadequate training. The system and method 10 of the present invention can monitor and trend these factors by utilizing wireless and high quality EEG electrodes 14. The present invention can also generate graphical outputs that identify areas of risk. Examples of graphical outputs of the present invention include creating heat maps of the areas with the highest risk, the equipment or machines with the highest risk, and the times with the highest risk. The system 10 can determine, among other things: · The average time it takes for an operator to become fatigued performing a task; · Distractions associated with the factory environment; · the effectiveness of training and the length of time it takes for the task to be adequately mastered or memorized after training; and · Operator impairment due to fatigue or chemicals.
[0095] Process steps for manufacturing optimization Referring to FIG. 13C, all operators A participating in the manufacturing optimization are fitted with an EEG headband or head accessory 12. Operator A then performs a manufacturing process step 101. Fatigue, concentration, and distraction are measured and trended 102 over time as Operator A performs the manufacturing step. In some cases, Operator A may need to have a baseline measurement of fatigue to calibrate the EEG system 10 to their individual situation. The fatigue trend 102 is compared to a threshold to determine when Operator A reaches a point of mental fatigue. This data is then used to optimize the process step for the human Operator A. Examples of redesign include, but are not limited to: Redesign processes to limit or reduce mental fatigue Plan your breaks at the best times Match operators with the processes that best suit their capabilities Determine in real time if an operator is too fatigued to carry out a process "Lock out" safety-critical processes if operators are fatigued or distracted Trending operators' frontal lobe activity days / weeks after training to understand the effectiveness of training · Trend fatigue over a period of several weeks to determine if fatigue is improved by process knowledge.
[0096] The collected data is displayed geographically within the plant to show "hot spots" or processes generating the highest fatigue, which may result in the replacement of Operator A. In this manner, each process is given a fatigue score of 100, both trended and in real time from the operator running the process, as illustrated in Figure 13B.
[0097] As described in more detail above, the timestamp and location data are generated automatically based on signal triangulation from Bluetooth, Wi-Fi, or other signals from the EEG headband 12 or the user's remote application device 20 or mobile phone.
[0098] Optimizing production through contact tracing Above we have described a system of the present invention that is used to optimize a manufacturing plant, a vessel, or any step in a process based on EEG data. This system 10 also includes a set of sensors and monitors for the safe functioning of manufacturing operations during a pandemic scenario.
[0099] Since the EEG headband or headgear 12 contacts the forehead, a non-EEG sensor 15, such as a temperature sensor, can be incorporated therein to monitor the worker or operator A's vitals, such as body temperature, for early warning of the onset of illness. This is particularly important to identify employees A who may pose an infection risk to other employees A. Since an increase in body temperature can be detected, the business can quickly isolate the employee. The temperature sensor A can also be used to detect the rise and fall of body temperature of a user A who spends time outdoors. In a hot environment, the system 10 can quickly detect hyperthermia, and in a cold environment, the system can quickly detect hypothermia. The user or employee A can then seek appropriate environmental protection.
[0100] The headband 12 of the system 10 can also include an acceleration sensor 15 for measuring movement and activity. The acceleration sensor 15 can measure the movement or lack of movement of the user. This is important in a medical setting where the presence or absence of movement of a patient must be monitored. The acceleration sensor 15 can also be used to monitor the health of a user A by detecting and measuring the physical activity of the user A. In the case of an employee A at work, it can be used to determine whether he or she is getting enough or some physical activity during his or her shift and day.
[0101] Signals from either the headband 12, the user's remote application device 20 such as a mobile phone, or other sensors 15 can further be used to indicate the proximity of User A or Operator A to one another. This aids in social distancing and can be extended to indicate contact tracing in the event of a pandemic virus outbreak. The system 10 can help update potentially infected people and provide data to enable healthy workers to keep a plant, store or office operating, resulting in reduced time to notification of illness.
[0102] User Interface Testing Referring to Figure 14, the system 10 can also be used for user interface testing. In this application, the media film content 70 can be substituted into a user interface 104, such as software or machine operation, and analytics can be collected to measure and optimize the user A or operator's experience. This can be performed in real-time evaluation or even within a simulated environment, such as an aircraft simulator. Figure 14 shows a diagram of the system architecture. As with the media screening application above, the system collects recordings of interactions with the user interface, along with physiological data.
[0103] The system 10 of the present invention may collect physiological data of User A as well as one or more records of interactions with one or more user interfaces.
[0104] FIG. 15 shows a flow chart of a user interface test for measuring the value of a user interface using the system 10 of the present invention. The first step 106 involves the UI designer loading the user interface into a data platform where it can be downloaded by user A. The next step 107 involves user A putting the head accessory 12 on his head and connecting it to the remote application device 20. If an eye-tracking device 72 is used, user A can put it on his head in step 108. In the next step 109, user A's eye-tracking is calibrated. In step 110, the remote application device 20 starts the user interface or simulator and data is time-stamped and / or synchronized. In step 111, the user starts using the user interface or simulator and all data is synchronized and / or time-stamped. In step 112, all data collected is time-stamped with respect to the user interface or simulator and sent to the algorithm platform 30 for analysis. In step 113, the algorithm platform 30 can perform analysis of the data and synchronize it with data from other individuals being monitored. In step 114 , eye-tracking and other data about the user interface is collected in a report and passed to monitor User B in step 115 .
[0105] Examples of insights that can be gained about user interfaces by analyzing EEG and eye-tracking data include: The distribution and proportion of gaze times in different areas of the user interface. · The length of time a user's eyes scan before implementing a particular function. · The length of time it takes for a user to react after an event, such as an emergency alarm, occurs in the simulator. · The degree of synchronicity among all users and the average variance in time it takes to perform a particular function or respond to an event. Distribution and measurement of the amount of time users spend on specific user interface features. A user’s cognitive responses to specific events and feature implementations, for example measuring levels of engagement, distraction, or stress while using a product or interface. · Synchronicity of cognitive responses between users to a particular event A user's fatigue profile over time and functional activity, allowing measurement of the specific fatigue caused by a particular event or function, as well as providing an overall measure of the cognitive load of a particular event or user interface operation. · Providing time allocation between users until fatigue-related cognitive impairment is likely to occur. Subset analysis of user / audience responses by demographics such as gender, age, location etc. This can be automatically selected or highlighted by the artificial intelligence algorithms employed in the data platform. · Automatic highlighting of certain features that cause excessive distraction or stress responses. Automatic highlighting and segmentation of specific events that could cause critical system failures, for example an aircraft crash in a simulator.
[0106] military use As briefly mentioned above, the system and method 10 of the present invention is ideally suited for use in law enforcement and the military. However, in military applications digital security is paramount, and wireless data transmission can pose a data security risk. Not only can biological data be intercepted, but wireless transmissions can provide a pathway for malicious actors to penetrate secure networks. For this reason, many military facilities that handle sensitive information, such as Sensitive Compartmented Intelligence Facilities (SCIFs), cannot allow wireless systems on their premises.
[0107] This poses a problem for using cognitive wearables such as EEG in a safe environment. Use cases such as monitoring operator fatigue, measuring cognitive responses to situational events, and improving human performance in sensitive and stressful environments can become more challenging. Not only is real-time data transfer impossible, but User A cannot interact with the wearable functionality through a wireless data link to a mobile device such as Bluetooth. In these situations, a separate architecture is needed for data collection and monitoring of device functionality. The architecture and design of the present invention overcomes these challenges while maintaining the usability of the system.
[0108] As illustrated in Fig. 11, in this embodiment of the invention, instead of a wireless link to the processing or mobile device 20, there is a wired connection 130 to the processing or mobile device 20, or to a storage platform or data collection device carried by the user. The data storage device can be connected to the EEG head accessory 12 by wired, wireless, or short-range wireless communication. In a wireless configuration, the data transmission distance is very short (e.g., a few inches to a foot or two). Data encryption can also be used to protect the data.
[0109] Instead of the tether 130, the EEG data is stored locally in the electronics of the headband 12. The data can be stored in memory or on a removable data storage device such as a flash SD card. In this configuration, the EEG head accessory or headband 12 includes an on-board real-time clock. The clock data is added to the EEG data stream stored locally in the storage device to provide timestamps for specific events and evoked responses.
[0110] Lead-off Display One challenge of an untethered, local storage EEG device such as the head accessory 12 is initiating operation of the device 12, monitoring the power or battery status, and monitoring the connection between the sensor electrodes 14 and user A's skin. The system 10 employs lead-off indicators that notify user A when a particular electrode 14 is not in strong electrical contact with user A's body. Because the mobile wireless EEG head accessory 12 is located on user A's head, user A cannot easily see indications of lead status or battery status. The testing process for these types of electrodes 14 is performed on the wireless and / or tethered system 10, thereby resolving potential adverse effects (e.g., security or data integrity flaws) that may occur if not performed.
[0111] To overcome these limitations, a series of visual indicators or lights 136 are employed on the EEG electrode device or headband 12 to provide an indication of the status of the electrodes 14. As illustrated in FIGS. 16A and 16B, these indicators 136 can be arranged in a variety of configurations. For example, they can be positioned opposite each electrode 14 or lined up together so that user A can see the status of the lead electrodes 14 by monitoring the indicators 136 on a reflective surface such as a mirror. The visual indicators 136 can also be operatively positioned on the user's helmet 12 above the eyes on a small visor 138 that extends downwardly from the helmet 12. In this configuration, user A can determine the status by the position of the lights relative to the helmet 12 or by the color of the status light. As illustrated in FIG. 16B, the visual indicators 136 can be positioned on the interior surface of the helmet 12 and / or visor 138 so as to be visible only to user A.
[0112] Alternatively, an audio subsystem may be included that can issue an audible warning of an open or low power electrode 14 to user A. The audible warning may be an audible beep or another sound. The system 10 includes different sounds that indicate different statuses, such as low batter or a faulty electrode 14. As illustrated in FIG. 16A, the audio system 140 comprises a sound emitting device, such as a speaker, that can be incorporated into the headband or helmet 12. The audio system 140 may be a recorded audio instruction to inform user A of the status of a component of the system 10 (e.g., an open electrode). This audio system 140 may be operated manually or automatically. The status of a particular component of the system 10 may be immediately relayed to a third party user B, such as an equipment technician, who can use the user monitor or system 40 to locate user A and advise user A on how to fix the indicated problem or provide user A with a method for fixing the malfunction. Third party user B may include a military strategist who can recall user A from combat while simultaneously moving one or more other users A to replace the retreating user A.
[0113] In another exemplary embodiment of the present invention, the system 10 may include one or more vibration devices 142 or transducers located in or on the EEG electrode fixture or headband 12. The vibration devices 142 may be located opposite or adjacent to each electrode 14, allowing the user to feel vibrations at the particular electrode 14 that is experiencing a problem. The vibration device 142 may also be placed anywhere that allows user A to sense the vibrations it emits.
[0114] In an exemplary embodiment, as illustrated in FIG. 16B, a remote device 144 such as a wristband may be used with one or more vibration devices 142 associated therewith. Each electrode 14 may correspond to a particular vibration device 142 on the wristband. In another exemplary embodiment, the vibration devices 142 are configured to vibrate in different patterns, each pattern associated with a particular electrode 14 or other component of the system 10. In this manner, user A may determine which electrode 14 has failed.
[0115] The wristband 144 may include a wireless transmitter and / or receiver 146 that communicates with other components of the system 10 of the present invention. The wristband 144 may be selectively limited or configured to only receive lead-off information for notifying user A. It may also be selectively configured to transmit lead-off information to a third party user B who is monitoring one or more users A wearing the system 10. The ability to select different receive and transmit states allows the system 10 to control the lead-off information.
[0116] The inability to see the EEG electrical signal and the status of the lead off indicators can create a situation where insufficient data is recorded without User A or B realizing that an adjustment is needed. In such a case, poor data quality from one electrode 14 can contaminate other electrodes during calculations, or, if the active bias electrode 14 is not in contact, can contaminate all electrodes 14, providing insufficient and, in the worst case, unusable data.
[0117] Alternatively, to make the system 10 more robust, the system 10 can sense and indicate in the data when the impedance of an electrode 14 exceeds a threshold. In this case, the data is marked as a lead off and data from that electrode 14 is ignored in the calculations. In the case of an active bias electrode 14, with the system 10 nominally positioned at FpZ (see FIG. 3), the loss of this electrode 14 may contaminate all electrodes 14 and / or make it difficult to extract any signal.
[0118] 17A, 17B, and 3, any changes from the nominal electrode 14 configuration are included in the time-stamped data so that these changes can be taken into account during subsequent EEG signal processing. To address failure of an active bias electrode 14, the system 10 of the present invention includes a backup electrode 14 switching subsystem that can automatically detect a failed electrode 14 and switch to a backup active bias electrode 14 or designate another electrode 14 as the active bias electrode 14. As illustrated in FIG. 16C, the system 10 includes a multiplexer 148 in the electronics that allows the active bias electrode 14 to be switched to another position, thereby maintaining the integrity of the remaining electrodes 14 and the collected data. The system 10 may perform the following example process: At step 200, it is detected that the electrode 14 is off. In step 201, the system 10 switches on the active bias electrode A1. In step 202, if active bias electrode A1 is detected to be disconnected, then switch to electrode Fp1. In step 203, if electrode Fp1 is detected as disconnected, the system 10 cycles through the remaining electrodes. In step 204, once electrode Fpz is restored, the system 10 returns the active bias to electrode Fpz. In step 205, if electrode Fpz is not restored, do not change anything.
[0119] 17B, the system 10 can also perform generating and tuning feedback loops taking into account any number of electrodes 14 and / or sensors 15 and their configurations (i.e., positions of reference and common mode feedback biases). The system can perform the following process: In step 207, electrode A2 is detected to be ON. In step 208, the system 10 determines whether electrode A1 is on. In step 209, if step 208 is "yes", the system 10 switches Vreff to electrode A1. In step 210, if step 208 is "no", the system 10 proceeds to electrodes F7 and F8. In step 211, if no electrodes are detected, the system 10 creates a log date.
[0120] Although the present invention has been described in connection with what are presently considered to be the most practical and preferred embodiments, it will be apparent to those skilled in the art that the present invention should not be limited to the disclosed embodiments. It will be readily apparent to those of ordinary skill in the art that many modifications and equivalent arrangements can be made without departing from the spirit and scope of the present disclosure, and such scope should be accorded the broadest interpretation of the appended claims to encompass all equivalent structures and products. Moreover, features or aspects of the various exemplary embodiments can be mixed and matched (even if such combinations are not expressly described herein) without departing from the scope of the present invention.
Claims
1. 1. A field-wearable neural activity monitoring system wearable for monitoring neural activity of a remote user, comprising:
1. A head accessory assembly comprising: a headband extending at least partially around a head of the remote user, the headband having an inner surface and an outer surface, the inner surface being positionable against the head of the remote user; the head accessory assembly comprising: a sensor having an electrode surface and a non-electrode surface, the sensor being removably coupled to a portion of the headband, the non-electrode surface being positioned against the inner surface of the headband; a mobile data collector in operative communication with the head accessory assembly for collecting neural activity data of a remote user detected by the sensor; The field-wearable neural activity monitoring system.
2. 10. The field-wearable neural activity monitoring system of claim 1, further comprising a plurality of sensors coupled to a connector extending between and operably coupled to the electrode surfaces of the plurality of sensors disposed proximate an outer surface of the headband.
3. 10. The field-wearable neural activity monitoring system of claim 1, further comprising a neural activity event configured to initiate neural activity of the remote user.
4. 10. The field-wearable neural activity monitoring system of claim 1, further comprising a remote computer in operative communication with the mobile data collector for receiving and processing the neural state of the remote user.
5. 4. The field-wearable neural activity monitoring system of claim 3, wherein the neural activity events comprise a program stored on a remote device and operated by the remote user.
6. 4. The field-wearable neural activity monitoring system of claim 3, wherein the neural activity events include activities performed or observed in real time by the remote user.
7. 5. The field-wearable neural activity monitoring system of claim 4, wherein the remote computer is configured to synchronize neural activity of multiple remote users to establish a collective neural state of the multiple remote users.
8. 10. The field-wearable neural activity monitoring system of claim 1, further comprising a coupler member disposed between the headband and the sensor, wherein the sensor is easily replaced by the remote user.
9. 9. The field-wearable neural activity monitoring system of claim 8, wherein the coupler comprises first and second spaced apart magnet members coupled to the non-electrode surfaces of the sensor, the first magnet member positionable proximate to the inner surface of the headband and the second magnet member positionable proximate to the outer surface of the headband, and the sensor is removably secured to the headband.
10. 2. The field-wearable neural activity monitoring system of claim 1, further comprising a random access memory coupled to the head accessory for receiving and at least temporarily storing the detected neural activity data, the mobile data collector wirelessly communicating with the head accessory to collect the detected neural activity data from the random access memory.
11. 5. The field-wearable neural activity monitoring system of claim 4, wherein the mobile data collector encrypts the neural activity data until it is transmitted to the remote computer.
12. 5. The field-wearable neural activity monitoring system of claim 4, wherein the detected neural condition comprises a normal condition, an Alzheimer's condition, a depressed condition, an anxious condition, a bipolar condition, a wasted condition, a confused condition, or an abnormal mental condition.
13. 1. A field-wearable neural activity monitoring system configured to detect mental states of a plurality of remote users and wearable by the plurality of remote users, comprising: Each remote user's system: a head accessory that extends around the head of a remote user, a plurality of sensors configured to detect neural activity data of a remote user experiencing an event, each of the plurality of sensors folded around a portion of the head accessory; a random access memory coupled to the plurality of sensors for at least temporarily storing the neural activity data; a program stored in said random access memory for controlling said plurality of sensors; a power source coupled to the head accessory and in operative communication with the plurality of sensors and the random access memory; a data collector in operative communication with the head accessory to collect neural activity data of the remote user over time; a remote computer in operative communication with the mobile data collector for receiving and processing the neural activity data to determine the mental state of the remote user; The field-wearable neural activity monitoring system.
14. 14. The field-wearable neural activity monitoring system of claim 13, further comprising a remote device containing a program used by a remote user to elicit neural activity.
15. 14. The field-wearable neural activity monitoring system of claim 13, wherein the head accessory has an inner surface and an outer surface, each of the plurality of sensors has an electrode surface and a non-electrode surface, and the sensors are foldable around the non-electrode surface that is positioned against the inner and outer surfaces of the head accessory.
16. 14. The field-wearable neural activity monitoring system of claim 13, wherein the event experienced by the remote user is a virtual event.
17. 14. The field-wearable neural activity monitoring system of claim 13, wherein the event experienced by the remote user is a non-virtual event.
18. 14. The field-wearable neural activity monitoring system of claim 13, wherein the plurality of sensors are removably coupled to a portion of the head accessory by one or more magnets.
19. 14. The field-wearable neural activity monitoring system of claim 13, wherein the remote computer detects neural activity states including a normal state, an Alzheimer's state, a depressed state, an anxious state, a bipolar state, a wasted state, a confused state, or an abnormal mental state.
20. 1. A method for monitoring a mental state of one or more remote users, comprising: providing a head accessory having one or more neural activity sensors configured to detect neural activity of the remote user; providing neural activity events for eliciting neural activity from the remote user; providing a neural activity data collector in operative communication with the one or more neural activity sensors to collect neural activity data of the remote user over time; providing a remote computer configured to receive the neural activity data from the neural activity data collector and to process the neural activity data of the remote user; and determining a mental state of the remote user by processing the experienced neural activity data in the remote computer; The method comprising: