Method and system for brain wave analysis based on event automatically quantified through various bio-signals

Abstract

The present invention relates to a multi-sensor and brain wave-based cognitive function monitoring system, and provides a multi-sensor and brain wave-based cognitive function monitoring system and a method thereof, the system including: an EEG collection unit for measuring brain waves of a user; a sensor unit including at least one sensor for collecting body response data of the user; and a calculation unit for evaluating the cognitive function of the user on the basis of the brain waves and the body response data, wherein the calculation unit includes an analysis module for analyzing patterns of changes in the brain waves before and after the occurrence of an event derived from the body response data measured by the sensor unit.

Description

Method and system for automatically quantifying event-based brainwave analysis using various biosignals

[0001] The present disclosure relates to a method and system for automatically quantifying event-based brainwave analysis using various biosignals. More specifically, the present disclosure relates to a multi-sensor and EEG-based cognitive function monitoring method and system that measures body response data and brainwaves using multiple sensors and EEG, and evaluates cognitive function based on the measured body response data and brainwaves.

[0002] Brain waves (neural oscillations) are wave-shaped signals that occur when signals are transmitted between cranial nerves in the nervous system, and their frequencies vary depending on the brain region. Brain waves are subdivided into various groups depending on the observed frequency, such as alpha (8–13 Hz), beta (13–30 Hz), gamma (38–45 Hz), delta (0.5–4 Hz), and theta (4–7 Hz). Alpha waves are observed when a person is in a relaxed state, beta waves are observed in a normal state, gamma waves are observed in a state of concentration or tension, delta waves are observed in a sleep state, and theta waves are observed in a state of falling asleep. Since brain waves appear in various frequency patterns as mentioned above, rather than observing the waveform as it is, power spectrum analysis is used to study the proportion of each brain wave according to frequency, or event-related potentials (ERPs) for the desired frequency are studied.

[0003] Electroencephalography (EEG) is an electrophysiological measurement method that records the brain's electrical activity through electrodes. It is used to assess brain damage or disease, and when diagnosing patients, it primarily analyzes event-related potentials or the spectral density of brain waves.

[0004] Event-related potentials (ERPs) are brainwave potential differences that occur in response to a specific event. Research using ERPs requires sufficient data for statistical analysis. In this case, a specific event refers to a stimulus or task presented during EEG measurement. Previously, cognitive function was assessed by assigning a specific task (event) under restrictive conditions and measuring the resulting brainwave potential.

[0005] However, the existing method of monitoring cognitive function through brain wave analysis involves measuring brain waves in response to specific tasks assigned under restrictive conditions, which has limitations in collecting long-term data and in identifying warning signs or symptoms of cognitive dysfunction that occur in everyday life rather than in experimental situations.

[0006] The present disclosure is intended to solve the above problems, and to provide a multi-sensor and brainwave-based cognitive function monitoring method and system that analyzes brainwave changes related to user's physical reactions occurring in daily life by turning them into events, rather than following the method of assigning specific tasks under restrictive conditions.

[0007] One embodiment of the present disclosure provides a multi-sensor and brainwave-based cognitive function monitoring method, a multi-sensor and brainwave-based cognitive function monitoring system, and a program stored in a computer-readable recording medium for executing the multi-sensor and brainwave-based cognitive function monitoring method.

[0008] One embodiment of the present disclosure provides a multi-sensor and brainwave-based cognitive function monitoring method, a multi-sensor and brainwave-based cognitive function monitoring system, and a program stored in a computer-readable recording medium for executing the multi-sensor and brainwave-based cognitive function monitoring method.

[0009] A multi-sensor and brainwave-based cognitive function monitoring system according to one embodiment of the present disclosure includes an EEG collection unit that measures a user's brainwaves, a sensor unit including at least one sensor that collects the user's physical response data, and a calculation unit that evaluates the user's cognitive function based on the brainwaves and the physical response data, wherein the calculation unit may include an analysis module that analyzes changes in brainwaves before and after an event derived from the physical response data measured by the sensor unit.

[0010] In one embodiment, the sensor unit may include at least one of a gyro sensor for measuring head movement, an eye blink sensor for measuring eye blinking, and a temperature sensor for measuring body temperature.

[0011] In one embodiment, the operation unit may further include a first signal processing module that processes brain wave data measured by the EEG collection unit.

[0012] In one embodiment, the first signal processing module can filter the brainwave data and classify the brainwave data by brainwave region.

[0013] In one embodiment, the operation unit may further include a second signal processing module that processes the body response data measured by the sensor unit.

[0014] In one embodiment, the second signal processing module can derive the event when the body response data changes beyond a set range.

[0015] In one embodiment, the operation unit may further include a synchronization module that synchronizes the time of the body response data measured by the sensor unit and the brain wave data measured by the EEG collection unit.

[0016] In one embodiment, the operation unit may further include a judgment module that evaluates cognitive function based on the brain wave change pattern.

[0017] A multi-sensor and brainwave-based cognitive function monitoring method according to one embodiment of the present disclosure may include a step of measuring a user's brainwaves through an EEG collection unit, a step of collecting the user's physical response data by at least one sensor, and a step of evaluating the user's cognitive function by analyzing a change pattern of the brainwaves before and after an event derived from the physical response data measured by the at least one sensor.

[0018] In one embodiment, the step of collecting the user's body response data by the at least one sensor may include at least one of a step of a gyroscope sensor measuring head movement, a step of an eye blink sensor measuring eye blinking, and a step of a temperature sensor measuring body temperature.

[0019] In one embodiment, the step of analyzing the brainwave change pattern may further include at least one of the steps of checking the response time of each region of the brainwave data for the event, the step of checking the amount of change in each region of the brainwave data for the event, and the step of checking the recovery time in each region of the brainwave data after the event.

[0020] In one embodiment, the step of analyzing the brain wave change pattern may further include the step of calculating a weighted sum of the brain wave data, and the step of adjusting the weighted sum according to the standard deviation of data for each brain wave region.

[0021] In one embodiment, the step of analyzing the brain wave change pattern may further include the step of calculating a weighted sum of the brain wave data, and the step of adjusting the weighted sum using the deviation between the left and right hemispheres of the brain.

[0022] In one embodiment, the method may further include a step of processing brain wave data measured by the EEG collection unit.

[0023] In one embodiment, the step of processing brain wave data measured by the EEG collection unit may include at least one of a step of filtering the brain wave data and a step of classifying the brain wave data by brain wave region.

[0024] In one embodiment, the method may further include a step of processing the body response data measured by the at least one sensor.

[0025] In one embodiment, the step of processing the physical response data may include a step of deriving the event when the physical response data changes beyond a set range.

[0026] In one embodiment, the method may further include a step of synchronizing the time of the body response data measured by the sensor unit and the brain wave data measured by the EEG collection unit.

[0027] According to one embodiment of the present disclosure, a program stored in a computer-readable recording medium may be provided to execute a multi-sensor and brainwave-based cognitive function monitoring method.

[0028] According to one embodiment of the present disclosure, events can be derived from user's physical response data in daily life without assigning a specific task, and changes in brain waves before and after the occurrence of the derived event can be analyzed.

[0029] Additionally, according to one embodiment of the present disclosure, cognitive function in daily life can be evaluated based on analysis of continuous brain wave change patterns.

[0030] FIG. 1 is a block diagram of a multi-sensor and brainwave-based cognitive function monitoring system according to one embodiment of the present disclosure.

[0031] FIG. 2 is a diagram illustrating a state of use of a multi-sensor and brainwave-based cognitive function monitoring system according to one embodiment of the present disclosure.

[0032] Figure 3 is a block diagram of an eye blink measurement sensor according to one embodiment of the present disclosure.

[0033] FIG. 4 is an exemplary diagram illustrating a strain gauge according to one embodiment of the present disclosure.

[0034] FIG. 5 is a usage state diagram of an eye blink measurement sensor according to one embodiment of the present disclosure.

[0035] Figure 6 is a block diagram of an operation unit in one embodiment of the present disclosure.

[0036] FIG. 7 is a flowchart illustrating a multi-sensor and brainwave-based cognitive function monitoring method according to one embodiment of the present disclosure.

[0037] Figure 8 is a flowchart illustrating an event derivation method according to one embodiment of the present disclosure.

[0038] FIG. 9 is a graph illustrating a method of analyzing changes in a gyro sensor and brain waves measured by a method according to one embodiment of the present disclosure.

[0039] To clarify the technical idea of ​​the present disclosure, embodiments of the present disclosure will be described in detail with reference to the attached drawings. In describing the present disclosure, if a detailed description of a related known function or component is determined to unnecessarily obscure the gist of the present disclosure, the detailed description will be omitted. Components having substantially the same functional configuration among the drawings are given the same reference numbers and symbols as possible even if they are shown in different drawings. For convenience of explanation, devices and methods are described together when necessary. Each operation of the present disclosure does not necessarily have to be performed in the described order and may be performed in parallel, selectively, or individually.

[0040] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, the applicant may arbitrarily select terms, and in such cases, their meanings will be described in detail in the description of the relevant embodiments. Therefore, the terms used in this specification should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0041] Throughout this disclosure, singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms such as "comprise" or "have" should be understood to indicate the presence of a feature, number, step, operation, component, part, or combination thereof, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. In other words, when it is said throughout this disclosure that a part "comprises" a certain component, unless specifically stated otherwise, this does not mean that other components may be included, but rather that other components may be excluded.

[0042] Expressions such as "at least one" modify the entire list of elements, not individual elements of the list. For example, "at least one of A, B, and C" and "at least one of A, B, or C" refer to only A, only B, only C, both A and B, both B and C, both A and C, all of A, B, and C, or any combination thereof.

[0043] In addition, terms such as “...part”, “...module”, etc. described in the present disclosure mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0044] Throughout this disclosure, when a part is said to be "connected" to another part, this includes not only cases where the parts are "directly connected," but also cases where the parts are "electrically connected" with other elements intervening. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes other components, unless otherwise specifically stated.

[0045] Throughout this disclosure, terms including ordinal numbers, such as "first," "second," etc., may be used to describe various components; however, the components are not limited by the terms. The terms are used solely to distinguish one component from another. For example, without departing from the scope of the present disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component." The term "and / or" includes any combination of multiple related items or any one of multiple related items.

[0046] The expression "configured to" as used throughout this disclosure can be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily mean something is "specifically designed to" in hardware. Instead, in some contexts, the expression "a system configured to" can mean that the system is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" can mean a dedicated processor for performing the operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) that can perform the operations by executing one or more software programs stored in memory.

[0047] Throughout this disclosure, an event is a sensory, cognitive, or motor event that can cause a change in brain waves, and can be a reference point for analyzing the pattern of brain wave changes when measuring the event-related potential (ERP) of brain waves.

[0048] One embodiment of the present disclosure provides a multi-sensor and brainwave-based cognitive function monitoring system, method, and program for executing the method.

[0049] In addition, one embodiment of the present disclosure provides a device and method for deriving an event from a user's physical response data and analyzing a change pattern of brain waves before and after the occurrence of the derived event, and a program for executing the method.

[0050] FIG. 1 is a block diagram of a multi-sensor and brainwave-based cognitive function monitoring system according to one embodiment of the present disclosure.

[0051] Referring to FIG. 1, a multi-sensor and brainwave-based cognitive function monitoring system (10) according to one embodiment may include an EEG collection unit (100) that measures a user's brainwaves, a sensor unit (200) including at least one sensor that collects the user's physical response data, and a calculation unit (300) that evaluates the user's cognitive function based on the brainwave and physical response data.

[0052] In one embodiment, the sensor unit (200) may include at least one of a gyro sensor (210) for measuring head movement, an eye blink sensor (220) for measuring eye blinking, and a temperature sensor (230) for measuring body temperature, but is not limited thereto, and may include various sensors for measuring the user's physical response data.

[0053] In one embodiment, the gyro sensor (210) may be implemented as at least one of an optical gyro sensor, a micro-electromechanical systems (MEMS) gyro sensor, an inertial measurement unit, and a wearable device equipped with a gyro sensor.

[0054] In one embodiment, the eye blink sensor (220) may be implemented as at least one of a skin-attached sensor that measures physical deformation, an infrared (IR) sensor, a camera vision sensor, and an electrooculogram (EOG) sensor. A skin-attached sensor according to one embodiment of the present disclosure will be described below with reference to FIGS. 3 to 5 .

[0055] For example, an infrared sensor measures blinks by emitting infrared light from one side of a pair of eyewear, reflecting it off the glasses, and then detecting the intensity of the reflected infrared light through an infrared receiver located on the opposite side or in a suitable location after it has been reflected again by the eyeball. This utilizes the principle that infrared transmittance increases and reflectivity decreases when the eyes are open compared to when they are closed.

[0056] For example, an electrooculogram (EOG) sensor uses electrooculogram technology to measure the potential between the cornea and the retina. It mainly places pairs of electrodes above and below the eye or on the left and right sides of the eye, and can measure eye blinking using the information obtained from the electrodes.

[0057] For example, a camera vision sensor can capture eye movements through a camera and measure eye blinks using camera vision technology.

[0058] In one embodiment, the temperature sensor (230) may be implemented as at least one of a contact thermometer and a non-contact thermometer.

[0059] In one embodiment, the EEG collection unit (100), the sensor unit (200), and the calculation unit (300) may be implemented in a single device. For example, the multi-sensor and brainwave-based cognitive function monitoring system (10) may be implemented in a single device, and the EEG collection unit (100), the sensor unit (200), and the calculation unit (300) may be internally connected. In another embodiment, the EEG collection unit (100), the sensor unit (200), and the calculation unit (300) may be implemented as separate devices, and may be connected therebetween via wired or wireless communication. In another embodiment, the EEG collection unit (100) and the sensor unit (200) may be implemented as a single device, and the calculation unit (300) may be implemented as a separate device, and data may be transmitted and received between these two devices via wired or wireless communication.

[0060] FIG. 2 is a diagram illustrating a state of use of a multi-sensor and brainwave-based cognitive function monitoring system according to one embodiment of the present disclosure.

[0061] Referring to FIG. 2, the EEG collection unit (100) and the sensor unit (200) may be implemented as a wearable device that a user can easily wear, and the operation unit (300) may be implemented as a separate device. For example, a wearable device including the EEG collection unit (100) and the sensor unit (200) may be wirelessly connected to a device including the operation unit (300). In one embodiment, data collected by the EEG collection unit (100) and the sensor unit (200) may be provided to the operation unit (300) via wireless communication. In one embodiment, data collected by the EEG collection unit (100) and the sensor unit (200) may be provided to the operation unit (300) after undergoing a signal processing process. For example, a wearable device including an EEG collection unit (100) and a sensor unit (200) may be equipped with a signal processing unit, and may pre-process data collected from the EEG collection unit (100) and the sensor unit (200) using the signal processing unit before transmitting the data to the operation unit (300).

[0062] In another embodiment, the EEG collection unit (100), the sensor unit (200), and the operation unit (300) may be implemented as a single wearable device. In yet another embodiment, the EEG collection unit (100), the sensor unit (200), and the operation unit (300) may each be implemented as separate devices. However, the present invention is not limited thereto, and the number of devices constituting the multi-sensor and brainwave-based cognitive function monitoring system is not limited, and some of the functions of the EEG collection unit (100), the sensor unit (200), and the operation unit (300) may be implemented in different devices. For example, some of the functions of the operation unit (300) may be implemented in a first device, and the remaining functions may be implemented in a second device.

[0063] FIG. 3 is a block diagram of an eye blink measurement sensor according to an embodiment of the present disclosure, and FIG. 4 is an exemplary diagram illustrating a strain gauge according to an embodiment of the present disclosure.

[0064] Referring to FIG. 3, the skin-attached eye blink sensor (220) may include a strain gauge (226), a conversion unit (227), a filter unit (228), and a communication unit (229).

[0065] In one embodiment, the strain gauge (226) is a gauge that measures the deformation (i.e., strain) of a structure, and can be attached to the surface of the structure to measure the state and amount of deformation of the structure. The strain gauge (226) used in one embodiment of the present disclosure is an electrical strain gauge, and when the structure undergoes deformation, the electrical resistance of the strain gauge attached to the surface changes, and thereby the strain rate of the structure can be measured.

[0066] Referring to FIG. 4(a), a typical strain gauge (226) may include a grid (221) in which long, thin conductive strips are arranged in parallel lines in a zigzag pattern so that the electrical resistance changes when the structure is deformed by being stretched or compressed, an electrode portion (222) to which an electrode for measuring the resistance of the grid (221) is connected, and a base (223) which is an insulator that protects and supports the grid (221). Generally, the strain gauge (226) is attached to the surface of a mechanical device or the like and is used to measure the deformation of a metal or the like, so a film having high stiffness (the amount of force required to create the same strain) is used as the base (223), and a special adhesive such as cyanoacrylate or epoxy adhesive may be used for surface adhesion.

[0067] Referring to FIG. 4(b), a strain gauge (226) used in one embodiment of the present disclosure may include a grid (221) whose resistance value changes according to physical deformation, a base (223) that protects and supports the grid, and an adhesive portion (224) that enables the strain gauge to be attached to the skin.

[0068] Since the strain gauge (226) used in one embodiment of the present disclosure is intended to be attached to the sensitive area of ​​the outer corner of the eye of the human body for long-term use, a soft and flexible material may be used to form the base (223) and encapsulate the grid (221). For example, considering human compatibility, appropriate rigidity, process convenience, and cost-effectiveness, the base (223) may be formed of a polymer silicone material that does not cause discomfort when blinking. However, the present invention is not limited thereto, and various soft and flexible materials may be used.

[0069] In one embodiment, the adhesive portion (224) may be made of a medical adhesive component suitable for use on human skin. For example, cyanoacrylate-based adhesives, fibrin glue, gelatin glue, and polyurethane-based adhesives may be used. In one embodiment, the strain gauge (226) may be manufactured with an area of ​​10 mm x 15 mm and a thickness of 0.5 mm to minimize user discomfort while maintaining performance when attached to the outer corner of the eye.

[0070] In one embodiment, the base (223) may be formed by solidifying a material such as silicone in a mold formed by 3D printing, then manufacturing a mask using a UV laser, etc., covering the mask over the silicone, and spraying conductive ink to form a conductive channel. In one embodiment, the base (223) may be formed by concavely forming a groove in the portion corresponding to the grid (221) and the electrode portion (222), and spraying conductive ink into the groove to form a conductive channel. For example, the base (223) may be manufactured by spraying nickel conductive ink after forming a basic shape with PDMS (Polydimethylsiloxane). However, the present invention is not limited thereto, and various materials with soft and flexible properties may be used as the base (223), and various types of conductive ink may be used. In addition, in the process of optimizing the performance of the strain gauge (226), the number of patterns and the thickness of the grid (221) may be different from those in FIG. 4(a) or FIG. 4(b).

[0071] Referring to FIG. 4(b), a strain gauge (226) according to one embodiment of the present disclosure may further include an inlet portion (225) for introducing a wire so that a wire connected to an electrode portion (222) can pass through without being bent. In one embodiment, the inlet portion (225) is formed in a concave groove in the base (223) together with the grid (221) and the electrode portion (222), and a conductive channel may be formed by spraying conductive ink only on the grid (221) and the electrode portion (222).

[0072] In one embodiment, a circuit may be connected to the electrode portion (222) of the strain gauge (226) to derive an electrical signal based on a change in electrical resistance of the strain gauge. For example, the circuit connected to the electrode portion (222) may be a Wheatstone bridge circuit capable of deriving an electrical signal from a change in electrical resistance of the strain gauge. However, this is merely an example, and various operational amplifiers capable of deriving an electrical signal from a change in resistance of the strain gauge may be used.

[0073] In one embodiment, the conversion unit (227) can convert an electrical signal based on a change in electrical resistance of the strain gauge (226) into a digital signal. For example, the conversion unit (227) can be implemented as an analog-digital converter (ADC).

[0074] In one embodiment, the filter unit (228) may include a low-pass filter that passes only low-frequency signals below a preset cutoff frequency among the electrical signals based on the electrical resistance change of the strain gauge (226). For example, the cutoff frequency of the low-pass filter may be set to a value that is 2.5 times the time required to blink an eye after converting it to a frequency. For example, since blinking takes about 0.25 seconds on average, signals above 10 Hz, which is 2.5 times that of about 4 Hz, may be filtered to improve performance. However, this is merely an example, and the cutoff frequency of the low-pass filter may be appropriately determined depending on the user or the data to be derived.

[0075] In one embodiment, the filter unit (228) may be implemented as a digital filter. For example, the output signal of the strain gauge (226) may be converted into a digital signal through the conversion unit (227) and then passed through the digital filter (228) that operates as a low-pass filter. In another embodiment, the filter unit (228) may be implemented as an analog filter. For example, the output signal of the strain gauge (226) may be processed by an analog filter that is a low-pass filter and then converted into a digital signal through the conversion unit (227).

[0076] In one embodiment, the communication unit (229) can transmit a digital signal generated through the strain gauge (226), the conversion unit (227), and the filter unit (228) to the receiving module of the eye blink data analysis system. In one embodiment, the communication unit (229) can utilize wireless communication. For example, the wireless communication can be implemented using at least one or more communication technologies such as wireless LAN (e.g., Wi-Fi), short-range communication (e.g., Bluetooth, Zigbee, infrared communication, etc.), and mobile communication.

[0077] In one embodiment, the skin-attached eye blink measurement sensor (220) may further include a processor (not shown). In one embodiment, the output signal of the strain gauge (226) may be filtered by a filter unit (228), converted into a digital signal through a converter unit (227), and then input to the processor. The processor may then calculate data on the user's eye blinks based on the input signal. For example, the processor may determine whether the user's eyes blink and measure the number of times the user blinks per unit time.

[0078] In one embodiment, the filter unit (228), the conversion unit (227), and the communication unit (229) may be implemented as at least one of a rigid hard PCB or a flexible printed circuit board (fPCB).

[0079] FIG. 5 is a usage state diagram of an eye blink measurement sensor according to one embodiment of the present disclosure.

[0080] Considering the muscles around the eyeball, the muscles around the eyelid (Tarsus) are directly involved in blinking. However, the inner part of the eye adjacent to the bridge of the nose has difficulty in securing space for attaching the blink measurement sensor (220) due to the bridge of the nose, and thus has limitations in use by eyeglass wearers. In addition, the upper part of the eyelid has limitations in use as well because the eyebrows are close. Ultimately, considering the ease of attaching the blink measurement sensor (220), the tail of the eye is the most appropriate part.

[0081] Additionally, the tail of the eye is anatomically related to the blinking motion and is a location where space is secured for proper attachment of the blinking measurement sensor (220).

[0082] Moreover, since the strain gauge (226) measures deformation in only one direction, it must be attached in accordance with the principal direction of deformation. The outer corner of the eye is an area where the principal direction of deformation is formed left and right, where deformation is concentrated in that direction, and where sufficient deformation is exhibited when blinking, making it an appropriate area from a mechanical engineering perspective. The shape of each person's eyes varies, and the principal direction of deformation in the area under the eyelid varies slightly depending on the shape and curvature of the eye. However, the outer corner of the eye has the principal direction of deformation formed left and right, so it has the advantage of being easy to universally apply to all people. Ultimately, in one embodiment of the present disclosure, the skin-attached eye blink sensor (220) can be attached near the outer corner of the user's eye to measure eye blinking, and can collect data on whether or not the eye blinks, as well as the speed and intensity of the blinking, in real time.

[0083] Figure 6 is a block diagram of an operation unit in one embodiment of the present disclosure.

[0084] Referring to FIG. 6, the operation unit (300) may include a signal processing module (310), a synchronization module (320), an analysis module (330), and a judgment module (340).

[0085] In one embodiment, the signal processing module (310) may include a first signal processing module (311) that processes brain wave data measured by the EEG collection unit (100), and a second signal processing module (312) that processes body response data measured by the sensor unit (200).

[0086] In one embodiment, the first signal processing module (311) can filter brain wave data measured by the EEG collection unit (100) and classify the brain wave data by brain wave region. In one embodiment, the filtering of the brain wave data can be implemented by at least one of a notch filter and a bandpass filter. For example, a notch filter for removing power line noise of 50 to 60 Hz from the brain wave data and a bandpass filter for removing high / low frequency noise can be used. In one embodiment, the brain wave data from which noise has been removed is digitally sampled to prevent phase distortion, and then passed through a digital filter to classify the brain wave region into bands such as commonly known alpha waves (8-12 Hz) and beta waves (12-30 Hz) and utilize them for analysis.

[0087] In one embodiment, the second signal processing module (312) can generate an event when the body response data measured by the sensor unit (200) changes beyond a preset range.

[0088] In one embodiment, the synchronization module (320) can synchronize the time of the body response data measured by the sensor unit (200) and the brain wave data measured by the EEG collection unit (100). For example, when the EEG collection unit (100) and the sensor unit (200) are implemented as an integrated device, the time of the two data can be easily synchronized because the same measurement time is used within the one device. As another example, when the EEG collection unit (100) and the sensor unit (200) are implemented as separate devices, the time of the two data can be synchronized by performing a process of synchronizing the time between the two devices before measurement or by calculating the time difference between the two devices after measurement and correcting it.

[0089] In one embodiment, the analysis module (330) can analyze the change pattern of brain waves before and after the occurrence of an event derived from the body response data measured by the sensor unit (200). In one embodiment, the occurrence of an event can be defined from the body response data measured by the sensor unit (200). For example, when the rate of change over time of the root mean square (RMS) of the three-axis angular velocity of the gyro sensor is greater than a certain threshold, the corresponding point in time can be defined as the event occurrence point in which a sudden head movement occurs. As another example, a change in eye blink intensity or a change in body temperature greater than a predetermined standard can be defined as the event occurrence point.

[0090] In one embodiment, the judgment module (340) can evaluate the user's cognitive function by analyzing brain wave data before and after the event occurrence time defined by the analysis module (330). For example, the judgment module (340) can analyze the user's alpha, beta, gamma, theta, and delta wave brain waves obtained by digitally filtering brain wave data measured before and after the event occurrence. In one embodiment, the judgment module (340) can analyze by utilizing power data of each brain wave. Here, power is defined as the square of the brain wave voltage value (V) as commonly used in electrical engineering. 2 )am.

[0091] FIG. 7 is a flowchart illustrating a multi-sensor and brainwave-based cognitive function monitoring method according to one embodiment of the present disclosure.

[0092] Referring to Fig. 7, the user's brain waves and physical response data can be measured through the EEG device (100) and the sensor unit (200), and the user's cognitive function can be evaluated based on the measured physical response data and brain waves.

[0093] At S410, the EEG device (100) can measure the user's brain waves. In one embodiment, after S410, the first signal processing module (311) can filter the user's brain wave data measured by the EEG collection unit (100) and classify the brain wave data by frequency domain. For example, noise in the brain wave data can be removed, the noise-removed brain wave data can be digitally sampled, and the brain wave data can be classified by frequency domain.

[0094] In S420, the sensor unit (200) can measure the user's physical response data. In one embodiment, the method may include at least one of a step of measuring head movement by a gyro sensor (210), a step of measuring eye blinking by an eye blink sensor (220), and a step of measuring body temperature by a temperature sensor (230), but is not limited thereto, and may include various steps of measuring the user's physical response data.

[0095] In S430, the synchronization module (320) can synchronize the time of the user's physical response data measured by the sensor unit (200) and the user's brain wave data measured by the EEG collection unit (100). By synchronizing the time of the physical response data and the brain wave data, it is possible to analyze the brain wave change pattern before and after the occurrence of the event derived from the physical response data in S450.

[0096] In S440, the second signal processing module (312) can generate an event when the body response data measured by the sensor unit (200) changes beyond a set range. For example, the timing of a rapid head movement, a change in body temperature, or a change in eye blinking can be generated as an event. Details regarding this will be described later with reference to FIG. 8.

[0097] In S450, the analysis module (330) can analyze the change pattern of brain waves before and after the occurrence of an event derived from the body response data measured by the sensor unit (200).

[0098] In one embodiment, the step of analyzing the change pattern of brain waves may further include at least one of a step of confirming the time point at which a brain wave response occurs, a step of confirming the amount of change in brain wave data by region, and a step of confirming the recovery speed of brain waves by region.

[0099] In one embodiment, the step of analyzing the brain wave change pattern may further include a step of calculating a weighted sum of brain wave data, and a step of adjusting the weighted sum according to the standard deviation of data for each brain wave region.

[0100] In one embodiment, the step of analyzing the brain wave change pattern may further include a step of calculating a weighted sum of brain wave data, and a step of adjusting the weighted sum using the deviation between the left and right hemispheres of the brain.

[0101] In S460, the judgment module (340) can evaluate cognitive function based on the change pattern of brain waves before and after the event occurrence analyzed by the analysis module (330). For example, a user with good cognitive function before and after an event such as a case where concentration is broken due to a sudden head movement shows a pattern of greater change in beta and gamma waves, and conversely, a user with poor cognitive function shows a pattern of greater change in alpha and theta waves. In addition, if the recovery speed to the original brain waves before and after the event is fast, the cognitive function can be determined to be good. At this time, the recovery speed to the original brain waves can be determined by utilizing the reciprocal of the time taken for the change in brain waves after the event occurrence to decrease and approach within a 10% error of the value before the event.

[0102] Figure 8 is a flowchart illustrating an event derivation method according to one embodiment of the present disclosure.

[0103] In S441, a normal range of data can be set to derive events from the user's physical response data. In one embodiment, the normal range can be set to a specific value externally in advance, or can be automatically set during the analysis of the user's physical response data.

[0104] In S442, the sensor unit (200) can measure the user's physical response data. In one embodiment, the method may include at least one of a step of measuring head movement by a gyro sensor (210), a step of measuring eye blinking by an eye blink sensor (220), and a step of measuring body temperature by a temperature sensor (230), but is not limited thereto, and may include various steps of measuring the user's physical response data.

[0105] In S443, the second signal processing module (312) can compare the normal range set in S441 with the body response data measured by the sensor unit (200). For example, the body response data for the rate of change of the root mean square (rms) of the three-axis angular velocity of the gyro sensor over time, body temperature, and the interval, speed, and intensity of eye blinking can be compared with the normal range.

[0106] In S444, the second signal processing module (312) can generate an event when the body response data measured by the sensor unit (200) changes beyond a set range.

[0107] In S445, the second signal processing module (312) can record the occurrence time of the derived event. The recorded occurrence time of the event can serve as a reference point for analyzing brain wave change patterns.

[0108] FIG. 9 is a graph illustrating a method of analyzing changes in a gyro sensor and brain waves measured by a method according to one embodiment of the present disclosure.

[0109] Fig. 9(a) is the user's physical response data measured by the gyro sensor measured in the sensor unit (200), and Fig. 9(b) is the brain wave data measured from the EEG collection unit (100) that is time-synchronized with the physical response data.

[0110] In one embodiment, the computation unit (300) can define an event from body response data. For example, referring to FIG. 9(a), the point in time at which a sudden change is observed in body response data measured by the gyro sensor can be specified as the point in time at which an event occurs.

[0111] According to one embodiment, the operation unit (300) can evaluate the user's cognitive function by analyzing at least one of the absolute size of the brain wave response, the brain wave response time, and the brain wave recovery time in the user's brain waves after an event occurs.

[0112] According to one embodiment, the calculation unit (300) can evaluate the user's cognitive function by calculating the absolute magnitude of the user's brainwave response to the occurrence of an event. For example, the calculation unit (300) can use the value of a positive peak (P300) that appears about 300 ms after the occurrence of an event as the absolute magnitude of the brainwave response. As another example, the calculation unit (300) can use the difference between the value of a positive peak (P300) that appears about 300 ms after the occurrence of an event and the value of a negative peak (N400) that appears about 400 ms after the occurrence of the event as the absolute magnitude of the brainwave response. In one embodiment, the user's cognitive function can be evaluated by utilizing the fact that a person with good cognitive function shows a greater change in beta waves and gamma waves before and after an event, and a person with low cognitive function shows a greater change in alpha waves and theta waves.

[0113] In one embodiment, the computation unit (300) can evaluate a user's cognitive function by measuring the user's brainwave response time to an event occurrence. For example, referring to FIG. 9(b), the user's cognitive function can be evaluated by measuring the response time up to the P300 time point. As another example, referring to FIG. 9(b), the user's cognitive function can be evaluated by measuring the response time up to the N400 time point.

[0114] According to one embodiment, the calculation unit (300) can evaluate the user's cognitive function by analyzing the user's brain wave recovery time after the occurrence of an event. Referring to FIG. 9(b), the user's cognitive function can be evaluated using the inverse of the time it takes for the user's brain wave fluctuations to decrease after the occurrence of an event and to approach the pre-event value within a predetermined range. For example, the user's cognitive function can be evaluated using the inverse of the time it takes for the user's brain wave to approach the pre-event value within a 10% error after the occurrence of an event. In one embodiment, the user's cognitive function can be evaluated by utilizing the fact that people with good cognitive function have short brain wave recovery times.

[0115] An embodiment of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically contains computer-readable instructions, data structures, or program modules, and includes any information delivery media.

[0116] Meanwhile, the embodiments disclosed in this specification and drawings are merely specific examples to easily explain the technical content of the present disclosure and to aid in understanding the present disclosure, and are not intended to limit the scope of the present disclosure. In other words, it will be apparent to those skilled in the art that other modifications based on the technical concept of the present disclosure are possible. Furthermore, each embodiment can be combined and operated as needed. For example, parts of one embodiment of the present disclosure and parts of another embodiment can be combined with each other.

[0117] [Explanation of symbols]

[0118] 10 Multi-sensor and brainwave-based cognitive function monitoring system

[0119] 100 EEG devices

[0120] 200 sensor section

[0121] 210 gyroscope sensor

[0122] 220 Blink Sensor

[0123] 230 temperature sensor

[0124] 300 operation units

[0125] 310 signal processing module

[0126] 311 First signal processing module

[0127] 312 Second signal processing module

[0128] 320 synchronization module

[0129] 330 Analysis Module

[0130] 340 judgment module

Claims

1. In a multi-sensor and brainwave-based cognitive function monitoring system, EEG collection unit that measures the user's brain waves; A sensor unit including at least one sensor for collecting the user's physical response data; and It includes a computational unit that evaluates the user's cognitive function based on the brain wave and physical response data, A multi-sensor and brainwave-based cognitive function monitoring system, wherein the above-mentioned operation unit includes an analysis module that analyzes the change pattern of brain waves before and after the occurrence of an event derived from the body response data measured by the sensor unit.

2. In paragraph 1, The above sensor part, Gyroscope sensor to measure head movements; A blink sensor that measures eye blinks; and A multi-sensor and brainwave-based cognitive function monitoring system comprising at least one temperature sensor for measuring body temperature.

3. In paragraph 1, The above operation unit, A multi-sensor and brainwave-based cognitive function monitoring system further comprising a first signal processing module that processes brainwave data measured by the EEG collection unit.

4. In paragraph 3, The above first signal processing module, A multi-sensor and brainwave-based cognitive function monitoring system that filters the brainwave data and classifies the brainwave data by brainwave region.

5. In paragraph 1, The above operation unit A multi-sensor and brainwave-based cognitive function monitoring system further comprising a second signal processing module that processes the body response data measured by the sensor unit.

6. In paragraph 5, The above second signal processing module, A multi-sensor and brainwave-based cognitive function monitoring system that generates the above event when the above physical response data changes beyond a set range.

7. In paragraph 1, The above operation unit, A multi-sensor and brainwave-based cognitive function monitoring system further comprising a synchronization module that synchronizes the time of the body response data measured by the sensor unit and the brainwave data measured by the EEG collection unit.

8. In paragraph 1, The above operation unit, A multi-sensor and brainwave-based cognitive function monitoring system further comprising a judgment module that evaluates the user's cognitive function based on the brainwave change pattern.

9. In a multi-sensor and brainwave-based cognitive function monitoring method, A step of measuring the user's brain waves through an EEG collection unit; A step of collecting the user's body response data by at least one sensor; and A multi-sensor and brainwave-based cognitive function monitoring method, comprising a step of evaluating the user's cognitive function by analyzing the change pattern of the brain waves before and after the occurrence of an event derived from the body response data measured by at least one sensor.

10. In paragraph 9, The step of collecting the user's body response data by at least one sensor is as follows: The step where the gyroscope sensor measures the movement of the head; A step in which the eye blink sensor measures eye blinks; and A multi-sensor and brainwave-based cognitive function monitoring method comprising at least one step of measuring body temperature by a temperature sensor.

11. In paragraph 9, The step of analyzing the above brain wave change pattern is: A step of checking the response time for each region of the brain wave data for the above event; A step of checking the amount of change in each region of the brain wave data for the above event; and A multi-sensor and brainwave-based cognitive function monitoring method, further comprising at least one step of checking the recovery time for each region of the brainwave data after the above event.

12. In paragraph 9, The step of analyzing the above brain wave change pattern is: A step of calculating a weighted sum of the above brain wave data; and A multi-sensor and brainwave-based cognitive function monitoring method further comprising a step of adjusting the above weighted sum according to the standard deviation of data for each brainwave region.

13. In paragraph 9, The step of analyzing the above brain wave change pattern is: A step of calculating a weighted sum of the above brain wave data; and A multi-sensor and brainwave-based cognitive function monitoring method further comprising a step of adjusting the above weighted sum using the deviation of the left and right hemispheres of the brain.

14. In paragraph 9, A multi-sensor and brainwave-based cognitive function monitoring method further comprising a step of processing brainwave data measured by the EEG collection unit.

15. In paragraph 14, The step of processing the brain wave data measured by the above EEG collection unit is as follows: a step of filtering the above brain wave data; and A multi-sensor and brainwave-based cognitive function monitoring method comprising at least one step of classifying the brainwave data by brainwave region.

16. In paragraph 9, A multi-sensor and brainwave-based cognitive function monitoring method further comprising a step of processing the body response data measured by at least one sensor.

17. In paragraph 16, The step of processing the above physical response data is: A multi-sensor and brainwave-based cognitive function monitoring method, comprising a step of deriving the event when the above-mentioned physical response data changes beyond a set range.

18. In paragraph 9, A multi-sensor and brainwave-based cognitive function monitoring method further comprising a step of synchronizing the time of the body response data measured by at least one sensor and the brainwave data measured by the EEG collection unit.

19. A program stored on a computer-readable recording medium that causes a computer to execute the method of any one of clauses 9 to 18.

Images