Device and method for generating a driving model for driver proficiency using error monitoring

Abstract

The present invention relates to an apparatus and method for generating a driving model for driver proficiency using error monitoring. An apparatus and method for generating a driving model are disclosed herein. The driving model generation method includes: sensing to collect electroencephalogram signals of a mobile vehicle driver within a predetermined time; determining a driver state by analyzing the electroencephalogram signals collected within the predetermined time; detecting at least one of operation information and driving information of the mobile vehicle based on the determined driver state, and storing the detected information. Herein, the electroencephalogram signals include event-related potentials (ERPs), and determining the driver state includes determining whether the driver is in an unstable state by analyzing the ERPs.

Description

[0001] Cross - reference to related applications

[0002] This application claims the priority and benefit of Korean Patent Application No. 10 - 2019 - 0131118, filed on October 22, 2019, the entire content of which is incorporated herein by reference for all purposes. Technical field

[0003] The present invention relates to a method and apparatus for controlling a mobile tool. More specifically, the present invention relates to a method and apparatus for controlling a mobile tool based on error monitoring. Background art

[0004] The statements in this section merely provide background information related to the present invention and do not constitute prior art.

[0005] As a means of transportation, a vehicle (or mobile tool) is a tool for living in the modern world. In addition, for some people, a vehicle itself can be considered an item with special significance.

[0006] With the progress of technology, the functions provided by vehicles have gradually developed. For example, in recent years, vehicles not only transport passengers to their destinations but also meet the needs of passengers to reach their destinations faster and more safely. In addition, new devices have been added to the vehicle system to meet the aesthetic tastes and comfort of passengers. Moreover, existing devices such as steering wheels, transmissions, and acceleration / deceleration devices have been developed so that more functions can be provided for users.

[0007] Meanwhile, a brain - computer interface or a brain - machine interface is a field that controls a computer or a machine according to a person's intention by using brainwave signals. Event - Related Potential (ERP) is closely related to cognitive functions. Summary of the invention

[0008] An object of the present invention is to provide an apparatus and a method for generating a driving model for driver proficiency.

[0009] Another object of the present invention is to provide an apparatus and a method for modeling the operation or driving condition of a mobile tool based on the driver's state.

[0010] Still another object of the present invention is to provide an apparatus and a method for determining the driver's state by using error monitoring.

[0011] Still another object of the present invention is to provide an apparatus and a method for detecting the operation or driving condition of a mobile tool based on the driver's state.

[0012] The technical object of the present invention is not limited to the above technical object, and those skilled in the art will clearly understand other technical objects not mentioned through the following description.

[0013] The present invention can provide a driving model generation device, which includes a sensor, an analyzer, and a memory. The sensor collects the electroencephalogram signals of the driver of the moving tool within a predetermined time. The analyzer determines the driver state by analyzing the electroencephalogram signals collected within the predetermined time, and detects at least one of the operation information and driving information of the moving tool based on the determined driver state. The memory stores the detected information. The electroencephalogram signals include event-related potentials (ERPs), and the analyzer determines whether the driver is in an unstable state by analyzing the ERPs.

[0014] According to one embodiment, the event-related potential (ERP) may include at least one of an error-related negativity (ERN) and an error positivity (Pe).

[0015] According to one embodiment, the event-related potential (ERP) may further include at least one of a correct-related negativity (CRN) and a correct positivity (Pc).

[0016] According to one embodiment, the analysis may compare the amplitude of the event-related potential collected within the predetermined time with a first threshold.

[0017] According to one embodiment, the first threshold may be determined to be different according to at least one of the type of ERP and the driver from whom the ERP is obtained.

[0018] According to one embodiment, the analyzer may divide the electroencephalogram signals collected within the predetermined time according to each frequency band, and further determine whether the driver is in an unstable state by comparing the amplitude of the electroencephalogram signals of each divided frequency band with a second threshold.

[0019] According to one embodiment, the electroencephalogram signals of each divided frequency band may include at least one of theta waves, alpha waves, and beta waves.

[0020] According to one embodiment, the second threshold may be determined to be different according to the type of the electroencephalogram signals of each frequency band.

[0021] According to one embodiment, when the analysis results of the ERPs and the analysis results of the electroencephalogram signals of each frequency band indicate that the driver is in an unstable state, the analyzer may finally determine that the driver is in an unstable state.

[0022] According to one embodiment, the sensor may further include at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the moving tool.

[0023] According to one embodiment, when the driver state is determined to be an unstable state, the analyzer can detect at least one of the operation information and driving information of the moving tool by using at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the moving tool.

[0024] According to one embodiment, the memory may include at least one of the detected operation information and driving information of the moving tool. The operation information of the moving tool may include the operation of the moving tool and at least one of the position, date, time, speed, and image corresponding to the operation. The driving information of the moving tool may include the driving path of the moving tool and at least one event that the moving tool is undergoing.

[0025] According to one embodiment, a simulation unit for modeling the driving path may also be included, and the simulation unit may model at least one of the driving condition and operation of the moving tool based on whether the driver state is determined to be an unstable state.

[0026] According to one embodiment, for a predetermined operation of the moving tool when the driver state is determined to be an unstable state, the simulation unit may model a virtual driving condition related to the predetermined operation.

[0027] In addition, the present invention may provide a driving model generation method, the method including: a sensing step of collecting electroencephalogram signals of a moving tool driver within a predetermined time; a determining step of determining the driver state by analyzing the electroencephalogram signals collected within the predetermined time; a detecting step of detecting at least one of the operation information and driving information of the moving tool based on the determined driver state; and a storing step of storing the detected information. The electroencephalogram signals include event-related potentials (ERPs), and the determining step includes determining whether the driver is in an unstable state by analyzing the ERPs.

[0028] According to one embodiment, the event-related potential (ERP) may include at least one of an error-related negativity (ERN) and an error positivity (Pe).

[0029] According to one embodiment, the event-related potential (ERP) may further include at least one of a correct-related negativity (CRN) and a correct positivity (Pc).

[0030] According to one embodiment, the analysis may compare the amplitude of the event-related potentials collected within the predetermined time with a first threshold.

[0031] According to one embodiment, the first threshold may be determined to be different according to at least one of the type of ERP and the driver from whom the ERP is obtained.

[0032] According to one embodiment, the determining step may further include: dividing the electroencephalogram signals collected within a predetermined time according to frequency bands, and determining whether the driver is in an unstable state by comparing the amplitudes of the electroencephalogram signals of each divided frequency band with a second threshold.

[0033] According to one embodiment, the electroencephalogram signals of each divided frequency band may include at least one of theta waves, alpha waves, and beta waves.

[0034] According to one embodiment, the second threshold may be determined to be different according to the type of electroencephalogram signals of each frequency band.

[0035] According to one embodiment, the determining step may further include: finally determining that the driver is in an unstable state when the analysis result of the ERP and the analysis results of the electroencephalogram signals of each frequency band indicate that the driver is in an unstable state.

[0036] According to one embodiment, the sensing step may further include at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the mobile tool.

[0037] According to one embodiment, the detecting step may include detecting at least one of the operation information and driving information of the mobile tool by using at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the mobile tool.

[0038] According to one embodiment, the storing step may include storing at least one of the detected operation information and driving information of the mobile tool. The operation information of the mobile tool may include at least one of the operation of the mobile tool and the position, date, time, speed, and image corresponding to the operation, and the driving information of the mobile tool may include the driving path of the mobile tool and at least one event that the mobile tool is undergoing.

[0039] According to one embodiment, a simulation step of modeling the driving path may be further included. The simulation step may include modeling at least one of the driving condition and operation of the mobile tool based on whether the driver state is determined to be an unstable state.

[0040] According to one embodiment, for a predetermined operation of the mobile tool when the driver state is determined to be an unstable state, the simulation step may include modeling a virtual driving condition related to the predetermined operation.

[0041] The features of the above brief summary of the present invention are merely exemplary aspects of the present invention described in detail below and do not limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To better understand the present invention, various embodiments of the present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0043] Figure 1 is a diagram showing the general waveform of the ERN showing one embodiment of the present invention;

[0044] Figure 2 is a diagram showing the general waveforms of the ERN and Pe according to one embodiment of the present invention;

[0045] Figure 3 is a diagram showing the deflection characteristics of the Pe according to another embodiment of the present invention;

[0046] Figure 4A and Figure 4B are diagrams respectively showing the measurement areas of the ERP and Pe of one embodiment of the present invention;

[0047] Figure 5 is a diagram showing the general waveforms of the ERN and CRN according to one embodiment of the present invention;

[0048] Figure 6 is a diagram showing the EEG measurement channels corresponding to the cerebral cortex region according to one embodiment of the present invention;

[0049] Figure 7 is a block diagram showing the configuration of a driving model generation device using error monitoring according to one embodiment of the present invention.

[0050] Figure 8 is a diagram showing the measurement time range when the target ERP is the ERN and Pe according to one embodiment of the present invention.

[0051] Figure 9 is a diagram showing the process of comparing the target ERP with a predetermined threshold when the target ERPs are the ERN and Pe respectively according to one embodiment of the present invention.

[0052] Figure 10A and Figure 10B is a diagram showing the process of comparing the brain wave signals of each frequency band with a predetermined threshold according to one embodiment of the present invention.

[0053] Figure 11 is a block diagram showing the configuration of a driving model generation device using error monitoring according to another embodiment of the present invention.

[0054] Figure 12 FIG. Figure 12 is a flowchart showing a driving model generation method using error monitoring according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following description is merely exemplary in nature and is not intended to limit the present invention, its application, or uses. It should be understood that throughout the drawings, corresponding reference numerals denote the same or corresponding components and features.

[0056] Exemplary embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily understand and implement the apparatus and methods provided by the present invention in conjunction with the drawings. However, the present invention can be implemented in various embodiments, and the scope of the present invention should not be construed as being limited to the exemplary embodiments.

[0057] When describing the embodiments of the present invention, well-known functions or configurations may obscure the spirit of the present invention, and thus they will not be described in detail.

[0058] In the present invention, it will be understood that when an element is referred to as being "connected to", "coupled to", or "combined with" another element, it can be directly connected to, coupled to, or combined with the other element, or there may be an intermediate element therebetween. It will be further understood that when used in the present invention, the terms "comprising", "including", "having", etc. indicate the presence of the described features, values, steps, operations, elements, components, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, values, steps, operations, elements, components, and / or combinations thereof.

[0059] It will be understood that although the terms "first", "second", etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish one element from another, and do not indicate the order or priority between the elements. For example, the first element discussed below may be referred to as the second element without departing from the teachings of the present invention. Similarly, the second element may also be referred to as the first element.

[0060] In the present invention, different elements are named to clearly describe the features of various elements, and it does not mean that these elements are physically separated from each other. That is, a plurality of different elements can be combined into a single hardware unit or a single software unit. Conversely, one element can be implemented by a plurality of hardware units or software units. Therefore, although not specifically described, the integrated form of various elements or the separated form of one element can fall within the scope of the present invention. Moreover, terms such as "unit" or "module" should be understood as units that process at least one function or operation and can be implemented in a hardware manner (e.g., a processor), a software manner, or a combination of a hardware manner and a software manner.

[0061] In the present invention, all constituent elements described in various forms should not be construed as essential elements, and some constituent elements may be optional elements. Therefore, embodiments configured by various subsets of constituent elements in a certain form may also fall within the scope of the present invention. In addition, embodiments configured by adding one or more elements to various elements also fall within the scope of the present invention.

[0062] As the electroencephalogram activity of neurons constituting the brain, electroencephalogram signals (or brain signals, brainwaves) represent biological signals that directly and indirectly reflect a person's conscious or unconscious state. Electroencephalogram signals can be measured in each region of the human scalp, with the frequency of their wavelengths mainly being 30 Hz or lower and the potential difference being several microvolts. Various waveforms may appear depending on brain activity and state. Research on interface control using electroencephalogram signals according to a person's intention is underway. Electroencephalogram signals can be obtained by using electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), or functional near-infrared spectroscopy (fNIRS). EEG uses electrical signals caused by brain activity, MEG uses magnetic signals that occur together with electrical signals, and fMRI or fNIRS uses changes in oxygen saturation in the blood. Although fMRI and fNIRS are useful techniques for measuring brain activity, generally, fMRI has a lower temporal resolution, while fNIRS has a lower spatial resolution. Due to these limitations, EEG signals are widely used due to their portability and temporal resolution.

[0063] Electroencephalogram signals change spatially and temporally according to brain activity. Since electroencephalogram signals are generally difficult to analyze and their waveforms are not easily visually analyzed, various processing methods have been proposed.

[0064] For example, according to the number of oscillations (frequency), electroencephalogram signals can be divided based on frequency bands (power spectrum division). This division regards the measured electroencephalogram signals as the linear sum of simple signals at each specific frequency, decomposes the signals into each frequency component, and indicates the corresponding amplitude. By using preprocessing, Fourier transform to the frequency domain, and a band-pass filter (BPF) commonly used for noise cancellation, electroencephalogram signals at each frequency can be obtained.

[0065] More specifically, according to frequency bands, brain waves can be divided into delta waves, theta waves, alpha waves, beta waves, and gamma waves. Delta waves are brain waves with a frequency of 3.5 Hz or less and an amplitude of 20 μV to 200 μV, which mainly appear in normal deep sleep or in newborns. In addition, delta waves may increase as our understanding of the physical world decreases. Generally, theta waves are brain waves with a frequency of 3.5 Hz to 7 Hz, which mainly appear in a state of emotional stability or during sleep.

[0066] In addition, theta waves are mainly generated in the parietal cortex and the occipital cortex and may appear during the calm concentration of recollection or meditation. Additionally, the amplitude of theta waves may increase when stress appears. Generally, alpha waves are brain waves with a frequency of 7 Hz to 12 Hz, which mainly appear in a relaxed and comfortable state. Additionally, alpha waves are usually generated in the occipital cortex during rest and may decrease during sleep. Generally, beta waves are brain waves with a frequency of 12 Hz to 30 Hz, which mainly appear in a tolerable state of tension or during a certain degree of attention. In addition, beta waves are mainly generated in the frontal cortex and are related to the waking state, or the concentration of brain activity, pathological phenomena, and drug effects. Beta waves may appear in a wide area of the entire brain. Additionally, specifically, beta waves can be divided into SMR waves with a frequency of 12 Hz to 15 Hz, mid-beta waves with a frequency of 15 Hz to 18 Hz, and high-beta waves with a frequency of 20 Hz or more. Since beta waves seem stronger under stress such as anxiety and tension, they are called stress waves. Gamma waves are brain waves that usually have a frequency of 30 Hz to 50 Hz, which mainly appear in a state of strong excitement or during the process of high-level cognitive information processing. In addition, gamma waves may appear during the state of conscious awakening and during the REM sleep process and may overlap with beta waves.

[0067] Each brain wave signal depending on the frequency band is related to a specific cognitive function. For example, delta waves are related to sleep, theta waves are related to working memory, and alpha waves are related to attention or suppression. Therefore, the nature of the brain wave signals in each frequency band selectively shows a specific cognitive function. Additionally, in each measurement site on the head surface, the brain wave signals in each frequency band may show some different appearances. The cerebral cortex can be divided into the frontal cortex, the parietal cortex, the temporal cortex, and the occipital cortex. These sites may have some different functions. For example, the occipital cortex corresponding to the back of the head has the primary visual cortex and can therefore mainly process visual information. The parietal cortex located near the top of the head has the somatosensory cortex and can therefore process motor / sensory information. Additionally, the frontal cortex can process information related to memory and thinking, and the temporal cortex can process information related to hearing and smell.

[0068] Meanwhile, for another example, electroencephalogram (EEG) signals can be analyzed by utilizing event-related potential (ERP). ERP is the electroencephalogram change related to external stimuli or internal mental processes. ERP refers to the signal including the electroencephalogram activity of the brain caused by a stimulus within a certain period of time after the appearance of the stimulus, and the stimulus includes specific information (such as images, voices, sounds, execution commands, etc.).

[0069] To analyze ERP, a process of separating the signal from the noise is required. The mean method can be mainly used. In particular, by taking the mean of the electroencephalogram measured based on the stimulus start time, the electroencephalogram unrelated to the stimulus can be removed, and only the relevant potential, that is, the brain activity usually related to stimulus processing, can be selected.

[0070] Since ERP has a high temporal resolution, it is closely related to the study of cognitive functions. ERP is an electrical phenomenon induced by external stimuli or related to internal states. According to the type of stimulus, ERP can be divided into auditory-related potential, visual-related potential, somatosensory-related potential, and olfactory-related potential. According to the nature of the stimulus, ERP can be divided into exogenous ERP and endogenous ERP. The waveform of exogenous ERP is determined by external stimuli, is related to automatic processing, and mainly appears in the initial stage of being stimulated. For example, exogenous ERP is brainstem potential. On the other hand, endogenous ERP is determined by internal cognitive processes or mental processes or states, is independent of stimuli, and is related to "controlled processes". For example, endogenous ERP is P300, N400, P600, contingent negative variation (CNV), etc.

[0071] The name of the ERP peak usually includes polarity and latency, and each peak of the signal has its own definition and meaning. For example, the positive potential is P, the negative potential is N, and P300 represents the positive peak measured about 300 ms after the start of the stimulus. In addition, 1, 2, 3 or a, b, c, etc. are applied according to the appearance order. For example, P3 represents the third positive potential in the waveform after the start of the stimulus.

[0072] Hereinafter, various ERPs will be described.

[0073] For example, N100 is related to the response to unpredictable stimuli.

[0074] Mismatch negativity (MMN) can be generated not only by focused stimuli but also by non-focused stimuli. MMN can be used as an indicator of whether sensory memory (echoic memory) is operating before initial attention. P300 described below appears during the process of attention and judgment, and analyzes MMN as the process occurring in the brain before attention.

[0075] For another example, N200 (or N2) is mainly generated based on visual and auditory stimuli and is related to short-term or long-term memory together with P300 described below. Short-term or long-term memory is the type of memory after attention.

[0076] For another example, P300 (or P3) mainly reflects the attention to stimuli, stimulus recognition, memory search, and reduction of uncertainty, and is related to the perceptual decision of distinguishing external stimuli. Since the generation of P300 is related to cognitive functions, P300 will be generated regardless of the type of stimuli that appear. For example, P300 can be generated in auditory, visual, and somatic stimuli. P300 is widely used in the research of brain-computer interfaces.

[0077] For another example, N400 is related to language processing and is elicited when a sentence with semantic errors or an auditory stimulus appears. In addition, N400 is related to the memory process and can reflect the process of retrieving or searching for information from long-term memory.

[0078] For another example, as an indicator representing the reconstruction or recovery process, P600 is related to the process of more accurately processing stimuli based on the information stored in long-term memory.

[0079] For another example, CNV refers to the potential that appears within 200 ms to 300 ms or even several seconds in the subsequent stage. It is also called slow potential (SP) and is related to anticipation, preparation, mental priming, association, attention, and motor activity.

[0080] For another example, ERN (error-related negativity) or Ne (error negativity) is an event-related potential (ERP) generated by a mistake or error. It may occur when a subject makes a mistake in a sensorimotor task or a similar task. More specifically, when a subject recognizes a mistake or error, ERN is generated, and its negative peak mainly appears in the frontal and central regions for about 50 ms to 150 ms. In particular, it may appear in situations where mistakes related to motor responses may occur and can also be used to indicate negative self-judgment.

[0081] Hereinafter, the main characteristics of ERN will be described in more detail.

[0082] Figure 1 is a diagram showing the typical waveform of ERN according to an embodiment of the present invention.

[0083] Reference Figure 1, negative potential values are depicted above the horizontal axis, and positive potential values are depicted below the horizontal axis. Additionally, it can be confirmed that an ERP with a negative peak is generated within a predetermined time range after the start of the response to any movement. Here, the response can indicate a situation where a mistake or error occurs (error response). Additionally, the predetermined time range can be approximately 50 ms to 150 ms. Alternatively, the predetermined time range can be approximately 0 to 100 ms. Furthermore, in the case of a correct response, the generated ERP has a relatively smaller negative peak than the ERN.

[0084] As an ERP of the initial negative wave, the ERN is time-locked until a response error occurs. Additionally, it is known that the ERN reflects the reinforcement activity of the dopaminergic system related to behavioral monitoring. The ERN includes the fronto-striatal loop, which includes the rostral cingulate area. At the same time, dopamine is related to the brain reward system that usually forms specific behaviors and motivates people, thereby providing a feeling of pleasure and fulfillment. When a behavior of repeatedly obtaining appropriate rewards is performed, it is learned as a habit. Additionally, more dopamine is released through emotional learning, and new behaviors are attempted due to the release of dopamine. Therefore, reward-driven learning is called reinforcement learning.

[0085] Furthermore, the ERN may be generated within 0 to 100 ms after the start of an error response caused during the execution of interference tasks (e.g., Go-noGo task, Stroop task, Flanker task, and Simon task) by the frontal cortex.

[0086] Additionally, together with the CRN described below, it is known that the ERN reflects the normal behavioral monitoring system that can distinguish correct behaviors from incorrect behaviors.

[0087] Additionally, the fact that the ERN reaches its maximum amplitude at the frontal cortex electrodes reflects that the intracranial generator is located in the rostral cingulate area or the dorsal anterior cingulate cortex (dACC) area.

[0088] Additionally, the ERN may show amplitude changes according to the negative emotional state.

[0089] Additionally, the ERN can be reported even in the case of behavioral monitoring based on external evaluation feedback (different from internal motor expression), and it can be classified into the FRN described below.

[0090] Additionally, the ERN can be generated not only when a mistake or error is recognized but also before a mistake or error is recognized.

[0091] Additionally, the ERN can be generated not only as a response to one's own mistake or error but also as a response to another person's mistake or error.

[0092] In addition, the ERN can be generated not only as a response to a mistake or error, but also as a response to anxiety or stress about a scheduled task to be performed or a subject.

[0093] In addition, when a larger ERN peak is obtained, it can be considered that it reflects a more serious mistake or error.

[0094] Meanwhile, for another example, as an event-related potential (ERP) generated after the ERN, Pe (error positivity) is an ERP with a positive value, which is mainly generated at the frontal cortex electrodes within about 150 ms to 300 ms after a mistake or error. It is known that Pe is a response to being aware of a mistake or error and paying more attention. In other words, Pe is related to an indicator of the conscious error information processing process after error detection. The ERN and Pe are called ERPs related to error monitoring.

[0095] Hereinafter, the main characteristics of Pe will be described in more detail.

[0096] Figure 2 is a diagram showing the typical waveforms of the ERN and Pe according to another embodiment of the present invention.

[0097] Reference Figure 2 , the negative potential value is shown above the positive potential value. In addition, it can be confirmed that an ERP (i.e., ERN) with a negative peak is generated within a first predetermined time range after the start of the response to any movement. Here, the response can represent a situation where a mistake or error occurs (error response). In addition, the first predetermined time range can be about 50 to 150 ms. Alternatively, the first predetermined time range can be about 0 to 200 ms.

[0098] In addition, it can be confirmed that an ERP (i.e., Pe) with a positive peak is generated within a second predetermined time range after the start of the ERN. In addition, the second predetermined time range can be about 150 ms to 300 ms after the start of the error. Alternatively, the second predetermined time range can represent about 200 ms to 400 ms.

[0099] Figure 3 is a diagram showing the deflection characteristics of Pe according to an embodiment of the present invention.

[0100] Reference Figure 3 , like P3, Pe also has a wide deflection characteristic, and the neural plexus generator includes not only the posterior cingulate cortex region and the insular cortex region, but also more of the anterior cingulate cortex region.

[0101] In addition, Pe can reflect the emotional evaluation of errors and the attention to stimuli like P300. Additionally, ERN represents the conflict between correct and incorrect responses, and Pe is considered a response to being aware of a mistake and paying more attention. In other words, ERN is generated during the detection of stimuli, and Pe is generated according to attention during the processing of stimuli. When ERN and / or Pe respectively have relatively large values, these values are known to be related to adaptive behaviors aimed at responding more slowly and accurately after a mistake.

[0102] Figure 4A and Figure 4B is a diagram showing the measurement regions of ERP and Pe according to an embodiment of the present invention.

[0103] ERN and Pe are referred to as ERPs related to error monitoring. Regarding the measurement regions of ERN and Pe, the maximum negative value and the maximum positive value can generally be measured in the central region. However, there may be some differences depending on the measurement conditions. For example, Figure 4A is the main region for measuring ERN, and the maximum negative value of ERN can generally be measured in the midline frontal lobe or the central region (i.e., FCZ). Additionally, Figure 4B is the main region for measuring Pe, and a relatively larger positive value of Pe can generally be measured in the posterior midline region compared to ERN.

[0104] Meanwhile, for another example, FRN (feedback-related negativity) is an event-related potential (ERP) that is related to error detection based on external evaluation feedback. ERN and / or Pe detect errors based on an internal monitoring process. However, in the case of FRN, when FRN is obtained based on external evaluation feedback, it can operate similarly to the ERN process.

[0105] In addition, FRN and ERN may share many electrophysiological properties. For example, FRN has a negative peak at frontal cortex electrodes approximately 250 ms to 300 ms after the onset of negative feedback and can be generated in the dorsal anterior cingulate cortex (dACC) region like ERN.

[0106] In addition, like ERN, FRN can reflect the reinforcement learning activity of the dopaminergic system. Additionally, FRN generally has a larger negative value than positive feedback and can have a larger value for unforeseen situations than for predictable outcomes.

[0107] For another example, CRN (correct-related negativity) is an ERP generated by correct trials and is a negative value smaller than ERN. Like ERN, CRN can be generated in the initial latency (e.g., 0 - 100 ms). Figure 5 is a diagram showing the typical waveforms of ERN and CRN according to an embodiment of the present invention.

[0108] For another example, Pc (correct positive wave) is an event-related potential generated after CRN. It is an event-related potential generated within about 150 ms to 300 ms after the start of a correct response. The relationship between CRN and Pc can be similar to the relationship between ERN and Pe.

[0109] In addition, ERPs can be divided into stimulus-locked ERPs and response-locked ERPs. Stimulus-locked ERPs and response-locked ERPs can be divided according to criteria such as the cause of inducing the ERP and the response time. For example, an ERP induced from the moment when a word or a picture is presented to a user from the outside can be called a stimulus-locked ERP. Additionally, for example, an ERP induced from the moment when a user speaks or presses a button can be called a response-locked ERP. Therefore, based on the above criteria, generally, stimulus-locked ERPs are N100, N200, P2, P3, etc., and response-locked ERPs are ERN, Pe, CRN, Pc, FRN, etc.

[0110] In addition, brain waves can be divided according to the manifestation motivation. Brain waves can be divided into spontaneous brain waves (spontaneous potentials) manifested by the user's will and evoked brain waves (evoked potentials) that are naturally manifested according to external stimuli and are independent of the user's will. When a user moves or imagines moving by himself / herself, spontaneous brain waves will be manifested, while evoked brain waves will be manifested through, for example, visual, auditory, olfactory, and tactile stimuli.

[0111] In addition, brain wave signals can be measured according to the international 10-20 system. The international 10-20 system determines the measurement points of brain wave signals according to the relationship between electrode positions and cerebral cortex regions.

[0112] Figure 6 FIG. is a diagram showing EEG measurement channels corresponding to cerebral cortex regions according to an embodiment of the present invention.

[0113] Reference Figure 6 , the brain regions (frontal lobe cortex FP1, FP2; frontal cortex F3, F4, F7, F8, FZ, FC3, FC4, FT7, FT8, FCZ; parietal cortex C3, C4, CZ, CP3, CP4, CPZ, P3, P4, PZ; temporal cortex T7, T8, TP7, TP8, P7, P8; occipital cortex O1, O2, OZ) correspond to 32 brain wave measurement channels. For each channel, data can be obtained and each cerebral cortex region can be analyzed by using the data.

[0114] Figure 7 FIG. is a block diagram showing the configuration of a driving model generation device using error monitoring according to an embodiment of the present invention.

[0115] Traffic accidents are usually caused by human factors, vehicle factors, and road environment factors. Most traffic accidents are caused by human factors. In particular, human factors include driver errors, carelessness, bad driving, drowsiness, drunkenness, and non-compliance with traffic rules, while vehicle factors include functional failures and lack of maintenance. Additionally, road environment factors include road design, lack of maintenance, construction, weather, visibility, and lighting.

[0116] At the same time, human factors such as driver errors, carelessness, and bad driving may be caused by lack of driving experience. For example, when a driver has just obtained his / her driver's license.

[0117] For another example, when a driver drives on a road for the first time, human factors such as bad driving may occur.

[0118] For another example, when a driver performs a specific action to turn left or change lanes, human factors such as bad driving may occur.

[0119] For another example, carelessness of the driver may occur when driving on a familiar road close to home.

[0120] Therefore, when a driver performs an operation or driving condition that he / she is not confident in or feels uneasy about, human factors such as bad driving, errors, and carelessness can be eliminated, thus helping to reduce the likelihood of traffic accidents.

[0121] When a mistake or error is detected, an ERP with response-locking occurs. Additionally, an ERP with response-locking can be generated not only as a response to one's own mistakes or errors, but also as a response to others' mistakes or errors. Additionally, an ERP with response-locking can be generated as a response to a pre-determined task to be performed or the anxiety or stress of the subject. Here, an ERP with response-locking can include ERN, Pe, CRN, Pc, and FRN.

[0122] At the same time, when a driver drives a mobile tool, a drastic change in the driver's brain wave signal may be observed within a predetermined time range. In other words, since brain wave signals are biological signals that directly and indirectly reflect a person's conscious or unconscious state, the anxiety, stress, or feeling of driving mistakes or errors of a driver when driving a mobile tool may lead to a drastic change in the driver's brain wave signal. Although there are multiple reasons, it is mainly when the driver driving the mobile tool feels discomfort or difficulty in terms of movement, operation, driving environment, and road that the feelings of anxiety, stress, or driving mistakes or errors occur. Here, movement, operation, driving environment, and road may vary from driver to driver.

[0123] For example, when changing lanes, a driver may feel anxious.

[0124] For another example, when making a permitted left turn, a driver may find the corresponding operation difficult.

[0125] For another example, when driving on an overpass, a driver may feel stressed.

[0126] For another example, when driving through a sharp turn, a driver may find the corresponding operation difficult.

[0127] For another example, when parking with the front of the vehicle first, a driver may find the corresponding operation difficult.

[0128] In addition, the brain wave signals divided according to frequency bands can reflect the feelings of a driver such as comfort, anxiety, and stress.

[0129] For example, the α wave is usually a brain wave with a frequency of 8 Hz to 12 Hz, which mainly appears in a relaxed and comfortable state. In addition, the α wave mainly occurs in the frontal lobe or occipital lobe and tends to decrease under stress.

[0130] For another example, the β wave is usually a brain wave with a frequency of 13 Hz to 30 Hz, which mainly appears in a tolerable state of tension or when there is a certain degree of attention. In addition, the β wave mainly occurs in the frontal lobe and seems to appear stronger under stresses such as anxiety and tension, so it is called a stress wave.

[0131] For another example, the θ wave is usually a brain wave with a frequency of 3.5 Hz to 7 Hz, which mainly occurs in the parietal lobe and occipital lobe. In addition, the θ wave tends to increase under stress.

[0132] As described above, when no stress occurs, the α wave can be measured as the main signal. On the other hand, when stress occurs, the α wave decreases, and the β wave and / or θ wave can be measured as the main signal.

[0133] Therefore, when analyzing the brain wave signals collected from a driver within a predetermined time, the mental state of the driver can be determined. In other words, when an event-related potential (ERP) locked in response to anxiety, stress, a mistake, or an error is obtained, or when the amplitudes of brain wave signals such as the α wave, β wave, and θ wave are detected at each frequency, it can be determined whether the driver is in a stable state or an unstable state.

[0134] In addition, the mobile tool can include a vehicle, a mobile / transport device, etc.

[0135] In addition, based on the determined driver state, operation information and / or driving information of the mobile vehicle can be detected. Herein, the operation information and / or driving information of the mobile vehicle can be detected by using various sensing devices included in the mobile vehicle.

[0136] In addition, by extracting the amplitudes of the brainwaves collected in the frequency domain within a predetermined time and using a partitioning model such as a support vector machine (SVM), the brainwave signals of each frequency can be partitioned into each frequency interval. The partitioning model is not limited to the above description and can include general methods for partitioning brainwave signals.

[0137] Reference Figure 7 , the driving model generation device 700 may include a sensor 710, an analyzer 720, and / or a memory 730. However, it should be noted that only some components necessary for explaining the present embodiment are shown, and the components included in the driving model generation device 700 are not limited to the above examples. For example, two or more constituent units may be implemented as one constituent unit, and the operations performed in one constituent unit may be partitioned and performed in two or more constituent units. In addition, some constituent units may be omitted, or additional constituent units may be added.

[0138] The driving model generation method and / or device of the present invention can determine the driver state by analyzing the brainwave signals collected for the driver of the mobile vehicle within a predetermined time. In addition, the driving model generation method and / or device can detect the operation information and / or driving information of the mobile vehicle based on the determined driver state. In addition, the driving model generation method and / or device can generate a driving model reflecting the driving characteristics of the driver based on the detected operation information and / or driving information of the mobile vehicle.

[0139] In particular, the driving model generation device 700 of the present invention can collect the brainwave signals of the driver of the mobile vehicle within a predetermined time. In addition, the sensor 710 can operate.

[0140] Herein, the brainwave signals can represent the ERP of each frequency and / or the brainwave signals.

[0141] Herein, the ERP can represent the response-locked ERP. In addition, the response-locked ERP can include ERN, Pe, CRN, Pc, and FRN. In addition, other ERPs obtained after the occurrence of the response (i.e., the start of the response) can be included in addition to ERN, Pe, CRN, Pc, and FRN. In addition, the response-locked ERP can include multiple ERPs.

[0142] In addition, herein, collecting electroencephalogram signals within a predetermined time may include the following process: measuring the electroencephalogram signals of a driver in a mobile tool and detecting ERPs from the measured electroencephalogram signals.

[0143] In Figures 1 to 6 as described above, in response to incorrect behaviors such as errors or mistakes or in response to correct behaviors, ERN, Pe, CRN, Pc, and / or FRN may be generated. Therefore, by utilizing ERPs, the mental state of the driver, such as anxiety, stress, and other feelings of mistakes or errors, can be determined while driving a mobile tool. Additionally, based on this determination, a driver adaptation driving model can be generated.

[0144] For example, when a driver is about to change lanes, ERN and / or Pe may be generated.

[0145] For another example, when a driver is about to pass through an overpass, ERN and / or Pe may be generated.

[0146] For another example, when a driver is on the way, driving on a road for the first time, or experiencing a tense and stressful situation on the road, ERN and / or Pe may be generated.

[0147] For another example, when a driver is about to perform a head-first parking for a mobile tool, ERN and / or Pe may be generated.

[0148] In addition, the predetermined time may represent about 0 to 400 ms after the start of a specific response. In addition, the predetermined time may include the time range during which ERPs locked to the above responses can be obtained. In addition, the predetermined time may vary according to the type of ERPs locked to the response and may have multiple time ranges. For example, a first time range may be given to obtain a first ERP, and a second time range may be given to obtain a second ERP.

[0149] For example, when the first ERP is ERN and the second ERP is Pe, the first time range may be about 0 to 150 ms (which is the main measurement interval for ERN), and the second time range may be about 150 ms to 400 ms (which is the main measurement interval for Pe). Figure 8 is a diagram showing the measurement time ranges when the target ERPs are ERN and Pe according to an embodiment of the present invention. Referring to Figure 8 , ERN can be obtained in the first time range 810, and Pe can be obtained in the second time range 820.

[0150] For another example, when the first ERP is ERN and the second ERP is CRN, the first time range can be approximately 0 to 200 ms (which is the main measurement interval of ERN), and the second time range can be approximately 0 to 200 ms (which is the main measurement interval of CRN).

[0151] In addition, the driver's brain wave signals can be measured and stored continuously or periodically.

[0152] The driving model generation device 700 of the present invention can determine the driver state by analyzing the brain wave signals collected within a predetermined time. In addition, the analyzer 720 can operate.

[0153] Here, the analysis can include a process of comparing the amplitude of the ERP collected within a predetermined time with a predetermined threshold value.

[0154] Here, the threshold value can be a preset value or a value input by the user. In addition, for each driver collecting ERP, the threshold value may have different amplitudes. For example, it can be a value reflecting the characteristics of the driver's brain wave signals. In order to reflect the analysis results of the brain wave signal characteristics, a predetermined learning process can be performed in advance for the ERP characteristics locked in response shown in the driver's brain wave signals. In addition, the threshold value can vary according to the type of ERP and can have multiple values. Figure 9 is a diagram showing a process of comparing a target ERP with a predetermined threshold value when the target ERPs are ERN and Pe respectively according to an embodiment of the present invention. Refer to Figure 9 , in the case of ERN, its amplitude can be compared with the first threshold value 910. In the case of Pe, its amplitude can be compared with the second threshold value 920. Here, the amplitude can represent an absolute value.

[0155] In addition, the analysis can include a process of determining whether the amplitude of the ERP within a predetermined time interval is equal to or greater than a predetermined threshold value (i.e., exceeds the predetermined threshold range). Refer to Figure 9 , in the case of ERN, the amplitude of ERN can be compared with the first threshold value 910 to check whether the amplitude of ERN is equal to or greater than the first threshold value 910 within the third time range 912. In the case of Pe, the amplitude of Pe can be compared with the second threshold value 920 to check whether the amplitude of Pe is equal to or greater than the second threshold value 920 within the fourth time range 922.

[0156] In addition, a process of identifying the start of the ERP by using the time when the characteristics of the brain wave signals appear and / or by using the pattern of the brain wave signals can be performed before the analysis. In addition, the analysis can include a process of extracting the ERP.

[0157] Additionally, the ERP used for analysis can be the statistical value of the ERP collected within a predetermined time. For example, the statistical value can represent an average value, a weighted average value, a maximum value, or a minimum value.

[0158] Additionally, the analysis can determine the driver's state from the point where the amplitude of the ERP or the amplitude of the brain wave signal is equal to or greater than a predetermined threshold.

[0159] Here, the driver's state can include a stable state, an unstable state, etc.

[0160] Since the peak of the ERP usually obtained is larger, the response-locked ERP can be considered as a response to a more serious mistake or error.

[0161] Therefore, as Figure 9 shown, for example, in the case of ERN, when the amplitude of ERN is compared with the first threshold 910 within the third time range 912 and found to be equal to or greater than the first threshold 910, the driver's state can be determined as an unstable state.

[0162] For another example, when the amplitude of ERN is less than the first threshold 910, the driver's state can be determined as a stable state.

[0163] For another example, in the case of Pe, when the amplitude of Pe is equal to or greater than the second threshold 920 within the fourth time range 922, the driver's state can be determined as an unstable state.

[0164] For another example, when the amplitude of Pe is less than the second threshold 920, the driver's state can be determined as a stable state.

[0165] For another example, when the peak of the ERP including ERN and / or Pe is equal to or greater than at least one of the first threshold 910 and the second threshold 920, the driver's state can be determined as an unstable state.

[0166] Additionally, the analysis can be performed by using the brain wave signal template for each driver. Here, the brain wave signal template can represent the brain wave signal in the time domain, which is obtained in advance within a predetermined time range after the start of the response to any movement. The response can include errors, mistakes, correct responses, etc. The prefabricated brain wave signal template can be scaled during the analysis. In other words, the amplitude of the brain wave signal graph can be increased or decreased at a predetermined ratio. For example, the analysis can be performed by comparing the amplitude-time graph waveform of a single ERP and / or multiple ERPs obtained within a predetermined time with a pre-determined brain wave signal template. The brain wave signal template can be obtained through a virtual simulation process or through a predetermined learning process.

[0167] Additionally, the analysis may include a process of comparing the amplitude of the brain wave signals at each frequency collected within a predetermined time with a predetermined threshold value.

[0168] Here, the threshold value may be a preset value or a value input by the user. Additionally, the threshold value may vary according to the brain wave signals at each frequency. Additionally, the threshold value may be a predetermined value of the brain wave signals appearing in the frequency domain or the time domain.

[0169] Figure 10A and Figure 10B is a diagram showing the process of comparing the brain wave signals in each frequency band with a predetermined threshold value according to an embodiment of the present invention.

[0170] For example, referring to Figure 10A , when the amplitude of the θ wave is equal to or greater than the predetermined threshold value, the driver's state may be determined to be an unstable state.

[0171] For another example, when the amplitude of the α wave is less than the predetermined threshold value, the driver's state may be determined to be an unstable state.

[0172] For another example, when the amplitude of the β wave is equal to or greater than the predetermined threshold value, the driver's state may be determined to be an unstable state. Referring to Figure 10B , during the time interval when the amplitude of the β wave is equal to or greater than the predetermined threshold value 1010, the driver's state may be determined to be an unstable state.

[0173] Furthermore, the driver's state may be determined by combining the brain wave signals at each frequency.

[0174] For example, when the ratio of the amplitude of the β wave to the amplitude of the α wave is equal to or greater than the predetermined threshold value, the driver's state may be determined to be an unstable state.

[0175] For another example, when the ratio of the amplitude of the θ wave to the amplitude of the α wave is equal to or greater than the predetermined threshold value, the driver's state may be determined to be an unstable state.

[0176] For another example, when the ratio of the amplitude of the linear combination of the θ wave and the β wave to the amplitude of the α wave is equal to or greater than the predetermined threshold value, the driver's state may be determined to be an unstable state.

[0177] Here, the amplitude of the brain wave signals at each frequency may represent the power of the frequency band within a predetermined range. In other words, the amplitude of the brain wave signals at each frequency may represent, for example, the power obtained by converting the measured signal into a frequency band in the frequency domain through Fourier transform.

[0178] In addition, it is possible to finally determine whether the driver is in a stable state or an unstable state by using the analysis result of the ERP collected within a predetermined time (hereinafter referred to as "the first result"), or the analysis result of the brain wave signals of each frequency collected within a predetermined time (hereinafter referred to as "the second result"), or the above two analysis results.

[0179] For example, when both the first result and the second result are in an unstable state, the driver's state can be finally determined to be in an unstable state.

[0180] For another example, when at least one of the first result and the second result is in an unstable state, the driver's state can be determined to be in an unstable state.

[0181] In addition, the predetermined time of the first result and the predetermined time of the second result can be different from each other.

[0182] The driving model generation device 700 of the present invention can detect the operation and / or driving information of the moving tool based on the driver's state. In addition, the analyzer 720 can operate.

[0183] The detection can be performed from the point where the amplitude of the ERP or the amplitude of the brain wave signal is equal to or greater than a predetermined threshold.

[0184] For example, the driving model generation device 700 can detect the operation information and / or driving information of the moving tool by using at least one sensing device included in the moving tool. In addition, the sensor 710 can include the sensing device.

[0185] In particular, when the driver's state is determined to be in an unstable state, the driving model generation device 700 can detect the operation information and / or driving information of the moving tool by using various sensing devices included in the moving tool.

[0186] Here, the sensing device can include a speed measurement device, an image acquisition device, a wheel monitoring device, and a manipulation device. The manipulation device can monitor the operations of the steering device / accelerator pedal / brake pedal. The image acquisition device can include a black box and a camera module.

[0187] Here, the operation information of the moving tool can include the operations of the moving tool, such as turning left, turning right, changing lanes, making a U-turn, accelerating, and decelerating, as well as the time and position at which each operation occurs.

[0188] In addition, the driving information of the moving tool can include the curve or overpass on which the moving tool is traveling, the parking condition, or the event that the moving tool is undergoing.

[0189] For example, when the driver's state is determined to be in an unstable state, the driving model generation device 700 can detect that the operation of the moving tool is a turning operation such as turning right or left based on the movement of the steering device.

[0190] For another example, when the driver state is determined to be an unstable state, the driving model generation device 700 may detect that the moving tool is accelerating or decelerating based on the movement of the accelerator pedal and / or the brake pedal.

[0191] For another example, when the driver state is determined to be an unstable state, the driving model generation device 700 may detect that the moving tool is changing lanes based on the movement of the steering device or the turn signal.

[0192] For another example, when the driver state is determined to be an unstable state, the driving model generation device 700 may detect that the moving tool is parking with the front of the vehicle first based on the image obtained from the image acquisition device.

[0193] The driving model generation device 700 of the present invention may store the detected operation information and / or driving information of the moving tool. In addition, the memory 730 may operate.

[0194] The storage may be performed from a point where the amplitude of the ERP or the amplitude of the brain wave signal is equal to or greater than a predetermined threshold.

[0195] For example, the detected operation information and / or driving information of the moving tool may be stored in a list form as shown in Table 1.

[0196] Table 1

[0197]

[0198] In Table 1, the operation item may represent the operation of the detected moving tool. The driving information item may represent the driving path of the detected moving tool or the event performed by the detected moving tool. The location item, date item, time item, speed item, and image item may respectively represent the location, date, time, speed, and image where the operation of the detected moving tool and / or the event of the detected moving tool are performed. At the same time, each item listed in Table 1 is only one implementation, but is not limited thereto, and the item may include the necessary information indicating the operation or event performed by the moving tool.

[0199] For another example, the detected operation information and / or driving information of the moving tool may be sorted and stored in ascending / descending order based on the number of items of the operation information or driving information.

[0200] Figure 11 is a block diagram showing the configuration of a driving model generation device using error monitoring according to another embodiment of the present invention.

[0201] Reference Figure 11, the driving model generation device 1100 may include a sensor 1110, an analyzer 1120, a memory 1130, and / or a simulation unit 1140. However, it should be noted that only some components necessary for explaining this embodiment are shown, and the components included in the driving model generation device 1100 are not limited to the above examples. For example, two or more constituent units may be implemented as one constituent unit, and the operations performed in one constituent unit may be divided and executed in two or more constituent units. Additionally, some constituent units may be omitted, or additional constituent units may be added.

[0202] Figure 11 The driving model generation device 1100, the sensor 1110, the analyzer 1120, and the memory 1130 may be Figure 7 respective embodiments of the driving model generation device 700, the sensor 710, the analyzer 720, and the memory 730.

[0203] The driving model generation device 1100 of the present invention may generate a driving model through Figure 7 the operation information and / or driving information of the moving tool described in

[0204] For example, the driving model generation device 1100 may model the entire path that the driver has traveled.

[0205] For another example, the driving model generation device 1100 may model the operation status and / or driving status of the moving tool when the driver's state is determined to be an unstable state.

[0206] For another example, the driving model generation device 1100 may model the entire path that the driver has traveled and provide the user with the operation status and / or driving status of the moving tool when the driver's state is determined to be an unstable state within the modeled entire path. For example, keywords such as left turn, U-turn, lane change, nose-in parking, and overpass when the driver's state is determined to be an unstable state may be provided to the user. Alternatively, points where the driver's state is determined to be an unstable state may be indicated on the modeled entire path.

[0207] For another example, when the operation of the moving tool whose driver's state is determined to be an unstable state is a left turn condition, the driving model generation device 1100 may model at least one virtual left turn condition and provide the modeled condition to the user. The virtual left turn condition may be obtained by using the detected information or through a virtual simulation process or a predetermined learning process.

[0208] For another example, when the operation of a moving vehicle with an unstable driver state is a head-first parking situation, the driving model generation device 1100 can model at least one virtual head-first parking situation and provide the modeled situation to the user. The virtual head-first parking situation can be obtained by using the detected information or through a virtual simulation process or a predetermined learning process.

[0209] Therefore, when providing a simulation program that models the operation status and / or driving status of a driver with an unstable state to the user, the user can repeatedly practice the simulation program to improve driving habits and further prevent traffic accidents.

[0210] Figure 12 It is a flowchart showing a driving model generation method using error monitoring according to an embodiment of the present invention.

[0211] In step S1201, the brain wave signals of the moving vehicle driver can be collected within a predetermined time.

[0212] Here, the brain wave signals can include ERP.

[0213] In addition, ERP can include at least one of error-related negativity (ERN) and error positivity (Pe). Additionally, ERP can further include at least one of correct-related negativity (CRN) and correct positivity (Pc).

[0214] In step S1202, the driver state can be determined by analyzing the brain wave signals collected within a predetermined time.

[0215] Here, the analysis can include comparing the amplitude of the ERP collected within a predetermined time with a first threshold.

[0216] Here, the first threshold can be determined in different ways according to at least one of the type of ERP and the driver from whom the ERP is obtained.

[0217] Here, determining the driver state can include analyzing the ERP to determine whether the driver is in an unstable state or a stable state.

[0218] In addition, determining the driver state can include: dividing the brain wave signals collected within a predetermined time according to frequency bands, and determining whether the driver is in an unstable state or a stable state by comparing the amplitude of the brain wave signals in each divided frequency band with a second threshold.

[0219] In addition, the brain wave signals in each divided frequency band can include at least one of theta wave, alpha wave, and beta wave.

[0220] In addition, the second threshold value can be determined differently according to the type of electroencephalogram signal of each frequency band.

[0221] In addition, determining the driver state may include: when the analysis result of the ERP and the analysis result of the electroencephalogram signal of each frequency band indicate that the driver is in an unstable state, finally determining that the driver is in an unstable state.

[0222] In step S1203, based on the determined driver state, at least one of the operation information and the driving information of the moving tool can be detected.

[0223] In addition, when the driver state is determined to be an unstable state, the detection may include detecting at least one of the operation information and the driving information of the moving tool by using at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the moving tool.

[0224] In step S1204, the detected information can be stored.

[0225] In addition, the storage may include storing at least one of the detected operation information of the moving tool and the detected driving information of the moving tool.

[0226] Here, the operation information of the moving tool may include at least one of the operation of the moving tool and the position, date, time, speed, and image corresponding to the operation.

[0227] Here, the driving information of the moving tool may include the driving path of the moving tool and at least one event that the moving tool is performing.

[0228] In addition, the method may perform a simulation for modeling the driving path, and the simulation may include modeling at least one of the operation and the driving condition of the moving tool for which the driver state is determined to be an unstable state.

[0229] In addition, for a predetermined operation of the moving tool based on the driver state being determined to be an unstable state, the simulation may include modeling a virtual driving condition related to the predetermined operation.

[0230] According to the present invention, an apparatus and a method for generating a driving model for driver proficiency can be provided.

[0231] In addition, according to the present invention, an apparatus and a method for modeling the operation or driving condition of a moving tool based on the driver state can be provided.

[0232] In addition, according to the present invention, an apparatus and a method for determining the driver state by using error monitoring can be provided.

[0233] In addition, according to the present invention, an apparatus and method for detecting an operation or driving condition of a mobile vehicle based on a driver's state can be provided.

[0234] The effects obtained in the present invention are not limited to the above effects, and according to the following description, those skilled in the art can clearly understand other effects not mentioned above.

[0235] Although, for the sake of clarity of description, the exemplary methods of the present invention are described as a series of operation steps, the present invention is not limited to the order or sequence of the above operation steps. The operation steps can be performed simultaneously or can be sequentially performed in a different order. To implement the method of the present invention, additional operation steps can be added and / or existing operation steps can be eliminated or replaced.

[0236] The various embodiments of the present invention are not presented as describing all available combinations, but rather as describing only representative combinations. The steps or elements in the various embodiments can be utilized alone or can be combined.

[0237] In addition, the various embodiments of the present invention can be implemented in the form of hardware, firmware, software, or a combination thereof. When the present invention is implemented as a hardware component, it can be, for example, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[0238] The scope of the present invention includes software or machine-executable instructions (e.g., an operating system (OS), an application, firmware, a program) that enable various forms of methods to be executed on a device or a computer, and a non-volatile computer-readable medium that stores such software or machine-executable instructions so that the software or instructions can be executed on the device or the computer.

[0239] The description of the present invention is essentially exemplary only, and thus, variations that do not depart from the essence of the present invention are intended to fall within the scope of the present invention. Such variations should not be regarded as departing from the spirit and scope of the present invention.

Claims

1. A driving model generation device, the device comprising: A sensor configured to collect the brain wave signals of a mobile tool driver within a predetermined time; An analyzer configured to determine the driver state by analyzing the brain wave signals collected within a predetermined time, and detect at least one of the operation information and driving information of the mobile tool based on the determined driver state; A memory configured to store the detected information; And A simulation unit configured to model the driving path; Wherein, the brain wave signals include event-related potential ERP; Wherein, the analyzer determines whether the driver is in an unstable state by analyzing the event-related potential ERP; Wherein, in order to learn the operations of the mobile tool performed during the unstable state, the simulation unit models a virtual driving situation related to the operations of the mobile tool that the driver has performed during the unstable state; Wherein, a simulation program for modeling the virtual driving situation is provided to the user, so that the user can improve driving habits by practicing the simulation program.

2. The driving model generation device according to claim 1, wherein: The event-related potential ERP includes at least one of error-related negativity (ERN), error positivity (Pe), correct-related negativity (CRN), and correct positivity (Pc).

3. The driving model generation device according to claim 1, wherein: The analysis includes comparing the amplitude of the event-related potential ERP collected within a predetermined time with a first threshold.

4. The driving model generation device according to claim 3, wherein: The first threshold is determined according to at least one of the type of the event-related potential ERP and the information of the driver.

5. The driving model generation device according to claim 1, wherein: The analyzer divides the brain wave signals collected within a predetermined time into brain wave signals of each frequency band, and further determines whether the driver is in an unstable state by comparing the amplitude of the divided brain wave signals of each frequency band with a second threshold.

6. The driving model generation device according to claim 1, wherein: The sensor further includes at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the mobile tool.

7. The driving model generation device according to claim 6, wherein: When the driver state is determined to be an unstable state, the analyzer detects at least one of the operation information and driving information of the mobile tool by using at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the mobile tool.

8. The driving model generation device according to claim 1, wherein: The memory includes at least one of the detected operation information and driving information of the mobile tool; The operation information of the mobile tool includes the operation of the mobile tool and at least one of the position, date, time, speed, and image corresponding to the operation; The driving information of the mobile tool includes the driving path of the mobile tool and at least one event that the mobile tool is undergoing.

9. The driving model generation device according to claim 1, Among them, wherein the simulation unit models at least one of the operation of the mobile tool and the driving condition based on whether the driver state is determined to be an unstable state.

10. A driving model generation method, the method comprising: collecting the electroencephalogram signal of the mobile tool driver by a sensor within a predetermined time; determining the driver state by analyzing the electroencephalogram signal collected within the predetermined time; detecting at least one of the operation information and the driving information of the mobile tool based on the determined driver state; storing the detected information; simulating the modeling of the driving path; wherein the electroencephalogram signal includes event-related potential ERP; wherein determining the driver state includes determining whether the driver is in an unstable state by analyzing the event-related potential ERP; wherein, in order to learn the operation of the mobile tool performed during the unstable state, a virtual driving condition related to the operation of the mobile tool that the driver has performed during the unstable state is modeled; wherein a simulation program for modeling the virtual driving condition is provided to the user, so that the user improves the driving habit by practicing the simulation program.

11. The driving model generation method according to claim 10, wherein: the event-related potential ERP includes at least one of error-related negativity (ERN), error positivity (Pe), correct-related negativity (CRN), and correct positivity (Pc).

12. The driving model generation method according to claim 10, wherein: the analysis includes comparing the amplitude of the event-related potential ERP collected within the predetermined time with a first threshold.

13. The driving model generation method according to claim 12, wherein: the first threshold is determined based on at least one of the type of the event-related potential ERP and the information of the driver.

14. The driving model generation method according to claim 10, wherein: determining the driver state further includes: dividing the electroencephalogram signal collected within the predetermined time into electroencephalogram signals of each frequency band; determining whether the driver is in an unstable state by comparing the amplitude of the electroencephalogram signal of each divided frequency band with a second threshold.

15. The driving model generation method according to claim 10, wherein: the sensor further includes at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the mobile tool.

16. The driving model generation method according to claim 15, wherein: when the driver state is determined to be an unstable state, detecting at least one of the operation information and the driving information of the mobile tool includes: detecting at least one of the operation information and the driving information of the mobile tool by using at least one of a speed measurement unit, an image acquisition unit, a sound acquisition unit, a wheel monitoring unit, and a control device unit included in the mobile tool.

17. The driving model generation method according to claim 10, wherein: storing the detected information includes storing at least one of the detected operation information and driving information of the mobile tool; The operation information of the mobile tool includes at least one of the operation of the mobile tool and the position, date, time, speed, and image corresponding to the operation; The driving information of the mobile tool includes the driving path of the mobile tool and at least one event that the mobile tool is undergoing.

18. The driving model generation method according to claim 10, Among them, The simulation for modeling the driving path includes modeling at least one of the operation and driving condition of the mobile tool based on whether the driver state is determined to be an unstable state.

Images