Device and method for generating an image using electroencephalogram signals

Abstract

The present invention relates to an apparatus and method for generating an image using an electroencephalogram signal. The method for generating an image using an electroencephalogram signal includes: collecting electroencephalogram signals of at least one passenger in a moving vehicle from multiple channels within a predetermined time; generating a first image of a service point from the electroencephalogram signals collected from the multiple channels using an artificial intelligence model; determining whether the generated first image is similar to a preset second image; and controlling the moving vehicle based on the determination result.

Description

[0001] Cross - reference to related applications

[0002] This application claims priority and the benefit of Korean Patent Application No. 10 - 2019 - 0135835, filed on October 29, 2019, which is incorporated herein by reference in its entirety. Technical field

[0003] The present invention relates to a method and apparatus for controlling a mobile tool. More specifically, the present invention relates to an apparatus and method for generating an image using an electroencephalogram signal. Background art

[0004] The statements in this section merely provide background information related to the present invention and do not constitute prior art.

[0005] As a means of transportation, a vehicle (or mobile tool) is a tool used for living in the modern world. In addition, for some people, a vehicle itself can be considered an item with special significance.

[0006] With the progress of technology, the functions provided by vehicles have also gradually developed. For example, in recent years, vehicles not only transport passengers to their destinations but also meet the needs of passengers to reach their destinations faster and safer. In addition, new devices have been added to the vehicle system to meet the aesthetic tastes and comfort of passengers. Moreover, existing devices such as steering wheels, transmissions, and acceleration / deceleration devices have been developed so that more functions can be provided for users.

[0007] Meanwhile, a brain - computer interface or a brain - machine interface is a field that controls a computer or a machine according to a person's intention by using an electroencephalogram signal. Event - Related Potential (ERP) is closely related to cognitive functions.

[0008] In addition, in recent years, there has been an increasing tendency to research using an artificial intelligence model to identify and classify objects included in an image and generate a new image. Summary of the invention

[0009] An object of the present invention is to provide an apparatus and method for generating an image based on a passenger's electroencephalogram signal.

[0010] Another object of the present invention is to provide an apparatus and method for generating an image from a passenger's electroencephalogram signal using an artificial intelligence model and controlling a mobile tool based on the generated image.

[0011] The technical objects of the present invention are not limited to the above - mentioned technical objects, and other technical objects not mentioned will be clearly understood by those skilled in the art from the following description.

[0012] According to the present invention, a device for generating an image using electroencephalogram signals includes a sensor and a controller. The sensor is configured to collect electroencephalogram signals of at least one passenger in a moving vehicle from multiple channels within a predetermined time. The controller is configured to generate a first image of a service point from the electroencephalogram signals collected from the multiple channels using an artificial intelligence model, determine whether the generated first image is similar to a preset second image, and control the moving vehicle based on the determination result.

[0013] The electroencephalogram signals collected from the multiple channels can be electroencephalogram signals in at least one of the time domain, frequency domain, or spatial domain.

[0014] The artificial intelligence model can be a generative adversarial network (GAN) model.

[0015] The first image can be at least one of a service point image indicating the service point or a product image. The service point image can be an image indicating the service point, and the product image can be an image indicating the product provided by the service point.

[0016] The second image can be one of a service point image and a product image.

[0017] The service point can be a drive-thru (DT) service providing place within a predetermined range from the moving vehicle.

[0018] When transmission and reception are performed between the moving vehicle and the service point based on a short-range communication network, the predetermined range can be a range capable of performing transmission and reception through the short-range communication network.

[0019] The product can be at least one of an item provided by the service point, a service provided by the service point, or information about the service point.

[0020] When the first image is a service point image, the controller can determine the second image as a preset service point image and determine whether the first image is similar to the second image.

[0021] When determining that the first image is similar to the second image, the controller can perform at least one of the following: change the driving route of the moving vehicle to the service point, provide a notification of the driving route to be changed to the passenger, or guide the passenger to select a route by providing the driving route to be changed to the passenger.

[0022] When the first image is a product image, the controller can determine the second image as a preset product image and determine whether the first image is similar to the second image.

[0023] When it is determined that the first image is similar to the second image, the controller may perform at least one of the following: changing the driving route of the mobile tool to the service point that provides the goods, providing the passenger with a notice of the driving route to be changed, guiding the passenger to select a route by providing the passenger with the driving route to be changed, or sending an order signal for the goods.

[0024] The controller may also be configured to determine whether the determination result meets the passenger's expectation.

[0025] According to the present invention, a method for generating an image using an electroencephalogram signal includes: collecting electroencephalogram signals of at least one passenger in a mobile tool from multiple channels within a predetermined time; using an artificial intelligence model to generate a first image of a service point from the electroencephalogram signals collected from the multiple channels; determining whether the generated first image is similar to a preset second image; and controlling the mobile tool based on the determination result.

[0026] The electroencephalogram signals collected from the multiple channels may be electroencephalogram signals in at least one of the time domain, frequency domain, or spatial domain.

[0027] The artificial intelligence model may be a generative adversarial network (GAN) model.

[0028] The first image may be at least one of a service point image indicating the service point or a goods image, the service point image may be an image indicating the service point, and the goods image may be an image indicating the goods provided by the service point.

[0029] The second image may be one of the service point image and the goods image.

[0030] The service point may be a drive-through (DT) service providing place within a predetermined range from the mobile tool.

[0031] When sending and receiving are performed between the mobile tool and the service point based on a short-range communication network, the predetermined range may be a range within which sending and receiving can be performed through the short-range communication network.

[0032] The goods may be at least one of an item provided by the service point, a service provided by the service point, or information about the service point.

[0033] When the first image is a service point image, determining whether the first image is similar to the second image may include: determining the second image as a preset service point image, and determining whether the first image is similar to the second image.

[0034] When it is determined that the first image is similar to the second image, the control of the mobile tool may include performing at least one of the following: changing the driving route of the mobile tool to the service point, providing the passenger with a notice of the driving route to be changed, or guiding the passenger to select a route by providing the passenger with the driving route to be changed.

[0035] When the first image is a product image, determining whether the first image is similar to the second image may include: determining the second image as a preset product image, and determining whether the first image is similar to the product image.

[0036] When it is determined that the first image is similar to the second image, the control of the moving tool may include performing at least one of the following: changing the driving route of the moving tool to a service point that provides the product, notifying the passenger of the driving route to be changed, guiding the passenger to select a route by providing the passenger with the driving route to be changed, or sending an order signal for the product.

[0037] The method may further include determining whether the determination result meets the passenger's expectations.

[0038] The features briefly outlined above regarding the present invention are merely exemplary aspects of the present invention described in detail below and do not limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To better understand the present invention, various forms of the present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0040] To better understand the present invention, various embodiments of the present invention will now be described by way of example with reference to the accompanying drawings, in which:

[0041] Figure 1 is a diagram showing the normal waveform of the ERN showing one embodiment of the present invention;

[0042] Figure 2 is a diagram showing the normal waveforms of the ERN and Pe according to one embodiment of the present invention;

[0043] Figure 3 is a diagram showing the deflection characteristics of the Pe according to another embodiment of the present invention;

[0044] Figure 4A and Figure 4B are diagrams respectively showing the measurement regions of the ERP and Pe of one embodiment of the present invention;

[0045] Figure 5 is a diagram showing the normal waveforms of the ERN and CRN according to one embodiment of the present invention;

[0046] Figure 6 is a diagram showing the EEG measurement channels corresponding to the cerebral cortex region according to one embodiment of the present invention;

[0047] Figure 7 is a block diagram showing the configuration of a device for generating an image based on a passenger's brain wave signal according to an embodiment of the present invention.

[0048] Figure 8 This is a diagram showing the range within which a mobile tool and a service point perform mutual transmission and reception according to an embodiment of the present invention.

[0049] Figure 9 This is a flowchart showing a method of an operation image generation device according to an embodiment of the present invention.

[0050] Figure 10 This is a flowchart showing a method of an operation image generation device according to an embodiment of the present invention. Detailed Description

[0051] The following description is merely exemplary in nature and is not intended to limit the present invention, its application, or uses. It should be understood that throughout the drawings, corresponding reference numerals denote the same or corresponding components and features.

[0052] Exemplary embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily understand and implement the devices and methods provided by the present invention in conjunction with the drawings. However, the present invention can be implemented in various embodiments, and the scope of the present invention should not be construed as being limited to the exemplary embodiments.

[0053] When describing embodiments of the present invention, well-known functions or configurations may be omitted from detailed description when they may obscure the spirit of the present invention.

[0054] In the present invention, it will be understood that when an element is referred to as being "connected to", "coupled to", or "combined with" another element, it can be directly connected to, coupled to, or combined with the other element, or there may be intervening elements therebetween. It will be further understood that when used in the present invention, the terms "comprising", "including", "having", etc. indicate the presence of the stated features, values, steps, operations, elements, components, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, values, steps, operations, elements, components, and / or combinations thereof.

[0055] It will be understood that although the terms "first", "second", etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish one element from another and do not indicate the order or priority between the elements. For example, the first element discussed below may be referred to as the second element without departing from the teachings of the present invention. Similarly, the second element may also be referred to as the first element.

[0056] In the present invention, different elements are named to clearly describe the features of various elements, which does not mean that these elements are physically separated from each other. That is, a plurality of different elements can be combined into a single hardware unit or a single software unit, and conversely, one element can be implemented by a plurality of hardware units or software units. Therefore, although not specifically described, the integration form of various elements or the separation form of one element can fall within the scope of the present invention. Moreover, terms such as "unit" or "module" should be understood as units that process at least one function or operation and can be implemented in a hardware manner (e.g., a processor), a software manner, or a combination of a hardware manner and a software manner.

[0057] In the present invention, all constituent elements described in various forms should not be construed as essential elements, and some constituent elements may be optional elements. Therefore, embodiments configured in a certain form by each subset of the constituent elements can also fall within the scope of the present invention. In addition, embodiments configured by adding one or more elements to various elements also fall within the scope of the present invention.

[0058] As the electroencephalogram activity of neurons that make up the brain, electroencephalogram signals (or brain signals, brainwaves) represent biological signals that directly and indirectly reflect a person's conscious or unconscious state. Electroencephalogram signals can be measured in each region of the human scalp, with the frequency of their wavelengths mainly being 30 Hz or lower and the potential difference being a few microvolts. Depending on brain activity and state, various waveforms may appear. Research on interface control using electroencephalogram signals according to a person's intention is underway. Electroencephalogram signals can be obtained by using electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), or functional near-infrared spectroscopy (fNIRS). EEG uses electrical signals caused by brain activity, MEG uses magnetic signals that occur together with electrical signals, and fMRI or fNIRS uses changes in oxygen saturation in the blood. Although fMRI and fNIRS are useful techniques for measuring brain activity, generally, fMRI has a lower temporal resolution, while fNIRS has a lower spatial resolution. Due to these limitations, EEG signals are widely used due to their good portability and temporal resolution.

[0059] Electroencephalogram signals change spatially and temporally according to brain activity. Since electroencephalogram signals are generally difficult to analyze and their waveforms are not easy to visually analyze, various processing methods have been proposed.

[0060] For example, according to the number of oscillations (frequency), the electroencephalogram (EEG) signal can be divided based on frequency bands (power spectrum division). This division regards the measured EEG signal as the linear sum of simple signals at each specific frequency, decomposes the signal into each frequency component, and indicates the corresponding amplitude. By using preprocessing commonly used for noise cancellation, Fourier transform to the frequency domain, and a band-pass filter (BPF), the EEG signal at each frequency can be obtained.

[0061] More specifically, according to frequency bands, EEG can be divided into delta waves, theta waves, alpha waves, beta waves, and gamma waves. Delta waves are EEGs with a frequency of 3.5 Hz or less and an amplitude of 20 μV to 200 μV, which mainly appear during normal deep sleep or in newborns. In addition, as our understanding of the physical world decreases, delta waves may increase. Generally, theta waves are EEGs with a frequency of 3.5 Hz to 7 Hz, which mainly appear in a state of emotional stability or during sleep.

[0062] In addition, theta waves are mainly generated in the parietal cortex and occipital cortex and may appear during the calm concentration of recollection or meditation. Generally, alpha waves are EEGs with a frequency of 8 Hz to 12 Hz, which mainly appear in a relaxed and comfortable state. In addition, alpha waves are usually generated in the occipital cortex during rest and may decrease during sleep. Generally, beta waves are EEGs with a frequency of 13 Hz to 30 Hz, which mainly appear in a tolerable state of tension or during a certain degree of attention. In addition, beta waves are mainly generated in the frontal cortex and are related to the waking state, concentrated brain activity, pathological phenomena, and drug effects. Beta waves may appear in a wide area of the entire brain. In addition, specifically, beta waves can be divided into SMR waves with a frequency of 13 Hz to 15 Hz, mid-beta waves with a frequency of 15 Hz to 18 Hz, and high-beta waves with a frequency of 20 Hz or more. Since beta waves seem to be stronger under stress such as anxiety and tension, they are called stress waves. Gamma waves are EEGs usually having a frequency of 30 Hz to 50 Hz, which mainly appear in a state of strong excitement or during the process of high-level cognitive information processing. In addition, gamma waves may appear during the state of conscious awakening and during REM sleep and may overlap with beta waves.

[0063] Each brain wave signal depending on the frequency band is related to a specific cognitive function. For example, delta waves are related to sleep, theta waves are related to working memory, and alpha waves are related to attention or inhibition. Therefore, the nature of the brain wave signals in each frequency band selectively shows a specific cognitive function. Additionally, in each measurement site on the head surface, the brain wave signals in each frequency band may show some different appearances. The cerebral cortex can be divided into the frontal cortex, parietal cortex, temporal cortex, and occipital cortex. These sites may have some different functions. For example, the occipital cortex corresponding to the back of the head has the primary visual cortex and thus can mainly process visual information. The parietal cortex near the top of the head has the somatosensory cortex and thus can process motor / sensory information. Additionally, the frontal cortex can process information related to memory and thinking, and the temporal cortex can process information related to hearing and smell.

[0064] Meanwhile, for another example, brain wave signals can be analyzed by utilizing Event-Related Potential (ERP). ERP is the electroencephalogram change related to external stimuli or internal mental processes. ERP refers to the signal including the electroencephalogram activity of the brain caused by a stimulus within a certain time after the appearance of the stimulus, and the stimulus includes specific information (such as images, voices, sounds, execution commands, etc.).

[0065] To analyze ERP, a process of separating the signal from the noise is required. The mean method can be mainly used. In particular, by taking the mean of the brain waves measured based on the start time of the stimulus, the brain waves unrelated to the stimulus can be removed, and only the relevant potential, that is, the brain activity usually related to stimulus processing, can be selected.

[0066] Since ERP has a high time resolution, it is closely related to the study of cognitive functions. ERP is an electrical phenomenon induced by external stimuli or related to internal states. According to the type of stimulus, ERP can be divided into auditory-related potential, visual-related potential, somatosensory-related potential, and olfactory-related potential. According to the nature of the stimulus, ERP can be divided into exogenous ERP and endogenous ERP. The waveform of exogenous ERP is determined by external stimuli, is related to automatic processing, and mainly appears in the initial stage of being stimulated. For example, exogenous ERP is brainstem potential. On the other hand, endogenous ERP is determined by internal cognitive processes or mental processes or states, is independent of stimuli, and is related to "controlled processes". For example, endogenous ERP is P300, N400, P600, Contingent Negative Variation (CNV), etc.

[0067] The names of ERP peaks usually include polarity and latency, and each peak of the signal has its own definition and meaning. For example, a positive potential is P, a negative potential is N, and P300 represents a positive peak measured approximately 300 ms after the start of the stimulus. Additionally, 1, 2, 3 or a, b, c, etc. are applied according to the order of appearance. For example, P3 represents the third positive potential in the waveform after the start of the stimulus.

[0068] In the following text, various ERPs will be described.

[0069] For example, N100 is related to the response to unpredictable stimuli.

[0070] The Mismatch Negativity (MMN) can be generated not only by focused stimuli but also by non-focused stimuli. MMN can be used as an indicator of whether sensory memory (echoic memory) is operating before initial attention. The P300 described below appears during the process of attention and making a judgment, and analyzes the MMN as a process that occurs in the brain before attention.

[0071] For another example, N200 (or N2) is mainly generated according to visual and auditory stimuli, and is related to short-term or long-term memory together with the P300 described below, which are memory types after attention.

[0072] For another example, P300 (or P3) mainly reflects attention to the stimulus, stimulus recognition, memory search, and reduction of the sense of uncertainty, and is related to the perceptual decision to distinguish external stimuli. Since the generation of P300 is related to cognitive functions, P300 will be generated regardless of the type of stimulus that appears. For example, P300 can be generated in auditory, visual, and somatic stimuli. P300 is widely used in the research of brain-computer interfaces.

[0073] For another example, N400 is related to language processing and is caused when a sentence with a semantic error or an auditory stimulus appears. Additionally, N400 is related to the memory process and can reflect the process of retrieving or searching for information from long-term memory.

[0074] For another example, as an indicator representing the reconstruction or recovery process, P600 is related to the process of more accurately processing stimuli based on the information stored in long-term memory.

[0075] For another example, CNV refers to the potential that appears within 200 ms - 300 ms or even several seconds in the subsequent stage. It is also called the slow potential (SP) and is related to anticipation, preparation, mental priming, association, attention, and motor activity.

[0076] For another example, the ERN (error-related negativity) or Ne (error negativity) is an event-related potential (ERP) generated by a slip or an error. It may occur when a subject makes a mistake in a sensorimotor task or a similar task. More specifically, when the subject recognizes a slip or an error, the ERN is generated, and its negative peak mainly appears in the frontal region and the central region for about 50 ms to 150 ms. In particular, it may occur in situations where slips related to motor responses may occur and can also be used to indicate negative self-judgment.

[0077] In the following, the main characteristics of the ERN will be described in more detail.

[0078] Figure 1 is a diagram showing the typical waveform of the ERN according to an embodiment of the present invention.

[0079] Reference Figure 1 , the negative potential values are depicted above the horizontal axis, and the positive potential values are depicted below the horizontal axis. Additionally, it can be confirmed that an ERP with a negative peak is generated within a predetermined time range after the start of the response to any movement. Here, the response can represent a situation where a slip or an error occurs (error response). Additionally, the predetermined time range can be about 50 ms to 150 ms. Alternatively, the predetermined time range can be about 0 to 100 ms. Furthermore, in the case of a correct response, the generated ERP has a relatively smaller negative peak than the ERN.

[0080] As an ERP of an initial negative wave, the ERN is time-locked until a response error occurs. Additionally, it is known that the ERN reflects the reinforcement activity of the dopaminergic system related to behavioral monitoring. The ERN includes a fronto-striatal loop, which includes the rostral cingulate area. At the same time, dopamine is related to the brain reward system that usually forms specific behaviors and motivates people, thereby providing a feeling of pleasure and fulfillment. When a behavior that repeatedly obtains appropriate rewards is performed, it is learned as a habit. Additionally, more dopamine is released through emotional learning, and new behaviors are attempted due to the release of dopamine. Therefore, reward-driven learning is called reinforcement learning.

[0081] Additionally, the ERN may be generated within 0 to 100 ms after the start of an error response caused during the reading of an executive interference task (e.g., Go-noGo task, Stroop task, Flanker task, and Simon task) through the frontal cortex.

[0082] Additionally, together with the CRN described below, it is known that the ERN reflects the normal behavioral monitoring system that can distinguish between correct and incorrect behaviors.

[0083] In addition, the fact that the ERN reaches its maximum amplitude at the frontal cortex electrodes reflects that the brain generator is located in the rostral cingulate area or the dorsal anterior cingulate cortex (dACC) area.

[0084] In addition, the ERN may show amplitude variations according to negative emotional states.

[0085] In addition, the ERN can be reported even when behavior monitoring is performed based on external evaluation feedback (different from internal motor expression), and it can be classified into the FRN described below.

[0086] In addition, the ERN can be generated not only when a mistake or error is recognized, but also before a mistake or error is recognized.

[0087] In addition, the ERN can be generated not only as a response to one's own mistakes or errors, but also as a response to others' mistakes or errors.

[0088] In addition, the ERN can be generated not only as a response to mistakes or errors, but also as a response to the anxiety or stress of a scheduled task to be performed or the subject.

[0089] In addition, when a larger ERN peak is obtained, it can be considered that it reflects a more serious mistake or error.

[0090] Meanwhile, for another example, as an event-related potential (ERP) generated after the ERN, Pe (error positivity) is an ERP with a positive value, which is mainly generated at the frontal cortex electrodes approximately 150 ms to 300 ms after a mistake or error. It is known that Pe is a reaction to being aware of a mistake or error and paying more attention. In other words, Pe is related to an indicator of the conscious error information processing process after error detection. The ERN and Pe are called ERPs related to error monitoring.

[0091] Hereinafter, the main characteristics of Pe will be described in more detail.

[0092] Figure 2 is a diagram showing the typical waveforms of the ERN and Pe according to another embodiment of the present invention.

[0093] Reference Figure 2, the negative potential value is shown above the positive potential value. Additionally, it can be confirmed that an ERP with a negative peak (i.e., ERN) is generated within a first predetermined time range after the start of the response to any movement. Here, the response can indicate a situation of making a mistake or error (error response). Additionally, the first predetermined time range can be about 50 ms to 150 ms. Alternatively, the first predetermined time range can be about 0 to 200 ms.

[0094] Furthermore, it can be confirmed that an ERP with a positive peak (i.e., Pe) is generated within a second predetermined time range after the start of ERN. Additionally, the second predetermined time range can be about 150 ms to 300 ms after the start of the error. Alternatively, the second predetermined time range can represent about 200 ms to 400 ms.

[0095] Figure 3 is a diagram showing the deflection characteristics of Pe in one embodiment of the present invention.

[0096] Reference Figure 3 , like P3, Pe also has a wide deflection characteristic, and the neural plexus generator includes not only the posterior cingulate cortex region and the insular cortex region, but also more anterior cingulate cortex regions.

[0097] In addition, Pe can reflect the emotional evaluation of errors as well as the attention to stimuli like P300. Additionally, ERN represents the conflict between correct and error responses, and Pe is considered a response to being aware of mistakes and paying more attention. In other words, ERN is generated during the process of detecting stimuli, and Pe is generated according to attention during the process of processing stimuli. When ERN and / or Pe respectively have relatively large values, it is known that these values are related to adaptive behaviors aimed at responding more slowly and accurately after mistakes.

[0098] Figure 4A and Figure 4B is a diagram showing the measurement regions of ERP and Pe according to one embodiment of the present invention.

[0099] ERN and Pe are considered ERPs related to error monitoring. Regarding the measurement regions of ERN and Pe, generally, the maximum negative value and the maximum positive value can be measured in the central region. However, there may be some differences according to the measurement conditions. For example, Figure 4A is the main region for measuring ERN, and generally, the maximum negative value of ERN can be measured in the midline frontal or central region (i.e., FCZ). Additionally, Figure 4B is the main region for measuring Pe, and generally, a larger positive value of Pe can be measured in the posterior midline region compared to ERN.

[0100] In addition, for another example, the FRN (feedback-related negativity) is an event-related potential (ERP) that is related to error detection obtained based on external evaluative feedback. The ERN and / or Pe detect errors based on an internal monitoring process. However, in the case of the FRN, when the FRN is obtained based on external evaluative feedback, it can operate similarly to the ERN process.

[0101] In addition, the FRN and ERN may share many electrophysiological properties. For example, the FRN has a negative peak at frontal cortex electrodes approximately 250 ms to 300 ms after the onset of negative feedback and can be generated as the ERN is in the dorsal anterior cingulate cortex (dACC) region.

[0102] In addition, like the ERN, the FRN can reflect the reinforcement learning activity of the dopaminergic system. In addition, the FRN generally has a more negative value than positive feedback and can have a greater value for unforeseen situations than for predictable outcomes.

[0103] For another example, the CRN (correct-related negativity) is an ERP generated by correct trials and is a less negative value than the ERN. Like the ERN, the CRN can be generated in the initial latency period (e.g., 0 to 100 ms). Figure 5 FIG. is a diagram showing the typical waveforms of the ERN and CRN of an embodiment of the present invention.

[0104] For another example, the Pc (positive component for correct responses) is an event-related potential generated after the CRN. It is an event-related potential generated approximately 150 ms to 300 ms after the onset of a correct response. The relationship between the CRN and Pc can be similar to the relationship between the ERN and Pe.

[0105] In addition, ERPs can be divided into stimulus-locked ERPs and response-locked ERPs. Stimulus-locked ERPs and response-locked ERPs can be divided according to criteria such as the cause of the induced ERP and the response time. For example, an ERP induced from the moment when a word or picture is presented externally to a user can be called a stimulus-locked ERP. In addition, for example, an ERP induced from the moment when a user speaks or presses a button can be called a response-locked ERP. Therefore, based on the above criteria, generally, stimulus-locked ERPs are N100, N200, P2, P3, etc., and response-locked ERPs are ERN, Pe, CRN, Pc, FRN, etc.

[0106] In addition, brainwaves can be classified according to the expressed motivation. Brainwaves can be divided into spontaneous brainwaves (spontaneous potentials) expressed by the user's will and evoked brainwaves (evoked potentials) that are naturally expressed according to external stimuli and are independent of the user's will. When the user moves or imagines moving on his / her own, spontaneous brainwaves are expressed, while evoked brainwaves are expressed through, for example, visual, auditory, olfactory, and tactile stimuli.

[0107] In addition, brainwave signals can be measured according to the international 10-20 system. The international 10-20 system determines the measurement points of brainwave signals based on the relationship between the electrode positions and the cerebral cortex regions.

[0108] Figure 6 It is a diagram showing EEG measurement channels corresponding to cerebral cortex regions according to an embodiment of the present invention.

[0109] Reference Figure 6 , the brain regions (frontal lobe cortex FP1, FP2; frontal cortex F3, F4, F7, F8, FZ, FC3, FC4, FT7, FT8, FCZ; parietal cortex C3, C4, CZ, CP3, CP4, CPZ, P3, P4, PZ; temporal cortex T7, T8, TP7, TP8, P7, P8; occipital cortex O1, O2, OZ) correspond to 32 brainwave measurement channels. For each channel, data can be obtained and each cerebral cortex region can be analyzed by using the data.

[0110] Figure 7 It is a block diagram showing the configuration of a device for generating an image based on a passenger's brainwave signal according to an embodiment of the present invention.

[0111] The drive-thru (DT) service refers to a service that allows customers to order, pay for, and pick up certain items without having to stop while driving a mobile vehicle. Since there is no need to stop or queue up, the DT service has gained public attention as an efficient and convenient service for customers. In recent years, the DT service has gradually become popular. For example, passengers in a mobile vehicle can conveniently and easily use the DT services provided by fast food restaurants and coffee shops in urban areas and on highways in their daily lives. The mobile vehicle can include vehicles, mobile devices / transportation devices, etc.

[0112] In addition, in the current DT service, passengers usually order the required goods after arriving at the place or location (hereinafter referred to as the DT point) that provides the DT service. In the present invention, an image generation device and method can be provided that enable the selection and ordering of goods (hereinafter referred to as DT goods) provided by the DT point in a mobile vehicle before arriving at the DT point.

[0113] Here, the DT point may be a DT service providing place within a predetermined range from the mobile tool.

[0114] In addition, the DT point of the present invention may include not only a place providing DT services, but also a business office that can provide DT goods to the mobile tool and receive information on goods selected and ordered by the mobile tool without providing DT services. For example, the DT point of the present invention may represent a business office where a customer in the mobile tool can select and order goods provided by the DT point, but the mobile tool needs to be parked / stopped at a separate place to pick up the ordered goods. That is, the DT point of the present invention may include a drive-in business office without a drive-through lane.

[0115] The image generation device of the present invention can use an artificial intelligence model to generate an image related to the DT point from the brain wave signals of the passengers. For example, an image related to the menu or items provided by the DT point can be generated. In addition, the image generation device of the present invention can select an image included in a predetermined list based on the generated image. For example, an image included in the list of items provided by the DT point or a predetermined menu can be selected. In addition, the image generation device of the present invention can control the mobile tool based on the selected image or provide information on a predetermined item selected by the passenger to the DT point.

[0116] In addition, artificial intelligence technology enables a computer to learn data and make decisions autonomously like a human. An artificial neural network is a mathematical model inspired by a biological neural network, and can represent an entire model that has the ability to solve problems by allowing artificial neurons formed by synaptic connections to learn and change the synaptic connection strength. An artificial neural network may include an input layer, a hidden layer, and an output layer. Neurons included in each layer are connected by weights, and an artificial neural network may have a form that can approximate a complex function through a linear combination of weights and neuron values and a non-linear activation function. The learning purpose of an artificial neural network is to find a weight that minimizes the difference between the output calculated at the output layer and the actual output value.

[0117] A deep neural network is an artificial neural network composed of multiple hidden layers between an input layer and an output layer, and can model complex non-linear relationships through multiple hidden layers. A neural network structure that can perform advanced abstraction by increasing the number of layers is called deep learning. In deep learning, since a large amount of data is learned when new data is input and the answer with the highest probability is selected based on the learning result, operations can be adaptively performed according to images, and feature factors can be automatically discovered based on data during the process of learning the model.

[0118] The deep learning-based models of the present invention may include, but are not limited to, fully convolutional neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), and deep belief networks (DBNs). Alternatively, in addition to deep learning, machine learning methods may be included, or a hybrid model that is a combination of deep learning and machine learning may be included. For example, features of an image may be extracted by applying a deep learning-based model, and the image may be classified and recognized based on the extracted features by applying a machine learning-based model. The machine learning-based model may include, but is not limited to, support vector machines (SVMs), AdaBoost, etc. Herein, the RNN may include long short-term memory (LSTM).

[0119] In addition, the method of learning the deep learning-based model of the present invention may include, but is not limited to, at least one of supervised learning, unsupervised learning, or reinforcement learning. Supervised learning is performed using a series of learning data and their corresponding labels (target output values), and a neural network model based on supervised learning may be a model that infers a function from training data. In supervised learning, a series of learning data and their corresponding target output values are received, errors are discovered through learning, the actual output value of the input data is compared with the target output value, and then the model is modified based on the corresponding result. Supervised learning may be classified into regression, classification, detection, and semantic segmentation according to the form of the result. The function obtained through supervised learning may be used again to predict a new result value. The neural network model based on supervised learning optimizes the parameters of the neural network model by learning a large amount of training data.

[0120] Different from supervised learning, unsupervised learning is a method of performing learning without labeling data. That is, unsupervised learning represents a learning method of teaching a learning algorithm without known output values or information, and the learning algorithm should only utilize the input data to extract knowledge from the data. For example, unsupervised transformation may represent a method of re-expressing data so that it is easier for a person or another machine of the learning algorithm to interpret the new data than the original data. For example, dimensionality reduction may be transformed to include only necessary features while reducing the number of features from many high-dimensional data. As another example, clustering may represent a method of dividing data with similar characteristics into groups, that is, representing a grouping with common features that appears in unlabeled pictures.

[0121] As another example, there is a generative adversarial network (GAN) model that has been actively studied in the field of creating or restoring images or imitating movements. The GAN model can represent an artificial neural network or deep learning, in which two types of relative systems perform learning in a competitive manner with each other, and can represent an algorithm that uses an adversarial learning method to learn a model created by deep learning to solve a generation problem. The GAN model can learn information through a process in which a generator and a discriminator compete with each other. Specifically, the generator can be used to generate imitation data similar to existing data. If the imitation data of the generator is close to the actual data, the discriminator may lose its discrimination function. In addition, the discriminator can be used to determine whether the input data is actual data or imitation data. If the discriminator determines that there is a 50% probability that the imitation data calculated by the generator is true, the learning can end. At the same time, the GAN model of the present invention can include a deep convolutional GAN (DCGAN) model that can perform more stable learning.

[0122] Reference Figure 7 , the image generation device 700 may include a sensor 710 and / or a controller 720. However, it should be noted that only some components necessary for explaining the present embodiment are shown, and the components included in the image generation device 700 are not limited to the above examples. For example, two or more constituent units may be implemented as one constituent unit, and the operations performed in one constituent unit may be divided and processed in two or more constituent units. In addition, some constituent units may be omitted, or additional constituent units may be added.

[0123] The image generation device 700 of the present invention can collect the electroencephalogram signals of at least one passenger in a moving tool from multiple channels within a predetermined time. In addition, the sensor 710 can perform the above operations.

[0124] Here, the electroencephalogram signals collected from multiple channels can represent electroencephalogram signals in at least one of the time domain, frequency domain, or spatial domain. Here, the spatial domain can represent the electroencephalogram signal measurement channels.

[0125] The image generation device 700 of the present invention can generate a first image related to a service point from the electroencephalogram signals collected from multiple channels using an artificial intelligence model. In addition, the controller 720 can perform the above operations.

[0126] Here, the service point can represent a DT point.

[0127] For example, the service point may be a DT service - providing place within a predetermined range from the mobile tool. Alternatively, when the mobile tool is within a predetermined range from the service point, the mobile tool may receive information about goods. The number or type of service points may vary according to the location of the mobile tool. The image - generating device 700 of the present invention may further include a receiver (not shown). The receiver may perform the above - mentioned operations.

[0128] Figure 8 FIG. is a diagram showing the range within which a mobile tool and a service point perform mutual transmission and reception according to an embodiment of the present invention.

[0129] Reference Figure 8 , the first service point 800 may send information about goods provided by the first service point 800 to the mobile tool 820 within the first range 802 from the first service point 800. Conversely, the second service point 810 does not have a mobile tool within the second range 812 from the second service point 810 to which it can send information about goods. Alternatively, the mobile tool 820 may receive information about goods from the first service point 800 and the second service point 810 within the third range 822 from the mobile tool 820.

[0130] As another example, the service point may represent a place input by the user or a place predetermined in the mobile tool. Alternatively, the service point may represent a place automatically detected according to a predetermined condition in the mobile tool's navigation device. The service point can be set in different ways according to the user in the mobile tool. For example, the service point can be set by inputting a user - preferred service point.

[0131] As another example, the service points can be grouped according to the nature of the service - providing place. For example, the service points can be grouped into fast - food restaurants, coffee shops, bakeries, convenience stores, banks, ticket offices according to the nature or type of goods and services provided, and at least one grouped list can be selected according to the choice of the passengers in the mobile tool. Additionally, for example, the grouped list can be displayed on the display of the mobile tool, and at least one grouped list can be selected according to the choice of the passengers in the mobile tool in response to the display.

[0132] Here, the predetermined range may represent a distance of several kilometers or dozens of kilometers from the mobile tool and / or the service point in the radial direction. Additionally, the predetermined range can be set based on a communication network. For example, when transmission and reception are performed between the mobile tool and the service point based on a short - range communication network, the predetermined range may be the range capable of performing transmission and reception through the short - range communication network. At this time, beacons can be used in the short - range communication network.

[0133] Here, the merchandise can represent DT merchandise. For example, the merchandise can represent items such as hamburgers, coffee, and bread. That is, the merchandise can represent items that a passenger wants to purchase.

[0134] In addition, the merchandise can include services provided by a service point.

[0135] In addition, the merchandise can include the name, image, logo, etc. of a service point.

[0136] Here, the information about the merchandise can represent the information about DT merchandise.

[0137] For example, the information about DT merchandise can include the image, type, price, quantity, and name information of the merchandise.

[0138] As another example, the information about DT merchandise can include reservation information provided by a service point. For example, the information about DT merchandise can include new menus, events, discount events, etc. provided by a service point.

[0139] As another example, the information about DT merchandise can be set based on the preferences of a passenger. To this end, the information about DT merchandise can be the result of pre-learning performed by a passenger. In addition, the information about DT merchandise can be updated in real time.

[0140] In addition, the artificial intelligence model can represent a generative adversarial network (GAN) model. The GAN model can include a deep convolutional GAN (DCGAN) model. In addition, the artificial intelligence model can represent a model obtained by combining an RNN-based model with a GAN model to process brain wave signals.

[0141] The GAN model can be used to pre-train the artificial intelligence model for the user's brain wave signals and the image of the corresponding service point. Here, the image of the service point can represent an image indicating a DT point or an image indicating the merchandise provided by a DT point.

[0142] Here, the first image is at least one of a service point image indicating a service point or a merchandise image. The service point image can represent an image indicating a service point, and the merchandise image can represent an image indicating the merchandise provided by a service point. Here, the merchandise image can represent an image of items, menus, products, etc. provided by a DT point.

[0143] For example, when the first image is a service point image, the first image can represent the logo or store image of a fast food restaurant, coffee shop, bakery, convenience store, bank, or ticket office.

[0144] As another example, when the first image is a merchandise image, the first image can represent an image of a hamburger or a drink provided by a fast food restaurant.

[0145] As another example, when the first image is a product image, the first image may represent an image of an item provided by a bakery.

[0146] In addition, during the process of generating the first image, electroencephalogram (EEG) signals collected from multiple channels may pass through a predetermined encoder to generate an EEG feature vector. Here, the predetermined encoder may include an LSTM layer and a non-linear layer (e.g., a fully connected layer including a ReLU non-linear activation function).

[0147] Referring again to Figure 7 , the image generation device 700 of the present invention may determine whether the first image is similar to a preset second image. Additionally, the controller 720 may perform the above operations.

[0148] Here, the second image may represent a service point image or a product image.

[0149] For example, the image generation device 700 of the present invention may determine the similarity based on the determination of the similarity between the first image and the second image.

[0150] Here, for the determination of similarity, various similarity determination methods commonly used in the field of image recognition or classification may be applied, such as a method of extracting feature points of the input image to determine similarity.

[0151] In addition, when determining the similarity, the degree of similarity between the images may be compared with a predetermined threshold. The predetermined threshold may vary according to the first image and / or the second image. Here, the degree of similarity between the images may be expressed as a probability or a predetermined value.

[0152] For example, the threshold when the first image is a service point image may be relatively smaller than the threshold when the first image is a product image. For example, if the threshold when the first image is a service point image is 0.6, then the threshold when the first image is a product image may be 0.8. For example, if the threshold when the first image is an image of a coffee shop logo is 0.6, then the threshold when the first image is a hamburger may be 0.8. That is, the degree of similarity between product images (e.g., hamburger images and beverage images) is generally greater than the degree of similarity between service point images (e.g., fast food restaurant images and coffee shop images). By differentiating the thresholds, compared with the case where the first image is a service point image, the degree of similarity between the images can be determined more strictly when the first image is a product image.

[0153] As another example, the threshold value when the first image in the merchandise image is a "hamburger" image can be relatively smaller than the threshold value when the first image is a "drink" image. For example, if the threshold value when the first image is a "hamburger" image is 0.5, then the threshold value when the first image is a "drink" image can be 0.7. That is, even when the types of merchandise images are the same, the degree of similarity of a specific merchandise image can be determined more strictly or less strictly than that of another merchandise image.

[0154] That is, the image generation device 700 of the present invention can select a service point determined to be desired by the passenger by determining the similarity. Alternatively, the image generation device 700 of the present invention can select a merchandise determined to be desired by the passenger by determining the similarity.

[0155] Here, the second image can be set by user input or can be preset in the mobile tool. In addition, the second image can be preset for each service point image and merchandise image. For example, the second image can be configured as a pair (service point image, merchandise image). For example, the second image can be configured as (fast food restaurant, predetermined hamburger), (coffee shop, predetermined coffee), etc.

[0156] In addition, the second image can be determined based on the generated first image. For example, when the first image is a service point image, the second image can be determined as a preset service point image. Alternatively, when the first image is a merchandise image, the second image can be determined as a preset merchandise image. That is, the second image can be determined as an image that matches the type of the first image in order to determine the similarity between the images.

[0157] Alternatively, the second image can be a preset image regardless of the generated first image. That is, the second image can be preset regardless of the nature, type, etc. of the first image. For example, the second image can be a predetermined "hamburger" image regardless of whether the first image is a service point image or a merchandise image. Alternatively, the second image can be a predetermined "coffee shop" image regardless of whether the first image is a service point image or a merchandise image.

[0158] That is, the image generation device 700 of the present invention can pre-store in the form of images the service points that the passenger often visits or the merchandise preferred by the passenger in the mobile tool, and determine whether the image generated based on the passenger's brain wave signal is similar to the stored image.

[0159] In addition, the image generation device 700 of the present invention can control the mobile tool or provide predetermined information to the service point based on the result of determining the similarity. In addition, the controller 720 can perform the above operations.

[0160] That is, as described above, the image generation device 700 of the present invention can determine what information the passenger expects or has in mind by using similarity determination. Additionally, based on the determination, the mobile tool can be controlled or a predetermined message can be sent to the service point to answer this purpose.

[0161] For example, when it is determined that the first image is similar to the second image, the driving route can be changed to the service point corresponding to the selected second image, or the driving route can be provided to the passenger to guide the passenger to select a route. Alternatively, a notice of changing the driving route can be provided to the passenger.

[0162] As another example, when it is determined that the first image is similar to the second image, an order signal for the product corresponding to the second image can be sent to the service point. For example, when the first image is an image of "coffee" and is determined to be similar to a preset second image, an order signal for coffee can be sent to the coffee shop corresponding to the coffee. Alternatively, the driving route can be changed to the coffee shop corresponding to the coffee, or the driving route can be provided to the passenger to guide the passenger to select a route. Alternatively, a notice of changing the driving route can be provided to the passenger.

[0163] Figure 9 is a flowchart showing a method of operating an image generation device according to an embodiment of the present invention.

[0164] In step S901, electroencephalogram signals of at least one passenger in the mobile tool can be collected from multiple channels within a predetermined time.

[0165] Here, the electroencephalogram signals collected from multiple channels can represent electroencephalogram signals in at least one of the time domain, frequency domain, or spatial domain.

[0166] In step S902, a first image of the service point can be generated from the electroencephalogram signals collected from multiple channels by using an artificial intelligence model.

[0167] Here, the artificial intelligence model can represent a generative adversarial network (GAN) model. The GAN model can include a deep convolutional GAN (DCGAN) model. Additionally, the artificial intelligence model can represent a model obtained by combining an RNN-based model with the GAN model to process electroencephalogram signals.

[0168] Here, the first image is at least one of a service point image indicating the service point or a product image. The service point image can represent an image indicating the service point, and the product image can represent an image indicating the product provided by the service point.

[0169] In addition, a GAN model can be used to pre-train an artificial intelligence model for the user's brainwave signals and the images of the service points corresponding thereto. Herein, the images of the service points can represent images indicating the DT points or images indicating the goods provided at the DT points.

[0170] In addition, in the process of generating the first image, the brainwave signals collected from multiple channels can pass through a predetermined encoder to generate EEG feature vectors. Herein, the predetermined encoder can include an LSTM layer and a non-linear layer (e.g., a fully connected layer including a ReLU non-linear activation function).

[0171] In step S903, it can be determined whether the generated first image is similar to a preset second image.

[0172] Herein, the second image can represent a service point image or a product image.

[0173] For example, the image generation device 700 of the present invention can determine the similarity based on the determination of the similarity between the first image and the second image.

[0174] Herein, for the determination of the similarity, various similarity determination methods commonly used in the field of image recognition or classification can be applied, such as a method of extracting feature points of the input image to determine the similarity.

[0175] In addition, when determining the similarity, the degree of similarity between the images can be compared with a predetermined threshold. The predetermined threshold can vary according to the first image and / or the second image. Herein, the degree of similarity between the images can be expressed as a probability or a predetermined value.

[0176] In step S904, the mobile tool can be controlled based on the determination result or predetermined information can be provided to the service point.

[0177] That is, what information the passenger expects or thinks of can be determined by using the similarity determination. In addition, according to the determination, the mobile tool can be controlled or predetermined information can be sent to the service point to answer this purpose.

[0178] Figure 10 It is a flowchart showing a method of operating an image generation device according to an embodiment of the present invention.

[0179] In addition, the image generation device of the present invention can update the artificial intelligence model through the following process: generating a first image of a service point from the passenger's brainwave signals and determining the similarity between the generated first image and a preset second image. That is, the passenger's brainwave signals and / or the first image can be added as learning data of the artificial intelligence model.

[0180] For example, the image generation device of the present invention may further perform a process of determining whether the similarity determination result meets the passenger's expectation. Additionally, based on whether the second image meets the passenger's expectation, the probability of generating the first image in the artificial intelligence model may be increased or decreased.

[0181] Steps S1001 to S1003 may respectively correspond to Figure 9 steps S901 to S903 of Figure 9 and thus their detailed processes have been described above with reference to

[0182] In step S1004, a process of determining whether the similarity determination result meets the passenger's expectation may be performed. For example, a preset second image or a name corresponding to the second image may be displayed on a predetermined display of the mobile tool, and in response to the display, a user input of selecting "yes" or "no" may be received. Alternatively, the user can determine whether the driving route has been changed to a service point corresponding to the preset second image, and in response thereto, a user input of selecting "yes" or "no" may be received. Alternatively, the user can determine whether an order signal for a commodity corresponding to the preset second image has been sent, and in response thereto, a user input of selecting "yes" or "no" may be received.

[0183] In step S1005, the artificial intelligence model may be updated based on whether the determination result meets the passenger's expectation.

[0184] For example, when the determination result meets the passenger's expectation, the probability of generating the first image in the artificial intelligence model is increased by a predetermined value, and when the determination result does not meet the passenger's expectation, the probability of generating the first image in the artificial intelligence model is decreased by a predetermined value.

[0185] According to the present invention, an apparatus and method for generating an image based on a passenger's brain wave signal can be provided.

[0186] According to the present invention, an apparatus and method for generating an image from a passenger's brain wave signal using an artificial intelligence model and controlling a mobile tool based on the generated image can be provided.

[0187] The effects obtained in the present invention are not limited to the above effects, and according to the following description, those skilled in the art can clearly understand other effects not mentioned above.

[0188] Although, for the sake of clarity of description, the exemplary method of the present invention is described as a series of operation steps, the present invention is not limited to the order or sequence of the above operation steps. The operation steps may be performed simultaneously or may be performed in a different order sequentially. To implement the method of the present invention, additional operation steps may be added and / or existing operation steps may be eliminated or replaced.

[0189] Rather than presenting all available combinations, the various embodiments of the present invention are presented as only describing representative combinations. The steps or elements in the various embodiments can be utilized alone or in combination.

[0190] In addition, the various embodiments of the present invention can be implemented in the form of hardware, firmware, software, or a combination thereof. When the present invention is implemented as a hardware component, it can be, for example, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[0191] The scope of the present invention includes software or machine-executable instructions (e.g., an operating system (OS), an application program, firmware, a program) that enable various forms of methods to be executed on a device or a computer, and a non-volatile computer-readable medium that stores such software or machine-executable instructions so that the software or instructions can be executed on the device or the computer.

[0192] The description of the present invention is exemplary in nature, and thus, variations that do not depart from the essence of the present invention are intended to fall within the scope of the present invention. Such variations should not be regarded as departing from the spirit and scope of the present invention.

Claims

1. An apparatus for generating an image using electroencephalogram signals, the apparatus comprising: A sensor configured to collect electroencephalogram signals of at least one passenger in a moving vehicle from multiple channels within a predetermined time; And A controller configured to generate a first image of a service point from the electroencephalogram signals collected from multiple channels using an artificial intelligence model, determine whether the generated first image is similar to a preset second image, and control the moving vehicle based on the determination result; Wherein the first image is at least one of a service point image indicating a service point or a commodity image, the service point image is an image indicating a service point, the commodity image is an image indicating a commodity provided by the service point, and the service point is a drive-through DT service providing place within a predetermined range from the moving vehicle.

2. The apparatus for generating an image using electroencephalogram signals according to claim 1, wherein, The electroencephalogram signals collected from multiple channels are electroencephalogram signals in at least one of the time domain, frequency domain or spatial domain.

3. The apparatus for generating an image using an electroencephalogram signal according to claim 1, wherein, The artificial intelligence model is a generative adversarial network GAN model.

4. The apparatus for generating an image using an electroencephalogram signal according to claim 1, wherein, The commodity is at least one of an item provided by the service point, a service provided by the service point, or information about the service point.

5. The apparatus for generating an image using an electroencephalogram signal according to claim 1, wherein, When the first image is a service point image, the controller determines the second image as a preset service point image and determines whether the first image is similar to the second image.

6. The apparatus for generating an image using an electroencephalogram signal according to claim 5, wherein, When it is determined that the first image is similar to the second image, the controller performs at least one of the following: changing the driving route of the moving vehicle to the service point, providing a notification of the driving route to be changed to the passenger, or guiding the passenger to select a route by providing the driving route to be changed to the passenger.

7. The apparatus for generating an image using an electroencephalogram signal according to claim 1, wherein, When the first image is a commodity image, the controller determines the second image as a preset commodity image and determines whether the first image is similar to the second image.

8. The apparatus for generating an image using an electroencephalogram signal according to claim 7, wherein, When it is determined that the first image is similar to the second image, the controller performs at least one of the following: changing the driving route of the moving vehicle to the service point providing the commodity, providing a notification of the driving route to be changed to the passenger, guiding the passenger to select a route by providing the driving route to be changed to the passenger, or sending an order signal for the commodity.

9. The apparatus for generating an image using an electroencephalogram signal according to claim 1, wherein, The controller is further configured to determine whether the determination result meets the passenger's expectation.

10. A method for generating an image using electroencephalogram signals, the method comprising: Collecting electroencephalogram signals of at least one passenger in a moving vehicle from multiple channels through a sensor within a predetermined time; Generating a first image of a service point from the electroencephalogram signals collected from multiple channels by a controller using an artificial intelligence model; Determining whether the generated first image is similar to a preset second image; Controlling the moving vehicle based on the determination result; Wherein the first image is at least one of a service point image indicating a service point or a commodity image, the service point image is an image indicating a service point, the commodity image is an image indicating a commodity provided by the service point, and the service point is a drive-through DT service providing place within a predetermined range from the moving vehicle.

11. The method for generating an image using electroencephalogram signals according to claim 10, wherein, The electroencephalogram signals collected from multiple channels are electroencephalogram signals in at least one of the time domain, frequency domain or spatial domain.

12. The method for generating an image using electroencephalogram signals according to claim 10, wherein, The artificial intelligence model is a generative adversarial network GAN model.

13. The method for generating an image using electroencephalogram signals according to claim 10, wherein, The commodity is at least one of an item provided by the service point, a service provided by the service point, or information about the service point.

14. The method for generating an image using electroencephalogram signals according to claim 10, wherein, When the first image is a service point image, determining whether the first image is similar to the second image includes: determining the second image as a preset service point image, and determining whether the first image is similar to the second image.

15. The method for generating an image using electroencephalogram signals according to claim 14, wherein, When it is determined that the first image is similar to the second image, the control of the mobile tool includes performing at least one of the following: changing the driving route of the mobile tool to the service point, providing the passenger with a notice of the driving route to be changed, or guiding the passenger to select a route by providing the passenger with the driving route to be changed.

16. The method for generating an image using electroencephalogram signals according to claim 10, wherein, When the first image is a commodity image, determining whether the first image is similar to the second image includes: determining the second image as a preset commodity image, and determining whether the first image is similar to the commodity image.

17. The method for generating an image using electroencephalogram signals according to claim 16, wherein, When it is determined that the first image is similar to the second image, the control of the mobile tool includes performing at least one of the following: changing the driving route of the mobile tool to the service point providing the commodity, providing the passenger with a notice of the driving route to be changed, guiding the passenger to select a route by providing the passenger with the driving route to be changed, or sending an order signal for the commodity.

18. The method for generating an image using an electroencephalogram signal according to claim 10, further comprising determining whether the determination result meets the passenger's expectation.

Images