Apparatus and method for presenting visual stimuli by using augmented reality

Abstract

The present disclosure relates to devices and methods for presenting visual stimuli by using augmented reality. The apparatus may include an augmented reality module configured to recognize a control target through augmented reality (AR) glasses and place a visual stimulus on the recognized control target; a brain wave measurement module configured to measure brain waves of the user gazing at the visual stimulation; a visual stimulus detection module configured to analyze the measured brain waves to detect a visual stimulus signal (VEP), and process the detected visual stimulus signal to classify the visual stimulus and identify a control target; and a control module configured to control the identified control target through a network.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of Korean Patent Application No. 10-2023-0156153 filed in the Korean Intellectual Property Office on November 13, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to an apparatus and method for presenting visual stimulation by using augmented reality. More specifically, the present disclosure relates to an apparatus and method for presenting visual stimulation by using augmented reality for an immersive augmented reality (AR) based brain computer interface (BCI) system. Background Art

[0004] BCI refers to a technology that operates external devices using only thoughts, and transmits desired commands mainly by collecting and analyzing brain wave signals. Brain wave signals can be divided into invasive brain waves and non-invasive brain waves. Invasive brain waves use sensors that invade the scalp, and non-invasive brain waves sense brain waves by attaching electrodes on the surface of the scalp. For practicality, non-invasive BCI technology is mainly studied. In particular, it is well known that steady-state visual evoked potentials (SSVEPs) generated when looking at visual stimuli that flicker at a specific frequency, or P300 potentials generated due to unpredicted visual stimuli, etc. can be observed in brain waves. BCI technologies based on visual evoked potentials (such as SSVEP or P300) are mainly used as communication means for patients.

[0005] Recently, due to the development of AR technology, which can augment virtual information in real space at any time, research for combining AR technology and BCI technology is being actively conducted. The advantage is that BCI technology can be used without spatial limitations because visual stimuli can be projected onto AR glasses without having to present them on a computer monitor as before.

[0006] The method for presenting SSVEP visual stimulation is mainly based on the research of existing monitors that simply borrows the 2D form. However, since the existing research uses 2D visual stimulation as it is, it hinders the user's natural AR interaction and may be degraded. Summary of the invention

[0007] The present disclosure attempts to provide an apparatus and method for presenting visual stimulation by using augmented reality, which is capable of synthesizing virtual steady-state visual evoked potential (SSVEP)-based visual stimulation into an object in the real world.

[0008] The present disclosure attempts to provide an apparatus and method for presenting visual stimulation by using augmented reality, which is capable of presenting visual stimulation at an appropriate position in consideration of a surrounding environment and a brain computer interface (BCI) control target.

[0009] An apparatus for presenting visual stimulation by using augmented reality may include: an augmented reality module configured to identify a control target through augmented reality (AR) glasses and place visual stimulation on the identified control target; a brain wave measurement module configured to measure brain waves of a user who is looking at the visual stimulation; a visual stimulation detection module configured to analyze the measured brain waves to detect a visual stimulation signal (VEP), and process the detected visual stimulation signal to classify the visual stimulation and identify the control target; and a control module configured to control the identified control target through a network.

[0010] The augmented reality module may be configured to register the image of the control target received from the database.

[0011] The augmented reality module may include a control target detector configured to detect a position of a control target gazed at by a user through the AR glasses and place a specific device for identifying the control target.

[0012] The specific device may include a virtual outline placed on the outline of the control target.

[0013] The augmented reality module may further include a visual stimulation placement unit configured to place a steady-state visual evoked potential (SSVEP)-based visual stimulation flickering at a specific frequency on the contour.

[0014] The visual stimulus placement unit may be configured to place the additional visual stimulus providing an interface for controlling the control target at a specific position that does not overlap with the outline of the identified control target.

[0015] The visual stimulation detection module may include: an EEG signal receiver configured to receive an EEG signal; a visual stimulation signal detector configured to analyze the received EEG signal to detect a visual stimulation signal corresponding to a specific frequency; a feature extraction unit configured to extract features of the visual stimulation signal; and a visual stimulation classifier configured to classify the visual stimulation and identify the control target based on the extracted features.

[0016] The augmented reality module can be configured to detect the positions of multiple control targets through AR glasses, place virtual contours on the contours of the multiple control targets respectively, and place VEP-based visual stimuli that flash at different frequencies on the virtual contours respectively, and the virtual contours can include three-dimensional contours.

[0017] The visual stimulation detection module can be configured to analyze the similarity between the visual stimulation signal detected from the brain waves of the user who is looking at the visual stimulation having a specific frequency and the reference signal according to a predetermined frequency to detect the frequency with the highest similarity, and identify the control target looked at by the user based on the detected frequency.

[0018] The visual stimulus detection module may be configured to place an additional visual stimulus providing an interface for additional interaction at a specific location that does not overlap with the location of the control target after identifying the control target.

[0019] A method for presenting visual stimulation by using augmented reality may include: identifying a control target that a user is gazing at through augmented reality (AR) glasses, and placing visual stimulation on the identified control target; measuring brain waves of the user gazing at the visual stimulation; analyzing the measured brain waves to detect visual stimulation signals (VEP), and processing the detected visual stimulation signals to classify the visual stimulation and identify the control target; and controlling the identified control target through a network.

[0020] Placing the visual stimulus may include detecting a position of the control target in three-dimensional space and placing the virtual outline on the three-dimensional outline of the control target.

[0021] Placing the visual stimulus may further include placing a steady-state visual evoked potential (SSVEP) based visual stimulus flickering at a specific frequency on the virtual contour.

[0022] Identifying the control target may include extracting features of the visual stimulation signal, and classifying the visual stimulation based on the extracted features.

[0023] Placing the visual stimulus may further include, when the control target is identified, placing an additional visual stimulus providing an interface for controlling the control target at a position that does not overlap with the control target.

[0024] Placing the visual stimulus may include detecting positions of the plurality of control targets in the three-dimensional space to place a three-dimensional virtual contour on each contour of the plurality of control targets.

[0025] Placing the visual stimulus may further include placing VEP-based visual stimuli that flicker at different frequencies on the virtual contour.

[0026] Identifying a control target may include analyzing similarities between a visual stimulation signal detected from a user gazing at a visual stimulation of a specific frequency visual stimulation having different frequencies and a reference signal based on a predefined frequency to detect a frequency having the highest similarity, and identifying a control target gazed by the user based on the detected frequency.

[0027] Arranging the visual stimulus may further include placing an additional visual stimulus for controlling the identified control target on the three-dimensional space so as not to overlap with the identified control target.

[0028] The visual stimulus may include an image of a checkerboard that is reversed at a specific frequency.

[0029] An apparatus and method for presenting visual stimulation by using augmented reality according to an embodiment may synthesize a visual stimulation based on a virtual steady-state visual evoked potential (SSVEP) into an object in the real world.

[0030] An apparatus and method for presenting visual stimulation by using augmented reality according to an embodiment may provide natural augmented reality brain-computer interaction by presenting visual stimulation at an appropriate location in consideration of a surrounding environment and a brain-computer interface (BCI) control target. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a diagram illustrating a brain computer interface (BCI) system including a device for presenting visual stimulation by using augmented reality according to an embodiment.

[0032] Figure 2 is a block diagram of an apparatus for presenting visual stimulation by using augmented reality according to an embodiment.

[0033] Figure 3 is a diagram illustrating an operation process of an apparatus for presenting visual stimulation by using augmented reality according to an embodiment.

[0034] Figure 4 and Figure 5 is a flow chart illustrating a method of presenting visual stimulation by using augmented reality according to an embodiment.

[0035] Figure 6 is an example diagram illustrating a process of presenting and controlling visual stimulation with respect to a single control target according to an embodiment.

[0036] Figure 7 is an example diagram illustrating a process of presenting and controlling visual stimuli with respect to multiple control targets according to an embodiment.

[0037] Figure 8 is a diagram for explaining a computing device according to an embodiment. DETAILED DESCRIPTION

[0038] Hereinafter, the embodiments of the present disclosure will be described more fully with reference to the accompanying drawings so that those skilled in the art can easily implement the embodiments. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present disclosure. In order to clarify the present disclosure, components not related to the description will be omitted, and the same elements or equivalents are represented by the same reference numerals throughout the specification.

[0039] In addition, unless explicitly described to the contrary, the word "comprise" and variations such as "comprises" or "comprising" will be understood to imply the inclusion of the elements described but not the exclusion of any other elements. Terms including common numbers such as first and second are used to describe various constituent elements, but the constituent elements are not limited by the terms. These terms are only used to distinguish one component from other components.

[0040] In addition, the terms "unit", "part" or "portion", "-er" and "module" in the specification refer to a unit that processes at least one function or operation, which can be implemented by hardware, software, or a combination of hardware and software.

[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0042] Figure 1 is a diagram illustrating a brain computer interface (BCI) system including a device for presenting visual stimulation by using augmented reality according to an embodiment. Figure 1 An immersive augmented reality (AR) brain-computer interface (BCI) system is shown according to an embodiment.

[0043] The apparatus 100 for presenting visual stimulation by using augmented reality (hereinafter, referred to as a visual stimulation presenting apparatus) proposes a new method for presenting visual stimulation for a BCI system using immersive augmented reality. The visual stimulation presenting apparatus 100 proposes a method for synthesizing virtual SSVEP visual stimulation into an object in the real world. That is, the visual stimulation presenting apparatus 100 can synthesize and present the virtual SSVEP visual stimulation to a control target 200, which is an object in the real world that a user is looking at through AR glasses AG.

[0044] The visual stimulation presenting apparatus 100 may analyze a brain wave signal (EEG) received through an electroencephalogram measuring headband (EHB) connected with electrodes for measuring brain waves of a user gazing at visual stimulation to detect the visual stimulation and recognize the control target 200 .

[0045] The visual stimulation presenting apparatus 100 may communicate with the identified control target 200 through a network, and may perform control, such as providing an operation command, according to the detected visual stimulation.

[0046] The control target 200 may include various external devices. The control target 200 may be a device that can communicate with the visual stimulation presentation apparatus 100 via Bluetooth, Wi-Fi, IoT, cloud, etc. For example, the control target 200 may include a smart home appliance, a robot arm, an electric wheelchair, etc.

[0047] Figure 2 is a block diagram of an apparatus for presenting visual stimulation by using augmented reality according to an embodiment.

[0048] refer to Figure 2 , an apparatus 100 for presenting visual stimulation by using augmented reality may include an augmented reality module 110 , a brain wave measurement module 120 , a visual stimulation detection module 130 , and a control module 140 .

[0049] The augmented reality module 110 may include AR glasses. The augmented reality module 110 may be connected to the AR glasses via a wired and / or wireless network. The augmented reality module 110 may identify a control target that the user is looking at through the AR glasses, and arrange or synthesize visual stimulation for the identified control target.

[0050] The augmented reality module 110 may detect the location where the control target exists in the three-dimensional real world. The augmented reality module 110 may detect the outline of the control target. The augmented reality module 110 may mark the control target based on the outline of the control target.

[0051] For example, the augmented reality module 110 may arrange the outline on the outline of the control target. The augmented reality module 110 may synthesize the three-dimensional virtual outline onto the outline of the control target in the three-dimensional real world.

[0052] The augmented reality module 110 may place a visual stimulus based on a steady-state visual evoked potential (SSVEP) on the virtual silhouette. The visual stimulus may include a virtual image that flickers at a specific frequency. The visual stimulus may include a checkerboard that reverses according to a specific frequency.

[0053] The brain wave measurement module 120 can measure the brain waves of the user who is looking at the visual stimulus. The brain wave measurement module 120 can be connected to a brain wave measurement headband wrapped around the user's head through a network. The brain wave measurement module 120 can detect EEG signals by measuring the user's brain waves through the brain wave measurement headband. The brain wave measurement module 120 can transmit the EEG signals to the visual stimulus detection module 130.

[0054] The visual stimulus detection module 130 may analyze the EEG signal to detect a visual stimulus signal. The visual stimulus signal may include a steady-state visual evoked potential (SSVEP). The visual stimulus detection module 130 may process the steady-state visual evoked potential to classify the visual stimulus.

[0055] The visual stimulation detection module 130 may classify the visual stimulation that the user is gazing at to identify the control target that the user is gazing at. The visual stimulation signal may include a brain wave signal generated according to the visual stimulation and may be referred to as a visual stimulation induced signal or a visual induced signal.

[0056] The control module 140 may be connected to the visual stimulus detection module 130 through a network and may control a control target. That is, the control module 140 may perform control with respect to the identified control target based on the detected visual stimulus.

[0057] Figure 3 is a diagram illustrating an operation process of an apparatus for presenting visual stimulation by using augmented reality according to an embodiment.

[0058] exist Figure 3 Among the objects located in the field of view (FOV) of the user wearing the AR glasses AG and detected by the user's gaze, the visual stimulus presentation device 100 (reference Figure 1 ) The object of communication may correspond to the control target 200. For example, an intelligent lighting device, an intelligent monitor, an intelligent computer, etc. may correspond to the control target 200. The visual stimulation presentation apparatus 100 may synthesize a visual stimulation STI based on a virtual SSVEP on the control target 200.

[0059] The augmented reality module 110 may receive and register a control target image IMG from a database. The control target image IMG may include image information about a control target 200 to be controlled by the visual stimulation presentation device 100. The control target image IMG may be used to generate a virtual visual stimulation STI. The control target image IMG may be used to generate a virtual contour.

[0060] The augmented reality module 110 may include a control target detector 111 and a visual stimulation arrangement unit 112 .

[0061] The control target detector 111 may detect the position of the control target 200 that the user is looking at through the AR glasses AG, and may place an identifying means for identifying the control target to the control target. For example, the control target detector 111 may arrange a virtual outline as the outline of the control target. The control target detector 111 may confirm the control target by using various means capable of identifying or specifying the control target.

[0062] The visual stimulation arrangement unit 112 may place the SSVEP-based visual stimulation STI flickering at a specific frequency on the virtual outline. In an embodiment, the visual stimulation arrangement unit 112 may arrange an additional visual stimulation providing an interface for controlling the control target 200 at a specific position that does not overlap with the outline of the control target.

[0063] The brainwave measurement module 120 may detect a brainwave signal (EEG) through a brainwave measurement headband (EHB) worn by a user. The brainwave measurement module 120 may include a sensor unit 121 and an AFE 122 .

[0064] The sensor unit 121 may sense a brain wave signal measured at a brain wave measurement headband (EHB), and the AFE 122 may amplify the sensed brain wave signal and convert it into a digital signal.

[0065] The brainwave measurement module 120 may be connected to the visual stimulation detection module 130 through a network.

[0066] The visual stimulation detection module 130 may include an EEG receiver 131 , a visual stimulation signal detector 132 , a feature extraction unit 133 , and a visual stimulation classifier 134 .

[0067] The brainwave signal receiver 131 may receive the EEG signal from the sensor unit 121 of the brainwave measurement module 120 .

[0068] The visual stimulation signal detector 132 may analyze the received brain wave signal to detect a visual stimulation signal (VEP) corresponding to a specific frequency. The visual stimulation signal detector 132 may perform pre-processing of the visual stimulation signal (SSVEP) for feature extraction.

[0069] The feature extraction unit 133 may extract features of the visual stimulation signal. The feature extraction unit 133 may extract feature data for classifying the visual stimulation from the pre-processed data.

[0070] The visual stimulus classifier 134 may classify the visual stimulus STI based on the extracted features, and may identify the control target 200 .

[0071] The control module 140 may perform control through a brain computer interface (BCI).

[0072] Figure 4 and Figure 5 1 is a flow chart showing a method for presenting visual stimulation by using augmented reality according to an embodiment. Figure 1 ) to execute Figure 4 and Figure 5 Method for presenting visual stimuli by using augmented reality. Figures 1 to 3 Give a description.

[0073] Figure 4 The steps (from step S410 to step S440 ) of the visual stimulation presentation device 100 synthesizing the visual stimulation STI through the augmented reality module 110 and the steps ( S450 to step S495 ) of the user controlling the control target 200 through the visual stimulation STI are shown.

[0074] exist Figure 4 In step S410, the augmented reality module 110 may register the control target image IMG from the database. In step S410, the augmented reality module 110 may detect the control target. Thereafter, in step S420, the augmented reality module 110 may detect the position of the control target and the outline of the control target.

[0075] Thereafter, at step S430, the augmented reality module 110 may place the virtual visual stimulus STI on the outline of the control target represented as the AR glasses AG. Thereafter, at step S440, the augmented reality module 110 may provide the placed visual stimulus STI for the user to gaze at.

[0076] Thereafter, at step S450, the visual stimulation detection module 130 may receive an EEG signal from the sensor unit 121. Thereafter, at step S460, the visual stimulation detection module 130 may analyze the EEG signal, and may perform feature extraction and classification of the visual stimulation signal. At step S470, the visual stimulation detection module 130 may detect a visual stimulation including the user's intention.

[0077] In step S480, the visual stimulus detection module 130 or the control module 140 may control the control target by using the detected visual stimulus. For example, the visual stimulus detection module 130 or the control module 140 may immediately perform an on / off operation of the control target by using the detected visual stimulus.

[0078] At step S490, the visual stimulus detection module 130 may identify a control target based on the detected visual stimulus. Thereafter, at step S495, when additional interaction is required, the augmented reality module 110 may generate a virtual additional visual stimulus around the identified control target so as not to overlap with the control target.

[0079] Thereafter, the visual stimulation detection module 130 may receive a brain wave signal of the user who is looking at the virtual additional visual stimulation, process the visual stimulation signal obtained through the analysis to detect the visual stimulation, and control the control target based thereon.

[0080] exist Figure 5In step S510, the device 100 for presenting visual stimulation by using augmented reality (hereinafter, the visual stimulation presenting device) can identify the control target that the user is gazing at in the three-dimensional space through the AR glasses, identify the position of the control target, and detect the outline of the control target.

[0081] The visual stimulation presenting device 100 may place an identification device capable of identifying the control target on the detected control target. For example, in step S520, the visual stimulation presenting device 100 may place a virtual outline on the outline of the control target and may place a visual stimulation based on SSVEP on the virtual outline.

[0082] The SSVEP-based visual stimulus may be provided as a virtual image, such as a checkerboard that flashes or reverses at a specific frequency. The recognition means may include various means for confirming a control target in order to place the visual stimulus.

[0083] In step S530 , the visual stimulation presentation apparatus 100 may analyze the electroencephalogram signal generated from the user gazing at the visual stimulation to detect the SSVEP induced signal, and classify the detected SSVEP induced signal to identify the control target.

[0084] In step S540, the visual stimulation presentation apparatus 100 may perform BCI control on the identified control target through the network.

[0085] Figure 6 and Figure 7 It shows that according to Figure 4 and Figure 5 An example diagram of a process of presenting and controlling visual stimulation by a visual stimulation presentation method using augmented reality according to an embodiment of the present invention.

[0086] Figure 6 is an example diagram illustrating a process of presenting and controlling visual stimuli with respect to a single control target according to an embodiment.

[0087] exist Figure 6 In the embodiment, when a user wears the AG glasses AG and looks at an actual smart lighting device, the visual stimulation presentation device 100 connected to the AG glasses can recognize the lighting device and can generate a virtual contour CT placed on the lighting device. The smart lighting device can be an IoT device that can communicate with the visual stimulation presentation device 100. Figure 6 It is shown that the virtual contour CT is a two-dimensional contour, but the virtual contour CT may be a three-dimensional contour surrounding the lighting device in an actual three-dimensional space.

[0088] Thereafter, the visual stimulation presenting apparatus 100 may generate the SSVEP visual stimulation STI on the virtual contour CT. The SSVEP visual stimulation STI may be synthesized on the lighting device in the same shape as the lighting device. The SSVEP visual stimulation STI may flicker at a specific frequency.

[0089] Thereafter, when the user gazes at the SSVEP visual stimulation STI, the visual stimulation presentation device 100 can measure the brain wave signal generated according to the frequency, detect the visual stimulation STI by performing feature extraction and classification on the received steady-state visual evoked potential (SSVEP) based on the measured brain wave signal, and identify the gazed control target 200 as a lighting device.

[0090] The visual stimulus presentation device 100 can control the lighting device according to the detected visual stimulus STI. For example, the visual stimulus presentation device 100 can perform BCI control, such as turning the lighting device on or off. When additional control is required, the visual stimulus presentation device 100 can generate additional visual stimuli that provide an interface around the lighting device.

[0091] Figure 7 is an example diagram illustrating a process of presenting and controlling visual stimuli with respect to multiple control targets according to an embodiment.

[0092] exist Figure 7 In the present invention, when a user wearing AR glasses looks at the actual three-dimensional living space, the visual stimulation presentation device 100 can identify the intelligent electronic device capable of BCI control as a control target. The visual stimulation presentation device 100 can detect the location where the identified control target is placed, and can generate a virtual outline or virtual image (segmentation) around the control target.

[0093] The visual stimulation presentation device 100 can represent the SSVEP visual stimulation (or SSVEP-induced visual stimulation) STI on a virtual image placed on a control target. The SSVEP visual stimulation STI placed on multiple control targets can each have a different frequency. That is, the multiple SSVEP visual stimulation STI can flash at different frequencies corresponding to the control targets, respectively.

[0094] The visual stimulus presentation device 100 can detect the specific visual stimulus that the user is looking at in multiple visual stimulus STIs. For example, the visual stimulus presentation device 100 can detect the specific visual stimulus that the user is looking at in multiple visual stimulus STIs by using an ABFCCA algorithm.

[0095] The visual stimulation presentation device 100 can detect the frequency of the reference signal based on the similarity between the visual stimulation signal detected from the user who is looking at the visual stimulation of a specific frequency in the visual stimulation STI with different frequencies and the reference signal based on the predefined frequency, and can identify the control target that the user is looking at based on the detected frequency. Here, the method for analyzing the similarity of the signal is not limited to a specific method, and various methods can be used.

[0096] For example, the visual stimulation presentation device 100 can analyze the correlation between a visual stimulation signal detected from a user who is gazing at a visual stimulation of a specific frequency in a visual stimulation STI with different frequencies and a reference signal based on a predefined frequency to detect the frequency of the reference signal with the highest correlation, and can identify the control target that the user is gazing at based on the detected frequency.

[0097] For example, when a user gazes at a visual stimulus placed in a smart lighting device, the visual stimulus presentation apparatus 100 may detect a reference signal having the highest correlation with the detected visual stimulus signal, detect a predefined frequency of the detected reference signal as the frequency of the visual stimulus gazed by the user, and identify the lighting device as a control target having the corresponding frequency as the control target gazed by the user.

[0098] Thereafter, the visual stimulus presentation device 100 may generate a control window providing an interface for controlling the selected lighting device as an additional visual stimulus STI2. The visual stimulus presentation device 100 may present the additional visual stimulus STI2 at a specific position that does not overlap with the lighting device based on the detected position of the lighting device. When the user gazes, in order to select one of the multiple options that appear in the control window through the AG glasses, the visual stimulus presentation device 100 may perform control for the selected option.

[0099] For example, when a user gazes at a specific option for turning on a lighting device shown in a control window, the visual stimulation presentation apparatus 100 may detect the user's intention through analysis of the user's brain waves and classification of visual stimulation, and turn on the lighting device.

[0100] Figure 8 is a diagram for explaining a computing device according to an embodiment.

[0101] refer to Figure 8 , an apparatus and method for presenting visual stimulation by using augmented reality according to an embodiment may be implemented by using a computing device 900 .

[0102] The computing device 900 may include at least one of a processor 910, a memory 930, a user interface input device 940, a user interface output device 950, and a storage device 960 that communicate via a bus 920. The computing device 900 may also include a network interface 970 that is electrically connected to the network 90. ​​The network interface 970 may send or receive signals with other entities via the network 90.

[0103] The processor 910 may be implemented in various types, such as a microcontroller unit (MCU), an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), etc., and may be any type of semiconductor device capable of executing instructions stored in the memory 930 or the storage device 960. The processor 910 may be configured to implement the above-mentioned Figures 1 to 7 The functions and methods described. For example, refer to Figures 1 to 7 The various elements described may be implemented by the processor 910 in whole or individually, so that the processor 910 may be configured to implement the above-described functions and methods.

[0104] The memory 930 and the storage device 960 may include various types of volatile or non-volatile storage media. For example, the memory may include a read-only memory (ROM) 931 and a random access memory (RAM) 932. In this embodiment, the memory 930 may be located inside or outside the processor 910, and the memory 930 may be connected to the processor 910 by various known means.

[0105] In some embodiments, at least some configurations or functions of the apparatus and method for presenting visual stimulation by using augmented reality according to the embodiments may be implemented as a program or software executable by the computing device 900, and the program or software may be stored in a computer-readable medium.

[0106] In some embodiments, at least some configurations or functions of the apparatus and method for presenting visual stimulation by using augmented reality according to the embodiments may be implemented by using hardware or circuits of the computing device 900, or may also be implemented as separate hardware or circuits that may be electrically connected to the computing device 900.

[0107] While the disclosure has been described in connection with what are presently considered to be practical embodiments, it is to be understood that the disclosure is not limited to the disclosed embodiments, but on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. An apparatus for presenting visual stimulation by using augmented reality, comprising: an augmented reality module, configured to identify a control target through augmented reality glasses and place the visual stimulus on the identified control target; a brain wave measurement module configured to measure brain waves of a user who is looking at the visual stimulus; a visual stimulation detection module configured to analyze the measured brain waves to detect visual stimulation signals, and process the detected visual stimulation signals to classify the visual stimulation and identify the control target; as well as The control module is configured to control the identified control target through the network.

2. The device according to claim 1, wherein: The augmented reality module is configured to register the image of the control target received from a database.

3. The device according to claim 1, wherein: The augmented reality module includes a control target detector configured to detect a position of the control target gazed at by the user through the augmented reality glasses and place a specific device for identifying the control target.

4. The device according to claim 3, wherein: The specific device includes a virtual outline placed on an outline of the control target.

5. The device according to claim 4, wherein: The augmented reality module further includes a visual stimulation placement unit configured to place a steady-state visual evoked potential-based visual stimulation flickering at a specific frequency on the outline.

6. The device according to claim 5, wherein: The visual stimulus placement unit is configured to place an additional visual stimulus providing an interface for controlling the control target at a specific position that does not overlap with the outline of the identified control target.

7. The device according to claim 1, wherein: The visual stimulus detection module comprises: a brain wave signal receiver configured to receive brain wave signals; a visual stimulation signal detector configured to analyze the received brain wave signal to detect the visual stimulation signal corresponding to a specific frequency; a feature extraction unit configured to extract features of the visual stimulus signal; and A visual stimulus classifier is configured to classify the visual stimulus based on the extracted features and identify the control target.

8. The device according to claim 1, wherein: The augmented reality module is configured to detect positions of the plurality of control targets through the augmented reality glasses, place virtual contours on the contours of the plurality of control targets respectively, and place visual stimuli based on visual stimulation signals flickering at different frequencies on the virtual contours respectively; and The virtual contour comprises a three-dimensional contour.

9. The device according to claim 8, wherein: The visual stimulation detection module is configured to analyze the similarity between the visual stimulation signal detected from the brain waves of the user who is gazing at the visual stimulation having a specific frequency and a reference signal according to a predetermined frequency to detect the frequency having the highest similarity, and to identify the control target gazed by the user based on the detected frequency.

10. The device according to claim 8, wherein: The visual stimulus detection module is configured to place an additional visual stimulus providing an interface for additional interaction at a specific position that does not overlap with the position of the control target after identifying the control target.

11. A method for presenting visual stimulation by using augmented reality, comprising the steps of: identifying, by means of augmented reality glasses, a control target that the user is looking at, and placing the visual stimulus on the identified control target; measuring brain waves of the user looking at the visual stimulus; analyzing the measured brain waves to detect a visual stimulation signal, and processing the detected visual stimulation signal to classify the visual stimulation and identify the control target; and The control target identified by the network control.

12. The method according to claim 11, wherein: Placing the visual stimulus includes detecting a position of the control target in a three-dimensional space and placing a virtual outline to the three-dimensional outline of the control target.

13. The method according to claim 12, wherein: Placing the visual stimulus further includes placing a steady-state visual evoked potential-based visual stimulus flickering at a specific frequency on the virtual outline.

14. The method according to claim 13, wherein: Identifying the control target comprises the following steps: extracting features of the visual stimulus signal; and The visual stimulus is classified based on the extracted features.

15. The method according to claim 14, wherein: Placing the visual stimulus further includes placing an additional visual stimulus providing an interface for controlling the control target at a position that does not overlap with the control target when the control target is identified.

16. The method according to claim 11, wherein: Placing the visual stimulus includes detecting positions of the plurality of control targets in a three-dimensional space to place a three-dimensional virtual outline on each outline of the plurality of control targets.

17. The method according to claim 16, wherein: Placing the visual stimulus further includes placing a visual stimulus based on a visual stimulus signal that flickers at different frequencies on the virtual outline.

18. The method according to claim 17, wherein: Identifying the control target includes analyzing the similarity between the visual stimulation signal detected from the user who is gazing at the visual stimulation of a specific frequency visual stimulation having different frequencies and a reference signal based on a predefined frequency to detect the frequency with the highest similarity, and identifying the control target gazed by the user based on the detected frequency.

19. The method according to claim 18, wherein: Placing the visual stimulus further includes placing an additional visual stimulus for controlling the identified control target on the three-dimensional space so as not to overlap with the identified control target.

20. The method according to claim 11, wherein: The visual stimulus includes a checkerboard image that is reversed at a specific frequency.

Images