Information presentation device and information presentation method

Abstract

An information presentation device according to an embodiment of the present invention includes an acquisition unit, an estimation unit, a collection unit, and a presentation unit. The acquisition unit acquires a brain wave signal of a subject user. The estimation unit that estimates a direction in which the subject user has interest on the basis of the brain wave signal acquired by the acquisition unit, and by using a trained model trained using, as training data, information pertaining to a direction in which a user has interest and the brain wave signal when the user is conscious of the direction in which the user has interest. The collection unit collects spot information, which is information about a spot present in the direction estimated by the estimation unit. The presentation unit presents the spot information to the user.

Description

Information presentation device and information presentation method

[0001] The embodiments relate to an information presentation device and an information presentation method.

[0002] With the widespread use of digital devices such as smartphones, users can easily search for information about their desired spot or nearby spots by launching a map app installed on the digital device and searching for spots such as stores and facilities on the map app (see, for example, Patent Document 1).

[0003] Patent No. 6775663

[0004] However, the above-mentioned search method requires the user to input information about the desired spot by text or voice. Therefore, for example, if the user is wearing gloves, the user must remove the gloves to input text into the digital terminal. Furthermore, if there is noise in the user's surroundings, voice input may be difficult. The present invention has been made in light of the above circumstances, and its purpose is to provide an information presentation device and an information presentation method that allow the user to easily collect information.

[0005] An information presentation device according to an embodiment includes an acquisition unit, an estimation unit, a collection unit, and a presentation unit. The acquisition unit acquires an electroencephalogram (EEG) signal of a target user. The estimation unit estimates the direction of interest of the target user based on the EEG signal acquired by the acquisition unit using a trained model that has been trained using, as training data, information about the direction of interest of the user and the EEG signal obtained when the user is conscious of the direction of interest. The collection unit collects spot information, which is information about spots located in the direction estimated by the estimation unit. The presentation unit presents the spot information to the user.

[0006] According to the embodiment, it is possible to provide an information presentation device and an information presentation method that allow a user to easily collect information.

[0007] FIG. 1 is a block diagram showing an information presentation device according to a first embodiment. FIG. 2 is a flowchart showing an example of operation of the information presentation device according to the first embodiment. FIG. 3 is a conceptual diagram showing a specific example of information presentation by the information presentation device according to the first embodiment. FIG. 4 is a diagram showing another example of information presentation by the information presentation device according to the first embodiment. FIG. 5 is a block diagram showing a learning device according to a second embodiment. FIG. 6 is a conceptual diagram of learning of a machine learning model. FIG. 7 is a block diagram showing the hardware configuration of the information presentation device and the learning device according to this embodiment.

[0008] Each embodiment will be described below with reference to the drawings. Each embodiment illustrates an apparatus or method for embodying the technical idea of ​​the invention. The drawings are schematic or conceptual. Hereinafter, the same reference numerals are used to designate components having substantially the same functions and configurations. The numbers following the letters that make up the reference numerals are used to distinguish between elements that are referred to by the reference numerals containing the same letters and that have similar configurations. When there is no need to distinguish between elements indicated by reference numerals containing the same letters or numbers, these elements will be referred to by reference numerals containing only letters or numbers.

[0009] First Embodiment A block diagram of an information presentation device according to a first embodiment will be described with reference to Fig. 1. The information presentation device 10 according to the first embodiment includes an acquisition unit 101, a determination unit 102, an electroencephalogram analysis unit 103, an estimation unit 104, a collection unit 105, a presentation unit 106, and a storage unit 107.

[0010] The acquisition unit 101 acquires an electroencephalography (EEG) signal of the target user to be measured. The electroencephalography signal may be acquired, for example, by an electroencephalography (EEG) sensor. The electroencephalography may be a device of any type, such as a hat type or a wearable type. The acquisition unit 101 also acquires position information and acceleration information of the user. The position information and acceleration information are values ​​acquired from a GPS sensor and an acceleration sensor built into a device held by the user, such as a smartphone, tablet, or smartwatch. The determination unit 102 determines whether the user is moving.

[0011] The electroencephalogram analysis unit 103 performs frequency analysis on the electroencephalogram signal to generate a frequency intensity distribution of the electroencephalogram signal. The estimation unit 104 uses a trained model to estimate the direction of interest of the target user based on the electroencephalogram signal of the target user. The trained model is generated by training a model using as training data directional information regarding the direction of interest of the user and the electroencephalogram signal when the user is conscious of the direction of interest.

[0012] The collection unit 105 searches for and collects spot information, which is information about spots located in a direction of interest to the target user, estimated by the estimation unit 104. Spots are assumed to be places such as restaurants, buildings such as shrines and temples, and facilities such as parks and theme parks. The presentation unit 106 presents the spot information to the target user. The storage unit 107 stores the trained model 1071, the electroencephalogram signal, etc.

[0013] Next, an example of the operation of the information presentation device 10 will be described with reference to the flowchart in Fig. 2. In step SA1, the acquisition unit 101 acquires an electroencephalogram signal, position information, and acceleration information of the user. It is assumed that the electroencephalogram signal, position information, and acceleration information are acquired in real time at predetermined sampling intervals.

[0014] In step SA2, the determination unit 102 determines whether the user has stopped. For example, if the acceleration value is less than a predetermined value, it may be determined that the user has stopped. Alternatively, it may be determined that the user has stopped if the fluctuation in the GPS information value is equal to or less than a threshold. If the user has stopped, the process proceeds to step SA3. If the user has not stopped, the process returns to step SA1 and the same process is repeated.

[0015] In step SA3, the EEG analysis unit 103 analyzes EEG signals before and after the user stops standing when the user stops. Specifically, the EEG analysis unit 103 performs an FFT on the acquired EEG signals to calculate a frequency intensity distribution. The frequency intensity distribution is divided into multiple sections and feature amounts are extracted. In step SA4, the estimation unit 104 estimates the direction of the user's interest using the feature amounts of the frequency intensity distribution and the trained model. Specifically, by inputting the feature amounts of the frequency intensity distribution obtained in step SA3 into the trained model, the trained model outputs an estimated direction of the user's interest based on the feature amounts.

[0016] In step SA5, the collection unit 105 collects spot information located in the direction of the user's interest. Specifically, the collection unit 105 searches for and displays information about surrounding spots located within a predetermined distance range in the direction of the user's interest, based on, for example, the user's current location, from information aggregated in a map application. Note that the information about surrounding spots is not limited to being collected through collaboration with a map application, and may also be collected by performing a text search on the Internet for the user's current location and spot information. In step SA6, the presentation unit 106 presents the spot information on the terminal used by the user.

[0017] In the above-described step SA4, it is assumed that the estimation unit 104 uses a trained model, but this is not limiting and the direction of the user's interest may be estimated by pattern matching. For example, if the degree of coincidence between a reference frequency distribution, which is an EEG signal when the user is interested in the right side of the traveling direction, and the frequency intensity distribution of the EEG signal analyzed in step SA3 is equal to or greater than a threshold, it may be estimated that the user is interested in the right side of the traveling direction. Furthermore, instead of using the reference frequency distribution of an individual user, the reference frequency distribution may be generated by averaging the frequency distributions of EEG signals obtained from multiple users.

[0018] Next, a specific example of information presentation by the information presentation device according to the first embodiment will be described with reference to the conceptual diagram of Fig. 3. In Fig. 3, it is assumed that a user is using the information presentation device on a smartphone and walking toward a destination (here, a restaurant). Although not specifically shown, it is also assumed that the user is wearing an electroencephalograph (EEG) for acquiring electroencephalogram signals.

[0019] In FIG. 3( a), a user is walking through town with a smartphone in hand. In FIG. 3( b), the user stops and begins to think that the restaurant they are looking for should be on the right side of their direction of travel. Since the user has stopped, the determination unit 102 determines that the user has stopped. The EEG analysis unit 103 then performs frequency analysis on the user's EEG signals acquired by the acquisition unit 101 to obtain a frequency intensity distribution. The estimation unit 104 inputs the obtained frequency intensity distribution into a trained model to estimate the direction of the user's interest. In this example, the user is looking right and expressing interest in the right side of their direction of travel, saying, "There should be... on the right side." Based on this frequency intensity distribution, the trained model can estimate that the user is interested in the right side. In FIG. 3( c), the collection unit 105 collects spot information located within a predetermined distance to the right of the point where the user stopped. The presentation unit 106 then displays the spot information on the smartphone display screen, asking, "Is the spot you're looking for around here?" It should be noted that the spot information may not only be displayed on the display screen, but may also be output by synthesized voice.

[0020] Next, another example of information presentation by the information presentation device 10 is shown in FIG. 4 . FIG. 4 is an example of a display screen 40, showing a state in which a user has launched a map application. As shown in FIG. 2 , when the information presentation device 10 estimates a direction of interest to a user, the device 10 prompts the user to decide whether to present spot information in the vicinity of the estimated direction. Here, a pop-up window 41 is displayed on the display screen 40, stating, "Are you looking for nearby spots? We will display spot information." In this manner, the spot information may not be automatically displayed, and the user may be left to decide whether to present the spot information. The notification is not limited to being displayed in the pop-up window 41. Any method that the user can perceive may be used to confirm whether to present the spot information, such as notifying the user by synthesized voice.

[0021] According to the first embodiment described above, when a user stops, the direction of the user's interest is estimated based on the user's electroencephalogram signal. Spot information present in the vicinity of the estimated direction is collected and displayed on the terminal held by the user. This allows the user to easily and appropriately collect and present information about surrounding spots simply by imagining or facing the direction of interest, without having to input the spot information the user wants to know by text or voice. In other words, the user can easily collect information.

[0022] Second Embodiment In the second embodiment, a learning device that generates a trained model to be used in the information presentation device 10 will be described. The learning device according to the second embodiment will be described with reference to the block diagram of Fig. 5. The learning device 20 according to the second embodiment includes an acquisition unit 201, an electroencephalogram analysis unit 202, a learning instruction unit 203, a learning unit 204, and a storage unit 205.

[0023] The acquisition unit 201 acquires an electroencephalogram signal of a user. The electroencephalogram analysis unit 202 analyzes the electroencephalogram signal. Specifically, the electroencephalogram analysis unit 202 performs an FFT on the electroencephalogram signal, similar to the electroencephalogram analysis unit 103, to calculate a frequency intensity distribution. The frequency intensity distribution is divided into multiple sections to extract feature amounts.

[0024] The learning instruction unit 203 instructs the user to take an action that encourages the user to be aware of the direction of the user's interest and acquires the user's brain waves at that time. Specifically, the instruction may include, for example, the user actually facing the specified direction, the user recalling the specified direction for 10 seconds 5 times, or the user exercising the arm on the side of the specified direction (for example, raising, rotating, or stretching the hand in response to voices such as "right" or "left"). In other words, the instruction may be any instruction that allows the user to be aware of the direction.

[0025] The learning unit 204 acquires, as training data, the instructions from the learning instruction unit 203 and the EEG signals generated when the user acts in response to those instructions, and trains a machine learning model to generate a trained model. The storage unit 205 stores the training data, the trained model 1071, and the like.

[0026] Next, learning of the machine learning model in the learning unit 204 will be described with reference to the conceptual diagram of Fig. 6. Fig. 6 shows how training data acquired from multiple users is input to a machine learning model 60. The training data may be input into the machine learning model 60 using directional information (e.g., left or right) 61 instructed by the learning instruction unit 203 as correct answer data (instruction label), and an electroencephalogram signal 62 of the user when the user is conscious of the instructed direction or when facing that direction as input data.

[0027] The machine learning model 60 is, for example, a random forest, and is trained to estimate which direction the input EEG signal 62 represents. The machine learning model 60 can be trained using a general supervised learning method, and therefore a detailed description thereof will be omitted here. Furthermore, the machine learning model 60 is not limited to a random forest; any machine learning model can be used, such as a decision tree, a deep convolutional neural network, a logistic regression, a support vector machine (SVM), or a naive Bayesian model, as long as it can learn the relationship between the EEG signal and the direction. The machine learning model 60 is trained using the training data (the direction information 61 and the EEG signal 62), thereby generating a trained model 1071.

[0028] According to the second embodiment described above, a machine learning model is trained using directional information about the direction in which the user is interested and the user's electroencephalogram signal at that time as training data, and a trained model is generated. This makes it possible to generate a trained model used in the information presentation device according to the first embodiment.

[0029] The learning device 20 may be incorporated into the information presentation device 10. In this case, the acquisition unit 201, the EEG analysis unit 202, and the storage unit 205 may be the same as the acquisition unit 101, the EEG analysis unit 103, and the storage unit 107 in the information presentation device 10.

[0030] Next, an example of the hardware configuration of the information presentation device 10 and the learning device 20 according to this embodiment will be described with reference to the block diagram shown in Fig. 7. As shown in Fig. 7, the information presentation device 10 and the learning device 20 include, for example, a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a display 14, a communication interface 15, an input / output interface 16, and storage 17. Note that the learning device 20 does not necessarily include the display 14.

[0031] The CPU 11 is an integrated circuit capable of executing various programs and controls the overall operation of the information presentation device 10. The ROM 12 is, for example, a non-volatile semiconductor memory and stores programs and control data for controlling the information presentation device 10. The RAM 13 is, for example, a volatile semiconductor memory and is used as a work area for the CPU 11. The display 14 is, for example, a liquid crystal display (LCD) or an electro-luminescence (EL) display. The display 14 displays a graphical user interface (GUI) corresponding to application software. The communication interface 15 is an input / output component, such as a USB, HDMI (registered trademark), or network interface, for exchanging information with the outside. The input / output interface 16 is a component for receiving input from a user, such as a keyboard, mouse, or touch display, and a component for outputting information to a user, such as a speaker. The storage 17 is a non-volatile storage device. The storage 17 stores system software for the information presentation device 10 and the learning device 20. The CPU 11 may also be called a "processor."

[0032] In the above embodiments, the CPU 11 of the information presentation device 10 and the learning device 20 may be other circuits (or processors). For example, the information presentation device 10 and the learning device 20 may include an MPU (micro processing unit) or the like instead of a CPU. Each of the processes described in each embodiment may be realized by dedicated hardware. The processes of the information presentation device 10 and the learning device 20 may be a mixture of processes executed by software and processes executed by hardware, or may be only one of them.

[0033] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0034] DESCRIPTION OF SYMBOLS 10... Information presentation device 11... CPU 12... ROM 13... RAM 14... Display 15... Communication interface 16... Input / output interface 17... Storage 40... Display screen 41... Pop-up window 60... Machine learning model 61... Direction information 62... EEG signal 101, 201... Acquisition unit 102... Determination unit 103, 202... EEG analysis unit 104... Estimation unit 105... Collection unit 106... Presentation unit 107, 205... Storage unit 203... Learning instruction unit 204... Learning unit 1071... Trained model

Claims

1. An information presentation device comprising: an acquisition unit that acquires an electroencephalogram signal of a target user; an estimation unit that estimates a direction in which the target user is interested based on the electroencephalogram signal acquired by the acquisition unit using a trained model that has been trained using, as training data, information regarding the direction in which the user is interested and the electroencephalogram signal when the user is conscious of the direction in which the user is interested; a collection unit that collects spot information that is information about spots located in the direction estimated by the estimation unit; and a presentation unit that presents the spot information to the user.

2. The information presentation device according to claim 1, wherein the collection unit collects the spot information from information aggregated on a map app within a predetermined distance range from the target user's location in the direction of the interest.

3. The information presentation device according to claim 1, wherein the estimation unit estimates the direction of interest of the target user when the target user stops.

4. An information presentation method in which an acquisition means acquires an electroencephalogram signal of a target user, an estimation means uses a trained model that has been trained using information regarding the direction of interest of the user and the electroencephalogram signal when the user is conscious of the direction of interest as training data to estimate the direction of interest of the target user based on the electroencephalogram signal acquired by the acquisition means, a collection means collects spot information that is information about spots located in the direction estimated by the estimation means, and a presentation means presents the spot information to the user.

Images