Information processing device, method, and program

Abstract

According to one embodiment, an information processing device includes a first acquisition unit that acquires information that indicates the line of sight of a user, a second acquisition unit that acquires video information for an area of movement for the line of sight of the user, and a control unit that, when information that is to attract the attention of the user is included in the acquired video information but the line of sight indicated by the information acquired by the first acquisition unit is not directed at the information that is to attract the attention of the user, controls the operation of a presentation device that can present olfactory stimulation to the user such that olfactory stimulation is presented to the user from the presentation device.

Description

Information processing device, method and program

[0001] FIELD Embodiments of the present invention relate to an information processing device, method, and program.

[0002] For example, when driving an automobile (hereinafter sometimes referred to as a car), a user needs to pay attention to multiple visual information, and depending on the situation, their attention may become distracted. Therefore, a method using olfactory stimulation has been used as a method to alert the user to the visual information instead (see, for example, Non-Patent Document 1).

[0003] Dmitrijs Dmitrenko, Emanuela Maggioni, and Marianna Obrist. 2018. I Smell Trouble: Using Multiple Scents To Convey Driving-Relevant Information. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI '18). Association for Computing Machinery, New York, NY, USA, 234-238. https: / / doi.org / 10.1145 / 3242969.3243015

[0004] In existing olfactory attention-requesting techniques such as that disclosed in Non-Patent Document 1, olfactory stimuli are presented to a user regardless of where the user is looking, etc. However, when the olfactory stimuli are presented relatively frequently, when the olfactory stimuli are used for a relatively long period of time, or when multiple types of scents are used as olfactory stimuli, the user's response to the olfactory stimuli may weaken due to the user's adaptation to the olfactory stimuli or lingering scent, or the user may have difficulty distinguishing between the scents relevant to the attention-requesting scents.

[0005] This invention was made in light of the above-mentioned circumstances, and its purpose is to provide an information processing device, method, and program that can assist in appropriate attention-raising through olfactory stimulation.

[0006] An information processing device according to one aspect of the present invention includes a first acquisition unit that acquires information indicating a user's line of sight, a second acquisition unit that acquires video information within a range in which the user's line of sight moves, and a control unit that controls the operation of a presentation device capable of presenting olfactory stimuli to a user so that, when information that should draw the user's attention is included in the acquired video information and the line of sight indicated by the information acquired by the first acquisition unit is not directed toward the information that should draw the user's attention, the presentation device presents olfactory stimuli to the user.

[0007] An information processing method according to one aspect of the present invention is a method performed by an information processing device, and includes: acquiring information indicating a user's line of sight by a first acquisition unit of the information processing device; acquiring video information within a range in which the user's line of sight moves by a second acquisition unit of the information processing device; and controlling, by a control unit of the information processing device, the operation of a presentation device capable of presenting olfactory stimuli to a user so that, when information that should draw the user's attention is included in the acquired video information and the line of sight indicated by the information acquired by the first acquisition unit is not directed toward the information that should draw the user's attention, the operation of the presentation device is controlled so that olfactory stimuli are presented to the user from the presentation device.

[0008] According to the present invention, it is possible to assist in appropriate attention-raising through olfactory stimulation.

[0009] Fig. 1 is a diagram showing an application example of an attention calling system according to an embodiment of the present invention. Fig. 2 is a flowchart showing an example of processing operations by the attention calling system. Fig. 3 is a diagram showing an implementation example of each unit of the attention calling system according to an embodiment of the present invention. Fig. 4 is a block diagram showing an example of the hardware configuration of an information processing device according to an embodiment of the present invention.

[0010] An embodiment of the present invention will be described below. In this embodiment, a configuration will be described in which an olfactory stimulus is presented to a user to draw the user's attention when important visual information is present outside the user's field of view or in a location where the user's gaze is not directed.

[0011] In this embodiment, olfactory stimuli are presented to the user only when visual information is present outside the user's line of sight or in a location where the user's line of sight is not directed, thereby reducing the effects of user adaptation or lingering scents on the user.

[0012] Fig. 1 is a diagram showing an application example of an attention calling system according to one embodiment of the present invention. As shown in Fig. 1, the attention calling system according to one embodiment of the present invention includes a camera image acquisition unit 10 that acquires external information captured by a camera, in this case, image information within the range in which a user's gaze moves, a gaze input unit 20 that acquires information about the user's gaze, a control unit 30 that determines whether the user is capturing necessary information and controls olfactory stimuli for the user, and an olfactory stimulus presentation unit 40 that presents olfactory stimuli to the user.

[0013] 2 is a flowchart showing an example of the processing operation of the attention warning system. First, the camera image acquisition unit 10 acquires camera image, which is an external image captured by a camera or the like (S10). This camera image contains information that the user wants to pay attention to. In addition to this acquisition, the gaze input unit 20 inputs information about the user's gaze measured using an existing gaze measurement device (S20).

[0014] At this time, the control unit 30 uses a device such as a PC (personal computer) to identify whether or not there is information in the external video that should draw the user's attention.

[0015] Next, the control unit 30 determines whether the user's line of sight is capturing information that should draw the user's attention, that is, whether the user's line of sight is capturing information that the user wants to pay attention to (S30).

[0016] If the information to which the user should pay attention is present within the user's field of view or the user's gaze is directed toward the information to which the user should pay attention (Yes in S30), the measurement continues without presenting an olfactory stimulus to the user. On the other hand, if the information to which the user should pay attention is present outside the user's field of view or the user's gaze is not directed toward the information to which the user should pay attention (No in S30), the control unit 30 controls the operation of the olfactory stimulus presentation unit 40 to present an olfactory stimulus to the user (S40). The above-mentioned method of identifying whether or not there is information to which the user should pay attention may involve using a device such as a PC to set information to which the user should pay attention based on information such as images stored in a database in advance, and the control unit 30 then identifies the information based on this set information. Alternatively, the control unit 30 may identify information that the user often looks at in the camera image in real time and automatically set this information as information to which the user should pay attention.

[0017] Fig. 3 is a diagram showing an example of implementation of each unit of an attention alert system according to an embodiment of the present invention. As shown in Fig. 3, the camera image acquisition unit 10 of the attention alert system can be implemented in a device equipped with a camera 100, such as a video camera, PC, or tablet. Furthermore, the gaze input unit 20 can be implemented in an existing gaze measurement device 200. The control unit 30 can be implemented in an information processing device 300, such as a device such as a PC or tablet. The olfactory stimulus presentation unit 40 can be implemented in an existing olfactory stimulus presentation device 400.

[0018] In the above-described attention alert system, for example, when used while a user is driving a car, the information to which the user wishes to pay attention, such as the speed of the car and whether or not a person has run out into the road at an intersection, is set and stored, for example, in the internal memory of the information processing device 300. When the speed of the car exceeds a specified value that is the standard for not stimulating the sense of smell with an aroma, if the user's gaze is directed toward the speedometer inside the car, the control unit 30 determines that the user's gaze is directed toward information that should draw the user's attention, and controls the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400 so that no aroma is presented to the user.

[0019] On the other hand, as described above, when the speed of the vehicle exceeds the specified value and the user's line of sight is not directed toward the speedometer inside the vehicle, the control unit 30 determines that the user's line of sight is not directed toward information that the user should pay attention to, and controls the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400 to present, for example, an orange scent to the user. Furthermore, the type of scent presented to the user is not limited to one type, and the control unit 30 may change the type of scent presented to the user depending on the current situation, for example, when the speed of the vehicle significantly exceeds the specified value as described above and the user's line of sight is not directed toward the speedometer inside the vehicle, the control unit 30 may control the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400 to present an orange scent to the user.

[0020] At the same time, the control unit 30 may use the image acquired by the camera image acquisition unit 10 to detect whether or not an object shown in the image should draw the user's attention, such as a person or animal jumping out into the roadway. For example, when an object jumps out beyond the line of sight of a user driving a car, the control unit 30 detects this jumping out and, in response to this detection, controls the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400 to present, for example, a mint scent to the user inside the car. Here, the scent to be presented may be selected for each individual user according to the content or number of information objects that the user wants to focus on.

[0021] Another example is not limited to driving a car, but also includes use in computer games. For example, in a multiplayer FPS (First-Person Shooter) game, there is a lot of visual information that changes within a short period of time, such as the player's movements, the opponent's movements, and information that supplements the battle situation.

[0022] Here, an example will be described in which information that determines the type of olfactory stimulation corresponding to this information is set in advance in the internal memory of the information processing device 300. For example, when the battle situation becomes advantageous for the player, the control unit 30 controls the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400 to present, for example, the scent of orange for a short period of time.

[0023] On the other hand, when the battle situation becomes unfavorable for the player, the control unit 30 performs control so that, for example, a mint scent is presented for a short period of time from the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400. However, when the player's gaze is directed at visual information that supplements the battle situation, the control unit 30 performs control so that these scents are not presented from the olfactory stimulation presentation unit 40 of the olfactory stimulation presentation device 400.

[0024] 4 is a block diagram showing an example of the hardware configuration of an information processing device according to an embodiment of the present invention. In the example shown in FIG. 4, the information processing device 300 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to this hardware processor 111A via a bus 115.

[0025] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0026] An input device 500 and an output device 600, which are attached to the information processing device 300 and used by a user or the like, are connected to the input / output interface 113. The input / output interface 113 can take in operation data input by a user or the like through the input device 500, such as a keyboard, a touch panel, or a touchpad, and can output output data to an output device 600, which includes a display device using a liquid crystal or an organic electroluminescence (EL) display, for display. The input device 500 and the output device 600 may be devices built into the information processing device 300, or may be input devices and output devices of other information terminals that can communicate with the information processing device 300 via a network.

[0027] The program memory 111B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and can store programs necessary to execute various control processes, etc., according to one embodiment.

[0028] The data memory 112 is a tangible storage medium that is, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and can be used to store various data or information acquired and created during various processes.

[0029] An information processing device 300 according to an embodiment of the present invention can be configured as an information processing device having the control unit shown in FIG. 1 as a software-based processing function unit.

[0030] The information storage unit used as a work memory or the like in the information processing device 300 can be configured by using the data memory 112 shown in Fig. 4. However, these configured storage areas are not essential components within the information processing device 300, and may be areas provided in an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0031] The processing function units in each of the above units can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0032] The methods described in each embodiment can be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), or semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables or data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0033] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0034] REFERENCE SIGNS LIST 10: camera image acquisition unit 20: gaze input unit 30: control unit 40: olfactory stimulus presentation unit 100: camera 200: gaze measurement device 300: information processing device 400: olfactory stimulus presentation device

Claims

1. An information processing device comprising: a first acquisition unit that acquires information indicating a user's line of sight; a second acquisition unit that acquires video information within a range in which the user's line of sight moves; and a control unit that controls the operation of a presentation device capable of presenting olfactory stimuli to a user when the acquired video information includes information that should draw the user's attention and the line of sight indicated by the information acquired by the first acquisition unit is not directed toward the information that should draw the user's attention, so that the presentation device presents olfactory stimuli to the user.

2. The information processing device of claim 1, wherein the control unit controls the operation of the presentation device so that, when the line of sight indicated by the information acquired by the first acquisition unit is not directed toward information that should attract the user's attention, the presentation device presents one of a plurality of types of scent as a stimulus to the user's sense of smell, depending on the situation.

3. The information processing device of claim 2, wherein the information to attract the user's attention includes numerical information indicating the situation, and the control unit controls the operation of the presentation device so that, when the line of sight indicated by the information acquired by the first acquisition unit is not directed toward the information to attract the user's attention and the numerical information does not satisfy the criterion for not stimulating the olfactory sense, the presentation device presents one of a plurality of types of fragrance corresponding to the numerical information as a stimulus to the user's olfactory sense.

4. The information processing device described in claim 2, wherein the control unit uses the video information acquired by the second acquisition unit to detect the type of each object shown in the video information, and when the line of sight indicated by the information acquired by the first acquisition unit is not directed toward information that should attract the user's attention, controls the operation of the presentation device so that the presentation device presents one of multiple types of scent corresponding to the type of each object detected by the second acquisition unit as a stimulus to the user's sense of smell.

5. A method performed by an information processing device, comprising: acquiring information indicating a user's line of sight by a first acquisition unit of the information processing device; acquiring video information within a range in which the user's line of sight moves by a second acquisition unit of the information processing device; and controlling, by a control unit of the information processing device, the operation of a presentation device capable of presenting olfactory stimuli to the user so that, when information that should draw the user's attention is included in the acquired video information and the line of sight indicated by the information acquired by the first acquisition unit is not directed toward the information that should draw the user's attention, the operation of the presentation device is controlled so that olfactory stimuli are presented to the user from the presentation device.

6. An information processing program that causes a processor to function as each part of the information processing device according to any one of claims 1 to 4.

Images