Information processing device, information processing system, information processing program, and information processing method

Abstract

Virtual objects are displayed to improve the visibility of the content the user is viewing and to reduce distractions from the user's surroundings. [Solution] The information processing device includes an information acquisition means for acquiring user eyeball information, an estimation means for estimating the gaze range that the user is fixating on based on the eyeball information, a notification acquisition means for acquiring notifications, and a determination means for determining the position where the notifications acquired by the notification acquisition means will be displayed. The determination means determines the position where the notifications will be displayed to be a position that is not included in the gaze range estimated by the estimation means.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus.

Background Art

[0002] In recent years, wearable terminals such as AR (Augmented Reality) glasses and head-mounted displays (HMDs) have been becoming popular. With these XR (Extended Reality) devices, it is possible to control a display screen using a user's line of sight and display information. For example, Patent Document 1 discloses a method of identifying an object being focused on from the movements of a user's head and line of sight, and displaying information about the object as a virtual object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art disclosed in the above-mentioned patent document, the visibility of the content being viewed by the user or the surrounding environment that can be seen through the virtual object displayed may be reduced. Therefore, there is a concern that it may become a factor inhibiting the visibility of the content being viewed by the user and the user's attention to the surrounding environment.

[0005] The present invention has been made in view of these problems, and an object thereof is to provide an information processing apparatus capable of displaying a virtual object so as to reduce the inhibition of the visibility of the content being viewed by the user and the user's attention to the surrounding environment.

Means for Solving the Problems

[0006] To achieve the above objective, the information processing apparatus of the present invention comprises: information acquisition means for acquiring user eyeball information; estimation means for estimating the gaze range that the user is fixating on based on the eyeball information; notification acquisition means for acquiring notifications; and determination means for determining a position for displaying the notifications acquired by the notification acquisition means, wherein the determination means determines the position for displaying the notifications to be a position that is not included in the gaze range estimated by the estimation means. [Effects of the Invention]

[0007] According to the present invention, virtual objects can be displayed in a way that reduces the visibility of the content being viewed by the user and the distraction of the user's attention to the surrounding environment. [Brief explanation of the drawing]

[0008] [Figure 1] This is an external view of AR glasses 100 according to the first embodiment of the present invention. [Figure 2] This is a schematic block diagram showing an example of the hardware configuration of AR glasses 100 according to the first embodiment of the present invention. [Figure 3] This figure shows an example of determining the position of a notification display according to the first embodiment of the present invention. [Figure 4] This figure shows a flowchart for determining the notification display position according to the first embodiment of the present invention. [Figure 5] This figure shows a flowchart for determining a reference point according to the first embodiment of the present invention. [Figure 6] This figure shows an example of determining the direction of notification display according to a second embodiment of the present invention. [Figure 7] This figure shows a flowchart for determining the notification display direction according to a second embodiment of the present invention. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the drawings. The same or equivalent components, members, and processes shown in each drawing will be denoted by the same reference numerals, and redundant explanations will be omitted as appropriate. Furthermore, some components, members, and processes will be omitted in each drawing. The following embodiments are not limiting to the present invention, and not all combinations of features described in these embodiments are essential to the solutions of the present invention. The configuration of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (operating conditions, operating environment, etc.). In the following embodiments, the same components will be denoted by the same reference numerals.

[0010] (First embodiment) <Internal configuration of information processing device> Figures 1(a) and 1(b) show external views of AR glasses 100, an information processing device to which the present invention can be applied. The AR glasses 100 comprises an eyeball information acquisition unit 101 (left eyeball information acquisition unit 101L, right eyeball information acquisition unit 101R), a display unit 102 (left display unit 102L, right display unit 102R), and a power button 103. In the first embodiment, AR glasses are assumed as an example of an information processing device, but the present invention is not limited to these embodiments. For example, it may be an HMD, a smartphone or tablet terminal equipped with a camera, or other devices such as a personal computer (PC) or digital camera. It may also be an information processing system equipped with each of the components of the information processing device of this embodiment.

[0011] The eyeball information acquisition unit 101 consists of an event sensor 214 (described later), an event data calculation unit 215, a gaze detection lens 216, and an infrared light-emitting diode 217, and is capable of detecting the eyeball information of the user looking at the display unit 102. The eyeball information acquisition unit 101 is further divided into a left eyeball detection unit 101L and a right eyeball detection unit 101R. The display unit 102 displays images and various information, and displays a GUI (Graphical User Interface) for various information displays and setting screens. The display unit 102 is also divided into a left eyeball display unit 102L and a right eyeball display unit 102R. The power button 103 is a user interface for turning the power ON / OFF.

[0012] Figure 2 is a block diagram showing an example configuration of AR glasses 100. Note that each component of AR glasses 100 may constitute an information processing system composed of individual hardware components.

[0013] The system control unit 201 controls the entire AR glasses 100. It implements each of the processes of this embodiment, described later, by executing a program recorded in the non-volatile memory 205, which will be described later. Furthermore, the system control unit 201 controls the display by controlling the memory 209, the memory control unit 208, and the display unit 102, etc. In addition, the system control unit 201 issues notifications when it performs specific processing or when the network interface 212 receives a specific signal from the network or wireless interface 220, and displays these notifications on the display unit 102. Here, notifications include, for example, notifications from the AR glasses 100 itself, notifications from applications that can be launched on the AR glasses 100, and notifications of receiving messages from others. These notifications may be in text form or video form. Note that instead of the system control unit 201 controlling the entire device, multiple hardware components may share the processing to control the entire device.

[0014] The operation unit 202 is an operation means for inputting various operation instructions and transmits user operations to the system control unit 201. The operation unit 202 includes, for example, a power button 103. When the pressing of the power button 103 is transmitted, the system control unit 201 instructs the power control unit 203 to transition to the power-off state if it is in the power-on state, and instructs it to transition to the power-on state if it is in the power-off state.

[0015] The power control unit 203 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching the energized block, etc., and detects the presence or absence of battery attachment, the type of battery, and the remaining battery level. Also, based on the detection result and the instruction of the system control unit 201, it controls the DC-DC converter and supplies the necessary voltage to each part including the recording medium 211 for the necessary period.

[0016] The power supply unit 204 consists of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, a Li battery, an AC adapter, etc.

[0017] The non-volatile memory 205 is an electrically erasable and recordable memory, and for example, an EEPROM or the like is used. The non-volatile memory 205 stores constants, programs, etc. for the operation of the system control unit 201 described above. Here, the program refers to the program for executing various flowcharts described later in this embodiment.

[0018] The system memory 206 is a RAM, and expands constants, variables, programs read from the non-volatile memory 205, etc. for the operation of the system control unit 201. The system memory 206 consists of, for example, a volatile memory (DRAM) using semiconductor elements.

[0019] The system timer 207 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0020] Memory 209 stores image data for display on the display unit 102. Memory 209 has sufficient storage capacity to store video and audio for a predetermined period of time.

[0021] The storage interface 210 is a communication interface with the recording medium 211, such as a hard disk.

[0022] The recording medium 211 is a recording medium such as a memory card, and is composed of semiconductor memory, magnetic disks, etc.

[0023] As described above, the eyeball information acquisition unit 101 consists of an event sensor 214, an event data calculation unit 215, a gaze detection lens 216, and an infrared light-emitting diode 217, and is capable of detecting the user's eyeball information. The infrared light emitted from the infrared light-emitting diode 217 is reflected by the user's eyeball (eye) 218, and the reflected infrared light is imaged onto the imaging surface of the event sensor 214 through the gaze detection lens 216.

[0024] The event sensor 214 is an event-based vision sensor that detects changes in the brightness of light incident on each pixel and outputs information about the pixel where the brightness change occurred, asynchronously from other pixels. The data output from the event sensor 214 (hereinafter referred to as event data) includes, for example, the position coordinates of the pixel where the brightness change (event) occurred, the polarity (positive or negative) of the brightness change, and timing information corresponding to the time the event occurred. Compared to existing frame-based synchronous sensors, the event sensor 214 eliminates redundancy in the output information and features high-speed operation, high dynamic range, and low power consumption. On the other hand, the event data is asynchronously output for each pixel. In order to determine the relationship between event data, it is necessary to accumulate event data that occurs over a predetermined period of time and perform various calculations on the results.

[0025] The event data calculation unit 215 is a calculation unit for detecting user eyeball information based on event data continuously and asynchronously output from the event sensor 214. For example, it accumulates event data that occurs over a predetermined period of time and processes them as a set of data to determine whether or not user eyeball information is present. By changing the accumulation time for event data, it is possible to detect eyeball information for multiple users with different occurrence speeds. User eyeball information includes eye movements, such as gaze position information, saccade information including the direction and speed of saccades, and microsaccade information including the frequency and amplitude of microsaccades. It may also include pupil information including pupil size and its changes, and blink information such as blink speed and number of blinks. The detectable user eyeball information is merely an example and is not limited to these. The event data calculation unit 215 may be configured to map the event data for the accumulation time as image data for one frame based on the event occurrence coordinates and perform image processing. This makes it possible to obtain eyeball information for multiple users from event data using the method conventionally used in frame-based image data processing.

[0026] The display unit 102 displays information on a display device such as a transmissive organic EL display, according to the digital signals from the memory 209. Output data from the system control unit 201 is written to the memory 209 via the memory control unit 208 or directly to the memory 209.

[0027] The network interface 212 communicates with networks such as LANs (Local Area Networks) and the Internet based on instructions from the system control unit 201.

[0028] The wireless interface 220 performs wireless communication using wireless communication methods such as Wi-Fi (Wireless Fidelity) (registered trademark) and Bluetooth (registered trademark).

[0029] The gaze range calculation unit 213 is a calculation unit that estimates and calculates the user's gaze range based on the user's eyeball information detected by the event data calculation unit 215. For example, the gaze range or degree of gaze (overview) can be estimated from the frequency and amplitude of microsaccades. Here, gaze range is synonymous with attention range and focus range. The degree of gaze is an index that is higher the narrower the gaze range and lower the wider it is. Overview is defined as the opposite of degree of gaze. The gaze range calculation unit 213 can be configured as a neural network or the like that takes parameters related to microsaccades, blinking, pupil diameter, etc., as input and outputs gaze range and degree of gaze (overview as its opposite). However, the configuration of the gaze range calculation unit 213 is not limited to the above, and other determination means may be used. The user's eyeball information used in the gaze range calculation unit 213 is not limited to the above, and other configurations may be used. Furthermore, the gaze range is the area that includes the trajectory of the gaze over a predetermined period of time.

[0030] The system control unit 201 controls the display unit 102 and can calculate the field of view based on user operations when a specific screen is displayed. The calculated field of view information is stored as a field of view angle in the recording medium 211. At this time, methods such as dynamic quantitative field of view measurement and static quantitative field of view measurement are used as the method for calculating the field of view. In addition, the system control unit 201 can display setting items on the display unit 102 to prompt the user to make a selection and record the selected content in the recording medium 211. Setting items include the method for determining the reference point, the display direction of notifications, and the display color settings, which will be described later.

[0031] In this embodiment, the display unit 102 is transparent and allows the wearer to see real images, as in AR glasses 100. However, the display unit 102 may also be opaque and display images from a camera capturing the surrounding environment, as in a head-mounted display. In other words, in the case of optical see-through, where the wearer can directly see real images, a virtual object for displaying notifications is displayed on the transparent display. In the case of video see-through, where the wearer can see captured images or composite images, the virtual object for displaying notifications is a composite image created by superimposing the virtual object onto the captured image and displaying it on the opaque display.

[0032] <How to determine the display position of notifications> The following describes a method for determining the display position of a notification from the user's eyeball (eye) 218 ​​in the AR glasses 100, with reference to Figure 3.

[0033] Figure 3 illustrates the user's gaze position 301, field of view 302, and gaze range 303 in the display unit 102 of the AR glasses 100. It also assumes a scenario where a virtual object displaying a notification is shown at the notification display position 304. The gaze position 301 represents the user's gaze position and is calculated by the event data calculation unit 215 as described above. The field of view 302 represents the field of view of both the user's eyes and is the range of the field of view angle acquired by the system control unit 201 and stored in the recording medium 211, relative to a reference point determined from the gaze position 301. The gaze range 303 represents the user's gaze range and is the range of the area calculated by the gaze range calculation unit 213, centered on the reference point determined from the gaze position 301. Here, the notification display position 304 is determined from the gaze position 301, field of view 302, and gaze range 303. Information that can be recognized as being displayed, such as icons or shapes, is displayed at the notification display position 304, and the user can recognize that a notification has been issued by recognizing the display of this information. Furthermore, in the example shown in Figure 3, the notification is displayed outside the gaze area but within the viewing area.

[0034] Referring to Figure 4, a flowchart for determining the notification display position 304 is explained. Here, this flowchart is a process that is executed at regular intervals after the AR glasses 100 are powered on.

[0035] In step S401, the system control unit 201 determines whether or not a notification has occurred (notification acquisition step). If the system control unit 201 determines that a notification has occurred, it proceeds to step S402; otherwise, it terminates the process.

[0036] In step S402, the system control unit 201 calculates the field of view range 302 of the display unit 102, centered on a reference point determined from the line of sight position 301, and then proceeds to step S403.

[0037] In step S403, the system control unit 201 acquires the gaze range 303 calculated by the gaze range calculation unit 213 and proceeds to step S404.

[0038] In step S404, the system control unit 201 displays the notification as a virtual object in an area within the field of view range 302 but outside the gaze range 303, and then terminates the process.

[0039] In this example, the notification display position was set within the field of view 302, but fixed values ​​could also be set, such as within the range of a typical human field of view of approximately 150-160 degrees horizontally and 130 degrees vertically relative to the reference point. Similarly, although it was set outside the gaze range 303, fixed values ​​could also be set, such as outside the range of a typical effective field of view of approximately 60 degrees horizontally and 40 degrees vertically relative to the reference point.

[0040] <Method for determining the reference point at the center of the field of view 302 from the line of sight position 301> Next, referring to Figure 5, we will explain how to determine the reference point that will be the center of the field of view 302 from the line of sight position 301. Figure 5 is a flowchart for determining the reference point, and it is a process that is executed at regular intervals after the AR glasses 100 are powered on.

[0041] In step S501, the system control unit 201 acquires the user's gaze position 301 detected by the event data calculation unit 215, and then proceeds to step S502.

[0042] In step S502, the system control unit 201 refers to the user settings of the recording medium 211 and determines whether or not to determine a reference point based on the change in the line of sight position 301. If the system control unit 201 determines that a reference point should be determined based on the change in the line of sight position 301, it proceeds to step S503; otherwise, it proceeds to step S505.

[0043] In step S503, the system control unit 201 determines whether the user's line of sight 301 remains within a predetermined range for a predetermined time or longer. If the system control unit 201 determines that the user's line of sight 301 remains within the predetermined range for a predetermined time or longer, it proceeds to step S504. If the system control unit 201 does not determine that the user's line of sight 301 remains within the predetermined range for a predetermined time or longer, it terminates the process.

[0044] In step S504, the system control unit 201 saves the line of sight position 301 as a reference point to the recording medium 211 and terminates the process.

[0045] In step S505, the system control unit 201 determines whether or not a notification is displayed on the display unit 102. If the system control unit 201 determines that a notification is displayed on the display unit 102, it proceeds to step S506; if it determines that a notification is not displayed on the display unit 102, it proceeds to step S504.

[0046] In step S506, the system control unit 201 refers to the user settings of the recording medium 211 and determines whether or not to update the reference point while the notification is displayed. If the system control unit 201 determines that the reference point should be updated, it proceeds to step S504; otherwise, it terminates the process.

[0047] In the flowchart shown in Figure 5, there are three main methods for determining the reference point. The first method is to update the reference point when transitioning from step S505 to step S504, determining the reference point based on the position of the user's gaze at the time the notification is issued. The second method is to update the reference point when transitioning from step S503 to step S504, determining the reference point based on the position of the user's gaze that was within a predetermined range for a predetermined time or longer before the notification was issued. The third method is to update the reference point when transitioning from step S505 to step S504 and when transitioning from step S506 to step S504, determining the reference point based on the position of the user's gaze while the notification is issued. Each method for determining the reference point has the following characteristics, and by using them appropriately according to the amount of eye movement of the user and the need to check the notification, notifications can be displayed without hindering the user's concentration or visibility.

[0048] In the first case, if the reference point is set to the position of the user's gaze at the time the notification is issued, the reference point will be set in the direction of the gaze even if the user moves their eyes. As a result, the notification will be displayed outside the gaze range 303, and the notification can be displayed without hindering the user's concentration or visibility.

[0049] Secondly, if the reference point is set to the position of the user's gaze that was within a specified range for a specified period of time or longer before the notification was issued, the reference point will be set to the position the user was most focused on. This means that even if a notification is issued when the user is looking in a different direction while looking in a specific direction and not frequently moving their gaze, the reference point will be set in that specific direction, and the notification will be displayed outside the gaze range 303. Therefore, notifications can be displayed without disrupting the user's concentration or visibility.

[0050] Thirdly, if the reference point is set to the position of the user's gaze while the notification is issued, the notification will be displayed outside the gaze range 303 regardless of whether the user's gaze is moving or stationary. This allows the notification to be displayed without disrupting the user's concentration or visibility. However, since the notification is always displayed outside the gaze range 303, the user cannot focus on it when they want to check the notification.

[0051] As described above, the AR glasses 100 of this embodiment can determine the position in which notifications are displayed relative to the user's eyeball (eye) 218, and display notifications in a way that reduces the possibility of hindering the user's concentration and visibility.

[0052] (Second Embodiment) <Method for determining the direction in which to display notifications within a field of view of 302 based on the issuing factor> The following describes a method for determining the direction in which a notification is displayed within the field of view 302 detected from the user's eyeball (eye) 218 ​​in the AR glasses 100, based on the emission factor, with reference to Figures 6 and 7.

[0053] Figure 6 illustrates the user's gaze position 301, field of view 302, gaze range 303, and notification display direction group 600 in the display unit 102 of the AR glasses 100.

[0054] The notification display direction group 600 indicates candidate directions for displaying notifications within the field of view range 302 and outside the gaze range 303. Notifications are issued when the system control unit 201 performs specific processing or when the network interface 212 receives a specific signal or specific data, and the display direction is determined from the notification display direction group 600 according to the cause of the notification. The relationship between the cause of the notification and the display direction is set by the system control unit 201, and the user can pre-define this setting, which is stored in the recording medium 211. The cause of the notification may also be, for example, which user the signal was received from (user type). If the signal is from a registered user, the notification is set to be displayed at a specific location; otherwise, the notification will be displayed at a different location. Therefore, by checking where it is displayed in the peripheral field of view, it may be possible to identify the cause of the notification without viewing the detailed content of the notification. It may also be which application issued the notification (application type). Similarly, if the signal is from a registered application, the notification is set to be displayed at a specific location; otherwise, the notification will be displayed at a different location. Therefore, by checking where the notification appears in the user's peripheral vision, it may be possible to identify which application the notification is from without viewing its detailed content. For example, if a user uses multiple social networking services (SNS), they can recognize which SNS the notification is from without viewing its detailed content. In this way, being able to identify the type of notification within the user's peripheral vision can reduce the disruption to the visibility of the content the user is viewing and to the user's attention to their surroundings.

[0055] Referring to Figure 7, a flowchart illustrating the determination of the display direction according to the notification issuance factor is shown. This flowchart is the process performed in step S404 described above.

[0056] In step S701, the system control unit 201 determines the range in which to display the notification from the field of view range 302 and the gaze range 303, and then proceeds to step S702.

[0057] In step S702, the system control unit 201 reads the setting of the relationship between the light source and the display direction from the recording medium 211 and proceeds to step S703.

[0058] In step S703, the system control unit 201 determines the display direction of the notification based on the notification issuance cause and the settings read in step S702, and then terminates the process.

[0059] Note that in this flow, the order of processing in step S701 and step S702 may be reversed. Also, this flow may be executed in the OS (Operation System) or within an application.

[0060] As described above, the AR glasses 100 of this embodiment can determine the display direction of a notification from the notification source within the field of view 302 detected from the user's eyeball (eye) 218. This allows the user to recognize the source by checking only the displayed direction in their peripheral vision without directly checking the notification, and by comparing it with the relationship between the display direction set by the user and the source source. In this embodiment, only the display direction has been mentioned, but other display characteristics such as display color and display orientation can also be determined in the same way.

[0061] (Other embodiments) Furthermore, the present invention can also be realized by performing the following process: that is, supplying software (program) that realizes the functions of the above-described embodiment to a system or device via a network or various storage media, and having the computer (or control unit or MPU, etc.) of the system or device read and execute the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0062] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0063] Furthermore, each functional unit in each of the above embodiments (each modified example) may or may not be individual hardware. The functions of two or more functional units may be implemented by common hardware. Each of the multiple functions of a single functional unit may be implemented by individual hardware. Two or more functions of a single functional unit may be implemented by common hardware. In addition, each functional unit may or may not be implemented by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. The functions of at least some of the functional units of the device may be implemented by the processor reading and executing the control program from the memory.

[0064] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0065] Furthermore, in each of the examples described above, "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPUs) and specialized processors (e.g., GPUs, ASICs, FPGAs, and programmable logic devices, etc.).

[0066] This embodiment includes the following configurations, methods, and programs.

[0067] [Configuration 1] A means for acquiring information to obtain the user's eyeball information, Estimation means for estimating the gaze range that the user is fixating on based on the eyeball information, Notification acquisition method for obtaining notifications, The system includes a determination means for determining the position where the notification acquired by the notification acquisition means will be displayed, The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means. An information processing device characterized by the following:

[0068] [Configuration 2] The estimation means further estimates the user's field of view based on the eyeball information, The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means, but is included in the field of view. The information processing device according to configuration 1, characterized by the above.

[0069] [Configuration 3] The estimation means estimates a predetermined range centered on a reference point estimated based on the eyeball information as the gaze range. An information processing device according to configuration 1 or 2, characterized by the above.

[0070] [Structure 4] The estimation means estimates a predetermined range centered on a reference point estimated based on the eyeball information as the field of view range. The information processing apparatus according to configuration 2, characterized in that...

[0071] [Composition 5] The determination means determines the position for displaying the notification to be a position that is not included in the gaze range estimated by the estimation means, based on the eye information at the time the notification is acquired by the notification acquisition means. An information processing device according to any one of configurations 1 to 4, characterized by the above.

[0072] [Composition 6] The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means at the time the notification is acquired by the notification acquisition means. An information processing device according to any one of configurations 1 to 5, characterized by the above.

[0073] [Composition 7] The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means, within a predetermined time until the notification is acquired by the notification acquisition means. An information processing device according to any one of configurations 1 to 6.

[0074] [Structure 8] The determination means controls the color or position in which the notification is displayed, according to the type of notification. An information processing device according to any one of configurations 1 to 7, characterized by the above.

[0075] [Composition 9] The determination means controls the color or position in which the notification is displayed, depending on the type of application to which the notification is issued. An information processing device according to any one of configurations 1 to 8.

[0076] [Configuration 10] The eyeball information includes at least one of the following: the frequency of microsaccade occurrence, the amplitude of the microsaccade, the position of the line of sight, the direction of the saccade, the speed of the saccade, the size of the pupil, and the change in pupil diameter. An information processing apparatus according to any one of configurations 1 to 9, characterized by the above.

[0077] [Composition 11] The gaze range is the range that includes the trajectory of the user's gaze during the predetermined time. An information processing apparatus according to any one of configurations 1 to 10, characterized by the above.

[0078] [method] An information acquisition step to obtain the user's eyeball information, An estimation step of estimating the gaze range that the user is fixating on based on the aforementioned eyeball information, The notification acquisition step to obtain the notification, The system includes a determination step for determining the position in which to display the notification obtained in the notification acquisition step, In the determination step, the position in which the notification is displayed is determined to be a position that is not included in the gaze range estimated in the estimation step. An information processing method characterized by the following:

[0079] [program] A program for causing a computer to function as one of the means of an information processing device described in any one of items 1 to 11.

[0080] [system] An information acquisition device that acquires user eyeball information, An estimation device that estimates the gaze range that the user is fixating on based on the aforementioned eyeball information, A notification acquisition device that acquires notifications, The device includes a determination device that determines the position for displaying the notification acquired by the notification acquisition device, The determination device determines the position for displaying the notification to be a position that is not included in the gaze range estimated by the estimation device. An information processing system characterized by the following:

Claims

1. A means for acquiring information to obtain the user's eyeball information, Estimation means for estimating the gaze range that the user is fixating on based on the eyeball information, Notification acquisition method for obtaining notifications, The system includes a determination means for determining the position where the notification acquired by the notification acquisition means will be displayed, The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means. An information processing device characterized by the following:

2. The estimation means further estimates the user's field of view based on the eyeball information, The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means, but is included in the field of view. The information processing apparatus according to feature 1.

3. The estimation means estimates a predetermined range centered on a reference point estimated based on the eyeball information as the gaze range. The information processing apparatus according to feature 1.

4. The estimation means estimates a predetermined range centered on a reference point estimated based on the eyeball information as the field of view range. The information processing apparatus according to feature 2.

5. The determination means determines the position for displaying the notification to be a position that is not included in the gaze range estimated by the estimation means, based on the eye information at the time the notification is acquired by the notification acquisition means. The information processing apparatus according to feature 1.

6. The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means at the time the notification is acquired by the notification acquisition means. The information processing apparatus according to feature 1.

7. The determination means determines the position in which the notification is displayed to be a position that is not included in the gaze range estimated by the estimation means, within a predetermined time until the notification is acquired by the notification acquisition means. The information processing apparatus according to feature 1.

8. The determination means controls the color or position in which the notification is displayed, according to the type of notification. The information processing apparatus according to feature 1.

9. The determination means controls the color or position in which the notification is displayed, depending on the type of application to which the notification is issued. The information processing apparatus according to feature 1.

10. The eyeball information includes at least one of the following: the frequency of microsaccade occurrence, the amplitude of the microsaccade, the position of the line of sight, the direction of the saccade, the speed of the saccade, the size of the pupil, and the change in pupil diameter. The information processing apparatus according to feature 1.

11. The gaze range is the range that includes the trajectory of the user's gaze during the predetermined time. The information processing apparatus according to feature 1.

12. An information acquisition step to obtain the user's eyeball information, An estimation step of estimating the gaze range that the user is fixating on based on the aforementioned eyeball information, The notification acquisition step to obtain the notification, The system includes a determination step for determining the position in which to display the notification obtained in the notification acquisition step, In the determination step, the position in which the notification is displayed is determined to be a position that is not included in the gaze range estimated in the estimation step. An information processing method characterized by the following:

13. A program for causing a computer to function as each of the means of the information processing apparatus described in claim 1.

14. An information acquisition device that acquires user eyeball information, An estimation device that estimates the gaze range that the user is fixating on based on the aforementioned eyeball information, A notification acquisition device that acquires notifications, The device includes a determination device that determines the position for displaying the notification acquired by the notification acquisition device, The determination device determines the position for displaying the notification to be a position that is not included in the gaze range estimated by the estimation device. An information processing system characterized by the following:

Images