Display Control Device

Abstract

A display control device comprising: a first acquisition unit that acquires at least one of a notification type or a notification timing; a calculation unit that calculates a display position for a virtual object corresponding to the notification in a three-dimensional virtual space displayed in a display device on the basis of the at least one of the notification type and the notification timing; and a display control unit that displays the virtual object in the virtual space on the basis of the display position calculated by the calculation unit.

Description

[Technical Field]

[0001] The present invention relates to a display control device, and more particularly to a display control device for displaying a virtual object corresponding to a notification. [Background technology]

[0002] In MR (Mixed Reality) technology, the real world perceived by the user is augmented by computer-based means. This technology makes it possible, for example, to precisely overlay a virtual world onto the real world perceived through MR glasses worn by the user.

[0003] When using an MR device, technology may be used to visualize some kind of notification in a three-dimensional virtual space.

[0004] Regarding notifications in a three-dimensional virtual space, for example, Patent Document 1 discloses a technology in which a "conversation event object" corresponding to a call between a first person and a second person via a terminal device and an "utterance phrase object" in which the main parts of the content of the call are recognized by voice and visualized are displayed. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-036871 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when some kind of notification is visualized in a three-dimensional virtual space perceived by a user wearing MR glasses, as the amount of notifications increases, the virtual space in front of the user's eyes becomes filled with visualized notifications. Furthermore, as the number of visualized notifications increases in response to an increase in the amount of notifications, it becomes difficult for the user to interact with the notifications. As a result, the user's notification processing efficiency decreases in inverse proportion to the increase in the amount of notifications. The technology disclosed in Patent Document 1 does not take into account this decrease in processing efficiency.

[0007] Therefore, the present invention aims to provide a display control device that visualizes notifications in a three-dimensional virtual space and controls display according to the amount of notifications, and that can suppress a decrease in notification processing efficiency by minimizing the degree to which visibility is obstructed according to the amount of notifications, even if the amount of notifications increases. [Means for solving the problem]

[0008] A preferred aspect of the present invention provides a display control device that includes: a first acquisition unit that acquires at least one of a type of notification and a timing of the notification; a calculation unit that calculates a display position of a virtual object corresponding to the notification in a three-dimensional virtual space displayed on a display device based on at least one of the type of notification and the timing of the notification; and a display control unit that displays the virtual object in the virtual space based on the display position calculated by the calculation unit. [Effects of the Invention]

[0009] According to the present invention, when notifications are visualized in a three-dimensional virtual space and display control is performed according to the amount of notifications, even if the amount of notifications increases, it is possible to suppress a decrease in notification processing efficiency by minimizing the degree to which the field of view is obstructed according to the amount of notifications. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing the overall configuration of an information processing system 1 according to a first embodiment. [Figure 2] 1 is a perspective view showing the appearance of MR glasses 20 according to a first embodiment. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of MR glasses 20 according to the first embodiment. [Figure 4] FIG. 1 is a block diagram showing an example of the configuration of a terminal device 10 according to a first embodiment. [Figure 5A] FIG. 4 is an explanatory diagram of the operation of the display control unit 113. [Figure 5B] FIG. 4 is an explanatory diagram of the operation of the display control unit 113. [Figure 5C] FIG. 4 is an explanatory diagram of the operation of the display control unit 113. [Figure 6] FIG. 2 is a block diagram showing an example of the configuration of a server 30 according to the first embodiment. [Figure 7] 4 is a flowchart showing the operation of the terminal device 10 according to the first embodiment. [Figure 8] FIG. 10 is a block diagram showing an example of the configuration of a terminal device 10A according to a second embodiment. [Figure 9] FIG. 10 is a block diagram showing an example of the configuration of a terminal device 10B according to a third embodiment. [Figure 10] FIG. 10 is an explanatory diagram of a virtual object VO7 indicating a storage body. DETAILED DESCRIPTION OF THE INVENTION

[0011] 1: First embodiment Hereinafter, the configuration of an information processing system 1 including a terminal device 10 as a display control device according to a first embodiment of the present invention will be described with reference to FIGS.

[0012] 1-1: Configuration of the first embodiment 1-1-1: Overall structure 1 is a diagram showing the overall configuration of an information processing system 1 according to a first embodiment of the present invention. The information processing system 1 is a system that provides a virtual space to a user U wearing MR glasses 20 (described later) using MR technology.

[0013] The information processing system 1 includes terminal devices 10-1 to 10-2, MR glasses 20, and a server 30. In the following description, when there is no need to distinguish between the terminal devices 10-1 to 10-2, the terminal device 10 will be used as a general term for the terminal devices 10-1 to 10-2. The terminal device 10 is an example of a display control device. In the information processing system 1, the terminal device 10 and the server 30 are connected to each other so as to be able to communicate with each other via a communication network NET. Furthermore, the terminal device 10-1 and the MR glasses 20 are connected to each other so as to be able to communicate with each other. Furthermore, although two terminal devices 10 and one MR glass 20 are shown in FIG. 1, these numbers are merely an example, and the information processing system 1 may include any number of terminal devices 10 and MR glasses 20.

[0014] 1, it is assumed that a user U uses a combination of a terminal device 10-1 and MR glasses 20. In the information processing system 1 shown in FIG. 1, a virtual object corresponding to a notification is displayed on the MR glasses 20. The notification may be a notification that the terminal device 10-1 has acquired from the terminal device 10-2. The notification may also be a notification that has been acquired from another terminal device not shown in FIG. 1. Furthermore, the notification may also be a notification that the terminal device 10-1 itself has generated.

[0015] The server 30 provides various data and cloud services to the terminal device 10 via the communication network NET.

[0016] The terminal device 10-1 displays virtual objects arranged in a virtual space on the MR glasses 20 worn by the user U on his / her head. The virtual space is, for example, a celestial sphere. Examples of the virtual objects include virtual objects representing data such as still images, videos, 3DCG models, HTML files, and text files, as well as virtual objects representing applications. Examples of text files include memos, source code, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files. The terminal device 10-1 is preferably, for example, a mobile terminal device such as a smartphone or a tablet.

[0017] The terminal device 10-2 outputs a notification to the terminal device 10-1. The terminal device 10-2 may also display a virtual object placed in a virtual space on a display 14 (described later) or XR glasses (not shown) connected to the terminal device 10-2. In this case, the configuration of the terminal device 10-2 basically includes the same configuration as the terminal device 10-1. Similarly to the terminal device 10-1, the terminal device 10-2 is preferably a mobile terminal device such as a smartphone or a tablet, for example.

[0018] The MR glasses 20 are a see-through wearable display worn on the user's head. The MR glasses 20 display virtual objects on display panels provided in each of the lenses for both eyes under the control of the terminal device 10. The MR glasses 20 are an example of a display device.

[0019] 1-1-2: Composition of MR glasses Fig. 2 is a perspective view showing the appearance of the MR glasses 20. As shown in Fig. 2, the appearance of the MR glasses 20 is similar to that of ordinary eyeglasses, and includes temples 91 and 92, a bridge 93, frames 94 and 95, and lenses 41L and 41R.

[0020] The bridge 93 is provided with an imaging device 26. The imaging device 26 captures an image of the outside world. The imaging device 26 also outputs imaging information indicating the captured image.

[0021] Each of the lenses 41L and 41R is provided with a half mirror. A liquid crystal panel or organic EL panel for the left eye is provided in the frame 94. The liquid crystal panel or organic EL panel will hereinafter be collectively referred to as a display panel. An optical member that guides light emitted from the display panel for the left eye to the lens 41L is provided in the frame 94. The half mirror provided in the lens 41L transmits external light to guide it to the left eye and reflects the light guided by the optical member to make it incident on the left eye. A display panel for the right eye and an optical member that guides light emitted from the display panel for the right eye to the lens 41R are provided in the frame 95. The half mirror provided in the lens 41R transmits external light to guide it to the right eye and reflects the light guided by the optical member to make it incident on the right eye.

[0022] The display 28, which will be described later, includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as a lens 41R, a display panel for the right eye, and an optical member for the right eye.

[0023] In the above configuration, the user U can observe the image displayed on the display panel in a see-through state in which it is superimposed on the outside world. Furthermore, in the MR glasses 20, of the binocular images with parallax, the image for the left eye is displayed on the display panel for the left eye, and the image for the right eye is displayed on the display panel for the right eye. The MR glasses 20 enable the user U to perceive the displayed image as if it had depth and a three-dimensional effect.

[0024] 3 is a block diagram showing an example of the configuration of the MR glasses 20. The MR glasses 20 include a processing device 21, a storage device 22, a gaze detection device 23, a GPS device 24, a motion detection device 25, an imaging device 26, a communication device 27, and a display 28. The elements of the MR glasses 20 are connected to each other by one or more buses for communicating information. Note that the term "device" in this specification may be replaced with other terms such as circuit, device, or unit.

[0025] The processing device 21 is a processor that controls the entire MR glasses 20. The processing device 21 is configured using, for example, one or more chips. The processing device 21 is also configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 21 may be realized by hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). The processing device 21 executes various processes in parallel or sequentially.

[0026] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 also stores a plurality of programs including a control program PR1 that the processing device 21 executes.

[0027] The gaze detection device 23 detects the gaze of the user U and generates gaze information indicating the detection result. Any method may be used for detecting the gaze by the gaze detection device 23. The gaze detection device 23 may detect the gaze information based on the position of the inner corner of the eye and the position of the iris, for example. The gaze information indicates the direction of the gaze of the user U. The gaze detection device 23 outputs the gaze information to the processing device 21 described below. The gaze information output to the processing device 21 is output to the terminal device 10 via the communication device 27.

[0028] The GPS device 24 receives radio waves from multiple satellites. The GPS device 24 also generates position information from the received radio waves. The position information indicates the position of the MR glasses 20. The position information may be in any format as long as it can identify the position. The position information indicates, for example, the latitude and longitude of the MR glasses 20. As an example, the position information is obtained from the GPS device 24. However, the MR glasses 20 may obtain the position information by any method. The obtained position information is output to the processing device 21. The position information output to the processing device 21 is output to the terminal device 10 via the communication device 27.

[0029] The motion detection device 25 detects the motion of the MR glasses 20. Examples of the motion detection device 25 include inertial sensors such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects acceleration on orthogonal X-, Y-, and Z-axes. The gyro sensor detects angular acceleration around the X-, Y-, and Z-axes as the central axes of rotation. The motion detection device 25 can generate posture information indicating the posture of the MR glasses 20 based on the output information of the gyro sensor. The motion information includes acceleration data indicating the acceleration of each of the three axes and angular acceleration data indicating the angular acceleration of each of the three axes. Furthermore, the motion detection device 25 outputs the posture information indicating the posture of the MR glasses 20 and the motion information related to the motion of the MR glasses 20 to the processing device 21. The posture information and motion information output to the processing device 21 are output to the terminal device 10 via the communication device 27.

[0030] The imaging device 26 outputs imaging information obtained by capturing an image of the outside world. The imaging device 26 also includes, for example, a lens, an imaging element, an amplifier, and an AD converter. Light collected through the lens is converted into an imaging signal, which is an analog signal, by the imaging element. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging information, which is a digital signal. The converted imaging information is output to the processing device 21. The imaging information output to the processing device 21 is output to the terminal device 10 via the communication device 27.

[0031] The communication device 27 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 27 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 27 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 27 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0032] The display 28 is a device that displays images. The display 28 displays various images under the control of the processing device 21. As described above, the display 28 includes the lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as the lens 41R, a display panel for the right eye, and an optical member for the right eye. As the display panel, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used.

[0033] The vibration device 29 is a vibrator. The vibration device 29 vibrates under the control of the processing device 21. The vibration device 29 includes a vibration device 29-1 and a vibration device 29-2. The vibration device 29-1 is provided on the frame 94. The vibration device 29-2 is provided on the frame 95. For example, if a predetermined condition is met, when the user U turns his / her face to the right while wearing the MR glasses 20 on his / her head, the processing device 21 vibrates the vibration device 29-1 provided on the frame 94 on the left side as seen from the user U. Similarly, if a predetermined condition is met, when the user U turns his / her face to the left while wearing the MR glasses 20 on his / her head, the processing device 21 vibrates the vibration device 29-2 provided on the frame 95 on the right side as seen from the user U. This vibration allows the user U to recognize that the predetermined condition is met.

[0034] The processing device 21 functions as an acquisition unit 211 and a display control unit 212, for example, by reading out a control program PR1 from the storage device 22 and executing it.

[0035] The acquisition unit 211 acquires image information indicating an image to be displayed on the MR glasses 20 from the terminal device 10-1.

[0036] Furthermore, the acquisition unit 211 acquires gaze information input from the gaze detection device 23, position information input from the GPS device 24, movement information input from the movement detection device 25, and imaging information input from the imaging device 26. Then, the acquisition unit 211 outputs the acquired gaze information, position information, movement information, and imaging information to the communication device 27.

[0037] Based on the image information acquired by the acquisition unit 211 from the terminal device 10-1, the display control unit 212 causes the display 28 to display an image indicated by the image information.

[0038] 1-1-3: Terminal device configuration 4 is a block diagram showing an example configuration of the terminal device 10. The terminal device 10 includes a processing device 11, a storage device 12, a communication device 13, a display 14, an input device 15, and an inertial sensor 16. The elements of the terminal device 10 are connected to each other by one or more buses for communicating information.

[0039] The processing device 11 is a processor that controls the entire terminal device 10. The processing device 11 is configured, for example, using one or more chips. The processing device 11 is configured, for example, using a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 11 may be realized by hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 11 executes various processes in parallel or sequentially.

[0040] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 also stores a plurality of programs including a control program PR2 that the processing device 11 executes.

[0041] The communication device 13 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 13 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0042] The display 14 is a device that displays images and text information. The display 14 displays various images under the control of the processing device 11. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display 14. Note that when XR glasses such as MR glasses 20 are connected to the terminal device 10, the display 14 does not need to be an essential component. In this case, the XR glasses will further have the same functions as the display 14.

[0043] The input device 15 receives operations from a user U wearing the MR glasses 20 on their head. For example, the input device 15 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 15 includes a touch panel, it may also serve as the display 14.

[0044] The inertial sensor 16 is a sensor that detects inertial force. The inertial sensor 16 includes, for example, one or more of an acceleration sensor, an angular velocity sensor, and a gyro sensor. The processing device 11 detects the attitude of the terminal device 10 based on the output information of the inertial sensor 16. Furthermore, the processing device 11 accepts the selection of a virtual object, the input of text, and the input of instructions in the spherical virtual space VS based on the attitude of the terminal device 10. For example, the user U operates the input device 15 while pointing the central axis of the terminal device 10 toward a predetermined area in the virtual space VS, thereby selecting a virtual object to be placed in the predetermined area. The user U's operation on the input device 15 is, for example, a double tap. By operating the terminal device 10 in this way, the user U can select a virtual object without looking at the input device 15 of the terminal device 10.

[0045] The processing device 11 functions as an acquisition unit 111, a calculation unit 112, and a display control unit 113 by reading and executing the control program PR2 from the storage device 12.

[0046] The acquisition unit 111 acquires at least one of the type of notification and the timing of the notification for the user U. Here, "notification" includes messages from other users to the user U. Messages from other users include, for example, emails and messages sent via SNS apps. In the terminal device 10-1, the "message from other users" may be a message generated by another user using the terminal device 10-2 and sent to the terminal device 10-1 via the communication network NET. Alternatively, the "message from other users" may be a message generated by another user using another device not shown in FIG. 1 and sent to the terminal device 10-1 via the communication network NET. The "other devices" may be, for example, mobile terminal devices such as smartphones and tablets. Alternatively, the "other devices" may be, for example, stationary personal computers (PCs). Furthermore, the "other devices" do not necessarily have to be devices that display virtual spaces.

[0047] The "notification" may include a message, such as a warning or a message, from each application corresponding to the virtual object displayed in the virtual space. The acquisition unit 111 is an example of a first acquisition unit. The notification is an example of a first notification.

[0048] The acquisition unit 111 also acquires image information indicating an image to be displayed on the MR glasses 20 from the server 30 via the communication device 13. Furthermore, the acquisition unit 111 uses the communication device 13 to acquire gaze information, position information, posture information, movement information, and imaging information from the MR glasses 20.

[0049] Furthermore, the acquisition unit 111 acquires an operation of the user U on a virtual object in the three-dimensional virtual space. The content of the operation of the user U is acquired, for example, from the MR glasses, and is generated by a generation unit (not shown) based on line-of-sight information, position information, posture information, movement information, and imaging information.

[0050] The calculation unit 112 calculates a display position of a virtual object corresponding to the notification in a three-dimensional virtual space displayed on the MR glasses 20 based on at least one of the type of the notification and the timing of the notification. Specifically, the calculation unit 112 calculates a positional relationship between the virtual object corresponding to the notification in the three-dimensional virtual space and the user U located at the center of the virtual space, and calculates the display position based on the positional relationship. More specifically, the positional relationship is a relative positional relationship of the virtual object with respect to the user U. For example, the positional relationship is a three-dimensional coordinate system with the position of the user U in the virtual space as the origin. As will be described later, the display control unit 113 displays a virtual object corresponding to each notification in the virtual space. When the display control unit 113 displays a virtual object in the virtual space, the calculation unit 112 calculates a positional relationship between the display position of the virtual object and the user U located at the center of the virtual space. Note that this positional relationship is an example of a first positional relationship. Furthermore, the virtual object is an example of a first virtual object.

[0051] The "notification type" above is, for example, the type of the sender of the notification. The calculation unit 112 may calculate a positional relationship such that the virtual object corresponding to the email in which the notification is an email and the sender is a person who is important to the user U is closer to the user U in the virtual space.

[0052] Furthermore, the above-mentioned "notification type" may be, for example, a type of importance or priority of the notification content. When the above-mentioned notification is an email and a flag attached to the email indicates that the email is more important, the calculation unit 112 may calculate a positional relationship such that the virtual object corresponding to the email is closer to the user U in the virtual space.

[0053] Alternatively, the above-mentioned "notification type" may be, for example, the type of an application that outputs a notification to the virtual space. Such applications may include, for example, a security-related application, a social networking application, and a time measurement application. The calculation unit 112 may calculate a positional relationship such that a virtual object corresponding to a notification that has a higher importance or priority for the user U regarding the output source application is closer to the user U in the virtual space.

[0054] Furthermore, if the "notification" is a message from another user to user U, the above-mentioned "timing of notification" refers to the timing when user U receives the message. However, the "timing of notification" is not limited to the timing when user U receives the message. For example, the "timing of notification" may be the timing when another user sends a message to user U. Furthermore, if the "notification" is a message from each application, the above-mentioned "timing of notification" refers to the timing when each application issues the message.

[0055] Furthermore, for example, the calculation unit 112 may calculate a positional relationship such that the virtual object corresponding to an earlier notification is farther away in the virtual space from the user U. That is, the calculation unit 112 may calculate a positional relationship such that the virtual object corresponding to the latest notification is closest to the user U. On the other hand, the calculation unit 112 may calculate a positional relationship such that the virtual object corresponding to an older notification is farther away from the user U.

[0056] The display control unit 113 displays the virtual object in the virtual space based on the positional relationship calculated by the calculation unit 112. More specifically, the display control unit 113 uses image information acquired from the server 30 via the communication device 13 and displays the virtual object in the virtual space VS based on the positional relationship. In addition, the display control unit 113 causes the virtual object that was displayed in the virtual space to disappear based on an operation performed by the user U.

[0057] 5A to 5C are explanatory diagrams of the operation of the display control unit 113. In the following description, it is assumed that the virtual space VS has an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other. As an example, the X-axis extends in the front-to-back direction of the user U. Furthermore, as viewed from the user U, the forward direction along the X-axis is defined as the X1 direction, and the backward direction along the X-axis is defined as the X2 direction. Furthermore, the Y-axis extends in the left-to-right direction of the user U. Furthermore, as viewed from the user U, the right direction along the Y-axis is defined as the Y1 direction, and the left direction along the Y-axis is defined as the Y2 direction. These X-axis and Y-axis form a horizontal plane. Furthermore, the Z-axis is orthogonal to the XY plane and extends in the up-down direction of the user U. Furthermore, as viewed from the user U, the downward direction along the Z-axis is defined as the Z1 direction, and the upward direction along the Z-axis is defined as the Z2 direction.

[0058] 5A, in the virtual space VS, the user U is located at coordinates (x, y, z) = (0, 0, 0), which is the center of the virtual space VS. The display control unit 113 displays virtual objects VO1 to VO5 in the virtual space VS. More specifically, as described above, the calculation unit 112 calculates the positional relationship between the virtual object VO corresponding to the notification in the virtual space VS and the user U located at the center of the virtual space VS, based on at least one of the type of notification and the timing of the notification. Specifically, the calculation unit 112 calculates five coordinates, from the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z) = (x5, y5, z5) of the virtual object VO5, based on at least one of the type of notification and the timing of the notification. The display control unit 113 displays the virtual objects VO1 to VO5 at the positions indicated by the coordinates calculated by the calculation unit 112. In FIG. 5A, as an example, it is assumed that virtual object VO1 is displayed at a position closest to user U. Note that in FIG. 5A, as an example, it is assumed that virtual objects VO1 to VO5 are spheres and that text files indicating the content of the notification are stored within the spheres. However, the shapes of virtual objects VO1 to VO5 are not limited to spheres and may be any shapes. Also, in FIG. 5A, five virtual objects VO, from virtual object VO1 to virtual object VO5, are shown. However, the number of virtual objects VO is not limited to five and may be any number.

[0059] Referring to FIG. 5B, the user U moves in the virtual space VS from a position at coordinates (x, y, z)=(0, 0, 0) to a position at coordinates (x, y, z)=(x u ,y u ,z u ) and then the user U comes into contact with the virtual object VO1 in the virtual space VS.

[0060] As a result, as illustrated in FIG. 5C, the display control unit 113 causes the sphere serving as the virtual object VO1 to disappear. After the disappearance, the display control unit 113 causes the virtual object VO6, which is a text file containing the content of the notification corresponding to the virtual object VO1 and which was stored in the sphere, to be displayed at coordinates (x, y, z)=(x6, y6, z6) within the sphere. Note that the method of displaying the virtual object VO6, which is a text file, is not limited to the method of disappearing the sphere serving as the virtual object VO1. For example, the display control unit 113 may apply morphing to transform the virtual object VO1 into the virtual object VO6.

[0061] In the virtual space VS, a virtual object VO corresponding to a notification with higher importance or priority for the user U is displayed closer to the user U. Furthermore, when the virtual object VO disappears, the area of ​​the region blocking the view of the user U decreases. Therefore, even if the amount of notifications to the user U increases, the terminal device 10-1 as a display control device can suppress a decrease in the processing efficiency of notifications by minimizing the degree to which the view is blocked in accordance with the amount of notifications.

[0062] 5A to 5C, the user U approaches the virtual object VO1 and then touches the virtual object VO1, causing the virtual object VO1 to disappear, but the method by which the user U touches the virtual object VO1 is not limited to this. For example, the user U may touch the virtual object VO1 by pulling on a string (not shown) attached to the virtual object VO1.

[0063] The display control unit 113 may also change the size of the virtual object VO depending on the amount of data in the text file included in the virtual object VO. Furthermore, the display control unit 113 may cause the virtual object VO to disappear after a predetermined time has elapsed since the virtual object VO was displayed.

[0064] 5A to 5C, for the sake of simplicity, the user U and the virtual objects VO1 to VO5 are displayed in two dimensions. However, in the actual virtual space VS, the user U and the virtual objects VO1 to VO5 are usually displayed in three dimensions. However, the display method of the user U and the virtual objects VO1 to VO5 is not limited to three-dimensional display, and may be two-dimensional display.

[0065] 1-1-4: Server configuration 6 is a block diagram showing an example of the configuration of the server 30. The server 30 includes a processing device 31, a storage device 32, a communication device 33, a display 34, and an input device 35. The elements of the server 30 are connected to each other by one or more buses for communicating information.

[0066] The processing device 31 is a processor that controls the entire server 30. The processing device 31 is configured, for example, using one or more chips. The processing device 31 is configured, for example, using a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 31 may be realized by hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 31 executes various processes in parallel or sequentially.

[0067] The storage device 32 is a recording medium that can be read from and written to by the processing device 31. The storage device 32 also stores a plurality of programs including a control program PR3 that the processing device 31 executes.

[0068] The communication device 33 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 33 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 33 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 33 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0069] The display 34 is a device that displays images and text information. The display 34 displays various images under the control of the processing device 31. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used as the display 34.

[0070] The input device 35 is a device that accepts operations by an administrator of the information processing system 1. For example, the input device 35 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 35 includes a touch panel, it may also serve as the display 34.

[0071] The processing device 31 functions as an acquisition unit 311 and a generation unit 312, for example, by reading out a control program PR3 from the storage device 32 and executing it.

[0072] The acquisition unit 311 acquires various data from the terminal device 10 using the communication device 33. The data includes, for example, data indicating the operation content for the virtual object VO, which is input to the terminal device 10 by a user U wearing the MR glasses 20 on his / her head.

[0073] The generation unit 312 generates image information indicating an image to be displayed on the MR glasses 20. This image information is transmitted to the terminal device 10 via the communication device 33. The display control unit 113 provided in the terminal device 10 displays the virtual object VO in the virtual space VS based on the image information acquired using the communication device 13 as described above.

[0074] 1-2: Operation of the first embodiment 7 is a flowchart showing the operation of the terminal device 10 according to the first embodiment. Hereinafter, the operation of the terminal device 10 will be described with reference to FIG.

[0075] In step S1, the processing device 11 functions as an acquisition unit 111. The processing device 11 acquires at least one of the type of notification and the timing of notification.

[0076] In step S2, the processing device 11 functions as the calculation unit 112. The processing device 11 calculates the display position of the virtual object VO corresponding to the notification in the three-dimensional virtual space VS displayed on the MR glasses 20, based on at least one of the type of notification and the timing of the notification.

[0077] In step S3, the processing device 11 functions as the display control unit 113. The processing device 11 displays the virtual object VO in the virtual space VS based on the display position calculated by the calculation unit 112. Thereafter, the processing device 11 ends all operations shown in FIG.

[0078] 1-3: Effects of the First Embodiment According to the above description, the terminal device 10 as a display control device includes an acquisition unit 111 as a first acquisition unit, a calculation unit 112, and a display control unit 113. The acquisition unit 111 acquires at least one of the type of notification and the timing of the notification. The calculation unit 112 calculates a display position of a virtual object VO corresponding to the notification in a three-dimensional virtual space VS displayed on the MR glasses 20, based on at least one of the type of notification and the timing of the notification. The display control unit 113 displays the virtual object VO in the virtual space VS, based on the display position calculated by the calculation unit 112.

[0079] The terminal device 10 has the above configuration, and thus the virtual object VO is arranged according to at least one of the notification type and the notification timing. Therefore, the user U can visually grasp at least one of the notification type and the notification timing corresponding to the virtual object VO in the three-dimensional virtual space VS. Therefore, compared to a case where the virtual object VO is not arranged according to at least one of the notification type and the notification timing, the terminal device 10 minimizes the degree of obstruction of the field of view according to the amount of notifications, thereby preventing a decrease in the efficiency with which the user processes notifications even when the amount of notifications increases. In particular, when most of the field of view of the user U, who is located at the center of the virtual space VS, is covered by multiple virtual objects VO corresponding to each notification, the user U may not be able to determine which virtual object VO to process most efficiently. As an example, the terminal device 10 may arrange the virtual object VO corresponding to a notification of high importance or priority for the user U, near the user U. This arrangement improves the efficiency with which the user U processes notifications.

[0080] According to the above description, the terminal device 10 includes the acquisition unit 111. The acquisition unit 111 acquires an operation on the virtual object VO. The display control unit 113 causes the virtual object VO to disappear from the virtual space VS based on the operation of the user U.

[0081] The terminal device 10 has the above configuration, and therefore can cause a virtual object VO to disappear in the virtual space VS on the condition that the user U touches the virtual object VO. For example, if the virtual object VO corresponding to a notification that the user U has already recognized disappears, the user U can improve the efficiency of notification processing.

[0082] 2: Second embodiment Hereinafter, a configuration of an information processing system 1A including a terminal device 10A as a display control device according to a second embodiment of the present invention will be described with reference to Fig. 8. In the following description, for the purpose of simplifying the description, among the components included in the information processing system 1A according to the second embodiment, the same components as those in the information processing system 1 according to the first embodiment will be designated by the same reference numerals, and their description may be omitted.

[0083] 2-1: Configuration of the second embodiment 2-1-1: Overall structure An information processing system 1A according to the second embodiment of the present invention differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10A instead of the terminal device 10. In other respects, the overall configuration of the information processing system 1A is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in Fig. 1, and therefore illustration and description thereof will be omitted.

[0084] 2-1-2: Terminal device configuration 8 is a block diagram showing an example of the configuration of the terminal device 10 A. The terminal device 10 A differs from the terminal device 10 in that it includes a processing device 11 A instead of the processing device 11 and a storage device 12 A instead of the storage device 12.

[0085] The storage device 12A differs from the storage device 12 in that it stores a control program PR2A instead of the control program PR2.

[0086] The processing device 11A includes a calculation unit 112A instead of the calculation unit 112 included in the processing device 11, and a display control unit 113A instead of the display control unit 113. In addition to the components included in the processing device 11, the processing device 11A also includes a recognition unit 114, a classification unit 115, and a visual control unit 116.

[0087] The recognition unit 114 recognizes an object placed in real space. In particular, the recognition unit 114 recognizes the shape of the object placed in real space and the position of the object in a composite space in which the real space and the virtual space VS are superimposed.

[0088] The classification unit 115 classifies notifications according to the user U's behavioral history in the virtual space VS and outputs the notification types to the acquisition unit 111. Here, the "user U's behavioral history" refers to, for example, the usage history of applications used by the user U in the virtual space VS. In this case, the classification unit 115 classifies notifications output from applications used more frequently by the user U into a category of notification with higher importance or priority for the user U. Alternatively, the "user U's behavioral history" may refer to, for example, the user U's movement history in the virtual space VS or a composite space in which the virtual space VS and real space are superimposed. In this case, the movement history is calculated based on measurement results by the GPS device 24 provided in the MR glasses 20. For example, while the user U is at work, the classification unit 115 classifies work-related notifications into a category of notification with higher importance or priority for the user U. Furthermore, the classification unit 115 classifies notifications related to the user U's private life into a category of notification with lower importance or priority for the user U. On the other hand, while the user U is at home, the classification unit 115 classifies notifications related to the user U's private life into a type of notification that is more important or has a higher priority for the user U. Also, while the user U is at home, the classification unit 115 classifies notifications related to the user U's work into a type of notification that is less important or has a lower priority for the user U. The classification unit 115 classifies notifications based on the user U's behavior history, and therefore the classification type is dynamically changed depending on the user U's situation. The classification unit 115 may be provided in the terminal device 10 described in the first embodiment.

[0089] Similar to the calculation unit 112, the calculation unit 112A calculates the display position of the virtual object VO in the three-dimensional virtual space VS based on at least one of the type of notification and the timing of the notification. At this time, the calculation unit 112A calculates the display position by further taking into account the type of notification that is the result of classification by the classification unit 115. For example, the calculation unit 112A calculates a positional relationship in which the virtual object VO corresponding to a notification classified by the classification unit 115 into a type with higher importance or priority is placed closer to the user U. As in the first embodiment, this positional relationship is an example of a first positional relationship.

[0090] Furthermore, the calculation unit 112A calculates the positional relationship between the virtual object VO and an object in the real space in a combined space in which the virtual space VS and the real space are superimposed, based on the recognition result by the recognition unit 114. More specifically, the calculation unit 112A calculates a positional relationship in the combined space in which the virtual object VO and the object do not overlap partially or entirely. For example, the calculation unit 112A calculates an area in the combined space in which the virtual object VO does not overlap with the object as the positional relationship. Note that the positional relationship between the virtual object VO and the object in the combined space is an example of a second positional relationship.

[0091] The calculation unit 112A further calculates a positional relationship between the virtual object VO and the user based on the calculated first positional relationship and second positional relationship, and calculates a display position of the virtual object VO based on this positional relationship. The calculated positional relationship is a positional relationship in which the virtual object VO does not overlap with an object and the virtual object VO is placed at a position corresponding to at least one of the notification type and the notification timing. In this calculation, the calculation unit 112A calculates a positional relationship indicating that the virtual object VO is to be placed at a position corresponding to at least one of the notification type and the notification timing, in an area in the composite space where it does not overlap with an object. In other words, the calculation unit 112A calculates the positional relationship based on the recognition result by the recognition unit 114 and at least one of the notification type and the notification timing.

[0092] The visual control unit 116 controls the visual behavior of the virtual object VO based on at least one of the type of notification and the timing of the notification. Here, the "visual behavior" may be, for example, vibration of the virtual object VO. Alternatively, the "visual behavior" may be, for example, behavior in which the virtual object VO moves from outside the field of view of the user U wearing the MR glasses 20 on their head, through the center of the field of view, and then out of the field of view again.

[0093] For example, the visual control unit 116 may cause a virtual object VO corresponding to a notification with high importance or priority to behave in a way that gives a stronger impression to the user U than a notification with low importance or priority. For example, the visual control unit 116 may control the virtual object VO corresponding to a notification with high importance or priority to vibrate more strongly than the virtual object VO corresponding to a notification with low importance or priority. Alternatively, the visual control unit 116 may control the virtual object VO corresponding to a new notification to vibrate, while not vibrating an existing virtual object VO corresponding to a previous notification.

[0094] 2-2: Operation of the second embodiment The basic operation of the terminal device 10A according to the second embodiment is the same as the operation of the terminal device 10 shown in FIG. 7, and therefore is not illustrated. As a detailed operation of the terminal device 10A, in step S2 of FIG. 7, the processing device 11A functions as a calculation unit 112A. The processing device 11A calculates the above-mentioned display position. Furthermore, in step S3 of FIG. 7, the processing device 11A functions as a display control unit 113A. Based on the calculated display position, the processing device 11A displays the virtual object VO in a composite space in which the virtual space VS and the real space are superimposed. Furthermore, in step S3, the visual control unit 116 controls the visual behavior of the virtual object VO based on at least one of the type of notification and the timing of the notification.

[0095] 2-3: Effects of the second embodiment According to the above description, the terminal device 10A serving as a display control device includes the calculation unit 112A, the display control unit 113A, and the recognition unit 114. The recognition unit 114 recognizes an object placed in real space. The calculation unit 112A calculates a display position based on the recognition result by the recognition unit 114 and at least one of the type of notification and the timing of the notification. Based on the display position, the display control unit 113A displays the virtual object VO in the composite space at a position that does not overlap with the object.

[0096] The terminal device 10A has the above configuration, and thus can display the virtual object VO in a manner that the virtual object VO does not overlap with an object either partially or entirely in the composite space. Because the virtual object VO does not overlap with an object, the user U can easily view the virtual object VO, and the efficiency of notification processing can be improved.

[0097] According to the above description, the terminal device 10A as a display control device includes the visual control unit 116. The visual control unit 116 controls the visual behavior of the virtual object VO based on at least one of the type of notification and the timing of the notification.

[0098] Since the terminal device 10A has the above configuration, the user U can visually recognize at least one of the type of notification and the timing of the notification from the behavior of the virtual object VO. For example, when the type of notification indicates high importance or priority, the virtual object VO corresponding to the notification makes a stronger impression on the user U compared to when the type of notification indicates low importance or priority. Therefore, the user U can process the virtual object VO corresponding to the notification with higher importance or priority first, thereby improving the efficiency of notification processing.

[0099] According to the above description, the terminal device 10A as a display control device includes the classification unit 115. The classification unit 115 outputs the type of notification by classifying the notification according to the behavior history of the user U in the virtual space VS or the real space.

[0100] The terminal device 10A has the above configuration, and therefore can control the display position of the virtual object VO in accordance with the behavior history of the user U. Since the behavior history of the user U is reflected in the display position of the virtual object VO, the user U can improve the efficiency of notification processing.

[0101] 3: Third embodiment Hereinafter, a configuration of an information processing system 1B including a terminal device 10B as a display control device according to a third embodiment of the present invention will be described with reference to Fig. 9. In the following description, for the purpose of simplifying the description, among the components included in the information processing system 1B according to the third embodiment, the same components as those in the information processing system 1 according to the first embodiment will be designated by the same reference numerals, and their description may be omitted.

[0102] 3-1: Configuration of the third embodiment 3-1-1: Overall structure An information processing system 1B according to the third embodiment of the present invention differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10B instead of the terminal device 10. In other respects, the overall configuration of the information processing system 1B is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in Fig. 1, and therefore illustration and description thereof will be omitted.

[0103] 3-1-2: Terminal device configuration 9 is a block diagram showing an example of the configuration of the terminal device 10B. The terminal device 10B differs from the terminal device 10 in that it includes a processing device 11B instead of the processing device 11 and a storage device 12B instead of the storage device 12.

[0104] The storage device 12B differs from the storage device 12 in that it stores a control program PR2B instead of the control program PR2.

[0105] The processing device 11B includes a first acquisition unit 111A instead of the acquisition unit 111 included in the processing device 11, and a display control unit 113B instead of the display control unit 113. In addition to the components included in the processing device 11, the processing device 11B also includes a second acquisition unit 117, an operation control unit 118, and a vibration control unit 119.

[0106] The function of the first acquisition unit 111A is the same as that of the acquisition unit 111.

[0107] The display control unit 113B displays a virtual object VO indicating a storage body that stores a plurality of virtual objects VO in the virtual space VS.

[0108] FIG. 10 is an explanatory diagram of a virtual object VO7 representing a storage body. The storage body represented by the virtual object VO7 stores virtual objects VO8 to VO13, each of which corresponds to a notification. The display control unit 113B may store all of the virtual objects VO corresponding to notifications in the storage body represented by the virtual object VO7. Alternatively, the display control unit 113B may store, in the storage body represented by the virtual object VO7, virtual objects VO corresponding to notifications that were displayed in the virtual space VS and remained unconfirmed by the user U for a predetermined period of time. Note that in FIG. 10, a total of six virtual objects VO8 to VO13 are stored in the storage body represented by the virtual object VO7. However, the number of virtual objects VO stored in the storage body is not limited to six and may be any number. Note that in FIG. 10, for simplicity of explanation, the virtual object VO7 representing the storage body and the virtual objects VO8 to VO13 stored in the storage body are displayed in two dimensions. However, in the actual virtual space VS, virtual objects VO7 to VO13 are usually displayed in three dimensions. However, the display method of virtual objects VO7 to VO13 is not limited to three-dimensional display, and they may be displayed in two dimensions. Furthermore, the display control unit 113A may set a predetermined installation area in the virtual space VS, instead of virtual object VO7 as a storage body, and install virtual objects VO8 to VO13 in that installation area.

[0109] The second acquisition unit 117 acquires the total volume of the plurality of virtual objects VO stored in the storage body indicated by the virtual object VO7 or the number of the plurality of virtual objects VO.

[0110] The operation control unit 118 controls the tracking of the movement of the virtual object VO7 as a storage body in the virtual space VS with respect to the movement of the user U in real space, in accordance with the volume or the quantity acquired by the second acquisition unit 117. Here, as the "control of tracking," the operation control unit 118 controls how much to delay the movement of the virtual object VO7 with respect to the movement of the user U.

[0111] Specifically, the larger the overall volume of the virtual objects VO stored in the storage body represented by the virtual object VO7, the greater the degree of delay in the movement of the virtual object VO7 displayed on the MR glasses 20 relative to the movement of the MR glasses 20 accompanying the movement of the head of the user U. Alternatively, the greater the number of virtual objects VO stored in the storage body represented by the virtual object VO7, the greater the degree of delay in the movement of the virtual objects VO7 displayed on the MR glasses 20 relative to the movement of the MR glasses 20 accompanying the movement of the head of the user U.

[0112] Since the operation control unit 118 has the above function, the larger the overall volume of the multiple virtual objects VO stored in the storage body represented by the virtual object VO7, or the larger the number of the multiple virtual objects VO, the more uncomfortable the user U will feel. Therefore, the user U will be more likely to notice that a large number of virtual objects VO have accumulated in the storage body represented by the virtual object VO7.

[0113] The vibration control unit 119 operates the vibration device 29-1 and the vibration device 29-2 provided in the MR glasses 20 as a display device worn by the user U, according to the volume or the quantity.

[0114] Specifically, when the volume of the entire virtual object VO stored in the storage body indicated by the virtual object VO7 is larger than a predetermined value and the face of the user U wearing the MR glasses 20 on his head turns to the right, the vibration device 29-1 provided in the frame 94, which is the frame on the left side of the MR glasses 20, is vibrated. On the other hand, when the volume of the entire virtual object VO stored in the storage body indicated by the virtual object VO7 is larger than a predetermined value and the face of the user U wearing the MR glasses 20 on his head turns to the left, the vibration device 29-2 provided in the frame 95, which is the frame on the right side of the MR glasses 20, is vibrated. Furthermore, when the volume of the entire virtual object VO stored in the storage body indicated by the virtual object VO7 is larger than a predetermined value and the user U wearing the MR glasses 20 on his head moves forward in the virtual space VS, both the vibration device 29-1 and the vibration device 29-2 are vibrated.

[0115] Alternatively, when the number of virtual objects VO stored in the storage body indicated by the virtual object VO7 is greater than a predetermined value and the face of the user U wearing the MR glasses 20 on his head turns to the right, the vibration device 29-1 provided in the frame 94, which is the frame on the left side of the MR glasses 20, is vibrated. On the other hand, when the number of virtual objects VO stored in the storage body indicated by the virtual object VO7 is greater than a predetermined value and the face of the user U wearing the MR glasses 20 on his head turns to the left, the vibration device 29-2 provided in the frame 95, which is the frame on the right side of the MR glasses 20, is vibrated. Furthermore, when the number of virtual objects VO stored in the storage body indicated by the virtual object VO7 is greater than a predetermined value and the user U wearing the MR glasses 20 on his head moves forward in the virtual space VS, both the vibration device 29-1 and the vibration device 29-2 are vibrated.

[0116] Because the vibration control unit 119 has the above-mentioned function, the larger the total volume of the multiple virtual objects VO stored in the storage body represented by the virtual object VO7, or the greater the number of the multiple virtual objects VO, the more the user U will feel a weight resisting rotation of the head around the neck or a weight resisting forward movement as vibration. This vibration makes it easier for the user U to notice that a large number of virtual objects VO have accumulated in the storage body represented by the virtual object VO7.

[0117] Note that the vibration control unit 119 may increase the vibration frequency of the vibrating device 29 as the total volume of the multiple virtual objects VO accumulated in the storage body represented by the virtual object VO7 increases. Alternatively, the vibration control unit 119 may increase the vibration frequency of the vibrating device 29 as the number of multiple virtual objects VO accumulated in the storage body represented by the virtual object VO7 increases.

[0118] 3-2: Operation of the third embodiment The basic operation of the terminal device 10B according to the third embodiment is the same as the operation of the terminal device 10 shown in FIG. 7, and therefore is not shown in the figure. As a detailed operation of the terminal device 10B, after repeating the operations of steps S1 to S3 of FIG. 7, the second acquisition unit 117 acquires the total volume of the multiple virtual objects VO stored in the storage body shown as the virtual object VO7 or the number of the multiple virtual objects VO. The operation control unit 118 controls the followability of the movement of the virtual object VO representing the storage body in the virtual space VS to the movement of the user U in the real space, in accordance with the volume or the number acquired by the second acquisition unit 117. The vibration control unit 119 operates the vibration device 29 provided in the MR glasses 20 serving as a display device worn by the user U, in accordance with the volume or the number.

[0119] 3-3: Effects of the third embodiment According to the above description, the terminal device 10B serving as a display control device includes a display control unit 113B. The display control unit 113B displays a virtual object VO7 representing a storage body that stores multiple virtual objects VO in a virtual space VS. The second acquisition unit 117 acquires the total volume of the multiple virtual objects VO stored in the storage body or the number of the multiple virtual objects VO. The operation control unit 118 controls the followability of the movement of the virtual object VO7 representing the storage body in the virtual space VS to the movement of the user U in real space, in accordance with the volume or the number acquired by the second acquisition unit 117.

[0120] Since the terminal device 10B has the above configuration, the tracking performance of the movement of the virtual object VO7 differs depending on the overall volume of the multiple virtual objects VO stored in the storage body or the number of the multiple virtual objects VO. Therefore, the user U can recognize the number of virtual objects VO contained in the storage body from the way the virtual objects VO move. Furthermore, the larger the volume or the larger the number, the slower the tracking performance, so the user U can easily grasp the number of virtual objects VO contained in the storage body. As a result, the user U can easily notice that a large number of virtual objects VO have accumulated in the storage body indicated by the virtual object VO7. This allows the user U to improve the efficiency of notification processing.

[0121] According to the above description, the terminal device 10B as a display control device includes a vibration control unit 119. The vibration control unit 119 operates the vibration devices 29-1 and 29-2 provided in the MR glasses 20 as a display device worn by the user U, according to the volume or the quantity.

[0122] Since the terminal device 10B has the above configuration, the user U feels more vibration as the total volume of the multiple virtual objects VO stored in the storage body represented by the virtual object VO7 increases, or as the number of the multiple virtual objects VO increases. The vibration makes it easier for the user U to notice that a large number of virtual objects VO have accumulated in the storage body represented by the virtual object VO7. This allows the user U to improve the efficiency of notification processing.

[0123] 4: Variation The present disclosure is not limited to the above-described exemplary embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples may be combined.

[0124] 4-1: Variation 1 The terminal device 10 according to the first embodiment includes an acquisition unit 111 as a first acquisition unit, a calculation unit 112, and a display control unit 113. The acquisition unit 111 acquires at least one of the type of notification and the timing of the notification. The calculation unit 112 calculates a display position of a virtual object VO corresponding to the notification in a three-dimensional virtual space VS based on at least one of the type of notification and the timing of the notification. The display control unit 113 displays the virtual object VO in the virtual space VS based on the display position calculated by the calculation unit 112. However, these operations may be executed by a device other than the terminal device 10. For example, if the server 30 includes components similar to the acquisition unit 111, the calculation unit 112, and the display control unit 113, these operations may be executed by the server 30. The same applies to the terminal device 10A according to the second embodiment and the terminal device 10B according to the third embodiment.

[0125] 4-2: Variation 2 In the information processing system 1 according to the above embodiment, the terminal device 10 and the MR glasses 20 are realized as separate entities. However, the method of realizing the terminal device 10 and the MR glasses 20 in the embodiment of the present invention is not limited to this. For example, the MR glasses 20 may have the same functions as the terminal device 10, and the terminal device 10 and the MR glasses 20 may be realized in a single housing. In other words, the MR glasses 20 may have components that are included in the terminal device 10 built in.

[0126] 4-3: Variation 3 The information processing system 1 according to the above embodiment includes MR glasses 20. However, the information processing system 1 may include any one of an HMD (Head Mounted Display) employing VR (Virtual Reality) technology, VR glasses employing VR technology, an HMD employing AR (Augmented Reality) technology, AR glasses employing AR technology, and an HMD employing MR technology, instead of the MR glasses 20. Alternatively, the information processing system 1 may include any one of a regular smartphone and a tablet equipped with an imaging device, instead of the MR glasses 20. These HMDs, VR glasses, AR glasses, smartphones, and tablets are examples of display devices.

[0127] 4-4: Variation 4 In the information processing system 1 according to the first embodiment, the generation unit 312 included in the server 30 generates image information indicating an image to be displayed on the MR glasses 20. However, the device that generates the image information is not limited to the server 30. For example, the terminal device 10 may generate the image information. In this case, the information processing system 1 does not need to include the server 30 as an essential component. The same applies to the information processing system 1A according to the second embodiment and the information processing system 1B according to the third embodiment.

[0128] 5:Other (1) In the above-described embodiment, storage devices 12-12B, storage device 22, and storage device 32 are exemplified by ROM and RAM, but may be flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, or other suitable storage media. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communications network NET via a telecommunications line.

[0129] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0130] (3) In the above-described embodiment, input and output information may be stored in a specific location (for example, a memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.

[0131] (4) In the above-described embodiment, the determination may be made by a value (0 or 1) represented using one bit, by a Boolean value (true or false), or by a comparison of numerical values ​​(e.g., comparison with a predetermined value).

[0132] (5) The order of the process procedures, sequences, flowcharts, etc. illustrated in the above-described embodiments may be rearranged unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0133] (6) Each function illustrated in Figures 1 to 10 is realized by any combination of at least one of hardware and software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are connected directly or indirectly (for example, by wire, wirelessly, etc.) and these multiple devices. A functional block may also be realized by combining software with the single device or the multiple devices.

[0134] (7) The programs exemplified in the above-described embodiments should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0135] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0136] (8) In each of the foregoing embodiments, the terms "system" and "network" are used interchangeably.

[0137] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0138] (10) In the above-described embodiments, the terminal device 10 and the server 30 may be mobile stations (MS). A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate term. In this disclosure, terms such as "mobile station," "user terminal," "user equipment (UE)," and "terminal" may be used interchangeably.

[0139] (11) In the above-described embodiments, the terms "connected," "coupled," or any variations thereof refer to any direct or indirect connection or coupling between two or more elements, including the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements are considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0140] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0141] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judgment" or "decision." In other words, "judgment" and "decision" can include regarding some action as having been "judgment" or "decision." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0142] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or" as used in this disclosure is not intended to be an exclusive or.

[0143] (15) In this disclosure, where articles are added by translation, such as a, an, and the in English, this disclosure may include that the nouns following these articles are plural.

[0144] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0145] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0146] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure. [Explanation of symbols]

[0147] 1, 1A, 1B... Information processing system, 10, 10A, 10B... Terminal device, 11, 11A, 11B... Processing device, 12, 12A, 12B... Storage device, 13... Communication device, 14... Display, 15... Input device, 16... Inertial sensor, 20... MR glasses, 21... Processing device, 22... Storage device, 23... Line of sight detection device, 24... GPS device, 25... Motion detection device, 26... Imaging device, 27... Communication device, 28... Display, 29... Vibration device, 30... Server, 31... Processing device, 32... Storage device, 33... Communication device, 34... Display, 35...input device, 41L, 41R...lenses, 91, 92...temples, 93...bridge, 94, 95...frame, 111...acquisition unit, 111A...first acquisition unit, 112, 112A...calculation unit, 113, 113A, 113B...display control unit, 114...recognition unit, 115...classification unit, 116...visual control unit, 117...second acquisition unit, 118...operation control unit, 119...vibration control unit, 211...acquisition unit, 212...display control unit, 311...acquisition unit, 312...generation unit, PR1, PR2, PR3...control program, U...user

Claims

1. a first acquisition unit that acquires at least one of a type of notification and a timing of the notification; a calculation unit that calculates a display position of a virtual object corresponding to the notification in a three-dimensional virtual space displayed on a display device based on at least one of the type of the notification and the timing of the notification; a display control unit that displays the virtual object in the virtual space based on the display position calculated by the calculation unit, the notification is a first notification; the virtual object is a first virtual object, the display control unit displays a virtual object indicating a storage body that stores a plurality of virtual objects including the first virtual object in the virtual space; a second acquisition unit that acquires a total volume of the plurality of virtual objects stored in the storage body or a quantity of the plurality of virtual objects; an operation control unit that controls the followability of a movement of a virtual object representing the storage body in the virtual space with respect to a movement of a user in real space, in accordance with the volume or the quantity acquired by the second acquisition unit; The display control device further comprises:

2. a recognition unit that recognizes a position of an object in a real space in a composite space in which the real space and the virtual space are superimposed, the calculation unit calculates the display position based on a position of the object recognized by the recognition unit, and at least one of a type of the notification and a timing of the notification; The display control device according to claim 1 , wherein the display control unit displays the virtual object in the composite space at a position where the virtual object does not overlap with the physical object, based on the display position.

3. The display control device according to claim 1 , further comprising a visual control unit that controls a visually recognizable behavior of the virtual object based on at least one of the type of the notification and the timing of the notification.

4. The display control device according to claim 1 or claim 2, further comprising a classification unit that outputs a type of the notification to the first acquisition unit by classifying the notification according to a behavior history of the user in the virtual space or the real space.

5. an acquisition unit that acquires a user's operation on the virtual object; The display control device according to claim 1 , wherein the display control unit causes the virtual object to disappear from the virtual space based on the operation.

6. The display control device according to claim 1 , further comprising a vibration control unit that activates a vibration device provided in the display device in accordance with the volume or the quantity.

Images