Display Control Device

Abstract

This display control device comprises an acquisition unit for acquiring a notification, a generation unit for generating a virtual object corresponding to the notification, a display control unit for displaying the virtual object in a virtual space where a first user is present, an assessment unit for assessing whether the first user is included among addressees of the notification and whether the distance between the virtual object and the first user in the virtual space is within a prescribed distance, and a movement control unit for bringing the virtual object closer to the user if the result of assessment by the assessment unit is affirmative.

Description

[Technical Field]

[0001] The present invention relates to a display control device, and more particularly to a display control device for displaying a virtual object corresponding to a notification in a virtual space. [Background technology]

[0002] XR glasses that apply XR technologies, including VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality), are known. XR glasses are worn on the user's head. Messages represented by virtual objects may be displayed in the virtual space displayed by the XR glasses.

[0003] For example, Patent Document 1 discloses a technology for sharing messages to facilitate communication between users in a virtual space. Specifically, Patent Document 1 discloses a technology for displaying a virtual object showing a "graffiti message" in a virtual space shared between users. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2010-535363 Summary of the Invention [Problem to be solved by the invention]

[0005] When displaying virtual objects corresponding to notifications containing messages in a virtual space, the number of virtual objects increases as the number of notifications increases. As a result, the virtual space displayed in the XR glasses worn by the user may become filled with multiple virtual objects. In other words, conventional technology has had the problem of reducing convenience by obstructing the user's field of view.

[0006] Therefore, an object of the present invention is to provide a display control device that, when displaying a virtual space including virtual objects corresponding to notifications, can suppress a decrease in convenience for the user even if the number of virtual objects increases in accordance with the number of notifications. [Means for solving the problem]

[0007] A preferred aspect of the present invention provides a display control device that includes: an acquisition unit that acquires a notification; a generation unit that generates a virtual object corresponding to the notification; a display control unit that displays the virtual object in a virtual space in which a first user is present; a determination unit that, when the first user is included in destination information indicating one or more destinations of the notification, determines whether a distance between the virtual object and the first user in the virtual space is within a predetermined distance; and a movement control unit that, when a determination result of the determination unit is positive, moves the virtual object to bring the virtual object closer to the first user, and, when a determination result of the determination unit is negative, does not move the virtual object. [Effects of the Invention]

[0008] According to the present invention, when a virtual space including virtual objects corresponding to notifications is displayed, even if the number of virtual objects increases in accordance with the number of notifications, a decrease in convenience for the user can be suppressed. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the overall configuration of an information processing system 1 according to a first embodiment. [Figure 2] 1 is a perspective view showing the appearance of XR glasses 20 according to a first embodiment. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of XR glasses 20 according to the first embodiment. [Figure 4] FIG. 1 is a block diagram showing an example of the configuration of a terminal device 10 according to a first embodiment. [Figure 5A]4 is an explanatory diagram illustrating an example of the operation of a generation unit 112 and a display control unit 113. [Figure 5B] 4 is an explanatory diagram illustrating an example of the operation of a generation unit 112 and a display control unit 113. [Figure 5C] 4 is an explanatory diagram illustrating an example of the operation of a display control unit 113 and a reception unit 116. FIG. [Figure 6] FIG. 2 is a block diagram showing an example of the configuration of a server 30 according to the first embodiment. [Figure 7] 4 is a flowchart showing the operation of the terminal device 10 according to the first embodiment. [Figure 8] FIG. 10 is a block diagram showing an example of the configuration of a server 30A according to the second embodiment. [Figure 9] 3 is an explanatory diagram illustrating an example of the operation of a generation unit 313 and a display control unit 314. FIG. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of a terminal device 10A according to a second embodiment. [Figure 11] 10 is a flowchart showing the operation of a server 30A according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] 1: First embodiment Hereinafter, with reference to FIGS. 1 to 7, a description will be given of the configuration of an information processing system 1 including a terminal device 10 as a display control device according to a first embodiment of the present invention.

[0011] 1-1: Configuration of the first embodiment 1-1-1: Overall structure FIG. 1 shows the overall configuration of an information processing system 1. As shown in FIG. 1, the information processing system 1 includes terminal devices 10-1 to 10-2, XR glasses 20, and a server 30. In the following description, when distinguishing between terminal devices 10 used by different users, a suffix "-X" is used in the reference numeral. X is an arbitrary integer equal to or greater than 1. The same applies to the components of each terminal device 10. In the following description, when there is no need to distinguish between the terminal devices 10-1 to 10-2, the terminal device 10 is used as a general term for the terminal devices 10-1 to 10-2. The terminal device 10 is an example of a display control device. In the information processing system 1, the terminal device 10 and the server 30 are connected to each other so as to be able to communicate with each other via a communication network NET. In addition, the terminal device 10 and the XR glasses 20 are connected to each other so as to be able to communicate with each other. Note that FIG. 1 shows two terminal devices 10-1 and 10-2 and one XR glass 20. However, this number is merely an example, and the information processing system 1 may include any number of terminal devices 10 and XR glasses 20.

[0012] In FIG. 1, it is assumed that a user U1 uses a combination of a terminal device 10-1 and XR glasses 20. In the information processing system 1 shown in FIG. 1, virtual objects corresponding to a plurality of notifications are displayed on the XR glasses 20. The notifications may include a notification transmitted from the terminal device 10-2 to the terminal device 10-1. The notifications may also include a notification transmitted from the terminal device 10-1 to the terminal device 10-2. The notifications may also include a message transmitted from another terminal device not shown in FIG. 1 to the terminal device 10-1. Furthermore, the notifications may be notifications generated by the terminal device 10-1 itself.

[0013] The server 30 provides various data and cloud services to the terminal device 10 via the communication network NET.

[0014] The terminal device 10-1 displays virtual objects arranged in a virtual space on the XR glasses 20 worn on the head of the user U1. The virtual space is, for example, a celestial sphere. Examples of the virtual objects include virtual objects representing data such as still images, videos, 3DCG models, HTML files, and text files, as well as virtual objects representing applications. Examples of text files include memos, source code, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files. The terminal device 10 is preferably, for example, a mobile terminal device such as a smartphone or a tablet. The terminal device 10-1 is an example of a display control device.

[0015] The terminal device 10-2 is a device for user U2 to send a message to user U1. The terminal device 10-2 may display a virtual object arranged in a virtual space on a display 14 (described later) or XR glasses (not shown) connected to the terminal device 10-2. In particular, when the terminal device 10-2 is a device that displays virtual objects, the configuration of the terminal device 10-2 is basically the same as that of the terminal device 10-1. Similarly to the terminal device 10-1, the terminal device 10-2 is preferably a mobile terminal device such as a smartphone or a tablet, for example.

[0016] The XR glasses 20 are a see-through wearable display worn on the head of the user U1. The XR glasses 20 are controlled by the terminal device 10 to display virtual objects on display panels provided in each of the lenses for both eyes. By wearing the XR glasses, the user U1 can view a virtual space including the virtual objects. The XR glasses 20 are an example of a display device. In the following, an example in which the XR glasses 20 are MR glasses will be described. However, the XR glasses 20 being MR glasses is merely an example, and the XR glasses 20 may also be VR glasses or AR glasses.

[0017] 1-1-2: XR Glasses Configuration Fig. 2 is a perspective view showing the appearance of the XR glasses 20. As shown in Fig. 2, the appearance of the XR glasses 20 is similar to that of ordinary eyeglasses, and includes temples 91 and 92, a bridge 93, frames 94 and 95, and lenses 41L and 41R.

[0018] The bridge 93 is provided with an imaging device 26. The imaging device 26 captures an image of the outside world. The imaging device 26 also outputs imaging information indicating the captured image.

[0019] Each of the lenses 41L and 41R is provided with a half mirror. A liquid crystal panel or organic EL panel for the left eye is provided in the frame 94. The liquid crystal panel or organic EL panel will hereinafter be collectively referred to as a display panel. The frame 94 is also provided with an optical member that guides light emitted from the display panel for the left eye to the lens 41L. The half mirror provided in the lens 41L transmits external light to guide it to the left eye and reflects the light guided by the optical member to make it incident on the left eye. The frame 95 is provided with a display panel for the right eye and an optical member that guides light emitted from the display panel for the right eye to the lens 41R. The half mirror provided in the lens 41R transmits external light to guide it to the right eye and reflects the light guided by the optical member to make it incident on the right eye.

[0020] The display 28, which will be described later, includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as a lens 41R, a display panel for the right eye, and an optical member for the right eye.

[0021] In the above configuration, the user U1 can observe the image displayed on the display panel in a see-through state, superimposed on the outside world. Furthermore, the XR glasses 20 display the left-eye image on the left-eye display panel and the right-eye image on the right-eye display panel, among the binocular images with parallax. Therefore, the XR glasses 20 allow the user to perceive the displayed image as if it has depth and a three-dimensional effect.

[0022] 3 is a block diagram showing an example configuration of the XR glasses 20. The XR glasses 20 include a processing device 21, a storage device 22, a gaze detection device 23, a GPS device 24, a motion detection device 25, an imaging device 26, a communication device 27, and a display 28. The elements of the XR glasses 20 are connected to each other by one or more buses for communicating information. Note that the term "device" in this specification may be replaced with other terms such as circuit, device, or unit.

[0023] The processing device 21 is a processor that controls the entire XR glasses 20. The processing device 21 is configured using, for example, one or more chips. The processing device 21 is also configured using, for example, a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 21 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The processing device 21 executes various processes in parallel or sequentially.

[0024] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 also stores a plurality of programs including a control program PR2 that the processing device 21 executes.

[0025] The gaze detection device 23 detects the gaze of the user U1 and generates gaze information indicating the detection result. Any method may be used for detecting the gaze by the gaze detection device 23. For example, the gaze detection device 23 may detect the gaze information based on the position of the inner corner of the eye and the position of the iris. The gaze information indicates the direction of the gaze of the user U1. The gaze detection device 23 supplies the gaze information to the processing device 21 described below. The gaze information supplied to the processing device 21 is transmitted to the terminal device 10 via the communication device 27.

[0026] The GPS device 24 receives radio waves from multiple satellites. The GPS device 24 also generates location information from the received radio waves. The location information indicates the location of the XR glasses 20. The location information may be in any format as long as it can identify the location. The location information indicates, for example, the latitude and longitude of the XR glasses 20. As an example, the location information is obtained from the GPS device 24. However, the XR glasses 20 may obtain the location information by any method. The obtained location information is supplied to the processing device 21. The location information supplied to the processing device 21 is transmitted to the terminal device 10 via the communication device 27.

[0027] The motion detection device 25 detects the motion of the XR glasses 20. Examples of the motion detection device 25 include inertial sensors such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects acceleration along orthogonal X-, Y-, and Z-axes. The gyro sensor detects angular acceleration around the X-, Y-, and Z-axes as central axes of rotation. The motion detection device 25 can generate attitude information indicating the attitude of the XR glasses 20 based on the output information of the gyro sensor. The motion information includes acceleration data indicating the acceleration along each of the three axes and angular acceleration data indicating the angular acceleration along each of the three axes. The motion detection device 25 also supplies the attitude information indicating the attitude of the XR glasses 20 and motion information related to the motion of the XR glasses 20 to the processing device 21. The attitude information and motion information supplied to the processing device 21 are transmitted to the terminal device 10 via the communication device 27.

[0028] The imaging device 26 outputs imaging information obtained by capturing an image of the outside world. The imaging device 26 also includes, for example, a lens, an imaging element, an amplifier, and an AD converter. Light collected through the lens is converted into an imaging signal, which is an analog signal, by the imaging element. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging information, which is a digital signal. The converted imaging information is supplied to the processing device 21. The imaging information supplied to the processing device 21 is transmitted to the terminal device 10 via the communication device 27.

[0029] The communication device 27 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 27 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 27 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 27 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0030] The display 28 is a device that displays images. The display 28 displays various images under the control of the processing device 21. As described above, the display 28 includes the lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as the lens 41R, a display panel for the right eye, and an optical member for the right eye. As the display panel, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used.

[0031] The processing device 21 functions as an acquisition unit 211 and a display control unit 212, for example, by reading out a control program PR2 from the storage device 22 and executing it.

[0032] The acquisition unit 211 acquires image information indicating an image to be displayed on the XR glasses 20 from the terminal device 10.

[0033] Furthermore, the acquisition unit 211 acquires gaze information input from the gaze detection device 23, position information input from the GPS device 24, posture information and movement information input from the movement detection device 25, and imaging information input from the imaging device 26. Then, the acquisition unit 211 outputs the acquired gaze information, position information, posture information, movement information, and imaging information to the communication device 27.

[0034] Based on the image information acquired by the acquisition unit 211 from the terminal device 10, the display control unit 212 causes the display 28 to display an image indicated by the image information.

[0035] 1-1-3: Terminal device configuration 4 is a block diagram showing an example of the configuration of the terminal device 10. The terminal device 10 includes a processing device 11, a storage device 12, a communication device 13, a display 14, an input device 15, and an inertial sensor 16. The elements of the terminal device 10 are connected to each other by one or more buses for communicating information. Note that the following description will basically be given assuming that the terminal device 10 is a terminal device 10-1 used by a user U1.

[0036] The processing device 11 is a processor that controls the entire terminal device 10. The processing device 11 is configured, for example, using one or more chips. The processing device 11 is configured, for example, using a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 11 may be realized by hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 11 executes various processes in parallel or sequentially.

[0037] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 also stores a plurality of programs including a control program PR1 executed by the processing device 11. The storage device 12 may also store image information indicating an image to be displayed on the XR glasses 20.

[0038] The communication device 13 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 13 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0039] The display 14 is a device that displays images and text information. The display 14 displays various images under the control of the processing device 11. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display 14. Note that when the XR glasses 20 are connected to the terminal device 10, the display 14 does not need to be an essential component. In this case, the XR glasses 20 will further have the same functions as the display 14.

[0040] The input device 15 receives operations from a user U1 wearing the XR glasses 20 on their head. For example, the input device 15 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 15 includes a touch panel, it may also serve as the display 14.

[0041] The inertial sensor 16 is a sensor that detects inertial force. The inertial sensor 16 includes, for example, one or more of an acceleration sensor, an angular velocity sensor, and a gyro sensor. The processing device 11 detects the attitude of the terminal device 10 based on the output information of the inertial sensor 16. Furthermore, the processing device 11 accepts the selection of a virtual object VO, the input of text, and the input of instructions in the spherical virtual space VS based on the attitude of the terminal device 10. For example, the user U1 operates the input device 15 while pointing the central axis of the terminal device 10 toward a predetermined area in the virtual space VS, thereby selecting a virtual object VO to be placed in the predetermined area. The operation by the user U1 on the input device 15 is, for example, a double tap. By operating the terminal device 10 in this way, the user U1 can select a virtual object VO without looking at the input device 15 of the terminal device 10.

[0042] In addition, when the XR glasses 20 are not connected to the terminal device 10, it is preferable that the terminal device 10 includes a GPS device similar to the GPS device 24 included in the XR glasses 20.

[0043] The processing device 11 reads out and executes the control program PR1 from the storage device 12, thereby functioning as an acquisition unit 111, a generation unit 112, a display control unit 113, a determination unit 114, a movement control unit 115, and a reception unit .

[0044] The acquisition unit 111 acquires notifications from the server 30. Here, when the terminal device 10 is the terminal device 10-1 used by the user U1, the "notification" includes a message from another user U to the user U1. The "message" includes, for example, an email and a message via an SNS app. The "notification" may also include a message from the user U1 to the other user U. The "notification" may also include a message from yet another user U to the other user U. The "notification" may also include a message, such as a warning or contact, from each application corresponding to the virtual object VO displayed in the virtual space VS. The "notification" may also include an advertisement aimed at an unspecified number of people.

[0045] The "notification" may be a text message or an image. If the "notification" is an image, the "notification" may be a still image or a video.

[0046] The acquisition unit 111 also acquires image information indicating an image to be displayed on the XR glasses 20 from the server 30 via the communication device 13. If image information indicating an image to be displayed on the XR glasses 20 is stored in the storage device 12, the acquisition unit 111 may acquire the image information from the storage device 12. Furthermore, the acquisition unit 111 acquires gaze information, position information, posture information, movement information, and imaging information from the XR glasses 20 via the communication device 13.

[0047] The generation unit 112 generates a virtual object VO corresponding to the notification acquired by the acquisition unit 111. More specifically, the generation unit 112 generates a virtual object VO corresponding to the notification acquired by the acquisition unit 111, using the image information acquired by the acquisition unit 111.

[0048] The display control unit 113 displays the virtual object VO generated by the generation unit 112 in the virtual space VS where the user U1 exists. Here, the user U1 is an example of a first user. Furthermore, the "virtual space VS where the user U1 exists" refers to the virtual space VS that can be perceived by the user U1 wearing the XR glasses 20.

[0049] FIG. 5A is an explanatory diagram illustrating an example of the operation of the generation unit 112 and the display control unit 113. In the following description, assume that the virtual space VS has an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other. As an example, the X-axis extends in the front-to-back direction of the user U1. Furthermore, as viewed from the user U1, the forward direction along the X-axis is defined as the X1 direction, and the backward direction along the X-axis is defined as the X2 direction. The Y-axis extends in the left-to-right direction of the user U1. Furthermore, as viewed from the user U1, the right direction along the Y-axis is defined as the Y1 direction, and the left direction along the Y-axis is defined as the X2 direction. These X-axis and Y-axis form a horizontal plane. Furthermore, the Z-axis is orthogonal to the XY plane and extends in the up-down direction of the user U1. Furthermore, as viewed from the user U1, the downward direction along the Z-axis is defined as the Z1 direction, and the upward direction along the Z-axis is defined as the Z2 direction.

[0050] 5A, in the virtual space VS, the user U1 is located at coordinates (x, y, z) = (0, 0, 0), which is the center of the virtual space VS. The generation unit 112 generates virtual objects VO1 to VO5 corresponding to the notification acquired by the acquisition unit 111. The display control unit 113 displays the virtual objects VO1 to VO5 in the virtual space VS. More specifically, the display control unit 113 calculates five coordinates in the virtual space VS, from the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z) = (x5, y5, z5) of the virtual object VO5. Furthermore, the display control unit 113 displays the virtual objects VO1 to VO5 at the positions indicated by the calculated coordinates. In FIG. 5A, D denotes the distance between the coordinates (x, y, z)=(0, 0, 0) of the user U1 and the coordinates (x, y, z)=(x1, y1, z1) of the virtual object VO1.

[0051] Here, the display control unit 113 may control the display form of each virtual object VO1 to VO5 depending on the destination of the notification corresponding to that virtual object. For example, the display control unit 113 may change the color of each virtual object VO1 to VO5 based on whether the destination of the notification corresponding to that virtual object is single or multiple. More specifically, the display control unit 113 may change at least one of the hue, brightness, and saturation of each virtual object VO1 to VO5 based on whether the destination of the notification corresponding to that virtual object is single or multiple. Alternatively, the display control unit 113 may change the transparency of each virtual object VO1 to VO5 based on whether the destination of the notification corresponding to that virtual object is single or multiple. Alternatively, the display control unit 113 may change the shape of each virtual object VO1 to VO5 based on whether the destination of the notification corresponding to that virtual object is single or multiple. Alternatively, the display control unit 113 may change the size of each of the virtual objects VO1 to VO5 in accordance with the number of notification destinations corresponding to each of the virtual objects VO1 to VO5. For example, the display control unit 113 may increase the size of each of the virtual objects VO1 to VO5 as the number of notification destinations corresponding to each of the virtual objects VO1 to VO5 increases. Increasing the size of the virtual object VO1 means increasing the volume of the virtual object VO1. Therefore, as long as the volume of the virtual object VO1 increases, the shape of the virtual object VO1 before and after the change in volume does not need to be similar.

[0052] As a result, the user U1 can distinguish the type of notification destination corresponding to the virtual object VO simply by visually recognizing the virtual object VO, which improves convenience for the user U1.

[0053] Here, the display control unit 113 may change the display form of each of the virtual objects VO1 to VO5 according to the amount of information included in the notification corresponding to that virtual object VO1 to VO5. For example, the larger the amount of information, the larger the size of the virtual object VO1 to VO5 may be.

[0054] As a result, the user U1 can grasp the amount of notification information corresponding to the virtual object VO simply by visually recognizing the virtual object VO, which improves convenience for the user U1.

[0055] Furthermore, the display control unit 113 may calculate five coordinates from the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z) = (x5, y5, z5) of the virtual object VO5 based on at least one of the time when the acquisition unit 111 acquired the notifications corresponding to the virtual objects VO1 to VO5 and the importance of the notifications corresponding to the virtual objects VO1 to VO5. Here, the "importance of the notification" may be derived, for example, by an importance setting unit (not shown) based on the sender of each notification. Alternatively, the "importance of the notification" may be derived by the importance setting unit based on the result of analyzing the title or content of each notification. For example, the display control unit 113 may display a virtual object VO closer to the user U1 for which the notification corresponding to each virtual object VO1 to VO5 was acquired earlier. Alternatively, the display control unit 113 may display a virtual object VO having a higher notification importance level corresponding to each of the virtual objects VO1 to VO5 at a position closer to the user U1.

[0056] As a result, the user U1 can, for example, preferentially check notifications that were received earlier and notifications with higher importance, which improves convenience for the user U1.

[0057] Furthermore, the display control unit 113 may display the virtual objects VO1 to VO5 at positions in the virtual space VS that correspond to locations in the real space associated with the destination of the notification acquired by the acquisition unit 111. For example, if the destination of the notification acquired by the acquisition unit 111 is the president of a company, the display control unit 113 may display the virtual object VO corresponding to the notification in the president's office of the company in a composite space in which the virtual space VS and the real space are combined.

[0058] As a result, user U1 can confirm that a notification addressed to him / her has arrived at a location that is highly relevant to him / her.

[0059] Furthermore, when the destinations of the notification acquired by the acquisition unit 111 include multiple users U, the display control unit 113 may display the virtual object VO at a position in the virtual space VS that corresponds to a location in the real space shared by the multiple users U. For example, when the destinations of the notification acquired by the acquisition unit 111 include all family members living in a house, the display control unit 113 may display the virtual object VO corresponding to the notification in the living room of the house that is shared by all family members.

[0060] As a result, each of the multiple users U can confirm that a notification has arrived in a place shared with the other users U, the notification address of which is the other users U in addition to the user U.

[0061] 5A, it is assumed, as an example, that the virtual objects VO1 to VO5 are spheres and that a text file indicating the content of the notification is stored inside the sphere. However, the shapes of the virtual objects VO1 to VO5 are not limited to spheres and may be any shape. Also, in FIG. 5A, five virtual objects VO are shown, from virtual object VO1 to virtual object VO5. However, the number of virtual objects VO is not limited to five and may be any number.

[0062] 4, when the user U1 is included in the destination information indicating one or more destinations of the notification acquired by the acquisition unit 111, the determination unit 114 determines whether the distance between the virtual object VO and the user U1 in the virtual space VS is within a predetermined distance. Note that the "predetermined distance" is an example of the first distance.

[0063] If the determination result by the determination unit 114 is positive, the movement control unit 115 moves the virtual object VO corresponding to the above notification closer to the user U1 in the virtual space VS. On the other hand, if the determination result by the determination unit 114 is negative, the movement control unit 115 does not move the virtual object VO corresponding to the above notification.

[0064] 5A and 5B are explanatory diagrams illustrating an example of the operation of the determination unit 114 and the movement control unit 115. At the time of FIG. 5A, it is assumed that the notification destinations corresponding to the virtual object VO1 include the user U1. Furthermore, it is assumed that the distance D between the coordinates (x, y, z) = (0, 0, 0) of the user U1 and the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 is within a predetermined distance. As a result, the determination result by the determination unit 114 is positive.

[0065] Since the determination result by the determination unit 114 is positive, in FIG. 5B, the movement control unit 115 changes the coordinates of the virtual object VO1 from (x, y, z)=(x1, y1, z1) to (x, y, z)=(x 1’ ,y 1’ ,z 1’ ), where (x,y,z)=(x 1’ ,y 1’ ,z 1’ ) is closer to the coordinates (x,y,z)=(0,0,0) of the user U1 compared to (x1,y1,z1). That is, the coordinates (x,y,z)=(0,0,0) of the user U1 and the new coordinates (x,y,z)=(x 1’ ,y 1’ ,z 1’ ) is shorter than the distance D.

[0066] 5A and 5B, the virtual objects VO2 to VO5 other than the virtual object VO1 either do not include the user U1 in the destinations of the corresponding notifications, or the distance between the coordinates of the user U1 and the coordinates of each of the virtual objects VO2 to VO5 exceeds the predetermined distance. In this case, the movement control unit 115 does not move the virtual objects VO2 to VO5.

[0067] As a result, even if the number of virtual objects VO increases according to the number of notifications, the terminal device 10 can prevent a decrease in convenience for the user U1. Specifically, when the destination information includes the user U1 and the distance between the virtual object VO corresponding to the notification and the user U1 in the virtual space VS is within a predetermined distance, the terminal device 10 moves the virtual object VO closer to the user U1. Therefore, the user U1 can check the virtual object VO corresponding to the notification for which the user U1 is included in the destination information preferentially over other virtual objects VO.

[0068] Furthermore, if the notification corresponding to the virtual object VO is an advertisement addressed to an unspecified number of users, the destination information includes user U1, but the movement control unit 115 may not move the virtual object VO even if the distance between user U1 and the virtual object VO is less than a predetermined distance.

[0069] Furthermore, when moving the virtual object VO1, the movement control unit 115 may move the virtual object VO1 in a first manner when destination information indicating one or more destinations of a notification corresponding to the virtual object VO1 includes only user U1, and may move the virtual object VO1 in a second manner different from the first manner when the destination information includes users U other than user U1. For example, the movement control unit 115 may move the virtual object VO1 at a faster speed when the destination information includes only user U1 compared to when the destination information includes users U other than user U1. Alternatively, the movement control unit 115 may move the virtual object VO1 while vibrating when the destination information includes only user U1, and may move the virtual object VO1 without vibrating when the destination information includes users U other than user U1.

[0070] As a result, the terminal device 10 can more effectively appeal to the user U1 about notifications addressed only to the user U1 than to other notifications. The user U1 can check notifications addressed only to himself / herself preferentially, which improves convenience for the user U1.

[0071] 4, the reception unit 116 receives an operation by the user U1 on the virtual object VO1. When the determination result of the determination unit 114 is positive as described above, the display control unit 113 displays the content of the notification corresponding to the virtual object VO1 in the virtual space VS in accordance with the operation received by the reception unit 116.

[0072] FIG. 5C is an explanatory diagram illustrating an example of the operation of the display control unit 113 and the reception unit 116. In FIG. 5C, it is assumed that the user U1 touches the virtual object VO1 in the virtual space VS. The reception unit 116 receives the touch of the user U1 on the virtual object VO1 as an operation of the user U1 on the virtual object VO1. As described above, since the determination result by the determination unit 114 is positive, the display control unit 113 displays, in response to the operation, a virtual object VO6, which is a text file containing the content of the notification corresponding to the virtual object VO1, at coordinates (x, y, z) = (x6, y6, z6) within the sphere. After displaying the virtual object VO6, the display control unit 113 causes the virtual object VO1 to disappear. Note that the display control unit 113 may display the virtual object VO6 after disappearing the virtual object VO1, or may simultaneously perform the disappearance of the virtual object VO1 and the display of the virtual object VO6. Note that the method of displaying the virtual object VO6, which is a text file, is not limited to the method of disappearing the sphere serving as the virtual object VO1. For example, the display control unit 113 may apply morphing to transform the virtual object VO1 into the virtual object VO6.

[0073] As a result, when the determination result of the determination unit 114 is positive, the user U1 can check the content of the notification for the virtual object VO. Furthermore, the user U1 can check the content of the notification for which he / she is included in the destination information in preference to other notifications.

[0074] Furthermore, the virtual object VO and the notification corresponding to the virtual object VO are not displayed simultaneously in the field of view of the user U1, so that the field of view of the user U1 is not unnecessarily narrowed, and the decrease in convenience for the user U1 is suppressed.

[0075] 5C, the reception unit 116 receives the user U1's contact with the virtual object VO1 as the user U1's operation on the virtual object VO1. However, the operation of the user U1 received by the reception unit 116 is not limited to contact with the virtual object VO1. For example, the user U1 may direct his / her gaze toward the virtual object VO1. More specifically, based on gaze information acquired by the terminal device 10 from the XR glasses 20, the reception unit 116 may receive, as the operation of the user U1, that the user U1's gaze is directed toward the virtual object VO1. In this case, when the determination result by the determination unit 114 is positive and the user U1's gaze is directed toward the virtual object VO1 for a predetermined period of time or longer, the display control unit 113 displays the content of the notification corresponding to the virtual object VO1 in the virtual space VS.

[0076] As a result, when the determination result of the determination unit 114 is positive, the user U1 can check the content of the notification corresponding to the virtual object VO simply by visually recognizing the virtual object VO for a predetermined period of time or more. Therefore, the user U1 can easily check the content of the notification corresponding to the virtual object VO.

[0077] 5A to 5C, for the sake of simplicity, the user U1 and the virtual objects VO1 to VO5 are displayed in two dimensions. However, in the actual virtual space VS, the user U1 and the virtual objects VO1 to VO5 are usually displayed in three dimensions. However, the method of displaying the user U1 and the virtual objects VO1 to VO5 in the actual virtual space VS is not limited to three-dimensional display, and may be two-dimensional display.

[0078] 1-1-4: Server configuration 6 is a block diagram showing an example of the configuration of the server 30. The server 30 includes a processing device 31, a storage device 32, a communication device 33, a display 34, and an input device 35. The elements of the server 30 are connected to each other by one or more buses for communicating information.

[0079] The processing device 31 is a processor that controls the entire server 30. The processing device 31 is configured, for example, using one or more chips. The processing device 31 is configured, for example, using a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Some or all of the functions of the processing device 31 may be realized by hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 31 executes various processes in parallel or sequentially.

[0080] The storage device 32 is a recording medium that can be read from and written to by the processing device 31. The storage device 32 also stores a plurality of programs including the control program PR3 executed by the processing device 31. The storage device 32 also stores image information that indicates the image to be displayed on the XR glasses 20.

[0081] The communication device 33 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 33 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 33 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 33 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0082] The display 34 is a device that displays images and text information. The display 34 displays various images under the control of the processing device 31. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used as the display 34.

[0083] The input device 35 is a device that accepts operations by an administrator of the information processing system 1. For example, the input device 35 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 35 includes a touch panel, it may also serve as the display 34.

[0084] The processing device 31 functions as an acquisition unit 311 and an output unit 312, for example, by reading out a control program PR3 from the storage device 32 and executing it.

[0085] The acquisition unit 311 acquires various data from the terminal device 10 via the communication device 33. The data includes, for example, data indicating the operation content for the virtual object VO, which is input to the terminal device 10 by the user U1 wearing the XR glasses 20 on his / her head.

[0086] The output unit 312 transmits a notification to the terminal device 10 via the communication device 33. In particular, when the terminal device 10 is the terminal device 10-1, the output unit 312 transmits to the terminal device 10 at least a notification addressed to the user U1.

[0087] Furthermore, the output unit 312 outputs image information indicating the image displayed on the XR glasses 20 to the terminal device 10 via the communication device 33. More specifically, the output unit 312 acquires the image information from the storage device 32. Furthermore, the output unit 312 outputs the acquired image information to the terminal device 10.

[0088] 1-2: Operation of the first embodiment 7 is a flowchart showing the operation of the terminal device 10 according to the first embodiment. Hereinafter, the operation of the terminal device 10 will be described with reference to FIG.

[0089] In step S1, the processing device 11 functions as an acquisition unit 111. The processing device 11 acquires a notification from the server 30.

[0090] In step S2, the processing device 11 functions as the generation unit 112. The processing device 11 generates a virtual object VO corresponding to the notification acquired in step S1.

[0091] In step S3, the processing device 11 functions as the display control unit 113. The processing device 11 displays the virtual object VO generated in step S2 in the virtual space VS in which the user U1 exists.

[0092] In step S4, the processing device 11 functions as the determination unit 114. The processing device 11 determines whether or not user U1 is included in the destination information indicating one or more destinations of the notification acquired in step S1. If user U1 is included in the destinations of the notification, that is, if the determination result of the processing device 11 is positive, the processing device 11 performs the operation of step S5. If user U1 is not included in the destination information, that is, if the determination result of the processing device 11 is negative, the processing device 11 performs the processing of step S1.

[0093] In step S5, the processing device 11 functions as the determination unit 114. The processing device 11 determines whether or not the distance D between the virtual object VO displayed in step S3 and the user U1 in the virtual space VS is within a predetermined distance. If the distance D is within the predetermined distance, that is, if the determination result of the processing device 11 is positive, the processing device 11 performs the operation of step S6. If the distance D exceeds the predetermined distance, that is, if the determination result of the processing device 11 is negative, the processing device 11 performs the processing of step S1.

[0094] In step S6, the processing device 11 functions as the movement control unit 115. The processing device 11 moves the virtual object VO closer to the user U1 in the virtual space VS.

[0095] In step S7, the processing device 11 functions as the reception unit 116. If the processing device 11 receives an operation by the user U1 on the virtual object VO, the processing device 11 executes the process of step S8. If the processing device 11 does not receive an operation by the user U1 on the virtual object VO, the processing device 11 executes the process of step S7.

[0096] In step S8, the processing device 11 functions as the display control unit 113. In response to the operation accepted in step S7, the processing device 11 displays the content of the notification corresponding to the virtual object VO in the virtual space VS. After that, the processing device 11 ends all the processes described in FIG.

[0097] 1-3: Effects of the First Embodiment According to the above description, the terminal device 10 as a display control device includes an acquisition unit 111, a generation unit 112, a display control unit 113, a determination unit 114, and a movement control unit 115. The acquisition unit 111 acquires a notification. The generation unit 112 generates a virtual object VO corresponding to the notification. The display control unit 113 displays the virtual object VO in a virtual space VS in which a user U1, who is a first user, exists. When the user U1 is included in destination information indicating one or more destinations of the notification, the determination unit 114 determines whether the distance D between the virtual object VO and the user U1 in the virtual space VS is within a predetermined distance. When the determination result of the determination unit 114 is positive, the movement control unit 115 moves the virtual object VO closer to the user U1, and when the determination result of the determination unit 114 is negative, the movement control unit 115 does not move the virtual object VO.

[0098] The terminal device 10 has the above configuration. When the terminal device 10 displays the virtual space VS including the virtual objects VO corresponding to notifications on the XR glasses 20, even if the number of the virtual objects VO increases in accordance with the number of notifications, the terminal device 10 can suppress a decrease in convenience for the user U1. Specifically, when the user U1 is included in the destinations of a message and the distance D between the virtual object VO corresponding to the notification and the user U1 in the virtual space VS is within a predetermined distance, the terminal device 10 moves the virtual object VO closer to the user U1. Therefore, the user U1 can preferentially check the virtual object VO corresponding to the notification for which the user U1 is included in the destinations, compared to other virtual objects VO. Furthermore, the terminal device 10 sets the condition for moving the virtual object VO closer to the user U1 as being that the distance D between the virtual object VO corresponding to the notification and the user U1 in the virtual space VS is within a predetermined distance. Therefore, a virtual object VO located farther than the predetermined distance from the user U1 does not approach the user U1. That is, a virtual object VO located farther than the predetermined distance does not suddenly move in front of the user U1, pushing aside a virtual object VO located near the user U1. Therefore, confusion does not occur in the field of view of the user U1.

[0099] According to the above description, the terminal device 10 further includes a reception unit 116. The reception unit 116 receives an operation by the user U1 on the virtual object VO. When the determination result of the determination unit 114 is positive, the display control unit 113 displays the content of a notification corresponding to the virtual object VO in the virtual space VS in response to the above operation.

[0100] Since the terminal device 10 has the above configuration, the user U1 can check the content of the notification for the virtual object VO when the determination result of the determination unit 114 is positive. Furthermore, the user U1 can check the content of the notification that includes the user U1 as a destination in preference to other notifications.

[0101] Furthermore, according to the above description, the operation on the virtual object VO may be directing the gaze toward the virtual object VO. In this case, when the determination result of the determination unit 114 is positive and the gaze is directed toward the virtual object VO for a predetermined period of time or longer, the display control unit 113 displays the content of the notification corresponding to the virtual object VO in the virtual space VS.

[0102] Since the terminal device 10 has the above configuration, when the determination result of the determination unit 114 is positive, the user U1 can check the content of the notification corresponding to the virtual object VO simply by visually checking the virtual object VO for a predetermined period of time or more. Therefore, the user U1 can easily check the content of the notification corresponding to the virtual object VO.

[0103] Furthermore, according to the above description, the display control unit 113 causes the content of the notification corresponding to the virtual object VO to be displayed in the virtual space VS, and then causes the virtual object VO to disappear in the virtual space VS.

[0104] The terminal device 10 has the above configuration, so that the virtual object VO and the notification corresponding to the virtual object VO are not displayed simultaneously in the field of view of the user U1. Therefore, an event in which the field of view of the user U1 is unnecessarily narrowed does not occur, and a decrease in convenience for the user U1 is suppressed.

[0105] Furthermore, according to the above description, the display control unit 113 controls the display form of the virtual object VO in accordance with the user U who is the destination of the notification.

[0106] Since the terminal device 10 has the above configuration, the user U1 can distinguish the type of notification destination corresponding to the virtual object VO simply by visually recognizing the virtual object VO, which improves convenience for the user U1.

[0107] Furthermore, according to the above explanation, when moving the virtual object VO, if the destination information includes only user U1, the movement control unit 115 moves the virtual object VO in a first manner. On the other hand, when moving the virtual object VO, if the destination information includes users U other than user U1, the movement control unit 115 moves the virtual object VO in a second manner.

[0108] The terminal device 10 has the above configuration, and therefore can more appeal to the user U1 with notifications addressed only to the user U1 than with other notifications. The user U1 can check notifications addressed only to himself / herself preferentially, which improves convenience for the user U1.

[0109] Furthermore, according to the above description, the display control unit 113 controls the position at which the virtual object VO is displayed in the virtual space VS based on at least one of the time when the acquisition unit 111 acquires the above notification and the importance of the above notification.

[0110] The terminal device 10 has the above configuration, so that the user U1 can, for example, preferentially check notifications that were received earlier and notifications with higher importance, thereby improving convenience for the user U1.

[0111] Furthermore, according to the above description, the display control unit 113 displays the virtual object VO at a position in the virtual space VS that corresponds to the location in real space associated with the destination of the notification.

[0112] The terminal device 10 has the above configuration, and thus can display a virtual object VO corresponding to a notification that includes the user U1 as a destination in a location that is highly relevant to the user U1. Therefore, the user U1 can check the notification addressed to him / her in a location that is highly relevant to him / her.

[0113] Furthermore, according to the above description, when the above destination information includes multiple users U, the display control unit 113 displays the virtual object VO at a position in the virtual space VS that corresponds to a location in the real space shared by the multiple users U.

[0114] The terminal device 10 has the above configuration, and thus displays, in a place shared by a plurality of users U, a virtual object VO corresponding to a notification that includes the plurality of users U as its destination. Therefore, each of the plurality of users U can check a notification that includes the other users U as its destination in addition to himself / herself in the place shared with the other users U.

[0115] 2: Second embodiment The configuration of an information processing system 1A including a terminal device 10A and a server 30A as a display control device according to a second embodiment of the present invention will be described below with reference to Figures 8 to 11. In the following description, for the purpose of simplifying the description, the same components of the information processing system 1A according to the second embodiment as those of the information processing system 1 according to the first embodiment will be designated by the same reference numerals, and their description may be omitted.

[0116] 2-1: Configuration of the second embodiment 2-1-1: Overall structure An information processing system 1A according to the second embodiment of the present invention differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10A instead of the terminal device 10 and a server 30A instead of the server 30. In other respects, the overall configuration of the information processing system 1A is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in FIG. 1, and therefore illustration and description thereof will be omitted. In the first embodiment, the terminal device 10 was an example of a display control device. However, in the second embodiment, the server 30A is an example of a display control device.

[0117] 2-1-2: Server configuration 8 is a block diagram showing an example of the configuration of the server 30A. The server 30A differs from the server 30 in that it includes a processing device 31A instead of the processing device 31 and a storage device 32A instead of the storage device 32.

[0118] The storage device 32A differs from the storage device 32 in that it stores a control program PR3A instead of the control program PR3.

[0119] The processing device 31A includes an acquisition unit 311A ​​instead of the acquisition unit 311 included in the processing device 31. Unlike the processing device 31, the processing device 31A does not require the output unit 312 as an essential component. Furthermore, in addition to the components included in the processing device 31, the processing device 31A includes a generation unit 313, a display control unit 314, a determination unit 315, and a movement control unit 316.

[0120] Similar to the acquisition unit 311, the acquisition unit 311A ​​acquires various data from the terminal device 10 via the communication device 33. The data includes, for example, data indicating the operation content for the virtual object VO, which is input to the terminal device 10 by the user U1 wearing the XR glasses 20 on his / her head.

[0121] The acquisition unit 311A ​​also acquires notifications, similar to the acquisition unit 111 included in the terminal device 10 according to the first embodiment. Here, the "notification" includes a message from another user U to at least one of the user U1 and the user U2. The "message" includes, for example, an email and a message via an SNS app. The "notification" may also include a message from one of the user U1 and the user U2 to the other user U. The "notification" may also include a message from yet another user U to the other user U. The "notification" may also include a message, such as a warning or contact, from each application corresponding to the virtual object VO displayed in the virtual space VS. The "notification" may also include an advertisement aimed at an unspecified number of people.

[0122] If the notification is stored in storage device 32A, acquisition unit 311A ​​acquires the notification from storage device 32A. Alternatively, acquisition unit 311A ​​may acquire the notification from another device different from server 30A, such as a mail server.

[0123] Furthermore, the acquisition unit 311A ​​acquires, from the terminal device 10A via the communication device 33, the operation of the user U1 on the virtual object VO1 accepted by the acceptance unit 116 of the terminal device 10A.

[0124] The generation unit 313 generates a virtual object VO corresponding to the notification acquired by the acquisition unit 311A, similar to the generation unit 112 included in the terminal device 10 according to the first embodiment. More specifically, the generation unit 313 generates a virtual object VO corresponding to the notification acquired by the acquisition unit 311A, using image information stored in the storage device 32A.

[0125] The display control unit 314 displays the virtual object VO generated by the generation unit 313 in each of the virtual space VS1 where the user U1 exists and the virtual space VS2 where the user U2 exists. It is assumed that the user U1 is located at the coordinates (x, y, z) = (0, 0, 0), which is the center of the virtual space VS1. It is also assumed that the user U2 is located at the coordinates (x, y, z) = (0, 0, 0), which is the center of the virtual space VS2. Here, the user U1 is an example of a first user. The user U2 is an example of a second user. The virtual space VS1 is an example of a first virtual space. The virtual space VS2 is an example of a second virtual space.

[0126] FIG. 9 is an explanatory diagram illustrating an example of the operation of the generation unit 313 and the display control unit 314. It is assumed in FIG. 9 that the user U2 is wearing XR glasses connected to the terminal device 10A-2 on his / her head. The display control unit 314 displays virtual objects VO1 to VO5 in the virtual space VS1, similar to the operation of the display control unit 113 included in the terminal device 10 according to the first embodiment illustrated in FIG. 5A. More specifically, the display control unit 314 outputs image information indicating images of the virtual objects VO1 to VO5 generated by the generation unit 313 to the terminal device 10A-1 via the communication device 33. The display control unit 314 also outputs control information including coordinate information indicating five coordinates, from the coordinates (x, y, z)=(x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z)=(x5, y5, z5) of the virtual object VO5, to the terminal device 10A-1 via the communication device 33. The display control unit 113A provided in the terminal device 10A-1, which will be described later, displays the virtual objects VO1 to VO5 in the virtual space VS1 based on image information indicating images of the virtual objects VO1 to VO5 acquired from the server 30A and coordinate information indicating five coordinates from the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z) = (x5, y5, z5) of the virtual object VO5.

[0127] 9, the display control unit 314 displays the virtual objects VO1 to VO5 in the virtual space VS2. More specifically, the display control unit 314 calculates the coordinates (x, y, z) of the virtual object VO1 in the virtual space VS2 as follows: 11 ,y 11 ,z 11 )~Coordinates of virtual object VO5 (x,y,z)=(x 15 ,y 15 ,z 15) of the virtual object VO1. The display control unit 113 then calculates the coordinates of the virtual object VO1 to VO5. The display control unit 113 then displays the virtual objects VO1 to VO5 at the positions indicated by the calculated coordinates. More specifically, the display control unit 314 transmits image information indicating images of the virtual objects VO1 to VO5 generated by the generation unit 313 to the terminal device 10A-2 via the communication device 33. The display control unit 314 also calculates the coordinates (x, y, z)=(x 11 ,y 11 ,z 11 )~Coordinates of virtual object VO5 (x,y,z)=(x 15 ,y 15 ,z 15 ) to the terminal device 10A-2 via the communication device 33. A display control unit 113A, which is provided in the terminal device 10A-2 and will be described later, transmits control information including coordinate information indicating the five coordinates of the virtual objects VO1 to VO5, acquired from the server 30A, together with image information indicating the images of the virtual objects VO1 to VO5 and the coordinates (x, y, z)=(x 11 ,y 11 ,z 11 )~Coordinates of virtual object VO5 (x,y,z)=(x 15 ,y 15 ,z 15 ) and the coordinate information indicating the five coordinates of the virtual objects VO1 to VO5, the virtual objects VO1 to VO5 are displayed in the virtual space VS2.

[0128] The coordinates of the virtual object VO1 in the virtual space VS2 are (x, y, z) = (x 11 ,y 11 ,z 11 ) may be the same as the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 in the virtual space VS1, or may be different from each other. Similarly, the coordinates (x, y, z) = (x 12 ,y 12 ,z 12 )~Coordinates of virtual object VO5 (x,y,z)=(x 15 ,y 15 ,z 15) may be the same as or different from the coordinates (x, y, z) = (x2, y2, z2) of virtual object VO2 to the coordinates (x, y, z) = (x5, y5, z5) of virtual object VO5 in virtual space VS1.

[0129] Returning to the explanation of Figure 8, the judgment unit 315, similar to the judgment unit 114 provided in the terminal device 10 according to the first embodiment, determines whether the distance D between the virtual object VO and the user U1 in the virtual space VS1 is within a predetermined distance when the user U1 is included in the destination information indicating one or more destinations of the notification acquired by the acquisition unit 311A.

[0130] Similar to the movement control unit 115 included in the terminal device 10 according to the first embodiment, when the determination result by the determination unit 315 is positive, the movement control unit 316 moves the virtual object VO corresponding to the above notification closer to the user U1 in the virtual space VS1. On the other hand, when the determination result by the determination unit 315 is negative, the movement control unit 316 does not move the virtual object VO corresponding to the above notification. Note that an example of the operation of the movement control unit 316 is similar to the example of the operation of the movement control unit 115 illustrated in FIG. 5B.

[0131] Furthermore, if the determination result by the determination unit 315 is positive, the display control unit 314 displays the content of the notification corresponding to the virtual object VO1 in the virtual space VS1 in accordance with the operation acquired by the acquisition unit 311A. For example, similar to the operation example of the display control unit 113 illustrated in FIG. 5C, the display control unit 314 displays, in the virtual space VS1, a virtual object VO6, which is a text file containing the content of the notification, as an example of the content of the notification corresponding to the virtual object VO1, at coordinates (x, y, z) = (x6, y6, z6) within the sphere. Furthermore, after displaying the virtual object VO6 in the virtual space VS1, the display control unit 314 causes the virtual object VO1 to disappear.

[0132] As a result, the server 30A can prevent a decrease in convenience for the user U1 even if the number of virtual objects VO increases according to the number of notifications. Specifically, when the user U1 is included in the destinations of a message and the distance between the virtual object VO corresponding to the notification and the user U1 in the virtual space VS1 is within a predetermined distance, the server 30A moves the virtual object VO closer to the user U1. Therefore, the user U1 can check the virtual object VO corresponding to the notification that includes the user U1 in the destinations preferentially over other virtual objects VO.

[0133] On the other hand, in the virtual space VS2, the display control unit 314 continues to display the virtual object VO1 until the content of the notification corresponding to the virtual object VO1, for example, the virtual object VO6, is displayed. That is, even after the virtual object VO1 disappears in the virtual space VS1, the display control unit 314 continues to display the virtual object VO1 in the virtual space VS2 until the user U2 confirms the content of the notification corresponding to the virtual object VO1 in the virtual space VS2.

[0134] As a result, even if the virtual object VO disappears in the virtual space VS1 when the user U1 checks the content of the notification, the content of the notification corresponding to the virtual object VO will not suddenly appear in the virtual space VS2 for the user U2, and therefore the user U2 will not be confused by the appearance.

[0135] The coordinates of the virtual object VO1 in the virtual space VS2 are (x, y, z) = (x 11 ,y 11 ,z 11 ) are the same as the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 in the virtual space VS1, the movement control unit 316 may move the virtual object VO1 in the virtual space VS2 along the same movement path as the virtual object VO1 in the virtual space VS1.

[0136] 2-1-3: Terminal device configuration 10 is a block diagram showing an example of the configuration of the terminal device 10 A. The terminal device 10 A differs from the terminal device 10 in that it includes a processing device 11 A instead of the processing device 11 and a storage device 12 A instead of the storage device 12.

[0137] The storage device 12A differs from the storage device 12 in that it stores a control program PR1A instead of the control program PR1.

[0138] The processing device 11A includes an acquisition unit 111A instead of the acquisition unit 111 included in the processing device 11, and a display control unit 113A instead of the display control unit 113. Unlike the processing device 11, the processing device 11A does not require the generation unit 112 as an essential component. Furthermore, the processing device 11A includes an output unit 117 in addition to the components included in the processing device 11.

[0139] The acquisition unit 111A, like the acquisition unit 111 provided in the terminal device 10 according to the first embodiment, acquires gaze information, position information, posture information, movement information, and imaging information from the XR glasses 20 via the communication device 13.

[0140] The acquisition unit 111A also acquires, from the server 30A, image information indicating an image to be displayed on the XR glasses 20. In particular, the acquisition unit 111A acquires, from the server 30A, image information indicating images of the virtual objects VO1 to VO5. Furthermore, the acquisition unit 111A acquires, from the server 30A, coordinate information indicating the coordinates of the virtual objects VO1 to VO5 in the virtual space VS. More specifically, when the terminal device 10A is the terminal device 10A-1, the acquisition unit 111A acquires, from the server 30A, coordinate information indicating five coordinates from the coordinates (x, y, z)=(x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z)=(x5, y5, z5) of the virtual object VO5. On the other hand, when the terminal device 10A is the terminal device 10A-2, the acquisition unit 111A acquires, from the server 30A, coordinate information indicating the coordinates (x, y, z)=(x 11 ,y 11 ,z 11 )~Coordinates of virtual object VO5 (x,y,z)=(x 15 ,y15 ,z 15 ) to obtain coordinate information indicating the five coordinates.

[0141] The display control unit 113A displays the virtual objects VO1 to VO5 in the virtual space VS based on image information indicating the images of the virtual objects VO1 to VO5 and coordinate information indicating the five coordinates up to the coordinates of the virtual objects VO1 to VO5, both of which have been acquired by the acquisition unit 111A. More specifically, when the terminal device 10A is the terminal device 10A-1, the display control unit 113A displays the virtual objects VO1 to VO5 in the virtual space VS1 based on image information indicating the images of the virtual objects VO1 to VO5 and coordinate information indicating the five coordinates from the coordinates (x, y, z)=(x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z)=(x5, y5, z5) of the virtual object VO5. On the other hand, when the terminal device 10A is the terminal device 10A-2, the display control unit 113A displays the virtual objects VO1 to VO5 in the virtual space VS1 based on image information indicating the images of the virtual objects VO1 to VO5 and coordinate information indicating the five coordinates from the coordinates (x, y, z)=(x1, y1, z1) of the virtual object VO1 to the coordinates (x, y, z)=(x5, y5, z5) of the virtual object VO5. 11 ,y 11 ,z 11 )~Coordinates of virtual object VO5 (x,y,z)=(x 15 ,y 15 ,z 15 ) and the coordinate information indicating the five coordinates of the virtual objects VO1 to VO5, the virtual objects VO1 to VO5 are displayed in the virtual space VS2.

[0142] The output unit 117 outputs the operation of the user U1 on the virtual object VO1 received by the receiving unit 116 to the server 30A via the communication device 13.

[0143] 2-2: Operation of the second embodiment 11 is a flowchart showing the operation of the server 30A according to the second embodiment. The operation of the server 30A will be described below with reference to FIG.

[0144] In step S11, the processing device 31A functions as an acquisition unit 311A. The processing device 31A acquires the notification.

[0145] In step S12, the processing device 31A functions as the generation unit 313. The processing device 31A generates a virtual object VO corresponding to the notification acquired in step S12.

[0146] In step S13, the processing device 31A functions as the display control unit 314. The processing device 31A displays the virtual object VO generated in step S12 in the virtual space VS1 where the user U1 exists and in the virtual space VS2 where the user U2 exists.

[0147] In step S14, the processing device 31A functions as the determination unit 315. The processing device 31A determines whether or not user U1 is included in the destination information indicating one or more destinations of the notification acquired in step S11. If user U1 is included in the destination information, that is, if the determination result of the processing device 31A is positive, the processing device 31A performs the operation of step S15. If user U1 is not included in the destination information, that is, if the determination result of the processing device 31A is negative, the processing device 31A performs the process of step S1.

[0148] In step S15, the processing device 31A functions as the determination unit 315. The processing device 31A determines whether or not the distance D between the virtual object VO displayed in step S13 and the user U1 in the virtual space VS1 is within a predetermined distance. If the distance D is within the predetermined distance, that is, if the determination result of the processing device 31A is positive, the processing device 31A performs the operation of step S16. If the distance D exceeds the predetermined distance, that is, if the determination result of the processing device 31A is negative, the processing device 31A performs the process of step S11.

[0149] In step S16, the processing device 31A functions as the movement control unit 316. The processing device 31A moves the virtual object VO within the virtual space VS1, thereby bringing the virtual object VO closer to the user U1.

[0150] In step S17, the processing device 31A functions as an acquisition unit 311A. When the processing device 31A acquires an operation by the user U1 on the virtual object VO in the virtual space VS1, the processing device 31A executes the process of step S18. When the processing device 31A does not accept an operation by the user U1 on the virtual object VO in the virtual space VS1, the processing device 31A executes the process of step S17.

[0151] In step S18, the processing device 31A functions as the display control unit 314. The processing device 31A displays the content of the notification corresponding to the virtual object VO in the virtual space VS1 in accordance with the operation acquired in step S17. Meanwhile, the processing device 31A continues to display the virtual object VO in the virtual space VS2 until the content of the notification corresponding to the virtual object VO is displayed in the virtual space VS2. Thereafter, the processing device 31A ends all of the processes described in FIG. 11.

[0152] 2-3: Effects of the second embodiment According to the above description, the server 30A as a display control device includes an acquisition unit 311A, a generation unit 313, a display control unit 314, a determination unit 315, and a movement control unit 316. The acquisition unit 311A ​​acquires a notification. The generation unit 313 generates a virtual object VO corresponding to the notification. The display control unit 314 displays the virtual object VO in a virtual space VS1 as a first virtual space in which a user U1 as a first user exists and in a virtual space VS2 as a second virtual space in which a user U2 as a second user exists. When the user U1 is included in destination information indicating one or more destinations of the notification, the determination unit 315 determines whether the distance D between the virtual object VO and the user U1 in the virtual space VS1 is within a predetermined distance. When the determination result of the determination unit 315 is positive, the movement control unit 316 moves the virtual object VO closer to the user U1 in the virtual space VS1. When the determination result of the determination unit 315 is negative, the movement control unit 316 does not move the virtual object VO in the virtual space VS1. After displaying the content of the notification corresponding to the virtual object VO in the virtual space VS1, the display control unit 314 causes the virtual object VO to disappear in the virtual space VS1. Meanwhile, the display control unit 314 continues to display the virtual object VO in the virtual space VS2 until displaying the content of the notification corresponding to the virtual object VO in the virtual space VS2.

[0153] The server 30A has the above-described configuration. When the server 30A causes the XR glasses 20 to display a virtual space VS including virtual objects VO corresponding to notifications, even if the number of the virtual objects VO increases in accordance with the number of notifications, the server 30A can prevent a decrease in convenience for the user U1. Specifically, when the destination information includes the user U1 and the distance D between the virtual object VO corresponding to the notification and the user U1 in the virtual space VS1 is within a predetermined distance, the terminal device 10 moves the virtual object VO closer to the user U1. Therefore, the user U1 can preferentially check the virtual object VO corresponding to the notification included in the destination information for the user U1, compared to other virtual objects VO. Furthermore, the terminal device 10 moves the virtual object VO closer to the user U1 only when the distance D between the virtual object VO corresponding to the notification and the user U1 is within a predetermined distance. Therefore, a virtual object VO located farther than the predetermined distance from the user U1 does not approach the user U1. That is, a virtual object VO located farther than the predetermined distance does not suddenly move in front of the user U1, pushing aside a virtual object VO located near the user U1. Therefore, confusion does not occur in the field of view of the user U1.

[0154] Furthermore, even if the server 30A causes the virtual object VO to disappear in the virtual space VS1, the server 30A does not cause the virtual object VO to disappear until it displays the content of the notification corresponding to the virtual object VO in the virtual space VS2. Therefore, even if the virtual object VO disappears in the virtual space VS1 after the user U1 checks the content of the notification, the content of the notification corresponding to the virtual object VO will not suddenly appear in the virtual space VS2 for the user U2. Therefore, the user U2 will not be confused by the appearance.

[0155] 3: Variation The present disclosure is not limited to the above-described exemplary embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected arbitrarily from the following examples may be combined.

[0156] 3-1: Variation 1 In the information processing system 1 according to the first embodiment, the terminal device 10 and the XR glasses 20 are realized as separate entities. However, the method for realizing the terminal device 10 and the XR glasses 20 in the embodiment of the present invention is not limited to this. For example, the XR glasses 20 may have the same functions as the terminal device 10, and the terminal device 10 and the XR glasses 20 may be realized in a single housing. The same applies to the information processing system 1A according to the second embodiment.

[0157] 3-2: Variation 2 The information processing system 1 according to the first embodiment includes XR glasses 20. In the above description, the XR glasses 20 are MR glasses as an example. However, the XR glasses 20 in the information processing system 1 may not be MR glasses, but may be any one of a head-mounted display (HMD) employing virtual reality (VR) technology, VR glasses employing VR technology, an HMD employing augmented reality (AR) technology, AR glasses employing AR technology, or an HMD employing MR technology. Alternatively, the information processing system 1 may include any one of a regular smartphone and a tablet equipped with an imaging device, instead of the XR glasses 20 as MR glasses. These HMDs, VR glasses, AR glasses, smartphones, and tablets are examples of display devices. The same applies to the information processing system 1A according to the second embodiment.

[0158] 4:Other (1) In the above-described embodiment, ROM and RAM are exemplified as storage devices 12-12A, storage device 22, and storage devices 32-32A. However, other suitable storage media may also be used, such as flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, and other suitable storage media. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communications network NET via a telecommunications line.

[0159] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0160] (3) In the above-described embodiment, input and output information may be stored in a specific location (for example, a memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be transmitted to another device.

[0161] (4) In the above-described embodiment, the determination may be made by a value (0 or 1) represented using one bit, by a Boolean value (true or false), or by a comparison of numerical values ​​(e.g., comparison with a predetermined value).

[0162] (5) The order of the process procedures, sequences, flowcharts, etc. illustrated in the above-described embodiments may be rearranged unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0163] (6) Each function illustrated in Figures 1 to 11 is realized by any combination of at least one of hardware and software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically coupled, or may be realized by using two or more devices that are physically or logically separated and connected directly or indirectly (for example, by wire, wirelessly, etc.). A functional block may also be realized by combining software with the one device or the multiple devices.

[0164] (7) The programs exemplified in the above-described embodiments should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0165] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0166] (8) In each of the foregoing embodiments, the terms "system" and "network" are used interchangeably.

[0167] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0168] (10) In the above-described embodiments, the terminal devices 10-10A and the servers 30-30A may be mobile stations (MS). A mobile station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate term. In addition, in the present disclosure, terms such as "mobile station," "user terminal," "user equipment (UE)," and "terminal" may be used interchangeably.

[0169] (11) In the above-described embodiments, the terms "connected," "coupled," or any variations thereof refer to any direct or indirect connection or coupling between two or more elements, including the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements are considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0170] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0171] (13) As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judgment" or "decision." In other words, "judgment" and "decision" can include regarding some action as having been "judgment" or "decision." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0172] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or" as used in this disclosure is not intended to be an exclusive or.

[0173] (15) In this disclosure, where articles are added by translation, such as a, an, and the in English, this disclosure may include that the nouns following these articles are plural.

[0174] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0175] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Notification of predetermined information (e.g., notification that "X is true") is not limited to explicit notification, but may be implicit (e.g., not notifying the predetermined information).

[0176] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure. [Explanation of symbols]

[0177] 1, 1A... Information processing system, 10, 10A... Terminal device, 11, 11A... Processing device, 12, 12A... Storage device, 13... Communication device, 14... Display, 15... Input device, 16... Inertial sensor, 20... XR glasses, 21... Processing device, 22... Storage device, 23... Gaze detection device, 24... GPS device, 25... Motion detection device, 26... Imaging device, 27... Communication device, 28... Display, 30, 30A... and server, 31, 31A... Processing device, 32, 32A... Storage device, 33... Communication device, 34... Display, 35... Input device, 41L, 4 1R...lens, 91, 92...temples, 93...bridge, 94...frame, 95...frame, 111, 111A...acquisition unit, 112...generation unit, 113, 113A...display control unit, 114...determination unit, 115...movement control unit, 116...reception unit, 117...output unit, 311, 311A...acquisition unit, 312...output unit, 313...generation unit, 314...display control unit, 315...determination unit, 316...movement control unit, PR1 to PR3A...control program, U, U1 to U2...user, VO, VO1 to VO6...virtual object, VS, VS1, VS2...virtual space

Claims

1. an acquisition unit that acquires the notification; a generation unit that generates a virtual object corresponding to the notification; a display control unit that displays the virtual object in a virtual space in which a first user exists; a determination unit that determines whether a distance between the virtual object and the first user in the virtual space is within a first distance when the first user is included in destination information indicating one or more destinations of the notification; a movement control unit that moves the virtual object closer to the first user when the determination result of the determination unit is positive, and does not move the virtual object when the determination result of the determination unit is negative; A display control device comprising:

2. a receiving unit that receives a user's operation on the virtual object, when the determination result of the determination unit is affirmative, the display control unit displays, in the virtual space, content of the notification corresponding to the virtual object in response to the operation. The display control device according to claim 1 .

3. the operation on the virtual object is directing a line of sight toward the virtual object, the display control unit, when the determination result of the determination unit is positive and the line of sight is directed to the virtual object for a predetermined period of time or more, causes the content of the notification corresponding to the virtual object to be displayed in the virtual space. The display control device according to claim 2 .

4. The display control device according to claim 2 , wherein the display control unit causes the content of the notification corresponding to the virtual object to be displayed in the virtual space, and then causes the virtual object to disappear in the virtual space.

5. The display control device according to claim 1 , wherein the display control unit controls a display form of the virtual object depending on a user who is a destination of the notification.

6. The movement control unit When the destination information includes only the first user, the virtual object is moved in a first manner; When the destination information includes a user other than the first user, the virtual object is moved in a second manner. The display control device according to claim 1 , comprising:

7. 4. The display control device according to claim 1, wherein the display control unit controls a position at which the virtual object is displayed in the virtual space based on at least one of a time when the acquisition unit acquires the notification and an importance of the notification.

8. The display control device according to claim 1 , wherein the display control unit displays the virtual object at a position in the virtual space that corresponds to a location in real space associated with a destination of the notification.

9. 4. The display control device according to claim 1, wherein, when the destination information includes multiple users, the display control unit displays the virtual object at a position in the virtual space that corresponds to a location in real space shared by the multiple users.

10. an acquisition unit that acquires the notification; a generation unit that generates a virtual object corresponding to the notification; a display control unit that displays the virtual object in a first virtual space in which a first user exists and in a second virtual space in which a second user exists; a determination unit that determines whether a distance between the virtual object and the first user in the first virtual space is within a predetermined distance when the first user is included in destination information indicating one or more destinations of the notification; a movement control unit that moves the virtual object closer to the first user in the first virtual space when the determination result of the determination unit is positive, and does not move the virtual object in the first virtual space when the determination result of the determination unit is negative, After displaying the content of the notification corresponding to the virtual object in the first virtual space, the display control unit causes the virtual object to disappear in the first virtual space, while continuing to display the virtual object in the second virtual space until the content of the notification corresponding to the virtual object is displayed in the second virtual space.

Images