Indicating a control device and a server

Abstract

This display control device comprises an acquiring unit for acquiring a plurality of messages, a generating unit for generating a plurality of individual objects corresponding on a one-to-one basis to the plurality of messages, and a display control unit for causing a display device to display a virtual object, which is a collection of the plurality of individual objects, wherein the display control unit increases a size of the virtual object and increases a distance from a center of a virtual space to a center of the virtual object in accordance with an increase in the number of the plurality of messages.

Description

Technical Field

[0001] The present invention relates to a display control device and a server. In particular, the present invention relates to a display control device and a server that cause a display device to display virtual objects corresponding to messages.

Background Art

[0002] With XR technology including VR (Virtual Reality) technology and AR (Augmented Reality) technology, messages indicated by virtual objects are displayed in a virtual space displayed on XR glasses worn on the user's head.

[0003] For example, Patent Document 1 discloses a technique related to a method and apparatus for an interactive virtual environment for communication. Specifically, Patent Document 1 discloses a technique for displaying a virtual object indicating a "scribbled message" in a virtual space where information can be exchanged between users.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When displaying a number of virtual objects corresponding to the number of messages in a virtual space, as the number of messages increases, the number of virtual objects also increases. For this reason, the virtual space displayed on the XR glasses worn on the user's head may be filled with a plurality of virtual objects. That is, in the conventional technology, there is a problem that the convenience is reduced because the user's view is obstructed.

[0006] Therefore, an object of the present invention is to provide a display control device that can suppress a decrease in user convenience even when the number of messages increases when displaying virtual objects corresponding to the number of messages in a virtual space.

Means for Solving the Problems

[0007] A display control device according to a preferred aspect of the present invention is a display control device that causes a display device worn on a user's head to display a virtual space including virtual objects, and includes an acquisition unit that acquires a plurality of messages, a generation unit that generates a plurality of individual objects corresponding one-to-one to the plurality of messages, and a display control unit that causes the display device to display a virtual object that is an aggregate of the plurality of individual objects. The display control unit sets the size of the virtual object to a first size when the number of the plurality of messages is a first number, and sets the distance from the center of the virtual space to the center of the virtual object in the virtual space to a first distance. When the number of the plurality of messages is a second number greater than the first number, the display control unit sets the size of the virtual object to a second size greater than the first size, and sets the distance from the center of the virtual space to the center of the virtual object in the virtual space to a second distance longer than the first distance.

Effects of the Invention

[0008] According to the present invention, when displaying virtual objects corresponding to the number of messages in a virtual space, it is possible to suppress a decrease in user convenience even when the number of messages increases.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Modes for Carrying Out the Invention

[0010] 1: First Embodiment Hereinafter, with reference to FIGS. 1 to 10, the configuration of the information processing system 1 including the terminal device 10 as a display control device according to the first embodiment of the present invention will be described.

[0011] 1-1: Configuration of the First Embodiment 1-1-1: Overall Configuration FIG. 1 is a diagram showing the overall configuration of the information processing system 1 according to the first embodiment of the present invention. The information processing system 1 is a system that provides a virtual space to the user U1 wearing the XR glasses 20 described later using XR technology.

[0012] The information processing system 1 includes a terminal device 10, XR glasses 20, and a server 30. The terminal device 10 is an example of a display control device. In the information processing system 1, the terminal device 10 and the server 30 are communicably connected to each other via a communication network NET. Also, the terminal device 10 and the XR glasses 20 are communicably connected to each other. In the following description, when distinguishing between the terminal device 10 and the XR glasses 20 used for each user, a subscript “-X” is used in the reference numeral. X is an arbitrary integer of 1 or more. The same applies to the components of the terminal device 10 and the XR glasses 20. In FIG. 1, the terminal device 10-1 and the XR glasses 20 are communicably connected to each other. Also, in FIG. 1, two terminal devices 10 and one XR glasses 20 are shown, but these numbers are merely examples, and the information processing system 1 may include an arbitrary number of terminal devices 10 and XR glasses 20.

[0013] In FIG. 1, it is assumed that the user U1 uses a set of the terminal device 10-1 and the XR glasses 20. In the information processing system 1 shown in FIG. 1, a plurality of individual objects described later corresponding to a message addressed to the user U1 are displayed on the XR glasses 20. The message may include a message transmitted from the terminal device 10-1 to the terminal device 10-2. Also, the message may include a message transmitted from another terminal device (not shown in FIG. 1) to the terminal device 10-1. Further, the message may be a message generated by the terminal device 10-1 itself.

[0014] Server 30 provides various data and cloud services to terminal device 10 via communication network NET.

[0015] Terminal device 10-1 causes XR glasses 20 worn by the user on the head to display virtual objects arranged in a virtual space. The virtual space is, for example, a celestial sphere-shaped space. Also, the virtual objects are, for example, virtual objects indicating data such as still images, videos, 3D CG models, HTML files, and text files, and virtual objects indicating applications. Here, examples of the text file include a memo, source code, diary, and recipe. Also, examples of the application include a browser, an application for using SNS, and an application for generating a document file. Note that the terminal device 10 is preferably a portable terminal device such as a smartphone and a tablet. Note that the terminal device 10-1 is an example of a display control device.

[0016] Terminal device 10-2 is a device for user U2 to send a message to user U1. The terminal device 10-2 may display virtual objects arranged in a virtual space on a display 14 described later or XR glasses (not shown) connected to the terminal device 10-2. In particular, when the terminal device 10-2 is a device that displays virtual objects, the configuration of the terminal device 10-2 basically includes the same configuration as that of the terminal device 10-1. Also, similar to the terminal device 10-1, the terminal device 10-2 is preferably a portable terminal device such as a smartphone and a tablet.

[0017] The XR Glasses 20 are a see-through type of wearable display that is worn on the head of the user U1. The XR Glasses 20 display virtual objects on display panels provided on each of the lenses for both eyes by being controlled by the terminal device 10-1. Note that the XR Glasses 20 are an example of a display device. Further, in the following, an example in the case where the XR Glasses 20 are AR glasses will be described. However, the XR Glasses 20 being AR glasses is merely an example, and the XR Glasses 20 may be VR glasses or MR (Mixed Reality) glasses.

[0018] 1-1-2: Configuration of XR Glasses FIG. 2 is a perspective view showing the appearance of the XR Glasses 20. As shown in FIG. 2, the appearance of the XR Glasses 20 has temples 91 and 92, a bridge 93, frames 94 and 95, and lenses 41L and 41R, similar to ordinary glasses.

[0019] An imaging device 26 is provided on the bridge 93. The imaging device 26 images the external world. Further, the imaging device 26 outputs imaging information indicating the imaged image.

[0020] Each of the lenses 41L and 41R includes a half mirror. A liquid crystal panel or an organic EL panel for the left eye is provided on the frame 94. The liquid crystal panel or the organic EL panel is hereinafter collectively referred to as a display panel. An optical member for guiding the light emitted from the display panel for the left eye to the lens 41L is provided on the frame 94. The half mirror provided on the lens 41L transmits the light of the external world and guides it to the left eye, and reflects the light guided by the optical member and makes it incident on the left eye. A display panel for the right eye and an optical member for guiding the light emitted from the display panel for the right eye to the lens 41R are provided on the frame 95. The half mirror provided on the lens 41R transmits the light of the external world and guides it to the right eye, and reflects the light guided by the optical member and makes it incident on the right eye.

[0021] The display 28 described below includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as a lens 41R, a display panel for the right eye, and an optical member for the right eye.

[0022] In the above configuration, the user U1 can observe the image displayed on the display panel in a see-through state where it is superimposed on the state of the external world. Also, in the XR glasses 20, by displaying the left-eye image among the binocular images with parallax on the display panel for the left eye and the right-eye image on the display panel for the right eye, it becomes possible to make the user U1 perceive the displayed image as if it has depth and a three-dimensional effect.

[0023] FIG. 3 is a block diagram showing a configuration example of the XR glasses 20. The XR glasses 20 include a processing device 21, a storage device 22, a gaze detection device 23, a GPS device 24, a motion detection device 25, an imaging device 26, a communication device 27, and a display 28. Each element included in the XR glasses 20 is interconnected by one or a plurality of buses for communicating information. Note that the term "device" in this specification may be read as other terms such as a circuit, a device, or a unit.

[0024] The processing device 21 is a processor that controls the entire XR glasses 20. The processing device 21 is configured using, for example, one or a plurality of chips. Also, the processing device 21 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and registers. Note that part or all of the functions of the processing device 21 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The processing device 21 executes various processes in parallel or sequentially.

[0025] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 stores a plurality of programs including the control program PR1 executed by the processing device 21.

[0026] The gaze detection device 23 detects the gaze of the user U1 and generates gaze information indicating the detection result. Any method may be used for the gaze detection by the gaze detection device 23. For example, the gaze detection device 23 may detect the gaze information based on the position of the head and the position of the iris. The gaze information indicates the direction of the gaze of the user U1. The gaze detection device 23 supplies the gaze information to the processing device 21 described later. The gaze information supplied to the processing device 21 is transmitted to the terminal device 10 via the communication device 27.

[0027] The GPS device 24 receives radio waves from a plurality of satellites. The GPS device 24 generates position information from the received radio waves. The position information indicates the position of the XR glasses 20. The position information may be in any format as long as the position can be specified. For example, the position information indicates the latitude and longitude of the XR glasses 20. As an example, the position information is obtained from the GPS device 24. However, the XR glasses 20 may obtain the position information by any method. The acquired position information is supplied to the processing device 21. The position information output to the processing device 21 is transmitted to the terminal device 10 via the communication device 27.

[0028] The motion detection device 25 detects the motion of the XR glasses 20. Examples of the motion detection device 25 include inertial sensors such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects the acceleration in the orthogonal X-axis, Y-axis, and Z-axis directions. The gyro sensor detects the angular acceleration with the X-axis, Y-axis, and Z-axis as the central axes of rotation. The motion detection device 25 can generate attitude information indicating the attitude of the XR glasses 20 based on the output information of the gyro sensor. The motion information includes acceleration data indicating the acceleration in each of the three axes and angular acceleration data indicating the angular acceleration in each of the three axes. Further, the motion detection device 25 supplies the attitude information indicating the attitude of the XR glasses 20 and the motion information related to the motion of the XR glasses 20 to the processing device 21. The supplied attitude information and motion information are transmitted to the terminal device 10 via the communication device 27.

[0029] The imaging device 26 outputs imaging information obtained by imaging the external world. Further, the imaging device 26 includes, for example, a lens, an image sensor, an amplifier, and an AD converter. The light collected through the lens is converted into an imaging signal, which is an analog signal, by the image sensor. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging information, which is a digital signal. The converted imaging information is supplied to the processing device 21. The supplied imaging information is transmitted to the terminal device 10 via the communication device 27.

[0030] The communication device 27 is hardware as a transceiver device for communicating with other devices. Further, the communication device 27 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 27 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 27 may include a wireless communication interface. Examples of the connector and the interface circuit for wired connection include products compliant with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products compliant with wireless LAN and Bluetooth (registered trademark).

[0031] The display 28 is a device that displays images. The display 28 displays various images under the control of the processing device 21. As described above, the display 28 includes the lens 41L, the display panel for the left eye, and the optical member for the left eye, as well as the lens 41R, the display panel for the right eye, and the optical member for the right eye. As the display panel, for example, various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used.

[0032] The processing device 21 functions as an acquisition unit 211 and a display control unit 212, for example, by reading and executing the control program PR1 from the storage device 22.

[0033] The acquisition unit 211 acquires image information indicating an image to be displayed on the XR glasses 20 from the terminal device 10-1.

[0034] In addition, the acquisition unit 211 acquires gaze information supplied from the gaze detection device 23, position information supplied from the GPS device 24, attitude information and motion information supplied from the motion detection device 25, and imaging information supplied from the imaging device 26. Then, the acquisition unit 211 supplies the acquired gaze information, position information, attitude information, motion information, and imaging information to the communication device 27.

[0035] The display control unit 212 causes the display 28 to display an image indicated by the image information based on the image information acquired from the terminal device 10-1 by the acquisition unit 211.

[0036] 1-1-3: Configuration of the Terminal Device FIG. 4 is a block diagram showing a configuration example of the terminal device 10. The terminal device 10 includes a processing device 11, a storage device 12, a communication device 13, a display 14, an input device 15, and an inertial sensor 16. Each element included in the terminal device 10 is interconnected by one or a plurality of buses for communicating information. Hereinafter, basically, the configuration of the terminal device 10-1 will be described as the configuration of the terminal device 10.

[0037] The processing device 11 is a processor that controls the entire terminal device 10. Further, the processing device 11 is configured using, for example, one or more chips. The processing device 11 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, registers, and the like. Note that part or all of the functions of the processing device 11 may be realized by hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 11 executes various processes in parallel or sequentially.

[0038] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. Further, the storage device 12 stores a plurality of programs including a control program PR2 executed by the processing device 11.

[0039] The communication device 13 is hardware as a transmission / reception device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, and a communication module. The communication device 13 may include a connector for a wired connection and an interface circuit corresponding to the connector. Further, the communication device 13 may include a wireless communication interface. Examples of the connector and the interface circuit for a wired connection include products compliant with a wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products compliant with a wireless LAN and Bluetooth (registered trademark).

[0040] The display 14 is a device that displays image and character information. The display 14 displays various images under the control of the processing device 11. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display 14. Note that when the XR glasses 20 are connected to the terminal device 10, the display 14 may not be an essential component. In this case, the XR glasses 20 will further have the same function as the display 14.

[0041] The input device 15 receives operations from the user U1 who wears the XR glasses 20 on the head. For example, the input device 15 is configured to include a pointing device such as a keyboard, a touch pad, a touch panel, or a mouse. Here, when the input device 15 includes a touch panel, it may also serve as the display 14.

[0042] The inertial sensor 16 is a sensor that detects inertial forces. The inertial sensor 16 includes, for example, one or more sensors among an acceleration sensor, an angular velocity sensor, and a gyro sensor. The processing device 11 detects the posture of the terminal device 10 based on the output information of the inertial sensor 16. Further, the processing device 11 receives the selection of the virtual object VO, the input of characters, and the input of instructions in the celestial sphere-shaped virtual space VS based on the posture of the terminal device 10. For example, when the user U1 operates the input device 15 with the central axis of the terminal device 10 directed toward a predetermined area in the virtual space VS, the virtual object VO arranged in the predetermined area is selected. The operation of the user U1 on the input device 15 is, for example, a double tap. In this way, the user U1 can select the virtual object VO without looking at the input device 15 of the terminal device 10 by operating the terminal device 10.

[0043] When the XR glasses 20 are not connected to the terminal device 10, it is preferable that the terminal device 10 be provided with a GPS device similar to the GPS device 24 provided in the XR glasses 20.

[0044] The processing device 11 functions as an output unit 111, an acquisition unit 112, a generation unit 113, and a display control unit 114 by reading and executing the control program PR2 from the storage device 12.

[0045] The output unit 111 outputs a message created by the user U1 using the input device 15 to the server 30. In the message, the sender of the message, the recipient of the message, and the content of the message are specified. The content of the message includes at least one of text and an image.

[0046] Furthermore, when the terminal device 10 is the terminal device 10-1 used by the user U1 who is the first user, when the message addressed to the user U1 acquired by the acquisition unit 112-1 described later becomes read by the user U1, the output unit 111 transmits information indicating that it has been read to the server 30.

[0047] The acquisition unit 112 acquires a plurality of messages addressed to the user U from the server 30. When the terminal device 10 is the terminal device 10-1 used by the user U1 who is the first user, the acquisition unit 112-1 acquires a plurality of messages addressed to the user U1 from the server 30.

[0048] The generation unit 113 generates a plurality of individual objects corresponding one-to-one to the plurality of messages acquired by the acquisition unit 112.

[0049] The display control unit 114 causes the XR glasses 20 as a display device to display a virtual object that is an aggregate of the plurality of individual objects generated by the generation unit 113.

[0050] As a result, the user U1 can visually recognize the number of the plurality of messages.

[0051] FIGS. 5 and 6 are explanatory diagrams showing an example of the operations of the generation unit 113 and the display control unit 114. In the following description, in the virtual space VS, it is assumed that there are an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other. As an example, the X-axis extends in the front-rear direction of the user U1. Further, as viewed from the user U1, the forward direction along the X-axis is the X1 direction, and the rearward direction along the X-axis is the X2 direction. Also, the Y-axis extends in the left-right direction of the user U1. Further, as viewed from the user U1, the right direction along the Y-axis is the Y1 direction, and the left direction along the Y-axis is the X2 direction. The horizontal plane is formed by these X-axis and Y-axis. Also, the Z-axis is orthogonal to the XY plane and extends in the up-down direction of the user U1. Further, as viewed from the user U1, the downward direction along the Z-axis is the Z1 direction, and the upward direction along the Z-axis is the Z2 direction.

[0052] Referring to FIG. 5, in the virtual space VS, the user U1 is assumed to be located at coordinates (x, y, z) = (x u1 , y u1 , z u1 ). It is assumed that the coordinates of this user U1 are approximately at the center of the virtual space VS. The "approximate center" means a position that includes, in addition to the center of the virtual space VS, an area within a range where there is no problem in the operation of the information processing system 1 according to the present invention, starting from the center. The coordinates of the user U1 in the virtual space VS correspond to the position of the user U1 in the real space. Also, the position of the user U1 in the real space is indicated by the position information generated in the XR glasses 20 worn on the head of the user U1.

[0053] In the example shown in FIG. 5, the virtual object VO1 displayed at the coordinates (x, y, z) = (x1, y1, z1) by the display control unit 114 is an aggregate of individual objects SO1 to SO3, each of which corresponds one-to-one to a message addressed to the user U1. In FIG. 5, the virtual object VO1 and the individual objects SO1 to SO3 are spheres as an example. Also, the coordinates (x, y, z) = (x2, y2, z2) of the individual object SO1, the coordinates (x, y, z) = (x3, y3, z3) of the individual object SO2, and the coordinates (x, y, z) = (x4, y4, z4) of the individual object SO3 are all coordinates within the virtual object VO1. Also, each sphere as the individual objects SO1 to SO3 is entirely inside the sphere as the virtual object VO1. That is, the virtual object VO1 includes the same number of individual objects SO1 to SO3 as the number of messages addressed to the user U1.

[0054] Note that the image information used when the display control unit 114 causes the XR glasses 20 to display the individual objects SO1 to SO3 may be information stored in the storage device 12. Alternatively, the image information may be information acquired by the acquisition unit 112 from the server 30.

[0055] Also, the individual objects SO1 to SO3 may correspond one-to-one to all messages addressed to user U1. Alternatively, the individual objects SO1 to SO3 may correspond one-to-one only to unread messages among all messages addressed to user U1. Alternatively, one individual object may correspond to a plurality of read messages.

[0056] Also, in FIG. 5, the viewing angle when user U1 views the virtual object VO1 is θ1.

[0057] Returning to FIG. 4 for explanation, the display control unit 114 further increases the size of the virtual object VO1 as the number of the plurality of messages acquired by the acquisition unit 112 increases. That is, the display control unit 114 increases the size of the virtual object VO1 as the number of individual objects SO included in the virtual object VO1 increases. Also, the display control unit 114 arranges the virtual object VO1 at a position farther from the user U1 in the virtual space VS as the number of the plurality of messages increases. In other words, the display control unit 114 increases the distance from the center of the virtual space VS to the center of the virtual object VO1 as the number of the plurality of messages increases. That is, when the number of the plurality of messages is the first number, the display control unit 114 sets the size of the virtual object VO1 to the first size and sets the distance from the center of the virtual space VS to the center of the virtual object VO1 to the first distance. Also, when the number of the plurality of messages is a second number greater than the first number, the display control unit 114 sets the size of the virtual object VO1 to a second size greater than the first size and sets the distance from the center of the virtual space VS to the center of the virtual object VO1 to a second distance longer than the first distance.

[0058] In the example shown in FIG. 6, the virtual object VO2 displayed at the coordinates (x, y, z) = (x5, y5, z5) is an aggregate of individual objects SO1 to SO6, each of which corresponds one-to-one to a message addressed to the user U1. In FIG. 6, the shapes of the virtual object VO2 and the individual objects SO1 to SO6 are spheres as an example. The virtual object VO2 includes the same number of individual objects SO1 to SO6 as the number of messages addressed to the user U1.

[0059] Comparing the virtual object VO1 shown in FIG. 5 and the virtual object VO2 shown in FIG. 6, the virtual object VO2 includes more individual objects SO than the virtual object VO1. Also, the virtual object VO2 is larger than the virtual object VO1. Furthermore, the coordinates (x, y, z) = (x5, y5, z5) of the virtual object VO2 are farther from the coordinates (x, y, z) = (x1, y1, z1) of the virtual object VO1 as seen from the coordinates (x, y, z) = (x U1 , y U1 , z U1 ) of the user U1, which is approximately the center of the virtual space VS.

[0060] Also, in FIG. 6, when the user U1 views the virtual object VO2, the viewing angle is θ2. Comparing FIG. 5 and FIG. 6, it is preferable that the viewing angle θ1 when the user views the virtual object VO1 in FIG. 5 is equal to the viewing angle θ2 when the user views the virtual object VO2 in FIG. 6. When θ1 and θ2 are equal, in the view of the user U1, the ratio occupied by the virtual object VO1 is equal to the ratio occupied by the virtual object VO2. Even if the number of individual objects SO corresponding one-to-one to the message addressed to the user U1 increases, the area of the virtual object VO2 in the view of the user U1 remains equal to the area of the virtual object VO1, so the user U1 does not feel that the field of view has become narrower. However, when the number of individual objects SO increases, the area of each individual object SO in the view of the user U1 becomes smaller. Note that the viewing angle θ1 and the viewing angle θ2 do not necessarily have to be equal.

[0061] For the sake of convenience in explanation, different reference numerals are used for the virtual object VO1 and the virtual object VO2. However, as the number of messages addressed to the user U1, that is, the number of individual objects SO increases, the same virtual object VO1 may gradually become larger.

[0062] In addition, the display control unit 114 changes the display mode of the virtual object VO1 according to the number of a plurality of messages addressed to the user U1. For example, as the number of individual objects SO included in the virtual object VO1 increases, the display control unit 114 may change at least one of the color of the virtual object VO1 and the color of the individual objects SO included in the virtual object VO1. Alternatively, the display control unit 114 may change the shape of the virtual object VO1 as the number of individual objects SO increases. For example, every time the number of individual objects SO reaches a predetermined number, in the virtual object VO1, a plurality of individual objects SO may be arranged to represent a specific character. FIG. 7 is a diagram showing an example of the arrangement mode of a plurality of individual objects SO1 to SO10. In the example shown in FIG. 7, in the virtual object VO3, a plurality of individual objects SO1 to SO10 are arranged to represent the character "N".

[0063] As a result, when the number of a plurality of messages increases, the terminal device 10 can more effectively convey to the user U1 that the number of the plurality of messages has increased.

[0064] In addition, the display control unit 114 may make the display modes of the individual objects SO different from each other according to the transmission source devices corresponding to the plurality of messages addressed to the user U1. For example, the display control unit 114 may change at least one of the shape and color of the individual object SO according to the transmission source device of the message.

[0065] As a result, the user U1 can distinguish a plurality of individual objects SO according to the transmission source devices of the plurality of messages only by visually recognizing the plurality of individual objects SO.

[0066] 1-1-4: Server Configuration FIG. 8 is a block diagram showing a configuration example of the server 30. The server 30 includes a processing device 31, a storage device 32, a communication device 33, a display 34, and an input device 35. Each element included in the server 30 is interconnected by one or more buses for communicating information.

[0067] The processing device 31 is a processor that controls the entire server 30. Further, the processing device 31 is configured using, for example, one or more chips. The processing device 31 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and registers. Note that part or all of the functions of the processing device 31 may be realized by hardware such as a DSP, an ASIC, a PLD, or an FPGA. The processing device 31 executes various processes in parallel or sequentially.

[0068] The storage device 32 is a recording medium that can be read from and written to by the processing device 31. Further, the storage device 32 stores a plurality of programs including a control program PR3 executed by the processing device 31. Furthermore, the storage device 32 stores a message database MD in which information related to messages transmitted and received among a plurality of users U is stored.

[0069] FIG. 9 is a table showing an example of the message database MD. As will be described later, the acquisition unit 311 provided in the server 30 acquires messages transmitted and received between the terminal device 10 and the user U. More specifically, the acquisition unit 311 acquires information indicating the sender of the message output from the terminal device 10, information indicating the recipient of the message, and information indicating the content of the message. These pieces of information are stored in the message database MD. Further, the acquisition unit 311 acquires information indicating that each message has been read by the user U. The message management unit 313 described later adds a flag indicating whether each message has been read or not in the message database MD based on the said information. As an example, a flag with a value of "0" indicates that the message is unread. On the other hand, a flag with a value of "1" indicates that the message has been read. In the message database MD shown in FIG. 9, as an example, a message ID "1", the sender "U1" of the message with the message ID = 1, the recipient "U2", the content of the message, and a flag "0" indicating that it is unread are stored in a state where they are associated with each other. Note that in the table shown in FIG. 9, "n" is an integer of 2 or more.

[0070] Returning to the explanation of FIG. 8, the communication device 33 is hardware as a transmission / reception device for communicating with other devices. The communication device 33 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 33 may include a connector for a wired connection and an interface circuit corresponding to the said connector. Further, the communication device 33 may include a wireless communication interface. Examples of the connector for a wired connection and the interface circuit include products compliant with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products compliant with wireless LAN and Bluetooth (registered trademark), etc.

[0071] The display 34 is a device that displays images and character information. The display 34 displays various images under the control of the processing device 31. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display 34.

[0072] The input device 35 is a device that receives operations by the administrator of the information processing system 1. For example, the input device 35 includes a pointing device such as a keyboard, a touch pad, a touch panel, or a mouse. Here, when the input device 35 includes a touch panel, it may also serve as the display 34.

[0073] The processing device 31 functions as an acquisition unit 311, an output unit 312, and a message management unit 313, for example, by reading and executing the control program PR3 from the storage device 32.

[0074] The acquisition unit 311 acquires various data from the terminal device 10 via the communication device 33. As an example, the data includes data indicating the operation content for the virtual object VO input to the terminal device 10 by the user U1 wearing the XR glasses 20 on the head.

[0075] In particular, the acquisition unit 311 acquires messages transmitted and received among the users U. Further, as an example, in the terminal device 10-1, when a message addressed to the user U1 has been read by the user U1, the acquisition unit 311 acquires information indicating that it has been read.

[0076] The output unit 312 transmits image information indicating the image displayed on the XR glasses 20 to the terminal device 10. The image information may be stored in the storage device 32. Alternatively, the image information may be generated by a generation unit (not shown).

[0077] Also, as an example, the output unit 312 transmits a message addressed to the user U1 stored in the message database MD to the terminal device 10-1. The same applies to other terminal devices 10.

[0078] The message management unit 313 manages the message database MD. As an example, in the terminal device 10-1, when the acquisition unit 311 acquires information indicating that a message addressed to the user U1 has been read by the user U1, the message management unit 313 changes the flag "0" indicating that the message is unread, which is associated with the message, to the flag "1" indicating that the message is read.

[0079] 1-2: Operations of the First Embodiment FIG. 10 is a flowchart showing the operations of the terminal device 10 according to the first embodiment, particularly the terminal device 10-1 used by the user U1. Hereinafter, with reference to FIG. 10, the operations of the terminal device 10-1 will be described.

[0080] In step S1, the processing device 11-1 functions as the acquisition unit 112-1. The processing device 11-1 acquires a plurality of messages addressed to the user U1.

[0081] In step S2, the processing device 11-1 functions as the generation unit 113-1. The processing device 11-1 generates a plurality of individual objects SO that correspond one-to-one to the plurality of messages.

[0082] In step S3, the processing device 11-1 functions as a display control unit 114-1. The processing device 11-1 causes a virtual object VO, which is an aggregate of a plurality of individual objects SO, to be displayed on the XR glasses 20 as a display device. In particular, the processing device 11-1 increases the size of the virtual object VO as the number of the plurality of messages acquired in step S1 increases. Also, the processing device 11-1 arranges a virtual object VO1 at a position farther from the user U1 in the virtual space VS as the number of the plurality of messages acquired in step S1 increases. Thereafter, the processing device 11-1 executes the process of step S1.

[0083] 1-3: Effects of the First Embodiment According to the above description, the terminal device 10 as a display control device is a display control device that causes a virtual space VS including a virtual object VO to be displayed on the XR glasses 20 as a display device worn on the head. The terminal device 10 includes an acquisition unit 112, a generation unit 113, and a display control unit 114. The acquisition unit 112 acquires a plurality of messages. The generation unit 113 generates a plurality of individual objects SO that correspond one-to-one to the plurality of messages. The display control unit 114 causes a virtual object VO, which is an aggregate of the plurality of individual objects SO, to be displayed on the XR glasses 20 as a display device. Also, when the number of the plurality of messages is a first number, the display control unit 114 sets the size of the virtual object VO to a first size and sets the distance from the center of the virtual space VS to the center of the virtual object VO in the virtual space VS to a first distance. Further, when the number of the plurality of messages is a second number greater than the first number, the display control unit 114 sets the size of the virtual object VO to a second size greater than the first size and sets the distance from the center of the virtual space VS to the center of the virtual object VO in the virtual space VS to a second distance longer than the first distance.

[0084] Since the terminal device 10 has the above configuration, while displaying a virtual object VO corresponding to the number of messages in the virtual space VS, even if the number of messages increases, it is possible to suppress a decrease in user convenience. Specifically, in the virtual space VS, as the number of individual objects SO corresponding to the number of messages increases, the terminal device 10 increases the size of the virtual object VO, which is an aggregate of the individual objects SO, and displays the virtual object VO at a position farther from the user U1. Since the terminal device 10 has this configuration, in the display 28 provided in the XR glasses 20 worn by the user U1 on the head, even if the number of individual objects SO increases, the listability is ensured. As a result, a decrease in user convenience is suppressed.

[0085] Also, according to the above description, the display control unit 114 changes the display mode of the virtual object VO according to the number of a plurality of messages.

[0086] Since the terminal device 10 has the above configuration, when the number of a plurality of messages increases, it is possible to more strongly appeal to the user U1 that the number of the plurality of messages has increased.

[0087] Also, according to the above description, the display control unit 114 makes the display modes of the plurality of individual objects SO different from each other according to the transmission source devices corresponding to each of the plurality of messages.

[0088] Since the terminal device 10 has the above configuration, the user U1 can distinguish the plurality of individual objects SO according to the transmission source devices of the plurality of messages only by visually recognizing the plurality of individual objects SO.

[0089] 2: Second Embodiment Hereinafter, with reference to FIGS. 11 to 14, the configuration of an information processing system 1A including a terminal device 10A as a display control device according to the second embodiment of the present invention will be described.

[0090] 2-1: Configuration of the Second Embodiment 2-1-1: Overall Configuration The information processing system 1A according to the second embodiment of the present invention is different from the information processing system 1 according to the first embodiment in that it includes a terminal device 10A instead of the terminal device 10. In other respects, since the overall configuration of the information processing system 1A is the same as that of the information processing system 1 according to the first embodiment shown in FIG. 1, its illustration and description are omitted.

[0091] 2-1-2: Configuration of Terminal Device FIG. 11 is a block diagram showing a configuration example of the terminal device 10A. Different from the terminal device 10, the terminal device 10A includes a processing device 11A instead of the processing device 11 and a storage device 12A instead of the storage device 12.

[0092] Different from the storage device 12, the storage device 12A stores a control program PR2A instead of the control program PR2.

[0093] The processing device 11A includes a display control unit 114A instead of the display control unit 114 provided in the processing device 11. In addition to the components provided in the processing device 11, the processing device 11A also includes a reception unit 115 and a determination unit 116.

[0094] The reception unit 115 receives an operation from the user U1 for the virtual object VO. For example, the reception unit 115 may receive an operation using the input device 15 by the user U1. Alternatively, the reception unit 115 may receive an operation in which the user U1 contacts the virtual object VO in the virtual space VS. These operations in which the virtual object VO is selected in the virtual space VS are an example of the first operation.

[0095] The reception unit 115 may also receive an operation in which the user U1 selects one message from a list of a plurality of messages displayed in the virtual space VS, as will be described later. These plurality of messages and the plurality of individual objects SO correspond one-to-one. When the user U1 selects one message from the list of the plurality of messages, one individual object SO is selected. Alternatively, the reception unit 115 may receive an operation in which the user U1 touches one individual object SO in the virtual space VS. These operations in which one individual object SO is selected in the virtual space VS are examples of the second operation.

[0096] The display control unit 114A has the same function as the display control unit 114. Further, when the operation by the user U1 is the above-described first operation, the display control unit 114A causes a list of a plurality of messages to be displayed in the virtual space VS. On the other hand, when the operation by the user U1 is the above-described second operation, the display control unit 114A causes the content of the message corresponding to the one individual object SO specified by the second operation to be displayed in the virtual space VS.

[0097] FIGS. 12 and 13 are explanatory diagrams of operation examples of the display control unit 114A and the reception unit 115. In FIG. 12, when the user U1 performs an operation of selecting the virtual object VO2 using the terminal device 10-1, a list L of a plurality of messages is displayed in the virtual space VS by the display control unit 114A. More specifically, as shown in FIG. 12, a list of the titles of messages A to F is displayed as the list L in the virtual space VS by the display control unit 114A. Each of messages A to F corresponds one-to-one to individual objects SO1 to SO6. In FIG. 12, as viewed from the user U1, the list L is displayed on the left side of the virtual object VO2, but the display location is merely an example. The list L may be displayed at any location in the virtual space VS. In particular, it is preferable that the list L is displayed at a position that does not overlap with the virtual object VO2 as viewed from the user U1. The same applies when the user U1 performs an operation of touching the virtual object VO7 in the virtual space VS.

[0098] As a result, the user U1 can visually recognize in a list format a plurality of messages that correspond one-to-one to a plurality of individual objects SO included in the virtual object VO.

[0099] In FIG. 12, when the user U1 performs an operation of selecting one message from the plurality of messages shown in the list L, as shown in FIG. 13, a message object MO1 showing the content of the message is displayed. The message object MO1 includes at least one of text and an image. In the example shown in FIG. 13, it shows the case where the user U1 performs an operation of selecting "Message F" corresponding to the individual object SO6 from the plurality of messages shown in the list L. The same applies when the user U1 performs an operation of contacting the individual object SO6 in the virtual space VS.

[0100] As a result, the user U1 can visually recognize the specific content of each of the plurality of messages that correspond one-to-one to the plurality of individual objects SO included in the virtual object VO.

[0101] Returning to the explanation with reference to FIG. 11, the determination unit 116 determines the importance for each of the plurality of messages. For example, the determination unit 116 may determine the importance by analyzing the content of each of the plurality of messages. Alternatively, the determination unit 116 may determine the importance based on the source device corresponding to each of the plurality of messages. Alternatively, the determination unit 116 may determine the importance based on the operation of the user U1 using the input device 15. For example, as shown in FIG. 13, with the content of the message corresponding to one individual object SO being displayed, the user U1 determines the importance of the message and inputs the determination result using the input device 15. The determination unit 116 may determine the importance based on the input content from the input device 15.

[0102] The display control unit 114A causes an individual object SO corresponding to a message with an importance level equal to or higher than a predetermined value among a plurality of individual objects SO included in the virtual object VO to be displayed closer to the user U1, that is, closer to the approximate center of the virtual space VS, as compared to an individual object SO corresponding to a message with an importance level lower than the predetermined value. Further, within the virtual object VO, the display control unit 114A may cause an individual object SO with a higher importance level to be displayed closer to the user U1, that is, closer to the approximate center of the virtual space VS. Note that the above "predetermined value" is an example of the "first value".

[0103] As a result, the user U1 can preferentially check messages with a higher importance level for the user himself / herself.

[0104] 2-2: Operations of the Second Embodiment FIG. 14 is a flowchart showing the operations of the terminal device 10A according to the second embodiment, particularly the terminal device 10A-1 used by the user U1. Hereinafter, the operations of the terminal device 10A-1 will be described with reference to FIG. 14.

[0105] In step S11, the processing device 11A-1 functions as the acquisition unit 112-1. The processing device 11A-1 acquires a plurality of messages addressed to the user U1.

[0106] In step S12, the processing device 11A-1 functions as the generation unit 113-1. The processing device 11A-1 generates a plurality of individual objects SO that correspond one-to-one to the plurality of messages.

[0107] In step S13, the processing device 11A-1 functions as the display control unit 114A-1. The processing device 11A-1 causes a virtual object VO, which is an aggregate of a plurality of individual objects SO, to be displayed on the XR glasses 20 as a display device. The processing device 11A-1 increases the size of the virtual object VO as the number of the plurality of messages acquired in step S11 increases. Further, the processing device 11A-1 arranges the virtual object VO1 at a position farther from the user U1 in the virtual space VS as the number of the plurality of messages acquired in step S11 increases.

[0108] In step S14, the processing device 11A-1 functions as the reception unit 115. The processing device 11A-1 receives an operation from the user U1. When the operation from the user U1 is a first operation on the virtual object VO, the processing device 11A-1 executes the process of step S15. When the operation from the user U1 is a second operation on the individual object SO, the processing device 11A-1 executes the process of step S16.

[0109] In step S15, the processing device 11A-1 functions as the display control unit 114A. The processing device 11A-1 causes a list L of a plurality of messages to be displayed in the virtual space VS. Then, the processing device 11A-1 executes the process of step S14.

[0110] In step S16, the processing device 11A-1 functions as the display control unit 114A. The processing device 11A-1 causes the content of the message corresponding to one individual object SO to be displayed in the virtual space VS. Then, the processing device 11A-1 executes the process of step S11.

[0111] 2-3: Effects Achieved by the Second Embodiment According to the above description, the terminal device 10A as a display control device further includes a reception unit 115 that receives an operation on the virtual object VO. When the above operation is a first operation, the display control unit 114 causes a list L of a plurality of messages to be displayed in the virtual space VS.

[0112] Since the terminal device 10A has the above configuration, the user U1 can visually recognize a plurality of messages corresponding one-to-one to a plurality of individual objects SO included in the virtual object VO in a list format.

[0113] Also, according to the above description, in the terminal device 10A as a display control device, when the display control unit 114 is a second operation for designating one individual object SO among the plurality of individual objects SO, the content of the message corresponding to the one individual object SO is displayed in the virtual space VS.

[0114] Since the terminal device 10A has the above configuration, the user U1 can visually recognize the specific content of each of the plurality of messages corresponding one-to-one to the plurality of individual objects SO included in the virtual object VO.

[0115] Also, according to the above description, the terminal device 10A as a display control device further includes a determination unit 116 that determines the importance of each of the plurality of messages. The display control unit 114 compares the individual object SO corresponding to the message whose importance is equal to or higher than the first value among the plurality of individual objects SO included in the virtual object VO with the individual object SO corresponding to the message whose importance is less than the first value, and displays it near the user U1.

[0116] Since the terminal device 10A has the above configuration, the user U can preferentially check the messages that are more important to himself / herself.

[0117] 3: Third Embodiment Hereinafter, with reference to FIG. 15, the configuration of an information processing system 1B including a terminal device 10B as a display control device according to the third embodiment of the present invention will be described.

[0118] 3-1: Configuration of the Third Embodiment 3-1-1: Overall Configuration The information processing system 1B according to the third embodiment of the present invention differs from the information processing system 1 according to the first embodiment in that it includes a terminal device 10B instead of the terminal device 10. In other respects, since the overall configuration of the information processing system 1B is the same as that of the information processing system 1 according to the first embodiment shown in FIG. 1, its illustration and description are omitted.

[0119] 3-1-2: Configuration of the Terminal Device FIG. 15 is a block diagram showing a configuration example of the terminal device 10B. Different from the terminal device 10, the terminal device 10B includes a processing device 11B instead of the processing device 11 and a storage device 12B instead of the storage device 12. In addition, the terminal device 10B includes a sound collection device 17 in addition to the components provided in the terminal device 10.

[0120] The sound collection device 17 collects the voice of the user U1 and converts the collected voice into an electrical signal. Specifically, the sound collection device 17 is a microphone. The electrical signal converted from the voice by the sound collection device 17 is output to a voice recognition unit 117 described later.

[0121] Different from the storage device 12, the storage device 12B stores a control program PR2B instead of the control program PR2.

[0122] Different from the processing device 11, the processing device 11B includes an acquisition unit 112B instead of the acquisition unit 112. In addition, the processing device 11B includes a voice recognition unit 117 and a message generation unit 118 in addition to the components provided in the processing device 11.

[0123] The voice recognition unit 117 recognizes the voice collected by the sound collection device 17. More specifically, the voice recognition unit 117 generates text by performing voice recognition based on the electrical signal acquired from the sound collection device 17.

[0124] The message generation unit 118 generates a message corresponding to the text generated by the speech recognition unit 117. The message generated by the message generation unit 118 is acquired by the acquisition unit 112A. That is, the plurality of messages acquired by the acquisition unit 112A includes the message generated by the message generation unit 118. Note that the plurality of messages acquired by the acquisition unit 112A may include messages generated by a plurality of users U including the user U1 in addition to the message generated by the message generation unit 118. In other words, the plurality of messages are generated by the terminal device 10B-1 as a display control device and one or a plurality of terminal devices 10B connected to the terminal device 10B-1 via the communication network NET.

[0125] Since the terminal device 10B has the above configuration, a message can be generated based on the voice uttered by the user U1, and an individual object SO can be generated based on the generated message.

[0126] 3-2: Operations of the Third Embodiment The operation of the terminal device 10B according to the third embodiment, particularly the operation of the terminal device 10B-1 used by the user U1, is basically the same as the operation of the terminal device 10-1 according to the first embodiment shown in FIG. 10, and thus the illustration and detailed description thereof are omitted. In the operation of the terminal device 10B-1, the plurality of messages acquired in step S1 includes the message generated by the message generation unit 118.

[0127] 3-3: Effects of the Third Embodiment According to the above description, in the terminal device 10B-1 as a display control device, the plurality of messages are generated by the terminal device 10B-1 as a display control device and one or a plurality of terminal devices 10B connected to the terminal device 10B-1 via the communication network NET.

[0128] Since the terminal device 10B has the above configuration, the user U1 can check the messages generated by users U other than the user U1.

[0129] Also, according to the above description, the terminal device 10B as a display control device further includes a sound collection device 17, a speech recognition unit 117, and a message generation unit 118. The sound collection device 17 collects the voice of the user U and outputs an electrical signal representing the voice. The speech recognition unit 117 generates text based on the electrical signal output from the sound collection device 17. The message generation unit 118 generates a message corresponding to the text generated by the speech recognition unit 117. Also, the above plurality of messages include the message generated by the message generation unit 118.

[0130] Since the terminal device 10B has the above configuration, an individual object SO can be generated based on the voice uttered by the user U1.

[0131] 4: Fourth Embodiment Hereinafter, with reference to FIGS. 16 to 19, the configuration of an information processing system 1C including a terminal device 10C as a display control device according to the fourth embodiment of the present invention will be described.

[0132] 4-1: Configuration of the Fourth Embodiment 4-1-1: Overall Configuration The information processing system 1C according to the fourth embodiment of the present invention is different from the information processing system 1 according to the first embodiment in that it includes a terminal device 10C instead of the terminal device 10 and a server 30A instead of the server 30. In other respects, since the overall configuration of the information processing system 1C is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in FIG. 1, its illustration and description are omitted.

[0133] 4-1-2: Configuration of the Terminal Device Unlike the terminal device 10, the terminal device 10C includes a processing device 11C instead of the processing device 11 and a storage device 12C instead of the storage device 12. Different from the storage device 12, the storage device 12C stores a control program PR2C instead of the control program PR2. Different from the processing device 11, the processing device 11C includes an output unit 111C instead of the output unit 111. In other respects, since the configuration of the terminal device 10C is the same as that of the terminal device 10 according to the first embodiment shown in FIG. 4, the illustration and description thereof are omitted.

[0134] The output unit 111C has the same functions as those of the output unit 111. Further, the output unit 111C outputs the device ID (identifier) of the terminal device 10, the user name of the user using the terminal device 10, and the position information acquired by the terminal device 10 from the XR glasses 20 to the server 30A. For example, when the terminal device 10 is the terminal device 10-1 used by the user U1 who is the first user, the output unit 111C-1 outputs the device ID of the terminal device 10-1, the user name "U1" of the terminal device 10-1, and the position information generated by the XR glasses 20 worn on the head of the user U1 to the server 30A. The terminal device 10-1 is an example of a first display control device. Also, the output unit 111C outputs the coordinates indicating the display position of the virtual object VO displayed in the virtual space VS by the display control unit 114 to the server 30A in the virtual space VS. A virtual space VS including the virtual object VO is displayed on the XR glasses 20 connected to the terminal device 10-1. The XR glasses 20 are an example of a first display device. The virtual space VS displayed on the XR glasses 20 is an example of a first virtual space.

[0135] 4-1-3: Configuration of Server FIG. 16 is a block diagram showing a configuration example of the server 30A. Unlike the server 30, the server 30A includes a processing device 31A instead of the processing device 31 and a storage device 32A instead of the storage device 32.

[0136] Unlike the storage device 32, the storage device 32A stores a control program PR3A instead of the control program PR3. In addition to the components stored by the storage device 32, the storage device 32A stores a position information database LD.

[0137] FIG. 17 is a table showing an example of the position information database LD. As described above, the terminal device 10C outputs to the server 30A the device ID of the terminal device 10C, the user name using the terminal device 10, and the position information acquired from the XR glasses 20 by the terminal device 10C, or the position information generated by the terminal device 10C. These device IDs, user names, and position information are stored in the position information database LD. In the position information database LD shown in FIG. 17, as an example, "0001", which is the device ID of the terminal device 10C-1 used by the user U1, "U1", which is the user name of the user U1, and the position information (x, y, z) = (x u1 , y u1 , z u1 ) acquired by the terminal device 10C-1 from the XR glasses 20 are stored in a linked state. In the table shown in FIG. 17, "L" is an integer of 2 or more.

[0138] Unlike the processing device 31, the processing device 31A includes an acquisition unit 311A instead of the acquisition unit 311 and an output unit 312A instead of the output unit 312. In addition to the components provided in the processing device 31, the processing device 31A includes a determination unit 314 and an extraction unit 315.

[0139] The determination unit 314 determines whether or not the number of a plurality of messages output by the output unit 312A to the terminal device 10C-1 is equal to or greater than a predetermined number. When the determination unit 314 determines that the number of the plurality of messages is equal to or greater than the predetermined number, the acquisition unit 311A acquires from the terminal device 10C-1 the coordinates indicating the display position of the virtual object VO displayed by the display control unit 114-1 in the virtual space VS. Note that the acquisition unit 311A may acquire, in addition to the coordinates indicating the display position of the virtual object VO, the coordinates indicating the display position of the individual object SO.

[0140] The extraction unit 315 extracts, among the users U who are permitted to share the virtual space VS with the user U1 by referring to the location information database LD, other users U other than the user U1 who exist within a predetermined distance from the display position of the virtual object VO in the virtual space VS. The extracted other users U are examples of the second user. The second user is a user U who exists within a predetermined distance from the display position in the virtual space VS and whose sharing of the virtual space VS is permitted when the number of a plurality of messages is equal to or greater than a predetermined number. Note that it is preferable that this "predetermined distance" increases according to the number of a plurality of messages output to the terminal device 10C-1 by the output unit 312A.

[0141] The output unit 312A transmits to the terminal device 10C used by the user U extracted by the extraction unit 315, a plurality of messages identical to the plurality of messages output to the terminal device 10C-1, and coordinates indicating the display position of the virtual object VO in the virtual space VS. The plurality of identical messages and the coordinates indicating the display position of the virtual object VO are an example of control information. Also, the terminal device 10C to which the output unit 312A transmits the control information is an example of a second display control device. Further, the XR glasses 20 connected to the terminal device 10C or the display 14 provided in the terminal device 10C is an example of a second display device. The terminal device 10C that has acquired the plurality of identical messages generates, in the same manner as the terminal device 10C-1, a plurality of individual objects SO that correspond one-to-one to the plurality of identical messages. Also, the terminal device 10C causes the virtual space VS in which the virtual object VO, which is an aggregate of the plurality of individual objects SO, is arranged at the display position acquired from the server 30A to be displayed on the XR glasses 20 connected to the terminal device 10C or the display 14 provided in the terminal device 10C. The virtual space VS displayed on the XR glasses 20 connected to the terminal device 10C or the display 14 provided in the terminal device 10C is an example of a second virtual space. Here, the plurality of identical messages output from the output unit 312A and the coordinates indicating the display position of the virtual object VO are information for causing the second display device to display the second virtual space including the virtual object VO. Note that, as described above, when the acquisition unit 311A acquires, in addition to the coordinates indicating the display position of the virtual object VO, the coordinates indicating the display position of the individual object SO, the output unit 312A outputs to the terminal device 10C the coordinates indicating the display position of the individual object SO in the virtual space VS. In this case, the terminal device 10C causes the individual object SO to be displayed at the display position in the virtual space VS acquired from the server 30A.

[0142] As a result, in the virtual space VS, when the number of individual objects SO included in the virtual object VO visually recognized by the user U1 is equal to or greater than a predetermined number, another user U2 can also visually recognize the virtual object VO.

[0143] FIG. 18 is an explanatory diagram showing an example of the operations of the determination unit 314, the extraction unit 315, and the output unit 312A. Assume that in the XR glasses 20 worn by the user U1 on the head, a virtual space VS including a virtual object VO4 is being displayed. As the number of a plurality of messages addressed to the user U1 increases, the number of individual objects SO included in the virtual object VO4 also increases. Further, as the number of individual objects SO included in the virtual object VO4 increases, the virtual object VO4 moves away from the user U1. When the determination unit 314 provided in the server 30A determines that the number of a plurality of messages addressed to the user U1 is equal to or greater than a predetermined number, the extraction unit 315 extracts a user U other than the user U1 who exists within a predetermined distance from the display position (x, y, z) = (x 12 , y 12 , z 12 ) of the virtual object VO4 in the virtual space VS. In the example shown in FIG. 18, assume that the extraction unit 315 extracts a user U2 located at (x, y, z) = (x U2 , y U2 , z U2 ) as the user U other than the user U1. The output unit 312A transmits the same plurality of messages as those transmitted to the terminal device 10C-1 to the terminal device 10C-2 used by the user U2. The acquisition unit 112 provided in the terminal device 10C-2 acquires the same plurality of messages from the server 30A and the coordinates indicating the display position of the virtual object VO4 in the virtual space VS. The acquisition unit 112 provided in the terminal device 10C-2 generates a plurality of individual objects SO that correspond one-to-one to the same plurality of messages. The display control unit 114 provided in the terminal device 10C-2 displays the virtual object VO4, which is an aggregate of the plurality of individual objects SO, at the display position (x, y, z) = (x 12 , y 12 , z 12 ) in the virtual space VS acquired from the server 30A. As a result, in the virtual space VS, both the user U1 and the user U2 can visually recognize the virtual object VO9.

[0144] 4-2: Operations of the Fourth Embodiment FIG. 19 is a flowchart showing the operation of the server 30A according to the fourth embodiment. Hereinafter, the operation of the server 30A will be described with reference to FIG. 19.

[0145] In step S21, the processing device 31A functions as the output unit 312A. The processing device 31A transmits a plurality of messages to the terminal device 10C-1 used by the user U1. The terminal device 10C-1 causes the XR glasses 20 to display the virtual object VO4 based on the plurality of messages.

[0146] In step S22, the processing device 31A functions as the determination unit 314. The processing device 31A determines whether the number of the plurality of messages transmitted to the terminal device 10C-1 is equal to or greater than a predetermined number. If the determination result is affirmative, that is, if the processing device 31A determines that the number of the plurality of messages is equal to or greater than the predetermined number, the processing device 31A executes the process of step S23. If the determination result is negative, that is, if the processing device 31A determines that the number of the plurality of messages is less than the predetermined number, the processing device 31A executes the process of step S21.

[0147] In step S23, the processing device 31A functions as the acquisition unit 311A. The processing device 31A acquires, from the terminal device 10C-1, coordinates indicating the display position of the virtual object VO4 displayed in the virtual space VS by the display control unit 114-1 in the virtual space VS.

[0148] In step S24, the processing device 31A functions as the extraction unit 315. The processing device 31A extracts, among the users U sharing the virtual space VS with the user U1, users U other than the user U1 who exist within a predetermined distance from the display position of the virtual object VO4 in the virtual space VS. Here, as an example, it is assumed that the processing device 31A extracts the user U2.

[0149] In step S25, the processing device 31A functions as the output unit 312A. The processing device 31A transmits to the terminal device 10C-2 used by the user U2 a plurality of messages identical to the plurality of messages output to the terminal device 10C-1, and coordinates indicating the display position of the virtual object VO4 in the virtual space VS. The terminal device 10C-2 that has acquired the same plurality of messages generates a plurality of individual objects SO that correspond one-to-one to the same plurality of messages, in the same manner as the terminal device 10C-1. Further, the terminal device 10C-2 displays the virtual object VO4, which is an aggregate of the plurality of individual objects SO, at the display position in the virtual space VS acquired from the server 30A.

[0150] 4-3: Effects of the Fourth Embodiment According to the above description, the server 30A is a server that transmits a plurality of messages to the terminal devices 10 to 10C as the above-described first display control device. In this case, the XR glasses 20 as the first display device are worn on the head of the user U1 as the first user. The server 30A includes an acquisition unit 311A and an output unit 312A. The acquisition unit 311A acquires the display position of the virtual object VO4 in the first virtual space VS. When the number of the plurality of messages is equal to or greater than a predetermined number, the output unit 312A transmits control information to the terminal devices 10 to 10C as the second display control device that exists within a predetermined distance from the above-described display position in the first virtual space VS and allows sharing of the virtual object VO. The terminal devices 10 to 10C as the second display control device display a second virtual space on the XR glasses 20 as the second display device worn on the head of the user U2 as the second user. The above-described control information is information for causing the second display device to display the virtual object VO4 in the second virtual space.

[0151] Since the server 30A has the above-described configuration, when the number of individual objects SO included in the virtual object VO4 displayed on the XR glasses 20 is equal to or greater than a predetermined number, another user U2 can also visually recognize the virtual object VO4 in the virtual space VS.

[0152] 5: Variation The present disclosure is not limited to the embodiments exemplified above. Specific forms of variation are exemplified below. Two or more aspects arbitrarily selected from the following examples may be combined.

[0153] 5-1: Variation 1 The components and technical features included in each of the information processing systems 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment may be configured to be combined with each other. For example, the terminal device 10B according to the third embodiment and the server 30A according to the fourth embodiment may be combined. FIG. 20 is an explanatory diagram of a virtual object VO5 generated when the terminal device 10B and the server 30A are combined. For example, the terminal device 10B may generate a plurality of individual objects SO based on the voices of individual spectators in a soccer stadium, and display a virtual object VO5 composed of the plurality of individual objects SO in the sky above the soccer stadium on the XR glasses 20 as AR glasses. The virtual object VO5 is shared by a plurality of spectators wearing the XR glasses 20 as AR glasses on their heads, and, for example, the plurality of individual objects SO may be aligned in a shape representing a specific character based on the number of the individual objects SO. In this case, the virtual object VO5 and the virtual space VS composed of the individual objects SO corresponding one-to-one to a plurality of messages are shared by a plurality of users U gathering at a predetermined location. The predetermined location may be a venue where some event is held or a public facility such as a school. Further, the plurality of users U sharing the virtual object VO5 and the virtual space VS may be participants in the same event. For example, an e-sports tournament corresponds to the event.

[0154] Also, in addition to the information processing system 1B according to the third embodiment, the terminal devices 10 to 10C may acquire messages transmitted from a second user U to a third user U in addition to the messages addressed to the user U1 and the messages generated by the user U1 himself / herself.

[0155] 5-2: Second Variant Example In the information processing systems 1 according to the first embodiment to the information processing systems 1C according to the fourth embodiment, the terminal devices 10 to 10C include the display control unit 114 or the display control unit 114A. However, instead of the terminal devices 10 to 10C, the servers 30 to 30A may be configured to include the display control unit 114 or the display control unit 114A. Further, the servers 30 to 30A may be configured to set coordinates indicating the display positions of the virtual object VO and the individual object SO in the virtual space VS.

[0156] 5-3: Third Variant Example In the information processing system 1C according to the fourth embodiment, the server 30A includes the determination unit 314. However, instead of the server 30A, the terminal device 10C may be configured to include the determination unit 314. Specifically, the terminal device 10C side may be configured to determine whether the number of acquired messages or the number of individual objects SO displayed on the XR glasses 20 is equal to or greater than a predetermined number, and output the determination result to the server 30A.

[0157] 5-4: Fourth Variant Example In the information processing systems 1 according to the first embodiment to the information processing systems 1C according to the fourth embodiment, the display control unit 114 or the display control unit 114A provided in the terminal devices 10 to 10C may disappear the individual object SO whose message content has been read by the user U. In particular, in the server 30A provided in the information processing system 1C according to the fourth embodiment, the determination unit 314 may determine whether the number of unread messages, rather than the number of messages output to the terminal device 10C-1, is equal to or greater than a predetermined number.

[0158] 5-5: Fifth Variant Example In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the acquisition unit 112 provided in the terminal device 10 to the terminal device 10C acquires a plurality of messages from the server 30 or the server 30A. However, the acquisition unit 112 may acquire only the message IDs corresponding to each of the plurality of messages from the server 30 or the server 30A. In this case, the generation unit 113 generates individual objects SO that correspond one-to-one to the plurality of message IDs instead of the plurality of messages. Also, in this case, as shown in FIG. 13 as an example, the acquisition unit 112 may acquire the content of the message from the server 30 or the server 30A for the first time when the user U1 views the content of the message corresponding to the individual object SO. After that, the display control unit 114 or the display control unit 114A may display the content of the message in the virtual space VS.

[0159] 5-6: Modification Example 6 In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the individual object SO corresponds one-to-one to the message generated by the user U. However, the message corresponding to the individual object SO is not limited to the message generated by the user U. For example, the message may be a notification to the user U generated by an application.

[0160] 5-7: Modification Example 7 In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the server 30 or the server 30A outputs a message stored in the message database MD to the terminal devices 10 to 10C. However, the method of outputting messages from the server 30 or the server 30A to the terminal devices 10 to 10C is not limited to this. For example, the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment may further include a content server. More specifically, the server 30 or the server 30A may acquire content including a message for the user U from the content server and output the acquired content to the terminal devices 10 to 10C.

[0161] 5-8: Variant 8 In the information processing system 1 according to the first embodiment to the information processing system 1C according to the fourth embodiment, the terminal devices 10 to 10C and the XR glasses 20 are realized as separate bodies. However, the method of realizing the terminal devices 10 to 10C and the XR glasses 20 in the embodiments of the present invention is not limited to this. For example, the XR glasses 20 may have the same functions as the terminal device 10. In other words, the terminal devices 10 to 10C and the XR glasses 20 may be realized in a single housing. The same applies to the information processing system 1A according to the second embodiment to the information processing system 1C according to the fourth embodiment.

[0162] 5-9: Variant 9 The information processing systems 1 according to the first embodiment to the information processing systems 1C according to the fourth embodiment include, as an example, the XR glasses 20 as AR glasses. However, the information processing systems 1 to 1C may include any one of an HMD (Head Mounted Display) employing VR (Virtual Reality) technology, an HMD employing MR (Mixed Reality) technology, and MR glasses employing MR technology as the XR glasses 20. Alternatively, the information processing systems 1 to 1C may include any one of a normal smartphone and a tablet equipped with an imaging device instead of the XR glasses 20. These HMDs, MR glasses, smartphones, and tablets are examples of display devices.

[0163] 6: Others (1) In the above-described embodiments, the storage devices 12 to 12C, the storage device 22, and the storage devices 32 to 32A are exemplified by ROM and RAM, but flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROM (Compact Disc-ROM), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, and other appropriate storage media. Also, the program may be transmitted from a network via a telecommunication line. Also, the program may be transmitted from the communication network NET via a telecommunication line.

[0164] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0165] (3) In the above-described embodiments, the input and output information, etc. may be stored in a specific location (e.g., memory), or may be managed using a management table. The input and output information, etc. can be overwritten, updated, or appended. The output information, etc. may be deleted. The input information, etc. may be transmitted to other devices.

[0166] (4) In the above-described embodiments, the determination may be made based on a value represented by 1 bit (0 or 1), may be made based on a boolean value (Boolean: true or false), or may be made by comparing numerical values (e.g., comparison with a predetermined value).

[0167] (5) The processing procedures, sequences, flowcharts, etc. exemplified in the above-described embodiments may be reordered as long as there is no contradiction. For example, for the methods described in the present disclosure, the elements of various steps are presented using an exemplary order and are not limited to the presented specific order.

[0168] (6) Each function illustrated in FIGS. 1 to 20 is realized by any combination of at least one of hardware and software. Also, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one physically or logically combined device, or two or more physically or logically separated devices may be directly or indirectly connected (e.g., using wired, wireless, etc.) and realized using these multiple devices. The functional block may be realized by combining software with the above one device or the above multiple devices.

[0169] (7) The programs exemplified in the above-described embodiments should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether called by the names of software, firmware, middleware, microcode, hardware description language, or other names.

[0170] Also, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cables, fiber optic cables, twisted pairs, digital subscriber lines (DSLs), etc.) and wireless technologies (such as infrared rays, microwaves, etc.), at least one of these wired and wireless technologies is included within the definition of the transmission medium.

[0171] (8) In each of the foregoing embodiments, the terms "system" and "network" are used interchangeably.

[0172] (9) The information, parameters, etc. described in this disclosure may be represented using absolute values, relative values from a predetermined value, or corresponding other information.

[0173] (10) In the above-described embodiments, the terminal devices 10 to 10C and the servers 30 to 30A may include cases where they are mobile stations (MS: Mobile Station). A mobile station may be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terms. Also, in the present disclosure, terms such as "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.

[0174] (11) In the above-described embodiments, the terms "connected" and "coupled", or any variations thereof, mean any direct or indirect connection or coupling between two or more elements, including the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read as "accessed". As used in the present disclosure, two elements are considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, and also, as some non-limiting and non-exhaustive examples, electromagnetic energy having wavelengths in the radio frequency region, microwave region, and optical (both visible and invisible) region.

[0175] (12) In the above-described embodiments, the description "based on" does not mean "only based on" unless otherwise specified. In other words, the description "based on" means both "only based on" and "at least based on".

[0176] (13) As used herein, the terms "determining" may encompass a wide variety of operations. "Determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up (e.g., searching in a table, database, or other data structure), ascertaining, and considering something as having been "determined". Also, "determining" may include considering something as having been "determined" after receiving (e.g., receiving information), transmitting (e.g., transmitting information), inputting, outputting, accessing (e.g., accessing data in memory). Further, "determining" may include considering something as having been "determined" after resolving, selecting, choosing, establishing, comparing, etc. That is, "determining" may include considering something as having been "determined" after performing some operation. Also, "determining" may be construed as "assuming", "expecting", "considering", etc.

[0177] (14) In the above-described embodiments, when the terms "include", "including", and their variants are used, these terms are intended to be inclusive in the same manner as the term "comprising". Further, the term "or" as used herein is not intended to be an exclusive disjunction.

[0178] (15) In the present disclosure, for example, when articles are added by translation, such as a, an, and the in English, the present disclosure may include that the nouns following these articles are in the plural form.

[0179] (16) In the present disclosure, the term "A and B are different" may mean that "A and B are different from each other". Note that the term may also mean that "A and B are each different from C". Terms such as "separate", "coupled", etc. may also be interpreted in the same way as "different".

[0180] (17) Each aspect / embodiment described in the present disclosure may be used alone, in combination, or switched and used during execution. Also, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicitly made notification, and may be made implicitly (for example, without making the notification of the predetermined information).

[0181] As described in detail above regarding the present disclosure, it is obvious to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented as modified and changed aspects without departing from the spirit and scope of the present disclosure determined by the description of the claims. Therefore, the description of the present disclosure is for the purpose of illustrative explanation and has no restrictive meaning for the present disclosure.

Description of Reference Numerals

[0182] 1 to 1C... Information processing system, 10 to 10C... Terminal device, 11 to 11C... Processing device, 12 to 12C... Memory device, 13... Communication device, 14... Display, 15... Input device, 16... Inertial sensor, 17... Sound collection device, 20... AR glasses, 21... Processing device, 22... Memory device, 23... Line-of-sight detection device, 24... GPS device, 25... Motion detection device, 26... Imaging device, 27... Communication device, 28... Display, 30, 30A... Server, 31, 31A... Processing device, 32, 32A... Memory device, 33... Communication device, 34... Display, 35... Input device, 41L, 41R... Lens, 91, 92... Temple, 93... Bridge, 94, 95... Frame, 111, 111C... Output unit, 112 to 112B... Acquisition unit, 113... Generation unit, 114, 114A... Display control unit, 115... Reception unit, 116... Decision unit, 117... Speech recognition unit, 118... Message generation unit, 311, 311A... Acquisition unit, 312, 312A... Output unit, 313... Message management unit, 314... Judgment unit, 315... Extraction unit, MO1... Message object, PR1 to PR3A... Control program, SO, SO1 to SO10... Individual object, U, U1 to U2... User, VO, VO1 to VO5... Virtual object

Claims

1. A display control device that causes a display device worn on a user's head to display a virtual space including virtual objects, comprising: an acquisition unit that acquires a plurality of messages; a generation unit that generates a plurality of individual objects corresponding one-to-one to the plurality of messages; a display control unit that causes the display device to display a virtual object that is an aggregate of the plurality of individual objects; a determination unit that determines the importance of each of the plurality of messages; and the display control unit when the number of the plurality of messages is a first number, sets the size of the virtual object to a first size, and in the virtual space, sets the distance from the center of the virtual space to the center of the virtual object to a first distance; when the number of the plurality of messages is a second number greater than the first number, sets the size of the virtual object to a second size greater than the first size, and in the virtual space, sets the distance from the center of the virtual space to the center of the virtual object to a second distance longer than the first distance; displays, in the vicinity of the user, a first individual object corresponding to a message having an importance equal to or greater than a first value among the plurality of individual objects included in the virtual object, as compared with a second individual object corresponding to a message having an importance less than the first value. Display control device.

2. The display control device according to claim 1, wherein the display control unit changes a display mode of the virtual object according to the number of the plurality of messages.

3. The display control device according to claim 1 or claim 2, wherein the display control unit makes the display modes of the plurality of individual objects different from each other according to the transmission source devices corresponding to each of the plurality of messages.

4. further comprising a reception unit that receives an operation of the user with respect to the virtual object, wherein the display control unit causes a list of the plurality of messages to be displayed in the virtual space when the operation is a first operation. The display control device according to any one of claims 1 to 3.

5. further comprising a reception unit that receives an operation of the user with respect to the virtual object, The display control unit causes the virtual space to display the content of a message corresponding to the one individual object when the operation is a second operation that designates one individual object among the plurality of individual objects. The display control device according to any one of claims 1 to 3.

6. The plurality of messages are generated by the display control device and one or more terminal devices connected to the display control device. The display control device according to any one of claims 1 to 5.

7. A sound collection device that collects the user's voice and outputs an electrical signal representing the voice, A voice recognition unit that generates text based on the electrical signal output from the sound collection device, A message generation unit that generates a message corresponding to the text generated by the voice recognition unit, further comprising: The plurality of messages include the message generated by the message generation unit. The display control device according to claim 6.

8. A server that transmits the plurality of messages to the display control device according to any one of claims 1 to 7, The display control device is a first display control device, the display device is a first display device worn on the head of the user as a first user, and the virtual space is a first virtual space. An acquisition unit that acquires the display position of the virtual object in the first virtual space from the first display control device, When the number of the plurality of messages is equal to or greater than a predetermined number, an output unit that transmits control information to a second display control device that exists within a predetermined distance from the display position in the first virtual space and allows sharing of the virtual object. The second display control device causes a second display device worn on the head of a second user to display a second virtual space. The control information is information for causing the second display device to display the virtual object in the second virtual space. Server

Images