Display control device

Abstract

To provide a display control device capable of grasping which content other participants are paying attention to in a virtual space.SOLUTION: An acquisition unit 311 acquires operation information indicating that one of a first user U1 and a second user U2 who share a virtual space VS performs a specific operation and first line-of-sight information regarding the line-of-sight of the first user. When the operation information is acquired, on the basis of the first line-of-sight information, a calculation unit 312 calculates a line-of-sight direction LS1 and a gaze position GP1 of the first user in the virtual space VS. A generation unit 313 generates a display image HG including a virtual object in a virtual space corresponding to the calculated line-of-sight direction LS1 and the gaze position GP1. A display control unit 314 displays the display image HG on a display 26 of a second terminal device 20 carried by the second user U2.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to a display control device.

Background Art

[0002] Conventionally, in a scenario where two or more users wearing a glass device such as a head-mounted display (HMD) share content in a virtual space, there are cases where two or more users want to grasp each other's attention parts. Patent Document 1 discloses an information processing apparatus that acquires gaze information indicating the gaze position of a first user with respect to a predetermined object and presents the gaze position of the first user to a second user based on input information by at least one of the first user and the second user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in a meeting held in a virtual space, it is assumed that a first user participating while wearing a glass device and a second user participating using a terminal having a two-dimensional display without wearing a glass device are mixed. However, in the conventional information processing apparatus, there has been a problem that it is difficult for the second user in the virtual space to grasp which content the first user is paying attention to.

[0005] The present invention has been made to solve the above problems, and an object thereof is to enable other participants in a virtual space to grasp which content they are paying attention to.

Means for Solving the Problems

[0006] A display control device according to a preferred aspect of the present invention includes an acquisition unit that acquires operation information indicating that one of a first user and a second user who share a virtual space has performed a specific operation and first gaze information regarding the gaze of the first user, a calculation unit that calculates at least one of the direction and the fixation position of the gaze of the first user on the virtual space based on the acquired first gaze information when the operation information is acquired, a generation unit that generates a first virtual object on the virtual space corresponding to at least one of the calculated gaze direction and fixation position, an image including the first virtual object, the calculated gaze direction, and a display image indicating at least one of the calculated fixation position, and a display control unit that displays the display image on a display of a device held by the second user.

Effects of the Invention

[0007] According to the display control device of the present invention, it is possible to grasp which content other participants are paying attention to in the virtual space.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Embodiments for Carrying Out the Invention

[0009] 1. First Embodiment Hereinafter, with reference to FIGS. 1 to 15, the configuration of the display control device according to the first embodiment of the present invention will be described.

[0010] 1.1. Configuration of the First Embodiment 1.1.1. Overall Configuration FIG. 1 is a diagram showing the overall configuration of an information processing system 1 including a display control device according to the first embodiment. The information processing system 1 includes a first terminal device 10, a second terminal device 20, a server 30, and a communication network NET. The first terminal device 10 is an example of a glass device, the second terminal device 20 is an example of a device possessed by a second user, and the server 30 is an example of a display control device.

[0011] The first terminal device 10 includes two first terminal devices 10a and 10b. The first user U1 corresponds to the first terminal device 10a. The third user U3 corresponds to the first terminal device 10b. The second user U2 corresponds to the second terminal device 20. The first user U1 wears the first terminal device 10a on the head. The second user U2 possesses the second terminal device 20. The third user U3 wears the first terminal device 10b on the head.

[0012] The first terminal device 10 is a head-mounted display (HMD) capable of experiencing VR (Virtual Reality) content. The first terminal device 10 displays virtual objects on display panels provided on each lens corresponding to both eyes of the user.

[0013] In this embodiment, the configurations of the first terminal devices 10a and 10b are the same as each other. Note that the first terminal device 10 may include terminal devices with different configurations. Further, the first terminal device 10 may be, for example, a system in which an information processing terminal such as a smartphone, a tablet terminal, a personal computer, a game device, etc. is connected to Google.

[0014] The second terminal device 20 includes devices having a flat display such as a personal computer, a tablet terminal, a smartphone, a smartwatch, etc. The flat display is a two-dimensional display capable of displaying two-dimensional images. As the second terminal device 20, a terminal with a relatively large display such as a personal computer or a tablet terminal is preferable.

[0015] In FIG. 1, two first terminal devices 10a and 10b and one second terminal device 20 are illustrated, but the number is merely an example. The information processing system 1 can include an arbitrary number of first terminal devices 10 and an arbitrary number of second terminal devices 20.

[0016] In the information processing system 1, the first terminal device 10a, the first terminal device 10b, the second terminal device 20, and the server 30 are communicably connected to each other via a communication network NET.

[0017] The server 30 is a server that provides an image display service. The server 30 collects various information related to the image display service from the first terminal device 10 and the second terminal device 20 via the communication network NET, and manages events based on the various information.

[0018] The information processing system 1 is a system that provides an image display service to any number of users wearing the first terminal device 10 and any number of users possessing the second terminal device 20.

[0019] FIG. 2 is a schematic diagram of a virtual space VS displayed by the first terminal device 10a. As shown in FIG. 2, one or more virtual objects VO are arranged in this virtual space VS. The virtual object VO includes five virtual objects VO1 to VO5. In the present embodiment, each of the virtual objects VO1 to VO5 is arranged inside a spherical surface R having a predetermined radius centered on the position of the head of the first user U1. Although the first user U1 is shown for convenience in FIG. 2, actually, the first user U1 is not displayed on the first terminal device 10a.

[0020] The one or more virtual objects VO are, for example, virtual objects indicating data such as still images, moving images, three-dimensional CG models, HTML files, text files, etc., and virtual objects indicating applications. Here, examples of the text file include a memo and source code. Examples of the application include a browser, an application for using SNS, and an application for generating a document file. Note that the number of the one or more virtual objects VO in FIG. 2 is merely an example, and the number of the one or more virtual objects VO is not limited to the present disclosure.

[0021] 1.1.2. Configuration of the First Terminal Device FIG. 3 is a block diagram showing a configuration example of the first terminal device 10 in FIG. 1. The first terminal device 10 includes a processing device 11, a storage device 12, a line-of-sight acquisition device 13, a motion detection device 14, a microphone 15, a speaker 16, a communication device 17, a display 18, and an input device 19. Each element of the first terminal device 10 is interconnected by a single or a plurality of buses for communicating information. Note that the term "device" in this specification may be read as other terms such as a circuit, a device, a unit, etc.

[0022] The processing device 11 is a processor that controls the entire first terminal device 10 and is configured using, for example, one or more chips. The processing device 11 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and registers. Note that part or all of the functions of the processing device 11 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 11 executes various processes in parallel or sequentially.

[0023] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, a RAM (Random Access Memory).

[0024] The storage device 12 stores a plurality of programs including a control program PR1 to be executed by the processing device 11. Further, the storage device 12 functions as a work area for the processing device 11.

[0025] The line-of-sight acquisition device 13 acquires the direction in which the user is looking, that is, the direction of the user's line of sight, by eye tracking that tracks the movement of the user's left and right eyeballs, and outputs line-of-sight information indicating the direction of the user's line of sight to the processing device 11 based on the acquisition result.

[0026] More specifically, the line-of-sight acquisition device 13 includes a pair of light sources and a pair of cameras. Each of the pair of light sources corresponds to the user's left eye and right eye, and each of the pair of cameras corresponds to the user's left eye and right eye. The light source corresponding to the user's left eye irradiates infrared light onto the user's left eye, and the camera corresponding to the user's left eye captures an image of the corneal reflection image of the user's left eye as an image formed by the reflection of the irradiated infrared light. Similarly, the light source corresponding to the user's right eye irradiates infrared light onto the user's right eye, and the camera corresponding to the user's right eye captures an image of the corneal reflection image of the user's right eye.

[0027] The line-of-sight acquisition device 13 detects the position of the user's canthus and the position of the iris from the acquired corneal reflection images of the left and right eyes, and acquires the user's line-of-sight information based on the detected positions of the user's canthus and the iris. Note that the method for acquiring the line of sight by the line-of-sight acquisition device 13 is not limited to the above method, and any method may be used.

[0028] The motion detection device 14 detects the motion of the first terminal device 10 and outputs motion data to the processing device 11. The motion data includes acceleration data indicating the accelerations in the directions of the X-axis, Y-axis, and Z-axis, respectively, and angular acceleration data indicating the angular accelerations with the respective axes of the X-axis, Y-axis, and Z-axis as the centers of rotation. The motion detection device 14 includes an acceleration sensor that detects acceleration, an inertial sensor such as a gyro sensor that detects angular acceleration, and a geomagnetic sensor that detects the geomagnetism in the directions of the X-axis, Y-axis, and Z-axis and detects the direction in which the first terminal device 10 is facing.

[0029] The acceleration sensor detects the accelerations in the directions of the X-axis, Y-axis, and Z-axis that are perpendicular to each other, respectively. The gyro sensor detects the angular accelerations with the respective axes of the X-axis, Y-axis, and Z-axis as the centers of rotation, respectively. The geomagnetic sensor detects the geomagnetism in the directions of the X-axis, Y-axis, and Z-axis, respectively, and detects the direction in which the first terminal device 10 is facing. The motion detection device 14 generates attitude information indicating the attitude of the first terminal device 10 based on the output information from the gyro sensor. In addition, the motion detection device 14 outputs motion information related to the motion of the first terminal device 10 to the processing device 11.

[0030] The microphone 15 converts the input voice into an electrical signal and outputs it. The voice input to the microphone 15 is, for example, the voice of the user speaking. The microphone 15 may be provided with an AD converter that converts an analog voice signal into voice information that is a digital signal. The voice information is output to the processing device 11.

[0031] The speaker 16 outputs voice. The voice output by the speaker 16 is, for example, the voice included in the VR content, the voice of another user participating in the VR conference, etc. The speaker 16 is controlled by the processing device 11.

[0032] The communication device 17 is hardware as a transmission and reception device for communicating with other devices. The communication device 17 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 17 is provided with a connector for a wired connection and may be provided with an interface circuit corresponding to the connector. Further, the communication device 17 may be provided with a wireless communication interface. Examples of the connector and the interface circuit for the wired connection include products compliant with wired LAN, IEEE1394, USB, etc. Examples of the wireless communication interface include products compliant with wireless LAN, Bluetooth (registered trademark), etc.

[0033] The display 18 is a device that displays images. The display 18 displays various images based on the control by the processing device 11. The display 18 includes a display for the left eye and a display for the right eye.

[0034] The input device 19 receives an input from the user. In the present embodiment, the input device 19 is used for, for example, selecting the VR content used by the user. The input device 19 may be, for example, a switch provided on a head-mounted display or a remote controller, etc.

[0035] The processing device 11 functions as an acquisition unit 111 and a display control unit 112, for example, by reading and executing the control program PR1 from the storage device 12.

[0036] The acquisition unit 111 acquires a control signal and image information from the first terminal device 10. The control signal is a signal for controlling the display on the display 18. The control signal is generated by a display control unit 113 (described later) provided in the first terminal device 10. The image information is information indicating an image to be displayed on the display 18. The image information is, for example, information including information indicating virtual objects VO1 to VO5. The image information may be stored in the storage device 12 of the first terminal device 10 or may be stored in a storage device 32 (described later) of the server 30.

[0037] In addition, the acquisition unit 111 acquires gaze information input from the gaze acquisition device 13, motion information input from the motion detection device 14, and voice information input from the microphone 15. Then, the acquisition unit 111 outputs the acquired gaze information, motion information, and voice information to the communication device 17.

[0038] The display control unit 112 controls the display on the display 18 based on the control signal acquired by the acquisition unit 111. In addition, the display control unit 112 causes the virtual object VO to be displayed on the display 18 based on the image information acquired by the acquisition unit 111.

[0039] 1.1.3. Configuration of the Second Terminal Device FIG. 4 is a block diagram showing a configuration example of the second terminal device 20 in FIG. 1. As shown in FIG. 4, the second terminal device 20 includes a processing device 21, a storage device 22, a microphone 23, a speaker 24, a communication device 25, a display 26, and an input device 27. Each element included in the second terminal device 20 is interconnected by a single or a plurality of buses for communicating information.

[0040] The processing device 21 is a processor that controls the entire second terminal device 20, and is configured using, for example, one or more chips. The processing device 21 is configured using, for example, a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, registers, and the like. Note that some or all of the functions provided in the processing device 21 may be realized by hardware such as a DSP, an ASIC, a PLD, or an FPGA. The processing device 21 executes various processes in parallel or sequentially.

[0041] The storage device 22 is a recording medium that can be read from and written to by the processing device 21. The storage device 22 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM.

[0042] The storage device 22 stores a plurality of programs including a control program PR2 to be executed by the processing device 21. The storage device 22 also functions as a work area for the processing device 21.

[0043] The microphone 23 converts the input voice into an electrical signal and outputs it. The voice input to the microphone 23 is, for example, the voice spoken by the user. The microphone 23 may include an AD converter that converts a voice signal, which is an analog signal, into voice information, which is a digital signal. The voice information is output to the processing device 21.

[0044] The speaker 24 outputs voice. The voice output by the speaker 24 is, for example, the voice included in the VR content, the voice spoken by another user participating in the VR conference, or the like. The speaker 24 is controlled by the processing device 21.

[0045] The communication device 25 is hardware as a transceiver device for communicating with other devices. The communication device 25 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 25 may include a connector for wired connection and an interface circuit corresponding to the connector. Further, the communication device 25 may include a wireless communication interface. Examples of the connector for wired connection and the interface circuit include products compliant with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products compliant with wireless LAN and Bluetooth (registered trademark).

[0046] The display 26 is a device for displaying images and character information. The display 26 displays various images based on the control by the processing device 21. For example, various display panels such as a liquid crystal panel and an organic EL panel are preferably used as the display 26.

[0047] The input device 27 receives operations from the user. For example, the input device 27 includes pointing devices such as a keyboard, a touch pad, a touch panel, and a mouse. Here, when the input device 27 includes a touch panel, it may also serve as the display 26.

[0048] The processing device 21 functions as an acquisition unit 211, an output unit 212, and a display control unit 213, for example, by reading and executing the control program PR2 from the storage device 22.

[0049] The acquisition unit 211 acquires instruction information corresponding to the user's operation. Here, the user's operation is, for example, an operation input by the user to the second terminal device 20 using the input device 27. More specifically, the user's operation may be the pressing of a specific part of the input device 27.

[0050] The acquisition unit 211 acquires image information indicating virtual objects VO1 to VO5 from the server 30. The acquisition unit 211 also acquires instruction information from the input device 27.

[0051] The acquisition unit 211 acquires image information from the server 30 by using the communication device 25. The acquisition unit 211 stores the acquired image information in the storage device 22. The acquisition unit 211 acquires the image information stored in the storage device 22.

[0052] The output unit 212 outputs the user's instruction information to the server 30 via the communication device 25 as necessary.

[0053] The display control unit 213 causes the display 26 to display various information based on the various information acquired by the acquisition unit 211.

[0054] 1.1.4. Configuration of the Server FIG. 5 is a block diagram showing a configuration example of the server 30 in FIG. 1. As shown in FIG. 5, the server 30 includes a processing device 31, a storage device 32, a communication device 33, a display 34, and an input device 35. Each element included in the server 30 is interconnected by a single or a plurality of buses for communicating information.

[0055] The processing device 31 is a processor that controls the entire server 30 and is configured using, for example, one or more chips. The processing device 31 is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and registers. Note that some or all of the functions of the processing device 31 may be implemented by hardware such as a DSP, an ASIC, a PLD, or an FPGA. The processing device 31 executes various processes in parallel or sequentially.

[0056] The storage device 32 is a recording medium that can be read from and written to by the processing device 31. The storage device 32 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM, an EPROM, and an EEPROM. The volatile memory is, for example, a RAM.

[0057] The storage device 32 stores a plurality of programs including a control program PR3 for execution by the processing device 31. Further, the storage device 32 functions as a work area for the processing device 31.

[0058] The communication device 33 is hardware as a transmission / reception device for communicating with other devices. The communication device 33 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 33 may include a connector for a wired connection and an interface circuit corresponding to the connector. Further, the communication device 33 may include a wireless communication interface. Examples of the connector for a wired connection and the interface circuit include products compliant with a wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products compliant with a wireless LAN and Bluetooth (registered trademark).

[0059] The display 34 is a device that displays images and character information. The display 34 displays various images based on the control by the processing device 31. For example, various display panels such as a liquid crystal panel and an organic EL panel are preferably used as the display 34.

[0060] The input device 35 is a device that receives operations by the administrator of the information processing system 1. For example, the input device 35 includes a pointing device such as a keyboard, a touch pad, a touch panel, and a mouse. Here, when the input device 35 includes a touch panel, it may also serve as the display 34. The administrator of the information processing system 1 can modify the control program PR3 by using the input device 35.

[0061] The processing device 31 functions as an acquisition unit 311, a calculation unit 312, a generation unit 313, and a display control unit 314, for example, by reading and executing the control program PR3 from the storage device 32.

[0062] Hereinafter, with reference to FIGS. 6 to 8, each function of the acquisition unit 311, the calculation unit 312, the generation unit 313, and the display control unit 314 will be described.

[0063] FIG. 6 is a schematic diagram showing an example of a virtual space VS displayed on the first terminal device 10a when the first user U1 participates in a VR meeting. The VR meeting is a meeting held on the virtual space VS. As shown in FIG. 6, the first user U1 can visually recognize the virtual object VO in the virtual space VS by wearing the first terminal device 10a on the head.

[0064] In the virtual space VS, as virtual objects VO, four slide materials SL1, SL2, SL3, and SL4 and four contents CO1, CO2, CO3, and CO4 are arranged. The first user U1 is gazing at the slide material SL3. The gazing position of the first user U1 is represented by GP1. Also, the direction of the line of sight of the first user U1 is represented by LS1. The range of the field of view of the first user U1 is represented by FOV1.

[0065] Note that the direction LS1 of the line of sight of the first user U1, the gazing position GP1 of the first user U1, and the field of view FOV1 of the first user U1 are shown for convenience in FIG. 6, but are not displayed on the display 18 of the first terminal device 10a. Therefore, the four slide materials SL1 to SL4 and the four contents CO1 to CO4 are visible to the first user U1, but the direction LS1 of the line of sight of the first user U1, the gazing position GP1 of the first user U1, and the field of view FOV1 of the first user U1 are not visible.

[0066] Also, the first user U1 and the first terminal device 10a are shown for convenience in FIG. 6 but are not displayed on the display 18 of the first terminal device 10a. Therefore, the first user U1 cannot visually recognize the first user U1 and the first terminal device 10a.

[0067] Also, a first sharing button SB1 may be displayed on the display 18 of the first terminal device 10a. When the first user U1 bends down, the first sharing button SB1 enters the field of view FOV1 of the first user U1, so the first user U1 can visually recognize the first sharing button SB1 on the display 28.

[0068] The first sharing button SB1 is a button for sharing, with a second user U2 sharing the same virtual space VS as the first user U1, a virtual object corresponding to the direction LS1 of the line of sight or the fixation position GP1 of the first user U1, or the direction LS1 of the line of sight or the fixation position GP1 of the first user U1.

[0069] For example, when the first user is a presenter, it is conceivable that the first user U1 may want to share the fixation position GP1 of the first user U1 himself / herself and the direction LS1 of the line of sight of the first user U1 himself / herself with the second user U2. In this case, when the first user U1 operates the input device 19 of the first terminal device 10a to press the first sharing button SB1, a display image based on the direction LS1 of the line of sight or the fixation position GP1 of the first user U1 is displayed on the display 26 of the second terminal device 20 held by the second user U2.

[0070] FIG. 7 is a diagram showing an example of a virtual space VS displayed on the second terminal device 20 when the second user U2 participates in a VR conference. As shown in FIG. 7, the second user U2 can visually recognize a partial area within the virtual space VS as a two-dimensional image on the display 26 of the second terminal device 20. The second user U2 can scroll up, down, left, or right a part of the virtual space VS displayed on the display 26 among the entire virtual space VS by operating the cursor button CUR on the display 26 or the cursor keys of the input device 27.

[0071] On the display 26, a part of the slide material SL2, the entire slide material SL3, a part of the slide material SL4, and the entire content CO3 are displayed. Also, on the display 26, the fixation position GP1 of the first user U1 and the direction LS1 of the line of sight of the first user U1 are displayed.

[0072] FIG. 8 is a diagram showing an example of the virtual space VS displayed on the second terminal device 20 when the first user U1 shares the first line-of-sight information with the second user U2 in a VR conference.

[0073] When the first user U1 presses the first sharing button SB1, a virtual object corresponding to the direction LS1 of the line of sight or the fixation position GP1 of the first user U1 is specified. Then, an image showing the area including the specified virtual object is displayed as the display image HG on the display 26 of the second terminal device 20.

[0074] The virtual object corresponding to the direction LS1 of the line of sight or the fixation position GP1 of the first user U1 is a virtual object existing in the direction LS1 of the line of sight of the first user U1 or a virtual object overlapping with the fixation position GP1 of the first user U1. In this example, the slide material SL3 corresponds to the virtual object corresponding to the direction LS1 of the line of sight or the fixation position GP1 of the first user U1.

[0075] Also, when there is no virtual object in the direction LS1 of the line of sight of the first user U1, or when there is no virtual object overlapping with the fixation position GP1 of the first user U1, the virtual object with the closest direction to the direction LS1 of the line of sight of the first user U1 or the virtual object closest to the fixation position GP1 of the first user U1 is specified as the virtual object corresponding to the direction LS1 of the line of sight or the fixation position GP1 of the first user U1.

[0076] That is, the display image HG is an image showing the area including the specified virtual object. The specified virtual object is the virtual object corresponding to the direction LS1 of the line of sight or the fixation position GP1 of the first user U1. In this embodiment, "pressing" includes the user pressing a key on the keyboard, clicking a button using a mouse, tapping a button on the touch panel, and the like.

[0077] More specifically, the first line-of-sight information is generated by the first terminal device 10a. When the first user U1 performs a specific operation, it is displayed on the display 18 of the first terminal device 10a that the specific operation has been performed. For example, when the first user U1 performs a specific operation, the display control unit 112 displays a message "Sharing line-of-sight information" on the display 18.

[0078] The first line-of-sight information and the operation information are transmitted to the server 30. In other words, the communication device 17 of the first terminal device 10a transmits the first line-of-sight information and the operation information to the server 30. The operation information is information indicating that the first user U1 has performed a specific operation. The specific operation is an operation of pressing the first sharing button SB1.

[0079] The acquisition unit 311 acquires the operation information and the first line-of-sight information from the first terminal device 10a.

[0080] When operation information is acquired, the calculation unit 312 calculates the direction LS1 of the line of sight and the fixation position GP1 of the first user U1 on the virtual space VS based on the first line-of-sight information acquired by the acquisition unit 311. For example, the calculation unit 312 calculates the fixation position GP1 of the first user U1 as the point where the line of sight of the left eye and the line of sight of the right eye of the first user U1 intersect. Further, the calculation unit 312 calculates the direction LS1 of the line of sight of the first user U1 as the direction from the midpoint between the right eye and the left eye of the first user U1 toward the fixation position GP1.

[0081] The generation unit 313 generates a display image HG. In the present embodiment, as shown in FIG. 8, the display image HG is represented by a frame line including the slide material SL3. The slide material SL3 is an example of a first virtual object.

[0082] The display control unit 314 displays the display image HG generated by the generation unit 313 on the display 26 of the second terminal device 20. Thereby, the second user U2 can easily grasp that the first user U1 is paying attention to the slide material SL3.

[0083] Further, as shown in FIG. 8, a second sharing button SB2 is displayed on the display 26 of the second terminal device 20. When the second user U2 presses the second sharing button SB2, the display image HG is displayed on the display 26 of the second terminal device 20.

[0084] FIG. 9 is a schematic diagram showing an example of the virtual space VS displayed by the first terminal devices 10a and 10b when the first user U1 and the third user U3 are participating in a VR meeting. As shown in FIG. 9, the first user U1 can visually recognize the virtual object VO in the virtual space VS by wearing the first terminal device 10a on the head. Similarly, the third user U3 can visually recognize the virtual object VO in the virtual space VS by wearing the first terminal device 10b on the head. The virtual space VS is shared by the first user U1, the second user U2, and the third user U3.

[0085] In the virtual space VS, as virtual objects VO, four slide materials SL1, SL2, SL3, and SL4 and four contents CO1, CO2, CO3, and CO4 are arranged. The first user U1 and the third user U3 are both gazing at the slide material SL2. The gazing position of the first user U1 is represented by GP1. The direction of the line of sight of the first user U1 is represented by LS1. The gazing position of the third user U3 is represented by GP3. Also, the direction of the line of sight of the third user U3 is represented by LS3.

[0086] Note that the direction LS1 and the gazing position GP1 of the line of sight of the first user U1, and the direction LS3 and the gazing position GP3 of the line of sight of the third user U3 are shown for convenience in FIG. 9, but on the display 18 of the first terminal device 10a, the direction LS1 and the gazing position GP1 of the line of sight of the first user U1 are not displayed. Also, on the display 18 of the first terminal device 10b, the direction LS1 and the gazing position GP3 of the line of sight of the third user U3 are not displayed.

[0087] Therefore, the four slide materials SL1 to SL4 and the four contents CO1 to CO4 are visible to the first user U1, but the direction LS1 and the gazing position GP1 of the line of sight of the first user U1 are not visible. However, the first user U1 can view the direction LS3 and the gazing position GP3 of the line of sight of the third user U3 by a predetermined operation.

[0088] The four slide materials SL1 to SL4 and the four contents CO1 to CO4 are visible to the third user U3, but the direction LS3 and the gazing position GP3 of the line of sight of the third user U3 are not visible. However, the third user U3 can view the direction LS1 and the gazing position GP1 of the line of sight of the first user U1 by a predetermined operation.

[0089] Also, the first user U1, the first terminal device 10a, the third user U3, and the first terminal device 10b are shown for convenience in FIG. 9, but are not displayed on the display 18 of the first terminal device 10a and the display 18 of the first terminal device 10b. Therefore, the first user U1 and the third user U3 cannot visually recognize the first user U1, the first terminal device 10a, the third user U3, and the first terminal device 10b.

[0090] Also, a first sharing button SB1 may be displayed on the display 18 of the first terminal device 10a. When the first user U1 bends down, the first sharing button SB1 enters the visual field FOV1 of the first user U1, so the first user U1 can visually recognize the first sharing button SB1 on the display 18.

[0091] The first sharing button SB1 is a button for the first user U1 to share the direction LS1 of the line of sight and the fixation position GP1 of the first user U1 with the second user U2. When the first user U1 wants to tell the second user U2 the direction LS1 of the line of sight and the fixation position GP1 of the first user U1, the first user U1 operates the input device 19 of the first terminal device 10a to press the first sharing button SB1. As a result, a display image HG corresponding to the direction LS1 of the line of sight and the fixation position GP1 of the first user U1 is displayed on the display 26 of the second terminal device 20.

[0092] A third sharing button SB3 may be displayed on the display 18 of the first terminal device 10b. When the third user U3 bends down, the third sharing button SB3 enters the visual field of the third user U3, so the third user U3 can visually recognize the third sharing button SB3 on the display 18.

[0093] The third sharing button SB3 is a button for the third user U3 to share the direction LS3 of the line of sight and the fixation position GP3 of the third user U3 with the second user U2. When the third user U3 wants to inform the second user U2 of the direction LS1 of the line of sight and the fixation position GP3 of the third user U3, the third user U3 operates the input device 19 of the first terminal device 10b to press the third sharing button SB3. As a result, a display image HG corresponding to the direction LS3 of the line of sight and the fixation position GP3 of the third user U3 is displayed on the display 26 of the second terminal device 20.

[0094] Note that the first sharing button SB1 is not visible to the third user U3, and the third sharing button SB3 is not visible to the first user U1.

[0095] FIG. 10 is a schematic diagram showing an example of a virtual space VS displayed on the display 26 of the second terminal device 20 when the first user U1, the second user U2, and the third user U3 are participating in a VR meeting. As shown in FIG. 10, the second user U2 can visually recognize a part of the region in the virtual space VS as a two-dimensional image on the display 26 of the second terminal device 20.

[0096] The entire slide material SL2 and the entire content CO2 are displayed on the display 26. Also, the second sharing button SB2 and the user selection menu SEL are displayed on the display 26. The user selection menu SEL is a menu for activating a function of specifying and displaying on the display 26 a part of the region in the virtual space VS including the position where a specific user among a plurality of users wearing the first terminal device 10 on their heads is currently fixating.

[0097] Buttons indicating each user participating in the VR meeting while wearing the first terminal device 10 on their heads are displayed on the user selection menu SEL. In the present embodiment, the button of the second user U2 himself / herself is not displayed, and the first user button UB1 for selecting the first user U1 and the third user button UB3 for selecting the third user U3 are displayed.

[0098] When the second user U2 selects the first user button UB1, operation information is generated by the second terminal device 20. The generated operation information is transmitted to the server 30. The acquisition unit 311 acquires the generated operation information from the second terminal device 20 via the communication device 33. When the operation information acquired by the acquisition unit 311 includes information indicating that the first user button UB1 has been selected, the acquisition unit 311 acquires the first line-of-sight information from the first terminal device 10a via the communication device 33.

[0099] The calculation unit 312 calculates the line-of-sight direction LS1 and the fixation position GP1 of the first user U1 on the virtual space VS based on the first line-of-sight information acquired by the acquisition unit 311.

[0100] The generation unit 313 generates a display image HG based on the calculated line-of-sight direction LS1 of the first user U1 and the fixation position GP1 of the first user U1.

[0101] The display control unit 314 displays a partial area including the fixation position GP1 of the first user U1 in the virtual space VS on the display 26 of the second terminal device 20, and also displays the display image HG generated by the generation unit 313 on the display 26 of the second terminal device 20.

[0102] On the other hand, when the second user U2 selects the third user button UB3, operation information is generated by the second terminal device 20. The generated operation information is transmitted to the server 30. The acquisition unit 311 acquires the generated operation information from the second terminal device 20 via the communication device 33. When the operation information acquired by the acquisition unit 311 includes information indicating that the third user button UB3 has been selected, the acquisition unit 311 acquires the third line-of-sight information from the first terminal device 10b via the communication device 33.

[0103] The calculation unit 312 calculates the line-of-sight direction LS3 and the fixation position GP3 of the third user U3 on the virtual space VS based on the third line-of-sight information acquired by the acquisition unit 311.

[0104] The generation unit 313 generates a display image HG3 based on the calculated direction LS3 of the line of sight of the third user U3 and the fixation position GP3 of the third user U3.

[0105] As shown in FIG. 10, the display control unit 314 displays a partial area including the fixation position GP3 of the third user U3 in the virtual space VS on the display 26 of the second terminal device 20, and also displays the display image HG3 generated by the generation unit 313 on the display 26 of the second terminal device 20.

[0106] Note that the acquisition unit 311 can determine that the device capable of outputting the first line-of-sight information is the first terminal device 10, and determine that the device incapable of outputting the first line-of-sight information is the second terminal device 20.

[0107] FIG. 11 is a diagram showing an example of an image displayed on the display 26 of the second terminal device 20. As shown in FIG. 11, an asynchronous mode button ASB is displayed on the display 26 of the second terminal device 20. When the asynchronous mode button ASB is pressed by the second user U2, operation information is transmitted from the second terminal device 20 to the server 30.

[0108] The acquisition unit 311 acquires the operation information transmitted from the second terminal device 20. When the operation information acquired by the acquisition unit 311 includes information indicating that the asynchronous mode button ASB has been pressed, the display control unit 314 switches the display on the display 26 to the display for the asynchronous mode.

[0109] FIG. 12 is a diagram showing an example of an image in the asynchronous mode displayed on the display 26 of the second terminal device 20. As shown in FIG. 12, in the asynchronous mode, thumbnail images TH1, TH2, TH3, and TH4 of each page of the slide material are displayed in the left region of the display 26, and the slide material SSL of the selected page is displayed in the right region of the display 26. Further, a normal mode button NMB for switching to the normal mode in which a partial region in the virtual space VS is displayed on the display 26 is displayed at the upper right of the display 26.

[0110] Note that the arrangement of the thumbnail images TH1 to TH4, the selected slide SSL, and the asynchronous mode button ASB is merely an example, and the arrangement within the display 26 is not limited to the above example. For example, the thumbnail images TH1 to TH4 may be arranged in the right region of the display 26, and the slide material SSL of the selected page may be arranged in the left region of the display 26.

[0111] Further, the thumbnail images TH1 to TH4 may be arranged side by side in the upper region of the display 26, or the thumbnail images TH1 to TH4 may be displayed side by side in the lower region of the display 26. Also, the asynchronous mode button ASB may be arranged at any position within the display 26.

[0112] 1.2. Operations of the display control device according to the first embodiment 1.2.1. First operation of the processing device 31 FIG. 13 is a flowchart showing the first operation of the processing device 31 in FIG. 5. Hereinafter, with reference to FIG. 13, the first operation of the processing device 31 will be described. The first operation is an operation when the first shared button SB1 is pressed by the first user U1. The routine in FIG. 13 is started, for example, when the processing device 31 is activated, and is executed every time a certain period of time elapses.

[0113] In step S11, the processing device 31 determines whether it has acquired operation information from the first terminal device 10a by functioning as an acquisition unit 311.

[0114] If the operation information cannot be acquired from the first terminal device 10a, that is, if the determination result in step S11 is negative, the processing device 31 temporarily ends this routine.

[0115] On the other hand, if the operation information is acquired from the first terminal device 10a, that is, if the determination result in step S11 is positive, the processing device 31 determines whether the first shared button SB1 has been pressed in step S12 by functioning as an acquisition unit 311.

[0116] If the first shared button SB1 has not been pressed, that is, if the determination result in step S12 is negative, the processing device 31 temporarily ends this routine.

[0117] On the other hand, if the first shared button SB1 has been pressed, that is, if the determination result in step S12 is positive, the processing device 31 acquires first gaze information from the first terminal device 10a in step S13 by functioning as an acquisition unit 311.

[0118] In step S14, the processing device 31 calculates the direction LS1 and the fixation position GP1 of the gaze of the first user U1 based on the first gaze information by functioning as a calculation unit 312.

[0119] In step S15, the processing device 31 generates an image including a virtual object corresponding to the direction LS1 and the fixation position GP1 of the gaze of the first user U1 as a display image HG by functioning as a generation unit 313.

[0120] In step S16, the processing device 31 functions as the display control unit 314 to display, on the display 26 of the second terminal device 20, an image including a virtual object corresponding to the direction LS1 of the line of sight and the fixation position GP1 of the first user U1 as the display image HG, and once ends this routine.

[0121] 1.2.2. Second operation of the processing device 31 FIG. 14 is a flowchart showing the second operation of the processing device 31 in FIG. 5. Hereinafter, with reference to FIG. 14, the second operation of the processing device 31 will be described. The second operation is an operation when the first user button UB1 is pressed by the second user U2. The routine in FIG. 14 is started, for example, when the processing device 31 is activated and is executed every time a certain period of time elapses.

[0122] In step S21, the processing device 31 functions as the acquisition unit 311 to determine whether operation information has been acquired from the second terminal device 20.

[0123] When the operation information cannot be acquired from the second terminal device 20, that is, when the determination result in step S21 is negative, the processing device 31 once ends this routine.

[0124] On the other hand, when the operation information is acquired from the second terminal device 20, that is, when the determination result in step S21 is negative, the processing device 31 functions as the acquisition unit 311 to determine, in step S22, whether the first user button UB1 or the third user button UB3 has been pressed.

[0125] When neither the first user button UB1 nor the third user button UB3 has been pressed, that is, when the determination result in step S22 is negative, the processing device 31 once ends this routine.

[0126] On the other hand, when the first user button UB1 or the third user button UB3 is pressed, that is, when the determination result in step S22 is affirmative, the processing device 31 functions as an acquisition unit 311 and acquires first gaze information from the first terminal device 10a or 10b in step S23. In the present embodiment, the first user button UB1 is pressed. In this case, the processing device 31 functions as an acquisition unit 311 and acquires first gaze information from the first terminal device 10a in step S23.

[0127] In step S24, the processing device 31 functions as a calculation unit 312 and calculates the direction LS1 of the gaze and the fixation position GP13 of the first user U1 based on the acquired first gaze information.

[0128] In step S15, the processing device 31 functions as a generation unit 313 and generates an image including a virtual object corresponding to the direction LS1 of the gaze and the fixation position GP1 of the first user U1 as a display image HG.

[0129] In step S16, the processing device 31 functions as a display control unit 314 and displays, on the display 26 of the second terminal device 20, an image including a virtual object corresponding to the direction LS1 of the gaze and the fixation position GP1 of the first user U1 as the display image HG, and once ends this routine.

[0130] 1.2.3. Third operation of the processing device 31 FIG. 15 is a flowchart showing a third operation of the processing device 31 in FIG. 5. Hereinafter, with reference to FIG. 15, the third operation of the processing device 31 will be described. The third operation is an operation when the asynchronous mode button ASB is pressed by the second user U2. The routine in FIG. 15 is started, for example, when the processing device 31 is activated and is executed every time a certain period of time elapses.

[0131] In step S31, the processing device 31 functions as an acquisition unit 311 to determine whether operation information has been acquired from the second terminal device 20.

[0132] If the operation information cannot be acquired from the second terminal device 20, that is, if the determination result in step S31 is negative, the processing device 31 temporarily ends this routine.

[0133] On the other hand, if the operation information is acquired from the second terminal device 20, that is, if the determination result in step S31 is positive, the processing device 31 functions as an acquisition unit 311 to determine whether the asynchronous mode button ASB has been pressed in step S32.

[0134] If the asynchronous mode button ASB has not been pressed, that is, if the determination result in step S32 is negative, the processing device 31 temporarily ends this routine.

[0135] On the other hand, if the asynchronous mode button ASB has been pressed, that is, if the determination result in step S32 is positive, the processing device 31 functions as a display control unit 314 to switch the display on the display 26 to the display for the asynchronous mode, and temporarily ends this routine.

[0136] 1.3. Effects of the First Embodiment According to the above description, the processing device 31 of the server 30 according to the first embodiment includes an acquisition unit 311, a calculation unit 312, a generation unit 313, and a display control unit 314. The acquisition unit 311 acquires operation information and first gaze information. The operation information is information indicating that one of the first user U1 and the second user U2 who share the virtual space VS has performed a specific operation. The first gaze information is information regarding the gaze of the first user U1.

[0137] When operation information is acquired, the calculation unit 312 calculates the direction LS1 of the line of sight and the fixation position GP1 of the first user U1 on the virtual space VS based on the acquired first line-of-sight information. The generation unit 313 generates a display image HG. The display image HG is an image including a virtual object on the virtual space VS corresponding to the calculated direction LS1 of the line of sight and the fixation position GP1. The display control unit 314 displays the display image on the display 26 of the second terminal device 20 held by the second user U2.

[0138] According to this aspect, when either the first user U1 or the second user U2 performs a specific operation, a display image HG corresponding to the direction LS1 of the line of sight and the fixation position GP1 of the first user U1 is displayed on the display 26 of the second terminal device 20 held by the second user U2. Therefore, the second user U2 can grasp which content in the virtual space VS the first user U1 is paying attention to.

[0139] Also, the first line-of-sight information is generated by the first terminal device 10a worn on the head of the first user U1. When the first user U1 executes a specific operation, the first line-of-sight information and the operation information are transmitted to the server 30. The acquisition unit 311 acquires the first line-of-sight information and the operation information.

[0140] According to this aspect, it is possible to easily inform the second user U2 of the content that the first user U1 himself / herself is paying attention to. Thereby, the second user U2 can grasp the content that the first user U1 is paying attention to.

[0141] Also, the first line-of-sight information is generated by the first terminal device 10a worn on the head of the first user U1. The virtual space VS is further shared with a third user U3. When the second user U2 selects the first user U1, the second terminal device 20 generates operation information. The first line-of-sight information generated by the first terminal device 10a and the operation information generated by the second terminal device 20 are transmitted to the server 30. The acquisition unit 311 acquires the first line-of-sight information and the operation information.

[0142] According to this aspect, the second user U2 can selectively grasp either the content that the first user U1 is paying attention to or the content that the third user U3 is paying attention to.

[0143] 2. Second Embodiment Hereinafter, with reference to FIGS. 16 to 21, the configuration of the display control device according to the second embodiment of the present invention will be described. In the following description, for the sake of simplicity of explanation, the same reference numerals are used for the same components as those in the first embodiment, and the description of their functions may be omitted. Further, in the following description, for the sake of simplicity of explanation, mainly, the differences between the second embodiment and the first embodiment will be described.

[0144] 2.1. Configuration of the Second Embodiment 2.1.1. Overall Configuration FIG. 16 is a diagram showing the overall configuration of an information processing system 1A including a display control device according to the second embodiment. As shown in FIG. 16, the information processing system 1A is different in that it includes a server 30A instead of the server 30 as compared with the information processing system 1 according to the first embodiment. The server 30A is an example of a display control device.

[0145] 2.1.2. Configuration of the First Terminal Device Since the first terminal device 10 has the same configuration as the first terminal device 10 according to the first embodiment, a detailed description thereof will be omitted.

[0146] 2.1.3. Configuration of the Second Terminal Device Since the second terminal device 20 has the same configuration as the second terminal device 20 according to the first embodiment, a detailed description thereof will be omitted.

[0147] 2.1.4. Configuration of the Server FIG. 17 is a block diagram showing a configuration example of the server 30A in FIG. 16. As shown in FIG. 17, the server 30A is different in that it includes a processing device 31A instead of the processing device 31 and a storage device 32A instead of the storage device 32 as compared with the server 30 according to the first embodiment.

[0148] The processing device 31A is a processor that controls the entire server 30A, and is configured using, for example, one or more chips. The processing device 31A is configured using, for example, a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and registers. Note that some or all of the functions provided in the processing device 31A may be realized by hardware such as a DSP, an ASIC, a PLD, or an FPGA. The processing device 31A executes various processes in parallel or sequentially.

[0149] The storage device 32A is a recording medium that can be read from and written to by the processing device 31A. The storage device 32A includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM, an EPROM, or an EEPROM. The volatile memory is, for example, a RAM.

[0150] The storage device 32A stores a plurality of programs including a control program PR4 to be executed by the processing device 31A. The storage device 32A also functions as a work area for the processing device 31A.

[0151] Since the communication device 33, the display 34, and the input device 35 are the same as the communication device 33, the display 34, and the input device 35 of the first embodiment, respectively, the description thereof is omitted.

[0152] The processing device 31A functions as an acquisition unit 311A, a calculation unit 312A, a generation unit 313A, and a display control unit 314A by, for example, reading and executing the control program PR4 from the storage device 32A.

[0153] The processing device 31A is different from the processing device 31 according to the first embodiment in that it functions as the acquisition unit 311A, the calculation unit 312A, the generation unit 313A, and the display control unit 314A.

[0154] FIG. 18 is a schematic diagram showing an example of a virtual space VS displayed by first terminal devices 10a to 10c when first users U11, U12, and U13 participate in a VR meeting. FIG. 19 is a diagram for explaining a gaze area and area images on the virtual space VS. FIG. 20 is a schematic diagram showing an example of the virtual space VS displayed on a second terminal device 20 when a second user U2 participates in the VR meeting.

[0155] As shown in FIG. 18, first users U11, U12, and U13 can visually recognize a virtual object VO in the virtual space VS by wearing first terminal devices 10a, 10b, and 10c on their heads respectively. The virtual space VS is shared by first users U11, U12, and U13 and a second user U2.

[0156] First users U11, U12, and U13 belong to a first group. The first group is a group of users who visually recognize the virtual space VS three-dimensionally using a first terminal device 10 worn on the head.

[0157] In the virtual space VS, as virtual objects VO, four slide materials SL1, SL2, SL3, and SL4 and four contents CO1, CO2, CO3, and CO4 are arranged. Both first users U11 and U12 are gazing at the slide material SL2. First user U13 is gazing at the slide material SL3. The gaze positions of first users U11, U12, and U13 are represented by GP11, GP12, and GP13 respectively. The directions of the lines of sight of first users U11, U12, and U13 are represented by LS11, LS12, and LS13 respectively.

[0158] Note that the directions LS11 to L13 of the lines of sight of first users U11 to U13 and the gaze positions GP11 to G13 of first users U11 to U13 are shown for convenience in FIG. 18, but these are different from the virtual objects.

[0159] In addition, although the first users U11 to U13 and the first terminal devices 10a to 10c are shown for convenience in FIG. 18, they are not displayed on the respective displays 18 of the first terminal devices 10a to 10c. Therefore, the first users U11 to U13 cannot visually recognize the first users U11 to U13 and the first terminal devices 10a to 10c.

[0160] The display 26 of the second terminal device 20 held by the second user U2 is a flat display. Here, the flat display is a two-dimensional display in which the virtual space VS is visually recognized in two dimensions. As shown in FIG. 19, the second user U2 can visually recognize a partial area within the virtual space VS as a two-dimensional image on the display 26 of the second terminal device 20.

[0161] The entire slide material SL2 and the entire content CO2 are displayed on the display 26. In addition, a second sharing button SB2 and a focus mode button AB2 are displayed on the display 26. The focus mode button AB2 is a button for activating a function of specifying a partial area within the virtual space VS including the slide or content currently being focused on by the first users U11 to U13 and displaying the specified partial area on the display 26.

[0162] When the focus mode button AB2 is pressed by the second user U2, operation information is transmitted from the second terminal device 20 to the server 30. The acquisition unit 311 acquires the operation information transmitted from the second terminal device 20. The operation information is information indicating that any one of the three first users U11 to U13 and the one second user U2 has performed a specific operation.

[0163] When the acquired operation information includes information indicating that the focus mode button AB2 has been pressed, the acquisition unit 311A acquires the first line-of-sight information of the first users U11, U12, and U13 belonging to the first group.

[0164] The calculation unit 312A calculates the line-of-sight directions LS11, LS12, and LS13 and the fixation positions GP11, GP12, and GP13 for each of the first users U11, U12, and U13 belonging to the first group.

[0165] The generation unit 313A includes a determination unit 313a. The determination unit 313a classifies the regions that each of the first users U11, U12, and U13 belonging to the first group is fixating on in the virtual space VS based on the calculation results of the calculation unit 312A. Thereby, the determination unit 313a determines one or more fixation regions on the virtual space VS.

[0166] More specifically, the determination unit 313a classifies the fixation positions GP11, GP12, and GP13 of each of the first users U11, U12, and U13 in the virtual space VS into one or more groups using, for example, a well-known clustering method. If the distance between each pair of fixation positions is within a predetermined distance, they are classified into the same group, and if the distance between each pair of fixation positions is greater than the predetermined distance, they are classified into different groups.

[0167] As shown in FIG. 19, in the present embodiment, two fixation regions GA1 and GA2 are determined. The fixation region GA1 is a region including the fixation positions GP11, GP12, and the slide material SL2. The fixation region GA2 is a region including the fixation position GP13 and the slide material SL3.

[0168] The determination unit 313a determines, among the two determined fixation regions GA1 and GA2, the fixation region that includes the virtual object on which the line of sight of the users belonging to the first group is most concentrated. In the example shown in FIG. 19, the fixation region GA1 is being fixated on by two first users U11 and U12, and the fixation region GA2 is being fixated on by one first user U13. Therefore, in this example, the determination unit 313a determines the region including the virtual object on which the line of sight of the users belonging to the first group is most concentrated as the fixation region GA1.

[0169] The generation unit 313A further generates a region image RG. The region image RG is an image including a virtual object at which the line of sight of the users belonging to the first group is most concentrated. The display control unit 314 further displays the region image RG on the display 26 of the second terminal device 20. In the examples shown in FIGS. 19 and 20, the region image RG is an image showing the fixation region GA1.

[0170] 2.2. Operations of the display control device according to the second embodiment FIG. 21 is a flowchart showing the operations of the processing device 31A in FIG. 17. Hereinafter, the operations of the processing device 31A will be described with reference to FIG. 21. The routine in FIG. 21 is started, for example, when the processing device 31 is activated, and is executed every time a certain period of time elapses.

[0171] In step S41, the processing device 31A determines whether or not it has acquired operation information from the second terminal device 20 by functioning as the acquisition unit 311A.

[0172] If the operation information cannot be acquired from the second terminal device 20, that is, if the determination result in step S41 is negative, the processing device 31A temporarily ends this routine.

[0173] On the other hand, if the operation information is acquired from the second terminal device 20, that is, if the determination result in step S41 is positive, the processing device 31A determines whether or not the attention mode button AB2 has been pressed in step S42 by functioning as the acquisition unit 311A.

[0174] If the attention mode button AB2 has not been pressed, that is, if the determination result in step S42 is negative, the processing device 31A temporarily ends this routine.

[0175] On the other hand, when the attention mode button AB2 is pressed, that is, when the determination result in step S42 is affirmative, the processing device 31A functions as an acquisition unit 311A, and in step S43, acquires first line-of-sight information from the first terminal devices 10a, 10b, and 10c.

[0176] In step S44, the processing device 31A functions as a calculation unit 312A, and calculates the line-of-sight direction LS11 and the fixation position GP11 of the first user U11, the line-of-sight direction LS12 and the fixation position GP12 of the first user U12, and the line-of-sight direction LS13 and the fixation position GP3 of the first user U13 based on the first line-of-sight information.

[0177] In step S45, the processing device 31A functions as a determination unit 313a, and determines the fixation areas of the first users U11 to U13 belonging to the first group.

[0178] In step S46, the processing device 31A functions as a determination unit 313a, and determines the fixation area GA1 including the slide material SL2, which is the virtual object where the lines of sight of the first users U11 to U13 belonging to the first group are most concentrated.

[0179] In step S47, the processing device 31A functions as a generation unit 313A, and generates a region image RG indicating the fixation area GA1 including the slide material SL2 where the lines of sight of the first users U11 to U13 belonging to the first group are most concentrated.

[0180] In step S48, the processing device 31A functions as a display control unit 314A, and displays the region image RG on the display 26 of the second terminal device 20, and once ends this routine.

[0181] 2.3. Effects of the Second Embodiment According to the above description, the first users U11 to U13 belong to the first group that visually recognizes the virtual space VS three-dimensionally using the first terminal devices 10a to 10c worn on their heads. The display 26 of the second terminal device 20 held by the second user U2 is a flat display. The first line-of-sight information is information regarding the lines of sight of the first users U11 to U13 belonging to the first group. The operation information indicates that any one of the first users U11 to U13 belonging to the first group and the second user U2 has performed a specific operation.

[0182] The calculation unit 312A calculates the line-of-sight directions LS11 to LS13 and the fixation positions GP11 to GP13 for the first users U11 to U13 belonging to the first group. The generation unit 313A includes a determination unit 313a.

[0183] Based on the calculation results of the calculation unit 312A, the determination unit 313a determines a fixation area GA1 that includes a second virtual object on which the lines of sight of the first users U11 to U13 belonging to the first group are most concentrated among one or more virtual objects on the virtual space VS.

[0184] The generation unit 313A further generates a region image RG indicating the fixation area GA1 determined by the determination unit 313a. The display control unit 314A further displays the region image RG on the display 26 of the second terminal device 20.

[0185] According to this aspect, the second user U2 can easily grasp the virtual object that is most focused on by the first users U11 to U13 belonging to the first group.

[0186] 3. Third Embodiment Hereinafter, with reference to FIGS. 22 to 24, the configuration of the display control device according to the third embodiment of the present invention will be described. In the following description, for the sake of simplicity of explanation, the same reference numerals are used for the same components as those in the first embodiment and the second embodiment, and the description of their functions may be omitted. Further, in the following description, for the sake of simplicity of explanation, mainly, the differences of the third embodiment compared with the first embodiment and the second embodiment will be described.

[0187] 3.1. Configuration of the Third Embodiment 3.1.1. Overall Configuration FIG. 22 is a diagram showing the overall configuration of an information processing system 1B including a display control device according to the third embodiment. As shown in FIG. 22, the information processing system 1B is different from the information processing system 1 according to the second embodiment in that it includes a server 30B instead of the server 30A. The server 30B is an example of a display control device.

[0188] 3.1.2. Configuration of the First Terminal Device Since the first terminal device 10 has the same configuration as the first terminal device 10 according to the first embodiment, a detailed description thereof will be omitted.

[0189] 3.1.3. Configuration of the Second Terminal Device Since the second terminal device 20 has the same configuration as the second terminal device 20 according to the first embodiment, a detailed description thereof will be omitted.

[0190] 3.1.4. Configuration of the Server FIG. 23 is a block diagram showing a configuration example of the server 30B in FIG. 22. As shown in FIG. 23, the server 30B is different from the server 30 according to the first embodiment in that it includes a processing device 31B instead of the processing device 31 and a storage device 32B instead of the storage device 32.

[0191] The processing device 31B is a processor that controls the entire server 30B, and is configured using, for example, one or more chips. The processing device 31B is configured using, for example, a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, and registers. Note that some or all of the functions provided in the processing device 31B may be realized by hardware such as a DSP, an ASIC, a PLD, or an FPGA. The processing device 31B executes various processes in parallel or sequentially.

[0192] The storage device 32B is a recording medium that can be read from and written to by the processing device 31B. The storage device 32B includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM, an EPROM, or an EEPROM. The volatile memory is, for example, a RAM.

[0193] The storage device 32B stores a plurality of programs including a control program PR5 for execution by the processing device 31B and a machine learning model LM1. The storage device 32B also functions as a work area for the processing device 31B.

[0194] Since the communication device 33, the display 34, and the input device 35 are the same as the communication device 33, the display 34, and the input device 35 of the first embodiment, respectively, the description thereof is omitted.

[0195] The processing device 31B functions as an acquisition unit 311B, a calculation unit 312A, a generation unit 313B, and a display control unit 314A by, for example, reading and executing the control program PR5 from the storage device 32B.

[0196] The processing device 31B is different from the processing device 31 according to the first embodiment in that it functions as an acquisition unit 311B, a calculation unit 312A, a generation unit 313B, and a display control unit 314A, and is different from the processing device 31A according to the second embodiment in that it functions as an acquisition unit 311B and a generation unit 313B.

[0197] The acquisition unit 311B acquires speech information. The speech information is information regarding the speech of the first users U11 to U13 belonging to the first group. The speech information is obtained from the output signals of the microphones 15 of the first terminal devices 10a to 10c that the first users U11 to U13 are respectively wearing on their heads. It is preferable that the speech information is acquired periodically. By acquiring it periodically, information regarding the frequency of speech can be obtained. Also, it is preferable that the speech information is acquired constantly. By acquiring it constantly, in addition to information regarding the frequency of speech, information regarding the duration of speech can be obtained.

[0198] The generation unit 313B includes a determination unit 313a and a specification unit 313b.

[0199] Based on the speech information, the specification unit 313b specifies one or more users who have spoken among the first users U11 to U13 belonging to the first group. For example, the specification unit 313b performs frequency analysis on the output signals of the microphones 15 included in the speech information, and specifies the users who are speaking with the frequency components of human voices as the users who have spoken.

[0200] More specifically, the specification unit 313b performs frequency analysis on the output signals of the microphones 15 and extracts feature amounts. The feature amounts include frequency bands and volume. The specification unit 313b determines whether it is a human voice or not by inputting the extracted feature amounts into a learned machine learning model LM1. The machine learning model LM1 has been pre-trained by associating a plurality of feature amounts with correct labels indicating whether it is a human voice or not.

[0201] When the specification unit 313b determines whether a human voice is included in the output signals corresponding to each user and specifies the users who have spoken, the acquisition unit 311B acquires first line-of-sight information regarding all the specified users. In the present embodiment, the description will continue assuming that all three first users U11, U12, and U13 are speaking.

[0202] Based on the acquired first line-of-sight information, the calculation unit 312A calculates the directions LS11 to LS13 and the fixation positions GP11 to GP13 of the lines of sight of the first users U11 to U13 on the virtual space VS, respectively.

[0203] The determination unit 313a classifies the areas being fixated on by the users identified by the identification unit 313b in the virtual space VS instead of the first users U11 to U13 belonging to the first group, thereby determining the fixation areas GA1 and GA2 on the virtual space VS. In the example of the present embodiment, the determined fixation areas GA1 and GA2 are the same as the fixation areas GA1 and GA2 in the second embodiment.

[0204] The determination unit 313a determines, as the fixation area GA1, the fixation area that includes the virtual object on which the lines of sight of the users belonging to the first group are most concentrated among the two determined fixation areas GA1 and GA2.

[0205] The generation unit 313B further generates the area image RG. The display control unit 314A further displays the area image RG on the display 26 of the second terminal device 20.

[0206] 3.2.2. Operations of the processing device 31B FIG. 24 is a flowchart showing the operations of the processing device 31B in FIG. 23. Hereinafter, the operations of the processing device 31B will be described with reference to FIG. 24. The routine in FIG. 24 is started, for example, when the processing device 31B is activated, and is executed every time a certain period of time elapses. In FIG. 24, the same step numbers are assigned to the same steps as in the flowchart of FIG. 21.

[0207] In step S41, the processing device 31B determines whether it has acquired operation information from the second terminal device 20 by functioning as the acquisition unit 311B.

[0208] If the operation information cannot be acquired from the second terminal device 20, that is, if the determination result in step S41 is negative, the processing device 31B temporarily ends this routine.

[0209] On the other hand, when the processing device 31B acquires operation information from the second terminal device 20, that is, when the determination result in step S41 is affirmative, the processing device 31B functions as an acquisition unit 311B and determines in step S42 whether or not the attention mode button AB2 has been pressed.

[0210] When the attention mode button AB2 has not been pressed, that is, when the determination result in step S42 is negative, the processing device 31B temporarily ends this routine.

[0211] On the other hand, when the attention mode button AB2 has been pressed, that is, when the determination result in step S42 is affirmative, the processing device 31B functions as an acquisition unit 311B and acquires speech information from the first terminal devices 10a, 10b, and 10c in step S51.

[0212] In step S52, the processing device 31B functions as a specifying unit 313b and specifies the user who has spoken based on the acquired speech information. In the present embodiment, it is the user who has spoken among all of the first users U11 to U13.

[0213] In step S43, the processing device 31B functions as an acquisition unit 311B and acquires the first line-of-sight information of the users who have spoken from the first terminal devices 10a, 10b, and 10c, that is, the first users U11 to U13.

[0214] In step S44, the processing device 31B functions as a calculation unit 312A and calculates the direction LS11 and the fixation position GP11 of the line of sight of the first user U11, the direction LS12 and the fixation position GP12 of the line of sight of the first user U12, and the direction LS13 and the fixation position GP3 of the line of sight of the first user U13 based on the first line-of-sight information.

[0215] In step S45, the processing device 31B functions as a determination unit 313a to determine the viewing areas of the first users U11 to U13 belonging to the first group.

[0216] In step S46, the processing device 31B functions as a determination unit 313a to determine a viewing area GA1 including the slide material SL2, which is the virtual object on which the gazes of the first users U11 to U13 belonging to the first group are most concentrated, as the viewing area.

[0217] In step S47, the processing device 31B functions as a generation unit 313B to generate a region image RG indicating the viewing area GA1 including the slide material SL2, on which the gazes of the first users U11 to U13 belonging to the first group are most concentrated.

[0218] In step S48, the processing device 31B functions as a display control unit 314A to display the region image RG on the display 26 of the second terminal device 20 and temporarily end this routine.

[0219] 3.3. Effects of the Third Embodiment

[0220] According to the above description, the acquisition unit 311B acquires speech information regarding the speech of the first users U11 to U13 belonging to the first group. The generation unit 313B includes an identification unit 313b. The identification unit 313b identifies one or more users who have spoken among the first users U11 to U13 belonging to the first group based on the speech information. The determination unit 313a determines a viewing area GA1 including the virtual object on which the gazes of the one or more users identified by the identification unit 313b are most concentrated, instead of the first users U11 to U13 belonging to the first group.

[0221] According to this aspect, the second user U2 can easily grasp the area on the virtual space VS that is most focused on by the speaking user. Therefore, the second user U2 can more accurately grasp the area to be focused on.

[0222] 4. Modification Example The present disclosure is not limited to the embodiments exemplified above. Specific modification modes are exemplified below. Two or more modes arbitrarily selected from the following examples may be combined. Also, the modes of the above embodiments and the following modification modes can be arbitrarily combined as long as they do not conflict with each other.

[0223] 4.1. Modification Example 1 In the first embodiment, the display image HG was an image including the slide material SL3, but the display image HG may be an image showing the direction LS1 of the line of sight of the first user U1 and an image showing the fixation position GP1. In other words, when the first user U1 presses the first sharing button SB1, instead of the image showing the area including the slide material SL3, which is a specified virtual object, an image showing the direction LS1 of the line of sight of the first user U1 and an image showing the fixation position GP1 may be displayed as the display image HG on the display 26 of the second terminal device 20.

[0224] FIG. 25 is a diagram showing an example of the virtual space VS displayed by the second terminal device 20 when the second user U2 is participating in a VR meeting. As shown in FIG. 25, the image showing the direction LS1 of the line of sight of the first user U1 is represented by an arrow, and the fixation position GP1 of the first user U1 is represented by a white circle. Note that the display image HG is not limited to the mode shown in FIG. 25, and may be either the direction LS1 of the line of sight of the first user U1 or the fixation position GP1. Also, as the display image HG, an image showing the direction LS1 of the line of sight of the first user U1 and an image showing the fixation position GP1 may be displayed together with the image showing the area including the slide material SL3, which is a specified virtual object.

[0225] Also, in the first embodiment, the display image HG was an image including the entire slide material SL3, but it may be an image including a part of the slide material SL3. Also, in Modification Example 1, the display image HG was an image showing the direction LS1 of the line of sight of the first user U1 and an image showing the fixation position GP1, but the display image HG may be an image within a certain range centered on the fixation position GP1 of the first user U1.

[0226] 4.2. Modification Example 2 In the first embodiment, as shown in FIG. 8, an image including the slide material SL3 is displayed on the display 26 of the second terminal device 20 held by the second user U2. In addition to or instead of this, the field of view FOV1 of the first user U1 may be displayed on the display 26.

[0227] 4.3. Modification Example 3 In the second and third embodiments, the region image RG was an image including the slide material SL2, but the region image RG may be an image showing the line-of-sight directions LS11 and LS12 of the first users U11 and U12 and the fixation positions GP11 and GP12. Note that the region image RG may be either the line-of-sight directions LS11 and LS12 or the fixation positions GP11 and GP12.

[0228] Also, in the second and third embodiments, the region image RG was an image including the entire slide material SL2, but it may be an image including a part of the slide material SL2. Also, in Modification Example 2, the region image RG was an image showing the line-of-sight direction LS11 of the first user U11, an image showing the line-of-sight direction LS12 of the first user U12, an image showing the fixation position GP11 of the first user U11, and an image showing the fixation position GP12 of the first user U12. Without being limited to this, the region image RG may be an image of a certain range including the fixation positions GP11 and GP12 of the first users U11 and U12.

[0229] 4.4. Modification Example 4 In each of the above-described embodiments, the first terminal device 10 was a head-mounted display capable of experiencing VR content. However, it is not limited to this, and it may be a glass device capable of experiencing other XR (Extended Reality) content such as AR (Augmented Reality) content and MR (Mixed Reality) content. Note that each lens and display panel of the MR glasses is see-through. Therefore, a user wearing the MR glasses can visually recognize the real space through each lens and display of the MR glasses.

[0230] 4.5. Modification Example 5 In the second and third embodiments, a heatmap regarding the gaze position of the users belonging to the first group may be displayed on the display 26. Also by this, the second user U2 can easily grasp the area that the users belonging to the first group are paying attention to.

[0231] 4.6. Modification Example 6 In the first embodiment, the direction LS1 and the gaze position GP1 of the line of sight of the first user U1 were shared with the second user U2, but they may be shared with the third user U3. Also in the second and third embodiments, the directions LS11 to LS13 and the gaze positions GP1 to GP3 of the respective lines of sight of the first users U11 to U13 were shared with the second user U2, but they may be shared among the first users U11 to U13.

[0232] 4.7. Modification Example 7 In the first embodiment, in the asynchronous mode, the display control unit 314 performed control to switch the display of the display 26 of the second terminal device 20 to the display for the asynchronous mode. Instead of this, data of a slide material may be downloaded to the second terminal device 20, and the display control unit 213 of the second terminal device 20 may be configured to control the display of the slide material.

[0233] 4.8. Modification Example 8 In the third embodiment, the speech information was acquired from the first terminal devices 10a to 10c when the attention mode button AB2 was pressed by the second user U2. However, the speech information may be separately acquired from the first terminal devices 10a to 10c periodically and stored in the storage device 32B.

[0234] 5. Others (1) In the above-described embodiments, the storage devices 12, 22, 32, 32A, and 32B were exemplified as ROM and RAM. However, they may be flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, and other appropriate storage media. Also, the program may be transmitted from a network via a telecommunication line. Further, the program may be transmitted from the communication network NET via a telecommunication line.

[0235] (2) In the above-described embodiments, the information, signals, etc. described may be represented using any of various different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0236] (3) In the above-described embodiments, the input / output information, etc. may be stored in a specific location (e.g., memory) or may be managed using a management table. The input / output information, etc. may be overwritten, updated, or appended. The output information, etc. may be deleted. The input information, etc. may be transmitted to other devices.

[0237] (4) In the above-described embodiments, the determination may be made based on a value represented by 1 bit (either 0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0238] (5) The processing procedures, sequences, flowcharts, etc. exemplified in the above-described embodiments may be reordered as long as there is no contradiction. For example, regarding the methods described in the present disclosure, the elements of various steps are presented using an exemplary order and are not limited to the specific order presented.

[0239] (6) Each function exemplified in FIGS. 1 to 25 is realized by any combination of at least one of hardware and software. Also, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one physically or logically combined device, or two or more physically or logically separated devices may be directly or indirectly connected (e.g., using wired, wireless, etc.) and realized using these multiple devices. The functional block may be realized by combining software with the above one device or the above multiple devices.

[0240] (7) The programs exemplified in the above-described embodiments should be broadly interpreted to mean instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, a hardware description language, or another name.

[0241] Also, software, instructions, information, etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cables, fiber optic cables, twisted pairs, digital subscriber lines (DSL), etc.) and wireless technologies (such as infrared, microwaves, etc.), at least one of these wired and wireless technologies is included within the definition of the transmission medium.

[0242] (8) In each of the above-described embodiments, the terms "system" and "network" are used interchangeably.

[0243] (9) The information, parameters, etc. described in the present disclosure may be represented using absolute values, relative values from a predetermined value, or in terms of corresponding other information.

[0244] (10) In the above-described embodiments, the servers 30, 30A, and 30B may include cases where they are mobile stations (MS: Mobile Station). A mobile station may also be referred to by those skilled in the art using terms such as subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other appropriate terms. Also, in the present disclosure, terms such as "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.

[0245] (11) In the above-described embodiments, the terms "connected" and "coupled," or any variations thereof, mean any direct or indirect connection or coupling between two or more elements and can include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements can be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read as "accessed." As used in this disclosure, two elements can be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as, by way of some non-limiting and non-exhaustive examples, electromagnetic energy having wavelengths in the radio frequency region, microwave region, and optical (both visible and invisible) region.

[0246] (12) In the above-described embodiments, the description "based on" does not mean "based only on" unless otherwise specified. In other words, the description "based on" means both "based only on" and "based at least on."

[0247] (13) As used herein, the terms "determining" and "deciding" may encompass a wide variety of operations. "Determining" and "deciding" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up (e.g., searching a table, database, or other data structure), ascertaining, and considering something as having been "determined" or "decided". Also, "determining" and "deciding" may include considering something as having been "determined" or "decided" based on receiving (e.g., receiving information), transmitting (e.g., transmitting information), inputting, outputting, accessing (e.g., accessing data in memory), and the like. Further, "determining" and "deciding" may include considering something as having been "determined" or "decided" based on resolving, selecting, choosing, establishing, comparing, and the like. That is, "determining" and "deciding" may include considering something as having been "determined" or "decided" based on performing some operation. Also, "determining (deciding)" may be replaced with "assuming", "expecting", "considering", etc.

[0248] (14) In the above-described embodiments, when the terms "include", "including", and their variants are used, these terms are intended to be inclusive in the same manner as the term "comprising". Further, the term "or" as used herein is not intended to be exclusive.

[0249] (15) In the present disclosure, for example, when articles are added by translation, such as a, an, and the in English, the present disclosure may include that the nouns following these articles are in the plural form.

[0250] (16) In the present disclosure, the term "A and B are different" may mean that "A and B are different from each other". Note that this term may also mean that "A and B are each different from C". Terms such as "separate", "coupled", etc. may also be interpreted in the same way as "different".

[0251] (17) Each aspect / embodiment described in the present disclosure may be used alone, in combination, or switched and used during execution. Also, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicitly made notification, and may be made implicitly (for example, by not making the notification of the predetermined information).

[0252] As described in detail above regarding the present disclosure, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented as modifications and variations without departing from the spirit and scope of the present disclosure determined by the description of the claims. Therefore, the description of the present disclosure is for illustrative purposes and has no restrictive meaning for the present disclosure.

Description of Reference Numerals

[0253] 10, 10a, 10b, 10c... First terminal device, 20... Second terminal device, 26... Display, 30, 30A, 30B... Server, 311, 311A, 311B... Acquisition unit, 312, 312A... Calculation unit, 313, 313A, 313B... Generation unit, 313a... Determination unit, 313b... Identification unit, 314, 314A... Display control unit, GA1, GA2... Fixation area, GP1, GP11, GP12, GP13, GP3... Fixation position, HG... Display image, LS1, LS11, LS12, LS13, LS3... Direction of line of sight, RG... Region image, U1, U11, U12, U13... First user, U2... Second user, U3... Third user, UB1... First image, UB3... Third image, VS... Virtual space

Claims

1. An acquisition unit that acquires operation information indicating that one of a first user and a second user who share a virtual space has performed a specific operation, and first line-of-sight information regarding the line of sight of the first user; A calculation unit that, when the operation information is acquired, calculates at least one of the direction and the fixation position of the line of sight of the first user on the virtual space based on the acquired first line-of-sight information; A generation unit that generates a first virtual object on the virtual space corresponding to at least one of the calculated direction and fixation position of the line of sight, an image including the first virtual object, the calculated direction of the line of sight, and a display image indicating at least one of the calculated fixation position; A display control unit that displays the display image on a display of a device possessed by the second user; A display control device comprising the above.

2. The first line-of-sight information is generated by a glass device worn on the head of the first user, When the first user executes the specific operation, The first line-of-sight information and the operation information are transmitted to the display control device, The acquisition unit acquires the first line-of-sight information and the operation information, The display control device according to claim 1.

3. The first line-of-sight information is generated by a glass device worn on the head of the first user, The virtual space is further shared with a third user, When the second user selects the first user, the operation information is generated by a device possessed by the second user, The first line-of-sight information generated by the glass device and the operation information generated by a device possessed by the second user are transmitted to the display control device, The acquisition unit acquires the first line-of-sight information and the operation information, The display control device according to claim 1.

4. The first user belongs to a first group that visually recognizes the virtual space in three dimensions using a glass device worn on the head, The display of the device possessed by the second user is a flat display, The first line-of-sight information is information regarding the line of sight of each user belonging to the first group, The operation information indicates that any one of the users belonging to the first group and the second user has performed the specific operation, The calculation unit calculates at least one of the direction of the line of sight and the fixation position for each user belonging to the first group, The generation unit includes a determination unit that determines a fixation area including a second virtual object on which the line of sight of each user belonging to the first group most concentrates among one or more virtual objects on the virtual space based on the calculation result of the calculation unit, Further generates a region image indicating the fixation area determined by the determination unit, The display control unit further displays the region image on the flat display, The display control device according to claim 1.

5. The acquisition unit acquires speech information regarding the speech of each user belonging to the first group, The generation unit includes a specifying unit that specifies one or more users who have spoken among the users belonging to the first group based on the speech information, The determination unit determines a fixation area including the second virtual object on which the line of sight of the one or more users specified by the specifying unit most concentrates instead of each user belonging to the first group, The display control device according to claim 4.

Images