Information processing device, display system, display method, and display program

Abstract

The present invention achieves smooth communication when a plurality of users communicate with each other while viewing the same three-dimensional model in a virtual space. This information processing device is provided with: an authority setting unit that sets operation authority for operating a three-dimensional model in a virtual space to any user; an authorized person image generation unit that, when the operation authority is set to a first user, generates, on the basis of the first user's operation instruction, an authorized person screen including an authorized person image obtained by operating a three-dimensional model in a virtual space based on the first user's viewpoint; a non-authorized person image generation unit that generates a non-authorized person screen including the authorized person image and a non-authorized person image that indicates a virtual space based on the viewpoint of a second user to whom the operation authority has not been set; and a control unit that performs control to display the authorized person screen on a first HMD worn by the first user, and display the non-authorized person screen on a second HMD worn by the second user.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a display system, a display method, and a display program.

Background Art

[0002] There is known a display system in which a plurality of users wear HMDS (Head Mounted Displays) and communicate while viewing the same 3D model in a virtual space in virtual reality or augmented reality. According to this display system, each user can communicate while sharing the same 3D model even in a situation where they cannot share physical objects because other users are remote from each other.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, in the above display system, a plurality of users wearing HMDS view the same 3D model from their respective viewpoints. Therefore, when a plurality of users view different positions from each other, smooth communication becomes difficult.

[0005] Also, in the above display system, a plurality of users operate the same 3D model in a virtual space using their respective controllers. Therefore, when the operation contents conflict with each other, each user cannot perform the desired operation, which hinders smooth communication.

[0006] This disclosure aims to enable smooth communication when multiple users communicate while viewing the same 3D model in a virtual space. [Means for solving the problem]

[0007] The information processing device relating to the first aspect of this disclosure is: A permission setting unit that grants permission to operate a 3D model in a virtual space to any user, When the aforementioned operation privileges are set for the first user, the authorized user image generation unit generates an authorized user image that operates a 3D model in a virtual space based on the first user's viewpoint, based on the operation instructions from the first user, and generates an authorized user screen that includes the authorized user image. An unauthorized user image generation unit generates an unauthorized user screen that includes an unauthorized user image showing a virtual space based on the perspective of a second user for whom the aforementioned operating privileges have not been set, and the aforementioned authorized user image. The system includes a control unit that controls the display of the authorized user screen on a first HMD worn by the first user, and the display of the unauthorized user screen on a second HMD worn by the second user.

[0008] A second aspect of this disclosure is an information processing apparatus described in the first aspect, The aforementioned authorized person image generation unit, If the aforementioned operation privileges are set for the second user, an image of the authorized user is generated, which operates a 3D model in a virtual space based on the second user's viewpoint, based on the operation instructions given by the second user. The aforementioned unauthorized person image generation unit, An image of an unauthorized user is generated that shows a virtual space based on the perspective of a first user for whom the aforementioned operational privileges have not been set. The control unit, The system controls the display of the authorized user screen on the second HMD worn by the second user, and the display of the unauthorized user screen on the first HMD worn by the first user.

[0009] A third aspect of this disclosure is an information processing device described in the first or second aspect, The control unit, When the aforementioned operation authority is set for the first user, control is performed to notify the authorized user image generation unit of the viewpoint data received from the first HMD and the operation instructions from the first user. The aforementioned authorized person image generation unit, The authorized user image is generated by generating an image of the virtual space based on the viewpoint of the first user based on viewpoint data received from the first HMD, and generating an image of the 3D model in the virtual space based on the viewpoint of the first user in operation based on operation instructions from the first user.

[0010] A fourth aspect of this disclosure is an information processing device as described in the second aspect, The control unit, When the aforementioned operation authority is set to the second user, the system controls the system to notify the authorized user image generation unit of the viewpoint data received from the second HMD and the operation instructions from the second user. The aforementioned authorized person image generation unit, The authorized user image is generated by generating an image of the virtual space based on the viewpoint data received from the second HMD, and generating an image of the 3D model in the virtual space based on the viewpoint of the second user in operation based on the operation instructions from the second user.

[0011] A fifth aspect of this disclosure is an information processing apparatus as described in the third aspect, The control unit, When the aforementioned operation authority is set for the first user, the system controls the system to notify the authorized user image generation unit of the location specification instruction made by the second user. The aforementioned authorized person image generation unit, Based on the location specification instructions from the second user, the authorized user image is generated, which includes an indicator showing the location in the virtual space based on the viewpoint of the first user.

[0012] A sixth aspect of the present disclosure is the information processing apparatus according to the fourth aspect, wherein the control unit when the operation authority is set for the second user, controls to notify the position designation instruction by the first user to the authority image generation unit, the authority image generation unit generates the authority image including an indicator indicating a position in the virtual space based on the viewpoint of the second user based on the position designation instruction by the first user.

[0013] A seventh aspect of the present disclosure is the information processing apparatus according to any one of the first to sixth aspects, further comprising a transmission / reception unit that transmits the voice of the first user to the second HMD and transmits the voice of the second user to the first HMD.

[0014] An eighth aspect of the present disclosure is the information processing apparatus according to any one of the first to seventh aspects, wherein the unauthorized person image generation unit when the second user without the set operation authority does not wear the second HMD, generates an unauthorized person screen including the authority image, the control unit controls to display the authority screen on the first HMD worn by the first user and display the unauthorized person screen on a display device visible to the second user.

[0015] A ninth aspect of the present disclosure is the information processing apparatus according to the second aspect, wherein the unauthorized person image generation unit when the first user without the set operation authority does not wear the first HMD, generates an unauthorized person screen including the authority image, the control unit controls to display the authority screen on the second HMD worn by the second user and display the unauthorized person screen on a display device visible to the first user.

[0016] A display system according to a tenth aspect of the present disclosure is The information processing apparatus according to the first or second embodiment, The device comprises the first HMD and the second HMD described in the first or second embodiment.

[0017] The display system relating to the 11th aspect of this disclosure is The information processing apparatus described in the eighth aspect, The present invention comprises the first HMD and the display device described in the eighth embodiment.

[0018] The display system relating to the twelfth aspect of this disclosure is: The information processing apparatus described in the ninth aspect, The present invention comprises the second HMD and the display device described in the ninth embodiment.

[0019] The method of display relating to the 13th aspect of this disclosure is: The process of granting a user the authority to manipulate a 3D model in a virtual space, When the aforementioned operation privileges are set for the first user, the process includes generating an authorized user screen that includes an authorized user image showing a 3D model in a virtual space based on the first user's viewpoint, based on the operation instructions given by the first user. A step of generating a non-authorized user screen that includes an image of a non-authorized user showing a virtual space based on the perspective of a second user for whom the aforementioned operating privileges have not been set, and an image of an authorized user. The computer performs the steps of: displaying the authorized user screen on the first HMD worn by the first user, and controlling the display of the unauthorized user screen on the second HMD worn by the second user.

[0020] The display program relating to the 14th aspect of this disclosure is: The process of granting a user the authority to manipulate a 3D model in a virtual space, When the aforementioned operation privileges are set for the first user, the process includes generating an authorized user screen that includes an authorized user image showing a 3D model in a virtual space based on the first user's viewpoint, based on the operation instructions given by the first user. A step of generating a non-authorized user screen that includes an image of a non-authorized user showing a virtual space based on the perspective of a second user for whom the aforementioned operating privileges have not been set, and an image of an authorized user. The computer is instructed to perform the steps of: displaying the authorized user screen on the first HMD worn by the first user, and displaying the unauthorized user screen on the second HMD worn by the second user. [Effects of the Invention]

[0021] According to this disclosure, smooth communication can be achieved when multiple users communicate while viewing the same 3D model in a virtual space. [Brief explanation of the drawing]

[0022] [Figure 1] Figure 1 is a diagram showing an example of the system configuration of the display system according to the first embodiment. [Figure 2] Figure 2 is the first diagram showing an example of a communication scene using the comparative example display system. [Figure 3] Figure 3 is the second figure showing an example of a communication scene using the comparative example's display system. [Figure 4A] Figure 4A is the first diagram showing an example of the hardware configuration of an information processing device. [Figure 4B] Figure 4B shows an example of the hardware configuration of the first HMD and the second HMD. [Figure 5] Figure 5 is the first diagram showing an example of the functional configuration of an information processing device. [Figure 6] Figure 6 shows a specific example of processing performed by the authorized person image generation unit. [Figure 7] Figure 7 shows a specific example of processing performed by the unauthorized person image generation unit. [Figure 8A] Figure 8A is the first diagram showing a specific example of processing by the control unit. [Figure 8B] Figure 8B is a second diagram showing a specific example of processing by the control unit. [Figure 9] Figure 9 is a first flowchart showing the flow of virtual space display processing by the information processing device. [Figure 10] Figure 10 is a flowchart showing the flow of audio transmission and reception processing by an information processing device. [Figure 11] Figure 11 is the first diagram showing an example of a communication scene when using the display system according to the first embodiment. [Figure 12] Figure 12 is a second diagram showing an example of a communication scene when using the display system according to the first embodiment. [Figure 13] Figure 13 shows an example of the system configuration of the display system according to the second embodiment. [Figure 14A] Figure 14A is a second diagram showing an example of the hardware configuration of an information processing device. [Figure 14B] Figure 14B shows an example of the hardware configuration of an information terminal. [Figure 15] Figure 15 is a second diagram showing an example of the functional configuration of an information processing device. [Figure 16] Figure 16 is a third diagram showing a specific example of processing by the control unit. [Figure 17] Figure 17 is a second flowchart showing the flow of virtual space display processing by the information processing device. [Figure 18] Figure 18 is the first diagram showing an example of a communication scene when using the display system according to the second embodiment. [Figure 19] Figure 19 is a second diagram showing an example of a communication scene when using the display system according to the second embodiment. [Modes for carrying out the invention]

[0023] Each embodiment will be described below with reference to the attached drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions will be omitted.

[0024] [First Embodiment] <System configuration of the display system> First, the system configuration of the display system according to the first embodiment will be described. Figure 1 is a diagram showing an example of the system configuration of the display system according to the first embodiment. As shown in Figure 1, the display system 100 is • Information processing device 110 and, • The first HMD (Head Mounted Display) 121, audio output device 122, audio input device 123, controller 124, • A second HMD (Head Mounted Display) 131, an audio output device 132, an audio input device 133, a controller 134, It holds.

[0025] In the display system 100, the information processing device 110 and the first HMD 121 are connected in a communicative manner. Furthermore, in the display system 100, the information processing device 110 and the second HMD 131 are also connected in a communicative manner. According to the display system 100, the first user 120 and the second user 130 can communicate while viewing the same 3D model in the virtual space.

[0026] The information processing device 110 communicates with the first HMD 121 to generate an image of a virtual space (for example, a virtual space image 140) based on the viewpoint of the first user 120 wearing the first HMD 121. The information processing device 110 also communicates with the first HMD 121 to receive operation instructions for the 3D model 141 contained in the virtual space image 140.

[0027] Similarly, the information processing device 110 communicates with the second HMD 131 to generate an image of the virtual space (e.g., a virtual space image 150) based on the viewpoint of the second user 130 wearing the second HMD 131. The information processing device 110 also communicates with the second HMD 131 to receive operation instructions for the 3D model 151 contained in the virtual space image 150.

[0028] The information processing device 110 receives a position specification instruction that specifies a position within a virtual space image (virtual space image 140 or 150) by communicating with the first HMD 121.

[0029] Similarly, the information processing device 110 receives a position specification instruction that specifies a position within the virtual space image (virtual space image 140 or 150) by communicating with the second HMD 131.

[0030] Furthermore, the authority to manipulate the 3D model in the virtual space is granted to either the first user 120 or the second user 130. The information processing device 110 is configured to indicate whether the authority has been granted to the first user 120 or the second user 130. The information processing device 110 switches its processing depending on whether the authority has been set to the first user 120 or the second user 130.

[0031] When the operation privileges of the information processing device 110 are set to the first user 120, The 3D model 141 within the generated virtual space image 140 is operated according to the operation instructions from the first user 120, and, • A virtual space image in which the indicator (not shown in Figure 1) has been moved to a position corresponding to a position specified by the first user 120 or the second user 130. Generate an image and use it as the authorized user's image.

[0032] Furthermore, if the information processing device 110 does not have operating privileges set to the first user 120, • 140 generated virtual space images, This will be considered an image of an unauthorized person.

[0033] When the information processing device 110 is set to have operation privileges for the second user 130, The 3D model 151 within the generated virtual space image 150 is operated in accordance with the operation instructions from the second user 130, and, • A virtual space image in which the indicator (not shown in Figure 1) has been moved to a position corresponding to a position specified by the first user 120 or the second user 130. It generates an image of the authorized user. Also, if the information processing device 110 does not have operation privileges set for the second user 130, • 150 generated virtual space images, This will be considered an image of an unauthorized person.

[0034] Furthermore, if the operation privileges are set to the first user 120, the information processing device 110 sends an authorized user screen, including an authorized user image generated based on the virtual space image 140, to the first HMD 121. If the operation privileges are not set to the first user, the information processing device 110 sends a non-authorized user screen, including a non-authorized user image generated based on the virtual space image 140 and an authorized user image generated based on the virtual space image 150, to the first HMD 121.

[0035] If the operation privileges are set to the second user 130, the information processing device 110 sends an authorized user screen, including an authorized user image generated based on the virtual space image 150, to the second HMD 131. If the operation privileges are not set to the second user, the information processing device 110 sends a non-authorized user screen, including a non-authorized user image generated based on the virtual space image 150 and an authorized user image generated based on the virtual space image 140, to the second HMD 131.

[0036] Furthermore, the information processing device 110 communicates with the first HMD 121 and the second HMD 131 to send and receive audio data so that the voice of the first user 120, acquired by the audio input device 123, is output by the audio output device 132. The information processing device 110 communicates with the first HMD 121 and the second HMD 131 to send and receive audio data so that the voice of the second user 130, acquired by the audio input device 133, is output by the audio output device 122.

[0037] The first HMD 121 is an HMD worn by the first user 120. The first HMD 121 has various sensors such as an accelerometer, a gyroscope, and a geomagnetic sensor, and acquires viewpoint data of the first user 120 by measuring the position of the first user 120's head, the orientation of the head, the direction of head rotation, etc. The first HMD 121 transmits the acquired viewpoint data to the information processing device 110.

[0038] The first HMD 121 communicates with the voice input device 123 and the voice output device 122. The first HMD 121 acquires voice data from the voice input device 123 and transmits it to the information processing device 110, and also receives voice data from the information processing device 110 and notifies the voice output device 122.

[0039] The first HMD 121 communicates with the controller 124. The first HMD 121 receives operation instructions or position designation instructions from the controller 124 and transmits them to the information processing device 110.

[0040] The first HMD121 has a display device. The first HMD121 displays an authorized user screen or an unauthorized user screen transmitted from the information processing device 110 on the display device.

[0041] The audio output device 122 outputs audio data notified from the first HMD 121. The audio input device 123 acquires the voice of the first user 120 and notifies the first HMD 121. The controller 124 has various sensors such as an accelerometer, gyroscope, and geomagnetic sensor, and measures various sensor data such as the position of the first user 120's hand, the orientation of the hand, and the direction of hand movement, and acquires operation instructions or position specification instructions from the first user 120. The controller 124 notifies the first HMD 121 of the acquired operation instructions or position specification instructions.

[0042] If the operation authority is set to the first user 120, the controller 124 will notify the first HMD 121 of either an operation instruction or a position specification instruction when a toggle switch (not shown) is operated. If the operation authority is not set to the first user 120, the controller 124 will notify the first HMD 121 of a position specification instruction, regardless of whether the toggle switch (not shown) is operated or not.

[0043] The second HMD 131 is an HMD worn by the second user 130. The second HMD 131 has various sensors such as an accelerometer, a gyroscope, and a geomagnetic sensor, and acquires viewpoint data of the second user 130 by measuring the position of the second user 130's head, the orientation of the head, the direction of head rotation, etc. The second HMD 131 transmits the acquired viewpoint data to the information processing device 110.

[0044] The second HMD 131 communicates with the voice input device 133 and the voice output device 132. The second HMD 131 acquires voice data from the voice input device 133 and transmits it to the information processing device 110, and also receives voice data from the information processing device 110 and notifies the voice output device 132.

[0045] The second HMD 131 communicates with the controller 134. The second HMD 131 receives operation instructions or position designation instructions from the controller 134 and transmits them to the information processing device 110.

[0046] The second HMD131 has a display device. The second HMD131 displays an authorized user screen or an unauthorized user screen transmitted from the information processing device 110 on the display device.

[0047] The audio output device 132 outputs audio data notified from the second HMD 131. The audio input device 133 acquires the voice of the second user 130 and notifies the second HMD 131. The controller 134 has various sensors such as an accelerometer, gyroscope, and geomagnetic sensor, and measures various sensor data such as the position, orientation, and direction of movement of the second user 130's hand, and acquires operation instructions or position specification instructions from the second user 130. The controller 134 notifies the second HMD 131 of the acquired operation instructions or position specification instructions.

[0048] If the operation authority is set to the second user 130, the controller 134 will notify the second HMD 131 of either an operation instruction or a position specification instruction when a toggle switch (not shown) is operated. If the operation authority is not set to the second user 130, the controller 134 will notify the second HMD 131 of a position specification instruction, regardless of whether the toggle switch (not shown) is operated or not.

[0049] The above explanation assumes that the first user 120 and the second user 130 are in positions where they cannot communicate directly, and that voice data is transmitted and received via the information processing device 110. However, if, for example, the first user 120 and the second user 130 are in the same room and can communicate directly, then the voice output device 122, voice input device 123, voice output device 132, and voice input device 133 are not essential components.

[0050] <Communication scenario using the comparative example display system> Next, using the comparative example display system, we will explain the problems that arise when a first user 120 and a second user 130 communicate while viewing the same 3D model in a virtual space.

[0051] Figure 2 is the first diagram showing an example of a communication scene using the comparative example display system.

[0052] In Figure 2, the virtual space image 210 is an example of a virtual space image displayed on the display device of the first HMD 121 in the comparative example's display system. In the comparative example's display system, the display device of the first HMD 121 displays a virtual space image 210 based on the viewpoint of the first user 120, where the indicator 211 is moved to a position corresponding to a position specification instruction transmitted from the first HMD 121.

[0053] Furthermore, in Figure 2, the virtual space image 220 is an example of a virtual space image displayed on the display device of the second HMD 131 in the comparative example's display system. As shown in Figure 2, in the comparative example's display system, the display device of the second HMD 131 displays a virtual space image 220 based on the viewpoint of the second user 130.

[0054] Under these display conditions, suppose the first user 120 says, for example, "I want to change the material of a specific face of the 3D model 141 indicated by the indicator 211." In this case, the second user 130 cannot understand which face of the 3D model 151 the first user 120 wants to change the material of, and therefore cannot communicate smoothly.

[0055] The virtual space image 220 viewed by the second user 130 is a virtual space image based on the second user 130's viewpoint, and the second user 130 cannot see the 3D model 141 to which the first user 120 pointed out a specific face. Furthermore, the indicator 211 pointed out by the first user 120 is not displayed in the virtual space image 220 viewed by the second user 130. Therefore, the second user 130 cannot recognize which face of the 3D model 151 corresponds to the face pointed out by the first user 120.

[0056] Figure 3 is the second figure showing an example of a communication scene using the comparative example's display system.

[0057] In Figure 3, the virtual space image 310 is an example of a virtual space image displayed on the display device of the first HMD 121 in the comparative example's display system. As shown in Figure 3, in the comparative example's display system, the display device of the first HMD 121 displays a virtual space image 310 based on the viewpoint of the first user 120. For example, if the first user 120 tries to view the window side of the 3D model 141 in the virtual space, the first user 120 moves the controller 124 in a counterclockwise arc, rotating the 3D model 141 90 degrees counterclockwise.

[0058] On the other hand, in Figure 3, the virtual space image 320 is an example of a virtual space image displayed on the display device of the second HMD 131 in the comparative example's display system. As shown in Figure 3, in the comparative example's display system, the display device of the second HMD 131 displays the virtual space image 320 based on the viewpoint of the second user 130. For example, if the second user 130 tries to view the window side of the 3D model 151 in the virtual space, the second user 130 moves the controller 134 in an arc from back to front, rotating the 3D model 151 180 degrees vertically.

[0059] Thus, in situations where the operations of the first user 120 and the second user 130 conflict, neither the first user 120 nor the second user 130 can move the 3D model 141 or the 3D model 151 in the desired direction. As a result, neither the first user 120 nor the second user 130 can see the surface they are trying to view, and smooth communication becomes impossible. This is because when multiple users simultaneously give different operation instructions to the same 3D model, that same 3D model will move in the direction of the sum of all the operation instructions.

[0060] Thus, when the first user 120 and the second user 130 communicate while viewing the same 3D model in the virtual space using the comparative example display system, • A scene in which either user specifies a location, or • Scenes where multiple users simultaneously issue instructions. In such situations, problems arise, such as the inability to communicate smoothly.

[0061] The display system 100 according to the first embodiment aims to solve these problems and enable smooth communication when multiple users communicate while viewing the same 3D model in a virtual space. The details of the display system 100 according to the first embodiment will be described below.

[0062] <Hardware configuration of the information processing device> First, the hardware configuration of the information processing device 110 will be described. Figure 4A is the first diagram showing an example of the hardware configuration of the information processing device. As shown in Figure 4A, the information processing device 110 includes a processor 401, memory 402, auxiliary storage device 403, input / output device 404, communication device 405, and drive device 406. Each piece of hardware of the information processing device 110 is interconnected via a bus 407.

[0063] The processor 401 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 401 reads various programs into memory 402 and executes them. These various programs include, for example, display programs, which will be described later.

[0064] Memory 402 has main memory devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 401 and memory 402 form a so-called computer. This computer realizes its various functions by the processor 401 executing various programs read from memory 402.

[0065] The auxiliary storage device 403 stores various programs and various data used when those programs are executed by the processor 401.

[0066] The input / output device 404 is a connection device that connects to the operating device 411 and the display device 412, which are examples of user interface devices. The communication device 405 is a communication device for communicating with the first HMD 121 and the second HMD 131 via a network.

[0067] The drive device 406 is a device for setting the recording medium 420. The recording medium 420 here includes media for recording information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 420 may also include media for recording information electrically, such as semiconductor memory, such as ROMs and flash memory.

[0068] The various programs to be installed on the auxiliary storage device 403 are installed, for example, when the distributed recording medium 420 is set in the drive device 406 and the various programs recorded on the recording medium 420 are read. Alternatively, the various programs to be installed on the auxiliary storage device 403 may be installed by downloading them from the network via the communication device 405.

[0069] <Hardware configuration of the first and second HMDs> Next, the hardware configurations of the first HMD 121 and the second HMD will be described. Figure 4B shows an example of the hardware configurations of the first HMD and the second HMD. As shown in Figure 4B(a), the first HMD 121 includes a processor 431, memory 432, auxiliary storage device 433, sensor 434, display device 435, wireless device 436, and communication device 437. Each piece of hardware in the first HMD 121 is interconnected via a bus 438.

[0070] The processor 431 has various computing devices such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 431 reads various programs into memory 432 and executes them.

[0071] Memory 432 has main memory devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The processor 431 and memory 432 form a so-called computer. This computer realizes its various functions by the processor 431 executing various programs read from memory 432.

[0072] The auxiliary storage device 433 stores various programs and various data used when those programs are executed by the processor 431.

[0073] Sensor 434 includes various sensors such as the acceleration sensor, gyroscope, and geomagnetic sensor mentioned above, and acquires viewpoint data from the first user 120. Display device 435 displays the authorized user screen or the unauthorized user screen mentioned above.

[0074] The wireless device 436 is a device for transmitting and receiving voice data, operation instructions, or location designation instructions between the voice output device 122, the voice input device 123, the controller 124, and the first HMD 121.

[0075] The communication device 437 is a device for transmitting and receiving viewpoint data, authorized user screens or unauthorized user screens, audio data, operation instructions, or location specification instructions between the information processing device 110 and the first HMD 121.

[0076] The various programs installed in the auxiliary storage device 433 are installed, for example, by being downloaded from the network via the communication device 437.

[0077] As shown in Figure 4B(b), the second HMD131 includes a processor 441, memory 442, auxiliary storage device 443, sensor 444, display device 445, wireless device 446, and communication device 447. The hardware components of the second HMD131 are interconnected via a bus 448. Since the hardware components of the second HMD131 are the same as those of the first HMD121, their description is omitted here.

[0078] <Functional Configuration of Information Processing Devices> Next, the functional configuration of the information processing device 110 will be described. Figure 5 is the first diagram showing an example of the functional configuration of the information processing device. As described above, a display program is installed in the information processing device 110, and by executing this program, the information processing device 110 realizes each of the functional units shown in Figure 5.

[0079] As shown in Figure 5, the functional units implemented in the information processing device 110 include a virtual space image provision unit 501, an authorized user image generation unit 511, and an unauthorized user image generation unit 512. Furthermore, the functional units implemented in the information processing device 110 include an authorized user viewpoint data acquisition unit 521, an authorized user instruction reception unit 522, an unauthorized user instruction reception unit 523, and an unauthorized user viewpoint data acquisition unit 524. Additionally, the functional units implemented in the information processing device 110 include an authorization setting unit 531, a control unit 532, and an audio data transmission / reception unit 540.

[0080] The virtual space image provider unit 501 provides a base virtual space image to the authorized user image generation unit 511 and the unauthorized user image generation unit 512. The virtual space image provided by the virtual space image provider unit 501 includes a three-dimensional model.

[0081] The authorized user image generation unit 511 obtains viewpoint data of users who have been granted operational privileges from the authorized user viewpoint data acquisition unit 521. The authorized user image generation unit 511 generates a virtual space image based on the acquired viewpoint data from the virtual space image provided by the virtual space image provision unit 501.

[0082] The authorized user image generation unit 511 receives operation instructions from the authorized user instruction receiving unit 522 from a user who has been granted operation privileges. The authorized user image generation unit 511 generates a virtual space image after the 3D model has been operated by operating the 3D model included in the virtual space image based on viewpoint data in accordance with the operation instructions.

[0083] The authorized user image generation unit 511 receives a location specification instruction from the unauthorized user instruction reception unit 523 for a user who does not have operation privileges set. The authorized user image generation unit 511 generates a virtual space image in the virtual space image after the 3D model operation, with the indicator moved to the position corresponding to the location specification instruction.

[0084] The authorized user image generation unit 511 notifies the unauthorized user image generation unit 512 of the virtual space image with the indicator moved as the authorized user image. The authorized user image generation unit 511 also generates an authorized user screen including the virtual space image with the indicator moved and notifies the control unit 532.

[0085] The authorized user viewpoint data acquisition unit 521 acquires viewpoint data of users who have been granted operational privileges and notifies the authorized user image generation unit 511.

[0086] The authorized user instruction receiving unit 522 obtains operation instructions from a user who has been granted operation privileges and notifies the authorized user image generation unit 511.

[0087] The non-authorized user instruction receiving unit 523 obtains a location specification instruction from a user for whom no operational authority has been set and notifies the authorized user image generation unit 511.

[0088] The unauthorized user image generation unit 512 acquires viewpoint data of users who do not have operational privileges set from the unauthorized user viewpoint data acquisition unit 524. The unauthorized user image generation unit 512 generates a virtual space image based on the acquired viewpoint data from the virtual space image provided by the virtual space image provision unit 501, and uses this as the unauthorized user image. The unauthorized user image generation unit 512 also acquires an authorized user image from the authorized user image generation unit 511. The unauthorized user image generation unit 512 generates an unauthorized user screen that includes both the authorized user image and the unauthorized user image, and notifies the control unit 532.

[0089] The unauthorized user viewpoint data acquisition unit 524 acquires viewpoint data of users for whom operation privileges were not set and notifies the unauthorized user image generation unit 512.

[0090] The authorization setting unit 531 sets the operation privileges for manipulating the 3D model in the virtual space to either the first user 120 or the second user 130. The authorization setting unit 531 notifies the control unit 532 of the user to whom operation privileges have been set.

[0091] The control unit 532 switches the notification destination or transmission destination of viewpoint data, authorized user screen or unauthorized user screen, operation instructions or location specification instructions transmitted and received between the first HMD 121 and the second HMD 131, according to the set operation permissions.

[0092] The audio data transmission / reception unit 540 transmits and receives audio data between the first HMD 121 and the second HMD.

[0093] <Details of processing performed by each functional unit of the information processing device> Next, we will explain in detail the processing performed by each functional unit of the information processing device 110 (in this case, the authorized person image generation unit 511, the unauthorized person image generation unit 512, and the control unit 532).

[0094] (1) Details of processing by the authorized person image generation unit 511 First, the details of the processing performed by the authorized person image generation unit 511 will be explained. Figure 6 shows a specific example of the processing performed by the authorized person image generation unit. As shown in Figure 6, the authorized person image generation unit 511 further includes a virtual space image acquisition unit 601, a viewpoint change unit 602, a model control unit 603, an indicator control unit 604, and an authorized person screen output unit 605.

[0095] The virtual space image acquisition unit 601 acquires the virtual space image provided by the virtual space image provision unit 501 and notifies the viewpoint change unit 602.

[0096] The viewpoint change unit 602 acquires viewpoint data of a user for whom operation privileges have been set from the authorized user viewpoint data acquisition unit 521. Here, it is assumed that operation privileges are set for the first user 120. The viewpoint change unit 602 generates a virtual space image 610 based on the viewpoint data of the first user 120. The virtual space image 610 includes, in addition to the 3D model 141, an indicator 611 that moves according to the operation instructions or position specification instructions of the first user 120, and an indicator 612 that moves according to the position specification instructions of the second user 130.

[0097] The model control unit 603 receives operation instructions from the authorized user instruction receiving unit 522 for the first user 120. The model control unit 603 generates a virtual space image 620 after the operation of the 3D model 141 by operating the 3D model 141 contained in the virtual space image 610 in accordance with the operation instructions of the first user 120.

[0098] The indicator control unit 604 receives a location specification instruction from the unauthorized user instruction reception unit 523 for the second user 130. The indicator control unit 604 moves the indicator 612 included in the virtual space image 620 according to the location specification instruction from the second user 130, thereby generating a virtual space image with the indicator 612 moved, which is designated as the authorized user image 630. The indicator control unit 604 notifies the unauthorized user image generation unit 512 of the authorized user image 630, and also notifies the authorized user screen output unit 605.

[0099] The authorized user screen output unit 605 generates an authorized user screen 640 including the authorized user image 630 and notifies the control unit 532.

[0100] (2) Details of processing by the unauthorized person image generation unit 512 Next, the details of the processing performed by the unauthorized person image generation unit 512 will be described. Figure 7 is a diagram showing a specific example of the processing performed by the unauthorized person image generation unit. As shown in Figure 7, the unauthorized person image generation unit 512 further includes a virtual space image acquisition unit 701, a viewpoint change unit 702, and an unauthorized person screen output unit 703.

[0101] The virtual space image acquisition unit 701 acquires the virtual space image provided by the virtual space image provision unit 501 and notifies the viewpoint change unit 702.

[0102] The viewpoint change unit 702 acquires viewpoint data of the second user 130, for whom no operational privileges have been set, from the unauthorized user viewpoint data acquisition unit 524. The viewpoint change unit 702 generates an unauthorized user image 710 based on the viewpoint data of the second user 130 and notifies the unauthorized user screen output unit 703.

[0103] The unauthorized user screen output unit 703 generates an unauthorized user screen 720 using the authorized user image 630 notified by the authorized user image generation unit 511 and the unauthorized user image 710 notified by the viewpoint change unit 702, and notifies the control unit 532. As shown in Figure 7, the unauthorized user screen 720 includes a main area and a sub-area, with the authorized user image 630 displayed in the main area and the unauthorized user image 710 displayed in the sub-area.

[0104] (3) Details of processing by the control unit 532 Next, the details of the processing performed by the control unit 532 will be described. Figure 8A is the first diagram showing a specific example of processing performed by the control unit. As described above, the control unit 532 switches the notification destination or transmission destination for viewpoint data, authorized user screen or unauthorized user screen, operation instructions or location specification instructions according to the set operation authority. Figure 8A shows the switch to the notification destination or transmission destination when the operation authority is set to the first user 120.

[0105] Specifically, when the operating authority is set to the first user 120, the control unit 532 will: The viewpoint data transmitted from the first HMD121 is notified to the authorized user viewpoint data acquisition unit 521. The operation instruction or location specification instruction transmitted from the first HMD121 is notified to the authorized user instruction receiving unit 522. The location specification instruction transmitted from the second HMD131 is notified to the unauthorized user instruction receiving unit 523. The authorized user screen generated by the authorized user image generation unit 511 is transmitted to the first HMD 121.

[0106] Furthermore, if the operating authority is set to the first user 120, the control unit 532 will The viewpoint data transmitted from the second HMD131 is notified to the unauthorized user viewpoint data acquisition unit 524. The unauthorized user screen generated by the unauthorized user image generation unit 512 is transmitted to the second HMD 131.

[0107] Figure 8B is a second diagram showing a specific example of processing by the control unit. As described above, the control unit 532 switches the notification destination or transmission destination for viewpoint data, authorized user screen or unauthorized user screen, operation instructions or location specification instructions according to the set operation authority. Figure 8B shows the switch to the notification destination or transmission destination when the operation authority is set to the second user 130.

[0108] Specifically, when the operating authority is set to the second user 130, the control unit 532 will: The viewpoint data transmitted from the second HMD131 is notified to the authorized user viewpoint data acquisition unit 521. The operation instruction or location specification instruction transmitted from the second HMD131 is notified to the authorized user instruction receiving unit 522. The location specification instruction transmitted from the first HMD121 is notified to the unauthorized user instruction receiving unit 523. The authorized user screen generated by the authorized user image generation unit 511 is transmitted to the second HMD 131.

[0109] Furthermore, if the operation authority is set to the second user 130, the control unit 532 will The viewpoint data transmitted from the first HMD121 is notified to the unauthorized user viewpoint data acquisition unit 524. The unauthorized user screen generated by the unauthorized user image generation unit 512 is transmitted to the first HMD 121.

[0110] <Flow of virtual space display processing by information processing device> Next, we will explain the flow of virtual space display processing by the information processing device 110. Figure 9 is the first flowchart showing the flow of virtual space display processing by the information processing device.

[0111] In step S901, the information processing device 110 accepts the setting of operation privileges.

[0112] In step S902, the information processing device 110 switches the notification destination or transmission destination of viewpoint data, authorized user screen or unauthorized user screen, operation instructions or location specification instructions transmitted between the first HMD 121 and the second HMD 131, according to the set operation authority.

[0113] In step S903, the information processing device 110 acquires a virtual space image.

[0114] In step S904, the information processing device 110 obtains viewpoint data of users for whom operation privileges have been set, operation instructions of users for whom operation privileges have been set, and location specification instructions of users for whom operation privileges have not been set.

[0115] In step S905, the information processing device 110 generates an authorized person image based on the viewpoint data, operation instructions, and position specification instructions acquired in step S904.

[0116] In step S906, the information processing device 110 acquires viewpoint data for users for whom operation privileges were not set.

[0117] In step S907, the information processing device 110 generates an image of an unauthorized person based on the viewpoint data acquired in step S906.

[0118] In step S908, the information processing device 110 generates an authorized user screen including an authorized user image and sends the authorized user screen to a user who has been granted operational privileges.

[0119] In step S909, the information processing device 110 generates a non-authorized user screen that includes an authorized user image and a non-authorized user image, and sends the non-authorized user screen to users for whom operation privileges have not been set.

[0120] In step S910, the information processing device 110 determines whether or not to terminate the virtual space display process. If it determines in step S910 to continue the virtual space display process (if the answer is NO in step S910), it returns to step S903. On the other hand, if it determines in step S910 to terminate the virtual space display process (if the answer is YES in step S910), it terminates the virtual space display process.

[0121] <Flowchart of voice transmission and reception processing by information processing equipment> Next, we will explain the flow of voice transmission and reception processing by the information processing device 110. Figure 10 is a flowchart showing the flow of voice transmission and reception processing by the information processing device.

[0122] In step S1001, the information processing device 110 receives the voice of the first user 120 acquired by the voice input device 123 from the first HMD 121 and transmits it to the second HMD 131, thereby causing the voice output device 132 to output the voice of the first user 120.

[0123] In step S1002, the information processing device 110 receives the voice of the second user 130 acquired by the voice input device 133 from the second HMD 131 and transmits it to the first HMD 121, thereby causing the voice output device 122 to output the voice of the second user 130.

[0124] Note that the order of the processes in step S1001 and step S1002 may be reversed. Alternatively, the processes in step S1001 may be executed in parallel with the processes in step S1002.

[0125] In step S1003, the information processing device 110 determines whether or not to terminate the voice transmission and reception process. If it determines in step S1003 to continue the voice transmission and reception process (if the answer is NO in step S1003), it returns to step S1001. On the other hand, if it determines in step S1003 to terminate the voice transmission and reception process (if the answer is YES in step S1003), it terminates the voice transmission and reception process.

[0126] <Communication scene using the display system according to the first embodiment> Next, we will describe a scenario in which a first user 120 and a second user 130 communicate while viewing the same 3D model in a virtual space using the display system 100 according to the first embodiment.

[0127] Figure 11 is a first diagram showing an example of a communication scene when using the display system according to the first embodiment, and shows an example of a communication scene when the operation authority is set to the first user 120. As shown in Figure 11, in the case of the display system 100, the display device 435 of the first HMD 121 displays an authorized user screen 640 including an authorized user image 1101. Also in the case of the display system 100, the display device 445 of the second HMD 131 displays a non-authorized user screen 720 including an authorized user image 1101 and a non-authorized user image 1102. However, in the display device 445 of the second HMD 131, the non-authorized user screen 720 may not include the non-authorized user image 1102, depending on the non-authorized user's selection or the default setting.

[0128] Under these display conditions, suppose the first user 120 says, for example, "I want to change the material of this surface." In this case, the second user 130 can see the authorized user image 1101 included in the unauthorized user screen 720 and understand that the surface whose material the first user 120 wishes to change is the surface indicated by the first user 120's indicator. As a result, the first user 120 and the second user 130 can achieve smooth communication.

[0129] Although not shown in Figure 11, let's assume that the second user 130, while viewing the authorized user image 1101 included in the unauthorized user screen 720, moves the second user 130's indicator to point to a different surface than the one pointed to by the first user 120's indicator.

[0130] Under these display conditions, suppose the second user 130 says, for example, "I want to change the material of this surface." In this case, the first user 120 can see the authoritative image 1101 included in the authoritative screen 640 and understand that the surface whose material the second user 130 wishes to change is the surface indicated by the second user 130's indicator. As a result, the first user 120 and the second user 130 can achieve smooth communication.

[0131] Figure 12 is a second diagram showing an example of a communication scene when using the display system according to the first embodiment, and shows an example of a communication scene when the operation privileges are set to the first user 120. As shown in Figure 12, in the case of the display system 100, operation privileges are set for only one user, and multiple users cannot have operation privileges at the same time.

[0132] Therefore, for example, even if the first user 120 and the second user 130 attempt to view the window-side surface of a 3D model in the virtual space, only the first user 120 can operate the 3D model. In other words, in the case of the display system 100 according to the first embodiment, it is possible to avoid situations where different operation instructions are given simultaneously.

[0133] Furthermore, the first user 120 rotates the 3D model 90 degrees counterclockwise by moving the controller 124 in a counterclockwise arc. This allows the first user 120 to view a desired face of the 3D model in the virtual space by viewing the authorized user image 1201' contained in the authorized user screen 640. In addition, the second user 130 can view a desired face of the 3D model in the virtual space by viewing the authorized user image 1201' contained in the unauthorized user screen 720.

[0134] <Summary> As is clear from the above description, the information processing device 110 according to the first embodiment is • Grant operational privileges to manipulate the 3D model within the virtual space to either User 120 (First User) or User 130 (Second User). If the operation privileges are set to the first user 120, an image of the authorized user is generated based on the operation instructions from the first user 120, which shows a 3D model in the virtual space based on the viewpoint of the first user 120, and an authorized user screen is generated that includes the image of the authorized user. • Generate a non-authorized user screen that includes a non-authorized user image showing the virtual space from the perspective of a second user 130 who does not have operational privileges, and an authorized user image. The system controls the display of the authorized user screen on the first HMD 121 worn by the first user 120, and the unauthorized user screen on the second HMD 131 worn by the second user 130.

[0135] Thus, in the information processing device 110 according to the first embodiment, operation privileges are set for one of the users, and the privileged user image is shared among multiple users. As a result, according to the first embodiment, smooth communication can be achieved when multiple users communicate while viewing the same 3D model in a virtual space.

[0136] [Second Embodiment] In the first embodiment described above, a communication scenario was described in which the first user 120 wears the first HMD 121 and the second user 130 wears the second HMD 131. In contrast, the second embodiment describes a communication scenario in which the first user 120 wears the first HMD and the second user 130 does not wear the second HMD 131 and views the display device. The explanation will focus on the differences from the first embodiment, and in the second embodiment, operation privileges are set only for the first user 120.

[0137] <System configuration of the display system> First, the system configuration of the display system according to the second embodiment will be described. Figure 13 is a diagram showing an example of the system configuration of the display system according to the second embodiment. As shown in Figure 13, the display system 1300 is • Information processing device 110 and, • The first HMD 121, audio output device 122, audio input device 123, controller 124, • Information terminal 1301, audio output device 132, audio input device 133, controller 134, It holds.

[0138] In the display system 1300, the information processing device 110 and the information terminal 1301 are connected in a communicative manner. According to the display system 1300, the first user 120 and the second user 130 can communicate with each other while viewing a 3D model in a virtual space.

[0139] The information processing device 110 receives a location specification instruction that specifies a location within the virtual space image 140 by communicating with the information terminal 1301.

[0140] The information processing device 110 transmits a screen for unauthorized users, including an image of an authorized user generated based on the virtual space image 140, to the information terminal 1301.

[0141] The information processing device 110 communicates with the first HMD 121 and the information terminal 1301 to send and receive audio data so that the voice of the first user 120, acquired by the voice input device 123, is output by the audio output device 132. The information processing device 110 communicates with the first HMD 121 and the information terminal 1301 to send and receive audio data so that the voice of the second user 130, acquired by the voice input device 133, is output by the audio output device 122.

[0142] The information terminal 1301 has a display device that is visible to the second user 130. The information terminal 1301 displays the unauthorized user screen transmitted from the information processing device 110 on the display device.

[0143] The information terminal 1301 communicates with the voice input device 133 and the voice output device 132. The information terminal 1301 acquires voice data from the voice input device 133 and transmits it to the information processing device 110, and also receives voice data from the information processing device 110 and notifies the voice output device 132.

[0144] The information terminal 1301 communicates with the controller 134. The information terminal 1301 receives a location specification instruction from the controller 134 and transmits it to the information processing device 110.

[0145] The voice output device 132 outputs voice data notified from the information terminal 1301. The voice input device 133 acquires the voice of the second user 130 and notifies the information terminal 1301. The controller 134 has various sensors such as an accelerometer, gyroscope, and geomagnetic sensor, and measures various sensor data such as the position of the second user 130's hand, the orientation of the hand, and the direction of hand movement, and acquires the position specification instruction from the second user 130. The controller 134 notifies the information terminal 1301 of the acquired position specification instruction.

[0146] <Hardware configuration of the information processing device> Next, the hardware configuration of the information processing device 110 will be described. Figure 14A is a second diagram showing an example of the hardware configuration of the information processing device. The difference from the hardware configuration of the information processing device 110 shown in Figure 4A is that in the hardware configuration shown in Figure 14A, the communication device 405 communicates with the first HMD 121 and the information terminal 1301.

[0147] <Hardware configuration of information terminals> Next, the hardware configuration of the information terminal 1301 will be described. Figure 14B shows an example of the hardware configuration of the information terminal. As shown in Figure 14B, the information terminal 1301 has a processor 1401, memory 1402, auxiliary storage device 1403, display device 1404, wireless device 1405, and communication device 1406. Each piece of hardware of the information terminal 1301 is interconnected via a bus 1407.

[0148] Since the hardware components of the information terminal 1301 are generally the same as those of the second HMD 131, a detailed explanation of each component will be omitted here.

[0149] <Functional Configuration of Information Processing Devices> Next, the functional configuration of the information processing device 110 will be described. Figure 15 is a second diagram showing an example of the functional configuration of the information processing device. The differences from the functional configuration shown in Figure 5 are that the function of the unauthorized person image generation unit 1511 is different from the function of the unauthorized person image generation unit 512 in Figure 5, and the function of the control unit 1521 is different from the function of the control unit 532 in Figure 5.

[0150] The unauthorized user image generation unit 1511 acquires an authorized user image from the authorized user image generation unit 511. The unauthorized user image generation unit 1511 generates an unauthorized user screen that includes the acquired authorized user image and notifies the control unit 1521.

[0151] The control unit 1521 notifies or transmits viewpoint data, authorized user screen or unauthorized user screen, operation instructions or location specification instructions, which are transmitted and received between the first HMD 121 and the information terminal 1301, to the notification destination or transmission destination.

[0152] <Details of processing performed by each functional unit of the information processing device> Next, we will explain the details of the processing performed by the control unit 1521 of the information processing device 110. Figure 16 is a third diagram showing a specific example of processing performed by the control unit.

[0153] In the display system 1300 according to the second embodiment, the control unit 1521 is The viewpoint data transmitted from the first HMD121 is notified to the authorized user viewpoint data acquisition unit 521. The operation instruction or location specification instruction transmitted from the first HMD121 is notified to the authorized user instruction receiving unit 522. The location specification instruction transmitted from the information terminal 1301 is notified to the unauthorized user instruction receiving unit 523. The authorized user screen generated by the authorized user image generation unit 511 is transmitted to the first HMD 121. The unauthorized user screen generated by the unauthorized user image generation unit 512 is transmitted to the information terminal 1301.

[0154] <Flow of virtual space display processing by information processing device> Next, the flow of virtual space display processing by the information processing device 110 according to the second embodiment will be described. Figure 17 is a second flowchart showing the flow of virtual space display processing by the information processing device. The difference from the first flowchart shown in Figure 9 is steps S1701 and S1702.

[0155] In step S1701, the information processing device 110 sets the notification destination or transmission destination for viewpoint data, authorized user screen or unauthorized user screen, operation instructions or location specification instructions that are transmitted and received between the first HMD 121 and the information terminal 1301.

[0156] In step S1702, the information processing device 110 generates a screen for unauthorized users that includes an image of an authorized user and transmits it to the information terminal 1301.

[0157] <Communication scene using the display system according to the second embodiment> Next, we will describe a scenario in which a first user 120 and a second user 130 communicate while viewing the same 3D model in a virtual space, using the display system 1300 according to the second embodiment.

[0158] Figure 18 is the first diagram showing an example of communication using the display system according to the second embodiment. As shown in Figure 18, in the case of the display system 1300, the display device 435 of the first HMD 121 displays an authorized person screen 640 including an authorized person image 1101. Also in the case of the display system 1300, the display device 1404 of the information terminal 1301 displays a non-authorized person screen 720 including an authorized person image 1101.

[0159] Under these display conditions, suppose the first user 120 says, for example, "I want to change the material of this surface." In this case, the second user 130 can see the authorized user image 1101 included in the unauthorized user screen 720 and understand that the surface whose material the first user 120 wishes to change is the surface indicated by the first user 120's indicator. As a result, the first user 120 and the second user 130 can achieve smooth communication.

[0160] Although not shown in Figure 18, let's assume that the second user 130, while viewing the authorized user image 1101 included in the unauthorized user screen 720, moves the second user 130's indicator to point to a different surface than the one pointed to by the first user 120's indicator.

[0161] Under these display conditions, suppose the second user 130 says, for example, "I want to change the material of this surface." In this case, the first user 120 can see the authoritative image 1101 included in the authoritative screen 640 and understand that the surface whose material the second user 130 wishes to change is the surface indicated by the second user 130's indicator. As a result, the first user 120 and the second user 130 can achieve smooth communication.

[0162] Figure 19 is a second diagram showing an example of a communication scene when using the display system according to the second embodiment. As shown in Figure 19, in the case of the display system 1300, only the first user 120 is granted operating privileges, and multiple users do not have operating privileges simultaneously.

[0163] Therefore, for example, even if the first user 120 and the second user 130 attempt to view the window-side surface of a 3D model in the virtual space, only the first user 120 can operate the 3D model. In other words, in the case of the display system 1300 according to the second embodiment, it is possible to avoid situations where different operation instructions are given simultaneously.

[0164] Furthermore, the first user 120 rotates the 3D model 90 degrees counterclockwise by moving the controller 124 in a counterclockwise arc. This allows the first user 120 to view a desired face of the 3D model in the virtual space by viewing the authorized user image 1201' contained in the authorized user screen 640. In addition, the second user 130 can view a desired face of the 3D model in the virtual space by viewing the authorized user image 1201' contained in the unauthorized user screen 720.

[0165] <Summary> As is clear from the above description, the information processing device 110 according to the second embodiment is • Grant the first user (user 120) the authority to manipulate the 3D model in the virtual space. Based on the operation instructions from the first user 120, an authorized user image is generated by operating a 3D model in a virtual space based on the viewpoint of the first user 120, and an authorized user screen including the authorized user image is generated. • Generate a screen for non-authorized users that includes an image of the authorized user. The system controls the display of the authorized user's screen on the first HMD 121 worn by the first user, and the display of the unauthorized user's screen on the information terminal 1301 viewed by the second user.

[0166] Thus, in the information processing device 110 according to the second embodiment, operating privileges are set for the user wearing the HMD, and the privileged user image is shared among multiple users. As a result, according to the second embodiment, smooth communication can be achieved when multiple users communicate while viewing the same 3D model in a virtual space.

[0167] [Third Embodiment] In the first embodiment described above, the display system 100 was described as having two sets of combinations of HMD, audio output device, audio input device, and controller. However, the number of combinations that the display system 100 has is not limited to two. For example, the display system 100 may have three or more combinations and be applied in communication scenes with three or more people.

[0168] Similarly, in the second embodiment described above, the case in which the display system 1300 has one set each of the combination of HMD, audio output device, audio input device, and controller, and the combination of information terminal, audio output device, audio input device, and controller was described. However, the number of combinations that the display system 1300 has is not limited to one set each. For example, the display system 1300 may have one set of the combination of HMD, audio output device, audio input device, and controller, and multiple sets of the combination of information terminal, audio output device, audio input device, and controller.

[0169] Alternatively, in the first embodiment described above, if the display system 100 has three or more combinations, one or more of these combinations may be replaced with a combination of an information terminal, an audio output device, an audio input device, and a controller.

[0170] [Other embodiments] In the embodiments described above, the virtual space image provided by the virtual space image providing unit 501 was described as an image that mimics the real world. However, the virtual space image provided by the virtual space image providing unit 501 is arbitrary. For example, if multiple users are performing molecular-level analysis of a material, molecular-level images of the material may be provided.

[0171] Furthermore, in the first embodiment described above, it was explained that a user with operational authority can switch between operation instructions and location specification instructions by operating a toggle switch on the controller. However, the functions of the information processing device 110 are not limited to this, and for example, it may be configured to change the display mode of the indicator in the authorized user image when the operation instructions and location specification instructions are switched.

[0172] Furthermore, in the first embodiment described above, it was explained that in the non-authorized user screen, the authorized user image is displayed in the main area and the non-authorized user image is displayed in the sub-area. However, the non-authorized user screen may be configured so that the authorized user image is displayed in the sub-area and the non-authorized user image is displayed in the main area. Alternatively, the non-authorized user screen may be configured so that the authorized user image and the non-authorized user image are switched and displayed based on a switching instruction from a user who does not have operational privileges set.

[0173] Furthermore, although the HMD, audio output device, and audio input device are configured as separate components in each of the above embodiments, the HMD, audio output device, and audio input device may be configured as an integrated unit.

[0174] Furthermore, in each of the above embodiments, the audio output device, audio input device, and controller are configured to transmit and receive information with the information processing device via the HMD or information terminal. However, the audio output device, audio input device, and controller may also be configured to transmit and receive information directly with the information processing device.

[0175] Furthermore, although the second embodiment described above assumes that the second user 130 inputs position specification instructions using the controller 134, the second user 130 may also input position specification instructions using, for example, an instruction device such as a mouse connected to the information terminal 1301.

[0176] Furthermore, in each of the above embodiments, the case in which the display method is applied to a virtual reality in which a 3D model is displayed in a virtual space image has been described. However, the application of the display method is not limited to virtual reality, and it may also be applied to augmented reality in which a virtual space image including a 3D model is displayed in an area where the real space is visible.

[0177] It should be noted that this disclosure is not limited to the configurations shown in the above embodiments, including combinations with other elements. These points can be modified without departing from the spirit of this disclosure and can be appropriately determined according to their application.

[0178] This application claims priority based on Japanese Patent Application No. 2023-185779, filed on 30 October 2023, which is incorporated herein by reference to the entire contents of the said Japanese Patent Application. [Explanation of Symbols]

[0179] 100: Display System 110: Information Processing Device 121: The first HMD 122: Audio output device 123: Voice input device 124: Controller 131: The second HMD 132: Audio output device 133: Voice input device 134: Controller 435:Display device 445:Display device 501: Virtual Space Image Provision Department 511: Authorized Person Image Generation Unit 512: Image generation unit for unauthorized users 531: Permissions setting section 532: Control Unit 540: Voice data transmission / reception unit 611, 612: Indicator 630: Image of an authorized person 640: Permissions screen 710: Image of an unauthorized person 720: Unauthorized User Screen 1300: Display System 1301: Information terminal 1511: Image generation unit for unauthorized users 1521: Control Unit

Claims

1. A permission setting unit that grants permission to operate a 3D model in a virtual space to any user, When the aforementioned operation privileges are set for the first user, the authorized user image generation unit generates an authorized user screen that includes an authorized user image of a 3D model in a virtual space based on the viewpoint of the first user, based on the operation instructions of the first user. An unauthorized user image generation unit generates an unauthorized user screen that includes an unauthorized user image showing a virtual space based on the perspective of a second user for whom the aforementioned operating privileges have not been set, and the aforementioned authorized user image. A control unit controls the display of the authorized user screen on the first HMD worn by the first user, and the display of the unauthorized user screen on the second HMD worn by the second user. An information processing device having

2. The aforementioned authorized person image generation unit, If the aforementioned operation privileges are set for the second user, an image of the privileged user is generated, which operates a 3D model in a virtual space based on the second user's viewpoint, based on the operation instructions given by the second user. The aforementioned unauthorized person image generation unit, An image of an unauthorized user is generated that shows a virtual space from the perspective of a first user for whom the aforementioned operational privileges have not been set. The control unit, The system controls the following: the authorized user screen is displayed on the second HMD worn by the second user, and the unauthorized user screen is displayed on the first HMD worn by the first user. The information processing apparatus according to claim 1.

3. The control unit, When the aforementioned operation authority is set for the first user, the system controls the system to notify the authorized user image generation unit of the viewpoint data received from the first HMD and the operation instructions from the first user. The aforementioned authorized person image generation unit, Based on viewpoint data received from the first HMD, an image of the virtual space based on the viewpoint of the first user is generated, and based on operation instructions from the first user, an image of the 3D model in the virtual space based on the viewpoint of the first user is generated, thereby generating the authorized person image. The information processing apparatus according to claim 1 or 2.

4. The control unit, When the aforementioned operation authority is set for the second user, the system controls the system to notify the authorized user image generation unit of the viewpoint data received from the second HMD and the operation instructions from the second user. The aforementioned authorized person image generation unit, Based on viewpoint data received from the second HMD, an image of the virtual space based on the viewpoint of the second user is generated, and based on operation instructions from the second user, an image of the 3D model in the virtual space based on the viewpoint of the second user is generated, thereby generating the authorized person image. The information processing apparatus according to claim 2.

5. The control unit, When the aforementioned operation authority is set for the first user, the system controls the system to notify the authorized user image generation unit of the location specification instruction made by the second user. The aforementioned authorized person image generation unit, Based on the location specification instruction by the second user, the authorized person image is generated, which includes an indicator showing the location in the virtual space based on the viewpoint of the first user. The information processing apparatus according to claim 3.

6. The control unit, When the aforementioned operation authority is set for the second user, the system controls the system to notify the authority user image generation unit of the location specification instruction made by the first user. The aforementioned authorized person image generation unit, Based on the location specification instruction by the first user, the system generates the authorized user image, which includes an indicator showing the location in the virtual space based on the viewpoint of the second user. The information processing apparatus according to claim 4.

7. The information processing apparatus according to claim 1, further comprising a transmitting and receiving unit for transmitting the voice of the first user to the second HMD and transmitting the voice of the second user to the first HMD.

8. The aforementioned unauthorized person image generation unit, If the second user, for whom the aforementioned operating privileges are not set, does not wear the second HMD, a non-authorized user screen including the authorized user image is generated. The control unit, The system controls the following to display the authorized user screen on the first HMD worn by the first user, and the unauthorized user screen on a display device viewed by the second user. The information processing apparatus according to claim 1.

9. The aforementioned unauthorized person image generation unit, If the first user, for whom the aforementioned operating privileges are not set, does not wear the first HMD, a non-authorized user screen including the authorized user image is generated. The control unit, The system controls the following to display the authorized user screen on a second HMD worn by the second user, and to display the unauthorized user screen on a display device viewed by the first user. The information processing apparatus according to claim 2.

10. The information processing apparatus according to claim 1 or 2, A display system comprising the first HMD and the second HMD according to claim 1 or 2.

11. The information processing apparatus according to claim 8, A display system comprising the first HMD and the display device described in claim 8.

12. The information processing apparatus according to claim 9, A display system comprising the second HMD and the display device described in claim 9.

13. The process of granting a user the authority to manipulate a 3D model in a virtual space, When the aforementioned operation privileges are set for the first user, the process includes generating an authorized user screen that includes an authorized user image showing a 3D model in a virtual space based on the first user's viewpoint, based on the operation instructions given by the first user. A step of generating a non-authorized user screen that includes an image of a non-authorized user showing a virtual space based on the perspective of a second user for whom the aforementioned operating privileges have not been set, and an image of an authorized user. The process involves controlling the following steps: displaying the authorized user screen on the first HMD worn by the first user, and displaying the unauthorized user screen on the second HMD worn by the second user. A method of displaying what a computer does.

14. The process of granting a user the authority to manipulate a 3D model in a virtual space, When the aforementioned operation privileges are set for the first user, the process includes generating an authorized user screen that includes an authorized user image showing a 3D model in a virtual space based on the first user's viewpoint, based on the operation instructions given by the first user. A step of generating a non-authorized user screen that includes an image of a non-authorized user showing a virtual space based on the perspective of a second user for whom the aforementioned operating privileges have not been set, and an image of an authorized user. The process involves controlling the following steps: displaying the authorized user screen on the first HMD worn by the first user, and displaying the unauthorized user screen on the second HMD worn by the second user. A display program that causes a computer to execute something.

Images