Display Control Device

Abstract

This display control device comprises: a generation unit that is disposed in a first virtual space which is viewed by a user who is the transmission source of a message, and that generates a virtual object related to the message; and a setting unit that is displayed in a second virtual space which is viewed by a user who is the transmission destination of a message, and that is for setting whether or not to change the external appearance of the virtual object.

Description

[Technical field]

[0001] The present invention relates to a display control device. [Background technology]

[0002] When a message or content is distributed in a virtual space, the specifications relating to the display of a virtual object representing the message or content may be changed.

[0003] For example, Patent Document 1 discloses a technology related to a content distribution server that provides educational content in a distance education system. In the content distribution server, a material data distribution unit distributes first specification data indicating specifications of a plurality of virtual objects arranged in a virtual space and motion data indicating the movements of the plurality of virtual objects to an education terminal. In addition, a material data change unit changes the first specification data to second specification data in response to a request received from the education terminal. Thereafter, the material data distribution unit distributes the second specification data to the education terminal. That is, in a conventional distance education system, the appearance of a virtual object displayed on a destination education terminal is changed in response to a request from a user to which the virtual object is to be transmitted. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-006894 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there are cases where it is desired to control the change of the appearance of a virtual object by the system, rather than leaving it up to the user at the destination. However, in the conventional technology, the appearance of a virtual object displayed on the destination educational terminal is changed in response to a request from the user at the destination of the virtual object. For this reason, it is not possible to determine whether or not to change the appearance of a virtual object in a server or the like that is the source of the virtual object.

[0006] Therefore, an object of the present invention is to provide a display control device that enables a transmission source to set whether or not the appearance of a virtual object displayed at a transmission destination can be changed. [Means for solving the problem]

[0007] A display control device according to a preferred embodiment of the present invention is a display control device that is arranged in a first virtual space viewed by a user who has sent a message, and includes a generation unit that generates a virtual object related to the message, and a setting unit that sets whether or not to allow a change in appearance of the virtual object displayed in a second virtual space viewed by a user to whom the message is sent. Effect of the Invention

[0008] According to the present invention, the sender can set whether or not the appearance of a virtual object displayed at the destination can be changed. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 shows the overall configuration of an information processing system 1. [Diagram 2] FIG. 2 is a perspective view showing the appearance of a display device 30-K. [Diagram 3] FIG. 11 is an explanatory diagram of a display method in a display device 30-K. [Figure 4] FIG. 2 is a block diagram showing a configuration example of a display device 30-K. [Diagram 5] FIG. 2 is a block diagram showing an example of the configuration of a terminal device 20-K. [Figure 6] FIG. 2 is a block diagram showing an example of the configuration of the server 10. [Figure 7] FIG. 2 is a sequence diagram showing the operation of the information processing system 1. [Figure 8] FIG. 2 is a block diagram showing a configuration example of a terminal device 20A-K. [Figure 9] FIG. 2 is a block diagram showing an example of the configuration of a server 10A. [Figure 10] FIG. 4 is a sequence diagram showing the operation of the information processing system 1A. [Figure 11] FIG. 13 is a diagram showing an example of a virtual object VO2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] 1: First embodiment Hereinafter, a configuration of an information processing system 1 including a server 10 as a display control device according to a first embodiment of the present invention will be described with reference to FIGS.

[0011] 1-1: Configuration of the first embodiment 1-1-1: Overall composition FIG. 1 shows the overall configuration of an information processing system 1. As shown in FIG. 1, the information processing system 1 includes a server 10, terminal devices 20-1, 20-2, ... 20-K, ... 20-M, ... 20-J, and display devices 30-K and 30-M. J is an integer of 1 or more. K is an integer of 1 or more and J or less. M is an integer of 1 or more and J or less, and different from K. In this embodiment, the terminal devices 20-1 to 20-J have the same configuration. However, terminal devices with different configurations may be included. In this embodiment, the display devices 30-K and 30-M have the same configuration. However, display devices with different configurations may be included.

[0012] In the information processing system 1, the server 10 and the terminal devices 20-1 to 20-J are connected to each other so as to be able to communicate with each other via a communication network NET. In addition, the terminal device 20-K and the display device 30-K are connected to each other so as to be able to communicate with each other. Similarly, the terminal device 20-K and the display device 30-K are connected to each other so as to be able to communicate with each other. In addition, in FIG. 1, a user U KAssume that a user U uses a combination of a terminal device 20-K and a display device 30-K. M It is assumed that a combination of a terminal device 20-M and a display device 30-M is used.

[0013] The server 10 provides various data and cloud services to the terminal devices 20-1, 20-2, . . . 20-K, . . . 20-M, . . . 20-J via the communication network NET.

[0014] The terminal device 20-K is a user U K The display device 30-K is mounted on the head of the user U and displays virtual objects arranged in a virtual space. The virtual space is, for example, a celestial sphere. The virtual objects are, for example, virtual objects representing data such as still images, videos, 3DCG models, HTML files, and text files, and virtual objects representing applications. Examples of text files include memos, source codes, diaries, and recipes. Examples of applications include browsers, applications for using SNS, and applications for generating document files. The terminal device 20-K is preferably, for example, a mobile terminal device such as a smartphone or a tablet. Similarly, the terminal device 20-M is also, for example, a mobile terminal device such as a smartphone or a tablet. M The user causes a head-mounted display device 30-M to display virtual objects arranged in a virtual space.

[0015] In particular, in this embodiment, a user U wearing the display device 30-K on his / her head K uses the terminal device 20-K to communicate with other users U M Specifically, as will be described later, K In the virtual space, a virtual object related to the message is placed, and the user U M By specifying the destination, user U M Send a message to.

[0016] Among the terminal devices 20-1 to 20-J, a display device similar to the display devices 30-K and 30-M may be connected to a terminal device other than the terminal devices 20-K and 20-M.

[0017] 1-1-2: Display device configuration As described above, the display device 30-K is a device that displays virtual objects arranged in a virtual space. K Alternatively, the display device 30-K may be a pair of VR glasses or VR goggles using a VR (Virtual Reality) technology that provides a virtual reality space to the user U. K Alternatively, the display device 30-K may be an AR glass or an AR goggle using an AR (Augmented Reality) technology that provides an augmented reality space to the user U. K The display device 30-K may be MR glasses or MR goggles using MR (Mixed Reality) technology that provides a mixed reality space for the display device 30-K. Alternatively, the display device 30-K may be a HMD (Head Mounted Display) that employs any of VR technology, AR technology, and MR technology. In particular, when the display device 30-K is an HMD, it may be an HMD that functions as two or more types of goggles among VR goggles, AR goggles, and MR goggles. The same applies to the display device 30-M.

[0018] 2 is a perspective view showing an external appearance of the display device 30-K when the display device 30-K is an AR glass as an example of the display device 30-K. As shown in FIG. 2, the external appearance of the display device 30-K includes temples 91 and 92, a bridge 93, frames 94 and 95, and lenses 41L and 41R, similar to general glasses.

[0019] The bridge 93 is provided with an imaging device 36. The imaging device 36 captures an image of the outside world. The imaging device 36 also outputs imaging information indicating the captured image.

[0020] Each of the lenses 41L and 41R includes a half mirror. The frame 94 includes a liquid crystal panel or an organic EL panel for the left eye. The liquid crystal panel or the organic EL panel will be collectively referred to as a display panel hereinafter. The frame 94 also includes an optical member that guides light emitted from the display panel for the left eye to the lens 41L. The half mirror provided on the lens 41L transmits light from the outside world and guides it to the left eye, and reflects the light guided by the optical member and allows it to enter the left eye. The frame 95 includes a display panel for the right eye and an optical member that guides light emitted from the display panel for the right eye to the lens 41R. The half mirror provided on the lens 41R transmits light from the outside world and guides it to the right eye, and reflects the light guided by the optical member and allows it to enter the right eye.

[0021] A display 39, which will be described later, includes a lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as a lens 41R, a display panel for the right eye, and an optical member for the right eye.

[0022] In the above configuration, user U K The user U can observe the image displayed by the display panel in a see-through state in which the image is superimposed on the outside world. Furthermore, the display device 30-K displays the image for the left eye on the display panel for the left eye, and the image for the right eye on the display panel for the right eye, among the binocular images with parallax. For this reason, the display device 30-K allows the user U to K To make a user perceive a displayed image as if it has depth and a three-dimensional effect.

[0023] 3 is an explanatory diagram of a display method in the display device 30-K realized in this embodiment. In FIG. 3, a user U who uses the display device 30-K as VR glasses as an example. K A virtual reality space VS is provided to a user U using a display device 30-M as an example of AR glasses. M In addition, the virtual reality space VS is an example of a first virtual space, and the augmented reality space AS is an example of a second virtual space.

[0024] User U K sets up a virtual object VO1 related to the message in the virtual reality space VS. Then, the user U K is addressed to user U M Then, a message corresponding to the virtual object VO1 is sent to the user U via the server 10. M When the message is transmitted to the terminal device 20-M, a virtual object VO1 related to the message is displayed as a virtual object VO2 in the augmented reality space AS. The appearance of the virtual object VO2 is basically the same as that of the virtual object VO1. However, since the virtual object VO2 is displayed in the augmented reality space AS, the appearance of the virtual object VO2 may differ from that of the user U depending on the environment in the real space included in the augmented reality space AS. M The appearance of the virtual object VO2 to the user U changes depending on the environment in the real space included in the augmented reality space AS. M The appearance of the virtual object VO2 is changed so that it appears optimal for the user.

[0025] FIG. 4 is a block diagram showing an example of the configuration of the display device 30-K when the display device 30-K is an AR glass, as an example of the display device 30-K. The display device 30-K includes a processing device 31, a storage device 32, a gaze detection device 33, a GPS device 34, a motion detection device 35, an imaging device 36, an environmental sensor 37, a communication device 38, and a display 39. The elements of the display device 30-K are connected to each other using a single or multiple buses for communicating information. Note that the term "apparatus" in this specification may be replaced with other terms such as a circuit, a device, or a unit.

[0026] The processing device 31 is a processor that controls the entire display device 30-K. The processing device 31 is configured, for example, using one or more chips. The processing device 31 is also configured, for example, using a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and a register. Some or all of the functions of the processing device 31 may be realized using hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). The processing device 31 executes various processes in parallel or sequentially.

[0027] The storage device 32 is a recording medium that can be read from and written to by the processing device 31. The storage device 32 also stores a plurality of programs including a control program PR3 that the processing device 31 executes.

[0028] The gaze detection device 33 detects the gaze of the user U K After detecting the gaze of the user U, the gaze detection device 33 generates gaze information indicating the detection result. Any method may be used as a method for the gaze detection device 33 to detect the gaze. For example, the gaze detection device 33 may detect the gaze information based on the position of the inner corner of the eye and the position of the iris. The gaze information is K The gaze detection device 33 supplies the gaze information to a processing device 31, which will be described later. The gaze information supplied to the processing device 31 is transmitted to the terminal device 20-K via a communication device .

[0029] The GPS device 34 receives radio waves from a plurality of satellites. The GPS device 34 generates location information from the received radio waves. The location information indicates the location of the display device 30-K. The location information may be in any format as long as it can identify the location. The location information indicates, for example, the latitude and longitude of the display device 30-K. As an example, the location information is obtained from the GPS device 34. However, the display device 30-K may obtain the location information using any method. The obtained location information is supplied to the processing device 31. The location information supplied to the processing device 31 is transmitted to the terminal device 20-K via the communication device 38.

[0030] The motion detection device 35 detects the motion of the display device 30-K. The motion detection device 35 includes inertial sensors such as an acceleration sensor that detects acceleration and a gyro sensor that detects angular acceleration. The acceleration sensor detects acceleration of the orthogonal X-axis, Y-axis, and Z-axis. The gyro sensor detects angular acceleration around the X-axis, Y-axis, and Z-axis as the central axis of rotation. The motion detection device 35 can generate attitude information indicating the attitude of the display device 30-K based on the output information of the gyro sensor. The motion information includes acceleration data indicating the acceleration of each of the three axes and angular acceleration data indicating the angular acceleration of each of the three axes. In addition, the motion detection device 35 supplies the attitude information indicating the attitude of the display device 30-K and the motion information related to the motion of the display device 30-K to the processing device 31. The attitude information and the motion information supplied to the processing device 31 are transmitted to the terminal device 20-K via the communication device 38.

[0031] The imaging device 36 outputs imaging information obtained by imaging the outside world. The imaging device 36 also includes, for example, a lens, an imaging element, an amplifier, and an AD converter. The imaging element converts light collected through the lens into an imaging signal, which is an analog signal. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified imaging signal, which is an analog signal, into imaging information, which is a digital signal. The converted imaging information is supplied to the processing device 31. The imaging information supplied to the processing device 31 is transmitted to the terminal device 20-K via the communication device 38.

[0032] The environmental sensor 37 senses the environment around the display device 30-K. The environmental sensor 37 includes an illuminance sensor that senses illuminance. The environmental sensor 37 may further sense temperature, humidity, air pressure, and the like, as necessary. The environmental sensor 37 generates environmental information based on the results of the sensing. In addition, the environmental sensor 37 supplies the generated environmental information to the processing device 31. The environmental information supplied to the processing device 31 is transmitted to the terminal device 20-K via the communication device 38.

[0033] The communication device 38 is hardware as a transmitting / receiving device for communicating with other devices. The communication device 38 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 38 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 38 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0034] The display 39 is a device that displays images. The display 39 displays various images under the control of the processing device 21. As described above, the display 39 includes the lens 41L, a display panel for the left eye, and an optical member for the left eye, as well as the lens 41R, a display panel for the right eye, and an optical member for the right eye. As the display panel, for example, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used.

[0035] The processing device 31, for example, reads out and executes a control program PR3 from the storage device 32. As a result, the processing device 31 functions as an acquisition unit 311 and a display control unit 312.

[0036] The acquisition unit 311 acquires image information indicating an image to be displayed on the display device 30-K from the terminal device 20-K.

[0037] Furthermore, the acquisition unit 311 acquires gaze information input from the gaze detection device 33, position information input from the GPS device 34, posture information and movement information input from the movement detection device 35, imaging information input from the imaging device 36, and environmental information input from the environmental sensor 37. Then, the acquisition unit 311 outputs the acquired gaze information, position information, posture information, movement information, imaging information, and environmental information to the communication device 38.

[0038] The display control unit 312 causes the display 39 to display an image indicated by the image information acquired by the acquisition unit 311 from the terminal device 20-K.

[0039] 1-1-3: Terminal device configuration 5 is a block diagram showing a configuration example of the terminal device 20-K. The terminal device 20-K includes a processing device 21, a storage device 22, a communication device 23, a display 24, an input device 25, and an inertial sensor 26. The elements of the terminal device 20-K are connected to each other using a single or multiple buses for communicating information.

[0040] The processing device 21 is a processor that controls the entire terminal device 20-K. The processing device 21 is configured, for example, using one or more chips. The processing device 21 is configured, for example, using a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and a register. Some or all of the functions of the processing device 21 may be realized using hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 21 executes various processes in parallel or sequentially.

[0041] The storage device 22 is a recording medium that can be read and written by the processing device 21. The storage device 22 also stores a plurality of programs including a control program PR2 executed by the processing device 21. The storage device 22 may further store image information indicating an image to be displayed on the display device 30-K.

[0042] The communication device 23 is hardware as a transmitting / receiving device for communicating with other devices. The communication device 23 is also called, for example, a network device, a network controller, a network card, and a communication module. The communication device 23 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 23 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), and the like.

[0043] The display 24 is a device that displays images and text information. The display 24 displays various images under the control of the processing device 21. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display 24. Note that, when the display device 30-K is connected to the terminal device 20-K, the display 24 does not have to be an essential component. Specifically, by using the display device 30-K as the display 24, the terminal device 20-K may not have the display 24.

[0044] The input device 25 is a display device 30-K mounted on the head of a user U K For example, the input device 25 includes a keyboard, a touch pad, a touch panel, or a pointing device such as a mouse. When the input device 25 includes a touch panel, it may also function as the display 24.

[0045] The inertial sensor 26 is a sensor that detects inertial force. The inertial sensor 26 includes, for example, one or more sensors selected from an acceleration sensor, an angular velocity sensor, and a gyro sensor. The processing device 21 detects the attitude of the terminal device 20-K based on the output information of the inertial sensor 26. Furthermore, the processing device 21 accepts the selection of a virtual object VO1, the input of characters, and the input of instructions in the virtual reality space VS or the augmented reality space AS based on the attitude of the terminal device 20-K. For example, when a user U K When the user U operates the input device 25 while pointing the central axis of the terminal device 20-K toward a predetermined area in the virtual reality space VS or the augmented reality space AS, the virtual object VO1 to be placed in the predetermined area is selected. K The operation is, for example, a double tap. K The user can select the virtual object VO1 by operating the terminal device 20-K without looking at the input device 25 of the terminal device 20-K.

[0046] The processing device 21 reads out and executes the control program PR2 from the storage device 22. As a result, the processing device 21 functions as an acquisition unit 211, a display control unit 212, and an output unit 213.

[0047] The acquisition unit 211 acquires the user U K The message and the destination information indicating the destination of the message are obtained from, for example, the user U. K The message and the destination information may be input to the terminal device 20-K by the processing device 21 using the input device 25. Alternatively, the message and the destination information of the message may be the message and the destination information acquired by the processing device 21 from an external device via the communication device 23.

[0048] In addition, the acquisition unit 211 K Obtain operation information indicating the operation of user U. KThe operation may be an operation using the input device 25, or may be an operation on the virtual object VO1 in the first virtual space. The acquisition unit 211 acquires line of sight information, position information, posture information, movement information, imaging information, and environmental information from the display device 30-K.

[0049] In addition, in the terminal device 20-M, not in the terminal device 20-K, the acquisition unit 211 receives the user U K In addition, in the terminal device 20-M, not in the terminal device 20-K, the acquisition unit 211 acquires the message created by the user U M Obtain operation information indicating the operation of user U. M The operation may be an operation using the input device 25, or may be an operation on the virtual object VO2 in the second virtual space.

[0050] Furthermore, the acquisition unit 211 acquires image information for displaying a virtual object on the display device 30-K from the server 10 via the communication device 23. In particular, the acquisition unit 211 acquires image information for displaying a virtual object VO1 related to the message acquired by the acquisition unit 211.

[0051] The acquisition unit 211 provided in the terminal device 20-M, not in the terminal device 20-K, acquires image information for displaying a virtual object on the display device 30-M from the server 10 via the communication device 23. In particular, the acquisition unit 211 provided in the terminal device 20-M acquires image information for displaying a virtual object VO2 related to the message acquired by the acquisition unit 211.

[0052] The display control unit 212 causes the display device 30-K to display a virtual object by using the image information acquired by the acquisition unit 211. In particular, the display control unit 212 causes the display device 30-K to display a virtual object VO1 related to a message by using the image information.

[0053] In the terminal device 20-M, not the terminal device 20-K, the display control unit 212 uses the image information acquired by the acquisition unit 211 to display a virtual object on the display device 30-M. In particular, the display control unit 212 uses the image information to display a virtual object VO2 related to a message on the display device 30-M. In addition, the display control unit 212 displays a message corresponding to the virtual object VO2 on the display device 30-M.

[0054] The output unit 213 transmits the message and destination information acquired by the acquisition unit 211 to the server 10 via the communication device 23.

[0055] The output unit 213 provided in the terminal device 20-M, not in the terminal device 20-K, outputs the environmental information acquired by the acquisition unit 211 to the server 10.

[0056] 1-1-4: Server configuration 6 is a block diagram showing a configuration example of the server 10. The server 10 includes a processing device 11, a storage device 12, a communication device 13, a display 14, and an input device 15. The elements of the server 10 are connected to each other using one or more buses for communicating information.

[0057] The processing device 11 is a processor that controls the entire server 10. The processing device 11 is configured, for example, using one or more chips. The processing device 11 is configured, for example, using a central processing unit (CPU) including an interface with peripheral devices, an arithmetic unit, and a register. Some or all of the functions of the processing device 11 may be realized using hardware such as a DSP, an ASIC, a PLD, and an FPGA. The processing device 11 executes various processes in parallel or sequentially.

[0058] The storage device 12 is a recording medium that can be read and written by the processing device 11. The storage device 12 also stores a plurality of programs including a control program PR1 executed by the processing device 11. The storage device 12 also stores image information indicating images to be displayed on the display devices 30-K and 30-M. In particular, the storage device 12 stores image information indicating virtual objects to be displayed on the display devices 30-K and 30-M.

[0059] Furthermore, the storage device 12 stores a registration information database RD. The registration information database RD includes information on users U who use the terminal devices 20-K and 20-J connected to the server 10. K and User U M And user U K and User U M The types of the display devices 30-K and 30-J to be worn on the head of the user U are stored in one-to-one correspondence with each other. Specifically, the types of the display devices 30-K and 30-J to be worn on the head of the user U are stored in one-to-one correspondence with each other. K and User U M In addition, the storage device 12 stores the user U K and User U M and the user U K and User U M The type of virtual space that the user U is viewing is stored in one-to-one correspondence with the type of virtual space that the user U is viewing. K and User U M The type of the virtual space that the user U is viewing is determined to be a virtual reality space, an augmented reality space, or a mixed reality space. K and User U M are stored in one-to-one correspondence with

[0060] The communication device 13 is hardware as a transmitting / receiving device for communicating with other devices. The communication device 13 is also called, for example, a network device, a network controller, a network card, a communication module, etc. The communication device 13 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 13 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include products that comply with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include products that comply with wireless LAN, Bluetooth (registered trademark), etc.

[0061] The display 14 is a device that displays images and text information. The display 14 displays various images under the control of the processing device 11. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used as the display 14.

[0062] The input device 15 is a device that accepts operations from an administrator of the information processing system 1. For example, the input device 15 includes a keyboard, a touch pad, a touch panel, or a pointing device such as a mouse. Here, when the input device 15 includes a touch panel, it may also serve as the display 14.

[0063] The processing device 11, for example, reads and executes a control program PR1 from the storage device 12. As a result, the processing device 11 functions as an acquisition unit 111, a registration unit 112, a generation unit 113, a setting unit 114, a change unit 115, and an output unit .

[0064] The acquisition unit 111 receives the user U K And user U K The acquisition unit 111 also acquires information on the type of the display device 30-K worn on the head of the user U from the terminal device 20-K via the communication device 13. K And user UK Similarly, the acquisition unit 111 acquires information in which the type of the virtual space that the user U is viewing is associated with the type of the virtual space that the user U is viewing from the terminal device 20-M via the communication device 13. M And user U M The acquisition unit 111 also acquires information on the type of the display device 30-M mounted on the head of the user U from the terminal device 20-M via the communication device 13. M And user U M The information obtained is associated one-to-one with the type of virtual space being viewed by the user.

[0065] Furthermore, the acquisition unit 111 acquires, via the communication device 13, a message from the terminal device 20-K and destination information indicating the destination of the message.

[0066] The acquisition unit 111 also acquires various data from the terminal device 20-K via the communication device 13. The data includes, for example, the following data: K The data includes data indicating the operation content for the virtual object VO1 inputted to the terminal device 20-K by the terminal device 20-K. The data may also include at least one of line-of-sight information, position information, posture information, movement information, and imaging information acquired by the terminal device 20-K from the display device 30-K.

[0067] The acquisition unit 111 also acquires various data from the terminal device 20-M via the communication device 13. The data includes, for example, the following data: M The data includes data indicating the operation content for the virtual object VO2 input by the terminal device 20-M to the terminal device 20-M. The data also includes environmental information acquired by the terminal device 20-M from the display device 30-M. Furthermore, the data may include at least one of gaze information, position information, posture information, movement information, and imaging information acquired by the terminal device 20-M from the display device 30-M.

[0068] The registration unit 112 registers the user U K and User UM And user U K and User U M The registration unit 112 stores information in a registration information database RD in which the types of the display devices 30-K and 30-M mounted on the heads of the users U are associated with each other on a one-to-one basis. K and User U M and the user U K and User U M The information in which the type of the virtual space visually recognized by the user is associated with the type of the virtual space visually recognized by the user is stored in a registration information database RD.

[0069] After acquiring the destination information, the acquisition unit 111 refers to the registration information database RD to acquire the user U corresponding to the destination information. M and acquiring information capable of distinguishing whether the second virtual space being viewed by the user is a virtual reality space, an augmented reality space, or a mixed reality space. Here, the information capable of distinguishing whether the second virtual space is a virtual reality space, an augmented reality space, or a mixed reality space is an example of virtual space information. Note that the virtual space information may be information indicating whether the second virtual space is an augmented reality space, an augmented reality space, or a mixed reality space.

[0070] The generation unit 113 uses the image information stored in the storage device 12 to generate a virtual image of the user U K The generation unit 113 generates a virtual object VO1 related to a message from the user. The generation unit 113 also generates a virtual object VO2 related to the above message, which is placed in the second virtual space, by using image information stored in the storage device 12. Note that when the generation unit 113 generates the virtual objects VO1 and VO2, they have the same appearance.

[0071] The setting unit 114 sets the virtual space for the user U based on the virtual space information acquired by the acquisition unit 111. MThe setting unit 114 sets whether or not to permit a change in the appearance of the virtual object VO2 in the second virtual space visually recognized by the user. More specifically, when the second virtual space is a virtual reality space, the setting unit 114 does not permit a change in the appearance of the virtual object VO2. On the other hand, when the second virtual space is an augmented reality space or a mixed reality space, the setting unit 114 permits a change in the appearance of the virtual object VO2.

[0072] As a result, the server 10 can set whether or not the appearance of the virtual object VO2 representing the message can be changed on the message receiving side. Furthermore, in order to make the appearance of the virtual object VO2 optimal for the user, the server 10 can set whether or not the appearance of the virtual object VO2 can be changed on the message receiving side. M There is no need for complicated operations. Also, the user U who sends the message K On the other hand, user U who receives the message M It is not necessary to manually set whether or not the appearance of the virtual object VO1, which is displayed as the virtual object VO2 in the second virtual space, can be changed depending on the situation.

[0073] In particular, the server 10 receives a message from a user U M The server 10 can set whether or not to permit a change in the appearance of the virtual object VO2 depending on whether the space being viewed by the user is a virtual reality space, an augmented reality space, or a mixed reality space.

[0074] When the setting unit 114 sets that the change of the appearance of the virtual object VO2 is permitted, the change unit 115 changes the appearance of the virtual object VO2 displayed in the second virtual space, based on the environmental information acquired by the acquisition unit 111. For example, when the environmental information acquired by the acquisition unit 111 is illuminance information indicating the illuminance measured by an illuminance sensor, the change unit 115 changes at least one of the brightness, saturation, and hue of the color of the virtual object VO2 according to the illuminance information.

[0075] As a result, the server 10 adjusts the appearance of the virtual object VO2 to the user U. MIn particular, the server 10 can provide an optimal appearance for the user U. M When the second virtual space viewed by the user U is an augmented reality space or a mixed reality space, the appearance of the virtual object VO2 is adjusted to match the environment in the augmented reality space or the real space included in the mixed reality space. M This allows for optimal viewing.

[0076] The output unit 116 transmits image information indicating an image to be displayed on the display device 30-K to the terminal device 20-K via the communication device 13. In particular, the output unit 116 transmits image information indicating the virtual object VO1 to the terminal device 20-K. Also, the output unit 116 transmits image information indicating an image to be displayed on the display device 30-M to the terminal device 20-M via the communication device 13. In particular, the output unit 116 transmits image information indicating the virtual object VO2 to the terminal device 20-M.

[0077] More specifically, when the setting unit 114 sets that changing the appearance of the virtual object VO2 is not permitted, or when the setting unit 114 sets that the change is permitted but the change unit 115 does not change the appearance of the virtual object VO2, the output unit 116 transmits, to the terminal device 20-M, image information showing the virtual object VO2 as it is generated by the generation unit 113. On the other hand, when the setting unit 114 sets that changing the appearance of the virtual object VO2 is permitted and the change unit 115 changes the appearance of the virtual object VO2, the output unit 116 transmits, to the terminal device 20-M, image information showing the virtual object VO2 whose appearance has been changed.

[0078] Furthermore, the acquisition unit 111 acquires, from the terminal device 20-M, the user U M When the output unit 116 acquires operation information indicating an operation instructing to display a message related to the virtual object VO2 as the operation of the user U, the output unit 116 transmits the message to the terminal device 20-M. K User U MThis is a message addressed to.

[0079] 1-2: Operation of the First Embodiment 7 is a sequence diagram showing the operation of the information processing system 1 according to the first embodiment. Hereinafter, the operation of the information processing system 1 will be described with reference to FIG.

[0080] In step S1, the processing device 21 included in the terminal device 20-K functions as an acquisition unit 211. K Get the message that was created by.

[0081] In step S2, the processing device 21 provided in the terminal device 20-K functions as the output unit 213. The processing device 21 outputs to the server 10 the user U K The processing device 11 in the server 10 functions as an acquisition unit 111. The processing device 11 receives a message created by the user U from the terminal device 20-K. K Get the message that was created by.

[0082] In step S3, the processing device 11 provided in the server 10 functions as a generation unit 113. K The virtual objects VO1 and VO2 are generated for the messages created by the user.

[0083] In step S4, the processing device 11 included in the server 10 functions as an output unit 116. The processing device 11 outputs image information indicating the virtual object VO1 to the terminal device 20-K. In addition, the processing device 21 included in the terminal device 20-K functions as an acquisition unit 211. The processing device 21 acquires image information indicating the virtual object VO1 from the server 10.

[0084] In step S5, the processing device 21 included in the terminal device 20-K functions as the display control unit 212. The processing device 21 uses the image information acquired in step S4 to cause the display device 30-K to display the virtual object VO1.

[0085] In step S6, the processing device 21 provided in the terminal device 20-K functions as an acquisition unit 211. The processing device 21 is the user U K acquires destination information indicating the destination of the message, which is input by, for example, using the input device 25. Further, the processing device 21 functions as an output unit 213. The processing device 21 transmits the acquired destination information to the server 10. Furthermore, the processing device 11 provided in the server 10 functions as an acquisition unit 111. The processing device 11 acquires destination information indicating the destination of the message acquired in step S2 from the terminal device 20-K. Here, it is assumed that the destination of the message is the user U M is.

[0086] In step S7, the processing device 11 provided in the server 10 functions as an acquisition unit 111. The processing device 11 refers to the registration information database RD using the destination information acquired in step S6. As a result, the processing device 11 is the user U M acquires virtual space information indicating which of the virtual reality space, augmented reality space, and mixed reality space is the second virtual space that the user U is viewing.

[0087] In step S8, the processing device 11 provided in the server 10 functions as a setting unit 114. The processing device 11 sets whether to permit the change of the appearance of the virtual object VO2 displayed in the second virtual space based on the virtual space information acquired in step S7. When the second virtual space is the virtual reality space, the processing device 11 sets not to permit the change of the appearance of the virtual object VO2. After that, the processing device 11 executes the operation of step S11. On the other hand, when the second virtual space is the augmented reality space or the mixed reality space, the processing device 11 sets to permit the change of the appearance of the virtual object VO2. After that, the processing device 11 executes the operation of step S9.

[0088] In step S9, the processing device 21 included in the terminal device 20-M functions as an output unit 213. The processing device 21 transmits environmental information to the server 10. Moreover, the processing device 11 included in the server 10 functions as an acquisition unit 111. The processing device 11 acquires the environmental information from the terminal device 20-M.

[0089] In step S9, the processing device 11 included in the server 10 may output a request signal requesting environmental information to the terminal device 20-M, and the processing device 21 included in the terminal device 20-M may transmit the environmental information to the server 10 in response to the request signal. Alternatively, the processing device 21 included in the terminal device 20-M may transmit the environmental information to the server 10 constantly or intermittently, and the processing device 11 included in the server 10 may acquire the environmental information.

[0090] In step S10, the processing device 11 included in the server 10 functions as the change unit 115. The processing device 11 changes the appearance of the virtual object VO2 displayed in the second virtual space, based on the environmental information acquired in step S9.

[0091] In step S11, the processing device 11 included in the server 10 functions as an output unit 116. The processing device 11 transmits image information indicating the virtual object VO2 to the terminal device 20-M. In addition, the processing device 21 included in the terminal device 20-M functions as an acquisition unit 211. The processing device 21 acquires the image information indicating the virtual object VO2 from the server 10.

[0092] In step S12, the processing device 21 included in the terminal device 20-M functions as a display control unit 212. The processing device 21 causes the display device 30-M to display the virtual object VO2, using the image information acquired in step S4.

[0093] In step S13, the processing device 11 provided in the server 10 functions as the output unit 116. The processing device 11 transmits a message corresponding to the virtual object VO2 to the terminal device 20-M. For example, the processing device 11 transmits a user U's input for the virtual object VO2 from the terminal device 20-M. M When operation information indicating an operation instructing to display a message related to the virtual object VO2 is acquired as the operation, the terminal device 20-M transmits the message to the terminal device 20-M. The processing device 21 included in the terminal device 20-M functions as an acquisition unit 211. The processing device 21 acquires the message from the server 10. The processing device 21 included in the terminal device 20-M also functions as a display control unit 212. The processing device 21 causes the display device 30-M to display the message.

[0094] Thereafter, the processor 21 in the terminal device 20-K, the processor 11 in the server 10, and the processor 21 in the terminal device 20-M complete all the processes shown in FIG.

[0095] 1-3: Advantages of the First Embodiment According to the above description, the server 10 as a display control device includes a generating unit 113 and a setting unit 114. The generating unit 113 generates a display control message for a user U K The setting unit 114 generates a virtual object VO1 related to the message, which is placed in the first virtual space that is visually recognized by the user U of the message destination. M This sets whether or not to permit a change in the appearance of the virtual object VO2 displayed in the second virtual space visually recognized by the user.

[0096] Since the server 10 has the above-mentioned configuration, the server 10 can set whether or not to permit a user receiving a message to change the appearance of the virtual object VO2 representing the message. MThe appearance of the virtual object VO2 differs depending on the situation. By allowing the server 10 to set whether or not to permit the appearance of the virtual object VO2 to be changed, the user U can set the appearance of the virtual object VO2 to be optimal for the user. M There is no need for complicated operations. Also, the user U who sends the message K Let U be the user who receives the message. M It is not necessary to manually set whether or not to permit the appearance of the virtual object VO1, which is displayed as the virtual object VO2 in the second virtual space, to be changed depending on the situation.

[0097] According to the above description, the server 10 further includes an acquisition unit 111. The acquisition unit 111 acquires virtual space information that can distinguish whether the second virtual space is a virtual reality space, an augmented reality space, or a mixed reality space. The setting unit 114 sets whether or not the change is possible based on the virtual space information.

[0098] The server 10 has the above configuration, and therefore, the user U M The server 10 can set whether or not to permit the appearance of the virtual object VO2 to be changed depending on whether the user U is viewing a virtual reality space, an augmented reality space, or a mixed reality space. K Let U be the user who receives the message. M In this case, it is not necessary for the user to manually set whether or not to allow the appearance of a virtual object VO1 displayed as a virtual object VO2 to be changed after determining whether the space being viewed is a virtual reality space, an augmented reality space, or a mixed reality space.

[0099] According to the above description, the acquisition unit 111 acquires environmental information indicating an environment in the real space of the message destination. The server 10 further includes a change unit 115. When the second virtual space is an augmented reality space or a mixed reality space and the setting unit 114 sets that changing the appearance of the virtual object VO2 is permitted, the change unit 115 changes the appearance of the virtual object VO2 in the second virtual space based on the environmental information.

[0100] Since the server 10 has the above configuration, the server 10 controls the appearance of the virtual object VO2 according to the user U. M In particular, the server 10 can provide an optimal appearance for the user U. M When the second virtual space viewed by the user U is an augmented reality space or a mixed reality space, the appearance of the virtual object VO2 is adjusted to match the environment in the augmented reality space or the real space included in the mixed reality space. M This allows for optimal viewing.

[0101] 2: Second embodiment Hereinafter, a configuration of an information processing system 1A including a terminal device 20A-K as a display control device according to a second embodiment of the present invention will be described with reference to Figures 8 to 10. In the following description, for the purpose of simplifying the description, among the components of the information processing system 1A according to the second embodiment, the same components as those of the information processing system 1 according to the first embodiment will be designated by the same reference numerals, and the description thereof may be omitted.

[0102] 2-1: Configuration of the second embodiment 2-1-1: Overall structure An information processing system 1A according to the second embodiment of the present invention differs from the information processing system 1 according to the first embodiment in that it includes a server 10A instead of the server 10, a terminal device 20A-K instead of the terminal device 20-K, and a terminal device 20A-M instead of the terminal device 20-M. In other respects, the overall configuration of the information processing system 1A is the same as the overall configuration of the information processing system 1 according to the first embodiment shown in Fig. 1, so illustration and description thereof will be omitted.

[0103] In the information processing system 1 according to the first embodiment, the server 10 generates virtual objects VO1 and VO2, and sets whether or not to permit a change in the appearance of the virtual object VO2. On the other hand, in the information processing system 1A according to the second embodiment, the terminal device 20A-K generates the virtual object VO1. In addition, the terminal device 20A-K sets whether or not to permit a change in the appearance of the virtual object VO1. Furthermore, the terminal device 20A-M generates the virtual object VO2. More specifically, when the virtual object VO1 is displayed in the second virtual space as the virtual object VO2, the terminal device 20-K sets whether or not to permit a change in the appearance of the virtual object VO1 when it is displayed as the virtual object VO2 in the second virtual space. In the second embodiment, the terminal device 20-K is an example of a display control device.

[0104] 2-1-2: Terminal device configuration 8 is a block diagram showing a configuration example of the terminal device 20A-K. The terminal device 20A-K differs from the terminal device 20-K in that it includes a processing device 21A instead of the processing device 21 and a storage device 22A instead of the storage device 22.

[0105] Unlike the storage device 22, the storage device 22A stores a control program PR2A instead of the control program PR2. The storage device 22A also stores image information indicating an image to be displayed on the display device 30-K. In particular, the storage device 22A stores image information indicating a virtual object VO1 to be displayed on the display device 30-K. In the terminal device 20A-M, not in the terminal device 20A-K, the storage device 22A stores image information indicating a virtual object VO2 to be displayed on the display device 30-M.

[0106] The processing device 21A includes an acquisition unit 211A instead of the acquisition unit 211 included in the processing device 21, a display control unit 212A instead of the display control unit 212, and an output unit 213A instead of the output unit 213. The processing device 21A further includes a generation unit 214, a reception unit 215, and a setting unit 216 in addition to the components included in the processing device 21.

[0107] The acquisition unit 211A has the same function as the acquisition unit 211 according to the first embodiment. Furthermore, in the terminal device 20A-M, not in the terminal device 20A-K, the acquisition unit 211A acquires change information regarding the change of the virtual object VO2 from the server 10A.

[0108] The generation unit 214 uses the image information stored in the storage device 22A to generate a virtual image of the user U that is placed in the first virtual space and acquired by the acquisition unit 211A. K In the terminal device 20A-M, not in the terminal device 20A-K, the generation unit 214 generates a virtual object VO1 related to the message created by the user U, which is placed in the second virtual space and acquired by the acquisition unit 211A, using the image information stored in the storage device 22A and the change information acquired by the acquisition unit 211A. K A virtual object VO2 is generated regarding the message from.

[0109] The display control unit 212A causes the display device 30-K to display the virtual object generated by the generation unit 214. In particular, in the terminal device 20A-K, the display device 30-K displays the virtual object VO1 generated by the generation unit 214. On the other hand, in the terminal device 20A-M, the display device 30-M displays the virtual object VO2 generated by the generation unit 214.

[0110] The reception unit 215 receives a user U's request regarding whether or not to permit the appearance of the virtual object VO1 to be changed. K Specifically, the user U KWhen the user U does not want to change the appearance of the virtual object VO2 from the appearance of the virtual object VO1 when the virtual object VO1 is displayed in the second virtual space as the virtual object VO2, the user U executes an operation to specify that the change is not permitted. K When the virtual object VO1 is displayed in the second virtual space as the virtual object VO2, if the appearance of the virtual object VO2 may be changed from the appearance of the virtual object VO1, the user U executes an operation to specify that the change is permitted. K may be an operation using the input device 25, and the user U K The operation may be performed on the virtual object VO1 from the virtual object VO1.

[0111] The reception unit 215 also receives a user U K In the terminal device 20A-M, not in the terminal device 20A-K, the reception unit 215 receives the operation of the user U M Accepts operations.

[0112] The setting unit 216 sets the user U M The setting unit 216 sets whether or not to permit a change in the appearance of the virtual object VO2 displayed in the second virtual space visually recognized by the user. Specifically, the setting unit 216 generates permission / prohibition information indicating whether or not a change in the appearance of the virtual object VO2 is permitted, based on the above operation.

[0113] As a result, the terminal device 20A-K can set, at the terminal device 20-K which is the sender of a message, whether or not to permit a change in the appearance of the virtual object VO2 representing the message on the message receiver side.

[0114] The output unit 213A has the same functions as the output unit 213 according to the first embodiment. Furthermore, the output unit 213 outputs the availability information generated by the setting unit 216 to the server 10A.

[0115] 2-1-3: Server configuration 9 is a block diagram showing an example of the configuration of the server 10A. The server 10A is different from the server 10 in that it includes a processing device 11A instead of the processing device 11 and a storage device 12A instead of the storage device 12.

[0116] The storage device 12A differs from the storage device 12 in that it stores a control program PR1A instead of the control program PR1.

[0117] The processing device 11A includes an acquisition unit 111A instead of the acquisition unit 111 included in the processing device 11, a generation unit 113A instead of the generation unit 113, a setting unit 114A instead of the setting unit 114, and an output unit 116A instead of the output unit 116. Furthermore, among the components included in the processing device 11, the processing device 11A does not consider the change unit 115 to be an essential component.

[0118] The acquisition unit 111A has the same functions as the acquisition unit 111 according to the first embodiment. Furthermore, the acquisition unit 111A acquires the above-mentioned availability information from the terminal device 20-K.

[0119] The setting unit 114A determines whether or not the user U M In more detail, the above-mentioned permission / prohibition information is set to a value that indicates whether or not the user U is permitted to change the appearance of the virtual object VO2 in the second virtual space that the user U views. K In the case where the permission information indicates that the appearance of the virtual object VO2 is not permitted to be changed, the setting unit 114A does not permit the appearance of the virtual object VO2 to be changed. K indicates that the appearance of the virtual object VO2 is permitted to be changed, but if the second virtual space is a virtual reality space, the setting unit 114A does not permit the appearance of the virtual object VO2 to be changed. K indicates that permission to change the appearance of the virtual object VO2 is specified, and when the second virtual space is an augmented reality space or a mixed reality space, the setting unit 114A permits the change of the appearance of the virtual object VO2.

[0120] When the setting unit 114A permits the change of the appearance of the virtual object VO2, the generation unit 113A generates change information related to the change of the appearance of the virtual object VO2 displayed in the second virtual space, based on the environmental information acquired by the acquisition unit 111A. For example, when the environmental information acquired by the acquisition unit 111A is illuminance information measured by an illuminance sensor, the generation unit 113A generates change information indicating the degree of change of at least one of the brightness, saturation, and hue of the color of the virtual object VO2 according to the illuminance information.

[0121] The output unit 116A outputs the change information generated by the generation unit 113A to the terminal device 20A-M.

[0122] 2-2: Operation of the second embodiment 10 is a sequence diagram showing the operation of the information processing system 1A according to the second embodiment. Hereinafter, the operation of the information processing system 1A will be described with reference to FIG.

[0123] In step S21, the processing device 21A included in the terminal device 20A-K functions as an acquisition unit 211A. K The message created by user U and destination information indicating the destination of the message are obtained. M It is assumed that.

[0124] In step S22, the processing device 21A included in the terminal device 20A-K functions as the generation unit 214. The processing device 21A generates a virtual object VO1 related to the message acquired in step S21.

[0125] In step S23, the processing device 21A included in the terminal device 20A-K functions as the display control unit 212. The processing device 21A causes the display device 30-K to display the virtual object VO1 generated in step S22.

[0126] In step S24, the processing device 21A included in the terminal device 20A-K functions as the setting unit 216. K Based on the operation of user U M In the second virtual space, the processing device 21A sets whether or not to permit a change in the appearance of the virtual object VO2 displayed in the second virtual space visually recognized by the user. Specifically, the processing device 21A generates permission / prohibition information indicating whether or not the change is permitted.

[0127] In step S25, the processing device 21A included in the terminal device 20A-K functions as an output unit 213A. The processing device 21A outputs the message and destination information acquired in step S21 to the server 10A. In addition, the processing device 11A included in the server 10A functions as an acquisition unit 111A. The processing device 11A acquires the message and destination information from the terminal device 20A-K.

[0128] In step S26, the processing device 21A included in the terminal device 20A-K functions as an output unit 213A. The processing device 21A outputs the propriety information generated in step S24 to the server 10A. In addition, the processing device 11A included in the server 10A functions as an acquisition unit 111A. The processing device 11A acquires the propriety information from the terminal device 20A-K.

[0129] In step S27, the processing device 11A included in the server 10A functions as an acquisition unit 111A. The processing device 11A refers to the registration information database RD using the destination information acquired in step S26. As a result, the processing device 11A acquires the user U M The present invention obtains virtual space information that enables a user to distinguish whether the second virtual space viewed by the user is a virtual reality space, an augmented reality space, or a mixed reality space.

[0130] In step S28, the processing device 11A of the server 10A functions as a setting unit 114A. The processing device 11A sets whether or not the appearance of the virtual object VO2 displayed in the second virtual space can be changed based on the possibility information acquired in step S26 and the virtual space information acquired in step S27. More specifically, the possibility information is set based on the possibility information acquired by the user U. K In the case where the permission information indicates that the change of the appearance of the virtual object VO2 is not permitted, the processing device 11A does not permit the change of the appearance of the virtual object VO2. K indicates that the appearance of the virtual object VO2 is permitted to be changed, but if the second virtual space is a virtual reality space, the processing device 11A does not permit the appearance of the virtual object VO2 to be changed. Then, the processing device 11A executes the operation of step S32. On the other hand, when the above-mentioned permission / prohibition information is K indicates that permission to change the appearance of the virtual object VO2 is specified, and if the second virtual space is an augmented reality space or a mixed reality space, the processing device 11A permits the change of the appearance of the virtual object VO2. Then, the processing device 11 executes the operation of step S29.

[0131] In step S29, the processing device 21A included in the terminal devices 20A-M functions as an output unit 213A. The processing device 21A outputs environmental information to the server 10A. The processing device 11A included in the server 10A functions as an acquisition unit 111A. The processing device 11A acquires the environmental information from the terminal devices 20A-M.

[0132] In step S29, the processing device 11A in the server 10A may output a request signal requesting environmental information to the terminal devices 20A-M, and the processing device 21A in the terminal devices 20A-M may output the environmental information to the server 10A in response to the request signal. Alternatively, the processing device 21A in the terminal devices 20A-M may output the environmental information to the server 10A constantly or intermittently, and the processing device 11A in the server 10A may acquire the environmental information.

[0133] In step S30, the processing device 11A included in the server 10A functions as a generation unit 113A. The processing device 11A generates change information related to a change in the appearance of the virtual object VO2 displayed in the second virtual space, based on the environmental information acquired in step S29.

[0134] In step S31, the processing device 11A included in the server 10A functions as an output unit 116A. The processing device 11A outputs the change information generated in step S30 to the terminal device 20-M. In addition, the processing device 21A included in the terminal device 20A-M functions as an acquisition unit 211A. The processing device 21A acquires the change information from the server 10A.

[0135] In step S32, the processing device 11A included in the server 10A functions as an output unit 116A. The processing device 11A outputs the message acquired in step S25 to the terminal device 20-M. In addition, the processing device 21A included in the terminal device 20A-M functions as an acquisition unit 211A. The processing device 21A acquires the message from the server 10A.

[0136] In step S33, the processing device 21A included in the terminal device 20A-M functions as the generation unit 214. The processing device 21A generates a virtual object VO2 related to the message acquired in step S32, using the image information stored in the storage device 22A and the change information acquired in step S31.

[0137] In step S34, the processing device 21A included in the terminal device 20A-M functions as a display control unit 212A. The processing device 21A causes the display device 30-M to display the virtual object VO2 generated in step S33. After that, the processing device 21A displays the message acquired in step S32. For example, the processing device 21A included in the terminal device 20A-M functions as a reception unit 215. The processing device 21A receives a user U's response to the virtual object VO2. MAs the operation, an operation of instructing to display a message related to the virtual object VO2 is accepted. Furthermore, the processing device 21A functions as a display control unit 212A. The processing device 21A causes the display device 30-M to display the message.

[0138] Thereafter, the processing devices 21A in the terminal devices 20A-K, the processing device 11A in the server 10A, and the processing device 21A in the terminal devices 20A-M complete all of the processing shown in FIG.

[0139] 2-3: Advantages of the Second Embodiment According to the above description, the terminal device 20A-K as a display control device includes a generation unit 214 and a setting unit 216. The generation unit 214 generates a display control message for a user U K The setting unit 216 generates a virtual object VO1 related to the message, which is placed in a virtual reality space VS as a first virtual space visually recognized by the user U. M This sets whether or not to permit changing of the appearance of the virtual object VO2 displayed in the augmented reality space AS as the second virtual space visually recognized by the user.

[0140] Since the terminal device 20A-K has the above-mentioned configuration, the terminal device 20A-K, which is the sender of the message, can set whether or not to permit the receiver of the message to change the appearance of the virtual object VO2 representing the message. M The appearance of the virtual object VO2 differs depending on the situation. By allowing the sender of the message to set whether or not the appearance of the virtual object VO2 can be changed, the sender of the message can set whether or not the appearance of the virtual object VO2 can be changed, so that the appearance of the virtual object VO2 can be optimized for the user. M There is no need to perform any complicated operations.

[0141] 3: Variations The present disclosure is not limited to the above-described embodiments. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples may be combined.

[0142] 3-1: Variation 1 In the server 10 according to the first embodiment, the setting unit 114 sets whether or not to permit a change in the appearance of the virtual object VO2. In the terminal devices 20A-K according to the second embodiment, the setting unit 216 sets whether or not to permit a change in the appearance of the virtual object VO2. The setting units 114 and 216 may further set whether to permit a change in the entire appearance of the virtual object VO2 or to permit a change in part of the appearance of the virtual object VO2.

[0143] Fig. 11 shows an example of a virtual object VO2. The virtual object VO2 shown in Fig. 11 has a different shape from the virtual object VO2 shown in Fig. 3. The virtual object VO2 shown in Fig. 11 has a protrusion portion P1 and a spherical portion P2. The setting unit 114 and the setting unit 216 may set, for example, whether to permit the entire appearance of the virtual object VO2 to be changed, or to permit the change of only one of the protrusion portion P1 and the spherical portion P2.

[0144] As a result, the server 10 or the terminal device 20A-K receives the user U M You can change only parts of the appearance of the virtual object VO2 to suit your needs.

[0145] 3-2: Variation 2 The server 10 according to the first embodiment includes a registration unit 112 and stores a registration information database RD. However, the server 10 may not include the registration unit 112 and may not store the registration information database RD. In this case, the acquisition unit 111 may acquire the virtual space information directly from the terminal device 20-M instead of acquiring the virtual space information by referring to the registration information database RD. The same is true for the server 10A according to the second embodiment.

[0146] 3-3: Variation 3 In the first embodiment, the virtual space information is “a user U corresponding to the destination information.” MThe virtual space information is not limited to the information. For example, as described above, the registration information database RD includes information on the user U, K and User U M And user U K and User U M The virtual space information is stored in one-to-one correspondence with the types of the display devices 30-K and 30-M mounted on the heads of the users U. M In this case, the setting unit 114 of the server 10 may set the device information of the display device 30-M that the user U is wearing based on the device information of the display device 30-M. M In more detail, the user U sets whether or not to permit a change in the appearance of the virtual object VO2 in the second virtual space viewed by the user U. M The display device 30-M worn by the user U is M The difference is whether the virtual space provided to the user U is a virtual reality space, an augmented reality space, or a mixed reality space. Therefore, the setting unit 114 of the server 10 can set whether or not to permit the change based on the device information of the display device 30-M as virtual space information. K and User U M And user U K and User U M The virtual space information may be stored in one-to-one correspondence with the types of the terminal devices 20-K and 20-M used by the user U. M The same applies to the server 10A according to the second embodiment.

[0147] 3-4: Variation 4 In the information processing system 1A according to the second embodiment, an acquisition unit 111A provided in a server 10A acquires virtual space information by referring to a registration information database RD, and a setting unit 114A uses the virtual space information to set a user U MHowever, the acquisition unit 211A included in the terminal device 20A-K reads the registration information database RD from the server 10A to acquire virtual space information, and the setting unit 216 uses the virtual space information to allow the user U to change the appearance of the virtual object VO2 in the second virtual space. M Alternatively, the server 10A may set whether or not to permit a change in the appearance of the virtual object VO2 in the second virtual space visually recognized by the user. In this case, the server 10A does not require the setting unit 114A as a necessary component.

[0148] 3-5: Variation 5 In the information processing system 1A according to the second embodiment, the generating unit 113A of the server 10A generates change information related to a change in the appearance of the virtual object VO2 displayed in the second virtual space based on the environmental information acquired from the terminal device 20A-M. However, the terminal device 20A-M may generate the change information instead of the server 10A. More specifically, when the server 10A outputs the possibility information to the terminal device 20A-M and the possibility information acquired by the terminal device 20A-M indicates that the change in the appearance of the virtual object VO2 is permitted, the terminal device 20A-M may generate the change information based on the environmental information. After that, the generating unit 214 of the terminal device 20A-M generates the virtual object VO2 using the image information stored in the storage device 22A and the above change information.

[0149] Similarly, in the information processing system 1 according to the first embodiment, the terminal device 20-M may generate modification information based on environmental information. In this case, the terminal device 20-M first displays the virtual object VO2 on the display device 30-M using image information acquired from the server 10, and then applies the modification information to the image information to modify the appearance of the virtual object VO2.

[0150] 3-6: Variation 6 In the information processing system 1A according to the second embodiment, the reception unit 215 included in the terminal device 20A-K receives a user U's request regarding whether or not to permit a change in the appearance of the virtual object VO1. KIn addition, the setting unit 216 of the terminal device 20A-K receives the operation of the user U based on the operation received by the reception unit 215. M In the previous example, the terminal device 20A-K set whether or not to permit a change in the appearance of the virtual object VO2 displayed in the second virtual space visually recognized by the user. However, the terminal device 20A-K may not require the reception unit 215 and the setting unit 216 as essential components. In this case, the terminal device 20A-K only generates the virtual object VO1, and the server 10A sets whether or not the appearance of the virtual object VO2 can be changed, as in the information processing system 1 according to the first embodiment.

[0151] 3-7: Variation 7 In the information processing system 1A according to the second embodiment, the terminal device 20A-K includes a reception unit 215 and a setting unit 216, whereas in the information processing system 1 according to the first embodiment, the terminal device 20-K does not include components similar to these. However, the terminal device 20-K may include components similar to the reception unit 215 and the setting unit 216. As a result, also in the information processing system 1 according to the first embodiment, the terminal device 20-K can set whether or not to permit a change in the appearance of the virtual object VO2.

[0152] 3-8: Variation 8 In the information processing system 1 according to the first embodiment, the server 10 autonomously sets whether or not to permit the appearance of the virtual object VO2 to be changed. In the information processing system 1A according to the second embodiment, the terminal device 20A-K includes the reception unit 215 and the setting unit 216, so that the user U K Based on the operation of the user U, the server 10 or the terminal device 20A-K sets whether or not to permit the appearance of the virtual object VO2 to be changed. K In this case, the user U may be asked whether to allow or disallow the change. K The user performs an operation to designate whether to set permission or non-permission of the change from the terminal device 20-K, and the server 10 or the terminal device 20-K sets permission or non-permission of the change based on the operation.

[0153] 3-9: Variation 9 In the information processing system 1 according to the first embodiment, the terminal device 20-K and the display device 30-K are realized as separate entities. However, the method of realizing the terminal device 20-K and the display device 30-K in the embodiment of the present invention is not limited to this. For example, the display device 30-K may have the same functions as the terminal device 20-K. In other words, the terminal device 20-K and the display device 30-K may be realized in a single housing. The same applies to the terminal device 20-M and the display device 30-M. The same applies to the information processing system 1A according to the second embodiment.

[0154] 4:Other (1) In the above-described embodiment, storage devices 12 and 12A, storage devices 22 and 22A, and storage device 32 are exemplified by ROM and RAM, but may be flexible disks, magneto-optical disks (e.g., compact disks, digital versatile disks, Blu-ray (registered trademark) disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy (registered trademark) disks, magnetic strips, databases, servers, or other suitable storage media. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communications network NET via a telecommunications line.

[0155] (2) In the above-described embodiments, the described information, signals, etc. may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0156] (3) In the above-described embodiment, the input / output information, etc. may be stored in a specific location (e.g., memory) or may be managed using a management table. The input / output information, etc. may be overwritten, updated, or added. The output information, etc. may be deleted. The input information, etc. may be transmitted to another device.

[0157] (4) In the above-described embodiments, the determination may be made based on a value represented using one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0158] (5) The order of the process steps, sequences, flow charts, etc. illustrated in the above-described embodiments may be changed without causing any inconsistency. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0159] (6) Each function illustrated in Fig. 1 to Fig. 11 is realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized by using one device that is physically or logically combined, or may be realized by using two or more devices that are physically or logically separated and directly or indirectly connected (for example, by wire, wirelessly, etc.). The functional block may be realized by combining the one device or the multiple devices with software.

[0160] (7) The programs exemplified in the above-described embodiments should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., regardless of whether they are called software, firmware, middleware, microcode, hardware description language, or by other names.

[0161] Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if the software is transmitted from a website, server, or other remote source using wired and / or wireless technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave, etc.), then these wired and / or wireless technologies are included within the definition of transmission media.

[0162] (8) In each of the above embodiments, the terms “system” and “network” are used interchangeably.

[0163] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0164] (10) In the above-described embodiment, the terminal devices 20-1 to 20-J, 20A-K, and 20A-M, and the servers 10 and 10A may be mobile stations (MS). A mobile station may be referred to by those skilled in the art as a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other suitable term. In the present disclosure, the terms "mobile station", "user terminal", "user equipment (UE)", "terminal", etc. may be used interchangeably.

[0165] (11) In the above-mentioned embodiments, the terms "connected" and "coupled" or any variation thereof refer to any direct or indirect connection or coupling between two or more elements, including the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, "connected" may be read with "access". As used in this disclosure, two elements are considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and light (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0166] (12) In the above embodiments, the phrase "based on" does not mean "based only on," unless otherwise specified. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0167] (13) The terms "determining" and "determining" as used in this disclosure may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, search, inquiry (e.g., searching in a table, database, or other data structure), ascertaining, and the like. Also, "determining" and "determining" may include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in a memory), and the like. In addition, "judgment" and "decision" can include considering resolving, selecting, choosing, establishing, comparing, etc., to be a "judgment" or "decision." In other words, "judgment" and "decision" can include considering some action to be a "judgment" or "decision." In addition, "judgment" can be interpreted as "assuming," "expecting," "considering," etc.

[0168] (14) In the above embodiments, when "include," "including," and variations thereof are used, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, the term "or" used in this disclosure is not intended to be an exclusive or.

[0169] (15) In this disclosure, where articles have been added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural.

[0170] (16) In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combined" may also be interpreted in the same way as "different."

[0171] (17) Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched according to execution. In addition, notification of specific information (e.g., notification that "X is the case") is not limited to explicit notification, but may be implicit (e.g., not notifying the specific information).

[0172] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented as modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is for illustrative purposes only and does not have any limiting meaning on the present disclosure. [Explanation of symbols]

[0173] 1, 1A...information processing system, 10, 10A...server, 11, 11A...processing device, 12, 12A...storage device, 13...communication device, 14...display, 15...input device, 20, 20A...terminal device, 21, 21A...processing device, 22, 22A...storage device, 23...communication device, 24...display, 25...input device, 26...inertial sensor, 30...display device, 31...processing device, 32...storage device, 33...gaze detection device, 34...GPS device, 35...motion detection device, 36...imaging device, 37...environmental sensor, 38...communication device, 39...display, 41L , 41R...lens, 91, 92...temples, 93...bridge, 94, 95...frame, 111, 111A...acquisition unit, 112...registration unit, 113, 113A...generation unit, 114, 114A...setting unit, 115...change unit, 116, 116A...output unit, 211, 211A...acquisition unit, 212, 212A...display control unit, 213, 213A...output unit, 214...generation unit, 215...reception unit, 216...setting unit, 311...acquisition unit, 312...display control unit, P1...protrusion unit, P2...sphere unit, PR1 to PR3A...control program, VO1, VO2...virtual object

Claims

1. a generating unit that is disposed in a first virtual space visually recognized by a user who is a sender of a message, and that generates a virtual object related to the message; a setting unit that sets whether or not to permit a change in the appearance of the virtual object displayed in a second virtual space viewed by a user who is a destination of the message; an acquisition unit that acquires virtual space information that can distinguish whether the second virtual space is a virtual reality space, an augmented reality space, or a mixed reality space; The setting unit sets whether or not to permit a change in an appearance of the virtual object based on the virtual space information.

2. Further comprising a change unit, the acquisition unit acquires environmental information indicating an environment in a real space of a destination of the message; The display control device according to claim 1 , wherein the change unit changes the appearance of the virtual object in the second virtual space based on the environmental information when the second virtual space is an augmented reality space or a mixed reality space and the change of the appearance is permitted by the setting unit.

3. 3. The display control device according to claim 1, wherein, when the appearance is changeable, the setting unit further sets whether or not changing of the entire appearance of the virtual object is permitted, or whether changing of part of the appearance of the virtual object is permitted.

Images