Control for use of information processing devices in virtual space

Abstract

A virtual space system according to one aspect of the present disclosure comprises one or more processors, wherein a first detecting process, a first displaying process, a second detecting process, an acquiring process, and a second displaying process are performed by at least one of the one or more processors. The first detecting process detects a communication device object selected from among a plurality of communication device objects in a virtual space. The first displaying process displays, in a predetermined display area, a display screen of a communication device of the same type as the type represented by the selected communication device object, the communication device being realized by a virtual machine. The second detecting process detects an operation with respect to the display screen in the display area. The acquiring process inputs information indicating the operation into the virtual machine, and thereby acquires a display screen that is displayed in accordance with the operation on the communication device achieved by the virtual machine. The second displaying process displays, in the display area, the display screen that is displayed in accordance with the operation.

Description

[Technical Field]

[0001] The present disclosure relates to a technology for using an information processing device provided in a virtual store represented in a virtual space. [Background technology]

[0002] In recent years, many technologies have been developed to allow users to use virtual stores that are displayed (expanded) in virtual spaces (virtual reality spaces). For example, Patent Document 1 discloses a technology for displaying details of a predetermined product on a second terminal device in response to a user's selection of the predetermined product in a virtual store displayed on a first terminal device. Furthermore, Patent Document 2 discloses a technology for changing (customizing) the specifications of a product selected by a user in a virtual store and paying for the product after the specifications have been changed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-131661 [Patent Document 2] Japanese Patent Application Publication No. 2022-36691 Summary of the Invention [Problem to be solved by the invention]

[0004] In a virtual store such as that described in the above document, products in the real world can be sold. This allows users to purchase products in the real world while in the virtual space. However, depending on the product, users may want to check the usability and convenience of the product. For example, in a virtual store that sells communication devices that can be used in the real world, users may want to check the usability and convenience of the communication devices. However, until now, no mechanism has been provided for using communication devices that can be used in the real world in the virtual space.

[0005] In view of such problems, the present disclosure aims to provide a mechanism for using, in a virtual space, an information processing device that can be used in a real space. [Means for solving the problem]

[0006] In order to solve the above problem, a virtual space system according to one aspect of the present disclosure includes one or more processors, and at least one of the one or more processors executes a first detection process, a first display process, a second detection process, an acquisition process, and a second display process. The first detection process is a process of detecting a communication device object selected from a plurality of communication device objects in a virtual space. The first display process is a process of displaying a display screen of a communication device, realized by a virtual machine, of the same model as represented by the selected communication device object in a predetermined display area. The second detection process is a process of detecting an operation on the display screen in the display area. The acquisition process is a process of inputting information indicating the operation into the virtual machine, thereby acquiring a display screen of the communication device, realized by the virtual machine, to be displayed in accordance with the operation. The second display process is a process of displaying the display screen, displayed in accordance with the operation, in the display area.

[0007] In order to solve the above problem, a virtual space control method according to one embodiment of the present disclosure includes a first detection step, a first display step, a second detection step, an acquisition step, and a second display step. In the first detection step, a communication device object selected from a plurality of communication device objects in a virtual space is detected. In the first display step, a display screen of a communication device realized by a virtual machine and having the same model as that represented by the selected communication device object is displayed in a predetermined display area. In the second detection step, an operation on the display screen in the display area is detected. In the acquisition step, information indicating the operation is input to the virtual machine, thereby acquiring a display screen displayed in accordance with the operation of the communication device realized by the virtual machine. In the second display step, the display screen displayed in accordance with the operation is displayed in the display area. [Effects of the Invention]

[0008] According to the technology of the present disclosure, a user can use, in a virtual space, an information processing device that can be used in a real space. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example configuration of a virtual space system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 illustrates an example configuration of a head-mounted display (HMD) according to an embodiment of the present disclosure. [Figure 3] FIG. 3 illustrates an example configuration of a VR content distribution device according to an embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates an example configuration of a screen generating device according to an embodiment of the present disclosure. [Figure 5] FIG. 5 shows an example of a communication sequence diagram between an HMD, a VR content distribution device, and a screen generation device. [Figure 6] FIG. 6 shows another example of a communication sequence diagram between an HMD, a VR content distribution device, and a screen generation device. [Figure 7A] FIG. 7A shows an example of a VR image displayed on an HMD. [Figure 7B] FIG. 7B shows an example of a VR image for selecting a user terminal to use in a virtual space. [Figure 7C] FIG. 7C shows an example of a VR image for selecting a terminal usage mode. [Figure 7D] FIG. 7D shows an example of a VR image in which a user terminal dedicated to an avatar operated by a user is displayed. [Figure 7E] FIG. 7E shows an example of a VR image including a display screen with embedded screen data. [Figure 7F] FIG. 7F shows an example of a VR image in which a user terminal dedicated to the user is superimposed. [Figure 7G] FIG. 7G shows an example of a VR image including a display screen displaying screen data. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Among the components disclosed below, components having the same functions are designated by the same reference numerals, and their description will be omitted. Note that the embodiment disclosed below is one form of the present disclosure, and should be appropriately modified or changed depending on the configuration of the device and various conditions, and is not limited to the following embodiment. Furthermore, not all of the combinations of features described in the present embodiment are necessarily essential to solving the above-mentioned problems.

[0011] In this embodiment, an example will be described in which a real user (human) uses a virtual reality (VR) space on the Internet where he or she acts as an alter ego called an avatar, or the metaverse, which is a mechanism for such a space. In the following description, the term virtual reality space (or virtual space) means a virtual space, such as the metaverse, where a user can act as an alter ego of the user.

[0012] [Virtual space system configuration] FIG. 1 shows an example of the configuration of a virtual space system 1 according to this embodiment. The virtual space system 1 includes a head-mounted display (HMD) 10, a VR content distribution device 20, and a screen data generation device 30. The HMD 10 is a device that can be worn on the head of a user (player) 12. The HMD 10 outputs VR content that constitutes a virtual space, distributed by the VR content distribution device 20, and the user 12 can view the VR content. The VR content includes at least VR images (image data that constitutes the virtual space) and may also include metadata and audio that indicate information about objects in the virtual space. In this embodiment, the term "image" is understood to include still images and / or videos. The user 12 can enjoy the virtual space by viewing the VR content. The VR content distribution device 20 and the screen data generation device 30 constitute a virtual reality (VR) system.

[0013] The HMD 10 may be a binocular type or a monocular type. The HMD 10 may be a see-through type that completely covers the eyes of the user 12, or may be a see-through type. A see-through type can provide a more immersive feeling. The HMD 10 may also be a goggle-type, eyeglass-type, or hat-type device. The user 12 can perform various controls and settings, such as controlling an avatar corresponding to the user 12, by holding the controller 11 and performing input operations. For example, the user 12 can perform desired input operations by tilting the controller 11 or pressing a button (not shown) provided on the controller 11. Note that while FIG. 1 shows an example in which the controller 11 is configured as a single piece of hardware, the controller 11 may also be configured as two controllers, one for the right hand and one for the left hand. Furthermore, the controller 11 is not limited to a form that can be held by the user 12, and may have any form as long as it allows input operations by the user 12.

[0014] The VR content distribution device 20 is a device that constitutes a VR system, and specifically, provides the VR content (i.e., a virtual space) to the user 12 by distributing the VR content to the HMD 10. As will be described later, the VR content distribution device 20 has a content database. The content database includes image data (VR images) representing the virtual space, model data of an avatar that is the user 12's alter ego, audio data, and the like. The VR content distribution device 20 uses the data stored in the content database to generate VR images corresponding to the movement of the HMD 10 and operations on the controller 11, and transmits (distributes) them to the HMD 10. In addition to the VR images, audio may also be transmitted to the HMD 10 in response to the movement of the HMD 10 and operations by the controller 11. The behavior of the avatar can be controlled by the operation of the controller 11 by the user 12. Note that the behavior of the avatar may also be controlled by the movement of the HMD 10.

[0015] The virtual space system 1 according to this embodiment is configured so that a user 12 can simulate the use of a communication device that can be used in real space in a virtual store that is expressed (deployed) in a virtual space provided by a VR content distribution device 20. In other words, the virtual space system 1 is configured so that a user 12 can simulate the use of a communication device that can be used in real space in the virtual space via an HMD 10.

[0016] The following describes an example of the configuration of a virtual space system that realizes such processing, an example of the configuration of each device (system) that constitutes the virtual space system, and an example of a specific processing procedure. In this embodiment, an object in the shape of a communication device is represented in a virtual store. For convenience of explanation, the object in the virtual space or virtual store will also be simply referred to as a communication device or a user terminal. An example will be described in which a user 12 selects one communication device from multiple communication devices placed in the virtual space and virtually (simulates) uses the selected communication device. Note that, although an example will be described below in which the object virtually used by the user 12 is a user terminal such as a smartphone, it may also be another communication device or an information processing device without a communication function. In other words, an information processing device with a communication function called a communication device may be virtually used, or an information processing device without a communication function may also be virtually used.

[0017] [HMD10 configuration] 2 shows an example configuration of an HMD 10 (first information processing device) according to this embodiment. The HMD 10 includes, as a hardware configuration, a CPU 101, a ROM 102, a RAM 103, an HDD 104, a display unit 105, a motion sensor 106, a controller I / F (interface) 107, and a communication unit 108. The HMD 10 may also include an external memory and an eye-gaze detection sensor (a sensor that detects the movement of the eye gaze of the user 12). The HMD 10 also includes, as functional components executed by the CPU 201, a terminal selection detection unit 111, a terminal operation detection unit 112, a user state detection unit 113, and a display control unit 114.

[0018] The CPU (Central Processing Unit) 101 is configured with one or more processors and performs overall control of the operations of the HMD 10. The CPU 101 may be replaced with one or more processors such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit). The functional configuration of the CPU 101 will be described later.

[0019] The ROM (Read Only Memory) 102 is a nonvolatile memory that stores control programs and the like required for the CPU 101 to execute processing. Note that the programs may be stored in a nonvolatile memory such as an HDD (Hard Disk Drive) 104 or an SSD (Solid State Drive), or in an external memory such as a removable storage medium (not shown). RAM (Random Access Memory) 103 is a volatile memory that functions as the main memory, work area, etc. of CPU 101. That is, when executing a process, CPU 101 loads necessary programs, etc. from ROM 102 into RAM 103 and executes the programs, etc. to realize various functional operations.

[0020] The HDD 104 stores, for example, various data and various information required when the CPU 101 performs processing using a program. The HDD 104 also stores, for example, various data and various information obtained when the CPU 101 performs processing using a program. Note that this storage may be performed using an external memory such as a nonvolatile memory such as an SSD or a removable storage medium together with the HDD 104 or instead of the HDD 104.

[0021] The display unit 105 is realized by a liquid crystal display or the like. For example, the display unit 105 is provided with a first display (first display area) set in front of the left eye of the user 12 and a second display (second display area) set in front of the right eye, enabling stereoscopic display. To enable such stereoscopic display, the display control unit 114 of the CPU 101 can generate an image for the left eye and an image for the right eye with different parallax, and display the image for the left eye on the first display and the image for the right eye on the second display. Note that the display unit 105 may be configured as an integrated display in which the first display and the second display are integrated.

[0022] The motion sensor 106 detects information indicating the posture of the user 12 wearing the HMD 10 (hereinafter referred to as posture information), such as the rotation angle and tilt of the HMD 10. The motion sensor 106 can be configured by combining a gyro sensor, an acceleration sensor, an angular acceleration sensor, and the like. The controller I / F 107 detects (receives) information indicating an operation performed by the user 12 on the controller 11 (hereinafter, operation information) from the controller 11. For example, the controller I / F 107 receives the operation information via the communication unit 108. The user's operation on the controller 11 includes, for example, pressing a button provided on the controller 11, the position, orientation, or tilt of the controller 11, and / or the level of force with which the controller 11 is gripped. In this embodiment, the HMD 10 is configured to receive the operation information from the controller 11 and transmit it to the VR content distribution device 20. Note that the controller 11 may also be configured to directly transmit the operation information to the VR content distribution device 20.

[0023] The communication unit 108 is an interface that controls communication between the HMD 10 and an external device. For example, the communication unit 108 is configured to communicate with the VR content distribution device 20 and the screen data generation device 30 via a communication network such as the Internet or a wireless LAN.

[0024] Next, the functional configuration of the CPU 101 of the HMD 10 will be described. The terminal selection detection unit 111 detects information corresponding to terminal selection (terminal selection information) from the posture information of the user 12 detected by the motion sensor 106 and the operation information of the user 12 detected by the controller I / F 107.

[0025] Terminal selection is an operation in which the user 12 selects one user terminal from multiple user terminals (communication device objects) placed in a virtual store depicted in a VR image (virtual space) displayed on the display unit 105. Terminal selection corresponds to, for example, an operation of selecting one user terminal from multiple user terminals in the virtual space using the controller 11. Alternatively, a predetermined movement or operation on the HMD 10 or the controller 11 may be set in advance to correspond to terminal selection. The terminal selection information indicates, for example, the position of the selection operation in the VR image or the virtual store. Alternatively, the terminal selection information may be information (identification information) that identifies one user terminal selected from multiple user terminals. Furthermore, the terminal selection information includes information on the usage mode (terminal usage mode) in the virtual space of the user terminal selected by the user 12. The terminal usage mode will be described later. The terminal selection information is transmitted to the VR content distribution device 20 via the communication unit 108 or from the controller 11.

[0026] The terminal operation detection unit 112 detects information corresponding to terminal operation (terminal operation information) from the posture information of the user 12 detected by the motion sensor 106 and the operation information of the user 12 detected by the controller I / F 107.

[0027] A terminal operation is an operation on a user terminal selected by the user 12. For example, a terminal operation corresponds to an operation performed on a predetermined area (also referred to as an operation area) by the controller 11. The operation area may be determined in advance, or may be determined by an operation via the controller 11 or the like. Note that the user terminal operated in the virtual space may have a different image display area and operation area, as in the case of a laptop, or may have the same image display area and operation area, as in the case of a smartphone touch panel. Therefore, the display area described below and the operation area described above may or may not match. Furthermore, even when the display area and the operation area match, they may be at least partially matched. Furthermore, predetermined movements and operations of the HMD 10 and the controller 11 may be set in advance to correspond to terminal operations. The terminal operation information may include position information and three-dimensional angle information where the terminal operation was performed. As will be described later, the terminal operation will be different depending on whether the user 12 uses a user terminal in a virtual space or a user terminal superimposed on the virtual space.

[0028] The user state detection unit 113 generates, as user state information, information that does not correspond to the above-mentioned terminal selection and terminal operation from among the posture information of the user 12 detected by the motion sensor 106 and the operation information of the user 12 detected by the controller I / F 107. The user state detection unit 113 can transmit the user state information to the VR content distribution device 20 via the communication unit 108. The user state information can also be transmitted from the controller 11. If the HMD 10 is equipped with a gaze detection sensor, the user state information can include information on the gaze movement of the user 12. Furthermore, in this embodiment, the user status information also includes information indicating an operation to terminate use of the user terminal in the virtual store that was being used by the user 12.

[0029] The display control unit 114 performs display control on the display unit 105. For example, the display control unit 114 performs control to display, on the display unit 105, a VR image generated by the VR content distribution device 20 in response to the user state information generated by the user state detection unit 113. The display control unit 114 can perform control for stereoscopic display as described above. Furthermore, the display control unit 114 may superimpose an image for viewing the display screen of a user terminal selected by the user 12 on the VR image. The image may be an image that resembles the user terminal selected by the user 12. For example, the image may include a frame of the user terminal and a display screen inside the frame. Furthermore, the display control unit 114 may obtain screen data generated by the screen data generating device 30 via the VR content distribution device 20 and display it on the display screen within the image.

[0030] [Configuration of VR content distribution device 20] 3 shows an example of the configuration of a VR content distribution device 20 (second information processing device) according to this embodiment. The VR content distribution device 20 distributes VR content to the HMD 10. When a predetermined login process (including authentication process for logging in with a non-general account) by the user 12 is required to use VR content provided by the VR content distribution device 20, the user 12 performs the login process using predetermined information (for example, an ID and password) by operating the HMD 10 or the controller 11, for example. The login process can be performed by the VR content distribution device 20.

[0031] The VR content distribution device 20 includes, as hardware components, a CPU 201, a ROM 202, a RAM 203, an HDD 204, a communication unit 205, and a content database 206. The VR content distribution device 20 also includes, as functional components executed by the CPU 201, a user state management unit 211, a content generation unit 212, a model information generation unit 213, and a terminal operation transfer unit 214. The basic components of the CPU 201, ROM 202, RAM 203, and HDD 204 are similar to those of the CPU 101, ROM 102, RAM 103, and HDD 104 in FIG. 2, and therefore description thereof will be omitted.

[0032] The communication unit 205 is an interface that controls communication between the VR content distribution device 20 and an external device. For example, the communication unit 205 is configured to communicate with the HMD 10 and the screen data generating device 30 via a communication network such as the Internet or a wireless LAN. The communication unit 205 may be configured to communicate with the controller 11.

[0033] The content database 206 is configured to store image data (VR images) constituting the virtual space, model data of avatars representing the users 12, audio data, and the like. Furthermore, in this embodiment, the content database 206 is configured to store information about a plurality of user terminals in the virtual store. Specifically, each of the plurality of user terminals in the virtual store corresponds to a user terminal that can be used in the real space, and the content database 206 stores information about the model of each user terminal in the virtual store (model information). The model information includes the manufacturer name, model number, and the like, and may be stored in association with information identifying each user terminal (identification information). Alternatively, the model information may be stored in association with the position of each user terminal placed in the virtual store (position in the virtual space or the virtual store). Furthermore, the model information is stored in association with shape data of the user terminal of the model information (an image including the frame and display screen of the user terminal).

[0034] The content database 206 and the VR content distribution device 20 may be separate devices. In this case, the VR content distribution device 20 can use the data stored in the content database 206 by communicating with the content database 206.

[0035] Next, the functional configuration of the CPU 201 of the VR content distribution device 20 will be described. The user state management unit 211 detects (receives) the user state information transmitted by the user state detection unit 113 of the HMD 10 via the communication unit 205, and outputs it to the content generation unit 212. When the user state information is information indicating an operation to terminate use of the user terminal in the virtual store, the user state management unit 211 transmits an end instruction to the screen data generation device 30 via the communication unit 205.

[0036] The content generation unit 212 generates VR content representing a virtual space from data acquired from the content database 206, based on the user state information detected by the user state management unit 211. The VR content includes VR images to be displayed on the HMD 10, and may also include audio. The content generation unit 212 distributes (transmits) the generated VR content to the HMD 20 via the communication unit 205. In this embodiment, the content generator 212 generates VR content representing a virtual store in which a plurality of user terminals are located, each of which may have identification information.

[0037] Furthermore, the content generation unit 212 can perform control to incorporate screen data received from the screen data generation device 30 via the communication unit 205 into a VR image. Specifically, the content generation unit 212 is configured to incorporate (fit) the received screen data into a display area such as a display screen of a user terminal in a virtual space. Note that the received screen data may also be transferred to the HMD 10 without being incorporated into the display area.

[0038] The model information generation unit 213 detects (receives) the terminal selection information transmitted by the terminal selection detection unit 111 (or the controller 11) of the HMD 10 via the communication unit 205, and identifies the user terminal according to the terminal selection information. Next, the model information generation unit 213 acquires model information (manufacturer name and model number) of the identified user terminal (i.e., selected by the user 12) from the content database 206. For example, if the terminal selection information is the position of a selection operation in a virtual store (or a VR image), the model information generation unit 213 acquires model information stored in the content database 206 in association with that position. Also, if the terminal selection information is user identification information, the model information generation unit 213 acquires the model information of the identified user terminal (i.e., selected by the user 12) from the content database 206. identification The model information generating unit 213 acquires model information stored in the content database 206 in association with the information. The model information generating unit 213 transmits the model information to the screen data generating device 30 via the communication unit 205. The transmitted model information indicates a request by the user 12 to use a user terminal having the model information (or to allocate a virtual machine having the model information). Furthermore, the model information generation unit 213 may output the model information to the content generation unit 212. Alternatively, the model information generation unit 213 may acquire shape data of the user terminal that is stored in the content database 206 in association with the model information.

[0039] The terminal operation transfer unit 214 is configured to receive terminal operation information transmitted by the terminal operation detection unit 112 (or the controller 11) of the HMD 10 via the communication unit 205 and transfer it to the screen data generating device 30.

[0040] [Configuration of screen data generating device 30] FIG. 4 shows an example of the configuration of a screen data generating device 30 (third information processing device) according to this embodiment. The screen data generating device 30 includes, as its hardware configuration, a CPU 301, a ROM 302, a RAM 303, a HDD 304, and a communication unit 305. The screen data generating device 30 also includes, as functional configurations executed by the CPU 301, a virtual machine (VM) management unit 311 and a screen data acquisition unit 312. The basic configurations of the CPU 301, ROM 302, and HDD 304 are similar to those of the CPU 101, ROM 102, and HDD 104 in FIG. 2, and therefore description thereof will be omitted.

[0041] The RAM 303 stores a hypervisor 321 and virtual machines (VMs) 331, 332, and 333. The virtual machine 331 includes a guest OS 341 and an application 351. The virtual machine 332 includes a guest OS 342 and an application 352. The virtual machine 333 includes a guest OS 343 and an application 353.

[0042] The hypervisor 321 is a program for constructing a virtual environment, and in the example of FIG. 4, a plurality of virtual machines 331, 332, and 333 constructed on the hypervisor 321 are run. The guest OSs are basic software for running various applications on the virtual machines. The guest OS 341 runs an application 351, the guest OS 342 runs an application 352, and the guest OS 343 runs an application 353. The application 351 is a program that runs on the guest OS 341, the application 352 is a program that runs on the guest OS 342, and the application 353 is a program that runs on the guest OS 343. 3 shows an example in which RAM 303 has three virtual machines 331, 332, and 333, but the number of virtual machines is not limited to a specific number. As will be described later, virtual machine management unit 311 can manage virtual machines, such as creating and deleting virtual machines.

[0043] In this embodiment, the application in each virtual machine is a program for virtually realizing a user terminal (specifically, at least some of the functions of the terminal) that can be used in real space. The application is configured to receive terminal operation information as input and output (generate) screen data that is displayed on the display screen of the user terminal (transitions in accordance with the terminal operation) in accordance with the terminal operation corresponding to the information. In other words, each virtual machine functions as a virtual user terminal that can generate screen data that is displayed in accordance with the operation of the user 12. Each virtual machine (virtual user terminal) is managed so as to be identifiable by the identification information (manufacturer name and model number) of the user terminal in real space. For example, virtual machines 331, 332, and 333 shown in FIG. 3 are identified by the following manufacturer name and model number. The symbol in parentheses represents the virtual machine identification information of each virtual machine, which combines the manufacturer name and model number. Virtual machine 331 → Manufacturer: Manufacturer A, Model number: a01 (Aa01) Virtual machine 332 → Manufacturer: Manufacturer A, Model number: a02 (Aa02) Virtual machine 333 → Manufacturer: Manufacturer B, Model number: b01 (Bb01) 3 may be created in advance or may be created by the virtual machine management unit 311. As will be described later, the virtual machine management unit 311 has a function of creating virtual machines.

[0044] The communication unit 305 is an interface that controls communication between the screen data generating device 30 and an external device. For example, the communication unit 305 is configured to communicate with the VR content distribution device 20 via a communication network such as the Internet or a wireless LAN.

[0045] Next, the functional configuration of the CPU 301 of the screen data generating device 30 will be described. The virtual machine management unit 311 manages virtual machines stored in the RAM 303. For example, the virtual machine management unit 311 creates virtual machines. Specifically, the virtual machine management unit 311 can create a virtual machine for model information received from the VR content distribution device 20 via the communication unit 305 (i.e., a virtual machine that virtually realizes at least some of the functions of the user terminal of the model information in real space). If a virtual machine for the received model information is already stored in the RAM 303, the virtual machine management unit 311 may use the stored virtual machine without creating a virtual machine for the received model information. Furthermore, the virtual machine management unit 311 may be configured to create a virtual machine in accordance with a predetermined instruction received via the communication unit 305.

[0046] The virtual machine management unit 311 can create multiple virtual machines with the same model information. For example, if multiple users, including user 12, select user terminals with the same model information in the virtual space, the virtual machine management unit 311 can create multiple virtual machines with the same model information. At this time, the virtual machine management unit 311 can create virtual machines that associate different identification numbers (for example, in ascending order) with the same model information. For example, if the above virtual machine 331 is the second virtual machine having the same virtual machine identification information (Aa01), the virtual machine identification information of virtual machine 331 will be Aa01-02.

[0047] In this way, the virtual machine management unit 311 is configured to be able to create a virtual machine based on the model information received from the VR content distribution device 20, and to be able to create multiple virtual machines based on the same model information. Therefore, even if only a limited number of user terminals can be placed in a virtual store due to store design, appearance, etc., the virtual machine management unit 311 can create any number of virtual machines regardless of the number. This allows, for example, multiple users or multiple avatars corresponding to multiple users to select and use the user terminal they want to use, regardless of the number of user terminals placed in the virtual store.

[0048] The virtual machine management unit 311 also manages the status of virtual machines in the RAM 303. For example, a virtual machine being used by one or more users, including the user 12 (i.e., being used by the screen data acquisition unit 312), is managed as "in use," and a virtual machine not being used by one or more users, including the user 12, is managed as "unused." The virtual machine management unit 311 may also adjust the number of virtual machines created. For example, when the number of virtual machines having the same model of communication device and a status of "unused" is equal to or less than a predetermined number, the virtual machine management unit 311 may create a virtual machine that realizes the model. This allows the virtual machine management unit 311 to respond to many operation requests for a popular model. The virtual machine management unit 311 may also reduce the number of virtual machines that realize a communication device of a model for which few operation requests are made. By managing the status in this way, the virtual machine management unit 311 can create new virtual machines and delete unnecessary (e.g., surplus) virtual machines, thereby enabling efficient management of the capacity of the RAM 303.

[0049] Furthermore, the virtual machine management unit 311 allocates a virtual machine for use by the user 12 based on the model information received from the VR content distribution device 20 (i.e., model information of the user terminal selected by the user 12). Specifically, the virtual machine management unit 311 allocates a virtual machine whose status is "unused" and which has the model information to the user 12. The virtual machine management unit 311 notifies the screen data acquisition unit 312 of the virtual machine identification information of the allocated virtual machine.

[0050] Furthermore, when the virtual machine management unit 311 receives an end-of-use instruction transmitted from the user state management unit 211 of the HMD 10 via the communication unit 305, it changes the status of the allocated virtual machine from "in use" to "unused." For example, when an end-of-use instruction is received from the HMD 10 by an operation of the user 12, the virtual machine management unit 311 changes the status of the virtual machine allocated to the user 123 from "in use" to "unused." Next, the virtual machine management unit 311 refers to the RAM 303 and counts the number of virtual machines with the "unused" status that realize communication devices of the same model as the virtual machine changed to "unused." If the number of virtual machines is less than a predetermined number, the virtual machine management unit 311 may initialize the allocated virtual machine (for example, delete and recreate it). On the other hand, if the number of virtual machines is equal to or greater than the predetermined number, the virtual machine management unit 311 may delete the allocated virtual machine.

[0051] The screen data acquisition unit 312 acquires (receives) terminal operation information from the HMD 10, the controller 11, or the terminal operation transfer unit 214 of the VR content distribution device 20 via the communication unit 305. The screen data acquisition unit 312 inputs the terminal operation information to a virtual machine having a virtual machine identification number notified by the virtual machine management unit 311. As described above, the virtual machine is configured to receive the terminal operation information as input and output (generate) screen data that transitions in accordance with the terminal operation corresponding to the information.

[0052] The HMD 10, the VR content distribution device 20, and the screen data generation device 30 may each have dedicated hardware for executing their respective functions, or may have some of their functions executed by hardware and the other parts executed by a computer running a program, or may have all of their functions executed by a computer and a program.

[0053] <Processing Flow (When Using a User Terminal by Displaying an Image from the Screen Data Generating Device 30 on an Object in a Virtual Space)> The flow of processing when using (operating) a user terminal in a virtual space according to this embodiment will be described with reference to Fig. 5. Fig. 5 shows an example of a communication sequence diagram between the HMD 10, the VR content distribution device 20, and the screen data generation device 30.

[0054] In S501, the HMD 10 logs in to the VR system via the VR content distribution device 20 in response to an operation by the user 12. During this login, the user 12 can set an avatar, which is an alter ego of the user 12 to be used in the virtual space. After the HMD 10 logs in, the content generation unit 212 of the VR content distribution device 20 generates VR content including VR images (S502) and provides the VR content to the HMD 10 via the communication unit 205 (S504). The display control unit 114 of the HMD 10 displays the VR images received via the communication unit 108 on the display unit 105 (S503).

[0055] In this embodiment, a virtual store in which multiple user terminals are placed is depicted in a VR image. An example of a VR image is shown in FIG. 7A. In a VR image 70A in FIG. 7A, multiple user terminals, including user terminals 72a, 72b, 72c, and 72d, are placed in a virtual store 71. The user terminals 72a, 72b, 72c, and 72d are user terminals selectable by the user 12, each corresponding to a user terminal available in real space and having unique model information. The model information of the user terminals 72a, 72b, 72c, and 72d may be the same or different. The user terminals 72a, 72b, 72c, and 72d have shapes (frames and display screens) based on their respective model information. An avatar 73 corresponds to the user 12. That is, the user 12 can freely move the avatar 73 in the virtual space by moving and operating the HMD 10 and the controller 11.

[0056] Next, the user state detection unit 113 of the HMD 10 transmits user state information including posture information of the HMD 10 and operation information of the controller 11 to the VR content distribution device 20 via the communication unit 108 (S505). In response to this, the VR content distribution device 20 generates VR content according to the user state information (S502) and distributes (transmits) it to the HMD 10 (S504). Such transmission of user state information (posture information and / or operation information) and distribution of VR content continues while the HMD 10 is logged in to the system (S51).

[0057] After logging in to the VR system, the user 12 uses the controller 11 (and / or HMD 10) to select one user terminal from multiple user terminals placed in the virtual store 71 (S506). FIG. 7B shows an example of a VR image for selecting a user terminal to use in the virtual space. In the VR image 70B in FIG. 7B, the user 12 selects the user terminal 72d via the avatar 73. Note that the user terminal may be selected when the avatar 73 comes into contact with the user terminal. Alternatively, in this case, a dialog box may be displayed inquiring whether or not to select the user terminal, and the user terminal may be selected by responding that the user should select it.

[0058] Next, the user 12 uses the controller 11 (and / or HMD 10) to select a usage mode (terminal usage mode) for the user terminal 72d in the virtual space. FIG. 7C shows an example of an image for selecting the terminal usage mode. In FIG. 7C, a selection area 74 is displayed superimposed on the VR image 70C. In this embodiment, the selectable terminal usage modes include two types: "in-VR mode" and "superimposed display mode."

[0059] The "in-VR mode" is a mode in which a user terminal is used by displaying the display screen of the user terminal by a virtual machine on the user terminal in a virtual space. In the "in-VR mode," the screen of the user terminal used by the user 12 in the virtual space can be seen (recognized) by other avatars. The "superimposed display mode" is a mode in which a user terminal is used by displaying the display screen of the user terminal by a virtual machine on an image superimposed on the virtual space (VR image). In other words, the display screen of the user terminal by the virtual machine is displayed in an area separate from the virtual space, not within the virtual space. In the "superimposed display mode," the screen of the user terminal used by user 12 in the virtual space cannot be seen (recognized) by other avatars. In this example, the user 12 selects the "in-VR mode" as indicated by the mark 75 (S507). By selecting the "in-VR mode", the virtual space is configured so that the user 12 can use the selected user terminal within the virtual space.

[0060] After S506 and S507, the terminal selection detection unit 111 of the HMD 10 transmits terminal selection information to the VR content distribution device 20 via the communication unit 108 (S508). The terminal selection information includes information about the user terminal selected in S506, and in this example, includes information about the location of the user terminal 72b in the virtual store 71 or identification information of the user terminal 72b, and information indicating the terminal use mode selected in S507 (="in-VR mode").

[0061] The model information generation unit 213 of the VR content distribution device 20 identifies the user terminal according to the terminal selection information received from the HMD 20 via the communication unit 205, and acquires model information of the identified user terminal from the content database 206 (S509). The model information generation unit 213 outputs the model information to the content generation unit 212. The model information generation unit 213 also transmits the model information to the screen data generation device 30 via the communication unit 205 (S510). The model information transmitted to the screen data generation device 30 corresponds to a request by the user 12 to use the user terminal having the model information (or to allocate a virtual machine of the model information).

[0062] The content generation unit 212 of the VR content distribution device 20 acquires, from the content database 206, shape data of the user terminal associated with the model information acquired from the model information generation unit 213. Next, the content generation unit 212 displays, in the VR image, the shape of the user terminal indicated by the shape data as a user terminal dedicated to the avatar 73 operated by the user 12 (S511). FIG. 7D shows an example of a VR image in which a user terminal dedicated to the avatar 73 operated by the user 12 is displayed. In the VR image 70D in FIG. 7D, the user terminal 76 is a user terminal dedicated to the avatar 73. The user terminal 76 corresponds to the user terminal 72d in the virtual store 71, and has a shape according to the model information of the user terminal 72d. Like the user terminal 72d, the user terminal 76 has a display screen (display screen 77).

[0063] The virtual machine management unit 311 of the screen data generating device 30 prepares a virtual machine (status is "unused") for the model information received from the VR content distribution device 20 via the communication unit 305. If a virtual machine (status is "unused") for the model information is stored in the RAM 303, the virtual machine is prepared; otherwise, the virtual machine management unit 311 prepares the virtual machine by creating it for the model information. Then, the virtual machine management unit 311 allocates the prepared virtual machine as a virtual machine to be used by the user 12 (i.e., the avatar 73) (S512). The virtual machine management unit 311 notifies the screen data acquisition unit 312 of virtual machine identification information of the allocated virtual machine.

[0064] When the terminal operation detection unit 112 of the HMD 10 detects a terminal operation on the display screen 77 of the user terminal 76 displayed in the virtual store 71 (S513), it transmits information corresponding to the operation (terminal operation information) to the VR content distribution device 20 via the communication unit 108 (S514). The terminal operation transfer unit 214 of the VR content distribution device 20 transfers the terminal operation information to the screen data generation device 30 (S515). The operation is, for example, at least one of a tap operation (touch operation), a slide operation, and a scroll operation by the avatar 73 on the display screen 77 in the virtual space. For example, when the user 12 moves the controller 11 or the like so that the avatar 73 taps on the display screen 77, the position of the tap operation becomes the terminal operation information. The operation position may be expressed including a three-dimensional angle.

[0065] When the screen data acquisition unit 312 of the screen data generation device 30 receives the terminal operation information via the communication unit 305, the screen data acquisition unit 312 inputs the terminal operation information to the virtual machine allocated in S512 and acquires screen data from the output of the virtual machine (S516). The acquired screen data is transmitted to the VR content distribution device 20 via the communication unit 305 (S517).

[0066] When the content generation unit 212 of the VR content distribution device 20 receives the screen data via the communication unit 205, it generates a VR image by embedding (incorporating) the screen data (i.e., an image based on the screen data) into a display screen 77 of a user terminal 76 in a virtual space (S518). FIG. 7E shows an example of a VR image including a display screen 77 into which the screen data has been embedded. In the VR image 70E in FIG. 7E, screen data 78 (hatched portion) displayed in response to an operation by the user 12 via the avatar 73 is displayed on the display screen 77. Thereafter, until the HMD 10 detects (receives) information indicating an operation to terminate use by the user 12, the display of the screen data in response to the terminal operation by the user 12 continues on the display screen 77 in the virtual space (S52).

[0067] When the user state management unit 211 of the VR content distribution device 20 detects that the user state information received from the HMD 10 is information indicating an operation to end use of the user terminal in the virtual store (S519), the user state management unit 211 transmits an end instruction to the screen data generating device 30 via the communication unit 205 (S520). This ends the use of the user terminal 76 by the user 12 in the virtual space. The end use operation is, for example, an operation by the avatar 73 to let go of the user terminal 76 or an operation to end the "in-VR mode."

[0068] When the virtual machine management unit 311 of the screen data generating device 30 receives a termination instruction from the VR content distribution device 20 via the communication unit 305, it releases the virtual machine allocated in S512 (S521). For example, the virtual machine management unit 311 changes the status of the allocated virtual machine from "in use" to "unused." Then, with reference to the RAM 303, it counts the number of virtual machines with the same model of communication device to be realized and with the status of "unused." If the number of virtual machines is less than a predetermined number, the virtual machine management unit 311 may initialize the allocated virtual machine (for example, delete and recreate it). On the other hand, if the number of virtual machines is equal to or greater than the predetermined number, the virtual machine management unit 311 may delete the allocated virtual machine. Thereafter, the HMD 10 logs out of the VR system via the VR content distribution device 20 (S522), whereby the VR content distribution device 20 stops providing the VR content to the HMD 10, and the processing of S51 ends.

[0069] <Processing Flow (When Using a User Terminal by Superimposing an Image from the Screen Data Generating Device 30 on a Virtual Space)> The flow of processing when using a virtual user terminal (operating a user terminal) superimposed on a virtual space according to this embodiment will be described with reference to Fig. 6. Fig. 5 shows another example of a communication sequence diagram between the HMD 10, the VR content distribution device 20, and the screen data generation device 30. Note that the same processes as those in Fig. 5 are assigned the same reference numerals and descriptions thereof will be omitted.

[0070] After the user 12 logs in to the VR system, the transmission of user state information (posture information and / or operation information) and the distribution of VR images continue while the HMD 10 is logged in to the system (S51). The user 12 selects a usage mode (terminal usage mode) of the user terminal 72d in the virtual space using the controller 11 (and / or the HMD 10). In this example, the "superimposed display mode" is selected in the VR image 70C selection area 74 shown in FIG. 7C (S601).

[0071] After S506 and S601, the terminal selection detection unit 111 of the HMD 10 transmits terminal selection information to the VR content distribution device 20 via the communication unit 108 (S602). The terminal selection information includes information about the user terminal selected in S506, and in this example, includes information about the location of the user terminal 72b in the virtual store 71 or identification information of the user terminal 72b, and information indicating the terminal usage mode selected in S601 (="superimposed display mode").

[0072] The model information generation unit 213 of the VR content distribution device 20 identifies the user terminal according to the terminal selection information received from the HMD 20 via the communication unit 205, and acquires model information of the user terminal from the content database 206 (S603). The model information generation unit 213 transmits the model information to the screen data generation device 30 via the communication unit 205 (S604). The model information generation unit 213 also acquires shape data of the user terminal associated with the model information, and transmits the shape data to the HMD 10 via the communication unit 205 (S605).

[0073] The display control unit 114 of the HMD 10 superimposes the shape data received from the model information generation unit 213 via the communication unit 108 onto the VR image (virtual space) displayed by the display unit 105 as a user terminal dedicated to the user 12 (S606). FIG. 7F shows an example of a VR image in which a user terminal dedicated to the user is superimposed. In the VR image 70F in FIG. 7F, the user terminal 79 is a user terminal dedicated to the user 12 superimposed on the VR image 70F. The user terminal 79 corresponds to the user terminal 72d in the virtual store 71 and has a shape according to the model information of the user terminal 72d. The user terminal 76 has a display screen (display screen 80), similar to the user terminal 72d. Note that the size of the user terminal 79 may be arbitrarily adjustable, and may be displayed at a larger size than normal for visibility.

[0074] The virtual machine management unit 311 of the screen data generating device 30 prepares a virtual machine (status is "unused") corresponding to the model information received from the VR content distribution device 20 via the communication unit 305. If a virtual machine (status is "unused") for the model information is stored in the RAM 303, the virtual machine is prepared; otherwise, the virtual machine management unit 311 prepares a virtual machine for the model information by creating it. Then, the virtual machine management unit 311 allocates the prepared virtual machine as a virtual machine for use by the user 12 (S607). The virtual machine management unit 311 notifies the screen data acquisition unit 312 of virtual machine identification information of the allocated virtual machine.

[0075] When the terminal operation detection unit 112 of the HMD 10 detects a terminal operation on the display screen 80 of the user terminal 79 superimposed on the virtual store 71 (S608), information corresponding to the operation (terminal operation information) is transmitted to the VR content distribution device 20 via the communication unit 108, and is then transferred by the VR content distribution device 20 to the screen data generation device 30 (S609). The operation is, for example, at least one of a tap operation (touch operation), a slide operation, and a scroll operation on the display screen 80. For example, when the user 12 taps on the display screen 80 with the controller 11 or the like, the position of the tap operation becomes the terminal operation information.

[0076] When the screen data acquisition unit 312 of the screen data generation device 30 receives the terminal operation information via the communication unit 305, it inputs the terminal operation information to the virtual machine allocated in S512 and acquires screen data from the output of the virtual machine (S516). The acquired screen data is transmitted to the VR content distribution device 20 via the communication unit 305, and is transferred to the HMD 10 by the VR content distribution device 20 (S610).

[0077] When the display control unit 114 of the HMD 10 receives the screen data via the communication unit 108, it displays (superimposes) the screen data (i.e., an image based on the screen data) on the display screen 80 of the user terminal 79 in the virtual space (S611). FIG. 7G shows an example of a VR image including a display screen 80 displaying the screen data. In the VR image 70G in FIG. 7G, screen data 81 (hatched portion) displayed in response to an operation by the user 12 is displayed on the display screen 80. The terminal operation on the display screen 80 may be an operation using the avatar 73, or may be an operation using a cursor 82 set by the display control unit 114. The cursor 82 may be controlled by the controller 11, for example.

[0078] Thereafter, when the user status management unit 211 of the VR content distribution device 20 detects that the user status information received from the HMD 10 is information indicating an operation to terminate use of the user terminal in the virtual store (S519), it transmits an end instruction to the screen data generating device 30 via the communication unit 205 (S520). This terminates use of the user terminal 79 superimposed on the virtual space by the user 12. The operation to terminate use is, for example, an operation to terminate the "superimposed display mode."

[0079] When the virtual machine management unit 311 of the screen data generating device 30 receives a termination instruction from the VR content distribution device 20 via the communication unit 305, it releases the virtual machine allocated in S607 (S612). For example, the virtual machine management unit 311 changes the status of the allocated virtual machine from "in use" to "unused." Then, with reference to the RAM 303, it counts the number of virtual machines with the same model of communication device to be realized and with the status of "unused." If the number of virtual machines is less than a predetermined number, the virtual machine management unit 311 may initialize the allocated virtual machine (for example, delete and reconfigure it). On the other hand, if the number of virtual machines is equal to or greater than the predetermined number, the virtual machine management unit 311 may delete the allocated virtual machine. Thereafter, the HMD 10 logs out of the VR system provided by the VR content distribution device 20 (S522), whereby the VR content distribution device 20 stops providing the VR content to the HMD 10, and the processing of S61 ends.

[0080] As described above, according to the embodiment described above, a user can virtually use a user terminal that can be used in the real space in a virtual store represented in a VR image (i.e., a virtual space) displayed on an HMD worn by the user. This allows the user to check the usability of a user terminal that can be used in the real space in the virtual space without going to a physical store in the real space.

[0081] In addition, virtual machines can be created (and deleted) as needed to provide screen data for user terminals in the virtual space. This allows users to select and use the user terminal they want to use, even if only a limited number of user terminals can be placed in the virtual store, without worrying about the number.

[0082] Although specific embodiments have been described above, these embodiments are merely examples and are not intended to limit the scope of the present disclosure. The devices and methods described herein may be embodied in forms other than those described above. Furthermore, appropriate omissions, substitutions, and modifications may be made to the above-described embodiments without departing from the scope of the present disclosure. Such omissions, substitutions, and modifications are included within the scope of the claims and their equivalents, and belong to the technical scope of the present disclosure.

[0083] (Embodiments of the present disclosure) The present disclosure includes the following embodiments. [1] A virtual space system having one or more processors, wherein at least one of the one or more processors executes a first detection process for detecting a communication device object selected from a plurality of communication device objects in a virtual space, a first display process for displaying a display screen of a communication device realized by a virtual machine that is the same model as that represented by the selected communication device object in a specified display area, a second detection process for detecting an operation on the display screen in the display area, an acquisition process for acquiring a display screen of the communication device realized by the virtual machine that is displayed in accordance with the operation by inputting information indicating the operation into the virtual machine, and a second display process for displaying the display screen that is displayed in accordance with the operation in the display area.

[0084] [2] The virtual space system described in [1], wherein the first display process includes displaying a new communication device object separate from the plurality of communication device objects and displaying the display screen inside the new communication device object.

[0085] [3] The virtual space system described in [2], wherein the shape of the new communication device object is the same as the shape of the object of the selected communication device.

[0086] [4] The virtual space system described in [2] or [3], wherein the second detection process includes detecting at least one of a tap operation, a slide operation, and a scroll operation on the display screen of the new communication device object as the operation.

[0087] [5] The virtual space system described in [1], wherein the first display process includes displaying an image simulating a frame of a communication device superimposed on an image representing the virtual space, and displaying the display screen inside the frame.

[0088] [6] The virtual space system according to [5], wherein the image simulating the frame of the communication device is simulating the frame of the selected communication device object.

[0089] [7] The virtual space system described in [5] or [6], wherein the second detection process includes detecting at least one of a tap operation, a slide operation, and a scroll operation on the display screen within the image that resembles the frame of a communication device as the operation.

[0090] [8] The virtual space system described in [1], wherein the first display process includes displaying the display screen inside one of the plurality of communication device objects.

[0091] [9] The virtual space system described in [1], wherein at least one of the one or more processors further executes a creation process for creating a plurality of virtual machines that virtually realize communication devices of the same model as each of the plurality of communication device objects represented by the plurality of communication device objects; an allocation process for allocating one of the plurality of virtual machines to the selected communication device object based on the model of the selected communication device object; and a release process for release of the allocation of the virtual machine allocated to the selected communication device object in response to an instruction to end use of the selected communication device object.

[0092]

[10] The virtual space system described in [9], wherein at least one of the one or more processors further executes a management process to manage, among the plurality of virtual machines, a virtual machine assigned to the selected communication device object as in use, and manage the other virtual machines as unused.

[0093]

[11] The virtual space system described in

[10] , wherein the creation process includes creating additional virtual machines if the number of virtual machines among the plurality of virtual machines that have the same model of communication device to be realized and are managed as unused is less than a predetermined number.

[0094]

[12] A virtual space system described in

[10] or

[11] , wherein at least one of the one or more processors further executes a deletion process to delete at least one of the plurality of virtual machines when the number of virtual machines that are managed as unused and that have the same model of communication device that they realize is greater than or equal to a predetermined number.

[0095]

[13] A virtual space system described in any of

[10] to

[12] , wherein at least one of the one or more processors further executes an initialization process to initialize the virtual machine whose allocation has been deallocated by the deallocation process.

[0096]

[14] A virtual space control method including: a first detection step of detecting a communication device object selected from a plurality of communication device objects in a virtual space; a first display step of displaying a display screen of a communication device realized by a virtual machine and having the same model as that represented by the selected communication device object in a predetermined display area; a second detection step of detecting an operation on the display screen in the display area; an acquisition step of acquiring a display screen to be displayed in accordance with the operation of the communication device realized by the virtual machine by inputting information indicating the operation into the virtual machine; and a second display step of displaying the display screen to be displayed in accordance with the operation in the display area. [Explanation of symbols]

[0097] 10: Head-mounted display (HMD), 11: Controller, 12: User, 20: VR content distribution device, 30: Screen data generation device, 101: CPU, 102: ROM, 103: RAM, 104: HDD, 105: Display unit, 106: Motion sensor, 107: Controller I / F, 108: Communication unit, 111: Terminal selection detection unit, 112: Terminal operation detection unit, 113: User state detection unit, 114: Display control unit, 201: CPU, 202: ROM, 203: RAM, 204 : HDD, 205: Communication unit, 206: Content database, 211: User state management unit, 212: Content generation unit, 213: Model information generation unit, 214: Terminal operation transfer unit, 301: CPU, 302, ROM, 303: RAM, 304: HDD, 305: Communication unit, 311: Virtual machine management unit, 312: Image data acquisition unit, 321: Hypervisor, 331; 332; 333: Virtual machine (VM), 341; 342; 343: Guest OS, 351; 352; 353: Application

Claims

1. a first detection process for detecting a communication device object selected from a plurality of communication device objects in a virtual space; a first display process, implemented by a virtual machine, for displaying a display screen of a communication device of the same model as that represented by the selected communication device object in a predetermined display area; a second detection process for detecting an operation on the display screen of the display area; an acquisition process of acquiring a display screen of the communication device realized by the virtual machine, the display screen being displayed in accordance with the operation by inputting information indicating the operation into the virtual machine; a second display process for displaying the display screen displayed in accordance with the operation in the display area; a creation process for creating a plurality of virtual machines that virtually realize communication devices of the same model as the model represented by each of the plurality of communication device objects; an allocation process of allocating one of the plurality of virtual machines to the selected communication device object based on a model of the selected communication device object; a management process for managing, among the plurality of virtual machines, a virtual machine allocated to the selected communication device object as being in use, and managing the other virtual machines as being unused; is executed, The creation process includes creating a further virtual machine if the number of virtual machines among the plurality of virtual machines that are managed as unused and have the same model of communication device to be realized is less than a predetermined number.

2. A first detection process for detecting a communication device object selected from a plurality of communication device objects in a virtual space; a first display process, implemented by a virtual machine, for displaying a display screen of a communication device of the same model as that represented by the selected communication device object in a predetermined display area; a second detection process for detecting an operation on the display screen of the display area; an acquisition process of acquiring a display screen of the communication device realized by the virtual machine, the display screen being displayed in accordance with the operation by inputting information indicating the operation into the virtual machine; a second display process for displaying the display screen displayed in accordance with the operation in the display area; a creation process for creating a plurality of virtual machines that virtually realize communication devices of the same model as the model represented by each of the plurality of communication device objects; an allocation process of allocating one of the plurality of virtual machines to the selected communication device object based on a model of the selected communication device object; a management process for managing, among the plurality of virtual machines, a virtual machine allocated to the selected communication device object as being in use, and managing the other virtual machines as being unused; a deletion process for deleting at least one virtual machine from the plurality of virtual machines when the number of virtual machines that are managed as unused and that realize the same type of communication device is equal to or greater than a predetermined number; A virtual space system in which

3. the first display process includes displaying a new communication device object separate from the plurality of communication device objects, and displaying the display screen inside the new communication device object.

3. The virtual space system according to claim 1.

4. the shape of the new communication device object has the same shape as the selected communication device object; The virtual space system according to claim 3 .

5. the second detection process includes detecting, as the operation, at least one of a tap operation, a slide operation, and a scroll operation on the display screen of the new communication device object. The virtual space system according to claim 3 .

6. the first display process includes displaying an image simulating a frame of a communication device superimposed on an image representing the virtual space, and displaying the display screen inside the frame.

3. The virtual space system according to claim 1.

7. the image simulating a frame of a communication device is a frame simulating a frame of the selected communication device object; The virtual space system according to claim 6 .

8. the second detection process includes detecting, as the operation, at least one of a tap operation, a slide operation, and a scroll operation on the display screen within the image simulating a frame of the communication device. The virtual space system according to claim 6.

9. the first display process includes displaying the display screen inside any one of the plurality of communication device objects; 3. The virtual space system according to claim 1 or 2.

10. A release process for releasing an allocation of a virtual machine allocated to the selected communication device object in response to an instruction to terminate use of the selected communication device object. The virtual space system according to claim 1 or 2, further comprising:

11. an initialization process is further executed to initialize the virtual machine whose allocation has been deallocated by the deallocation process among the plurality of virtual machines; The virtual space system according to claim 10.

12. A virtual space control method executed by an information processing device, comprising: a first detection step of detecting a communication device object selected from a plurality of communication device objects in a virtual space; a first display step of displaying, in a predetermined display area, a display screen of a communication device that is the same model as the communication device represented by the selected communication device object, which is realized by a virtual machine; a second detection step of detecting an operation on the display screen of the display area; an acquisition step of acquiring a display screen displayed in accordance with the operation of the communication device realized by the virtual machine by inputting information indicating the operation into the virtual machine; a second display step of displaying the display screen displayed in accordance with the operation in the display area; a creation step of creating a plurality of virtual machines that virtually realize communication devices of the same model as the model represented by each of the plurality of communication device objects; an allocating step of allocating one of the plurality of virtual machines to the selected communication device object based on a model of the selected communication device object; a management step of managing, among the plurality of virtual machines, a virtual machine allocated to the selected communication device object as being in use, and managing the other virtual machines as being unused; Including, A virtual space control method, wherein the creation process includes creating additional virtual machines if the number of virtual machines among the plurality of virtual machines that have the same model of communication device to be realized and are managed as unused is less than a predetermined number.

13. A virtual space control method executed by an information processing device, comprising: a first detection step of detecting a communication device object selected from a plurality of communication device objects in a virtual space; a first display step of displaying, in a predetermined display area, a display screen of a communication device that is the same model as the communication device represented by the selected communication device object, which is realized by a virtual machine; a second detection step of detecting an operation on the display screen of the display area; an acquisition step of acquiring a display screen displayed in accordance with the operation of the communication device realized by the virtual machine by inputting information indicating the operation into the virtual machine; a second display step of displaying the display screen displayed in accordance with the operation in the display area; a creation step of creating a plurality of virtual machines that virtually realize communication devices of the same model as the model represented by each of the plurality of communication device objects; an allocating step of allocating one of the plurality of virtual machines to the selected communication device object based on a model of the selected communication device object; a management step of managing, among the plurality of virtual machines, a virtual machine allocated to the selected communication device object as being in use, and managing the other virtual machines as being unused; a deletion step of deleting at least one virtual machine from the plurality of virtual machines when the number of virtual machines that are managed as unused and that realize the same type of communication device is equal to or greater than a predetermined number; A virtual space control method comprising:

Images