Virtual space display device, virtual space display method, program, and recording medium

Abstract

The purpose of the present invention is to provide a device, a method, a program, and a recording medium with which it is possible to display a virtual space that more closely approximates a real space.　In one embodiment of the present invention, a processor performs: a process for acquiring first information regarding an actual image capture device; a process for selecting second information corresponding to the first information from a database in which object information regarding image capture device objects that can be displayed in a virtual space and second information regarding image capture devices corresponding to the object information are stored in association with each other; and a process for displaying, on a display device, a virtual space in which an image capture device object based on the object information associated with the second information corresponding to the first information is drawn.

Description

Virtual space display device, virtual space display method, program, and recording medium

[0001] The present invention relates to a virtual space display device, a virtual space display method, a program, and a recording medium for displaying a virtual space in which an object is drawn on a display device.

[0002] In a technology that provides a user with an image of a virtual space such as a virtual reality (VR) space or a metaverse space, for example, an object that moves in conjunction with the user's actions, such as an avatar or a character, may be displayed in the virtual space. In this case, the user can have various experiences in the virtual space through the object, and can, for example, interact with other users in the virtual space (see, for example, Patent Document 1).

[0003] Patent Literature 1 describes a technology for moving a character in a virtual space in accordance with the actual movements of a user. This technology allows two or more users to respectively move characters displayed in the virtual space in accordance with the movements of their faces or eyes, thereby recreating communication that is close to reality in the virtual space.

[0004] JP 2017-55851 A

[0005] When a virtual space is provided to a user using the technology described in Patent Literature 1 or the like, it is expected that the user will feel a sense of familiarity with the virtual space if the virtual space is made closer to real space. On the other hand, while conventional technologies including the technology described in Patent Literature 1 can reproduce communication close to reality in the virtual space, further improvements are required to make the virtual space closer to real space to the extent that the user will feel a sense of familiarity.

[0006] One embodiment of the present invention has been made in consideration of the above circumstances, and aims to provide an apparatus, method, program, and recording medium that can display a virtual space that is closer to real space.

[0007] The above object is achieved by a virtual space display device according to any one of [1] to

[16] below. [1] A virtual space display device including a processor for displaying a virtual space on a display, wherein the processor executes the following processes: acquiring first information about an actual imaging device; selecting second information corresponding to the first information from a database in which object information about imaging device objects displayable in a virtual space and second information about the imaging devices corresponding to the object information are stored in association with each other; and displaying on the display a virtual space in which an imaging device object based on the object information associated with the second information according to the first information is drawn. [2] The virtual space display device according to [1], wherein the processor acquires the first information based on imaging device information acquired in a real space in which an actual imaging device is placed. [3] The virtual space display device according to [1] or [2], wherein the first information and the second information include information about an image. [4] The virtual space display device according to any one of [1] to [3], wherein the processor acquires first information based on imaging device information acquired in a real space in which an actual imaging device is placed, the imaging device information being an image of the actual imaging device, the first information including feature quantities of the image of the actual imaging device, and the second information including feature quantities of the image of the imaging device corresponding to the object information. [5] The virtual space display device according to [1] or [2], wherein the first information and the second information include text information. [6] The virtual space display device according to [1], [2], or [5], wherein the processor acquires first information based on imaging device information acquired in a real space in which an actual imaging device is placed, the imaging device information being information output from a user terminal based on a user's input operation performed on the actual imaging device, the first information including matching information based on the output information from the user terminal, and the second information including matching information registered for the imaging device corresponding to the object information. [7] A virtual space display device described in any of [1] to [6], wherein in the process of selecting second information corresponding to first information, the processor selects second information from a database whose degree of similarity with the first information satisfies a selection condition.[8] The virtual space display device according to any one of [1] to [7], wherein the second information includes information on attributes of an imaging device corresponding to the object information, and functions of the imaging device corresponding to the attributes are associated with the imaging device object, and when an operation according to the function is instructed for the imaging device object, the processor performs a visual presentation based on the function in the virtual space. [9] The virtual space display device according to [8], wherein a user's avatar that moves according to the user's movements is rendered in the virtual space, and when the avatar performs an operation according to the function for the imaging device object, the processor performs a visual presentation based on the function in the virtual space.

[10] The virtual space display device according to [8] or [9], wherein the first information is information specific to the model of an actual imaging device, and the second information is information specific to the model of an imaging device corresponding to the object information.

[11] The virtual space display device according to

[10] , wherein the number and type of functions associated with the imaging device object are determined according to the model of an imaging device corresponding to the object information of the imaging device object.

[12] The virtual space display device according to [9], wherein the imaging device object is associated with a function for generating a print of the captured image, and when an avatar performs a printing action on the imaging device object, the processor implements a visual effect in which a print object corresponding to the print is generated by the imaging device object in the virtual space.

[13] The virtual space display device according to

[12] , wherein when the avatar performs a printing action on the imaging device object, the processor implements a visual effect in which a print object including an extracted image extracted from a range corresponding to the imaging device object is generated by the imaging device object in the virtual space, and causes a real imaging device having a function for generating a print to generate a print on which an image based on the extracted image is printed.

[14] The virtual space display device according to

[13] , wherein the processor causes a real imaging device having a function for generating a print to output a print on which a composite image in which another image is superimposed on the extracted image in the print object is printed.

[15] The virtual space display device according to [9], wherein the imaging device object is associated with object information and a function related to imaging by an imaging device corresponding to the object information, and when an avatar performs an imaging action on the imaging device object, the processor performs a visual effect in the virtual space based on the imaging function.

[16] The virtual space display device according to

[15] , wherein when the avatar performs an imaging action on the imaging device object, the processor obtains an extracted image by extracting a range in the virtual space corresponding to the imaging device object, and stores the extracted image in an area in the real space that can be stored by a real imaging device.

[0008] The object of one embodiment of the present invention described above is achieved by a virtual space display method described in

[17] to

[19] below.

[17] A virtual space display method for displaying a virtual space on a display device, the virtual space display method including the steps of: acquiring first information about an actual imaging device by a processor; selecting second information corresponding to the first information from a database in which object information about imaging device objects displayable in the virtual space and second information about the imaging devices corresponding to the object information are stored in association with each other; and displaying, on the display device, a virtual space in which an imaging device object based on the object information associated with the second information corresponding to the first information is drawn by the processor.

[18] The virtual space display method described in

[17] , in which, in the step of selecting second information corresponding to the first information, the processor selects second information from the database whose degree of similarity to the first information satisfies a selection condition.

[19] The virtual space display method described in

[17] or

[18] , further comprising a step in which the second information includes information on attributes of the imaging device corresponding to the object information, the imaging device object is associated with a function of the imaging device corresponding to the attribute, and when an operation according to the function is instructed to the imaging device object, the processor performs a visual presentation based on the function in the virtual space.

[0009] A program according to one embodiment of the present invention is a program for causing a computer to execute each step included in the virtual space display method described in any one of

[17] to

[19] above. A recording medium according to one embodiment of the present invention is a computer-readable recording medium on which a program for causing a computer to execute each step included in the virtual space display method described in any one of

[17] to

[19] is recorded.

[0010] According to one embodiment of the present invention, by drawing an object corresponding to an imaging device recognized in real space in a virtual space, it is possible to link the real space and the virtual space and provide the user with a virtual space that is closer to real space.

[0011] 1 is a diagram showing a user using a virtual space displayed by a virtual space display device according to an embodiment of the present invention. FIG. 2 is a diagram showing a virtual space displayed by a virtual space display device according to an embodiment of the present invention. FIG. 3 is a diagram showing a user using a virtual space in which an imaging device object is drawn by a virtual space display device according to an embodiment of the present invention, and a real imaging device. FIG. 4 is a diagram showing a virtual space in which an imaging device object is drawn by a virtual space display device according to an embodiment of the present invention, and the imaging device object. FIG. 5 is a diagram showing a state in which a user instructs an imaging device to perform an operation based on an imaging function in real space. FIG. 6 is a diagram showing a state in which an avatar performs an operation according to the imaging function on an imaging device object in virtual space. FIG. 7 is a diagram showing a virtual space in which a visual effect in which a printed object is generated is performed. FIG. 8 is a diagram showing a state in which a printed object is generated and output by a real imaging device in real space. FIG. 9 is a diagram showing a state in which a printed object is being handed over to an avatar of another user in virtual space. FIG. 10 is a diagram showing an information processing system including a virtual space display device according to an embodiment of the present invention. FIG. 11 is a diagram showing an example of object information. FIG. 12 is a diagram showing an example of second information. FIG. 13 is a diagram showing a virtual space display flow according to an embodiment of the present invention. FIG. 14 is a diagram showing a virtual space displayed by a virtual space display device according to a modified example of the present invention. FIG. 15 is an explanatory diagram of visual effect performed in a virtual space displayed by a virtual space display device according to a modified example of the present invention. FIG. 10 is a diagram showing a part of a virtual space displayed by a virtual space display device according to a modified example of the present invention, displayed on the monitor of an actual imaging device in real space.

[0012] Specific embodiments of the present invention will be described below. In the following description, for convenience of explanation, the description may be given from the perspective of a GUI (Graphical User Interface).

[0013] Furthermore, the basic data processing technologies (communication / transmission technologies, data acquisition technologies, data recording technologies, data processing / analysis technologies, machine learning technologies, image processing technologies, image display technologies, visualization technologies, etc.) required to realize the present invention are well-known technologies, and therefore a description thereof will be omitted. It should be noted that the devices and apparatuses used to apply the above-mentioned well-known technologies may be appropriately selected from those available at the time of implementing the present invention.

[0014] In this specification, the concept of "device" includes not only a single device that performs a specific function, but also a combination of multiple devices that exist independently and in a distributed manner but cooperate (link) to perform a specific function. In addition, in this specification, the concept of "system" includes a system that is composed of multiple devices connected in a state where they can communicate with each other, and a system that is composed of a single device.

[0015] In this specification, a "user" refers to a person who uses the virtual space display device of the present invention for the purpose of enjoying the benefits of implementing the present invention. Specifically, a user of the present invention is a person who is provided with a virtual space that can be displayed by the virtual space display device of the present invention and who uses the space collaboration service described below within the virtual space.

[0016] In addition, in this specification, the term "person" refers to an entity that performs a specific action, and includes individuals, groups, corporations such as companies, and organizations, as well as computers and devices that constitute artificial intelligence (AI). Artificial intelligence (AI) is a technology that realizes intelligent functions such as inference, prediction, and judgment using hardware and software resources.

[0017] Additionally, in this specification, machine learning algorithms may include neural networks, convolutional neural networks, recurrent neural networks, attention, transformers, variational autoencoders, generative adversarial networks, deep learning neural networks, Boltzmann machines, matrix factorization, factorization machines, m-way factorization machines, field-aware factorization machines, field-aware neural factorization machines, support vector machines, Bayesian networks, decision trees, and random forests, as well as other machine learning algorithms.

[0018] Furthermore, in this specification, "space" may be either a three-dimensional space or a two-dimensional space, but in the following description, it will be assumed to be a three-dimensional space unless otherwise specified.

[0019] <<About one embodiment of the present invention>> In one embodiment of the present invention (hereinafter referred to as the first embodiment), a virtual space is displayed on a display used by a user, and the user can use services that can be provided by the present invention within the virtual space.

[0020] A virtual space (hereinafter referred to as virtual space V) is, for example, a space constructed and reproduced by a computer, and specifically includes a virtual reality (VR) space, an augmented reality (AR) space, and a mixed reality (MR) space. The virtual space V may also include a commercial virtual space such as the metaverse. One or more objects are rendered within the virtual space V. The objects are elements that make up the virtual space V, and include, for example, people (specifically, avatars), characters, objects, living things, vehicles, plants, buildings, and backgrounds that can be placed within the virtual space V.

[0021] The display device is, for example, a goggle-type head-mounted display (hereinafter referred to as HMD 30) that can be worn by a user on the head, as shown in FIG. 1A . Wearing the goggle-type HMD 30 makes it easier for the user to immerse themselves in the virtual space V displayed on the HMD 30. However, the display device is not limited to the HMD 30 and may be a stationary display or a large monitor installed in a store or the like. The display device may also be a touch panel display mounted on a tablet terminal or a smartphone, or a display on a wearable device in the form of glasses, such as smart glasses.

[0022] 1B, a humanoid object corresponding to the user, i.e., an avatar Ab, is rendered in the virtual space V. The avatar Ab moves in accordance with the user's movements within the virtual space V. That is, the user's movements are detected by a detection device in the real space R, and the avatar Ab that moves in accordance with the detected movements is rendered in the virtual space V. As the detection device, a wearable device 40 that detects the user's physical movements can be used, as shown in FIG. 1A.

[0023] The wearable device 40 includes, for example, a device equipped with a gyro sensor, a device equipped with an angular velocity sensor, a device equipped with an acceleration sensor, etc. Specifically, the wearable device 40 includes a smart watch, a device for motion capture, etc. The wearable device 40 may also include a glove-type device equipped with a tactile sensor, such as a so-called tactile feedback glove. Note that the detection device is not limited to the wearable device 40, and may be a controller that the user holds and operates in their hand, or a video camera that captures the user's movements.

[0024] Furthermore, in the virtual space V, the avatar Ab can be moved in conjunction with the user's movements in the real space R, for example, the avatar Ab can be made to perform the same movements as the user's hand movements in the real space R. Such movements include holding an object (strictly speaking, an object in the virtual space V) and operating a button, switch, or the like provided on the object.

[0025] Furthermore, the range of the virtual space V that is displayed on the HMD 30, i.e., the range that the user can see through the HMD 30, is set according to the line of sight of the avatar Ab, and the line of sight of the avatar Ab can be changed according to the direction of the user's face. This allows the user to view any range in the virtual space V through the HMD 30.

[0026] Furthermore, in the first embodiment, multiple users can simultaneously use the same virtual space V. That is, in the first embodiment, multiple users can simultaneously view images of the virtual space V by specifying the same virtual space V. Furthermore, each of the multiple users can move a corresponding avatar Ab (i.e., each user's avatar Ab) within the same virtual space V. This allows the multiple users to interact with each other through their respective avatars Ab within the virtual space V. Specifically, each user can move their own avatar Ab to interact with the avatars Ab of other users.

[0027] Here, one of the multiple users is referred to as the first user, and the remaining users are referred to as the second users. Hereinafter, the avatar Ab of the first user will be referred to as the first avatar, and the avatar Ab of the second user will be referred to as the second avatar. The first user can move the first avatar and, for example, give an object (strictly speaking, an object) to the second avatar in the virtual space V. The second user can use the received object in the virtual space V, specifically, can give the object to the second avatar, and can also view the area in the virtual space V where the object is located using the HMD 30.

[0028] As described above, the first user and the second user can reproduce communication that is close to reality in the virtual space V by moving their respective avatars Ab.

[0029] Furthermore, in the first embodiment, an imaging device object that resembles a device that actually exists in the real space R, specifically an imaging device such as a camera, can be rendered in the virtual space V. To explain in more detail, for example, as shown in FIG. 2A , a user recognizes a camera that exists around the user in the real space R, and obtains image information of the camera as information about the camera (strictly speaking, imaging device information, which will be described later). The image information of the camera can be obtained, for example, by a CCD (Charged-Coupled Device) camera, which is the image sensor 32 built into the HMD 30.

[0030] If the model of the camera identified from the acquired image information has been registered in advance, an imaging device object representing a camera of the same model as the camera from which the image information was acquired is rendered in the virtual space V used by the user, as shown in Figure 2B. The imaging device object is one of the objects that can be displayed in the virtual space V.

[0031] That is, when information (more specifically, second information) about a real camera captured by the image sensor 32 is registered in the database 12 described below, an imaging device object corresponding to the real camera is rendered in the virtual space V. The virtual space V in which the imaging device object is rendered is displayed on the HMD 30, and the user can visually recognize the imaging device object in the virtual space V, as shown in FIG. 2B .

[0032] Furthermore, the user can move the imaging device object through the avatar Ab as if it were a real imaging device, and specifically, as shown in Figures 3A and 3B, can capture images using the imaging device object in the virtual space V. With this effect, the virtual space V and the real space R can be linked so that the virtual space V becomes closer to the real space R. As a result, the virtual space V can be provided to the user with a virtual space V that gives a sense of familiarity.

[0033] The following describes an example of a service (hereinafter referred to as a space linking service) provided to a user by linking the virtual space V and the real space R. Note that, hereinafter, an imaging device that actually exists in the real space R will be referred to as a real camera Gc, and an imaging device object rendered in the virtual space V will be referred to as a virtual camera Oc.

[0034] 2B and 3B, the virtual camera Oc has the same appearance as the corresponding real camera Gc, and the virtual camera Oc is provided with operation parts such as switches and buttons in the same arrangement as those on the real camera Gc. Attributes of the corresponding real camera Gc are set for the virtual camera Oc. The attributes of the real camera Gc are attributes of the imaging device, and specifically include the type, model and model number, format, specifications, structure, manufacturer (manufacturer) of the real camera Gc, and other attributes related to the functions of the real camera Gc. Examples of types of real cameras Gc include digital cameras, instant cameras, and film cameras.

[0035] Furthermore, the virtual camera Oc is associated with a function of the real camera Gc that corresponds to the attribute set for the virtual camera Oc. The association of the function of the real camera Gc with the virtual camera Oc means that information necessary for the virtual camera O to reproduce a scene in which the function of the real camera Gc is exercised, specifically, information related to the visual effects corresponding to the function (the visual effects will be described later), is stored in association with the virtual camera Oc.

[0036] Here, the real camera Gc corresponding to an attribute refers to a real camera Gc that belongs to a type or model, which is one of the attributes. Furthermore, the functions of the real camera Gc are functions related to image capture and image output. The image capture functions include a function to capture an image and a function to adjust the image capture conditions. The image capture condition adjustment functions include, for example, a zoom function, an autofocus function, and a telephoto function. The image output functions include, for example, a function to generate a print of the captured image if the real camera Gc is an instant camera.

[0037] The user can then instruct the virtual camera Oc to perform an action corresponding to a function associated with the virtual camera Oc. Specifically, as shown in Figures 3A and 3B, the user moves the avatar Ab to cause the avatar Ab to perform an action corresponding to a function to be performed by the virtual camera Oc, and more specifically, causes the avatar Ab to press a button provided on the virtual camera Oc that corresponds to the function. This allows the user to instruct the virtual camera Oc to perform an action corresponding to the function.

[0038] When the avatar Ab performs an action according to a function with respect to the virtual camera Oc, a visual representation based on the function is performed in the virtual space V. The visual representation based on the function is a representation in which a scene in which the function is performed with the real camera Gc is reproduced with the virtual camera Oc, and specifically, an image of the same scene as that scene is displayed (played back) in the virtual space V.

[0039] In order to explain the visual presentation in more detail, we will use as an example a case in which a real camera Gc having the function of printing captured images exists in real space R, and a virtual camera Oc that imitates this real camera Gc is drawn in virtual space V.

[0040] In this case, the real camera Gc is an instant camera with a print-producing function. That is, the real camera Gc accommodates one or more sheets of photosensitive film in the camera body, and when the capture button is pressed, it captures an image within the field of view and prints the captured image on the photosensitive film. The photosensitive film on which the captured image is printed is then output as a print, specifically, ejected from the camera body (see FIG. 4B ).

[0041] On the other hand, as shown in Fig. 4A , a user using the virtual space V can move an avatar Ab and have the avatar Ab hold a virtual camera Oc that imitates a real camera Gc, which is an instant camera. The user can also move the avatar Ab to instruct the virtual camera Oc to perform an operation corresponding to the print production function. In other words, as shown in Fig. 4A , the avatar Ab performs an operation corresponding to the print production function on the virtual camera Oc based on the user's instruction, such as pressing a portion of the virtual camera O that corresponds to the shooting button.

[0042] When the above operation is performed, this triggers a visual effect based on the print generation function in the virtual space V. Specifically, as shown in Fig. 4A, a part of the virtual space V is captured by the virtual camera Oc, and an image of an object resembling a print made by printing the captured image is output from the virtual camera Oc and displayed (played) in the virtual space V. Hereinafter, the print object will be referred to as the print object Po.

[0043] Here, the image captured by the virtual camera Oc is an extracted image obtained by extracting a range corresponding to the orientation of the virtual camera Oc from the virtual space V. In other words, capturing an image by the virtual camera Oc means extracting (capturing) a range corresponding to the orientation of the virtual camera Oc from the image of the virtual space V, specifically, a range corresponding to the angle of view of the virtual camera Oc, and acquiring the extracted image.

[0044] 4A , by pointing a virtual camera Oc at an avatar Ab to capture an image, it is possible to obtain a captured image (strictly speaking, an extracted image) with the avatar Ab as the subject. In this case, if multiple avatars Ab exist in the virtual space V and an extracted image is captured with two or more avatars Ab within the angle of view of the virtual camera Oc, a print object Po with images of the two or more avatars Ab printed on it is displayed in the virtual space V. This makes it possible to recreate, in the virtual space V, a scene in which a group photo of multiple users' avatars Ab is taken and a printout of the photo is output.

[0045] Furthermore, the user can move the avatar Ab holding the printed object Po and carry the printed object Po within the virtual space V. Furthermore, the user can move the avatar Ab within the virtual space V and place the printed object Po near another user's avatar Ab, thereby showing it to the other user. Furthermore, as shown in FIG. 5 , the user can hand over the printed object Po to another user's avatar Ab through the user's avatar Ab within the virtual space V. In this way, the user can transfer the printed object Po to another user within the virtual space V.

[0046] Furthermore, when the above-described visual effect is implemented in the virtual space V, i.e., when the print object Po is output from the virtual camera Oc, a print P is also generated in the real space R in conjunction with this and output from the real camera Gc, as shown in FIG. 4B . More specifically, in the real camera Gc, which is an instant camera, an image based on the print image (strictly speaking, an extracted image) of the print object Po in the virtual space V is actually printed on a photosensitive film in the real camera Gc. The image based on the print image of the print object Po may be the same image as the print image, or, as shown in FIG. 4B , may be a composite image in which another image is superimposed on the print image. Examples of other images that can be superimposed on the print image include decorative images representing patterns or designs, images of characters, stamp images for messages, and images that add effects to the background image in the print image.

[0047] 4B , a print P on which the image is printed is output from the real camera Gc. This allows the user to acquire, in the real space R, a print P corresponding to the print object Po in the virtual space V. As a result, the virtual space V and the real space R can be linked more effectively, and a virtual space V that the user feels more familiar with can be provided.

[0048] In the real space R, when a print P is generated and output by the real camera Gc, the photosensitive film in the real camera Gc is consumed, and the remaining quantity of photosensitive film is reduced by the consumed amount. Accordingly, in the virtual space V, the remaining number of times that a print object Po can be generated by the virtual camera Oc, in other words, the remaining number of times that a visual effect related to the generation of the print object Po can be performed, is reduced. In the real space R, the remaining quantity of photosensitive film can be increased by loading new photosensitive film into the real camera Gc. In this case, the remaining number of times that a visual effect related to the generation of the print object Po can be performed is also increased in the virtual space V.

[0049] <<Configuration Example of Virtual Space Display System>> An information processing system (hereinafter, virtual space display system 100) including a virtual space display device according to the first embodiment will be described with reference to Fig. 6. The virtual space display system 100 is a system constructed to provide users with a virtual space V and a spatial collaboration service that can be experienced within the virtual space V. As shown in Fig. 6, the virtual space display system 100 is composed of a virtual space display device 10 and a user unit 20. Note that while Fig. 6 shows two user units 20 constituting the system, this is not limited to this, and in reality, the number of user units 20 present corresponds to the number of users who use the virtual space V and the spatial collaboration service.

[0050] The user unit 20 is made up of equipment used by the user, and includes a user terminal 22, an HMD 30, an image sensor 32, and a wearable device 40, as shown in FIG.

[0051] The HMD 30 and the wearable device 40 are worn by the user when the user uses the virtual space V and the spatial collaboration service. While the user is using the virtual space V, the HMD 30 displays the virtual space V, and the wearable device 40 detects the user's movements. The image sensor 32 is mounted, for example, in the HMD 30, and while the user is using the virtual space V, captures an image of a range of the real space R that corresponds to the position and orientation of the user's face, and acquires the captured image.

[0052] The user terminal 22 is a real terminal that the user operates when using the virtual space V and the spatial collaboration service. In other words, the user terminal 22 accepts a series of input operations that the user performs in the real space R to use the virtual space V and the spatial collaboration service. The input operations accepted by the user terminal 22 include operations performed directly on the user terminal 22 and operations performed via the wearable device 40 or the like.

[0053] The user terminal 22 may be a personal computer, smartphone, mobile phone, tablet device, wearable device, or the like. The user terminal 22 may also be a computer housed within the casing of the HMD 30 and integrated with the HMD 30. As shown in FIG. 6 , the user terminal 22 includes a processor 22a, a memory 22b, and a communication interface 22c. The configuration of these devices is similar to the processor 10a, memory 10b, and communication interface 10c of the virtual space display device 10. As shown in FIG. 6 , the user terminal 22 further includes an input device 22d including a touch panel, a mouse, a keyboard, and the like, and an output device 22e including a display, a speaker, and the like. The input device 22d may include a microphone as a sound collection device and a camera as an image reading device.

[0054] In addition, a program for an OS (Operating System) and a predetermined application program (hereinafter, an app) are installed on the user terminal 22. By launching the app on the user terminal 22, the user can use the virtual space V and the space linkage service using the user unit 20. More specifically, by launching the app, the user logs in to the virtual space V, and the user's avatar Ab appears in the virtual space V.

[0055] Furthermore, the user terminal 22 is capable of communicating with the HMD 30, the image sensor 32, and the wearable device 40, and transmits and receives data to and from these devices. Specifically, the user terminal 22 transmits display data of the virtual space V to the HMD 30, and when the HMD 30 receives the display data, the virtual space V is displayed on the HMD 30 based on the display data. Furthermore, image data (image data) captured by the image sensor 32 is transmitted from the image sensor 32 to the user terminal 22. Furthermore, data (motion data) related to the user's movements detected by the wearable device 40 is transmitted from the wearable device 40 to the user terminal 22.

[0056] Furthermore, the user terminal 22 can communicate with the real camera Gc and can control the real camera Gc to perform the functions of the real camera Gc. Specifically, when the user terminal 22 receives a control instruction from the virtual space display device 10, it controls the real camera Gc based on the control instruction.

[0057] The communication method between the HMD 30, the image sensor 32, the wearable device 40, and the real camera Gc and the user terminal 22 may be a wired communication method or a wireless communication method. As the wireless communication method, a method using Bluetooth (registered trademark), a method using Wi-Fi (registered trademark), an infrared communication method, and other wireless communication methods can be used.

[0058] The virtual space display device 10 is composed of a computer, and specifically, may be configured as a personal computer, a workstation, a server, or the like. When the virtual space display device 10 is configured as a server, the server may be a server for a cloud service. Servers for cloud services include servers for ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). In this case, when necessary information is input into a client terminal, the server performs various processes and calculations based on the input information, and the calculation results are output on the client terminal side. In other words, the functions provided by the server can be used on the client terminal side. The virtual space display device 10 may be configured as a single computer or as multiple computers distributed in parallel.

[0059] The configuration of the virtual space display device 10 will be described. The computer that constitutes the virtual space display device 10 has, as hardware devices, a processor 10a, a memory 10b, a communication interface 10c, and a storage 10d, as shown in FIG.

[0060] The processor 10a may be configured with a central processing unit (CPU), a micro-processing unit (MPU), a microcontroller unit (MCU), a graphics processing unit (GPU), a digital signal processor (DSP), a tensor processing unit (TPU), an application specific integrated circuit (ASIC), etc. The memory 10b may be configured with semiconductor memory such as a read only memory (ROM) and a random access memory (RAM).

[0061] The communication interface 10c may be configured, for example, by a network interface card, a communication interface board, etc. The standard of data communication by the communication interface 10c is not particularly limited, and examples include communication by a wireless LAN based on Wi-fi (registered trademark), communication by a mobile communication system of 3G to 5G or later generations, or communication based on LTE (Long Term Evolution).

[0062] The virtual space display device 10 is capable of communicating information with each user's user terminal 22 via the communication interface 10c. Specifically, the virtual space display device 10 transmits information (hereinafter, virtual space information) for displaying the virtual space V on each user's HMD 30 to the user terminal 22. The user terminal 22 generates display data for the virtual space V based on the virtual space information received from the virtual space display device 10, and the virtual space V based on the generated display data is thereby displayed on the HMD 30.

[0063] Furthermore, through communication with the user terminal 22, the virtual space display device 10 can acquire information on images captured by the image sensor 32 and information on the user's movements detected by the wearable device 40. Furthermore, the virtual space display device 10 edits the virtual space information based on the acquired information and transmits the edited virtual space information to the user terminal 22. The user terminal 22 then generates display data for the virtual space V based on the edited virtual space information. As a result, the HMD 30 displays the virtual space V in which the avatar Ab that moves in conjunction with the user's movements and objects such as the imaging device object are drawn.

[0064] Furthermore, when the image sensor 32 captures an image of the real camera Gc in the real space R, the virtual space display device 10 analyzes the captured image of the real camera Gc and identifies feature quantities related to the captured image. This allows the virtual space display device 10 to acquire information (hereinafter, "first information") about the real camera Gc, including feature quantities of the captured image. The feature quantities of the image are quantified features of the image (e.g., the type, position, size, captured scene, and image quality of the subject in the image), and more specifically, the features are expressed by numerical values, vectors, tensors, or the like. Note that known image analysis techniques, such as techniques used for subject recognition, can be used as techniques for identifying the feature quantities of the image.

[0065] The first information is an attribute of the real camera Gc, more specifically, information specific to the model, and is used as matching information in the selection process described below. In the first embodiment, the first information includes information related to the image, and more specifically, as described above, includes feature amounts of the captured image obtained by the image sensor 32 capturing an image of the real camera Gc. Here, the captured image of the real camera Gc corresponds to the "imaging device information" of the present invention, and is information that can be acquired in the real space R in which the real camera Gc is placed. Note that the first information is not limited to information including feature amounts of the captured image, and may include the captured image itself, more specifically, data on the gradation values ​​of each of the multiple pixels that make up the captured image.

[0066] Furthermore, the virtual space display device 10 can communicate with the user terminal 22 to control the real camera Gc via the user terminal 22 and enable the real camera Gc to perform its functions. Specifically, the virtual space display device 10 generates control instruction data and transmits it to the user terminal 22. Upon receiving the control instruction data, the user terminal 22 identifies the content of the control instruction from the data and controls the real camera Gc based on the identified content of the control instruction. As a result, the virtual space display device 10 can, for example, control the real camera Gc to save a predetermined image in an area that can be stored by the real camera Gc, specifically, in the memory within the real camera Gc or in a recording medium such as an SD memory card. Furthermore, if the real camera Gc is an instant camera that has the function of generating a print P, the virtual space display device 10 can control the real camera Gc to generate a print P on which a predetermined image is printed.

[0067] The storage 10d may be configured with a flash memory, a hard disc drive (HDD), a solid state drive (SSD), a flexible disc (FD), a magneto-optical disc (MO disc), a compact disc (CD), a digital versatile disc (DVD), a secure digital card (SD card), a universal serial bus memory (USB memory), or the like. The storage 10d may be built into the computer constituting the virtual space display device 10, may be attached to the computer main body in an external format, or may be a network attached storage (NAS), or the like. The storage 10d may also be configured with a third computer, such as a database server or online storage, communicably connected to the virtual space display device 10.

[0068] The storage 10d stores and accumulates various types of information required to provide the spatial collaboration service, thereby constructing the database 12 shown in Fig. 6. The database 12 stores object information about a virtual camera Oc, which is an imaging device object, and second information about a real camera Gc corresponding to the object information, in association with each other.

[0069] More specifically, the database 12 stores, for each virtual camera, object information for each of multiple types of virtual cameras Oc that can be displayed in the virtual space V. As shown in FIG. 7 , each piece of object information includes an identification ID (object ID) of the virtual camera Oc, an appearance image of the virtual camera Oc, attributes, and functions associated with the virtual camera Oc. The attributes in the object information are attributes of the real camera Gc corresponding to the object information of the virtual camera Oc, specifically, the type and model of the real camera Gc. The functions associated with the virtual camera Oc are functions possessed by the real camera Gc corresponding to the object information of the virtual camera Oc, in other words, functions of the real camera Gc that correspond to the above attributes. Here, the real camera Gc corresponding to the object information of the virtual camera Oc is a real camera Gc that has attributes included in the object information of the virtual camera Oc, for example, a real camera Gc of a model that has the same appearance as the virtual camera Oc.

[0070] Furthermore, the number and types of functions associated with a virtual camera Oc are determined according to the model of the real camera Gc corresponding to the object information of the virtual camera Oc. That is, a virtual camera O that simulates a real camera Gc having functions f1, f2, ..., fn (n is a natural number) is associated with functions f1, f2, ..., fn. To explain using a specific example, when a real camera Gc that is a digital camera corresponds to the object information of the virtual camera Oc, the virtual camera O is associated with functions such as an image capture function, a function for displaying captured images, and a function for saving captured images. Furthermore, when a real camera Gc that is an instant camera corresponds to the object information of the virtual camera Oc, the virtual camera O is associated with an image capture function and a function for generating prints of the captured images.

[0071] 7, the object information further includes the content of the visual effect based on the function associated with the virtual camera Oc, the content of the visual effect, and the conditions for starting the visual effect. The content of the visual effect is the specific content of the scene reproduced in the virtual space V, and is video information of the scene. The conditions for starting the visual effect are information indicating the content of the user's action that is detected in the real space R and triggers the start of the visual effect. Note that the object information may further include information other than the above.

[0072] The second information is information about the real camera (real camera Gc) corresponding to each piece of object information stored in the database 12, and is registered together with the object information in the database 12. Therefore, the same number of pieces of second information as the number of pieces of object information are stored in the database 12. The second information is also an attribute of the real camera Gc corresponding to the object information, more specifically, information specific to the model, and is used as matching information in the selection process described below.

[0073] Each piece of second information includes the feature amount of the image captured by the real camera Gc, the attributes of the real camera Gc, and the object ID (the identification ID of the virtual camera Oc) included in the object information corresponding to the real camera Gc, as shown in Fig. 8. Since the second information includes the object ID in this way, the second information and the object information are associated with each other via the object ID.

[0074] The image features included in the second information are information about the image of the real camera Gc and are identified by analyzing an image showing the appearance of the real camera Gc. As with the first information, known image analysis techniques, such as techniques used for subject recognition, can be used to identify the image features. The attributes included in the second information are information about the attributes of the real camera Gc corresponding to the object information, specifically, information about the type and model of the real camera Gc. The attribute information can be obtained based on a catalog or instruction manual for the real camera Gc, or information provided by the manufacturer of the real camera Gc. The second information may also include information other than the above. Furthermore, the information about the image of the real camera Gc included in the second information is not limited to the image features, but may also be information about the image itself, more specifically, the gradation values ​​of each of the multiple pixels that make up the image.

[0075] The number of pieces of object information and second information stored in the database 12 increases as new object information and second information are stored. For example, when a new camera is developed in the real space R and distributed on the market, second information is generated for that camera, and object information is generated for a virtual camera Oc that imitates that camera. The generated object information and second information are then associated with each other and newly stored in the database 12.

[0076] Returning to the explanation of the configuration of the virtual space display device 10, a program for an operating system (OS) and an information processing program for displaying a virtual space are installed on the computer that constitutes the virtual space display device 10. These programs correspond to the "programs" of the present invention and are programs that control the computer that constitutes the virtual space display device 10. In other words, the above-mentioned programs are executed by the above-mentioned processor 10a, causing the above-mentioned computer to function as the virtual space display device 10. Specifically, the processor 10a executes a series of processes described below in accordance with the above-mentioned programs in order to provide the virtual space V and the space linkage service.

[0077] The information processing program for displaying the virtual space described above may be obtained by reading it from a computer-readable recording medium, or may be obtained (downloaded) via a network such as the Internet or an intranet.

[0078] <<Example of Operation of Virtual Space Display Device According to First Embodiment>> In the virtual space display device 10 according to the first embodiment, for the purpose of providing a virtual space V and a space linkage service to a user, the processor 10a of the computer constituting the virtual space display device 10 executes a series of processes shown in Fig. 9. Hereinafter, the flow of processing executed by the processor 10a will be referred to as a virtual space display flow, and the virtual space display flow will be described with reference to Fig. 9.

[0079] The virtual space display flow uses the virtual space display method of the present invention. In other words, each step (process) in the virtual space display flow shown in Figure 9 corresponds to a component of the virtual space display method of the present invention. However, the flow shown in Figure 9 is merely an example, and some steps in the flow may be deleted, new steps may be added, or the order of execution of two steps in the flow may be reversed, within the scope of the spirit of the present invention.

[0080] Before the virtual space display flow is executed, the database 12 is constructed in advance. That is, when the virtual space display flow is executed, the database 12 stores object information of the virtual camera Oc and second information of the real camera Gc corresponding to the object information in a mutually associated state.

[0081] In each step of the virtual space display flow, the processor 10a mainly reads an information processing program for virtual space display and executes data processing corresponding to each step. In addition, in the processing of each step, the processor 10a appropriately communicates with the user terminal 22 of the user (hereinafter also referred to as the user) who intends to use the virtual space V, and transmits information necessary for using the virtual space V to the user terminal 22 and receives information from the user terminal 22.

[0082] Specifically, when the user starts the app on the user terminal 22, the processor 10a starts the display process (S001) as a trigger. In the display process, the processor 10a communicates with the user terminal 22 of the user and displays a virtual space V on the HMD 30 worn by the user. The user's avatar Ad is depicted in the virtual space V.

[0083] Furthermore, if the user moves while the virtual space V is displayed on the HMD 30, the movement is detected by the wearable device 40 worn by the user. In this case, the processor 10a acquires information about the detected movement of the user through communication with the user terminal 22, and controls the display of the virtual space V so that the avatar Ab moves in conjunction with the movement.

[0084] Furthermore, while the virtual space V is displayed on the HMD 30, a real camera Gc positioned around the user in the real space R captures an image using the image sensor 32 (S002). In this case, the captured image is transmitted as imaging device information from the user terminal 22 to the virtual space display device 10. Upon receiving the captured image (S003), the processor 10a executes an acquisition process (S004). The acquisition process is executed based on the image captured by the real camera Gc. Specifically, in the acquisition process, the captured image is analyzed to acquire first information including feature quantities of the captured image.

[0085] After acquiring the first information, the processor 10a executes a selection process (S005). In the selection process, second information corresponding to the first information acquired in the previous step S004 is selected from the database 12. That is, in the selection process, the first information is compared with each piece of second information stored in the database 12, and second information of a camera model that matches or is similar to the actual camera Gc captured by the image sensor 32 is searched for.

[0086] Specifically, in the selection process, the processor 10a calculates the degree of similarity between the feature amount of an image included in the first information and the feature amount of an image included in each piece of second information stored in the database 12. In this case, the degree of similarity of the feature amounts of the images can be calculated using known techniques such as a technique for calculating Mahalanobis distance, Euclidean distance, or cosine similarity as a measure of similarity. Furthermore, artificial intelligence (AI) can be used as a means for calculating the degree of similarity of the feature amounts of the images.

[0087] The processor 10a then selects, from the second information stored in the database 12, second information whose degree of similarity to the first information satisfies a selection condition. The selection condition is, for example, that the degree of similarity to the first information is the greatest. However, the selection condition is not limited to this, and the selection condition may be that the degree of similarity to the first information is in the top kth place (k is a natural number) from the greatest. Alternatively, the selection condition may be that the degree of similarity to the first information is equal to or greater than a set reference value.

[0088] After the selection process is performed, the processor 10a performs a drawing process (S006). In the drawing process, object information associated with the second information selected in the previous step S005 is identified from the object information stored in the database 12. Here, the second information selected in step S005 is selected according to the first information acquired in step S004, and therefore, the object information associated with the second information can be said to be object information according to the first information.

[0089] Then, in the drawing process, the processor 10a draws a virtual camera Oc based on the object information identified in the above manner in the virtual space V, and displays the virtual space V in which the virtual camera Oc is drawn on the user's HMD 30. This allows the user looking at the virtual space V displayed on the HMD 30 to recognize the virtual camera Oc, which imitates the real camera Gc placed near the user in the real space R, within the virtual space V. As a result, the virtual space V and the real space R can be linked, making it possible to provide a virtual space V that the user feels more familiar with.

[0090] A user viewing a virtual space V can have an avatar Ab hold a virtual camera Oc depicted in the virtual space V (see FIGS. 3B and 4A). The user can also instruct the virtual camera Oc, via the avatar Ab, to perform an operation according to a function associated with the virtual camera O. To explain this in detail, we will assume that the virtual camera Oc depicted in the virtual space V is associated with a function for generating a print P of a captured image, i.e., the function of an instant camera.

[0091] After the rendering process is executed, the avatar Ab moves in conjunction with the user's movements in the real space R, and performs an operation on the virtual camera Oc according to the function of generating a print, i.e., an operation for printing (S007). Specifically, the user causes the avatar Ab to press a part of the virtual camera Oc that corresponds to the shooting button. This triggers the processor 10a to execute a performance process (S008).

[0092] In the rendering process, the processor 10a performs a visual rendering based on the above functions within the virtual space V. Specifically, the visual rendering is performed in which a print object Po corresponding to a print is generated by the virtual camera Oc within the virtual space V. In this visual rendering, the print object Po output from the virtual camera Oc includes an extracted image obtained by extracting a range within the virtual space V that corresponds to the orientation of the virtual camera Oc at the time the above printing operation is performed. In other words, in the visual rendering, an image is displayed (played) within the virtual space V as if the virtual camera Oc were photographing a range corresponding to the angle of view of the virtual camera Oc, and the photographed image were printed to generate a print.

[0093] As described above, in the first embodiment, by implementing the above visual effects, it is possible to reproduce in the virtual space V, using the virtual camera Oc that imitates the real camera Gc, the state in which the functions of the real camera Gc are exercised. As a result, it is possible to more effectively link the virtual space V with the real space R, and it is possible to provide a virtual space V that the user feels even more familiar with.

[0094] A user viewing the virtual space V can have his / her avatar Ab hold the printed object Po drawn in the virtual space V and carry it to any location within the virtual space V. The user can also transfer the printed object Po to another user by moving his / her avatar Ab as if handing the printed object Po held in the hand of his / her avatar Ab to the other user's avatar Ab.

[0095] Furthermore, when the above-described visual effect has been implemented, the processor 10a further executes a control process (S009). In the control process, the processor 10a cooperates with the user terminal 22 in the real space R to control the real camera Gc so that the real camera Gc performs its function (specifically, the function of generating a print). Through this control process, an actual print P is generated by the real camera Gc in the real space R and output from the real camera Gc (see FIG. 4B ).

[0096] The print P output from the real camera Gc has printed thereon an image based on the print image of the print object Po drawn in the virtual space V (i.e., an extracted image of a portion of the virtual space V). This image may be the same as the print image of the print object Po, or may be a composite image in which another image is superimposed on the print image. When printing a composite image, the other image to be superimposed on the print image may be determined according to the print image; specifically, it may be determined according to the facial expression of the avatar Ab included in the print image, the atmosphere created by the print image, the scene of the print image, etc.

[0097] As described above, in the first embodiment, the print object P is output from the real camera Gc in the real space R in conjunction with the output of the print object Po from the virtual camera Oc in the virtual space V. This makes it possible to reproduce the visual effects performed in the virtual space V in the real space R. As a result, the virtual space V and the real space R can be linked together more effectively, making it possible to provide a virtual space V that the user feels even more familiar with.

[0098] Of the above-described series of steps, steps S007 to S009 are repeatedly performed until the user finishes using the virtual space V (S010). The virtual space display flow ends when the user finishes using the virtual space V. The virtual space display flow is also performed each time the user launches the app on the user terminal 22 and logs into the virtual space V as the user.

[0099] <<Other Embodiments>> Specific embodiments of the present invention have been described above, but the above-described embodiments are merely examples given to facilitate understanding of the present invention and are not intended to limit the present invention. That is, the present invention may be modified or improved from the embodiments described below without departing from the spirit of the present invention. Furthermore, the present invention includes equivalents thereof. Furthermore, embodiments of the present invention may include a combination of the above-described embodiments with one or more of the following modifications.

[0100] (Regarding Modifications of the First Information and the Second Information) In the first embodiment, the first information and the second information used as the matching information in the selection process are information related to the image, and more specifically, information including image feature quantities. However, this is not limited to this, and for example, it is also conceivable that the first information and the second information include information other than image feature quantities, and that this information is used as the matching information. This form is referred to as the second embodiment of the present invention, and the second embodiment will be described below.

[0101] In the second embodiment, in an acquisition process during the virtual space display flow, the processor 10a of the virtual space display device 10 acquires imaging device information for the real camera Gc that can be acquired in the real space R. Here, the imaging device information is information output from the user terminal 22 based on an input operation by the user performed on the real camera Gc. Specifically, the user performs an input operation to input the model, device name, serial number, etc. of the real camera Gc using the input device 22d of the user terminal 22. In this case, the text information input by the user is output from the user terminal 22 as the imaging device information.

[0102] When the processor 10a acquires the imaging device information (i.e., the output information from the user terminal 22), it acquires first information based on that information. The first information includes text information, specifically, text information input by the user. As described above, this text information indicates the model, device name, serial number, etc. of the real camera Gc, and is used as matching information in the selection process. That is, in the second embodiment, the first information includes the above text information as matching information based on the output information from the user terminal 22. This text information is information specific to the model of the real camera Gc.

[0103] Meanwhile, the database 12 stores object information of the virtual camera Oc and second information as matching information registered for the real camera (real camera Gc) corresponding to the object information. In the second embodiment, the second information is text information, specifically, information indicating the model, device name, serial number, etc. of the real camera Gc. This text information is information specific to the model of the real camera Gc (i.e., the real camera corresponding to the object information).

[0104] In the second embodiment, in the selection process, the processor 10a compares the acquired first information with each piece of second information stored in the database 12 and selects second information corresponding to the first information. Specifically, the processor 10a calculates the degree of similarity between text information included in the first information and the text included in each piece of second information, and selects second information whose degree of similarity to the first information satisfies the selection condition. Note that known techniques can also be used to calculate the degree of similarity of text information.

[0105] As described above, in the second embodiment, information other than image features is used as the first information and the second information. As a result, in the second embodiment, the user can use the spatial collaboration service by inputting the model, device name, serial number, or the like of the real camera Gc into the user terminal 22.

[0106] In the second embodiment, the user inputs the model, device name, serial number, etc. of the real camera Gc via the input device 22d of the user terminal 22. However, this is not limited to this. For example, an identification code, such as a two-dimensional barcode, written on the body of the real camera Gc may be read by a camera built into the user terminal 22. In this case, the scanned image of the identification code corresponds to the imaging device information. Then, the scanned image of the identification code may be analyzed to obtain first information including information on the identification code. Furthermore, the second information stored in the database 12 may include the identification code of a real camera (real camera) corresponding to the object information. This allows the first information to be compared with each piece of second information stored in the database 12, and second information having a matching or similar identification code to the first information to be selected as second information corresponding to the first information.

[0107] (Regarding Modified Examples of the Rendition Process and the Control Process) In the first embodiment, a function for generating a printout is associated with the virtual camera Oc. Furthermore, in the first embodiment, when the avatar Ab performs a printing action on the virtual camera Oc in the virtual space V, a visual effect is implemented in which a printout object Po is generated by the virtual camera Oc. Furthermore, in the first embodiment, in the real space R, the real camera Gc is controlled in conjunction with the implementation of the visual effect, and an actual printout P is generated by the real camera Gc. However, the rendition process and the control process are not limited to the above, and other examples are also possible. A case in which rendition process and control process different from those described above are performed is defined as a third embodiment of the present invention, and the third embodiment will be described below with reference to FIGS. 10 to 12 .

[0108] In describing the third embodiment, it is assumed that a real camera Gc, which is a digital camera, is placed near a user in real space R. In this case, object information regarding the model of the digital camera and second information regarding a real camera (i.e., real camera Gc) corresponding to the object information are associated with each other and stored in the database 12. As a result, when an image of the real camera Gc is captured by the image sensor 32 in real space R, the processor 10a of the computer constituting the virtual space display device 10 executes an acquisition process, a selection process, and a rendering process in the same manner as in the first embodiment. As a result, a virtual camera Oc simulating the real camera Gc is rendered in the virtual space V, as shown in FIG. 10 .

[0109] A function related to image capture by a real camera Gc, which is a digital camera, is associated with the virtual camera Oc depicted in the virtual space V. The function related to image capture (hereinafter referred to as the image capture function) includes a function to acquire a captured image and a function to adjust image capture conditions, and the function to adjust image capture conditions includes, for example, a zoom function, an autofocus function, and a telephoto function.

[0110] In the third embodiment, assume that the user moves the avatar Ab, and the avatar Ab performs an image capturing action on the virtual camera Oc, for example, pressing a portion of the virtual camera Oc corresponding to the camera button. The processor 10a then performs a visual effect based on the image capturing function in the virtual space V. In this visual effect, a video of a scene capturing an image within the field of view of the virtual camera Oc in the virtual space V is displayed in the virtual space V. Capturing an image within the field of view of the virtual camera Oc means extracting (capturing) a range of the virtual space V corresponding to the orientation of the virtual camera Oc and acquiring the extracted image. Therefore, for example, if the user performs an image capturing action while the virtual camera Oc is facing the avatar Ab, an image of the avatar Ab is captured in the visual effect. Hereinafter, the image captured by the virtual camera Oc, i.e., the extracted image of a portion of the virtual space V, will be referred to as the "captured image in the virtual space V."

[0111] In addition, in the above visual presentation, as shown in FIG. 11 , an image captured in the virtual space V is displayed on a portion of the virtual camera Oc that corresponds to the rear monitor (hereinafter referred to as the monitor portion Om). Strictly speaking, the image on the monitor portion Om in the virtual space V is replaced with the image captured in the virtual space V. This allows the user to check the image captured in the virtual space V by looking at the monitor portion Om of the virtual camera Oc in the virtual space V. Furthermore, if another user's avatar Ab (hereinafter referred to as the other avatar) is present in the virtual space V, the user may move their avatar Ab to bring the virtual camera Oc to the other avatar and point the monitor portion Om of the virtual camera Oc toward the other avatar. This allows the other user to see the image captured in the virtual space V.

[0112] In the third embodiment, in conjunction with the implementation of the above visual effects, the processor 10a executes control processing based on the captured image in the virtual space V. Specifically, the processor 10a generates data for an extracted image (captured image) that extracts a range from the virtual space V according to the orientation of the virtual camera Oc. The processor 10a then controls the real camera Gc in the real space R to store the captured image in the virtual space V in an area that can be stored by the real camera Gc. The area that can be stored by the real camera Gc is a storage area for the real camera Gc, which is a digital camera, to store image data, and specifically, is an internal memory mounted on the camera body and an SD card inserted into a slot in the camera body. The storage destination (storage destination) of the captured image in the virtual space V may be specified in advance or may be changed as appropriate based on a change operation by the user.

[0113] As described above, in the third embodiment, captured images in the virtual space V can be stored in a predetermined storage area in the real space R. This allows the captured images in the virtual space V to be used in the real space R. For example, if an SD card storing captured images in the virtual space V is removed from the real camera Gc and inserted into a slot of another terminal, the captured images in the virtual space V can be read from the SD card by that terminal and displayed on a display or the like. Alternatively, as shown in FIG. 12 , captured images in the virtual space V can be read from an internal memory built into the real camera Gc and displayed on the rear display Gm of the real camera Gc.

[0114] In the third embodiment, it is possible to store and use images captured in the virtual space V in the real space R, thereby effectively linking the real space R with the virtual space V. Furthermore, the user can view the images captured in the virtual space V even after finishing using the virtual space V.

[0115] (Regarding modified examples of the configuration of the virtual space display device) In the above embodiment, the virtual space display device 10 is configured by a computer, specifically, a computer different from the user terminal 22, but some of the functions of the virtual space display device 10 may be performed by the user terminal 22. In that case, the virtual space display device of the present invention is configured by cooperation between the computer and the user terminal 22.

[0116] (Regarding the processor configuration) The virtual space display device of the present invention includes at least one or more processors, which may include various types of processors. The various processors include, for example, a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units. Furthermore, each process in each embodiment of the present invention may be executed by any computer. Furthermore, any computer may execute these processes using a processor, a program, or a combination thereof. The any computer may be a system such as a general-purpose computer, a computer for a specific application, a workstation, or other hardware element capable of executing a program.

[0117] The processor may be configured with one or more pieces of hardware, and the type of hardware is not limited. For example, the processor may be configured with hardware such as a central processing unit (CPU), a micro processing unit (MPU), a programmable logic device such as a field programmable gate array (FPGA), a dedicated circuit for executing specific processing such as an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or a neural processing unit (NPU).

[0118] Furthermore, the processor has each unit or each means that executes various processes in the present invention. Furthermore, the type of hardware may be a combination of different types of hardware. When multiple pieces of hardware are configured to execute one or more processes of a certain processor, the multiple pieces of hardware may exist in devices that are physically separate from each other, or may exist in the same device. Furthermore, in any embodiment of the present invention, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware is configured by an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0119] Furthermore, each embodiment of the present invention may be realized by hardware, software, firmware, microcode, or a combination thereof. Software, firmware, and microcode are configured by a program. A program may also be, for example, a group of program modules, each function of which may be implemented by a processor configured to perform the respective function. The program may also be program code and multiple code segments stored in one or more non-transitory computer-readable media (e.g., storage media or other storages). The program may also be stored in multiple non-transitory computer-readable media that reside in physically separate devices. Program code or code segments may represent procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending or receiving information, data, arguments, parameters, or memory contents.

[0120] REFERENCE SIGNS LIST 10 Virtual space display device 10a Processor 10b Memory 10c Communication interface 10d Storage 12 Database 20 User unit 22 User terminal 22a Processor 22b Memory 22c Communication interface 22d Input device 22e Output device 30 HMD (display) 32 Image sensor 40 Wearable device 100 Virtual space display system Ab Avatar Gc Real camera (real imaging device) Gm Rear monitor Oc Virtual camera (imaging device object) Om Monitor portion P Printed material Po Printed material object R Real space V Virtual space

Claims

1. A virtual space display device having a processor that displays a virtual space on a display, wherein the processor executes the following processes: acquiring first information about an actual imaging device; selecting second information corresponding to the first information from a database in which object information about imaging device objects that can be displayed in the virtual space and second information about imaging devices corresponding to the object information are stored in association with each other; and displaying on the display the virtual space in which the imaging device objects based on the object information associated with the second information corresponding to the first information are drawn.

2. The virtual space display device according to claim 1, wherein the processor acquires the first information based on imaging device information acquired in a real space in which an actual imaging device is placed.

3. The virtual space display device according to claim 1, wherein the first information and the second information include information about an image.

4. A virtual space display device as described in claim 1, wherein the processor acquires the first information based on imaging device information acquired in a real space in which a real imaging device is placed, the imaging device information being an image of the real imaging device, the first information including features of the image of the real imaging device, and the second information including features of the image of the imaging device corresponding to the object information.

5. The virtual space display device according to claim 1, wherein the first information and the second information include text information.

6. A virtual space display device as described in claim 1, wherein the processor acquires the first information based on imaging device information acquired in a real space in which an actual imaging device is placed, the imaging device information being information output from a user terminal based on a user's input operation performed on the actual imaging device, the first information including matching information based on the output information from the user terminal, and the second information including matching information registered for the imaging device corresponding to the object information.

7. A virtual space display device as described in claim 1, wherein in the process of selecting the second information according to the first information, the processor selects from the database the second information whose degree of similarity with the first information satisfies a selection condition.

8. A virtual space display device as described in claim 1, wherein the second information includes information on attributes of an imaging device corresponding to the object information, the imaging device object is associated with a function of the imaging device corresponding to the attribute, and when an operation corresponding to the function is instructed to the imaging device object, the processor performs a visual presentation based on the function in the virtual space.

9. A virtual space display device as described in claim 8, wherein an avatar of the user that moves in accordance with the user's movements is depicted in the virtual space, and when the avatar performs an action in accordance with the function on the imaging device object, the processor performs a visual presentation based on the function in the virtual space.

10. A virtual space display device as described in claim 8, wherein the first information is information specific to the model of the real imaging device, and the second information is information specific to the model of the imaging device corresponding to the object information.

11. The virtual space display device according to claim 10, wherein the number and type of functions associated with the imaging device object are determined according to the object information of the imaging device object and the model of the corresponding imaging device.

12. A virtual space display device as described in claim 9, wherein the imaging device object is associated with the function of generating a print of a captured image, and when the avatar performs a printing action on the imaging device object, the processor performs the visual effect in which a print object corresponding to the print is generated by the imaging device object in the virtual space.

13. A virtual space display device as described in claim 12, wherein, when the avatar performs the printing action on the imaging device object, the processor performs the visual effect in which the print object including an extracted image extracted from a range of the virtual space corresponding to the imaging device object is generated by the imaging device object, and causes the real imaging device having the function of generating the print object to generate the print object on which an image based on the extracted image is printed.

14. A virtual space display device as described in claim 13, wherein the processor causes the real-world imaging device having the function of generating the print to output the print on which a composite image in which another image is superimposed on the extracted image in the print object is printed.

15. A virtual space display device as described in claim 9, wherein the imaging device object is associated with a function related to imaging by an imaging device corresponding to the object information, and when the avatar performs an imaging action on the imaging device object, the processor performs the visual presentation based on the imaging function in the virtual space.

16. A virtual space display device as described in claim 15, wherein, when the avatar performs an imaging action on the imaging device object, the processor obtains an extracted image of a range of the virtual space corresponding to the imaging device object, and stores the extracted image in an area in real space that can be stored by the real imaging device.

17. A virtual space display method for displaying a virtual space on a display, comprising: a step by a processor acquiring first information related to an actual imaging device; a step by the processor selecting second information corresponding to the first information from a database in which object information related to imaging device objects that can be displayed in the virtual space and second information related to the imaging devices corresponding to the object information are stored in association with each other; and a step by the processor displaying on the display the virtual space in which the imaging device objects based on the object information associated with the second information corresponding to the first information are drawn.

18. A virtual space display method as described in claim 17, wherein in the step of selecting the second information corresponding to the first information, the processor selects from the database the second information whose degree of similarity with the first information satisfies a selection condition.

19. A virtual space display method as described in claim 17, further comprising the step of: the second information includes information on attributes of an imaging device corresponding to the object information; the imaging device object is associated with a function of the imaging device corresponding to the attribute; and when an operation according to the function is instructed to the imaging device object, the processor performs a visual presentation based on the function in the virtual space.

20. A program for causing a computer to execute each step included in the virtual space display method according to any one of claims 17 to 19.

21. A computer-readable recording medium having recorded thereon a program for causing a computer to execute each step included in the virtual space display method according to any one of claims 17 to 19.

Images