Program, information processing apparatus, and information processing method

Abstract

To provide an information processing apparatus that can link information on a virtual picture and information on a virtual reality space to each other, while making use of the characteristics of the virtual reality space.SOLUTION: A program causes a computer to function as an information processing apparatus comprising: a photographic information acquisition unit (221) that acquires photographic information including picture information on a virtual picture photographed by a virtual camera used in a virtual reality space, and space specification information for specifying a virtual reality space where the virtual picture is photographed by using the virtual camera, of a plurality of virtual reality spaces different from each other; and a linking unit (225) that links the picture information and the space specification information to each other.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, an information processing device, and an information processing method. [Background technology]

[0002] Virtual reality technology is a technology that allows people to experience a virtual world constructed on a computer as if it were real. This virtual world is a world made up of various virtual objects (hereafter simply referred to as "objects") and is called a virtual reality space. One of the objects is a virtual camera that imitates a real camera. The virtual camera can be operated by, for example, an avatar in the virtual reality space. The avatar is an object that is used as an alter ego of a real person (hereinafter referred to as a "participant") when the real person participates in the virtual reality space. Conventionally, there is a technique for automatically linking information about a virtual photograph taken using this virtual camera with information about an object appearing in the virtual photograph (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6952065 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the characteristics of a virtual reality space is that the computer that provides the virtual reality space knows what state the participant is currently in. Due to this characteristic, when a virtual photograph is taken using a virtual camera, the computer that provides the virtual reality space can know what state the virtual reality space was in when the virtual photograph was taken. However, conventional techniques are limited to linking information related to virtual photographs with information related to objects, and have the problem of not taking advantage of the characteristics of virtual reality spaces as described above.

[0005] The present invention is devised to solve the above-mentioned problems, and has an object to take advantage of the characteristics of virtual reality space to effectively utilize information relating to virtual photographs and information relating to virtual reality space. [Means for solving the problem]

[0006] The program of the present invention causes one or more computers to function as an information processing device having a shooting information acquisition unit that acquires shooting information including photographic information regarding a virtual photograph taken by a virtual camera used in a virtual reality space and space identification information that identifies the virtual reality space in which the virtual photograph was taken using the virtual camera among a plurality of different virtual reality spaces, and a linking unit that links the photographic information with the space identification information. Effect of the Invention

[0007] According to the present invention, the characteristics of the virtual reality space can be utilized to effectively utilize information related to virtual photographs and information related to the virtual reality space. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration for implementing the functions of an information processing device according to a first embodiment. [Diagram 3] FIG. 13 is a conceptual diagram showing an example of virtual photography. [Figure 4] FIG. 11 is a screen diagram showing an example of a virtual photograph. [Diagram 5] FIG. 11 is a screen diagram showing an example of object designation in a virtual photograph. [Figure 6]11 is a screen diagram showing an example of a display of objects and object information in a virtual photograph. FIG. [Figure 7] 4 is a flowchart showing an information processing method according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiment 1 Fig. 1 is a block diagram showing a configuration of an information processing system 1 according to the first embodiment. In Fig. 1, the information processing system 1 is a system in which an information processing device 2, a service providing server 3, and a participating terminal 4 are connected via a network 5. An existing electric communication line can be used as the network 5. The network 5 is, for example, the Internet. The information processing device 2, the service providing server 3, and the participating terminal 4 are capable of communicating with each other via the network 5.

[0010] (Outline of information processing device) The information processing device 2 acquires shooting information including photographic information regarding a virtual photograph taken by a virtual camera used in a virtual reality space and space identification information that identifies the virtual reality space in which the virtual photograph was taken using the virtual camera among a plurality of different virtual reality spaces, and links the photographic information to the space identification information.

[0011] In the information processing system 1, some or all of the functions of the information processing device 2 may be provided by the service providing server 3 or by the participating terminal 4. In addition, in the information processing system 1, some or all of the functions of the information processing device 2 may be provided in a duplicated manner by the service providing server 3 and the participating terminal 4. In the following, unless otherwise specified, it is assumed that all of the functions of the information processing device 2 are provided in a single server having a physical configuration independent of the service providing server 3 and the participating terminal 4. However, all of the functions of the information processing device 2 may be realized by a single server or multiple servers. The information processing device 2 will be described in detail later.

[0012] (Service provider server) The service providing server 3 provides services related to the virtual reality space expressed by spatial data to users such as participants in the virtual reality space via the network 5. For example, a person who wishes to use the service can register as a user of the service by accessing the service providing server 3 from his / her own terminal and performing a predetermined procedure. The predetermined procedure includes, for example, registration of a name (real name or handle name) desired by the user as a name for identifying the user. The service providing server 3 assigns a unique ID (hereinafter referred to as "user ID") to the registered user and manages the user by linking the user ID with information about the user such as the user's name. The service is provided to the user through, for example, a service app installed on a terminal used by the user such as the participant terminal 4.

[0013] The service providing server 3 manages three-dimensional data of a virtual reality space (hereinafter referred to as "spatial data"). The service providing server 3 can manage multiple pieces of spatial data that are different from each other. The service providing server 3 manages the spatial data by assigning a unique ID as metadata to each piece of spatial data.

[0014] In addition, the service providing server 3 may assign, as metadata to each of the multiple spatial data, a unique ID indicating the space creator (hereinafter referred to as the "space creator ID"), a unique ID indicating the event organizer (hereinafter referred to as the "organizer ID"), or a unique ID indicating the space monitor (hereinafter referred to as the "monitor ID").

[0015] A space creator is a person who creates space data excluding three-dimensional data of avatars. The space creator can create space data for expressing a virtual reality space in a state where an avatar is not placed, using a terminal for space creation (not shown) by a known method. The terminal for space creation may be, for example, a smartphone, tablet terminal, or PC (Personal Computer) owned by the space creator. In the service provided by the service providing server 3, for example, any space creator having the above-mentioned user ID can operate the terminal for space creation to create space data and upload the created space data to the service providing server 3. At that time, the space creator can upload the virtual reality space represented by the space data created by the space creator by including an arbitrary and unique name given by the space creator, a thumbnail image of the virtual reality space, and an explanation of the content of the virtual reality space, etc., in the metadata of the space data. The name of the space creator may be, for example, a real name or a handle name. A thumbnail image is a two-dimensional still image of a virtual reality space viewed from a certain viewpoint, and is, for example, an image created by the space creator to well represent the characteristics of the virtual reality space created by the space creator. When spatial data is uploaded from a terminal used for creating space, the service providing server 3 assigns a unique ID to the spatial data, links it to the user ID of the space creator, and manages the spatial data and its metadata, such as its name.

[0016] An event organizer is a person who uses a virtual reality space represented by spatial data as a venue for a specific event. A person who wishes to hold various events using a virtual reality space represented by spatial data managed by the service providing server 3 can register an event using a virtual reality space by obtaining the above-mentioned user ID, for example, accessing the service providing server 3 from a terminal used by the person, and inputting information such as the type of event desired, a thumbnail image of the event, the name of the event, a description of the content of the event, and the spatial data desired to be used. The service providing server 3 manages the event by linking, for example, information specifying the registered event, the type of event, the thumbnail image of the event, the name of the event, an ID unique to the spatial data used for the event, and the user ID (hereinafter also referred to as "organizer ID") of the person who registered the event (event organizer). The information for identifying a registered event is, for example, an ID unique to the event (hereinafter referred to as "event ID"). When registering an event, the service providing server 3 assigns a unique event ID to the event. The event ID may be included in the metadata of the spatial data. Hereinafter, the organizer ID or event ID may also be referred to as "event information."

[0017] As long as permission is obtained from the creator of the virtual reality space, the event organizer can arbitrarily select in which virtual reality space and when to hold an event with a certain content. For example, the event organizer may hold an event with a certain content multiple times in one virtual reality space at different dates and times while keeping the content of the event substantially the same, or may hold the event in multiple different virtual reality spaces at the same time on the same day, or may hold the event in multiple different virtual reality spaces at different dates and times, or may hold the event in multiple different virtual reality spaces at dates and times where the time periods partially overlap. In such cases, even if an event has substantially the same content, each time the event is held in a different situation (different virtual reality space or different holding date and time), a unique event ID may be assigned to each event. Hereinafter, a group of multiple events that have substantially the same content but are held in different situations is referred to as a "same content related event group."

[0018] Additionally, event organizers may hold events with similar content, although they may not be said to have substantially the same content as a certain event, or a series of events whose content follows on from that of a certain event. Each of these events is also assigned a unique event ID. Hereinafter, a group of multiple events with similar content will be referred to as a "similar content-related event group," and a group of multiple events with the same content will be referred to as a "series content-related event group." In addition, event organizers may manage multiple events that were held at different times in a chronological order as a channel consisting of related events. Hereinafter, such a group of multiple events on one channel will be referred to as a "same-channel event group." In addition, a group of multiple events held by a single event organizer regardless of the content of the events is called a "same-organizer related event group," and a group of multiple events that are specifically designated as being related by a single event organizer is called a "designated related event group." Hereinafter, the same content related event group, similar content related event group, series of content related event group, same channel event group, same organizer related event group, and designated related event group will be collectively referred to as "related event group".

[0019] As described above, related event groups consisting of a plurality of events that have a predetermined specific relationship with each other (hereinafter simply referred to as a "specific relationship") are stored and managed in a storage unit (not shown) accessible from the service providing server 3 (hereinafter referred to as a "service providing server storage unit") for each related event group by, for example, linking the event IDs of the plurality of events included in each related event group to each other in the service providing server 3. A unique ID (hereinafter referred to as a "related event group ID") may be assigned to each of the plurality of related event groups by the service providing server 3. Hereinafter, for any one event included in a related event group, any other event included in that related event group, that is, an event that has a specific relationship with a certain event, is referred to as a "related event".

[0020] The type of relationship between events that defines them as related events having a specific relationship is determined in advance, for example, by the event organizer or a service provider using the service providing server 3. Information describing the specific relationship (hereinafter referred to as "specific relationship description information") is stored in a storage unit for the service providing server. For example, when an event is registered by an event organizer, the service providing server 3 refers to the specific relationship description information, and if a related event group in which the event should be included already exists, it performs processing to include the event in that related event group, and if a related event exists but a related event group does not yet exist, it may create a new related event group, assign a related event group ID, and store the created group in the service providing server memory unit. Note that a given event may be included in multiple related event groups.

[0021] The space monitor is a person who monitors whether the virtual reality space represented by the spatial data is being used appropriately by the participants. The space monitor is, for example, a person who belongs to an organization such as a company that provides a service using the service providing server 3 and is in charge of monitoring the use of the virtual reality space. The service providing server 3 manages the spatial data by linking the ID of the person in charge of monitoring (for example, an employee ID, etc.) with an ID unique to the spatial data.

[0022] Hereinafter, when the term "spatial data" is used, it is assumed that the metadata of the spatial data is also included.

[0023] A single piece of spatial data includes three-dimensional data for each of one or more objects positioned within the virtual reality space represented by the spatial data. The three-dimensional data of an object includes, for example, three-dimensional shape information, size information, color information, position information, and orientation information of the object. The position information is represented, for example, by coordinate values ​​of a global coordinate system set in a virtual reality space. The orientation information is represented, for example, by a rotation angle with respect to a coordinate axis of a global coordinate system set in a virtual reality space. Furthermore, when a participant participates as an avatar in a virtual reality space represented by certain spatial data, the object also includes the avatar. When the object is an avatar, the 3D data of the object includes not only each piece of information such as the 3D shape information described above, but also the posture information of the avatar. The posture information is represented, for example, by position information or orientation information of each of the multiple bones that make up the avatar.

[0024] In addition, the three-dimensional data of the object may include, as metadata, an ID unique to the object (hereinafter referred to as the "object ID"), the object name, an ID unique to the object's owner (hereinafter referred to as the "owner ID"), an ID unique to the object's creator (hereinafter referred to as the "object creator ID"), an ID unique to the object's seller (hereinafter referred to as the "seller ID"), or a unique ID of the object's favorites subscriber (hereinafter referred to as the "favorite subscriber ID").

[0025] An object owner is a person who manages or controls the three-dimensional data of an object. For example, a person who purchases three-dimensional data of an avatar from a seller of the three-dimensional data of the avatar is a person who manages or controls the three-dimensional data of the avatar, and is the owner of the avatar. A participant may participate in a virtual reality space using an avatar whose three-dimensional data he or she owns. Therefore, the service providing server 3 can use, for example, the user ID of a participant who is participating in a virtual reality space using an avatar as the owner ID of the avatar.

[0026] The creator of an object is a person who creates the three-dimensional data of the object. For example, objects include various structures in a virtual reality space, various parts that make up the structures, plants, avatars, and items that the avatars can use. Since the three-dimensional data of these objects is all created by someone, the creator of the three-dimensional data of each object can provide the three-dimensional data to others, such as the space creator, after giving the creator his / her own unique ID as metadata of the three-dimensional data of the object. Therefore, the metadata of the three-dimensional data of an object included in the space data managed by the service providing server 3 can include the object creator ID of the object. The service providing server 3 may manage, as an object creator ID, the user ID of a user who uploads an object to the service providing server 3 for the purpose of providing an object that can be used for a fee or free of charge in the service that the service providing server 3 provides. Note that the users who upload objects may also include object sellers.

[0027] An object seller is a person who sells three-dimensional data of an object. A seller of three-dimensional data of an object can sell the three-dimensional data to others, such as participants, after assigning his / her own unique ID as metadata of the three-dimensional data of the object. Therefore, the metadata of the three-dimensional data of an object included in the spatial data managed by the service providing server 3 can include the seller ID of the object. Note that sales of objects may be conducted via the service providing server 3. After acquiring the above-mentioned user ID, the seller of the object can sell the object to other users by, for example, accessing the service providing server 3 from a terminal used by the seller and registering the three-dimensional data of the object to be sold, the selling name or object name of the object, the selling price, etc.

[0028] A favorite registrant of an object is, for example, a person who has registered so-called "favorite" information by performing a specific operation on the three-dimensional data of an object for sale. A favorite registrant ID that identifies the person who has registered the favorite can be managed by a server (not shown) in association with the three-dimensional data or object ID of the object for sale. The service providing server 3 can access the server (not shown) as necessary for an object included in the spatial data it manages, and obtain the favorite registrant ID that is managed in association with the three-dimensional data or object ID of the object.

[0029] In the following description, it is assumed that both the seller and the registrant who can register favorites are users of the service provided by the service providing server 3 and have a user ID. In this case, the seller ID may be the same as the user ID, and the favorite registrant ID may also be the same as the user ID. In addition, hereinafter, when the term "three-dimensional data of an object" is used, the metadata of the three-dimensional data of the object is also included.

[0030] The service providing server 3 may, for example, copy the spatial data excluding the three-dimensional data of the avatar to create one or more new copies of the spatial data. Hereinafter, the newly created spatial data by copying is referred to as "duplicated spatial data." For example, when the number of participants who can participate as avatars in certain spatial data is limited, or when only certain participants wish to participate in certain spatial data, it is highly convenient for participants to use duplicate spatial data, which is a duplicate of the spatial data excluding the three-dimensional data of the avatar. The service providing server 3 also assigns a unique ID to the replicated spatial data and manages the replicated spatial data. The service providing server 3 manages the ID unique to the replicated spatial data by linking it to, for example, an ID unique to the original spatial data. Hereinafter, the ID unique to the spatial data and the ID unique to the replicated spatial data are also collectively referred to as the "spatial data ID." Hereinafter, the spatial data and replicated spatial data are also collectively referred to simply as the "spatial data."

[0031] The service providing server 3 provides participants with services using a virtual reality space represented by spatial data via a network 5. 1, the information processing system 1 has only one service providing server 3. However, the information processing system 1 may have one or more service providing servers 3. When the information processing system 1 has multiple service providing servers 3, the service providing servers 3 may, for example, manage different spatial data from each other, or when one service providing server 3 manages certain spatial data, the other service providing servers 3 may manage replicated spatial data that is a replicate of the spatial data. In the following, it is assumed that the information processing system 1 has one service providing server 3, similar to the one shown in FIG.

[0032] The service providing server 3 can access the service providing server storage unit. The service providing server storage unit stores space data for expressing one or more virtual reality spaces. The service providing server storage unit also stores a user ID, a user name registered by the user indicated by the user ID when using the service, information indicating a means of notification to the user indicated by the user ID (for example, an email address, or an in-app notification of a service app, etc.). The service providing server storage unit also stores three-dimensional data of an object, an object creator ID, sales information in the case where the object is for sale or has been sold, or a favorite registrant ID, etc., linked to information that can identify an object, such as an object ID. The sales information of an object may include, for example, a seller ID of the object, a price at which the object was sold or is being sold, a date on which the object was sold, a date on which the sale of the object started, a sale ID unique to the sale indicating that the object was sold and purchased in a specific sale, or a regular price sale ID indicating that the object was sold and purchased at a regular price.

[0033] (Participating devices) The participation terminal 4 is a terminal used when a participant participates in a virtual reality space. The participant uses the participation terminal 4 to access the service providing server 3 and downloads desired spatial data to the participation terminal 4. The participant uses the participation terminal 4 to participate as an avatar in the virtual reality space represented by the spatial data. The participant can also participate in the virtual reality space by simply viewing the virtual reality space without using an avatar. When a participant participates in the virtual reality space as an avatar, the participant can change the state of the avatar or other objects by operating the participant terminal 4, such as by moving the avatar within the virtual reality space, changing the avatar's posture, or moving the avatar to move or use other objects.

[0034] The participating terminal 4 is, for example, a smartphone, a tablet terminal, or a PC. The participating terminal 4 may also be a head-mounted display with a communication function that is used together with a controller. For example, a participant wearing a head-mounted display on their head and holding a controller in their hand can operate the virtual reality space displayed on the head-mounted display by moving their head or hands or by operating buttons on the controller. In other words, the participant terminal 4 may be any device that displays a virtual reality space and allows operations on the virtual reality space.

[0035] (Sharing spatial data) When a participant uses a participation terminal 4 to participate in a virtual reality space as an avatar and operates the participation terminal 4 to change the state of the avatar or other objects, the participation terminal 4 modifies the downloaded spatial data based on information indicating the operation (hereinafter referred to as "operation information"), and changes the state of the avatar or other objects in the virtual reality space. The participation terminal 4 also transmits the operation information to the service providing server 3. The service providing server 3 modifies, based on the operation information acquired from the participation terminal 4, the spatial data of the virtual reality space in which the above-mentioned avatar is participating, among the spatial data it manages, and changes the state of the avatar or other objects in the virtual reality space. In this way, the participating terminals 4 and the service providing server 3 can share the states of avatars and objects in a single virtual reality space in almost real time. In other words, the participating terminals 4 and the service providing server 3 can share the spatial data of a single virtual reality space in almost real time.

[0036] 1 shows only one participant terminal 4. However, the information processing system 1 normally has a plurality of participant terminals 4.

[0037] For example, if multiple participants are participating as avatars in one and the same virtual reality space from their own participating terminals 4, when the service providing server 3 acquires operation information from one participating terminal 4, it also transmits the operation information to all other participating terminals 4. When all other participating terminals 4 acquire the above-mentioned operation information from the service providing server 3, they modify the downloaded space data and change the states of the avatars or other objects in the virtual reality space. In this way, the states of avatars and objects in a single virtual reality space can be shared in almost real time among multiple participating terminals 4 and the service providing server 3. In other words, the spatial data of a single virtual reality space can be shared in almost real time among multiple participating terminals 4 and the service providing server 3.

[0038] (Virtual Camera) One of the objects is a virtual camera that imitates a real camera. A virtual camera is an object that is used in a virtual reality space to define a shooting range in the virtual reality space according to its position, orientation, etc., and to record (as a virtual photograph) the state of the virtual reality space within the shooting range seen from the position of the virtual camera at a specific moment. A virtual camera can be used to take a virtual photograph with an object existing in a virtual reality space as a subject, for example, typically by being held and operated by an avatar in a virtual reality space, in the same way as a person holds and operates a camera in a real space to take a photograph with an object existing in the real space as a subject. Also, a virtual camera can be used to take a group (virtual) photograph or a selfie virtual photograph of a plurality of avatars existing in a virtual reality space, for example, by being fixed in a predetermined position and operated by an avatar in a virtual reality space, in the same way as a person fixes a camera to a tripod, places it in a predetermined position, and operates it in a real space to take a group photograph or a so-called selfie photograph in the real space.

[0039] In other words, this virtual camera is different from a device that simply determines from what position and at what angle the virtual reality space is viewed in order to display the virtual reality space on the display of participant terminal 4, that is, a device that simply has the role of determining a viewpoint (which is also generally called a camera, hereinafter referred to as a "viewpoint camera").

[0040] Since the virtual camera is one of the objects, like other objects, its substance is three-dimensional data including three-dimensional shape information, size information, color information, position information, and orientation information. In addition to the above information, the three-dimensional data of the virtual camera also includes information specific to the virtual camera, such as the angle of view, as metadata.

[0041] (Virtual photo) A virtual photograph captures one or more objects in a virtual reality space as seen from the position and orientation of the virtual camera at the time the virtual camera is operated to take the photograph (hereinafter referred to as "time of photographing"). A virtual photograph is a two-dimensional image, and its substance is two-dimensional data having a certain number of pixels in each of the vertical and horizontal directions, just like a real digital photograph.

[0042] Also, although the substance of a virtual photograph is two-dimensional data, it can be treated as one of the objects. In other words, the two-dimensional data of a virtual photograph can be treated as three-dimensional data without thickness. The three-dimensional data of a virtual photograph includes three-dimensional shape information, size information, color information, position information, and orientation information, just like other objects. For example, an avatar in a virtual reality space can hold a virtual photo like any other object and change its state by, for example, moving it.

[0043] The virtual photograph is generated by at least one of the participating terminal 4 and the service providing server 3.

[0044] For example, when an operation for taking a virtual photograph is performed on the virtual camera, first, of one or more objects arranged in the virtual reality space, one or more objects that are included in the shooting range of the virtual camera at the time of shooting are identified (hereinafter referred to as "in-range object identification"). The shooting range of the virtual camera at the time of shooting is identified based on the position information, orientation information, and angle of view information of the virtual camera at the time of shooting. Hereinafter, information including at least the position information, orientation information, and angle of view information of the virtual camera at the time of shooting is also referred to as "camera information". The shooting range has a three-dimensional shape, for example, a pyramid shape with the virtual camera position as its apex, or a truncated pyramid shape excluding a portion up to a certain distance from the apex position of the pyramid. One or more objects contained within the shooting range of the virtual camera at the time of shooting can be identified from three-dimensional shape information indicating the shooting range and the three-dimensional shape information, size information, position information and orientation information at the time of shooting of one or more objects placed in the virtual reality space.

[0045] Once the within-range objects have been identified, it is then determined for each of the identified objects (hereinafter referred to as "within-range objects") whether or not part or all of it is invisible from the position of the virtual camera due to the presence of other objects that act as occlusions (hereinafter referred to as "occlusion determination"). For example, if two objects are placed in a virtual reality space, both of which are within the shooting range of a virtual camera, and one object is in front of the other object as viewed from the position of the virtual camera, one object will appear in the virtual photograph, but the part of the other object behind the first object will not appear in the virtual photograph. Depending on the relationship between the size and position of one object and the size and position of the other object, the whole of the other object may be behind the first object as viewed from the position of the virtual camera, and the other object may not appear at all in the virtual photograph. The occlusion determination is a process of determining whether or not such a state occurs for each object within the range.

[0046] Once the occlusion determination has been made, the color, etc. of each pixel of the virtual photograph is identified based on the result of the occlusion determination and color information, etc. of the objects within the range, and two-dimensional data of the virtual photograph can be generated.

[0047] As described above, a virtual photograph can be generated based on three-dimensional shape information, size information, and color information of one or more objects in the virtual reality space, position information, orientation information, and posture information of each object at the time of shooting, and position information, orientation information, and angle of view information of the virtual camera at the time of shooting. All of this information is included in the spatial data, and the spatial data is shared almost in real time between the participating terminal 4 and the service providing server 3. Therefore, when the participating terminal 4 transmits operation information, including operation information indicating a photographing operation described below, to the service providing server 3, both the participating terminal 4 and the service providing server 3 can generate a virtual photograph.

[0048] In the following description, it is assumed that at least the participating terminal 4 generates the virtual photo, and the below-described photo information and the like relating to the virtual photo are transmitted from the participating terminal 4 to the information processing device 2. However, as described above, the service providing server 3 can also generate the virtual photo. When the service providing server 3 generates the virtual photo, the photo information and the like may be transmitted from the service providing server 3 to the information processing device 2 instead of being transmitted from the participating terminal 4 to the information processing device 2. In addition, if both the participating terminal 4 and the service providing server 3 generate virtual photographs, it is also possible for the rules for generating the photo ID described below to be common rules between the participating terminal 4 and the service providing server 3.

[0049] A unique ID (hereinafter referred to as "photo ID") is assigned to the two-dimensional data of the virtual photo by the participating terminal 4 or the service providing server 3. The participating terminal 4 may link the two-dimensional data of the virtual photo it has generated with the photo ID and store it in a memory unit (hereinafter referred to as "memory unit for participating terminals") (not shown) that the participating terminal 4 has. In addition, if the service providing server 3 does not generate the virtual photo, the participating terminal 4 may link the two-dimensional data of the virtual photo it has generated with the photo ID and the user ID of the participant who used the participating terminal 4 to participate in the virtual reality space and take the virtual photo, and transmit them to the service providing server 3. When the service providing server 3 acquires this information from the participating terminal 4, it links the two-dimensional data of the virtual photo and the photo ID with the user ID already stored in the memory unit for service providing server, and stores them in the memory unit for service providing server.

[0050] When a participant uses the participation terminal 4 to participate in the virtual reality space as, for example, an avatar, and operates the participation terminal 4 to take a virtual photo using a virtual camera (hereinafter referred to as a "photographing operation"), the participation terminal 4 generates a virtual photo and assigns a photo ID to the generated virtual photo. In addition, the participation terminal 4 transmits camera information, photo information of the generated virtual photo, and information on objects within the range (hereinafter referred to as "object information") to the information processing device 2. The photo information includes at least a photo ID, and may include two-dimensional data of the generated virtual photo, and may include the time when the virtual photo was taken. The object information includes at least three-dimensional shape information, size information, object ID, and position information at the time of shooting of each object within the range. The object information may also include orientation information at the time of shooting of each object within the range, posture information at the time of shooting, or metadata other than the object ID. The metadata other than the object ID may include, for example, an owner ID, sales information regarding the sale of the object, etc.

[0051] (Details of information processing device) The following describes in detail the information processing device 2. As shown in FIG. The communication unit 21 communicates with the service providing server 3 and the participating terminal 4 via the network 5. For example, the communication unit 21 is a communication device capable of mobile communication using a communication method such as LTE, 3G, 4G, or 5G, and communicates with other devices such as the service providing server 3 and the participating terminal 4 connected to the network 5. The communication unit 21 may also include a short-range wireless communication means such as Bluetooth (registered trademark).

[0052] The calculation unit 22 controls the overall operation of the information processing device 2. The calculation unit 22 includes an imaging information acquisition unit 221, an object information acquisition unit 222, a determination unit 223, a determination result acquisition unit 224, a linking unit 225, a search key acquisition unit 226, a search result acquisition unit 227, a notification unit 228, and a display information generation unit 229. The calculation unit 22 executes an information processing application, whereby the calculation unit 22 realizes each of the functions of the imaging information acquisition unit 221, the object information acquisition unit 222, the determination unit 223, the determination result acquisition unit 224, the linking unit 225, the search key acquisition unit 226, the search result acquisition unit 227, the notification unit 228, and the display information generation unit 229.

[0053] The storage unit 23 stores, for example, an information processing application and information used in the arithmetic processing of the arithmetic unit 22. The storage unit 23 is a storage device provided in a computer functioning as the information processing device 2, and includes storage such as a hard disk drive (HDD) or a solid state drive (SSD), or the memory 103 of Fig. 2. Note that the storage unit 23 may be provided outside the information processing device 2 as long as it is accessible by the information processing device 2.

[0054] 2 is a block diagram showing a hardware configuration that realizes the functions of the information processing device 2. For example, the information processing device 2 has a communication interface 100, an input / output interface 101, a processor 102, and a memory 103 as hardware configurations. Each function of the shooting information acquisition unit 221, the object information acquisition unit 222, the determination unit 223, the determination result acquisition unit 224, the linking unit 225, the search key acquisition unit 226, the search result acquisition unit 227, the notification unit 228, the display information generation unit 229, the space selection screen display information acquisition unit 230, the event selection screen display information acquisition unit 231, the selection unit 232, the introduction screen display information acquisition unit 233, and the space data acquisition unit 234 included in the information processing device 2 is realized by executing an information processing application in these hardware configurations.

[0055] The communication interface 100 outputs data received from the service providing server 3, participating terminals 4, etc. via the network 5 to the processor 102, and transmits data generated by the processor 102 to the service providing server 3, participating terminals 4, etc. via the network 5. The processor 102 reads and writes data from the storage unit 23 in FIG. 1 via the input / output interface 101.

[0056] The programs constituting the information processing application for realizing each function of the information processing device 2, namely, the shooting information acquisition unit 221, the object information acquisition unit 222, the judgment unit 223, the judgment result acquisition unit 224, the linking unit 225, the search key acquisition unit 226, the search result acquisition unit 227, the notification unit 228, the display information generation unit 229, the space selection screen display information acquisition unit 230, the event selection screen display information acquisition unit 231, the selection unit 232, the introduction screen display information acquisition unit 233, and the space data acquisition unit 234, are stored in the memory unit 23. The processor 102 reads out a program stored in the storage unit 23 via the input / output interface 101, loads it into the memory 103, and executes the program loaded into the memory 103. In this way, the processor 102 realizes the functions of a shooting information acquisition unit 221, an object information acquisition unit 222, a determination unit 223, a determination result acquisition unit 224, a linking unit 225, a search key acquisition unit 226, a search result acquisition unit 227, a notification unit 228, a display information generation unit 229, a space selection screen display information acquisition unit 230, an event selection screen display information acquisition unit 231, a selection unit 232, an introduction screen display information acquisition unit 233, and a space data acquisition unit 234. The memory 103 is, for example, a RAM (Random Access Memory).

[0057] (Photography information acquisition section) The shooting information acquisition unit 221 acquires shooting information including photo information regarding a virtual photo taken by a virtual camera used in a virtual reality space, and space identification information that identifies the virtual reality space in which the virtual photo was taken using the virtual camera among a plurality of different virtual reality spaces. The shooting information acquisition unit may acquire, as the space specification information, a space data ID that is an ID unique to the three-dimensional data of the virtual reality space.

[0058] Furthermore, the photographing information acquisition unit 221 may acquire photographing information including camera information related to a virtual camera used in the virtual reality space, in addition to the photo information and the space specification information. In the following, it is assumed that the photography information acquisition unit 221 acquires photo information, space specification information, and camera information.

[0059] When a participant is participating in a virtual reality space using the participation terminal 4, for example as an avatar, and performs a shooting operation to take a virtual photograph using the virtual camera held by the avatar, the participation terminal 4 generates a virtual photograph and assigns a photo ID to the generated virtual photograph, and further transmits space-specific information (e.g., space data ID) of the virtual reality space in which the virtual photograph was taken, camera information, and photo information of the generated virtual photograph as shooting information to the shooting information acquisition unit 221 of the information processing device 2.

[0060] When the imaging information acquisition unit 221 acquires the imaging information from the participation terminal 4, it outputs the acquired imaging information to the determination unit 223 and the linking unit 225. In addition, the imaging information acquisition unit 221 causes the storage unit 23 to store the imaging information.

[0061] (Object information acquisition part) The object information acquisition unit 222 acquires object information regarding one or more objects (in-range objects) that are included within the shooting range of the virtual camera when the virtual photograph was taken, out of one or more objects placed in the virtual reality space. When generating a virtual photograph, the participating terminal 4 performs processing for identifying objects within the range to identify the objects within the range. The participating terminal 4 transmits the object information of the objects within the range to the object information acquisition unit 222 of the information processing device 2. The object information acquisition unit 222 outputs the acquired object information of the within-range objects to the determination unit 223. In addition, the object information acquisition unit 222 causes the storage unit 23 to store the acquired object information of the within-range objects.

[0062] Fig. 3 is a conceptual diagram showing an example of virtual photography. Fig. 3 shows four avatars A1 to A4, one tree object C1, two luggage objects C2 to C3, and a virtual camera VC, which are arranged in a virtual reality space. Note that in addition to the above objects shown in Fig. 3, one or more objects may be arranged in the virtual reality space, but these objects are not shown in the figure because they are arranged outside the shooting range.

[0063] In Fig. 3, a virtual camera VC is held by an avatar A1. Within the shooting range of the virtual camera VC (indicated by a dotted line in Fig. 3), there are three avatars A2 to A4, one tree object C1, and two luggage objects C2 to C3. In the state shown in FIG. 3, for example, when viewed from the position of the virtual camera, the lower body portion of avatar A2 is occluded by a luggage object C2, and most of avatar A4, including the upper body, is occluded by a tree object C1. In this state, when a participant participating in the virtual reality space using avatar A1 performs a shooting operation on participant terminal 4 to take a virtual photograph using virtual camera VC, participant terminal 4 generates a virtual photograph.

[0064] The participating terminal 4, for example, by the above-mentioned process of identifying within-range objects, identifies the three avatars A2 to A4, one tree object C1, and two luggage objects C2 to C3 that exist within the shooting range of the virtual camera VC as within-range objects. In addition, the participating terminal 4 determines, for example, by the above-mentioned occlusion determination process, that the lower body portion of avatar A2 is occluded by luggage object C2, and that most of avatar A4, including its upper body, is occluded by tree object C1, and therefore will not appear in the virtual photograph. As described above, the participating terminal 4 transmits object information of the objects within the range, that is, in the example shown in FIG. 3, object information of three avatars A2 to A4, one tree object C1, and two luggage objects C2 to C3, to the object information acquisition unit 222 of the information processing device 2.

[0065] Fig. 4 is a screen diagram showing an example of a virtual photograph. Fig. 4 shows a virtual photograph VP generated by a photographing operation performed in the state of Fig. 3. The virtual photograph VP shows three avatars A2 to A4, one tree object C1, and two luggage objects C2 to C3, but does not show most of the avatar A2, including the lower body and the upper body of the avatar A4. As described above, when the participating terminal 4 generates a virtual photo VP, it assigns a photo ID to the generated virtual photo VP, and further transmits the camera information and the photo information of the generated virtual photo VP as shooting information to the shooting information acquisition unit 221 of the information processing device 2.

[0066] (Judgment Department) The determination unit 223 acquires shooting information from the shooting information acquisition unit 221 and acquires object information of objects within the range from the object information acquisition unit 222, and then determines, based on the camera information and object information contained in the shooting information, whether or not each of the objects within the range is a subject object captured in the virtual photograph (hereinafter referred to as "subject object determination").

[0067] The conditions (hereinafter referred to as "judgment conditions") that determine whether or not the judgment unit 223 judges which within-range object is a subject object in the subject object judgment are set in advance and stored, for example, in the storage unit 23. The judgment conditions are decided, for example, by an organization or individual that provides a service using the service providing server 3, and are set in advance by an operation on the information processing device 2. The determination unit 223 determines whether or not each of the one or all of the multiple in-range objects is a subject object.

[0068] In subject object determination, the determination unit 223 uses the camera information and the object information of the within-range object to calculate the size or the orientation or posture of the within-range object to be determined in the virtual photograph, or the occlusion state such as the extent to which the within-range object is occluded or which part of the within-range object is occluded, or other shooting states, etc. The determination unit 223 can perform subject object determination by comparing the result of the calculation with the determination conditions.

[0069] The contents of the judgment conditions can be set arbitrarily. For example, the judgment condition may be that, for the range object being judged, if the area of ​​the part actually appearing in the virtual photograph is equal to or greater than a predetermined threshold value with respect to the area that would appear in the virtual photograph if the range object were not occluded at all, the range object is judged to be a subject object, and if the area is less than the predetermined threshold value, the range object is judged to be a range object that is not a subject object (hereinafter referred to as a "non-subject object"). The predetermined threshold value may be, for example, 50%.

[0070] Also, for example, the judgment condition may be that, for the range object to be judged, if the ratio of the area that can be captured in the virtual photograph when the range object is assumed to be completely unobstructed to the area of ​​the entire virtual photograph is equal to or greater than a predetermined threshold, the range object is judged to be a subject object, and if the ratio is less than the predetermined threshold, the range object is judged to be a non-subject object. The predetermined threshold may be, for example, 0.1%. Also, for example, the judgment condition may be that if the ratio of the area of ​​the object in the virtual photograph to the area of ​​the entire virtual photograph is equal to or greater than a predetermined threshold, the object is judged to be a subject object, and if the ratio is less than the predetermined threshold, the object is judged to be a non-subject object. The predetermined threshold may be, for example, 0.1%.

[0071] Also, for example, the judgment condition may be that if the center point of the range object being judged is captured in the virtual photograph, it is judged to be a subject object, and if not, it is judged to be a non-subject object, or if the ratio of the number of representative points (which may include the center point) that are preset in the range object and are within a certain distance from the position of the virtual camera and captured in the photograph is equal to or greater than a predetermined threshold, it is judged to be a subject object, and if it is less than the predetermined threshold, it is judged to be a non-subject object. The predetermined threshold is, for example, 50%.

[0072] Also, for example, the judgment condition may be that, when the range object being judged is an avatar, if the area of ​​the avatar's head that is actually captured in the virtual photograph is equal to or larger than a predetermined threshold value relative to the area of ​​the avatar's head that would appear in the virtual photograph if the avatar were not occluded at all, the avatar is judged to be a subject object regardless of whether other parts of the avatar are occluded or not, and if the area is less than the predetermined threshold value, the avatar is judged to be a non-subject object. The predetermined threshold value is, for example, 50%.

[0073] Also, for example, the judgment condition may be that when the range object being judged is an avatar, if the direction of the front of the avatar's head is within a predetermined angle range with respect to a line connecting the virtual camera and the head, it is judged to be a subject object, and if it is outside the predetermined angle range, it is judged to be a non-subject object. The predetermined angle range is, for example, a range of -90° to +90°, assuming that the angle of the front of the avatar's head when facing the virtual camera is 0°. Furthermore, the judgment condition may be a combination of two or more of the above examples. As described above, the contents of the judgment conditions can be set arbitrarily and are not limited to the above examples.

[0074] The determination unit 223 assigns the result of the subject object determination for each in-range object to the object information of the in-range object. For example, when a certain in-range object is determined to be a subject object, the determination unit 223 assigns a flag of "1" to the object information of the in-range object indicating the result of the subject object determination, and when the in-range object is determined to be a non-subject object, the determination unit 223 assigns a flag of "0" to the object information of the in-range object indicating the result of the subject object determination. Hereinafter, information that associates at least information that can identify an object within the range (for example, an object ID) with the result of the subject object determination will be referred to as a "determination result."

[0075] The determination unit 223 outputs the determination result to the determination result acquisition unit 224. For example, for all of one or more in-range objects, the determination unit 223 outputs the determination result to the determination result acquisition unit 224 by outputting object information of each in-range object to which a determination result has been assigned to the determination result acquisition unit 224. Alternatively, the determination unit 223 may output the determination result to the determination result acquisition unit 224 by linking the determination result for each in-range object with the object ID of the in-range object for one or more in-range objects and outputting the result to the determination result acquisition unit 224.

[0076] The determination unit 223 may be provided in either the participation terminal 4 or the service providing server 3, rather than in the information processing device 2. Details of this case will be described later.

[0077] (Judgment result acquisition part) The judgment result acquisition unit 224 acquires a judgment result based on the camera information and object information, which determines whether or not one or more objects (in-range objects) included within the shooting range are to be subject objects captured in the virtual photograph. When the determination result acquisition unit 224 acquires the determination result from the determination unit 223 , the determination result acquisition unit 224 outputs the acquired determination result to the linking unit 225 . The determination result acquisition unit 224 outputs the determination result to the linking unit 225, for example, by outputting, for all of one or more in-range objects, object information of each in-range object to which a determination result has been assigned, to the linking unit 225. Alternatively, the determination result acquisition unit 224 may output the determination result to the linking unit 225, for example, by linking the determination result for each in-range object with the object ID of the in-range object, for all of one or more in-range objects, and outputting the result to the linking unit 225.

[0078] (Linking section) The linking unit 225 links the photograph information with the space specifying information. When the linking unit 225 acquires the photographing information from the photographing information acquisition unit 221, the linking unit 225 links at least the photo ID included in the photograph information with the space specifying information and stores the space specifying information in the storage unit 23. The linking unit may link a space data ID as the space specifying information with the photo ID and store the space specifying information in the storage unit 23.

[0079] Furthermore, the linking unit 225 may link the object information and the photo information based on the determination result. When the linking unit 225 acquires a judgment result from the judgment result acquisition unit 224, it links at least the object ID of the information contained in the object information of the within-range object to be linked to the photo ID contained in the photo information based on the acquired judgment result, and stores the same in the memory unit 23. The linking unit 225 may link the object information and the photo information in various ways based on the determination result.

[0080] For example, the linking unit 225 may perform processing such that object information of within-range objects that are determined to be subject objects is linked to photographic information, and object information of non-subject objects, which are within-range objects that are determined not to be subject objects, is not linked to photographic information.

[0081] Also, for example, the linking unit 225 may perform a process of distinguishing between object information of objects within the range that have been determined to be subject objects and object information of non-subject objects, which are objects within the range that have been determined not to be subject objects, and linking the object information to photograph information. In this case, the linking unit 225, for example, links the object ID of the subject object and a flag "1" indicating the subject object to the photo ID and stores them in the memory unit 23, and links the object ID of the non-subject object and a flag "0" indicating the non-subject object to the photo ID and stores them in the memory unit 23.

[0082] Also, for example, the linking unit 225 may perform a process of linking the object information of a pre-set specific object within the range with the photograph information, regardless of whether the object within the range is determined to be a subject object or a non-subject object, or conversely, a process of not linking the object information of the object within the range with the photograph information, regardless of whether the object within the range is determined to be a subject object or a non-subject object. Regarding the former, for example, one or more specific avatars, all avatars, one or more specific items, or all items may be preset as specific in-range objects whose object information is always linked to photo information. Regarding the latter, for example, objects as backgrounds constituting the virtual reality space and objects similar to the background may be preset as specific in-range objects whose object information is not linked to photo information.

[0083] After the linking process is completed, the linking unit 225 may transmit data in which the space specification information, object information, and photo information are linked (hereinafter referred to as "linked photo information") to the participating terminal 4 that transmitted the shooting information including the photo information. In this case, the shooting information transmitted from the participating terminal 4 to the shooting information acquisition unit 221 may include, for example, the user ID of the participant who participated in the virtual reality space as an avatar using the participating terminal 4 and performed a shooting operation to take a virtual photo using a virtual camera. The linking unit 225 can identify the user ID from the shooting information acquisition unit 221 via the storage unit 23, etc. Furthermore, the linking unit 225 may transmit the linked photograph information to the service providing server 3 after the linking process is completed.

[0084] In addition, since the participating terminal 4 has already specified the objects within the range in the process of generating the virtual photograph, it can also be interpreted that the photo information of the virtual photograph and the object information are linked at the time when the participating terminal 4 generates the virtual photograph. Based on this interpretation, the linking process by the linking unit 225 can also be said to be a process of correcting the object information already linked to the photo information based on the determination result. In other words, the linking process by the linking unit 225 may include either or both of a process of linking new object information to photo information that is not linked to any object information, or a process of modifying linked object information in photo information that is already linked to object information.

[0085] As described above, the object also includes an avatar. Therefore, when one or more avatars are included in the shooting range, the determination unit 223 performs subject object determination for each of the objects within the range based on the camera information included in the shooting information and the object information of the objects within the range including the one or more avatars. Then, the judgment result acquisition unit 224 acquires the judgment result of the subject object judgment for each of one or more avatars as one or more objects (in-range objects) included within the shooting range, and the linking unit 225 links the object information regarding the avatar with the photo information based on the judgment result.

[0086] In addition to the space identification information, or in addition to the space identification information and the object information, the linking unit 225 may link the photo information with user identification information that identifies a user of the virtual reality space who took the virtual photo using the virtual camera. A user of the virtual reality space who took a virtual photograph using a virtual camera is, for example, a participant who participated in the virtual reality space as an avatar using a participant terminal 4 and performed a shooting operation to take a virtual photograph using the virtual camera. In the above case, the user identification information is the user ID of the participant.

[0087] In this case, the photographing information transmitted from the participant terminal 4 to the photographing information acquisition unit 221 may include, for example, the user ID of the participant who participated in the virtual reality space as an avatar using the participant terminal 4 and performed a photographing operation to take a virtual photo using a virtual camera. The linking unit 225 can identify the user ID from the photographing information acquisition unit 221 via the storage unit 23, etc.

[0088] Alternatively, the linking unit 225 may link the photo information with event information related to an event that was being held in the virtual reality space in which the virtual photograph was taken using the virtual camera at the time the virtual photograph was taken, in addition to the space identification information, or in addition to the space identification information and the object information. In this case, the photographing information transmitted from the participation terminal 4 to the photographing information acquisition unit 221 may include, for example, event information (event ID, organizer ID, or both IDs) related to the event that was being held at the time the virtual photograph was taken in the virtual reality space in which the participant was participating using the participation terminal 4. Alternatively, in a certain virtual reality space, when a plurality of events are held at the time of taking a virtual photograph, it is sufficient to include at least event information related to the event in which the participant's avatar is participating at the time of taking the virtual photograph. In a certain virtual reality space, when a plurality of events are held at the time of taking a virtual photograph, each of the plurality of events is usually held at a different position from each other in the virtual reality space. Therefore, the event in which the participant's avatar is participating can be specified, for example, from the position coordinates indicating the location of the event in the virtual reality space and the holding date and time (the holding date and the holding time period), and the position coordinates indicating the location of the participant's avatar who took the virtual photograph and the taking date and time.

[0089] In addition, the linking unit 225 may link the event information to the photo information, further including event information related to a related event that has a specific relationship with the event that was being held in the virtual reality space in which the virtual photo was taken using the virtual camera at the time the virtual photo was taken. A plurality of events having a specific relationship with each other can be included in one related event group as related events having a specific relationship with each other, and can be stored in the service providing server storage unit. The information processing device 2 can obtain information indicating the related event group from the service providing server storage unit in advance, and store the information in the storage unit 23. In this case, when the linking unit 225 acquires the shooting information from the shooting information acquisition unit 221, it accesses the memory unit 23, for example, and searches whether there is a related event group that includes an event that was held at the time when the virtual photo indicated by the currently acquired photo information was taken, and if a related event group is found, it links event information (e.g., event ID) related to one or more other related events included in the related event group to the photo information of the virtual photo.

[0090] In contrast, the information processing device 2 does not need to acquire information indicating the related event group from the service providing server storage unit in advance. In this case, instead, the event information of the event in the service provided by the service providing server 3 is acquired and stored in advance in the storage unit 23. When the linking unit 225 acquires the photographing information from the photographing information acquisition unit 221, the linking unit 225 first acquires the above-mentioned specific relationship description information, which is information describing a specific relationship, from the storage unit 23. Then, the linking unit 225 refers to the specific relationship description information to search for related events that have a specific relationship with an event related to an event held at the time of shooting the virtual photo indicated by the currently acquired photograph information, and if one or more related events exist, the linking unit 225 includes event information related to the related events in the event information and links the event information to the photograph information. If the information processing device 2 performs such processing every time it acquires photographing information from the photographing information acquisition unit 221, as a result, the related event group is stored in the storage unit 23 in a form linked to one piece of photograph information. However, since the above processing is performed only when a virtual photo is shot in a virtual reality space during an event, the related event group stored in the storage unit 23 is usually only a part of the related event group stored in the service providing server storage unit.

[0091] When object information and photo information are linked, the linking unit 225 may link the photo information with sales information regarding the sale of an object corresponding to the object information, in addition to the object information or as the object information. As described above, the sales information is associated with information capable of identifying an object, such as an object ID, and is stored in the service providing server storage unit of the service providing server 3. For example, the association unit 225 can access the service providing server 3 and transmit, as a search key, the object ID of the within-range object from which sales information is to be acquired, thereby acquiring, as a search result, the sales information of the within-range object stored in the service providing server storage unit from the service providing server 3. As described above, sales information may also be included in the object information. In this case, for example, sales information of the object may be included in the object information transmitted from the participating terminal 4 to the object information acquisition unit 222. The linking unit 225 can acquire the sales information of the objects within the range from the object information acquisition unit 222 via the storage unit 23, etc.

[0092] When object information and photo information are linked, the linking unit 225 may link searchability information, which indicates whether or not a search based on metadata including the object information linked to the photo information, to the photo information for each object. In particular, when the photo information includes two-dimensional data of a virtual photo, the object information linked to the photo information, in other words, the object information included in the linked photo information, can be said to be metadata of the photo information. The metadata of the photo information includes, for example, the above-mentioned user ID, space data ID, sales information, etc. The space data ID and the user ID of the space creator among the user IDs may be linked to the photo information, but are metadata other than the object information. The photo information may also include the time when the virtual photo was taken, and this time can also be said to be metadata of the photo information. The metadata linked to the photo information in this way may also include metadata other than the object information.

[0093] When a considerable amount of linked photo information is stored in the storage unit 23 of the information processing device 2, the information processing device 2 can provide a service to a user for searching for photo information using any of the metadata linked to the photo information as a search key, as described below. In this case, the photo information includes a photo ID and two-dimensional data of a virtual photo. However, there are cases where the owner of the within-range object does not want others to obtain, through a search, the photo information of the virtual photo in which the within-range object appears. For example, when the within-range object is an avatar, if a participant who participates in the virtual reality space using the avatar obtains photo information that is not intended by the participant, the participant may feel that his / her privacy has been violated.

[0094] Therefore, the service providing server 3 may have a function that enables the owner of an object to register searchability information indicating whether or not the object is searchable, in association with information that can identify the object, such as an object ID. For example, a participant who plans to participate in a virtual reality space using an avatar registers searchability information in the service providing server 3, linking it to the object ID of the avatar he or she owns, when registering as a user of the service. The service providing server 3 stores the searchability information in the service providing server storage unit, linking it to the object ID of the avatar owned by the user, by, for example, setting a flag of "1" if the searchability is possible and setting a flag of "0" if the searchability is not possible.

[0095] The linking unit 225 can access the service providing server 3 and send the object ID of the within-range object for which searchability information is to be obtained as a search key, thereby obtaining from the service providing server 3 the searchability information of the within-range object stored in the service providing server memory unit as a search result.

[0096] The linking unit 225 may also determine whether or not search is possible based on the environment in which the virtual photograph was taken, and generate searchability information. For example, searchability information such as a flag indicating whether or not search is possible may be linked to each spatial data ID and stored in advance in the service providing server storage unit of the service providing server 3.

[0097] (Search key acquisition section, search result acquisition section) The information processing device 2 may further include a search key acquisition unit 226 that acquires object information as a search key for searching for photo information, and a search result acquisition unit 227 that acquires photo information searched based on the object information. When a considerable amount of linked photo information is stored in the storage unit 23 of the information processing device 2, the information processing device 2 can provide a user with a service (hereinafter referred to as a "search service") for searching for photo information using, as a search key, for example, an object ID, an owner ID, etc., among the object information linked to the photo information. In this case, the photo information includes the photo ID and two-dimensional data of the virtual photo. A user of the search service is, for example, a user who is registered as a user of a service that uses a virtual reality space provided by the service providing server 3.

[0098] When a user of the search service operates his / her own terminal (hereinafter referred to as the "search terminal") to access the information processing device 2 and then inputs and transmits a search key to the search terminal, the search key acquisition unit 226 of the information processing device 2 acquires the search key transmitted from the search terminal. Note that the search terminal may be a terminal used as a participating terminal 4. When the search key acquisition unit 226 acquires the search key, it outputs the acquired search key to the search result acquisition unit 227 . When the search result acquisition unit 227 acquires the search key from the search key acquisition unit 226, it accesses the storage unit 23 and acquires, as a search result, photo information linked to object information corresponding to the search key, or the photo information and object information linked to the photo information (linked photo information). If the photo information linked to object information corresponding to the search key is not stored in the storage unit 23, the search key acquisition unit 226 acquires, as a search result, information indicating that the photo information linked to object information corresponding to the search key could not be acquired.

[0099] When the search result acquisition unit 227 acquires the search result, it transmits the acquired search result to the searching terminal.

[0100] The information processing device 2 may include a search key acquisition unit 226 that acquires space specific information as a search key for searching for photo information, and a search result acquisition unit 227 that acquires photo information searched based on the space specific information. The information processing device 2 can provide a user with a search service for searching for photo information using the space-specific information, i.e., the space data ID, as a search key, from among the metadata of the object information linked to the photo information. In this case, the photo information includes the photo ID and the two-dimensional data of the virtual photo. The processes performed by search key acquisition unit 226 and search result acquisition unit 227 are similar to those in the case where object information is used as a search key described above, and therefore a description thereof will be omitted.

[0101] The search key acquisition unit 226 may acquire only object information as a search key, may acquire only space specific information, or may acquire both. Furthermore, the linking unit 225 may also link with, for example, event information, etc. Therefore, the search key acquisition unit 226 may acquire metadata of the photo information other than the object information and space specific information as a search key. In either case, the search result acquisition unit 227 acquires search results based on the search key acquired from the search key acquisition unit 226.

[0102] The information processing device 2 includes a search key acquisition unit 226 that acquires object information as a search key for searching for photo information, and a search result acquisition unit 227 that acquires photo information searched based on the object information and linked to searchability information indicating that the photo information is searchable. As described above, when the linking unit 225 links the searchability information to photo information for each object, and the searchability information is stored in the memory unit 23 linked to the photo information, the search result acquisition unit 227, even if photo information linked to object information corresponding to the search key is searched for, if the searchability information linked to the photo information indicates that it is not searchable (for example, flag "0"), does not acquire the searched photo information, but instead acquires, as the search result, information indicating that the photo information linked to the object information corresponding to the search key could not be acquired. On the other hand, if the search result acquisition unit 227 searches for photo information linked to object information corresponding to the search key, and the searchability information linked to the photo information indicates that the photo information is searchable (for example, flag "1"), the search result acquisition unit 227 acquires the searched photo information as the search result.

[0103] In either case, when the search result acquisition unit 227 acquires the search results, it transmits the acquired search results to the searching terminal. The search service by the information processing device 2 can be provided as long as the photo information is linked to some object information. In other words, the linked photo information that is the target of the search service does not necessarily have to be linked based on the result of the subject object determination.

[0104] (Notification Department) The information processing device 2 may further include a notification unit 228 that, when object information of an object within the range determined to be a subject object is linked to photo information, notifies a notification destination identified in accordance with predetermined notification conditions of the photo information. When the information processing device 2 includes the notification unit 228, the linking unit 225 performs processing for linking the object information with the photo information, and then outputs the linked photo information to the notification unit 228. In this case, the metadata included in the linked photo information includes user IDs of various users.

[0105] After the notification unit 228 acquires the linked photo information, notification conditions that define to whom the acquired linked photo information is to be notified are set in advance and stored, for example, in the storage unit 23. The notification conditions are information that describes to which notification destination the photo information is to be notified, depending on the object information of the subject object associated with the photo information, the space specification information (space data ID) associated with the photo information, or the event information associated with the photo information. The notification conditions are, for example, determined by an organization or individual that provides a service using the service providing server 3, and are set in advance by an operation on the information processing device 2. When the notification unit 228 acquires the linked photo information from the linking unit 225, it acquires the notification conditions from the storage unit 23 and notifies the notification destination that satisfies the notification conditions of the photo information or the linked photo information.

[0106] If the metadata included in the linked photo information includes information indicating the means of notification to the notification destination (e.g., an email address or an in-app notification of a service app, etc.), the notification unit 228 uses the indicated notification means to notify the notification destination of the photo information or the linked photo information. If the metadata included in the linked photo information does not include information indicating the notification means to the notification destination, the notification unit 228 accesses the service providing server 3 to acquire information indicating the notification means. The notification destination is usually a user registered in the service provided by the service providing server 3, and as described above, information indicating the notification means to the user is stored in the service providing server storage unit in association with the user ID. Therefore, if the metadata included in the linked photo information does not include information indicating the notification means to the notification destination, the notification unit 228 accesses the service providing server 3 and transmits the user ID to acquire information indicating the notification means from the service providing server 3.

[0107] For example, the notification unit 228 notifies the notification destination of the photo information, the owner of the object, the creator of the object, or the seller of the object, specified according to the notification conditions. Furthermore, for example, the notification unit 228 may notify the notification recipient of the photo information, the notification recipient being the space creator of the virtual reality space in which the virtual photograph was taken, the event organizer who was holding an event using the virtual reality space at the time the virtual photograph was taken within the virtual reality space, or the space monitor of the virtual reality space in which the virtual photograph was taken, all of which are identified according to the notification conditions. The notification condition may describe only one type of notification destination, or may describe multiple types of notification destinations.

[0108] The notification service by the information processing device 2 can be provided as long as the photo information is linked to some object information. In other words, the linked photo information that is the subject of the notification service does not necessarily have to be linked based on the result of the subject object determination.

[0109] (Display information generation section) The information processing device 2 may further include a display information generation unit 229 that generates display information for displaying object information of a specified object when a subject object is specified by an operation of specifying the subject object in a virtual photograph performed on a virtual photograph placed in a virtual reality space. Hereinafter, it is assumed that the storage unit 23 stores object information similar to the object information of the subject object stored in the service providing server storage unit of the service providing server 3.

[0110] As described above, the substance of a virtual photograph is two-dimensional data, but it can also be one of the objects. In other words, the two-dimensional data of a virtual photograph can be treated as three-dimensional data without thickness. For example, an avatar in a virtual reality space can hold a virtual photograph like any other object, and can change its state by moving it, etc.

[0111] For example, for in-range objects appearing in a virtual photograph, information regarding which part of the photograph corresponds to which in-range object (hereinafter referred to as "in-photo location information") may be added as metadata linked to the photograph information of the virtual photograph. Such information can be obtained during the process of generating the virtual photograph, for example, because when the final pixel color, etc. is specified, the pixel color, etc. is specified using the result of determining which pixel is the color of which range object. The in-photo location information is information that links the position of each pixel in the virtual photo with the object ID of the object within the range corresponding to that pixel, and is managed by linking it with the photo ID of the virtual photo. The in-photo location information is a type of metadata for the two-dimensional data of the virtual photo.

[0112] A participant can first perform an appropriate operation on the participant terminal 4, specify a desired virtual photo from one or more virtual photos stored as linked photo information in the storage unit 23 of the information processing device 2, acquire the virtual photo, and place it in the virtual reality space. If a virtual photo is stored as linked photo information in the participant terminal storage unit of the participant terminal 4 or the service providing server storage unit of the service providing server 3, a participant can also perform an appropriate operation on the participant terminal 4, specify a desired virtual photo from these storage units, and place it in the virtual reality space. For example, a participant can use the participant terminal 4 to view one or more virtual photos taken by the participant using an album object that imitates a real album, and can specify one of the virtual photos to place in the virtual reality space. Also, for example, a participant can operate the participant terminal 4 to place a virtual photo acquired using the above-mentioned search service in the virtual reality space.

[0113] When a virtual photo is designated by operating the participating terminal 4, the participating terminal 4 can access the information processing device 2 and transmit the photo ID of the designated virtual photo to obtain linked photo information of the designated virtual photo stored in the storage unit 23. When the participating terminal 4 obtains the virtual photo as linked photo information, it places the virtual photo in the virtual reality space.

[0114] Thereafter, the participant can operate the participant terminal 4 to specify one of the subject objects in the virtual photograph placed in the virtual reality space. The operation for designation (hereinafter referred to as "designation operation") is an operation of pressing a point on the object captured in the virtual photograph with a finger or the like of the avatar, or an operation of pressing a point on the object captured in the virtual photograph with a designation pointer arranged in the virtual reality space, etc. The designation operation may be any operation that can designate a point on the object captured in the virtual photograph.

[0115] When the avatar's fingers or the like move on the virtual photograph, the participating terminal 4 may identify the position on the virtual photograph where the avatar's fingers or the like are present, and may determine the subject object present at the identified position based on the in-photo position information and information on whether the object within the range linked to the photo information is a subject object, and may display a frame line surrounding the contour of the subject object shown in the virtual photograph. If the identified position corresponds to an object or the like other than the subject object, the participating terminal 4 does not display a frame line surrounding the contour. When the participating terminal 4 displays such a frame line, the participant can more easily know which subject object can be specified at the current position of the avatar's finger, etc., compared to when the frame line is not displayed.

[0116] The virtual photo may be displayed outside the virtual reality space. For example, the virtual photo may be displayed on a WEB browser that is normally displayed on the screen of a terminal used by a user such as the participant terminal 4. In this case, the designation operation may be an operation of clicking a point on the object shown in the virtual photo with a mouse, or an operation of tapping a point on the object shown in the virtual photo on a touch panel.

[0117] When a point on an object captured in a virtual photograph is specified, the participating terminal 4 identifies the coordinates of the specified point on the virtual photograph, in other words, the pixel position, and identifies the object ID of the specified object based on the identified pixel position and position information within the photograph. When the participation terminal 4 identifies the object ID of the specified subject object, it transmits the identified object ID to the display information generation unit 229 of the information processing device 2. The display information generation unit 229 acquires, from the storage unit 23, object information associated with the object ID as a search result.

[0118] In this case, the object information includes three-dimensional data such as three-dimensional shape information or size information of the subject object, or metadata such as an object ID, an owner ID, or sales information regarding the sale of the object, and does not include information specific to the time of shooting, such as location information at the time of shooting, orientation information at the time of shooting, or posture information at the time of shooting. However, if information specific to the time of photographing is associated with the photo ID and stored in the storage unit 23, the information specific to the time of photographing may be included in the object information. In this case, the participating terminal 4 transmits the photo ID to the display information generating unit 229 of the information processing device 2 in addition to the object ID of the specified subject object.

[0119] When the display information generation unit 229 acquires the object information, it generates display information for displaying the object information of the specified subject object. Then, the display information generation unit 229 transmits the generated display information to the participating terminal 4. When the participating terminal 4 acquires the display information, it displays the object information based on the display information.

[0120] As described above, the linking unit 225 links at least the object ID of the information included in the object information of the range object with the photo ID included in the photo information and stores it in the storage unit 23. Therefore, the object information included in the linked photo information may be only the object ID, or may have a relatively small amount of information. In response to this, the information processing device 2 can provide more detailed information to the participants by, for example, acquiring detailed object information stored in the storage unit 23. When the displayed object information is selected, the participating terminal 4 may display an object purchase screen, or a favorites registration screen or the like.

[0121] FIG. 5 is a screen diagram showing an example of object designation in a virtual photograph. Fig. 5, like Fig. 4, shows a virtual photograph VP generated by a photographing operation performed in the state of Fig. 3. The virtual photograph VP shows three avatars A2 to A4, one tree object C1, and two luggage objects C2 to C3, but does not show most of the avatar A2, including the lower body and the upper body of the avatar A4, for example. Here, among the objects within the range captured in the virtual photograph VP, only the avatar A4 is a non-photographed object, and all other objects within the range are photographed objects.

[0122] 5 shows designation pointers SL1-SL2 for designating subject objects in addition to within-range objects. The designation pointers SL1-SL2 are a type of object located in the virtual reality space, but are not within-range objects captured in the virtual photograph VP. A participant can move the designation pointers SL1-SL2 to any position on the virtual photograph VP by operating the participant terminal 4, and can also use the designation pointers SL1-SL2 to designate a point on the subject object captured in the virtual photograph VP.

[0123] Fig. 6 is a screen diagram showing a display example of objects and object information in a virtual photograph. In Fig. 5, it is assumed that a participant operates the participant terminal 4 to specify a point on the luggage object C2 using the specification pointer SL1 and to specify a point on the luggage object C3 using the specification pointer SL2. In this case, as shown in Fig. 6, a speech bubble OF1 is displayed for a luggage object C2 captured in the virtual photograph VP, and a speech bubble OF2 is displayed for a luggage object C3 captured in the virtual photograph VP. The speech bubbles OF1-OF2 are also a type of object placed in the virtual reality space, but are not objects within the range captured in the virtual photograph VP. The speech bubbles OF1-OF2 may be objects that can be seen by participants other than the participant who performed the operation to display them, or they may be objects that cannot be seen by those participants. Inside the speech bubble OF1, object information of a parcel object C2 is displayed, and inside the speech bubble OF2, object information of a parcel object C3 is displayed.

[0124] The information processing device 2 may also include a display information generation unit 229 that, when a subject object is designated by an operation of designating a subject object in a virtual photograph performed on a virtual photograph placed in a virtual reality space, displays object information of the designated subject object, and, when a non-subject object is designated by an operation of designating a non-subject object, generates display information for displaying the object information of the designated non-subject object.

[0125] A non-focused object may appear in its entirety in the virtual photograph, even if it is small, may appear only in part, or may not appear at all. If a non-subject object appears in the virtual photograph in some manner, the participant can specify the non-subject object in the virtual photograph by operating the participant terminal 4, in the same manner as in the case of the subject object described above, and cause object information of the non-subject object to be displayed.

[0126] In contrast, in order to allow a non-subject object to be specified even when the non-subject object does not appear at all in the virtual photograph, for example, the participating terminal 4 can display a list indicating at least the object IDs of one or more non-subject objects based on the object information of the non-subject objects included in the linked photograph information. Also, if the object information of the non-subject objects included in the linked photograph information includes an object name, the participating terminal 4 can display a list indicating the object names of one or more non-subject objects instead of or in addition to the object ID. The participating terminal 4 may display the list within the virtual reality space, or may display it outside the virtual reality space. In addition, the participating terminal 4 may display the list together with the object ID or object name of the subject object.

[0127] A participant operates the participating terminal 4 to display the list and select a non-subject object in the list, whereby the participating terminal 4 transmits the object ID of the selected non-subject object to the display information generating unit 229 and displays the object information of the selected non-subject object through the same process as described above. Also, if the participating terminal 4 is displaying the list together with the object ID or object name of the subject object, the participant can similarly display the object information of the selected subject object by selecting the subject object in the list.

[0128] In addition, when a virtual photograph is generated by participating terminal 4, if it is possible to identify the position within the virtual photograph at which an object within the range that does not appear at all in the virtual photograph would appear in the virtual photograph if it were not obstructed at all, then it is also possible to include that position in the object information linked to the photograph information. In this case, for example, when a participant operates the participating terminal 4 to specify a desired range on the virtual photograph, the participating terminal 4 can also identify the object IDs of non-subject objects that are partly or entirely included within that range but are not shown in the virtual photograph. Then, the participating terminal 4 transmits the object ID of the identified non-object to the display information generating unit 229, and through the same processing as described above, the object information of the identified non-object can be displayed.

[0129] The object information display service by the information processing device 2 can be provided as long as the photo information of the virtual photo is linked to some object information. In other words, the linked photo information of the virtual photo that is the subject of the object information display service does not necessarily have to be linked based on the result of the subject object determination.

[0130] (Space selection screen display information acquisition unit) The information processing device 2 may further include a space selection screen display information acquisition unit 230 that acquires space selection screen display information for displaying a space selection screen that allows a user to select one virtual reality space for participation from among a plurality of different virtual reality spaces. The space selection screen displays space recognition information that allows each of a plurality of different virtual reality spaces to be recognized, and displays a virtual photo associated with the space identification information of the virtual reality space for each piece of space recognition information. A user can select a space in which he or she wants to participate and participate in the space with his or her avatar. The space recognition information is information that allows a user who sees the information to easily recognize a virtual reality space, and is, for example, a thumbnail image of the virtual reality space corresponding to the space recognition information, the name of the virtual reality space, and the name of the space creator who created the space data of the virtual reality space and uploaded it to the service providing server 3.

[0131] For example, when a service user accesses the service providing server 3 from a participation terminal 4, the space selection screen can be displayed on the display of the participation terminal 4 either initially or after a predetermined operation by the user.

[0132] The space selection screen is a screen displayed for allowing the user to select the virtual reality space in which he or she will participate from among a plurality of different virtual reality spaces. On the space selection screen, a plurality of pieces of space recognition information different from each other are displayed, for example, in n rows and m columns (n ​​and m are arbitrary integers). For example, one piece of space recognition information is displayed in such a manner that the name of the virtual reality space is arranged below the thumbnail image, and the name of the space creator is arranged below that. Then, around each piece of space recognition information (for example, below the name of the space creator), one or more virtual photos taken in the virtual reality space corresponding to that piece of space recognition information are displayed side by side. These one or more virtual photos include at least virtual photos that are not created by the space creator (for example, thumbnail images or virtual photos taken by the space creator as a participant). In other words, around a certain piece of space recognition information, at least one or more virtual photos taken in the virtual reality space by one or more participants who are participants other than the space creator of the virtual reality space corresponding to that piece of space recognition information are displayed side by side.

[0133] For example, when the service providing server 3 is accessed by a participation terminal 4, it transmits to the information processing device 2 space identification information (e.g., space data ID) of a plurality of virtual reality spaces to be displayed on the space selection screen, space recognition information, and the user ID of the user who has accessed from the participation terminal 4. When the information processing device 2 acquires the space identification information, space recognition information, and user ID of a plurality of virtual reality spaces from the service providing server 3, it extracts a virtual photo linked to the space identification information from the storage unit 23, and generates space selection screen display information according to a predetermined format. When the space selection screen display information acquisition unit acquires the generated space selection screen display information, it transmits the acquired space selection screen display information to the participation terminal 4 identified by the user ID. When the participation terminal 4 acquires the space selection screen display information from the information processing device 2, it displays the space selection screen on the display.

[0134] The user refers to the multiple pieces of space recognition information and virtual photos displayed on the space selection screen to determine the virtual reality space in which he or she will participate, and performs a selection operation to select the space recognition information of that virtual reality space, for example by a click operation. When the selection operation is performed on the participating terminal 4, the participating terminal 4 transmits space identification information of the selected virtual reality space to the service providing server 3, and in response, the service providing server 3 transmits space data of the virtual reality space identified by the space identification information to the participating terminal 4. When the participating terminal 4 obtains the space data from the service providing server 3, it displays the virtual reality space represented by the space data on the display.

[0135] In the above explanation, an example was shown in which when a user performs a selection operation to select one piece of spatial recognition information from among a plurality of pieces of spatial recognition information, spatial data is immediately transmitted to the participating terminal 4, but instead of this, when the above selection operation is performed, a detailed explanation screen (hereinafter referred to as a "detailed explanation screen") of only the selected virtual reality space may be displayed first. The detailed explanation screen also displays the spatial recognition information and one or more virtual photos, as well as a detailed explanation of the virtual reality space, such as an explanation of the type of space the selected virtual reality space is, the size of the spatial data, the release date, whether it can be used in the event, etc. Here, the detailed explanation screen is also included in the space selection screen.

[0136] In the above description, an example was shown in which a virtual photograph is displayed around each of a plurality of pieces of space recognition information displayed on the space selection screen. Alternatively, when a plurality of pieces of space recognition information are displayed on the space selection screen, the virtual photograph may not be displayed, and the virtual photograph may be displayed only on the detailed explanation screen after one piece of space recognition information (corresponding virtual reality space) is selected from the space selection screen in this state.

[0137] (Event selection screen display information acquisition unit) The information processing device 2 may further include an event selection screen display information acquisition unit 231 that acquires event selection screen display information for displaying an event selection screen on which one event for participation can be selected from among a plurality of different events. The event selection screen displays event recognition information that allows a plurality of different events to be recognized, and displays a virtual photo associated with the event information for each event recognition information. A user can select an event that the user wants to participate in and participate in the space in which the event is held with his / her avatar. The event recognition information is information that allows a user who sees the information to easily recognize the contents of the event, and is, for example, a thumbnail image of the event corresponding to the event recognition information, the name of the event, or the name of the event organizer of the event.

[0138] For example, when a service user accesses the service providing server 3 from a participation terminal 4, the event selection screen can be displayed on the display of the participation terminal 4 either initially or after a predetermined operation by the user.

[0139] The event selection screen is a screen displayed for a user to select an event in which the user will participate from among a plurality of different events. On the event selection screen, multiple different event recognition information are displayed, for example, aligned in n rows and m columns (n ​​and m are any integers). For example, one piece of event recognition information is displayed in such a manner that the name of the event is placed below a thumbnail image, and the name of the event organizer is placed further below that. Then, around each piece of event recognition information (for example, below the name of the event organizer), one or more virtual photographs taken in a virtual reality space when the event corresponding to that event recognition information is being held are displayed side by side.

[0140] For example, when the service providing server 3 is accessed by the participation terminal 4, it transmits to the information processing device 2 event information (e.g., event IDs) of multiple events to be displayed on the event selection screen, event recognition information, and the user ID of the user who has accessed from the participation terminal 4. When the information processing device 2 acquires the event information of multiple events, the event recognition information, and the user ID from the service providing server 3, it extracts a virtual photo linked to the event information from the storage unit 23 and generates event selection screen display information according to a predetermined format. When the event selection screen display information acquisition unit acquires the generated event selection screen display information, it transmits the acquired event selection screen display information to the participation terminal 4 identified by the user ID. When the participation terminal 4 acquires the event selection screen display information from the information processing device 2, it displays the event selection screen on the display.

[0141] The user decides which event he or she will participate in while referring to the multiple pieces of event recognition information and virtual photos displayed on the event selection screen, and performs a selection operation to select the event recognition information for that event, for example, by a click operation. When a selection operation is performed on the participation terminal 4, the participation terminal 4 transmits to the service providing server 3 the space specification information of the selected event and the virtual reality space in which the event is held, and in response, the service providing server 3 transmits space data of the virtual reality space specified by the space specification information to the participation terminal 4. This space data also includes data for displaying the event being held. When the participation terminal 4 obtains the space data from the service providing server 3, it displays the virtual reality space represented by the space data on the display.

[0142] In the above explanation, an example was shown in which when a user performs a selection operation to select one piece of event recognition information from a plurality of pieces of event recognition information, spatial data is immediately transmitted to the participant terminal 4, but instead of this, when the above selection operation is performed, a detailed explanation screen (hereinafter referred to as a "detailed explanation screen") of only the selected event may be displayed first. The detailed explanation screen also displays the event recognition information and one or more virtual photographs, as well as a detailed explanation of the event, such as an explanation of the content of the selected event, the date and time of the event, etc. Here, the detailed explanation screen is also included in the event selection screen.

[0143] In the above description, an example has been shown in which a virtual photograph is displayed around each of a plurality of pieces of event recognition information displayed on the event selection screen. Alternatively, when a plurality of pieces of event recognition information are displayed on the event selection screen, the virtual photograph may not be displayed, and the virtual photograph may be displayed only on the detailed explanation screen after one piece of event recognition information (corresponding event) is selected from the event selection screen in this state.

[0144] (Selection section) Information processing device 2 may further include a selection unit that selects, based on a selection condition, a virtual photo to be displayed on the space selection screen from among a plurality of virtual photos associated with space specifying information of the virtual reality space. When a plurality of virtual photographs are linked to space identification information of one virtual reality space and stored in the storage unit 23, it may not be possible to display all of the virtual photographs around the space recognition information of the virtual reality space on the space selection screen. In addition, it is preferable that the virtual photograph to be displayed on the space selection screen is one that provides an incentive for the user to participate in the virtual reality space.

[0145] The selection conditions are set in advance and stored in the storage unit 23. The selection conditions may be common to all virtual reality spaces, or may be set for each virtual reality space. If the selection conditions are common to all virtual reality spaces, the selection conditions are set, for example, by a service provider using the service providing server 3. If the selection conditions are set for each virtual reality space, for example, each selection condition is set by a space creator of each virtual reality space.

[0146] The content of the selection condition may be any content of the condition. However, in the selection condition, an upper limit number of selections may be set for one virtual reality space (one piece of space recognition information), and in consideration of the viewability of the space selection screen, it is desirable that the upper limit number be common to all virtual reality spaces. The upper limit number is set, for example, by a service provider using the service providing server 3. The selection conditions may be, for example, a completely random selection condition, a selection condition that does not bias the subject object to virtual photographs of a specific object (for example, a condition to exclude virtual photographs linked to the same subject object as the subject object linked to the selected virtual photograph, etc.), a selection condition that does not bias the photographer to virtual photographs taken by a specific participant (for example, a condition to exclude virtual photographs taken by the same photographer as the participant who took the selected virtual photograph (hereinafter referred to as the "photographer")), a selection condition that does not bias the event ID linked to the virtual photograph to a specific event ID (for example, a condition to exclude virtual photographs linked to the same event ID as the event ID linked to the selected virtual photograph, etc.), a selection condition that prioritizes the most recently taken virtual photograph, a virtual reality space those that make the selection conditional on giving priority to a virtual photo taken by a photographer who has spent the most time in the virtual reality space, those that make the selection conditional on giving priority to a virtual photo linked to object information of a subject object who has spent the most time in the virtual reality space, those that make the selection conditional on giving priority to a virtual photo where the space creator is not the photographer, those that make the selection conditional on giving priority to a virtual photo with a large number of linked subject objects, those that make the selection conditional on displaying a virtual photo previously selected by the space creator of the virtual reality space, those that make the selection conditional on giving priority to a virtual photo linked as a subject object to a friend of a user displaying the space selection screen when a service is provided in which users can register each other as friends, and those that make the selection conditional on giving priority to a virtual photo with a reaction such as "like" when the virtual photo is posted on a social networking site or the like. The selection condition may be, for example, a combination of any of the above-mentioned multiple selection conditions.

[0147] When generating space selection screen display information as described above, the information processing device 2 first obtains, from the memory unit 23, selection conditions to be applied to the spatial recognition information of each virtual reality space, and in accordance with the selection conditions, selects and extracts virtual photos to be displayed on the space selection screen from the multiple virtual photos stored in the memory unit 23, and generates space selection screen display information using the extracted virtual photos. The function of selecting a virtual photo by the selection unit 232 can also be applied to the selection of a virtual photo to be displayed on the event selection screen. In this case, the selection conditions and the processing procedure by the information processing device 2 are similar to those in the space selection screen, and therefore detailed explanations are omitted. In addition, the function of the selection unit in the information processing device 2 can be realized as long as the photograph information and various information are linked in advance by some method. In other words, the function of acquiring spatial data from the introduction screen does not necessarily have to be achieved by linking the photograph information and various information performed by the photograph information acquisition unit 221 and the linking unit 225 as described above.

[0148] (Introduction screen display information acquisition unit, spatial data acquisition unit) The information processing device 2 may further include an introduction screen display information acquisition unit 233 that acquires introduction screen display information for displaying an introduction screen on which one or more virtual photographs and spatial recognition information capable of recognizing the virtual reality space in which the virtual photographs were taken, or event recognition information capable of recognizing an event that was being held in the virtual reality space in which the virtual photographs were taken at the time the virtual photographs were taken are displayed, and a spatial data acquisition unit 234 that, when one virtual photograph out of the one or more virtual photographs is selected on the introduction screen, identifies the virtual reality space in which the virtual photograph was taken based on the space identification information or event information linked to the selected virtual photograph, acquires spatial data that is three-dimensional data of the identified virtual reality space, and transmits the acquired spatial data to the participating terminal on which the operation of selecting the virtual photo was performed.

[0149] The introduction screen may be any screen that displays one or more virtual photos and displays space recognition information or event recognition information corresponding to each virtual photo around the virtual photo. Therefore, the introduction screen also includes the space selection screen or event selection screen described above. The introduction screen may also include a feed screen or a thumbnail screen. The feed screen is, for example, one of the services similar to a general SNS that is provided by the service providing server 3 to users of the service, and allows users of any or a specific service to post, view, quote or repost virtual photos, as well as to give ratings such as "likes" or bookmarks, etc. Furthermore, the thumbnail screen, in a limited screen area around the home screen of the above-mentioned service, displays, for example, one or more thumbnail images of virtual reality spaces or events that the user viewing the screen is likely to be interested in. This function, like the display of recommended information on general websites, can be realized by storing the participation history of virtual reality spaces or events for each user, analyzing the user's preferences based on the history, and selecting a similar virtual reality space or event that matches the preferences.

[0150] For example, when a service user accesses the service providing server 3 from a participation terminal 4, the introduction screen can be displayed on the display of the participation terminal 4 either initially or after a predetermined operation by the user. The introduction screen is a screen displayed so that the user can select a virtual reality space or event in which he or she will participate from among multiple displayed virtual photos.

[0151] For example, when the service providing server 3 is accessed by the participation terminal 4, it requests the information processing device 2 for information for displaying an introduction screen. This request may include the user ID of the user using the participation terminal 4. When the information processing device 2 acquires the request for information for displaying an introduction screen from the service providing server 3, it extracts a virtual photo to be displayed on the introduction screen from the storage unit 23 according to a predetermined condition, and extracts space recognition information or event recognition information from the storage unit 23 based on the space specification information or event information linked to the virtual photo, and generates introduction screen display information according to a predetermined format. When the introduction screen display information acquisition unit acquires the generated introduction screen display information, it transmits the acquired introduction screen display information to the participation terminal 4 identified by the user ID. When the participation terminal 4 acquires the introduction screen display information from the information processing device 2, it displays the introduction screen on the display.

[0152] The user decides on a virtual photo of the virtual reality space or event in which the user will participate while referring to the multiple virtual photos displayed on the introduction screen, and performs a selection operation to select the virtual photo, for example, by a click operation. The selection operation may be a selection of space recognition information or event recognition information displayed around the virtual photo, for example, by a click operation. When the selection operation is performed on the participation terminal 4, the participation terminal 4 transmits to the information processing device 2 space specification information of the virtual reality space or event information of the event linked to the photo information of the selected virtual photo. When the space data acquisition unit 234 of the information processing device 2 acquires the space specification information or the event information, it acquires the space data of the virtual reality space indicated by the space specification information, or the space data of the virtual reality space in which the event indicated by the event information is held. This allows the user to easily participate with his or her avatar in the virtual space in which the virtual photo was taken (the virtual reality space indicated by the space specification information or the virtual reality space in which the event indicated by the event information is held) via the introduction screen.

[0153] For example, the storage unit 23 of the information processing device 2 may store all spatial data previously acquired from the storage unit for the service providing server, in which case the spatial data acquisition unit 234 acquires the spatial data from the storage unit 23. Alternatively, when the spatial data acquisition unit 234 acquires the space specifying information or the event information from the participating terminal 4, it may acquire the spatial data by accessing the storage unit for the service providing server of the service providing server 3.

[0154] When the spatial data acquisition unit 234 acquires the spatial data, it transmits the acquired spatial data to the participating terminal 4. When the participating terminal 4 acquires the spatial data, it displays the virtual reality space represented by the spatial data on the display.

[0155] In the above description, an example was shown in which when a user performs a selection operation to select one virtual photograph from among a plurality of virtual photographs, spatial data is immediately transmitted to the participant terminal 4. Alternatively, when the selection operation is performed, a detailed explanation screen (hereinafter referred to as a "detailed explanation screen") of only the selected virtual reality space or event may be displayed first. In the detailed explanation screen, in addition to displaying space recognition information or event recognition information and one or more virtual photographs linked to (the space specification information of) the virtual reality space indicated by the space recognition information or (the event information of) the event indicated by the event recognition information, a detailed explanation of the virtual reality space or event, such as an explanation of the contents of the selected virtual reality space or event, is displayed. Then, by selecting a virtual photograph in this detailed explanation screen, spatial data may be transmitted to the participant terminal 4. Here, the detailed explanation screen is also included in the introduction screen.

[0156] The function of acquiring space data from an introduction screen (including a space selection screen, an event selection screen, etc.) in the information processing device 2 can be realized as long as the space specification information or the event information is linked to the photo information in advance by some method. In other words, the function of acquiring space data from an introduction screen does not necessarily have to be achieved by the linking of the space specification information or the event information with the photo information performed by the photographing information acquisition unit 221 and the linking unit 225 as described above.

[0157] (Information processing method) FIG. 7 is a flowchart showing the information processing method according to the first embodiment. The shooting information acquisition unit 221 acquires shooting information including photo information regarding a virtual photograph taken by a virtual camera used in a virtual reality space, and space identification information that identifies the virtual reality space in which the virtual photograph was taken using the virtual camera among a plurality of different virtual reality spaces (step ST1). The linking unit 225 links the photograph information with the space specifying information (step ST2).

[0158] In addition, the information processing device 2 only needs to have the functions of at least the shooting information acquisition unit 221 that acquires shooting information including photo information and space identification information, and the linking unit 225 that links the photo information and the space identification information, and the object information acquisition unit 222, the judgment unit 223, the judgment result acquisition unit 224, the search key acquisition unit 226 and the search result acquisition unit 227, the notification unit 228, the display information generation unit 229, or the space selection screen display information acquisition unit 230, the event selection screen display information acquisition unit 231, the selection unit 232, the introduction screen display information acquisition unit 233, or the space data acquisition unit are functions that can be added arbitrarily.

[0159] In the explanation so far, it has been assumed that at least the participating terminal 4 generates the virtual photograph, and that the later-described photo information and the like relating to the virtual photograph are transmitted from the participating terminal 4 to the information processing device 2. However, as described above, the participating terminal 4 and the service providing server 3 can share the states of avatars and objects in one and the same virtual reality space in almost real time, so that the service providing server 3 can also generate the virtual photograph. When the service providing server 3 generates the virtual photograph, the photo information and the like may be transmitted from the service providing server 3 to the information processing device 2 instead of being transmitted from the participating terminal 4 to the information processing device 2.

[0160] In the explanation so far, it has been assumed that all of the functions of the information processing device 2 are provided in a single server having a physical configuration independent of the service providing server 3 and the participating terminal 4. However, as described above, in the information processing system 1, some or all of the functions of the information processing device 2 may be provided in the service providing server 3 or in the participating terminal 4. In the information processing system 1, some or all of the functions of the information processing device 2 may be provided in a duplicated manner in the service providing server 3 and the participating terminal 4.

[0161] For example, the determination unit 223 may be provided in the participation terminal 4 instead of the information processing device 2. In that case, the shooting information acquisition unit 221 does not need to acquire camera information from the participation terminal 4. Furthermore, the participation terminal 4 transmits the shooting information and the object information of the objects within the range to the shooting information acquisition unit 221 and the object information acquisition unit 222 of the information processing device 2, respectively, and the determination unit provided in the participation terminal 4 transmits the determination result to the determination result acquisition unit 224 of the information processing device 2. Alternatively, the determination unit 223 may be provided in the service providing server 3 instead of in the information processing device 2. In that case, the configuration is the same as the above-described configuration in which the determination unit 223 is provided in the participation terminal 4.

[0162] However, although the functions of the search key acquisition unit 226 and the search result acquisition unit 227 can be provided in the participating terminal 4, since the participating terminal 4 is usually for personal use, taking into consideration the load on the participating terminal 4, if the information processing device 2 does not have those functions, or if those functions are provided in a redundant manner with the information processing device 2 within the information processing system 1, it is preferable for the service providing server 3 to have those functions.

[0163] Furthermore, in the information processing device 2 described above, the object information acquisition unit 222 acquires object information from the participating terminal 4, but the object information acquisition unit 222 may also acquire spatial data at the time of shooting from the participating terminal 4 (or the service providing server 3) and acquire the results by identifying objects within the range based on the camera information and the spatial data.

[0164] As described above, the information processing device 2 according to embodiment 1 includes a shooting information acquisition unit 221 that acquires shooting information including photo information relating to a virtual photo taken by a virtual camera used in a virtual reality space and space identification information that identifies the virtual reality space in which the virtual photo was taken using the virtual camera among a plurality of different virtual reality spaces, and a linking unit 225 that links the photo information with the space identification information. Therefore, the program of embodiment 1 which causes one or more computers to function as information processing device 2 can be executed by the one or more computers to take advantage of the characteristic of the virtual reality space in which the computer providing the virtual reality space knows the current state of the virtual reality space in which the participants are participating, thereby linking photo information (photo ID, etc.) related to the virtual photo with space-specific information (space data ID, etc.).

[0165] In the information processing device 2 according to embodiment 1, the shooting information acquisition unit 221 may acquire a space data ID, which is an ID unique to the three-dimensional data of the virtual reality space, as space identification information, and the linking unit 225 may link the photo information with the space data ID. Therefore, the program of embodiment 1, which causes one or more computers to function as information processing device 2, can be executed by the one or more computers to identify the virtual reality space in which the virtual photograph was taken by using the spatial data ID linked to the photograph information.

[0166] The information processing device 2 according to the first embodiment may further include a space selection screen display information acquisition unit 230 that acquires space selection screen display information for displaying a space selection screen on which one virtual reality space for participation can be selected from among a plurality of different virtual reality spaces. This space selection screen displays space recognition information that allows each of a plurality of different virtual reality spaces to be recognized, and displays a virtual photo associated with space identification information of the virtual reality space for each piece of space recognition information. In this case, the program according to the first embodiment that causes one or more computers to function as the information processing device 2 can be executed by the one or more computers to provide a user with space recognition information that allows the user to distinguish and recognize each of the virtual reality spaces, and a space selection screen on which virtual photos taken in each virtual reality space are displayed, and a technology can be provided in which the user can select one of a plurality of different virtual reality spaces while referring to the virtual photos. For the user, it is highly convenient to select the virtual reality space in which the user will participate while referring to the virtual photos. In addition, for the space creator of the virtual reality space, a highly convenient technology is provided in which the display of the virtual photos can make the user more interested in the virtual reality space created by the user.

[0167] In the information processing device 2 according to embodiment 1, the linking unit 225 may link the photo information with event information related to an event that was being held at the time the virtual photograph was taken in the virtual reality space in which the virtual photograph was taken using the virtual camera. In this case, the program of embodiment 1 which causes one or more computers to function as information processing device 2 can be executed by the one or more computers, thereby taking advantage of the characteristic of the virtual reality space in which the computer providing the virtual reality space knows what event is currently being held in which a participant is participating, and linking photo information (photo ID, etc.) related to the virtual photo, space identification information (space data ID, etc.), and event information (event ID, etc.). This makes it possible to identify, for example, from the virtual photograph at which event the virtual photograph was taken and the state of the virtual reality space in which the event was held.

[0168] In the information processing device 2 according to the first embodiment, the linking unit 225 may link the event information with the photo information, further including event information related to a related event having a specific relationship with the event. In this case, the program of embodiment 1 which causes one or more computers to function as information processing device 2 can be executed by the one or more computers, thereby taking advantage of the characteristic of the virtual reality space in which the computer providing the virtual reality space knows what event is currently being held in which the participant is participating, and in addition to linking photo information (photo ID, etc.) related to the virtual photo, space identification information (space data ID, etc.), and event information (event ID, etc.), it can also link with event information of related events that have a specific relationship with the event. This makes it possible to determine, for example, from the virtual photograph at which event the virtual photograph was taken, the state of the virtual reality space in which the event was held, and further, what related events have a specific relationship with the event.

[0169] The information processing device 2 according to the first embodiment may further include an event selection screen display information acquisition unit 231 that acquires event selection screen display information for displaying an event selection screen on which one event for participation can be selected from among a plurality of different events. This event selection screen displays event recognition information that allows each of a plurality of different events to be recognized, and displays a virtual photograph linked to the event information for each piece of event recognition information. In this case, the program according to the first embodiment that causes one or more computers to function as the information processing device 2 can be executed by the one or more computers to provide a user with event recognition information that allows the user to distinguish and recognize each event, and a space selection screen that displays virtual photos taken in the virtual reality space in which each event was held, and allows the user to select one event from among a plurality of different events while referring to the virtual photos. For the user, it is highly convenient to select an event to participate in or a virtual reality space in which an event was held while referring to the virtual photos. In addition, for the event organizer or the space creator of the virtual reality space, a highly convenient technology is provided that allows the user to become more interested in the event that the event organizer holds or the virtual reality space that the user created by displaying the virtual photos.

[0170] The information processing device 2 according to the first embodiment may further include a selection unit 232 that selects a virtual photo to be displayed on the space selection screen from among a plurality of virtual photos linked to the space identification information of the virtual reality space based on a selection condition. In this case, the program of embodiment 1 that causes one or more computers to function as information processing device 2 can be executed by the one or more computers to select from among the multiple virtual photographs those that meet the selection conditions and display them on the space selection screen. When a plurality of virtual photographs are linked to space identification information of one virtual reality space and stored in the storage unit 23, it may not be possible to display all of the virtual photographs around the space recognition information of the virtual reality space on the space selection screen. In addition, it is preferable that the virtual photograph to be displayed on the space selection screen is one that provides an incentive for the user to participate in the virtual reality space. By providing the above-mentioned selection unit 232, virtual photographs that meet the selection conditions can be selected and displayed on the space selection screen, which means that more useful virtual photographs are displayed for the user, and the space creator can provide a higher incentive to the user. For the service provider, this leads to increased use of the services related to the virtual reality space that they provide, and therefore a highly convenient technology is provided for all parties involved.

[0171] The information processing device 2 according to the first embodiment may further include an introduction screen display information acquisition unit 233 that acquires introduction screen display information for displaying an introduction screen on which one or more virtual photographs and spatial recognition information capable of recognizing the virtual reality space in which the virtual photographs were taken or event recognition information capable of recognizing an event that was being held in the virtual reality space in which the virtual photographs were taken at the time the virtual photographs were taken are displayed, and a spatial data acquisition unit 234 that, when one virtual photograph out of the one or more virtual photographs is selected on the introduction screen, identifies the virtual reality space in which the virtual photograph was taken based on the space identification information or event information linked to the selected virtual photograph, acquires spatial data that is three-dimensional data of the identified virtual reality space, and transmits the acquired spatial data to the participating terminal on which the operation of selecting the virtual photo was performed. In this case, the program of embodiment 1 that causes one or more computers to function as information processing device 2 can be executed by the one or more computers to provide a technology that allows a user to participate in a virtual reality space by selecting a virtual photo displayed on the introduction screen.

[0172] The information processing device 2 of embodiment 1, which executes the program of embodiment 1, can take advantage of a characteristic of the virtual reality space in which the state of the virtual reality space in which a participant is currently participating is understood by the computer providing the virtual reality space, and can link photo information (photo ID, etc.) related to the virtual photo with space identification information (space data ID, etc.).

[0173] The information processing method according to the first embodiment includes a step (ST1) in which the shooting information acquisition unit 221 acquires shooting information including photo information relating to a virtual photo taken by a virtual camera used in a virtual reality space and space identification information that identifies the virtual reality space in which the virtual photo was taken using the virtual camera among a plurality of different virtual reality spaces, and a step (ST2) in which the linking unit 225 links the photo information with the space identification information. As a result, the information processing method of embodiment 1 can take advantage of a characteristic of virtual reality space, in which the state of the virtual reality space in which a participant is currently participating is understood by the computer providing the virtual reality space, and can link photo information (photo ID, etc.) related to the virtual photo with space identification information (spatial data ID, etc.).

[0174] The various embodiments disclosed in this specification will be summarized below as appendices. (Appendix 1) One or more computers, a photographing information acquisition unit that acquires photographing information including camera information related to a virtual camera used in a virtual reality space and photograph information related to a virtual photograph taken by the virtual camera; an object information acquisition unit that acquires object information regarding one or more objects that are included in a shooting range of the virtual camera when the virtual photograph is taken, among one or more objects arranged in the virtual reality space; a determination result acquisition unit that acquires a determination result of whether or not each of one or more of the objects included in the shooting range is to be a subject object captured in the virtual photograph based on the camera information and the object information; and a linking unit that links the object information and the photo information based on the determination result. A program characterized by: (Appendix 2) The one or more computers, The linking unit links the object information of the object determined to be the subject object with the photograph information, and does not link the object information of a non-subject object, which is the object determined not to be the subject object, with the photograph information. Functioning as the information processing device 2. The program according to claim 1, (Appendix 3) The one or more computers, The linking unit links the object information of the object determined to be the subject object and the object information of a non-subject object, which is the object determined not to be the subject object, to the photograph information while distinguishing between the object information of the object determined to be the subject object and the non-subject object. Functioning as the information processing device 2. The program according to claim 1, (Appendix 4) The one or more computers, the determination result acquisition unit acquires the determination result as to whether or not one or more avatars as the one or more objects included in the shooting range are to be the subject object, The linking unit links the object information and the photo information related to the avatar based on the determination result. Functioning as the information processing device 2. The program according to claim 1, (Appendix 5) The one or more computers, The linking unit links the photo information with user identification information that identifies a user of the virtual reality space who took the virtual photo using the virtual camera, in addition to the object information. Functioning as the information processing device 2. The program according to claim 1, (Appendix 6) The one or more computers, The linking unit links the photo information with space specification information that specifies the virtual reality space in which the virtual photo was taken using the virtual camera, in addition to the object information. Functioning as the information processing device 2. The program according to claim 1, (Appendix 7) The one or more computers, The linking unit links, in addition to the object information, event information related to an event that was being held in the virtual reality space in which the virtual photograph was taken using the virtual camera at the time the virtual photograph was taken, with the photo information. Functioning as the information processing device 2. The program according to claim 1, (Appendix 8) The one or more computers, The linking unit links the photo information with sales information regarding the sale of the object corresponding to the object information in addition to the object information or as the object information. Functioning as the information processing device 2. The program according to claim 1, (Appendix 9) The one or more computers, The linking unit links searchability information, which indicates whether or not a search can be performed based on metadata including the object information linked to the photo information, to the photo information for each of the objects. Functioning as the information processing device 2. The program according to claim 1, (Appendix 10) The one or more computers, a search key acquisition unit that acquires the object information as a search key for searching the photo information; and a search result acquisition unit that acquires the photo information searched based on the object information. 2. The program according to claim 1, (Appendix 11) The one or more computers, a search key acquisition unit that acquires the space specification information as a search key for searching the photo information; and a search result acquisition unit that acquires the photo information searched based on the space specification information. 7. The program according to claim 6, (Appendix 12) The one or more computers, a search key acquisition unit that acquires the object information as a search key for searching the photo information; and a search result acquisition unit that acquires the photo information searched based on the object information, the photo information being linked to the searchability information indicating that the photo information is searchable. 10. The program according to claim 9, (Appendix 13) The one or more computers, and a notification unit that, when the object information of the object determined to be the subject object is linked to the photo information, notifies a notification destination specified in accordance with a predetermined notification condition of the photo information. 2. The program according to claim 1, (Appendix 14) The one or more computers, The notification unit notifies the notification destination of the photograph information by designating an owner, creator, or seller of the object specified in accordance with the notification condition as the notification destination. Functioning as the information processing device 14. The program according to claim 13, (Appendix 15) The one or more computers, and a display information generating unit that generates display information for displaying the object information of the specified object when the object is specified by an operation of specifying the object in the virtual photograph performed on the virtual photograph arranged in the virtual reality space. 2. The program according to claim 1, (Appendix 16) The one or more computers, and a display information generating unit configured to, when a subject object is designated by an operation of designating the subject object in the virtual photograph performed on the virtual photograph arranged in the virtual reality space, display the object information of the designated subject object, and to, when a non-subject object is designated by an operation of designating the non-subject object, generate display information for displaying the object information of the designated non-subject object. 4. The program according to claim 3. (Appendix 17) An information processing device that executes the program according to any one of Supplementary Note 1 to Supplementary Note 16. (Appendix 18) An information processing method performed by an information processing device, A step in which a photography information acquisition unit acquires photography information including camera information related to a virtual camera used in a virtual reality space and photo information related to a virtual photo taken by the virtual camera; an object information acquisition unit acquiring object information regarding one or more objects that are included in a shooting range of the virtual camera when the virtual photograph is taken, among one or more objects arranged in the virtual reality space; a determination result acquisition unit acquiring a determination result as to whether or not each of one or more of the objects included in the shooting range is a subject object captured in the virtual photograph based on the camera information and the object information; and a linking unit links the object information and the photo information based on the determination result. 23. An information processing method comprising:

[0175] The effects of the various embodiments shown in the appendix will be summarized below. The information processing device 2 according to the first embodiment includes a shooting information acquisition unit 221 that acquires shooting information including camera information on a virtual camera used in the virtual reality space and photo information on a virtual photo taken by the virtual camera, an object information acquisition unit 222 that acquires object information on one or more objects (in-range objects) that are included within the shooting range of the virtual camera when the virtual photo was taken, among one or more objects arranged in the virtual reality space, a judgment result acquisition unit 224 that acquires a determination result based on the camera information and the object information as to whether or not each of the one or more objects included within the shooting range is to be a subject object captured in the virtual photo, and a linking unit 225 that links the object information and the photo information based on the determination result. Therefore, the program of embodiment 1 which causes one or more computers to function as information processing device 2 can be executed by the one or more computers to take advantage of the characteristic of the virtual reality space, in which the positions of all objects placed within the virtual reality space are understood by the computer providing the virtual reality space, and link photo information (photo ID, etc.) related to the virtual photo with object information (object ID, etc.). The linked object information can be used in various ways, such as searching for a virtual photo that includes a particular object, or displaying object information for objects within a range.

[0176] In the information processing device 2 relating to embodiment 1, the linking unit 225 links object information of objects within the range that are determined to be subject objects with photograph information, and does not link object information of non-subject objects, which are objects that are determined not to be subject objects, with photograph information. The program of embodiment 1, which causes one or more computers to function as the above-mentioned information processing device 2, can be executed by the one or more computers to link information about a subject object determined by utilizing the characteristics of the virtual reality space with photographic information about the virtual photograph.

[0177] In the information processing device 2 according to embodiment 1, the linking unit 225 may distinguish between object information of objects within the range that have been determined to be subject objects and object information of non-subject objects that have been determined not to be subject objects, and link these to the photograph information. The program of embodiment 1, which causes one or more computers to function as the above-mentioned information processing device 2, can be executed by the one or more computers to link information about subject objects and non-subject objects determined by utilizing the characteristics of the virtual reality space with photographic information about the virtual photograph.

[0178] In the information processing device 2 relating to embodiment 1, the judgment result acquisition unit 224 acquires a judgment result for each of one or more avatars as one or more objects (in-range objects) included in the shooting range, in which it is determined whether or not to set each of them as a subject object, and the linking unit 225 links the object information related to the avatar with the photo information based on the judgment result. The program of embodiment 1, which causes one or more computers to function as the above-mentioned information processing device 2, can be executed by the one or more computers to link information about an avatar as a subject object determined by utilizing the characteristics of the virtual reality space with photographic information about the virtual photograph.

[0179] In information processing device 2 according to embodiment 1, linking unit 225 may link, in addition to object information, user identification information that identifies a user of the virtual reality space who took a virtual photo using a virtual camera with photo information. The program of embodiment 1 that causes one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to link information identifying a user of the virtual reality space who took a virtual photograph using a virtual camera, for example, information identifying a participant who participated in the virtual reality space as an avatar and operated the virtual camera as the avatar to take a virtual photograph (user ID), with photo information regarding the virtual photograph. The information for identifying the linked user can be used in various ways, such as searching for virtual photos taken by a specific user.

[0180] In information processing device 2 according to embodiment 1, linking unit 225 may link, in addition to object information, space specifying information that specifies a virtual reality space in which a virtual photograph was taken using a virtual camera with photo information. The program of embodiment 1 that causes one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to link space identification information (e.g., space data ID of the virtual reality space) that identifies the virtual reality space in which a virtual photograph was taken using a virtual camera with photo information related to the virtual photograph. The associated space-specific information can be used in various ways, such as searching for virtual photographs taken in a specific virtual reality space.

[0181] In the information processing device 2 according to embodiment 1, the linking unit 225 may link, in addition to the object information, the photo information with event information related to an event that was being held in the virtual reality space in which the virtual photo was taken using the virtual camera at the time the virtual photo was taken. The program of embodiment 1 that causes the one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to link event information (e.g., event ID, organizer ID, or both) related to an event that was being held at the time the virtual photograph was taken in the virtual reality space where the virtual photograph was taken using the virtual camera with photo information related to the virtual photograph.

[0182] In the information processing device 2 according to the first embodiment, the linking unit 225 may link, in addition to the object information, or as the object information, sales information regarding the sale of an object corresponding to the object information with the photo information. The program of embodiment 1 that causes one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to link sales information regarding the sale of objects with photo information regarding virtual photos. The linked sales information can be used in various ways, such as displaying sales information for objects within a range specified on the virtual photograph.

[0183] In the information processing device 2 according to the first embodiment, the linking unit 225 may link searchability information indicating whether or not a search based on metadata including object information linked with the photo information is possible to the photo information for each object within the range. The program of embodiment 1 that causes one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to link searchability information with information related to the virtual photograph. The associated searchability information can be used when searching for a virtual photo that includes a specific object. The searchability information can address a problem in which, for example, the owner of an object within the range does not want others to obtain photo information of a virtual photo that includes the object within the range through a search.

[0184] The information processing device 2 of embodiment 1 may include a search key acquisition unit 226 that acquires object information as a search key for searching photo information, and a search result acquisition unit 227 that acquires photo information searched based on the object information. The program according to the first embodiment, which causes one or more computers to function as the information processing device 2, can be executed by the one or more computers to search for photo information based on a search key and obtain search results. These functions can be used to provide users with a virtual photo search service, etc.

[0185] The information processing device 2 according to embodiment 1 may include a search key acquisition unit 226 that acquires space specific information as a search key for searching for photo information, and a search result acquisition unit 227 that acquires photo information searched based on the space specific information. The program according to the first embodiment, which causes one or more computers to function as the information processing device 2, can be executed by the one or more computers to search for photo information using space-specific information (e.g., space data ID) as a search key and obtain search results. These functions can be used to provide users with a search service for virtual photos using the space-specific information.

[0186] The information processing device 2 according to embodiment 1 may include a search key acquisition unit 226 that acquires object information as a search key for searching for photo information, and a search result acquisition unit 227 that acquires photo information searched based on the object information and linked with searchability information indicating that the photo information is searchable. The program of embodiment 1, which causes one or more computers to function as the above-mentioned information processing device 2, is executed by the one or more computers, so that when searching for photo information based on a search key, only photo information that is linked with searchability information indicating that it is searchable can be obtained. This enables the program to address problems, for example, when the owner of an within-range object does not want others to obtain photo information of a virtual photo that includes the within-range object through a search.

[0187] The information processing device 2 according to the first embodiment may also include a notification unit 228 that, when object information of an object determined to be a subject object is linked to photographic information, notifies a notification destination identified in accordance with predetermined notification conditions of the photographic information. The program according to the first embodiment, which causes one or more computers to function as the information processing device 2, can be executed by the one or more computers to notify a notification destination specified according to a notification condition when object information is linked to photo information. This allows a notification destination, for example, a user, to know that a virtual photo related to him or her has been taken.

[0188] In the information processing device 2 according to the first embodiment, the notification unit 228 notifies the notification destination of the photograph information, specifying the owner, creator, or seller of the subject object specified according to the notification condition as the notification destination. The program according to the first embodiment, which causes one or more computers to function as the information processing device 2, can be executed by the one or more computers to notify the owner, creator, or seller of the object when the object information is linked to the photograph information, thereby enabling these persons to know, for example, that a virtual photograph related to them has been taken.

[0189] The information processing device 2 according to the first embodiment may include a display information generation unit 229 which, when a subject object is designated by an operation of designating a subject object in a virtual photograph, which is performed on a virtual photograph placed in a virtual reality space, generates display information for displaying object information of the designated subject object. The program of embodiment 1 that causes one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to, for example, display object information of a subject object on the display of the participation terminal 4 when a participant who participates in a virtual reality space using the participation terminal 4 specifies a subject object in a virtual photograph.

[0190] The information processing device 2 according to the first embodiment may also include a display information generation unit 229 that, when a subject object is designated by an operation for designating a subject object in a virtual photograph performed on a virtual photograph placed in a virtual reality space, displays object information of the designated subject object, and, when a non-subject object is designated by an operation for designating a non-subject object, generates display information for displaying the object information of the designated non-subject object. The program of embodiment 1 that causes one or more computers to function as the above-mentioned information processing device 2 can be executed by the one or more computers to, for example, display object information of a participant that participates in a virtual reality space using a participation terminal 4 on the display of the participation terminal 4, regardless of whether the participant specifies a subject object in a virtual photograph or a non-subject object in the virtual photograph.

[0191] The information processing device 2 according to the first embodiment, which executes the program according to the first embodiment, can link photo information (photo ID, etc.) related to a virtual photo with object information (object ID, etc.) by taking advantage of the characteristic of the virtual reality space that the positions of all objects arranged in the virtual reality space are grasped by the computer that provides the virtual reality space. The linked object information can be used in various aspects, such as searching for a virtual photo in which a specific object appears, or displaying object information of objects within a range.

[0192] The information processing method according to the first embodiment includes a step in which a shooting information acquisition unit 221 acquires shooting information including camera information on a virtual camera used in the virtual reality space and photo information on a virtual photograph taken by the virtual camera; a step in which an object information acquisition unit 222 acquires, based on the camera information, object information on one or more objects (in-range objects) among one or more objects arranged in the virtual reality space and which are included in the shooting range of the virtual camera when the virtual photograph was taken; a step in which a judgment result acquisition unit 224 acquires, based on the object information, a judgment result as to whether or not each of one or more objects included in the shooting range is to be a subject object captured in the virtual photograph; and a step in which a linking unit 225 links the object information and the photo information based on the judgment result. As a result, the information processing method according to the first embodiment can link photo information (photo ID, etc.) related to a virtual photo with object information (object ID, etc.) by taking advantage of the characteristics of the virtual reality space that the positions of all objects arranged in the virtual reality space are known by the computer that provides the virtual reality space. The linked object information can be used in various aspects, such as searching for a virtual photo that includes a specific object, or displaying object information of objects within a range. [Explanation of symbols]

[0193] 1 Information processing system, 2 Information processing device, 3 Service providing server, 4 Participating terminal, 5 Network, 21 Communication unit, 22 Calculation unit, 23 Storage unit, 100 Communication interface, 101 Input / output interface, 102 Processor, 103 Memory, 221 Shooting information acquisition unit, 222 Object information acquisition unit, 223 Judgment unit, 224 Judgment result acquisition unit, 225 Linking unit, 226 Search key acquisition unit, 227 Search result acquisition unit, 228 Notification unit, 229 Display information generation unit, 230 Space selection screen display information acquisition unit, 231 Event selection screen display information acquisition unit, 232 Selection unit, 233 Introduction screen display information acquisition unit, 234 Space data acquisition unit, A1 to A4 Avatar, C1 Tree object, C2 to C3 Baggage object, VC Virtual camera, VP Virtual photo, SL1 to SL2 Designation pointer, OF1 to OF2 Speech bubble

Claims

1. One or more computers, a photographing information acquisition unit that acquires photographing information including photographic information on a virtual photograph taken by a virtual camera used in a virtual reality space and space identification information that identifies the virtual reality space in which the virtual photograph was taken using the virtual camera among a plurality of virtual reality spaces different from each other; and a linking unit that links the photograph information and the space specification information. A program characterized by:

2. The one or more computers, the photographing information acquisition unit acquires, as the space identification information, a space data ID that is an ID unique to the three-dimensional data of the virtual reality space; The linking unit links the photo information with the spatial data ID. Functioning as the information processing device 2. The program according to claim 1 .

3. The one or more computers, Further, a space selection screen display information acquisition unit is provided that acquires space selection screen display information for displaying a space selection screen on which one of the plurality of virtual reality spaces different from one another can be selected for participation; The space selection screen displays space recognition information capable of recognizing each of the plurality of virtual reality spaces different from one another, and displays the virtual photograph associated with the space specification information of the virtual reality space for each of the space recognition information. Functioning as the information processing device 2. The program according to claim 1 .

4. The one or more computers, The linking unit links the photo information with event information related to an event that was being held in the virtual reality space in which the virtual photo was taken using the virtual camera at the time the virtual photo was taken. Functioning as the information processing device 2. The program according to claim 1 .

5. The one or more computers, The linking unit links the event information with the photo information, further including event information related to a related event having a specific relationship with the event. Functioning as the information processing device 5. The program according to claim 4.

6. The one or more computers, The event selection screen display information acquisition unit is configured to acquire event selection screen display information for displaying an event selection screen on which one of the plurality of events different from one another can be selected for participation; The event selection screen displays event recognition information capable of recognizing each of the plurality of events different from one another, and displays the virtual photograph associated with the event information for each of the event recognition information. Functioning as the information processing device 5. The program according to claim 4.

7. The one or more computers, Further, a selection unit is provided that selects, based on a selection condition, the virtual photo displayed on the space selection screen from among the plurality of virtual photos associated with the space identification information of the virtual reality space. Functioning as the information processing device 4. The program according to claim 3.

8. The one or more computers, an introduction screen display information acquisition unit that acquires introduction screen display information for displaying an introduction screen on which one or more of the virtual photographs and space recognition information capable of recognizing the virtual reality space in which the virtual photographs were taken or event recognition information capable of recognizing an event that was being held in the virtual reality space in which the virtual photographs were taken at the time the virtual photographs were taken are displayed; and and a spatial data acquisition unit that, when one of the one or more virtual photographs is selected on the introduction screen, identifies the virtual reality space in which the virtual photograph was taken based on the space identification information or the event information associated with the selected virtual photograph, acquires spatial data that is three-dimensional data of the identified virtual reality space, and transmits the acquired spatial data to a participating terminal on which an operation for selecting the virtual photograph has been performed. Functioning as the information processing device 5. The program according to claim 4.

9. An information processing device that executes the program according to any one of claims 1 to 8.

10. An information processing method performed by an information processing device, A step in which a photographing information acquisition unit acquires photographing information including photographic information on a virtual photograph taken by a virtual camera used in a virtual reality space, and space identification information that identifies the virtual reality space in which the virtual photograph was taken using the virtual camera among a plurality of virtual reality spaces different from each other; A linking unit links the photograph information with the space specification information.

23. An information processing method comprising:

Images