Program, information processing device, and information processing method

Abstract

To provide a program, an information processing device, and an information processing method that can easily construct a virtual store in a virtual space and can prevent unauthorized use of image data of an existing building.SOLUTION: A program causes a computer to realize: an image acquisition function of acquiring image data showing an image regarding a building of a first enterprise in a real space; an enterprise acquisition function of acquiring first enterprise information regarding the first enterprise and including location information of the building, from at least one of a second enterprise device of a second enterprise that does business with the first enterprise and a public institution device of a public institution to which the first enterprise information is registered; and an output function of causing a user device of a user to output a virtual space based on a map of the real space, and performing output so that a virtual building corresponding to the building is arranged in the virtual space on the basis of the image data and the first enterprise information.SELECTED DRAWING: Figure 7

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , ,

[0003] , , , , , ,

[0001] The present invention relates to a program, an information processing apparatus, and an information processing method.

Background Art

[0002] Conventionally, for a virtual building (hereinafter also referred to as "virtual building") of a business operator in a virtual space such as a virtual store, a technique of outputting using image data obtained by photographing an actual building corresponding to this virtual building is known (for example, Patent Document 1 below). According to such a technique, it is possible to easily construct a virtual store in a virtual space.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In using image data obtained by photographing an actual building, there is a possibility of unauthorized use by a third party, such as the third party using this image data for their own virtual store without permission. However, with the above conventional technology, it is difficult to prevent such unauthorized use.

[0005] Therefore, an object of the present invention is to provide a program, an information processing apparatus, and an information processing method that can easily construct a virtual store in a virtual space and can prevent unauthorized use of image data of an actual building.

Means for Solving the Problems

[0006] A program according to one aspect of the present invention provides a computer with an image acquisition function that acquires image data showing images of a building of a first business operator in real space; a business acquisition function that acquires first business operator information relating to the first business operator, including location information of the building, from at least one of a second business operator device of a second business operator with which the first business operator does business, and a public institution device of a public institution where the first business operator information is registered; and an output function that causes a user's device to output a virtual space based on a map of real space, and outputs a virtual building corresponding to a building in the virtual space based on the image data and the first business operator information.

[0007] An information processing device according to one aspect of the present invention includes: an image acquisition unit that acquires image data showing an image of a building of a first business operator in real space; a business operator acquisition unit that acquires first business operator information relating to the first business operator, including location information of the building, from at least one of a second business operator's device of a second business operator with which the first business operator does business, and a public institution device of a public institution where the first business operator information is registered; and an output function that causes a user's device to output a virtual space based on a map of real space, and outputs a virtual building corresponding to a building in the virtual space based on the image data and the first business operator information.

[0008] An information processing method according to one aspect of the present invention involves a computer acquiring image data showing an image of a building of a first business operator in real space, acquiring first business operator information relating to the first business operator, including location information of the building, from at least one of a second business operator's device of a second business operator with which the first business operator does business, and a public institution device of a public institution where the first business operator information is registered, and outputting a virtual space based on a map of real space to a user's device, wherein the output arranges a virtual building corresponding to the building in the virtual space based on the image data and the first business operator information. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a program, an information processing device, and an information processing method that enable the easy construction of virtual stores in a virtual space and deter the unauthorized use of image data of real-world buildings. [Brief explanation of the drawing]

[0010] [Figure 1] This diagram illustrates an example of the system configuration of the VR system according to this embodiment. [Figure 2] This is a diagram illustrating the overview of the VR system according to this embodiment. [Figure 3] This is a diagram illustrating the overview of the VR system according to this embodiment. [Figure 4] This is a diagram illustrating the overview of the VR system according to this embodiment. [Figure 5] This is a diagram illustrating the overview of the VR system according to this embodiment. [Figure 6] This is a diagram illustrating the overview of the VR system according to this embodiment. [Figure 7] This figure shows an example of the functional configuration of the server device according to this embodiment. [Figure 8] This figure shows an example of the operation of the server device according to this embodiment. [Figure 9] This figure shows an example of the hardware configuration of the server device according to this embodiment. [Modes for carrying out the invention]

[0011] A preferred embodiment of the present invention (hereinafter referred to as "this embodiment") will be described with reference to the attached drawings. In each drawing, components denoted by the same reference numerals have the same or similar configuration.

[0012] In the present invention, "part," "means," "apparatus," or "system" does not merely mean physical means, but also includes cases where the functions of such "part," "means," "apparatus," or "system" are realized by software. Furthermore, even if the functions of one "part," "means," "apparatus," or "system" are realized by a combination of two or more physical means, devices, or software modules, the functions of two or more "parts," "means," "apparatus," or "systems" may be realized by a single physical means, device, or software module.

[0013] <1. System Configuration> Referring to Figure 1, an example of the system configuration of the VR system 1 according to this embodiment will be described. The VR system 1 is a system for providing a virtual space to the user using VR (Virtual Reality) technology.

[0014] VR System 1 makes it possible to easily reproduce buildings such as stores in the real world in a virtual space based on the real world. Specifically, VR System 1 can use image data (hereinafter also referred to as "image data") that shows images of a building owned by a first business operator in the real world (hereinafter also simply referred to as "building") to construct a virtual building corresponding to this building in the virtual space. Furthermore, this virtual space based on the real world may be generated, for example, from 3D map data generated from map information of the real world.

[0015] Images relating to the building of the first business operator (hereinafter also referred to as "actual building images") may include still images or moving images (hereinafter simply referred to as "images") such as photographs of the building of the first business operator. Actual building images may also be photographs of the interior or exterior of the building. Furthermore, if the building is a store, actual building images may also be photographs of the staff and / or owner, or the displayed goods.

[0016] The image data may include two-dimensional or three-dimensional images. The image data may also include information attached to these images (hereinafter also referred to as "attached information"). The attached information may include, for example, shooting date and time information indicating the date and time when the image was shot (hereinafter also referred to as "shooting date and time"), and / or shooting position information indicating the position where the image was shot (hereinafter also referred to as "shooting position"). Further, the attached information may include, for example, information regarding the image quality in the case of image data. The information regarding the image quality may be, for example, the color tone or brightness of the image. The attached information may also include comment information indicating comments (including annotations) attached to the image data. This comment may be a comment introducing the store if the building is a store.

[0017] As shown in FIG. 1, the VR system 1 includes a server device 100 that provides a virtual space, a store user device 200 used by a user (hereinafter also referred to as a "store user") such as a store clerk or a store owner in the store of the first operator, a customer user device 300 used by a user (hereinafter also referred to as a "customer user") who uses the virtual store (a form of virtual building) of the first operator in the virtual space as a customer, and a second operator device 400 used by the second operator. The store user device 200 is a form of the first operator device of the first operator. The server device 100, the store user device 200, the customer user device 300, and the second operator device 400 are connected to each other via the network N. Note that the store user device 200 and the customer user device 300 are also collectively referred to as "user devices" when there is no particular need to distinguish them.

[0018] The second operator is an operator that has transactions with the first operator. The second operator may be, for example, a settlement operator (such as an acquirer or a settlement agency company) or a financial institution (such as a bank) that has transactions with the first operator.

[0019] Network N consists of wireless or wired networks. Examples of networks include mobile phone networks or PHS (Personal Handy-phone System) networks, wireless LAN (Local Area Network, including communication compliant with IEEE 802.11 (so-called Wi-Fi®)), 3G (3rd Generation), LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation), WiMAX®, infrared communication, visible light communication, Bluetooth®, wired LAN, telephone lines, power line communication, power line networks, and networks compliant with IEEE 1394, etc.

[0020] [Server equipment] The server device 100 is an information processing device capable of communicating with the user device, the second operator device 400, and the external system 500. The server device 100 is a device used, for example, by the operator of the VR system 1 (hereinafter also simply referred to as the "operator"). The server device 100 generates or processes a virtual space based on material data including image data by executing a predetermined program, and outputs the generated or processed virtual space to the user device. In this virtual space, multiplayer by multiple users may be possible, and it may be a so-called metaverse. This multiplayer may also be multiplayer in a virtual space realized on a cloud including the server device 100 (for example, cloud games). In the case of a virtual space realized on a cloud, the server device 100 may be realized by streaming distribution technology, as in so-called cloud games. With such a configuration, the user device can use the virtual space via a general-purpose viewer (for example, a web browser).

[0021] [User device] The user device is, for example, an information processing device such as a smartphone or laptop or a head-mounted display, which can output a virtual space to the user, accept requests to process the outputted virtual space, and communicate with the server device 100.

[0022] The user device may have, for example, a pre-defined application program (hereinafter referred to as "VR app") specifically for the VR system 1 installed. The VR app may be a native app that provides each user with a user interface for using the VR system 1. The user device installs and uses the VR app from an application store or server device 100, etc.

[0023] The specified program may be, for example, a VR application, a web browser, or a third-party application implemented using a white label provided for VR system 1. Furthermore, for example, VR system 1 may allow both VR applications and web browsers to be used as user interfaces.

[0024] [Second operator equipment] The second business operator device 400 is an information processing device capable of communicating with the server device 100 and the external system 500. The second business operator device 400 registers and manages information relating to the first business operator (hereinafter also referred to as "first business operator information") in its own storage unit for transactions with the first business operator.

[0025] The first business operator information may include, for example, business operator identification information for identifying the first business operator, and store information for one or more stores owned, operated, or franchised by the first business operator. The store information may include, for example, store identification information for identifying each store, store name, store location information (e.g., store address or store latitude and longitude information), and / or information about the store owner.

[0026] The first business operator information may include, for example, if the first business operator is a corporation, the corporate registration number, corporate name, trade name, address of the head office or branch office, contact information for the head office or branch office, representative, officers, and / or certificate information relating to the corporation.

[0027] [External System] The external system 500 is a so-called third-party system. The external system 500 may be a system that provides external information, such as information about the environment of real space or virtual space (hereinafter also referred to as "environmental information"). The external system 500 may be a system that provides, for example, an online storage service for managing user image data, an SNS service where user image data is registered, or a homepage / blog creation service where user image data is registered.

[0028] Environmental information is, for example, information about the surrounding environment of a specific area and period (or a specific location and time) in real or virtual space. This specific period may be, for example, a specific day, month, or year, or a specific decade. Environmental information may include, for example, weather information around a specific location, map information, urban information for the surrounding city (including, for example, image data or CAD data of structures, buildings, or structures in the city), and / or information on events held.

[0029] Environmental information may include, for example, information about a specific region and a specific time period in real or virtual space. This information about a time period may include, for example, images such as photographs or videos of people or landscapes in the area surrounding the building of the first business operator during a specific time period. This information about a time period may also include, for example, information about events that occurred in the area surrounding the building of the first business operator during a specific time period (so-called Tips information).

[0030] The external system 500 may be, for example, a system that provides a function (hereinafter also referred to as "online wallet") for storing and controlling account information for one or more payment methods available to customer users in an online environment. These payment methods may include, for example, card payments such as credit or debit cards, bank transfers, and cryptocurrency payments. Customer users can access this online wallet in the virtual space and purchase goods from the virtual store by specifying one or more payment methods stored in this online wallet.

[0031] <2. Overview> An overview of the VR system 1 according to this embodiment will be described with reference to Figures 2-6.

[0032] <2-1. Building a virtual store using images of a physical store> Referring to Figure 2, an example of constructing a virtual store using images of a physical store provided by VR System 1 will be explained. In this example, the image data will be photographic images D taken by a store user of their own store in real space (hereinafter also referred to as "physical store"). In this example, the first business operator will be Company A (indicated as "○○ Corporation" in Figure 2), the building owned by Company A will be store a, which is a physical store owned by Company A, and the address of store a will be Minami Aoyama XX-chome, Minato-ku, Tokyo.

[0033] (1) As shown in Figure 2, a store user of store a takes a photograph of store a in the real world using the store user device 200. (2) The store user device 200 uploads this photographic image D to the server device 100. The server device 100 retrieves this uploaded photographic image D.

[0034] (3) The server device 100 obtains first business information, including store information for store a, from the second business device 400 of a second business that does business with company A. (4) The server device 100 compares the obtained store information with the photographic image D. For example, in order to verify whether the photographic image D is an image of company A's physical store, the server device 100 checks whether there is a predetermined relationship between the shooting location, which is supplementary information of the photographic image D, and the address of store a indicated by the store information, and whether there is a predetermined relationship between them, and whether the shooting date and time, which is supplementary information of the photographic image D, is on or after the opening date of store a.

[0035] (5) The server device 100 causes the store user device 200 and the customer user device 300 to output a virtual space S based on a map of the real world. If the server device 100 determines, as a result of the matching in (4) above, that the photographic image D is an image of company A's physical store, it outputs a virtual store corresponding to store a in the virtual space S based on the acquired image data. Specifically, the server device 100 constructs at least a part of the virtual store (in this example, the front) with this image data. For example, the server device 100 may analyze the image data to generate a polygon of the virtual store, or if the image data only represents a two-dimensional planar image of store a, it may construct this planar image as the texture of the virtual store. The server device 100 may also place the virtual store in the virtual space based on the store information of store a. Specifically, it places the virtual store at a location corresponding to the address of store a on the map of the virtual space (in this example, Minami Aoyama, Minato Ward, Tokyo, XX-chome, XX-banchi).

[0036] Under the above configuration, VR system 1 can place a virtual store of store a in the virtual space at a location corresponding to the actual address of store a. Furthermore, since this actual address information is obtained from a second business operator, its reliability can be ensured. Therefore, for example, it is possible to prevent store employees from falsifying store a's information to suit their own convenience. In addition, VR system 1 can compare the actual address of store a with the shooting location to verify that there are no inconsistencies between the two. Furthermore, VR system 1 can similarly compare the opening date of store a with the shooting date and time to verify that there are no inconsistencies between the two. Thus, VR system 1 makes it possible to easily construct a virtual store in the virtual space and can deter the misuse of image data of real buildings.

[0037] <2-2. Overall picture of virtual store generation and output> Referring to Figure 3, an example of the overall structure of generating and outputting virtual stores in a virtual space in VR System 1 is explained.

[0038] <Store Inquiry / Registration> (1) As shown in Figure 3, a store user of a store operated by the first business operator (hereinafter also simply referred to as "store") requests registration of the corresponding store from the store user device 200 in order to open their own virtual store in the virtual space. (2) The server device 100 queries the second business operator device 400a of second business operator A and the second business operator device 400b of second business operator B, which have dealings with the store or the first business operator, for this store. In this example, second business operator A is the credit card company that customers can use at the store, and second business operator B is the financial institution that provides the payment methods that can be used at the store. (3) If the server device 100 can confirm that this store actually exists as requested as a result of the query, it registers the store. (4) The server device 100 obtains first business operator information from second business operator device 400a and / or second business operator device 400b in order to collect information on the first business operator of the registered store. Furthermore, the server device 100 may acquire payment history information from the second business device 400a and / or the second business device 400b as customer user behavior history information, which shows the customer user's payment history using the payment methods provided by the first business A and the second business B, respectively.

[0039] <Generation / Processing> (5a) The server device 100 receives a request from the store user device 200 to generate a virtual store, which includes image data for generating the virtual store. (6a) When the server device 100 receives the generation request, it generates a virtual store in the virtual space based on the acquired image data and the acquired first business information. (5b) When the server device 100 processes the virtual store using material data, it receives a request from the store user device 200 to process the virtual store, which includes material data. This material data may be provided by the server device 100 free of charge or for a fee, in which case the processing request may include a specification of the material data to be used for processing, rather than including the material data itself. (6b) When the server device 100 receives the processing request, it processes the virtual store based on the acquired material data (or the specified material data).

[0040] <Output / VR Commerce> (7) The server device 100 receives an output request from the customer user device 300 to output a virtual store. (8) Upon receiving this request, the server device 100 analyzes the customer user's payment trends, i.e., product purchase trends, based on the customer user's payment history information, and based on the results of the analysis, changes the output format from the default and causes the customer user device 300 to output the virtual space. With this configuration, the server device 100 can create a virtual space that suits the customer user's preferences, taking into account the customer user's product purchase trends. (9) In the virtual store in the virtual space, the server device 100 receives a product purchase request from the customer user device 300. (10)(11) Based on this purchase request, the server device 100 instructs the second business device 400a or the second business device 400b to make a payment for the product purchase via the online wallet provided by the external system 500. (12) The second business operator's device 400a or the second business operator's device 400b receives the payment instruction and executes the payment.

[0041] Under the above configuration, VR System 1 can easily construct virtual stores in a virtual space based on information about the first business operator and image data of the stores obtained from the first business operator A and the second business operator B, and allow them to open stores while ensuring the reliability of the information.

[0042] <2-3. Processing Flow for New Store Registration> Referring to Figure 4, an example of the process in the new store account registration process performed by the entire VR system 1 is illustrated. In the example in Figures 4-6, the first business operator is Company B, and the building of the first business operator is Store b, which is operated by Company B.

[0043] (1) As shown in Figure 4, the store user requests the server device 100 from the store user device 200a to register a new account for store b in order to open their store b in the virtual space (including store information). The acquisition unit 111 of the server device 100 acquires this registration request. (2) The inquiry unit 114 of the server device 100 inquires with the second business device 400 of the second business operator about store b for which registration has been requested. Specifically, the inquiry unit 114 asks the second business device 400 whether store b actually exists as requested. The inquiry unit 114 acquires a response to this inquiry (in this example, the result is that store b actually exists). The business operator acquisition unit 111b acquires the first business operator information of company B and store information of store b (for example, location information and / or opening information of store b) included in this response. The registration unit 115 of the server device 100 registers the acquired first business information of company B and the store information of store b in the storage unit 130 in order to register a new account for store b.

[0044] <2-4. Flowchart for registering material data for virtual stores and generating / processing virtual store data> Referring to Figure 5, an example of the processing in the registration of material data for virtual stores and the generation and processing of virtual store data performed by the entire VR system 1 is explained. Virtual store data is data used to generate a virtual store corresponding to store b in the virtual space.

[0045] (1) As shown in Figure 5, the store user requests login to the registered account from the store user device 200a. (2) When the authentication unit (not shown) of the server device 100 receives this login request, it performs account authentication.

[0046] (3) If the account authentication in (2) above is successful, i.e., the user logs in, the store user sends image data of the store from the store user device 200, i.e., uploads it to the server device 100. The image acquisition unit 111a of the server device 100 acquires the image data sent from the store user device 200. (4) When the image data is acquired, the business acquisition unit 111b of the server device 100 may reacquire the latest store information of store b and the latest first business information of company B from the second business device 400. (5) The matching unit 113 of the server device 100 compares the image data with the store information to confirm that there is no problem in using the image data for the virtual store of store b. (6) When the matching unit 113 confirms that there is no problem, the registration unit 115 of the server device 100 registers the image data in the storage unit 130 in association with the store information of store b.

[0047] (7) A store user requests the creation or processing of a virtual store from the store user device 200 based on the material data including the registered image data. The acquisition unit 111 of the server device 100 acquires this request. (8) The business acquisition unit 111b of the server device 100 may acquire material data to be used for creating or processing a virtual store from an external system 500. Specifically, if the external system 500 is a system that provides an online storage service for storing image data related to stores, the business acquisition unit 111b of the server device 100 may acquire the stored image data related to stores. (9) The generation and processing unit 117a of the server device 100 refers to the storage unit 130 which stores the material data including the acquired image data related to stores in response to the generation or processing request. (10) The generation and processing unit 117a generates or processes virtual store data for generating a virtual store in the virtual space based on this material data. (11) The generation and processing unit 117a stores the generated or processed virtual store data in the storage unit 130.

[0048] <2-5. Processing flow for output and product purchase in virtual stores> Referring to Figure 6, an example of the processing in the output and product purchase process of the virtual store performed by the entire VR system 1 is illustrated.

[0049] (1) As shown in Figure 6, the customer user requests the output of a virtual space from the customer user device 300. The acquisition unit 111 of the server device 100 acquires this output request. (3) In response to this request, the output unit 117 of the server device 100 causes the customer user device 300 to output a virtual space in which a virtual store is located, based on the virtual space data (including virtual store data and store information) for generating the virtual space.

[0050] (4) A customer user requests to purchase an item by specifying the icon of the item they wish to buy from among the icons of one or more items listed in the virtual store. This item icon is one form of input means for receiving purchase requests from customer users (hereinafter also referred to as "purchase request input means"). The acquisition unit 111 of the server device 100 acquires this purchase request for an item. (5) The purchase processing unit 118 of the server device 100 performs processing for purchasing the requested item (hereinafter also referred to as "purchase processing") based on this purchase request for an item. (6) The output unit 117 of the server device 100 may, for example, output a virtual space where the virtual store is located to a store user device 200b installed in a store. A customer user or store user may request to purchase an item by specifying the icon of the item they wish to buy from among the icons of one or more items listed in the virtual store output to this store user device 200b. (5) The purchase processing unit 118 of the server device 100 performs purchase processing for the requested item based on this purchase request for an item.

[0051] <3. Functional Configuration> Referring to Figure 7, the functional configuration of the server device 100 according to this embodiment will be described. As shown in Figure 7, the server device 100 comprises a control unit 110, a communication unit 120, and a storage unit 130.

[0052] The control unit 110 includes an acquisition unit 111 and an output unit 117. The control unit 110 may also include, for example, an extraction unit 112, a matching unit 113, an inquiry unit 114, a registration unit 115, a determination unit 116, an output unit 117, and / or a purchase processing unit 118.

[0053] [Acquisition Department] The acquisition unit 111 acquires various requests and / or various information from the user's equipment, the second operator's equipment 400, or an external system 500.

[0054] The acquisition unit 111 includes an image acquisition unit 111a. The image acquisition unit 111a acquires image data showing images of the building of the first business operator in real space from a user device, an external system 500, or a storage unit 130. The building of the first business operator may be the head office or branch office of the first business operator, in addition to the stores shown in the examples in Figures 2 to 6 above. The image acquisition unit 111a may store the acquired image data in the storage unit 130 as material data for a virtual store.

[0055] The acquisition unit 111 includes a business operator acquisition unit 111b. The business operator acquisition unit 111b acquires first business operator information from at least one of the second business operator device 400 of the second business operator and the public institution device of a public institution where the business operator information is registered. The first business operator information includes, for example, location information of the first business operator's building. This public institution may be, for example, a legal affairs bureau where the real estate registration information of the first business operator's building is registered, or a government agency where government registration information such as G-BizID is registered.

[0056] The acquisition unit 111 may acquire environmental information relating to the real space from, for example, the storage unit 130 or an external system 500. The acquisition unit 111 may, for example, search for environmental information stored in the storage unit 130 using the date and time of shooting and / or the shooting location included in the supplementary information of the image data as search keys, and acquire environmental information that matches, i.e., environmental information in which the date and time of shooting and a specific period (or specific time), and / or the shooting location and a specific region (or specific point) are within a predetermined range.

[0057] The acquisition unit 111 may include, for example, a first user acquisition unit 111c. The first user acquisition unit 111c acquires information indicating the access environment of the user device (hereinafter also referred to as "access environment information") from the user device. This access environment information may include, for example, access location information at the time of access to the virtual space of the customer user device 300, or access date and time information indicating the date and time of access.

[0058] The first user acquisition unit 111c may acquire, for example, information indicating the customer user's activity history (hereinafter also referred to as "activity history information"). This activity history information may include, for example, the date and time of the activity, the location of the activity, the history of operations on the customer user device 300 associated with the activity, and the content of the activity, as a history of the customer user's activities. More specifically, if the customer user's activity is a trip from one year ago, the activity history information may include, for example, the period of the trip, the location of the trip, the history of operations on map applications and / or travel websites associated with the trip, and the itinerary of the trip. The first user acquisition unit 111c may also acquire, for example, settlement history information indicating the settlement history for a predetermined period from the user device or the second business device 400. In the case of settlement of a sales transaction for goods, the settlement history information may include, for example, product identification information to identify the goods sold, the number of goods sold, the price of the goods sold, the type of goods sold, and the date and time the sales transaction was settled.

[0059] The acquisition unit 111 may include, for example, a second user acquisition unit 111d. The second user acquisition unit 111d acquires, for example, user location information indicating the user's location from a user device or an external system 500 (for example, a GPS system).

[0060] [Extraction part] The extraction unit 112 extracts a specific image from image data of an actual building.

[0061] The extraction unit 112 includes a store employee extraction unit 112a. If the building is a store of the first business operator and the virtual building is a virtual store of the first business operator, the store employee extraction unit 112a extracts store employee images of the store of the first business operator from the image data of the store. The store employee images may be, for example, images of only a part of the store employee's body (for example, only the face or only the feet).

[0062] The extraction unit 112 includes a product extraction unit 112b. If the building is a store of the first business operator and the virtual building is a virtual store of the first business operator, the product extraction unit 112b extracts product images of one or more products displayed in the store from the image data of the store.

[0063] [Verification Unit] The matching unit 113 compares image data of existing buildings with various other information.

[0064] The matching unit 113 includes, for example, a location matching unit 113a. The location matching unit 113a matches the location information contained in the first business operator information with the shooting location information contained in the image data of the first business operator's building. Specifically, the matching unit 113 may match whether the location and shooting location of the first business operator's building have a predetermined relationship. This predetermined relationship may be, for example, a relationship in which the shooting location is included within a predetermined range from the location of the building.

[0065] The matching unit 113 includes, for example, a product matching unit 113b. The product matching unit 113b refers to a product storage unit that stores product information relating to products sold by the business operator and matches the product information with the product image extracted by the product extraction unit. The matching unit 113 may, for example, match the product image with a sample image of the product included in the product information, and if the matching result indicates that the similarity between them is above a predetermined threshold, it may determine that they are the same product or similar products. As another example, the matching unit 113 may extract part or all of the product name from the product image, and match part or all of the extracted product name with the product name included in the product information, and if the matching result indicates that the similarity between them is above a predetermined threshold, it may determine that they are the same product or similar products.

[0066] Product information may include, for example, product identification information to identify each product (e.g., GTIN (JAN code)), product name, product price, sample image of the product, product ingredients, product size, product type, and / or business identification information to identify the first business that sells the product.

[0067] [Inquiries Department] The inquiry unit 114 inquires with the second business operator's equipment 400 of the second business operator about the first business operator or the first business operator's store that has requested new account registration. Specifically, the inquiry unit 114 asks the second business operator's equipment 400 whether the first business operator or store actually exists as requested for registration. The inquiry unit 114 obtains a response to this inquiry.

[0068] [Registration Department] If the result of the inquiry by the inquiry unit 114 indicates that the first business operator or store actually exists as requested for registration, the registration unit 115 causes the first business operator information or store information to be registered in the storage unit 130 in order to register a new account for the first business operator or the first business operator's store.

[0069] [Judgment section] The determination unit 116 determines whether the user is within a predetermined range from the building's location based on the user's location information and the building's location information.

[0070] [Output section] The output unit 117 causes the user device to output various information. For example, the output unit 117 may generate output information for outputting the virtual space as a screen and sound based on virtual space data, and transmit this output information to the user device. For example, if the server device 100 renders an image of the virtual space, the output unit 117 may output this rendered image. Alternatively, if the VR application on the user device renders the image, the output unit 117 may output parameters for rendering in the VR application.

[0071] The output unit 117 causes the user device to output a virtual space based on a map of the real world. The output unit 117 also outputs virtual buildings corresponding to the buildings of the first business operator, placed within the virtual space, based on image data of the physical store and information of the first business operator obtained from the second business operator device 400 or a public institution device.

[0072] For example, a first business operator might illegally photograph a real building belonging to a third party and use it for their own virtual building. With the above configuration, the output unit 117 can ensure the reliability of the image data based on first business operator information obtained from a second business operator or public institution, and use this image data to easily output the virtual store into the virtual space. Therefore, it is possible to easily construct a virtual store in the virtual space and deter the illegal use of image data of real buildings.

[0073] The output unit 117 may, for example, generate a virtual building based on image data of a building if the result of the position matching unit 113a indicates that there is a predetermined relationship between the shooting location and the location of the building of the first business operator. Specifically, the output unit 117 may generate at least some of the polygons and / or textures of the virtual building using the image data of this building. In this case, the output unit 117 may also use polygon data and / or texture data of a general-purpose model of the virtual building as appropriate. On the other hand, if the result of the position matching unit 113a indicates that there is no predetermined relationship between the shooting location and the location of the building of the first business operator, the output unit 117 may generate a virtual building without using image data of this building. Specifically, the output unit 117 does not use image data of this building to generate polygons and textures of the virtual building, but instead uses polygon data and / or texture data of a general-purpose model of the virtual building.

[0074] According to the above configuration, the output unit 117 verifies whether the image data is of a building belonging to the first business operator by comparing location information obtained from a reliable third-party organization such as a second business operator or a public institution with the shooting location of the image data, and then generates a virtual building in the virtual space based on this image data.

[0075] The output unit 117 may, for example, output a store clerk object representing a virtual store clerk, based on the store clerk image extracted by the store clerk extraction unit 112a, so as to place it inside or around the virtual store in the virtual space. Specifically, the output unit 117 may use the store clerk image for at least a portion of the image of the store clerk object. The output unit 117 may, for example, operate this store clerk object in response to an operation input from a store user to the store user device 200. With such a configuration, the output unit 117 can easily output a store clerk object that reflects a real store clerk inside or around the virtual store.

[0076] The output unit 117 may, for example, if the result of the product matching unit 113b indicates that the product image and the product information indicate a predetermined relationship, cause the user device to output a purchase request input means for purchasing this product based on the product image. Having this predetermined relationship may, for example, mean that the similarity between the product indicated by the product image and the product information indicates is above a predetermined threshold. The purchase request input means may also be an icon of the product placed in the virtual space output to the user device. The purchase process may be executed directly from this product icon, or the page for the corresponding product on the e-commerce site provided by the external system 500 may be called from this product icon. With this configuration, the output unit 117 can easily list the same or similar products sold in physical stores in the virtual store.

[0077] The output unit 117 may change the output mode of the virtual space according to, for example, the access environment information acquired by the first user acquisition unit 111c. For example, if the access environment of the customer user device 300 is the United States, the output unit 117 may change the text and audio output in the virtual space from Japanese, which is pre-set as the language normally used, to the language most widely used in the United States (English). For example, if the access environment of the customer user device 300 is the United States, the output unit 117 may switch the day / night mode in the first virtual space from the day / night mode indicated by the system time to the day / night mode corresponding to US time.

[0078] According to the above configuration, the output unit 117 can realize output that takes into account the access environment when the customer user is accessing the virtual space. Therefore, it is possible to provide the customer user with a realistic space that is suitable for the environment they are accessing.

[0079] The output unit 117 may change the output configuration of the virtual space according to, for example, the behavioral history information acquired by the first user acquisition unit 111c. The output configuration of the virtual space may include, for example, the output configuration of store clerk objects placed in the virtual space. If the behavioral history information is the payment history information of a customer user, the output unit 117 may analyze the trends in the products purchased by the customer user. For example, if the result of this analysis shows that the customer user tends to purchase a relatively large amount of products related to the Edo period, the output unit 117 may have the acquisition unit 111 acquire environmental information of the Edo period from an external system 500 or the like, and set the output configuration of the virtual space to the specifications of the Edo period.

[0080] For example, if the activity history shows that the customer traveled to Kyoto five years ago, and the customer user's avatar is also in (virtual) Kyoto in the virtual space, the output unit 117 may cause the VR application of the customer user device 300 to output a message indicating that the customer is in a location in the virtual space corresponding to a place visited five years ago.

[0081] With the above configuration, the output unit 117 can realize output that takes into account the behavioral history of the user to whom it is published. Therefore, it is possible to provide the user to whom it is published with a realistic space that is suited to their own behavioral tendencies or preferences.

[0082] The output unit 117 may change the output configuration of the virtual space if, for example, the determination unit 116 determines that a user is within a predetermined range from the location of the building. Being within a predetermined range from the location of the building means, for example, if the building is a store and the user is a customer, that the customer may be visiting the store. For example, if the store is a flower shop, the output unit 117 may, if it determines that a customer may be visiting, change the background of the virtual space to a floral pattern and output flower objects to be placed around the virtual store. With such a configuration, the output unit 117 can link the output configuration of the virtual store in the virtual space to the actions of a user visiting a real building in the real world, thereby realizing a more realistic virtual space.

[0083] For example, multiple virtual spaces may exist for each user. The output unit 117 may generate a virtual building for each of the multiple virtual spaces. The output unit 117 may also change the output manner of the virtual building according to each virtual space. For example, if the acquisition unit 111 receives a request from the user to output virtual spaces for multiple eras (e.g., the present, 10 years ago, and 100 years ago), the output unit 117 may output these virtual spaces for each era to the user device simultaneously or by switching them according to the user's operation. In this case, the output unit 117 may have the acquisition unit 111 acquire environmental information corresponding to each of the multiple eras, and then output the virtual spaces for each era based on this acquired environmental information (e.g., an image of the actual building from 10 years ago).

[0084] The output unit 117 may place and output virtual buildings in each of the multiple virtual spaces based on image data of real buildings corresponding to the era set for each space. For example, if it is the present, the output unit 117 may use the latest image data of real buildings, or if it is the last 10 years, it may use image data of real buildings taken 10 years ago to output virtual buildings in the virtual space. The output unit 117 may also generate employee objects in each of the multiple virtual spaces based on environmental information corresponding to the era set for each space. For example, the output unit 117 may change the character image, audio file, audio conversion specifications, and operation control of the employee object to match each era and output the employee object.

[0085] With the above configuration, the output unit 117 can realize diverse virtual spaces, each with a sense of realism.

[0086] The output unit 117 includes a generation and processing unit 117a. The generation and processing unit 117a generates or processes virtual store data or virtual space data based on material data stored in the storage unit 130 or external information provided from the external system 500. For example, the output unit 117 may have the acquisition unit 111 acquire page information of products sold by a first business operator on an e-commerce site provided by the external system 500 as external information, and generate virtual store data based on this page information.

[0087] The generation and processing unit 117a may process the virtual space data based on environmental information acquired by the acquisition unit 111, for example. If the environmental information acquired by the acquisition unit 111 is weather information relating to the date and time and location where the image data was taken, the generation and processing unit 117a may process the virtual space data to reproduce the weather indicated by this weather information (e.g., sunny or rainy) in the virtual space. Alternatively, if the environmental information is event information held at the date and time and location where the image data was taken, the generation and processing unit 117a may process the virtual space data to reproduce the event indicated by this event information (e.g., a musician's live performance or a local festival) in the virtual space.

[0088] For example, the user experience contained within the camera's frame is limited, and the surrounding environment may not be fully captured. With the above configuration, the surrounding environment that cannot be fully captured by image data can be reproduced in the virtual space using environmental information. Therefore, a more realistic virtual space can be realized.

[0089] [Purchase Processing Department] The purchase processing unit 118 performs the purchase processing of the requested goods based on the purchase request for the goods received by the purchase request input means, which is output by the output unit 117. For example, as part of the purchase processing, the purchase processing unit 118 may instruct the second business device 400a of the second business A to make a card payment for the purchase of the goods via an online wallet.

[0090] [g section] The communication unit 120 transmits and receives various information between the user equipment, the second operator equipment 400, and / or the external system 500 via the network N.

[0091] [Storage] The storage unit 130 stores various types of information, such as first business operator information and virtual space data, which are processed by the VR system 1. The storage unit 130 may store these types of information in an associated manner, for example. The storage unit 130 may store various types of information using a database management system (DBMS) or using a file system. When using a DBMS, a table may be created for each type of information, and the various types of information may be managed by associating these tables.

[0092] <4. Example of operation> Referring to Figure 8, an example of the operation of the server device 100 will be explained. Note that the order of processing shown below is just one example and may be changed as appropriate.

[0093] As shown in Figure 8, the image acquisition unit 111a of the server device 100 acquires image data showing images of the building of the first business operator in real space (S10). The business operator acquisition unit 111b acquires the first business operator information from at least one of the second business operator device of the second business operator with which the first business operator does business, and the public institution device of the public institution where the first business operator information is registered (S11). The output unit 117 outputs to the user's user device so that it places a virtual building corresponding to the building in the virtual space based on the image data and the first business operator information (S12).

[0094] <5. Hardware Configuration> Referring to Figure 9, an example of a hardware configuration when the server device 100 described above is implemented using a computer 800 will be explained. Note that the functions of each device can also be implemented by dividing them among multiple devices.

[0095] As shown in Figure 9, the computer 800 includes a processor 801, a memory 803, a storage device 805, an input I / F unit 807, a data I / F unit 809, a communication I / F unit 811, and a display device 813.

[0096] The processor 801 controls various processes in the computer 800 by executing programs stored in memory 803. For example, each functional unit of the control unit 110 of the server device 100 can be realized by the processor 801 executing programs temporarily stored in memory 803.

[0097] Memory 803 is a storage medium such as RAM (Random Access Memory). Memory 803 temporarily stores the program code of the program executed by the processor 801, as well as data required during program execution.

[0098] The storage device 805 is a non-volatile storage medium such as a hard disk drive (HDD) or flash memory. The storage device 805 stores the operating system and various programs for realizing the above configurations. In addition, the storage device 805 can also store tables for registering various information such as image data, virtual space data, material data, first business operator information, or output information, and a database for managing those tables. Such programs and data are loaded into memory 803 as needed and accessed by the processor 801.

[0099] The input interface unit 807 is a device for receiving input from the user. Specific examples of the input interface unit 807 include keyboards, mice, touch panels, various sensors, and wearable devices. The input interface unit 807 may be connected to the computer 800 via an interface such as USB (Universal Serial Bus).

[0100] The data interface unit 809 is a device for inputting data from outside the computer 800. Specific examples of the data interface unit 809 include drive devices for reading data stored on various storage media. The data interface unit 809 may also be located outside the computer 800. In that case, the data interface unit 809 would be connected to the computer 800 via an interface such as USB.

[0101] The communication interface unit 811 is a device for performing data communication with external devices of the computer 800 via a network N, either wired or wirelessly. The communication interface unit 811 may also be located outside the computer 800. In that case, the communication interface unit 811 is connected to the computer 800 via an interface such as USB.

[0102] The display device 813 is a device for displaying various types of information. Specific examples of the display device 813 include liquid crystal displays, organic EL (Electro-Luminescence) displays, and displays for wearable devices. The display device 813 may be located outside the computer 800. In that case, the display device 813 is connected to the computer 800, for example, via a display cable. Furthermore, if a touch panel is used as the input I / F unit 807, the display device 813 can be integrated with the input I / F unit 807.

[0103] The above embodiments are illustrative examples for explaining the present invention and are not intended to limit the present invention to these embodiments alone. Furthermore, the present invention can be modified in various ways without departing from its essence. Moreover, those skilled in the art can adopt embodiments in which each of the elements described above is replaced with equivalent ones, and such embodiments are also included within the scope of the present invention.

[0104] Furthermore, the components of the server device described in the above embodiment are such that a program stored in the storage device 805 is executed by the processor 801, thereby realizing a defined process in cooperation with other hardware. In other words, these components are envisioned as both software or firmware, and as corresponding hardware, and in both of these concepts, they may be described and interpreted as "function," "means," "part," "processing circuit," "unit," or "module."

[0105] [Differentiation] Although the present invention has been described based on the embodiments described above, the following cases are also included in the present invention.

[0106] [Example 1] In the above embodiment, a client-server format was described as an example, but the VR system according to the present invention is not limited to this. The VR system according to the present invention may, for example, employ a P2P format (star type or full mesh type) that is implemented only by the user device on the edge side. In this case, all or part of each component of the server device 100 may be provided by the user device. The user device may, for example, implement the functions of each functional unit provided by the control unit 110 of the server device 100 by running an installed VR application.

[0107] [Differentiation 2] In the above embodiment, a single VR platform was described as an example for the sake of clarity, but the VR platform according to the present invention is not limited to this. For example, there may be multiple VR platforms in which the VR system according to the present invention interacts. Each of the multiple virtual spaces may be generated in each of the multiple interacting VR platforms. In this case, the virtual space data may be stored in the storage unit 130 for each of the multiple VR platforms.

[0108] [Difference 3] In the above embodiment, an example was shown in which the storage unit for storing virtual space data is the storage unit of the server device 100, but the present invention is not limited to this. For example, at least a portion of these storage units may be provided by the devices of the external system 500. [Explanation of Symbols]

[0109] 1...VR system, 100...Server device, 110...Control unit, 111...Acquisition unit, 120...Communication unit, 130...Storage unit, 200...Store user device, 300...Customer user device, 400...Second business operator device, 500...External system, 800...Computer, 801...Processor, 803...Memory, 805...Storage device, 807...Input I / F unit, 809...Data I / F unit, 811...Communication I / F unit, 813...Display device.

Claims

1. On the computer, Image acquisition function that acquires image data showing an image of a building of the first business operator in real space, which includes shooting location information indicating the shooting location where the image was taken, A business acquisition function that acquires first business information relating to the first business operator, including location information of the building, from at least one of the second business operator equipment of a second business operator that does business with the first business operator, and the public institution equipment of a public institution where the first business operator information is registered. A location matching function that compares the aforementioned shooting location information with the aforementioned location information, An output function is provided to cause a user's device to output a virtual space based on a map of the real space, and the output function is provided to cause a virtual building corresponding to the building to be placed in the virtual space based on the image data and the first business operator information, The output function described above is If the result of the comparison indicates that the shooting location and the location of the building have a predetermined relationship, the virtual building is generated based on the image data. If the result of the comparison indicates that the shooting location and the location do not have the predetermined relationship, the virtual building is generated without relying on the image data. program.

2. The aforementioned building is the store of the first business operator, The aforementioned virtual building is the virtual store of the first business operator, The aforementioned computer is equipped with a function to extract images of store employees from the aforementioned image data. The output function outputs a store employee object representing a virtual store employee, based on the store employee image, so as to place it inside or around the virtual store within the virtual space. The program according to claim 1.

3. The aforementioned building is the store of the first business operator, The aforementioned virtual building is the virtual store of the first business operator, To the aforementioned computer, A product extraction function that extracts product images of one or more products displayed in the store from the aforementioned image data, A product matching function is implemented that compares the product information with the extracted product image by referring to a product storage unit that stores product information relating to products sold by the first business operator. The output function, when the result of the matching indicates that the product image and the product have a predetermined relationship, causes the user device to output an input means for receiving a purchase request for the product from the user in the virtual store, based on the product image. The program according to claim 1.

4. The computer is provided with a first user acquisition function that acquires at least one of the following: behavioral history information indicating the user's activity history, and access environment information indicating the access environment of the user device. The output function changes the output configuration of the virtual space according to at least one of the behavior history information and the access environment information. The program according to claim 1.

5. To the aforementioned computer, A second user acquisition function that acquires user location information indicating the location of the aforementioned user, The computer is provided with a determination function that determines whether the user is within a predetermined range from the location of the building, based on the user location information and the location information. If it is determined that the user is within the predetermined range, the output function changes the output mode of the virtual space. The program according to claim 1.

6. Multiple virtual spaces exist, The output function generates the virtual structure for each of the plurality of virtual spaces and changes the output configuration of the virtual structure according to each virtual space. The program according to claim 1.

7. An image acquisition unit acquires image data that shows an image of a building belonging to a first business operator in real space, and which includes shooting location information indicating the shooting location where the image was taken. A business information acquisition unit that acquires first business information relating to the first business operator, including location information of the building, from at least one of the second business equipment of a second business operator that does business with the first business operator, and the public institution equipment of a public institution where the first business information is registered. A location matching unit that compares the aforementioned shooting location information with the aforementioned location information, An output unit that causes a user's user device to output a virtual space based on a map of the real space, comprising an output unit that, based on the image data and the first business operator information, outputs a virtual building corresponding to the building to be placed in the virtual space, The output unit is, If the result of the comparison indicates that the shooting location and the location of the building have a predetermined relationship, the virtual building is generated based on the image data. If the result of the comparison indicates that the shooting location and the location do not have the predetermined relationship, the virtual building is generated without relying on the image data. Information processing device.

8. Computers To acquire image data showing an image of a building owned by the first business operator in real space, which includes location information indicating the location where the image was taken, The first business information relating to the first business operator, including location information of the building, is obtained from at least one of the second business operator equipment of a second business operator with which the first business operator has business dealings, and the public institution equipment of a public institution where the first business information is registered. The aforementioned shooting location information and the aforementioned location information are compared, The user's device is configured to output a virtual space based on a map of the real space, and to output a virtual building corresponding to the building within the virtual space based on the image data and the first business operator information. The above output means, If the result of the comparison indicates that the shooting location and the location of the building have a predetermined relationship, the virtual building is generated based on the image data. If the result of the comparison indicates that the shooting location and the location do not have the predetermined relationship, the process includes generating the virtual building without relying on the image data. Information processing methods.

Images