Information processing system, information processing method, and program

Abstract

To provide an information processing system configured to complete settlement operation in an augmented reality space or a virtual reality space, a method and a program.SOLUTION: In a virtual-space settlement system serving as an information processing system, a retail store terminal 400 transmits store information and commodity information selected by a user on a web page. A wallet provision server 100 transmits the received store information and commodity information to an e-commerce server 500 and a virtual space management server 200. A user terminal 300 transmits user information acquired through installation of a virtual space APP and a wallet APP and operation input on the wallet APP. The wallet provision server specifies a wallet UI compatible with user settlement from the received user information. When a commodity purchase operation is performed on the user terminal, the virtual space management server transmits commodity purchase information therefor. The e-commerce server executes transfer processing to a retail store, on the basis of the purchase information received via the wallet provision server.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] A system that allows product purchasing procedures to be carried out within a virtual space has been disclosed (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-79439 Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 is configured to allow a user to perform actions in a virtual space, such as adding an object to a virtual cart (cart-in) or expressing a favorable impression of a product (like). The system also enables users to purchase products by adding products to their virtual cart in cooperation with an electronic commerce (EC) site. However, when making a purchase in a virtual space, a system is generally adopted in which the purchase is completed in a browser tab on the EC site. In this case, the system requires the user to put on and take off a terminal device such as goggles, since the purchase operation cannot be completed in the virtual space. This creates a problem in that the user loses their sense of immersion in the virtual space every time they make a purchase in the virtual space.

[0005] Therefore, in order to solve the above-mentioned problems, an object of the present invention is to provide a system for completing payment-related operations in an augmented reality space or a virtual reality space. [Means for solving the problem]

[0006] An information processing system according to one embodiment of the present invention comprises a display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment method available to a user for making payment for selected goods, which are goods selected by a selection operation by a user wearing a display device that displays the augmented reality space or the virtual reality space for a three-dimensionally displayed product object placed in the augmented reality space or the virtual reality space, and a payment unit that completes operations relating to payment for the selected goods in the augmented reality space or the virtual reality space in accordance with a selection operation by the user on the user interface relating to the payment method.

[0007] An information processing method according to one embodiment of the present invention includes a computer executing a process for displaying, on a display unit of a display device, a user interface relating to a three-dimensionally displayed payment method available to a user for making a payment for a selected commodity, which is a commodity selected by a selection operation by a user wearing a display device that displays the augmented reality space or the virtual reality space, for a three-dimensionally displayed commodity object placed in the augmented reality space or the virtual reality space, and completing an operation relating to the payment for the selected commodity in the augmented reality space or the virtual reality space in accordance with a selection operation by the user on the user interface relating to the payment method.

[0008] A program according to one embodiment of the present invention causes a computer to execute processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment method available to a user for making payment for a selected commodity, which is a commodity selected by a selection operation by a user wearing a display device displaying the augmented reality space or the virtual reality space for a three-dimensionally displayed commodity object placed in the augmented reality space or the virtual reality space, and to complete an operation relating to payment for the selected commodity in the augmented reality space or the virtual reality space in accordance with a selection operation by the user on the user interface relating to the payment method. [Effects of the Invention]

[0009] According to the present invention, payment can be completed in an augmented reality space or a virtual reality space, thereby preventing the user from losing their sense of immersion in the virtual space. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of the configuration of a virtual space payment system 10. FIG. [Figure 2] FIG. 1 is a diagram showing an example of an overview of the processing of the virtual space payment system 10. [Figure 3] FIG. 2 is a diagram showing an example of the configuration of a wallet providing server 100. [Figure 4] FIG. 10 is a diagram showing the configuration of user information D111. [Figure 5] FIG. 10 is a diagram showing the configuration of retail store information D112. [Figure 6] FIG. 11 is a diagram showing the configuration of UI information D113. [Figure 7] FIG. 10 is a diagram illustrating an example of a store identification UI in a virtual space. [Figure 8] FIG. 10 is a diagram illustrating an example of a product confirmation UI in a virtual space. [Figure 9] FIG. 10 is a diagram illustrating an example of a product selection UI in a virtual space. [Figure 10] FIG. 10 is a diagram showing another example of a product selection UI in a virtual space. [Figure 11] FIG. 10 is a diagram illustrating an example of a menu UI in a virtual space. [Figure 12] A figure showing an example of a first wallet UI in a virtual space. [Figure 13] A figure showing an example of a second wallet UI in a virtual space. [Figure 14] A figure showing an example of a third wallet UI in a virtual space. [Figure 15] A figure showing an example of a fourth wallet UI in a virtual space. [Figure 16] A figure showing another example of the fifth wallet UI in the virtual space. [Figure 17] FIG. 10 is a diagram illustrating an example of a purchase confirmation UI in a virtual space. [Figure 18] FIG. 2 is a diagram illustrating an example of the configuration of a virtual space management server 200. [Figure 19] FIG. 10 is a diagram showing an example of store space information D211. [Figure 20] FIG. 10 is a diagram showing an example of product placement information D212. [Figure 21] FIG. 10 is a diagram showing an example of product information D213. [Figure 22] FIG. 10 is a diagram showing an example of action information D214. [Figure 23] FIG. 2 illustrates an example of the configuration of a user terminal 300. [Figure 24] 1 is a flow chart showing an example of processing of the virtual space payment system 10. FIG. [Figure 25] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0011] Below, a virtual space payment system 10 according to one embodiment of the present invention will be described in detail with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or applications of technologies not explicitly stated below. In other words, the present invention can be implemented by various modifications or combinations of embodiments within the scope of its purpose. Furthermore, in the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals.

[0012] ===Outline of Virtual Space Payment System 10=== The virtual space payment system 10 is a system that allows a user to complete a purchase procedure for a product or the like in an augmented reality space (hereinafter referred to as "augmented reality space") or a virtual three-dimensional space (hereinafter referred to as "virtual reality space") projected onto a terminal device such as goggles worn by the user, without losing the sense of immersion in the space. Hereinafter, for convenience, augmented reality space and virtual reality space may be collectively referred to as "virtual space." In other words, the virtual space payment system 10 allows a user to complete a purchase procedure for a 3D avatar without having to leave the virtual space (for example, without removing a terminal device such as goggles). This prevents the user from losing the sense of immersion in the virtual space.

[0013] Augmented reality space is, for example, a space including images that are displayed as if projected onto real space when a user wears a terminal device that is a transparent head-mounted display. Virtual reality space is, for example, a virtual space configured with images displayed on a terminal device that is a non-transparent head-mounted display worn by a user. Note that virtual reality space also includes so-called mixed reality space (a space in which information from the real world is reflected in a digital space). The image displayed on the terminal device is, for example, an image that the user can visually recognize as a three-dimensional image. Hereinafter, among the images displayed on the terminal device, an object that the user can visually recognize in the virtual space as an actual object may be referred to as a "virtual object."

[0014] First, an overview of the configuration of the virtual space payment system 10 will be described with reference to Figure 1. Figure 1 is a diagram showing an example of the configuration of the virtual space payment system 10. As shown in Figure 1, the virtual space payment system 10 may include, for example, a wallet providing server 100, a virtual space management server 200, a user terminal 300, and a retail store terminal 400.

[0015] The wallet providing server 100 is a server that processes information for enabling a user to complete a purchase procedure in a virtual space. The wallet providing server 100 processes information for a retailer to open a store in a virtual space (e.g., information about virtual objects for the store, information about virtual objects for products, etc.), and processes information about a user interface for enabling a user to complete a purchase procedure in the virtual space (e.g., information about virtual objects related to payment). The wallet providing server 100 may provide the information for opening a store and the information about the user interface to the virtual space management server 200 (described later) in a software development kit (hereinafter referred to as "SDK").

[0016] As a result, the virtual space payment system 10 can easily enable retailers to open stores and users to complete purchase procedures in the virtual space. Furthermore, the SDK allows businesses operating virtual spaces to easily enable retailers to open stores and users to complete purchase procedures, thereby attracting retailers and encouraging consumer participation. As a result, the virtual space payment system 10 can provide the economic benefit of enabling businesses to expand their business areas.

[0017] The virtual space management server 200 is, for example, a server for providing a virtual space to a user. The user terminal 300 provides the virtual space to the user based on information provided by the virtual space management server 200. The virtual space management server 200 causes the user terminal 300 to display a predetermined user interface based on information related to the user interface provided by the wallet providing server 100, for example.

[0018] The user terminal 300 is, for example, a non-transparent head-mounted display or a transparent head-mounted display, and is a device that displays virtual objects in a virtual space so that the user can visually recognize them. The user terminal 300 has a function of recognizing the user's actions.

[0019] The retail store terminal 400 is, for example, a device that transmits information about a store opened in a virtual space and products to be displayed. The retail store terminal 400 generates various information by, for example, accepting operational input from the manager of the retail store.

[0020] Although the above description has been given assuming that the wallet providing server 100 and the virtual space management server 200 are provided separately, this is not limiting. The wallet providing server 100 and the virtual space management server 200 may be configured as a single server or may be a cloud server. Furthermore, the wallet providing server 100 may be configured as multiple servers divided by function, for example.

[0021] Next, an overview of the processing of the virtual space payment system 10 will be described with reference to FIG. 2. FIG. 2 is a diagram showing an example of an overview of the processing of the virtual space payment system 10. First, in step S1, the retailer terminal 400 accepts, for example, a user's operation input on a web page provided by the wallet providing server 100, and transmits information about the store (hereinafter referred to as "store information") and information about the product (hereinafter referred to as "product information") to the wallet providing server 100. Although not shown here, the wallet providing server 100 may, for example, provide the store information and product information to a business that creates three-dimensional data, thereby obtaining three-dimensional data of the store and the product from the business. Note that the wallet providing server 100 may, or may not, automatically obtain three-dimensional data of the store and the product from the business by providing the store information and product information to the business.

[0022] Next, in step S2, the wallet providing server 100 transmits, for example, store information and product information to the e-commerce server 500. This enables the wallet providing server 100 to transfer funds to a retail store when the user performs a procedure in the virtual space to purchase a product provided by the retail store.

[0023] Next, in step S3, the wallet providing server 100 transmits, for example, store information, product information, and three-dimensional data of the store and products to the virtual space management server 200. This allows retailers to open stores in virtual space and display products with simple input operations. This eliminates the barrier that retailers have to prepare three-dimensional data of the store and products when opening stores in virtual space.

[0024] Next, in step S4, the user terminal 300 installs, for example, application software (hereinafter referred to as "virtual space APP") for displaying the virtual space on the display unit 1007 of the user terminal 300. Note that the user terminal 300 may display the virtual space on the display unit 1007 via a web browser instead of using the virtual space APP.

[0025] Next, in step S5, the user terminal 300 installs, for example, application software (hereinafter referred to as "wallet APP") that allows the user to use a payment method for purchasing products in the virtual space.

[0026] Next, in step S6, the user terminal 300 transmits information about the user (hereinafter referred to as "user information") acquired based on user input via the wallet APP, for example, to the wallet providing server 100. The user information may include, for example, information about the payment method used by the user (hereinafter referred to as "payment method information").

[0027] Next, in step S7, the wallet providing server 100 identifies a user interface (hereinafter referred to as a "wallet UI") corresponding to the payment method used by the user, for example, based on the user information. The wallet providing server 100 transmits the user information and information related to the wallet UI to the virtual space management server 200. This allows the user to purchase products in the virtual space using the desired payment method.

[0028] Next, in step S8, the user terminal 300 accepts user input, for example, to add a product from a store in the virtual space to a cart. The user terminal 300 accepts user input, for example, to select a product from the cart, and executes the purchase procedure. For convenience, the operations from when the user selects a product to when the purchase procedure is executed will be referred to as a "purchase operation," and the information for executing payment through the purchase operation will be referred to as "payment execution information." The user interface provided to the user through the user terminal 300 during the purchase operation will be described later.

[0029] Next, in step S9, the virtual space management server 200 transmits information about the purchased product (hereinafter referred to as "purchase information") to the wallet providing server 100. The purchase information includes, for example, information about the product that the user has purchased. Next, in step S10, the wallet providing server 100 transmits the purchase information to the e-commerce server 500. This allows the user to complete the purchase procedure without removing the user terminal 300.

[0030] Next, in step S11, the electronic commerce server 500 transfers the sales proceeds of the product for which the user has completed the purchase procedure to the retail store, thereby enabling the retail store to obtain the sales proceeds in the virtual space.

[0031] In this way, even if there are multiple virtual space management servers 200 in the virtual space payment system 10, each virtual space management server 200 can transmit purchase information to the appropriate e-commerce server 500 via the wallet providing server 100. In other words, the user can complete a transaction, for example, within a VR (Virtual Reality) SNS (Social Networking Service) without having to put on or take off the user terminal 300, thereby eliminating the problem of the user losing their sense of immersion in the virtual space.

[0032] ===Configuration of wallet provider server 100=== Next, the configuration of the wallet providing server 100 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the wallet providing server 100. As shown in Fig. 3, the wallet providing server 100 includes functional units such as a storage unit 110, an acquisition unit 120, a UI information providing unit 130, a display processing unit 140, a payment unit 150, and a product classification unit 160. Each functional unit is a function realized, for example, when a processor 1001 reads out a program stored in a memory 1002.

[0033] The storage unit 110 stores user information D111, retail store information D112, and UI information D113.

[0034] First, the user information D111 will be described with reference to FIG. 4. FIG. 4 is a diagram showing the configuration of the user information D111. As shown in FIG. 4, the user information D111 may include, for example, the following fields: [User ID], [Name], [Email Address], [Address], [Telephone Number], [Payment Method], [Purchased Product], and [UI ID]. [User ID] stores a user ID that can uniquely identify a user. [Name] stores the user's name. [Email Address] stores the email address used by the user. [Address] stores the user's current address. [Telephone Number] stores the user's telephone number. [Payment Method] stores the payment method used by the user in the virtual space. In this example, the user with user ID "U0001" has registered credit card A, bank account B, virtual currency C, and electronic money D as payment methods. The wallet providing server 100 generates information about a wallet UI linked to the user with user ID "U0001." [Purchased Product] stores data about products the user previously purchased in the virtual space. [UIID] stores a UIID that can uniquely identify each UI.

[0035] Next, the retail store information D112 will be described with reference to FIG. 5. FIG. 5 is a diagram showing the configuration of the retail store information D112. As shown in FIG. 5, the retail store information D112 may include the following items, for example: [Retail Store ID], [Retail Store Name], [Email Address], [Address], [Telephone Number], [Store 3D Data], [Product 3D Data], and [Sales History]. [Retail Store ID] stores a retail store ID that can uniquely identify a retail store. [Retail Store Name] stores the name of the retail store. [Email Address] stores the email address used by the retail store. [Address] stores the current location of the retail store. [Telephone Number] stores the telephone number of the retail store. [Store 3D Data] stores three-dimensional data related to the retail store. [Product 3D Data] stores three-dimensional data related to products linked to the retail store. [Sales History] stores the sales history of products at the retail store in the virtual space.

[0036] Next, the UI information D113 will be described with reference to FIG. 6. FIG. 6 is a diagram showing the configuration of the UI information D113. As shown in FIG. 6, the UI information D113 may include, for example, items such as [UI ID], [UI name], [UI type], [UI data], and [virtual space ID]. [UI ID] stores a UI ID that can uniquely identify various UIs. [UI name] stores the name of various UIs. [UI type] stores the type of UI, such as a UI for identifying a store, a UI for checking products, a UI for executing a payment method, a UI for checking purchased products, a UI related to a menu, etc. [UI data] stores data for displaying a UI in a virtual space. [Virtual space ID] stores a virtual space ID that can uniquely identify the virtual space corresponding to the UI.

[0037] The acquisition unit 120 acquires information from other devices such as the virtual space management server 200, the user terminal 300, the retail store terminal 400, and the e-commerce server 500, for example.

[0038] The UI information providing unit 130 generates information about UIs related to purchasing operations by the user (hereinafter referred to as "UI information"). The UIs related to purchasing operations include, for example, a UI for identifying a store in the virtual space (hereinafter referred to as a "store identification UI"), a UI for checking products (hereinafter referred to as a "product confirmation UI"), a UI for selecting products in a cart (hereinafter referred to as a "product selection UI"), a wallet UI for executing a payment method, a UI for checking purchased products (hereinafter referred to as a "purchase product confirmation UI"), and a UI related to a menu (hereinafter referred to as a "menu UI"). The information about the UIs generated by the UI information providing unit 130 is transmitted to the virtual space management server 200. This enables the virtual space management server 200 to provide the user terminal 300 with a UI suited to the user in the virtual space. Hereinafter, for convenience, the term "user's hand" refers to the hand of an avatar corresponding to the user in the virtual reality space, and the user's actual hand in the augmented reality space.

[0039] First, an example of a store identification UI will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of a store identification UI in a virtual space. As shown in FIG. 7, the store identification UI includes an identification object U1 that is displayed in the virtual space for stores that accept payment methods. The identification object U1 is displayed, for example, when the user approaches the store within a predetermined distance (here, "L") from the store. The identification object U1 is an object that indicates, for example, a logo or the type of payment method. This allows the virtual space payment system 10 to easily find stores that accept the user's own payment method, thereby improving user convenience.

[0040] Next, an example of a product confirmation UI will be described with reference to FIG. 8. FIG. 8 is a diagram illustrating an example of a product confirmation UI in a virtual space. As shown in FIG. 8, the product confirmation UI includes a linear object (hereinafter referred to as a "line object U2") that is displayed in the virtual space in response to a user's hand gesture recognized by the user terminal 300. Furthermore, when the user pinches an object by overlapping their index finger and thumb (hereinafter referred to as a "pinch gesture") while the line object U2 overlaps or intersects with the product, the product confirmation UI brings the product closer to the user and displays the product in a rotatable and deformable manner. This allows the virtual space payment system 10 to improve user convenience by allowing the user to check the product from various angles at hand without approaching the product. Furthermore, the product confirmation UI may include a price display object U3 that displays the price, specifications, etc. of the product when the product is designated by the line object U2. This allows the user to easily check the price, specifications, etc. of the product, improving user convenience.

[0041] As shown in FIG. 8 , the product confirmation UI may include a display object U4 that displays, within a predetermined area, multiple products related to a product designated by the user. For example, if the product designated by the user is "Bag A," related products may include bags of the same brand, bags of a similar shape, bags of a similar color, bags with a similar purpose, etc., in relation to Bag A. The display object U4 is a spherical object that displays 3D image data of the product within the interior area of ​​the sphere. The multiple display objects U4 may be displayed in a line away from the user so that at least a portion of each of the display objects U4 overlaps. This improves the user's visibility of the products in the virtual space. The product confirmation UI may also be displayed to move in response to the user's hand movements over the multiple display objects U4. This allows the user to view products through intuitive operations.

[0042] Next, an example of a product selection UI will be described with reference to FIG. 9. FIG. 9 is a diagram illustrating an example of a product selection UI in a virtual space. As illustrated in FIG. 9, the product selection UI includes, for example, a cart object U5 and a product display object U6 that displays selected products. The cart object U5 is a three-dimensional object for storing selected products. The cart object U5 is displayed so as to cover one of the user's palms and moves in accordance with the movement of the palm. The product display object U6 is an object that displays a reduced version of a product within a predetermined area when the product is moved to the cart object U5. The product display object U6 is an object that displays three-dimensional image data of the product within a spherical inner area in response to a pinching action of the product with the user's other hand. The product display object U6 is displayed so as to be movable by the user while accepting the user's pinching action. This allows the user to select products through intuitive operations.

[0043] Another example of the product selection UI will be described with reference to FIG. 10 . FIG. 10 is a diagram illustrating another example of the product selection UI in a virtual space. As illustrated in FIG. 10 , the product selection UI includes, for example, a product display object U7 that displays multiple products added to a cart by a user. The product display object U7 is, for example, an area in which each product moved to the cart object U5 illustrated in FIG. 9 is displayed in a reduced size within a predetermined area. The product selection UI may, for example, display multiple product display objects U7 in a line surrounding the upper body (torso) of the user's avatar. This makes it easier for the user to view the selected products, thereby improving user convenience. The product selection UI also cancels the selection of a product in response to a user's hand gesture that moves the product display objects U7 in a line away from the line. This allows the user to cancel the selection of a product through an intuitive operation. The product selection UI also moves a product along the line of the product display objects U7 in response to a hand gesture that moves the product along the line. This allows the user to view the selected product through an intuitive operation.

[0044] Next, an example of a menu UI will be described with reference to FIG. 11. FIG. 11 is a diagram illustrating an example of a menu UI in a virtual space. As illustrated in FIG. 11, the menu UI includes, for example, icons U8 arranged so as to surround a user's hand with the thumb pointed forward. Surrounding the hand includes, for example, arranging the icons U8 in a circle or arc shape around the wrist. The icons U8 are menu items (e.g., items that transition to checkout, cart, or inventory). This allows the user to easily understand the items included in the menu, thereby improving user convenience. The menu UI may also include an indicator object U9 that points to one of the multiple icons U8. The indicator object U9 is displayed so as to point in the direction indicated by the tip of the user's thumb, for example. The menu UI may change the icon U8 pointed to by the indicator object U9 in response to, for example, twisting the user's wrist in one direction. This allows the user to select a menu item through an intuitive operation.

[0045] Next, an example of the first wallet UI will be described with reference to FIG. 12. FIG. 12 is a diagram illustrating an example of the first wallet UI in a virtual space. As shown in FIG. 12, the first wallet UI includes, in the virtual space, card-shaped objects U10 corresponding to payment methods in the real space. In the first wallet UI, multiple card-shaped objects U10 are displayed in one direction so that some of the objects overlap. This allows the user to intuitively select a payment method. The display position of the first wallet UI may be changed depending on the frequency of use by the user. Specifically, the first wallet UI may display the card-shaped objects U10 in descending order of frequency of use (for example, in FIG. 12, "Credit Card A" is most frequently used). This allows the user to easily select a frequently used payment method. The first wallet UI may also display a limit display object U11 that displays the credit limit for each payment method. This allows the user to easily grasp the available balance for each payment method, thereby improving user convenience.

[0046] Next, an example of the second wallet UI will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of the second wallet UI in a virtual space. As shown in Fig. 13, the second wallet UI includes an object U12 in a virtual space, the object having a shape corresponding to the type of asset related to the payment method. The second wallet UI may include, for example, a wallet-shaped object corresponding to a regular account, a piggy bank-shaped object corresponding to a savings account, and a three-dimensional polygonal object corresponding to a crypto asset. This allows the user to select a payment method through intuitive operations.

[0047] Next, an example of the third wallet UI will be described with reference to FIG. 14. FIG. 14 is a diagram showing an example of the third wallet UI in a virtual space. As shown in FIG. 14, the third wallet UI includes an input object U13 that allows a user to input secret information in the virtual space. The input object U13 is, for example, a combination lock. The input object U13 is an object that accepts input of secret information in response to a user's action of pinching and turning the combination lock. When the authenticity of the secret information is confirmed, the third wallet UI causes the wallet providing server 100 to transmit payment execution information for executing a payment. This allows the user to execute a payment using an intuitive operation. The third wallet UI may also be an input object representing a pin pad, a signature pad, or a keyboard.

[0048] Next, an example of the fourth wallet UI will be described with reference to FIG. 15. FIG. 15 is a diagram showing an example of the fourth wallet UI in a virtual space. As shown in FIG. 15, the fourth wallet UI includes a slide object U14 that slides in response to the user's hand movement. The slide object U14 may be provided on a plate-shaped object U15. The fourth wallet UI may also display the slide object U14 in one of the user's palms. The fourth wallet UI transmits payment execution information in response to the user's hand sliding the slide object U14. This allows the user to perform payments through intuitive operations.

[0049] Another example of the fifth wallet UI will be described with reference to Fig. 16. Fig. 16 is a diagram showing another example of the fifth wallet UI in a virtual space. As shown in Fig. 16, the fifth wallet UI may include a card-shaped slide object U16 and a gate object U17 that allows the slide object U16 to pass through. In this case, the fifth wallet UI transmits payment execution information in accordance with the movement of the user's hand sliding the slide object U16. This allows the user to perform payments through intuitive operations.

[0050] Next, an example of a purchase item confirmation UI will be described with reference to FIG. 17. FIG. 17 is a diagram showing an example of a purchase item confirmation UI in a virtual space. As shown in FIG. 17, the purchase item confirmation UI includes icons U18 (e.g., in a folder format) corresponding to product categories (e.g., clothing, games, etc.) classified by a product classification unit (described later). The purchase item confirmation UI may, for example, display products associated with the icon U18 selected by the user in a predetermined format. This improves user operability. The user can easily find 3D image data of purchased products. For example, in the case of an icon U18 corresponding to the clothing category, the predetermined format may include a format in which 3D image data of the purchased clothing is displayed hanging in a wardrobe U19. This allows the user to view purchased products as if checking them in real space, thereby improving user convenience. The purchase item confirmation UI may also display a combination of multiple purchased products, for example. Specifically, as shown in FIG. 17, the purchase confirmation UI may display clothing items hung in a wardrobe U19, and clothing items selected by the user may be applied to a human-shaped object U20. This allows the user to check how the items coordinate with the clothes they currently own while shopping, improving user convenience. Furthermore, even for large, highly customizable items with complex mechanisms, such as cars, motorcycles, and bicycles, users can consider combinations and preview functions before purchasing, improving user convenience.

[0051] The display processing unit 140 executes a process of transmitting information about the various UIs to the virtual space management server 200, for example, so that the virtual space management server 200 can display the various UIs generated by the UI information providing unit 130 on the display unit 1007 of the user terminal 300. The display processing unit 140 of the wallet providing server 100 may transmit information about the various UIs directly to the user terminal 300, so that the various UIs are displayed on the display unit 1007 of the user terminal 300. The information about the various UIs may include a program for displaying the various UIs on the display unit 1007 of the user terminal 300.

[0052] The settlement unit 150 completes operations related to the payment of products selected by the user in the virtual space, for example, in response to a selection operation by the user on a user interface related to a payment method (e.g., the first to fourth wallet UIs). Specifically, the user terminal 300 transmits payment execution information to the virtual space management server 200 based on a selection operation by the user on, for example, the first or second wallet UI, or on, for example, the third, fourth, or fifth wallet UI, related to a payment method. The settlement unit 150 executes a payment process based on purchase information transmitted from the virtual space management server 200 based on the payment execution information. The payment process is, for example, a process of transmitting purchase information including information about products purchased by the user to the e-commerce server 500. The settlement unit 150 executes the payment process, thereby completing the payment by the user in the virtual space.

[0053] The product classification unit 160 classifies information about purchased products, which are products paid for by the payment settlement unit 150, into one of a plurality of categories. The product classification unit 160 stores the classified information about purchased products in the user information D111.

[0054] ===Configuration of virtual space management server 200=== Next, the configuration of virtual space management server 200 will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of the configuration of virtual space management server 200. As shown in Fig. 18, virtual space management server 200 includes functional units such as a storage unit 210, an acquisition unit 220, a display processing unit 230, and a purchase information transmission unit 240. Each functional unit is a function realized, for example, by processor 1001 reading out a program stored in memory 1002.

[0055] The storage unit 110 stores, for example, store space information D211, product placement information D212, product information D213, action information D214, UI information D215, and user information D216. Note that the UI information D215 and the user information D216 are similar to the UI information D113 and the user information D111, and therefore a description thereof will be omitted.

[0056] First, the store space information D211 will be described with reference to FIG. 19. FIG. 19 is a diagram showing an example of the store space information D211. The store space information D211 is a database that stores three-dimensional data of a store in a virtual space. The three-dimensional data of a store includes data related to the three-dimensional shape of a room consisting of walls, floors, ceilings, product display shelves, etc. As shown in FIG. 19, the store space information D211 may include items such as [Store Space ID], [Store Space Name], and [Update History]. [Store Space ID] stores a store space ID that can uniquely identify a store in the virtual space. [Store Space Name] stores the name of the store space. The name of the point p space may be, for example, the name of the store. [Update History] stores the history of updates to the store space.

[0057] Next, the product placement information D212 will be described with reference to FIG. 20. FIG. 20 is a diagram showing an example of the product placement information D212. The product placement information D212 stores the placement of objects. An object is an object in a virtual space that a user focuses on. As shown in FIG. 20, the product placement information D212 may include, for example, the following items: [Store Space ID], [Object ID], [3D Coordinates], [Scale], and [Update Date and Time]. [Store Space ID] stores the store space ID. [Object ID] stores the object ID of an object placed in the virtual space. [3D Coordinates] stores the coordinates (x, y, z) indicating the position of an object placed in a store in the virtual space. [Scale] stores the scale used when displaying an object in the virtual space. [Update Date and Time] stores the update date and time of each record.

[0058] Next, the product information D213 will be described with reference to Fig. 21. Fig. 21 is a diagram showing an example of the product information D213. The product information D213 stores attributes of objects, etc. As shown in Fig. 21, the product information D213 may include items such as [Object ID], [Product Name], [Price], and [Update Date and Time]. [Object ID] stores the object ID of the corresponding object. [Product Name] stores the product name. [Price] stores the price of the product. [Update Date and Time] stores the update date and time of each record.

[0059] Next, the action information D214 will be described with reference to FIG. 22. FIG. 22 is a diagram illustrating an example of the action information D214. The action information D214 stores actions corresponding to user actions. As illustrated in FIG. 22, the action information D214 may include, for example, the following items: [Action ID], [Action Name], [Recognized Action], [UI ID], and [Execution Date and Time]. [Action ID] stores an action ID that can uniquely identify the type of action. [Action Name] stores the name of the action. [Recognized Action] stores information corresponding to the user action recognized by the user terminal 300. Specifically, the recognized user action includes, for example, the user's hand movements shown in FIGS. 8, 9, 10, 14, 15, 16, and 11. [UI ID] stores a UI ID that can uniquely identify various UIs. [Execution Date and Time] stores the date and time when the action was performed by the user.

[0060] The acquisition unit 220 acquires information from other devices such as the wallet providing server 100 and the user terminal 300. The various information acquired by the acquisition unit 220 is stored in the storage unit 210.

[0061] The display processing unit 230 executes a process for displaying, for example, three-dimensional image data relating to the virtual space stored in the storage unit 210 on the display unit 1007 of the user terminal 300. Specifically, the display processing unit 230 causes the display unit 1007 of the user terminal 300 to display a virtual space including a UI such as those shown in FIGS.

[0062] For example, when the purchase information transmission unit 240 acquires payment execution information from the user terminal 300, the purchase information transmission unit 240 identifies the product that is the target of the purchase operation. The purchase information transmission unit 240 transmits purchase information related to the product that is the target of the purchase operation to the wallet providing server 100.

[0063] ===User terminal 300=== Next, the configuration of the user terminal 300 will be described with reference to Fig. 23. Fig. 23 is a diagram showing an example of the configuration of the user terminal 300. As shown in Fig. 23, the user terminal 300 includes functional units such as a storage unit 310, an acquisition unit 320, an action recognition unit 330, and a display processing unit 340. Each functional unit is a function that is realized, for example, when the processor 1001 reads out a program stored in the memory 1002.

[0064] The storage unit 310 stores, for example, user information D311 and action information D312. The user information D311 is similar to the user information D111, and therefore its description will be omitted. The action information D312 is similar to the action information D214, and therefore its description will be omitted. The storage unit 310 may also store information similar to the UI information D215. The acquisition unit 320 acquires information from other devices, such as the wallet providing server 100 and the virtual space management server 200. The various information acquired by the acquisition unit 320 is stored in the storage unit 310. The action recognition unit 330 recognizes a user's action based on, for example, information acquired by the sensor group 1008 included in the user terminal 300. Specifically, the action recognition unit 330 generates information related to the action (hereinafter referred to as "action information") by referring to the action information D312. The user terminal 300 transmits the action information to the virtual space management server 200. This allows a UI corresponding to the user's actions to be displayed on the display unit 1007. The display processing unit 340 causes the display unit 1007 to display, for example, three-dimensional image data related to the virtual space acquired from the virtual space management server 200 and three-dimensional image data related to various UIs.

[0065] ===Processing of the virtual space payment system 10=== An example of the processing of the virtual space payment system 10 will be described with reference to Figure 24. Figure 24 is a flow diagram showing an example of the processing of the virtual space payment system 10. Below, an example of the processing flow will be described, from a retail store opening a store in a virtual space to a user finding the retail store in the virtual space, purchasing clothes at the retail store, and completing the payment processing. The following description will also refer to Figures 7 to 17 as appropriate.

[0066] In step S100, the retail store terminal 400 of the retail store transmits store information and product information to the wallet providing server 100. The wallet providing server 100 may create 3D image data of the store and the product based on the store information and the product information. The wallet providing server 100 associates the store information, product information, and 3D image data of the store and the product with a store ID and stores them in the storage unit 110.

[0067] Next, in step S101, the wallet providing server 100 transmits store information, product information, and 3D image data of the store and products to the virtual space management server 200. At this time, the wallet providing server 100 may transmit the store information, product information, and 3D image data of the store and products together with a program for applying the information to the virtual space provided by the virtual space management server 200. In other words, the wallet providing server 100 may transmit the store information, product information, and 3D image data of the store and products to the virtual space management server 200 using an SDK. This allows retailers to easily open stores in virtual space.

[0068] Next, in step S102, virtual space management server 200 stores store information, product information, and three-dimensional image data of the store and the product in storage unit 210. Virtual space management server 200 reflects the store and the product in the virtual space.

[0069] Next, in step S200, the wallet providing server 100 transmits a wallet APP for enabling various UIs to be used in the virtual space in response to a request from the user terminal 300. Also, the virtual space management server 200 transmits a virtual space APP for displaying the virtual space in response to a request from the user terminal 300.

[0070] Next, in step S201, the user terminal 300 accepts an operational input from the user and transmits the user information and payment method information to the wallet providing server 100. The wallet providing server 100 stores the user information and payment method information in the storage unit 110.

[0071] Next, in step S202, the wallet providing server 100 associates UI information corresponding to the payment method information with the user information and transmits it to the virtual space management server 200. This allows the virtual space management server 200 to provide a wallet UI to the user. In step S203, the virtual space management server 200 associates the wallet UI with the user information and stores it in the storage unit 210.

[0072] Next, in step S300, the user terminal 300 transmits authentication information to the virtual space management server 200. In step S301, the virtual space management server 200 transmits information about the virtual space to the user terminal 300. In step S302, the user terminal 300 displays the virtual space on the display unit 1007. The user terminal 300 recognizes the user's actions by referring to the action information D312 and transmits action information to the virtual space management server 200. In step S303, the virtual space management server 200 transmits UI information about various UIs corresponding to the user's actions to the user terminal 300 based on the action information. Next, in step S304, the user terminal 300 displays the various UIs on the display unit 1007.

[0073] The following describes the process of displaying various UIs by the user terminal 300. Note that, although the following describes the virtual space management server 200 referencing the UI information 215 to display various UIs on the display unit 1007 of the user terminal 300, for example, if the UI information is stored in the user terminal 300, the user terminal 300 may refer to the UI information to display various UIs on the display unit 1007.

[0074] 7, when the virtual space management server 200 determines that the distance between the avatar and a store in the virtual space has reached a predetermined distance, it refers to the UI information D215 and displays a store identification UI on the display unit 1007. This allows the user to easily recognize stores that accept the registered payment method, thereby improving user convenience.

[0075] As shown in FIG. 8, when a user checks a product in a store, the virtual space management server 200 refers to UI information D215 and displays a product confirmation UI (here, a line object U2) on the display unit 1007 of the user terminal 300. The virtual space management server 200 acquires motion information related to the user's hand movement recognized by the user terminal 300 with reference to action information D312, thereby rotating, shrinking, and enlarging the product as it is drawn toward the user's hand. The virtual space management server 200 also refers to product information D213 to identify products related to the product specified by the line object U2. The virtual space management server 200 displays 3D image data of each of the identified products within the area of ​​a spherical display object U4. Based on the motion information related to the user's hand movement recognized by the user terminal 300, the virtual space management server 200 moves the spherical display object U4 in accordance with the hand movement. This allows the user to check related products at a glance, and the virtual space payment system 10 enables the user to browse products intuitively.

[0076] As shown in FIG. 9, the virtual space management server 200 displays a product selection UI (here, a cart object U5 and a product display object U6) for a user to add products in a store to a cart on the display unit 1007 of the user terminal 300. The virtual space management server 200 acquires motion information related to the user's motion of sticking out the palm of their left hand, which is recognized by the user terminal 300 with reference to action information D312. The virtual space management server 200 displays the cart object U5 on the display unit 1007 of the user terminal 300 based on the UI information D215. This allows the virtual space payment system 10 to display the cart using an intuitive motion, thereby improving user convenience. The virtual space management server 200 acquires motion information related to the user's motion of picking up a product, which is recognized by the user terminal 300 with reference to the action information D312. The virtual space management server 200 displays a product display object U6 including three-dimensional image data of the product on the display unit 1007 of the user terminal 300 with reference to the UI information D215. When the virtual space management server 200 acquires the operation information regarding the operation of moving the product display object U6 to the cart object U5, it adds the product to a purchase list (hereinafter referred to as a "purchase list"). The purchase list is, for example, a list stored in the memory unit 210 of the virtual space management server 200. This allows the virtual space payment system 10 to add products to the cart with an intuitive operation, thereby improving user convenience.

[0077] 10, the virtual space management server 200 displays a product selection UI (here, a product display object U7) for viewing products stored in the purchase list on the display unit 1007 of the user terminal 300. When the virtual space management server 200 receives from the user terminal 300 operation information related to an operation of selecting, for example, an icon U8 indicating a cart on the menu UI, the virtual space management server 200 displays the product display object U7 on the display unit 1007. When the virtual space management server 200 receives information related to an operation of removing the product display object U7 from the row, the virtual space management server 200 removes the product displayed in the product display object U7 from the purchase list. This allows the virtual space payment system 10 to easily view the products in the cart and to remove products from the cart through intuitive operations, thereby improving user convenience.

[0078] As shown in FIG. 11 , the virtual space management server 200 displays a menu UI for selecting an item on the display unit 1007 of the user terminal 300. The virtual space management server 200 acquires motion information related to the motion of extending one hand with the thumb facing forward, which is recognized by the user terminal 300 with reference to the action information D312. The virtual space management server 200 displays an icon U8 included in the menu UI on the display unit 1007 of the user terminal 300, surrounding the user's extended hand. This allows the user to easily check the menu items, improving user convenience. When the virtual space management server 200 acquires motion information related to the operation of selecting a specific icon U8 in the menu UI from the user terminal 300, the virtual space management server 200 displays an object corresponding to the icon U8 on the display unit 1007 of the user terminal 300.

[0079] As shown in Figures 12 and 13, virtual space management server 200 displays a wallet UI (here, a first wallet UI or a second wallet UI) on display unit 1007 of user terminal 300, which allows the user to select a payment method when purchasing a product. For example, when virtual space management server 200 acquires operation information related to an operation for paying for a product in a purchase list recognized by user terminal 300 by referring to action information D312, virtual space management server 200 displays the first wallet UI or the second wallet UI on display unit 1007. Virtual space management server 200 generates purchase information based on the purchase operation for the payment method (e.g., credit card U10 or wallet object U12) selected by the user from the first wallet UI or the second wallet UI.

[0080] 14, 15, and 16, virtual space management server 200 displays a wallet UI (here, the third wallet UI, the fourth wallet UI, or the fifth wallet UI) on display 1007 of user terminal 300, which accepts an action to confirm a purchase when making a payment with a predetermined payment method. For example, when virtual space management server 200 acquires operation information related to an operation to select a payment method as shown in FIGS. 12 and 13, virtual space management server 200 displays the third wallet UI, the fourth wallet UI, or the fifth wallet UI on display 1007 of user terminal 300. When user terminal 300 recognizes, with reference to action information D312, an action to execute a user payment using the third wallet UI, the fourth wallet UI, or the fifth wallet UI, it transmits payment execution information to execute the payment to virtual space management server 200.

[0081] Next, in step S305, purchase information is generated based on the payment execution information and transmitted to the wallet providing server 100. Next, in step S306, the wallet providing server 100 transmits the purchase information to the e-commerce server 500 to settle the product that is the subject of the payment. This allows the user to complete payment-related operations in the virtual space by operating the user terminal 300 without removing the user terminal 300.

[0082] Next, in step S306, the wallet providing server 100 identifies the e-commerce server 500 corresponding to the purchased product based on the purchase information and transmits the purchase information to the e-commerce server 500. This allows the e-commerce server 500 to execute payment processing to the retailer linked to the purchase information. This allows the user to complete the transaction within, for example, a VR (Virtual Reality) SNS (Social Networking Service).

[0083] Next, in step S307, the user terminal 300 receives from the wallet providing server 100 a payment availability notification regarding whether or not payment is possible at the e-commerce server 500. This allows the user to know whether or not payment in the virtual space is possible, thereby improving user convenience.

[0084] Next, in step S308, the virtual space management server 200 stores the purchased product information in the user information D216. As shown in Fig. 17, the virtual space management server 200 may refer to the user information D216 and cause the display unit 1007 of the user terminal 300 to display a purchased product confirmation UI that displays, as a three-dimensional image, three-dimensional image data related to the product for which the purchase procedure has been completed.

[0085] === Variations === The virtual space payment system 10 may further include a configuration for issuing a token number that is encrypted (anonymized) based on confidential information (e.g., card number) of a payment method, such as a credit card. Specifically, for example, after step S6 in FIG. 2, the user terminal 300 transmits information about the card number to the service provider that issues the token number, based on a user's input. The user terminal 300 acquires the token number corresponding to the card number. The user terminal 300 stores the token number in association with the virtual space APP. When the user terminal 300 receives an action to purchase a product from the user in the virtual space, it transmits payment execution information to the virtual space management server 200. The virtual space management server 200 transmits purchase information including the token number to the wallet providing server 100 (step S9 in FIG. 2). This allows the user to enjoy a secure purchase process.

[0086] The virtual space payment system 10 may also manage three-dimensional data related to purchased products on a blockchain platform. Specifically, a purchase certificate for the purchased product is issued to the user, and the three-dimensional data related to the product is stored in distributed storage. The transaction history of the product is also stored on the blockchain platform. This allows the user to prove that they are a legitimate purchaser of the product, and allows the data to be used even if the content distribution provider's service is suspended.

[0087] ===Hardware Configuration=== The wallet providing server 100 and virtual space management server 200 that implement the functionality of the above-described embodiments may be, for example, a server computer, a personal computer (e.g., desktop, laptop, tablet, etc.), a media computer platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computer device (e.g., PDA, email client, etc.), or any other type of computer or communication platform.

[0088] The user terminal 300 and retail store terminal 400 that realize the functions related to the above-described embodiments may be, for example, a smartphone, a mobile phone (feature phone), a personal computer (e.g., a desktop, laptop, tablet, etc.), a media computer platform (e.g., a cable or satellite set-top box, digital video recorder), a handheld computer device (e.g., a personal digital assistant (PDA), email client, etc.), a wearable device (e.g., a glasses-type device, a watch-type device, etc.), or another type of computer or communication platform. It is preferable that the user terminal 300 be a transparent head-mounted display or a non-transparent head-mounted display.

[0089] 25, an example of a hardware configuration in which the wallet providing server 100, virtual space management server 200, user terminal 300, and retailer terminal 400 are realized by a computer 1000 will be described. Note that the various functions of the wallet providing server 100, virtual space management server 200, user terminal 300, and retailer terminal 400 can be realized by dividing them into multiple devices.

[0090] 25 is a diagram showing an example of the hardware configuration of a computer. As shown in FIG. 25, the computer 1000 includes, for example, a processor 101, a memory 1002, a storage device 1003, an input I / F unit 1004, a data I / F unit 1005, a communication I / F unit 1006, a display unit 1007, and a sensor group 1008. Note that the sensor group 1008 is hardware provided in the user terminal 300, and does not necessarily have to be provided in the wallet providing server 100, the virtual space management server 200, the user terminal 300, or the retailer terminal 400.

[0091] The processor 1001 is a control unit that controls various processes in the computer 1000 by executing programs stored in the memory 1002 .

[0092] The memory 1002 is a storage medium such as a RAM (Random Access Memory), etc. The memory 1002 temporarily stores the program code of the program executed by the processor 1001 and data required when the program is executed.

[0093] The storage device 1003 is a non-volatile storage medium such as a hard disk drive (HDD), flash memory, etc. The storage device 1003 stores an operating system and various programs for realizing the above-mentioned components.

[0094] The input I / F unit 1004 is a device for receiving input from a user. Specific examples of the input I / F unit 1004 include a keyboard, a mouse, a touch panel, various sensors, and a wearable device. The input I / F unit 1004 may be connected to the computer 1000 via an interface such as a USB (Universal Serial Bus).

[0095] The data I / F unit 1005 is a device for inputting data from outside the computer 1000. A specific example of the data I / F unit 1005 is a drive device for reading data stored in various storage media. The data I / F unit 1005 may be provided outside the computer 1000. In this case, the data I / F unit 1005 is connected to the computer 1000 via an interface such as a USB.

[0096] The communication I / F unit 1006 is a device for performing data communication via the Internet N, either wired or wirelessly, with devices external to the computer 1000. The communication I / F unit 1006 may be provided outside the computer 1000. In this case, the communication I / F unit 1006 is connected to the computer 1000 via an interface such as a USB.

[0097] The display unit 1007 is a device for displaying various types of information. Specific examples of the display unit 1007 include a liquid crystal display, an organic EL (Electro-Luminescence) display, and a display of a wearable device. The display unit 1007 may be provided outside the computer 1000. In this case, the display unit 1007 is connected to the computer 1000 via, for example, a display cable. Furthermore, when a touch panel is adopted as the input I / F unit 1004, the display unit 1007 can be configured as an integral part of the input I / F unit 1004.

[0098] In the above, when the display unit 1007 of the user terminal 300 is referred to, the display unit 1007 includes a liquid crystal display panel. If the user terminal 300 is a transparent head-mounted display, the real space is not displayed on the display unit 1007. If the user terminal 300 is a non-transmissive head-mounted display or a smartphone, an image obtained by superimposing a captured image of the real space on an image of the virtual reality space may be generated and displayed on the display unit 1007. If the user terminal 300 is a transparent head-mounted display or a non-transmissive head-mounted display, the display unit 1007 may include two display panels and display a stereo image.

[0099] The sensor group 1008 includes various sensors. The sensor group 1008 includes sensors for grasping the space around the user, sensors for grasping the user's position, posture, and hand movements, and sensors for detecting the user's viewpoint and line of sight. The sensors for grasping the space around the user include an RGB image sensor and a depth sensor. The sensors for detecting the user's position, posture, hand movements, and line of sight include a gyro sensor, a digital compass, an acceleration sensor, a proximity sensor, and a motion tracking sensor.

[0100] ===Summary=== The virtual space payment system 10 of this embodiment includes a display processing unit 140 that executes processing to display, on the display unit 1007 of the user terminal 300 (display device), a user interface related to three-dimensionally displayed payment methods available to the user for making a payment for selected goods, which are goods selected by a selection operation by a user wearing a user terminal 300 (display device) that displays the augmented reality space or the virtual reality space for three-dimensionally displayed product objects located in the augmented reality space or the virtual reality space, in the augmented reality space or the virtual reality space, and a payment unit 150 that completes an operation related to the payment for the selected goods in the augmented reality space or the virtual reality space in response to a selection operation by the user on the user interface related to the payment method. This allows the operation related to the payment to be completed in the augmented reality space or the virtual reality space, thereby preventing the user from losing their sense of immersion in the virtual space.

[0101] In addition, in the virtual space payment system 10 of this embodiment, the settlement unit 150 completes the operation related to the settlement of the selected product by the user in the augmented reality space or the virtual reality space based on the token number (identification information) that conceals the identification code that identifies the payment method. This allows the user to enjoy a secure purchase process.

[0102] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying identification objects U1 (predetermined identification signs) in the augmented reality space or virtual reality space for stores that accept payment methods. This allows users to easily find stores that accept their own payment methods, improving user convenience.

[0103] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays a linear object U2 (linear object) that moves in response to the user's hand movements recognized by the user terminal 300 (display device) in the augmented reality space or virtual reality space, and if there is a designated product object that overlaps or intersects with the linear object U2 (linear object), executes processing to bring the designated product object closer to the user in response to the user's pinch operation. This allows the user to check the product from various angles at hand without getting close to it, improving user convenience.

[0104] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays, in the augmented reality space or virtual reality space, display objects U4 that display multiple related products related to the product of the designated product object in a predetermined display area, each of the multiple related products. The multiple display objects U4 are displayed in a line away from the user so that at least a portion of each of the multiple display objects U4 overlaps, and executes processing to move the multiple display objects U4 in response to the user's hand movement relative to the multiple display objects U4. This improves the user's visibility of the products in the virtual space. Furthermore, the system allows the user to view the products through intuitive operations.

[0105] In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for reducing and displaying the selected product object (product display object U6 in this case) within a predetermined area in response to a user's pinching action of the selected product object, which is recognized on the user terminal 300 (display device) in the augmented reality space or virtual reality space. This allows the user to select products through intuitive operations.

[0106] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays, in the augmented reality space or virtual reality space, a plurality of selected products set as candidates for purchase by a user's selection operation on the selected product objects, on product display objects U6, and each of the plurality of product display objects U7 is displayed in a line surrounding the torso of the user or the user's avatar, and executes processing to move the plurality of product display objects U7 in accordance with the user's hand movements with respect to the plurality of product display objects U7. This makes it easier for the user to view the selected products, thereby improving user convenience.

[0107] In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to hide the product display object U7 on which the selected product is displayed so that the selected product is removed from the selection list of purchase candidates in response to a user action such as moving the selected product displayed on the product display object U7 recognized on the user terminal 300 (display device) out of a predetermined range in the augmented reality space or virtual reality space. This allows the user to cancel the selection of the product by an intuitive operation.

[0108] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying, in the augmented reality space or virtual reality space, a plurality of icons U8 constituting a menu UI (menu objects) that are displayed so as to surround at least one hand of the user as recognized by the user terminal 300 (display device). This allows the user to easily grasp the items included in the menu, thereby improving user convenience.

[0109] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 displays, in the augmented reality space or virtual reality space, an instruction object U9 that indicates one of the icons U8 included in the menu UI (menu object), and executes processing to change the icon U8 indicated by the instruction object U9 in response to the user's wrist twisting action in one direction, as recognized by the user terminal 300 (display device). This allows the user to select a menu item through an intuitive operation.

[0110] In the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing to display, in the augmented reality space or virtual reality space, user interfaces related to payment methods in the shape of cards corresponding to the payment methods in the real space, and to display multiple card-shaped objects U10 (user interfaces) in a row in one direction so that some of the objects overlap, thereby enabling the user to select a payment method through intuitive operations.

[0111] In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying a user interface for a payment method in the augmented reality space or virtual reality space in a shape (here, object U12) corresponding to the type of asset, thereby enabling the user to select a payment method through intuitive operations.

[0112] In the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying an input object U13 representing a combination lock for allowing the user to input secret information in the augmented reality space or virtual reality space, and the settlement unit 150 executes the user's settlement based on information input to the input object in response to the user's hand movements recognized on the display device. This allows the user to execute settlement through intuitive operations.

[0113] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying a slide object U16 that slides in the augmented reality space or the virtual reality space in response to the user's hand movements recognized on the user terminal 300 (display device), and the settlement unit 150 executes the user's payment based on information input to the slide object U16 in response to the user's hand movements recognized on the user terminal 300 (display device). This allows the user to execute payment through intuitive operations.

[0114] In addition, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for displaying a slide object U14 on one palm of the user, which is recognized by the user terminal 300 (display device), in the augmented reality space or the virtual reality space. This allows the user to perform payment by intuitive operation.

[0115] Furthermore, the virtual space payment system 10 of this embodiment further includes a product classification unit 160 that classifies information about purchased products, which are products paid for by the settlement unit 150, into one of a plurality of categories, and the display processing unit 140 displays a plurality of folders corresponding to the plurality of categories in the augmented reality space or virtual reality space, and in response to a user's selection operation on one of the plurality of folders, executes processing to display a purchased product object (here, 3D image data of clothing), which is a purchased product classified into the category corresponding to the folder that received the selection operation. This improves user operability.

[0116] Furthermore, in the virtual space payment system 10 of this embodiment, the display processing unit 140 executes processing for combining and displaying, in the augmented reality space or virtual reality space, a plurality of purchase item objects (here, 3D image data of clothing) according to product categories, based on information about the purchase item objects (here, 3D image data of clothing) stored in the product classification unit 160. This allows the user to view the purchase items as if they were being viewed in real space, thereby improving user convenience.

[0117] In addition, in the virtual space payment system 10 of this embodiment, the product classification unit 160 stores information about the user's clothing, which is the purchased product, and the display processing unit 140 executes processing to hang and display clothing objects, which are clothing, in the wardrobe U19 so that they can be viewed by the user in the augmented reality space or virtual reality space. This allows the user to shop while checking how the new clothing coordinates with the clothes they currently own, thereby improving user convenience. [Explanation of symbols]

[0118] 10...Virtual space payment system, 100...Wallet providing server, 200...Virtual space management server, 300...User terminal, 400...Retailer terminal.

Claims

1. a display processing unit that executes processing to display, on a display unit of a display device, a user interface relating to a three-dimensionally displayed payment means that can be used by a user to make a payment for a selected commodity, which is a commodity selected by a selection operation by the user on a three-dimensionally displayed commodity object that is located in the augmented reality space or the virtual reality space, which is a virtual three-dimensional space; a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with the display processing unit executes processing for displaying, in the augmented reality space or the virtual reality space, an identification sign that indicates that the payment means available to the user is available at a store where the payment means available to the user can be used for payment; Information processing system.

2. The display processing unit executes a process for displaying the identification sign for the store when the distance between the user and the store falls within a predetermined range. The information processing system according to claim 1 .

3. the payment unit completes an operation related to payment for the selected product by the user in the augmented reality space or the virtual reality space based on identification information obtained by concealing an identification code that identifies the payment means. The information processing system according to claim 1 .

4. the display processing unit executes a process for reducing and displaying the selected product object within a predetermined area in response to an action of the user pinching the selected product object, which is recognized on the display device, in the augmented reality space or the virtual reality space. The information processing system according to any one of claims 1 to 3.

5. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with The display processing unit In the augmented reality space or the virtual reality space, displaying on a product display object a plurality of selected products that have been set as candidates for purchase by a selection operation by the user on a selected product object that indicates the selected products; displaying the plurality of product display objects in a series so as to surround a body of the user or an avatar of the user; In response to an action of the user moving the selected product displayed in the product display object out of a predetermined range, which is recognized by the display device, a process is executed to hide the product display object on which the selected product is displayed so that the selected product is not set as a candidate for purchase. Information processing system.

6. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with The display processing unit displaying, in the augmented reality space or the virtual reality space, a plurality of icons constituting a menu object displayed so as to surround at least one hand of the user recognized on the display device; displaying an instruction object that indicates one of the icons included in the menu object, and executing a process for changing the icon indicated by the instruction object in accordance with a twisting motion of the user's wrist in one direction that is recognized by the display device; Information processing system.

7. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with the display processing unit executes processing to display, in the augmented reality space or the virtual reality space, a user interface related to the payment means in the shape of a card corresponding to the payment means in the real space, and to display a plurality of the card-shaped user interfaces in a row in one direction so that parts of the user interfaces overlap with each other. Information processing system.

8. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with the display processing unit executes processing for displaying a user interface related to the payment method in a shape corresponding to the type of asset in the augmented reality space or the virtual reality space. Information processing system.

9. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with the display processing unit executes processing for displaying, in the augmented reality space or the virtual reality space, an input object representing a combination lock for prompting the user to input secret information; the payment unit executes payment by the user based on information input to the input object in response to a hand movement of the user recognized by the display device. Information processing system.

10. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; Equipped with the display processing unit executes processing for displaying, in the augmented reality space or the virtual reality space, a slide object that slides in response to a hand movement of the user recognized by the display device; the settlement unit executes the settlement of the user based on information input to the slide object in accordance with the hand movement of the user recognized on the display device. Information processing system.

11. the display processing unit executes processing for displaying the slide object on one palm of the user recognized by the display device in the augmented reality space or the virtual reality space. The information processing system according to claim 10.

12. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected products, which are products selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; a product classification unit that classifies information about purchased products that have been paid for by the payment unit into one of a plurality of categories; Equipped with The display processing unit displaying a plurality of folders corresponding to the plurality of categories in the augmented reality space or the virtual reality space; executes a process for displaying, in response to a selection operation by the user on one of the plurality of folders, a purchase item object that is the purchase item classified into a category corresponding to the folder for which the selection operation has been accepted; Information processing system.

13. the display processing unit executes processing for combining and displaying a plurality of the purchase item objects in accordance with the product categories in the augmented reality space or the virtual reality space based on information about the purchase item objects stored in the product classification unit. The information processing system according to claim 12.

14. A display processing unit that executes processing to display on a display unit of a display device a user interface relating to a three-dimensionally displayed payment means available to a user for making payment for selected goods, which are goods selected by a user's selection operation on a three-dimensionally displayed product object placed in an augmented reality space or a virtual reality space that is a virtual three-dimensional space; and a payment unit that completes an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment means; a product classification unit that classifies information about purchased products that have been paid for by the payment unit into one of a plurality of categories; Equipped with the product classification unit stores information about the user's clothing, which is the purchased product; the display processing unit executes processing for displaying a clothing object, which is the clothing, in the augmented reality space or the virtual reality space by hanging it from a wardrobe so that it is visible to the user; Information processing system.

15. The computer Execute a process for displaying, on a display unit of a display device, a user interface relating to a three-dimensionally displayed payment means available to a user for making a payment for a selected commodity, which is a commodity selected by a selection operation by a user on a three-dimensionally displayed commodity object located in an augmented reality space or a virtual reality space, which is a virtual three-dimensional space; completing an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment method; In the augmented reality space or the virtual reality space, displaying an identification sign indicating that the payment method available to the user is available at a store where the payment method is available to the user; An information processing method that performs the above.

16. On the computer, Execute a process for displaying, on a display unit of a display device, a user interface relating to a three-dimensionally displayed payment means available to a user for making a payment for a selected commodity, which is a commodity selected by a selection operation by a user on a three-dimensionally displayed commodity object located in an augmented reality space or a virtual reality space, which is a virtual three-dimensional space; completing an operation related to payment for the selected product in the augmented reality space or the virtual reality space in response to a selection operation by the user on a user interface related to the payment method; In the augmented reality space or the virtual reality space, displaying an identification sign indicating that the payment method available to the user is available at a store where the payment method is available to the user; A program that executes the following.

Images