Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program, which complete merchandise selection operation more intuitively in an augmented reality space or a virtual reality space.SOLUTION: A method executes: processing for displaying an instruction object U2 for instructing a three-dimensionally displayed merchandise object, which moves according to a motion of a hand of a user wearing a display device for displaying an augmented reality space or a virtual reality space, on a display section of a display device; and processing for moving a designated merchandise object close to the user by pinch operation of the user and displaying it when there is a designated merchandise object that is a merchandise object overlapping or intersecting with the instruction object. The method completes the operation concerning the settlement of selected merchandise according to the user's selection operation to a user interface concerning settlement means for carrying out the settlement of the selected merchandise, which is merchandise corresponding to the designated merchandise object selected by the user's selection operation.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] A system that enables a purchase procedure for a product in a virtual space has been disclosed (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The system described in Patent Document 1 is configured such that a user can perform actions such as putting an object into a virtual cart (cart-in) or indicating that they like a product (liking). Further, the system starts an action such as cart-in or "liking" triggered by a predetermined gesture such as quickly bending the user's index finger twice, enabling the purchase of a product. However, in this system, there is more room for improvement in the actions that trigger the user to select a product and the detection process of such actions in order to ensure a deeper sense of immersion of the user in the virtual space.

[0005] Therefore, an object of the present invention is to provide a system that more intuitively completes a product selection operation in an extended reality space or a virtual reality space in order to solve the above problems.

Means for Solving the Problems

[0006] An information processing system according to one aspect of the present invention executes a process for causing a display unit of a display device to display an instruction object for instructing a product object three-dimensionally displayed in the augmented reality space or the virtual reality space, which moves according to a hand motion of a user wearing the display device that displays the augmented reality space or the virtual reality space which is a virtual three-dimensional space. When there is a designated product object which is a product object overlapping or intersecting with the instruction object, a display processing unit executes a process for causing the designated product object to be displayed closer to the user by a pinch operation of the user, and a settlement unit completes an operation related to settlement of a selected product which is a product corresponding to the designated product object selected by a selection operation by the user, in response to the selection operation by the user on a user interface related to a settlement means for settling the selected product.

[0007] An information processing method according to one aspect of the present invention includes a computer executing a process for causing a display unit of a display device to display an instruction object for instructing a product object three-dimensionally displayed in the augmented reality space or the virtual reality space, which moves according to a hand motion of a user wearing the display device that displays the augmented reality space or the virtual reality space which is a virtual three-dimensional space, and when there is a designated product object which is a product object overlapping or intersecting with the instruction object, executing a process for causing the designated product object to be displayed closer to the user by a pinch operation of the user, and completing an operation related to settlement of a selected product which is a product corresponding to the designated product object selected by a selection operation by the user, in response to the selection operation by the user on a user interface related to a settlement means for settling the selected product.

[0008] A program according to an aspect of the present invention causes a computer to execute a process for displaying, on a display unit of a display device, an instruction object for instructing a product object three-dimensionally displayed in an augmented reality space or a virtual reality space that moves according to an operation of a hand of a user wearing the display device for displaying the augmented reality space or the virtual reality space, which is a virtual three-dimensional space; when there is a designated product object that is a product object overlapping or intersecting with the instruction object, execute a process for displaying the designated product object closer to the user by a pinch operation of the user; and complete an operation related to settlement of a selected product in response to a selection operation by the user on a user interface related to a settlement means for settling a selected product that is a product corresponding to the designated product object selected by the selection operation of the user.

Advantages of the Invention

[0009] According to the present invention, since product selection in a virtual space can be performed by a gesture more similar to a product selection action in the real world, a more intuitive operation in the virtual space realizes a product purchase experience, thereby providing a deeper sense of immersion to the user.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Embodiments for Carrying Out the Invention

[0011] Hereinafter, the virtual space payment system 10 in an embodiment of the present invention will be described in detail with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention to exclude various modifications and applications of technologies not explicitly described below. That is, the present invention can be implemented with various modifications or by combining each embodiment without departing from its gist. In addition, in the description of the following drawings, the same or similar parts are denoted by the same or similar reference numerals. ===Overview of the Virtual Space Payment System 10===

[0012] The virtual space payment system 10 is a system that completes the purchase procedure of products or the like in an extended reality space (hereinafter referred to as the "extended reality space") or a virtual three-dimensional space (hereinafter referred to as the "virtual reality space") projected onto a terminal device such as goggles worn by a user, without losing the sense of immersion in the space. Hereinafter, for convenience, the extended reality space and the virtual reality space may be collectively referred to as the "virtual space". That is, the virtual space payment system 10 can complete the purchase procedure of a 3D avatar by the user without detaching the user from the virtual space (for example, without removing the terminal device such as goggles). Thereby, the virtual space payment system can prevent the user from losing the sense of immersion in the virtual space.

[0013] The extended reality space is a space including an image that is displayed so as to be projected onto the real space, for example, when a user wears a terminal device that is a transmissive head-mounted display. The virtual reality space is a virtual space composed of images displayed on a terminal device that is a non-transmissive head-mounted display worn by a user, for example. Note that the virtual reality space includes a so-called mixed reality space (a space in which information of the real world is reflected in the digital space). The image displayed on the terminal device is, for example, an image that a user visually recognizes as a three-dimensional image. Hereinafter, among the images displayed on the terminal device, an object that a user can visually recognize as an actual object in the virtual space may be referred to as a "virtual object".

[0014] First, referring to FIG. 1, an overview of the configuration of the in-virtual-space payment system 10 will be described. FIG. 1 is a diagram showing an example of the configuration of the in-virtual-space payment system 10. As shown in FIG. 1, the in-virtual-space payment system 10 may include, for example, a wallet providing server 100, a virtual space management server 200, a user terminal 300, and a retail store terminal 400.

[0015] The wallet providing server 100 is a server that processes information for realizing a user's purchase procedure in a virtual space. The wallet providing server 100 processes information for a retail store to open a store in the virtual space (for example, information regarding a virtual object of the store, information regarding a virtual object of a product, etc.), and processes information regarding a user interface for realizing a user's purchase procedure in the virtual space (for example, information regarding a virtual object related to payment). The wallet providing server 100 may provide the information for opening a store and the information regarding the user interface to the virtual space management server 200 described later with a software development kit (hereinafter referred to as "SDK").

[0016] Thereby, the in-virtual-space payment system 10 can easily realize allowing a retail store to open a store and allowing a user to perform a purchase procedure in the virtual space. Further, the operator who operates the virtual space can easily realize store opening by a retail store and a purchase procedure by a user with the SDK, so that the retail store can be incorporated and consumer participation can be promoted. Thereby, the in-virtual-space payment system 10 can give an economic effect of expanding the business area to the operator.

[0017] The virtual space management server 200 is, for example, a server for providing a virtual space to a user. The user terminal 300 provides a virtual space to the user based on the information provided by the virtual space management server 200. The virtual space management server 200 causes the user terminal 300 to display a predetermined user interface based on, for example, information regarding the user interface provided by the wallet providing server 100.

[0018] The user terminal 300 is, for example, a non-transmissive head-mounted display or a transmissive head-mounted display, etc., and is a device that enables a user to visually recognize a virtual object in a virtual space. The user terminal 300 has a function of recognizing the operations of the user.

[0019] The retail store terminal 400 is, for example, a device that transmits information regarding stores opened and products displayed in a virtual space. The retail store terminal 400 generates various types of information, for example, by receiving an operation input by an operator of the retail store.

[0020] Note that in the above description, the wallet providing server 100 and the virtual space management server 200 are described as being separately provided, but the present invention is not limited thereto. The wallet providing server 100 and the virtual space management server 200 may be configured by one server, or may be a cloud server. Further, the wallet providing server 100 may be configured by, for example, a plurality of servers divided according to functions.

[0021] Next, with reference to FIG. 2, an overview of the processing of the in-virtual-space payment system 10 will be described. FIG. 2 is a diagram showing an example of an overview of the processing of the in-virtual-space payment system 10. First, in step S1, the retail store terminal 400 receives, for example, a user's operation input for a web page provided by the wallet providing server 100, and transmits information about the store (hereinafter referred to as "store information") and information about the product (hereinafter referred to as "product information") to the wallet providing server 100. Here, although not shown, the wallet providing server 100 may obtain 3D data of the store and the product from the operator by providing the store information and the product information to the operator who creates 3D data. Note that the wallet providing server 100 may automatically obtain 3D data of the store and the product from the operator by providing the store information and the product information to the operator, or may not.

[0022] Next, in step S2, the wallet providing server 100 transmits, for example, the store information and the product information to the e-commerce server 500. Thereby, when the wallet providing server 100 executes a procedure for the user to purchase a product provided by the retail store in the virtual space, a transfer to the retail store becomes possible.

[0023] Next, in step S3, the wallet providing server 100 transmits, for example, the store information, the product information, and the 3D data of the store and the product to the virtual space management server 200. Thereby, the retail store can open a store in the virtual space and display products with a simple operation input. It is possible to eliminate the barrier that it was difficult to prepare 3D data of the store and products when the retail store opened a store in the virtual space.

[0024] Next, in step S4, the user terminal 300 installs, for example, application software (hereinafter referred to as "virtual space APP") for displaying the virtual space on the display unit 1007 of the user terminal 300. Note that the user terminal 300 may display the virtual space on the display unit 1007 through a web browser without using the virtual space APP.

[0025] Next, in step S5, the user terminal 300 installs application software (hereinafter referred to as the "wallet APP") for allowing the user to use settlement means for purchasing goods in the virtual space, for example.

[0026] Next, in step S6, the user terminal 300 transmits information about the user (hereinafter referred to as "user information") obtained based on the user's operation input through the wallet APP, for example, to the wallet providing server 100. The user information may include, for example, information about the settlement means used by the user (hereinafter referred to as "settlement means information").

[0027] Next, in step S7, the wallet providing server 100 identifies a user interface corresponding to the settlement means used by the user (hereinafter referred to as the "wallet UI") based on the user information, for example. The wallet providing server 100 transmits information about the user information and the wallet UI to the virtual space management server 200. Thereby, the user can purchase goods using the desired settlement means in the virtual space.

[0028] Next, in step S8, the user terminal 300 receives, for example, the user's operation input for adding the goods of the store in the virtual space to the cart. The user terminal 300 receives, for example, the user's operation input for selecting the goods in the cart and executes the purchase procedure. Hereinafter, for convenience, the operation until the goods are selected by the user and the purchase procedure is executed is referred to as the "purchase operation", and the information for executing the settlement by the purchase operation is referred to as the "settlement execution information". The user interface provided to the user through the user terminal 300 in the purchase operation will be described later.

[0029] Next, in step S9, the virtual space management server 200 transmits information about the purchased product (hereinafter referred to as "purchase information") to the wallet providing server 100. The purchase information includes, for example, information about the product that is the target of the purchase operation by the user. Next, in step S10, the wallet providing server 100 transmits the purchase information to the e-commerce server 500. As a result, the user can complete the purchase procedure without detaching the user terminal 300.

[0030] Next, in step S11, the e-commerce server 500 performs a transfer process of the sales amount of the product for which the user has executed the purchase procedure to the retail store. As a result, the retail store can obtain the sales amount in the virtual space.

[0031] In this way, in the in-virtual-space payment system 10, for example, even if there are a plurality of virtual space management servers 200, each virtual space management server 200 can transmit purchase information to an appropriate e-commerce server 500 through the wallet providing server 100. In other words, since the user can complete the transaction, for example, within a VR (Virtual Reality) SNS (Social Networking Service) without detaching the user terminal 300, the problem of losing the user's sense of immersion in the virtual space can be solved. ===Configuration of Wallet Providing Server 100 ===

[0032] Next, with reference to FIG. 3, the configuration of the wallet providing server 100 will be described. FIG. 3 is a diagram showing an example of the configuration of the wallet providing server 100. As shown in FIG. 3, the wallet providing server 100 includes functional units such as a storage unit 110, an acquisition unit 120, a UI information providing unit 130, a display processing unit 140, a settlement unit 150, and a product classification unit 160. Each functional unit is a function realized by, for example, a processor 1001 reading a program stored in a memory 1002.

[0033] The storage unit 110 stores user information D111, retail store information D112, and UI information D113.

[0034] First, referring to FIG. 4, user information D111 will be described. FIG. 4 is a diagram showing the configuration of user information D111. As shown in FIG. 4, the user information D111 may include items such as [User ID], [Name], [Email Address], [Address], [Phone Number], [Payment Method], [Purchased Items], and [UIID]. [User ID] stores a user ID that can uniquely identify the user. [Name] stores the user's name. [Email Address] stores the email address used by the user. [Address] stores the user's current address. [Phone Number] stores the user's phone number. [Payment Method] stores the payment methods used by the user in the virtual space. Here, the user with the user ID "U0001" has registered credit card A, bank account B, virtual currency C, and electronic money D as payment methods. The wallet providing server 100 generates information regarding the wallet UI associated with the user with the user ID "U0001". [Purchased Items] stores data regarding the items purchased by the user in the virtual space in the past. [UIID] stores a UIID that can uniquely identify various UIs.

[0035] Next, referring to FIG. 5, retail store information D112 will be described. FIG. 5 is a diagram showing the configuration of retail store information D112. As shown in FIG. 5, the retail store information D112 may include items such as [Retail Store ID], [Retail Store Name], [Email Address], [Address], [Phone Number], [Store 3D Data], [Product 3D Data], and [Sales History]. [Retail Store ID] stores a retail store ID that can uniquely identify the retail store. [Retail Store Name] stores the name of the retail store. [Email Address] stores the email address used by the retail store. [Address] stores the current location of the retail store. [Phone Number] stores the phone number of the retail store. [Store 3D Data] stores 3D data regarding the store of the retail store. [Product 3D Data] stores 3D data regarding the products associated with the retail store. [Sales History] stores the sales history of the products of the retail store in the virtual space.

[0036] Next, with reference to FIG. 6, the UI information D113 will be described. FIG. 6 is a diagram showing the configuration of the UI information D113. As shown in FIG. 6, the UI information D113 may include items such as [UIID], [UI name], [UI type], [UI data], and [virtual space ID], for example. [UIID] stores a UIID that can uniquely identify various UIs. [UI name] stores the names of various UIs. [UI type] stores the types of UIs, such as a UI for identifying a store, a UI for checking a product, a UI for executing a payment method, a UI for checking purchased products, and a UI related to a menu. [UI data] stores data for displaying a UI in a virtual space. [Virtual space ID] stores a virtual space ID that can uniquely identify the virtual space corresponding to the UI.

[0037] The acquisition unit 120 acquires information from other devices such as, for example, the virtual space management server 200, the user terminal 300, the retail store terminal 400, and the e-commerce server 500.

[0038] The UI information providing unit 130 generates information regarding a UI related to a purchase operation by the user (hereinafter referred to as "UI information"). The UI related to the purchase operation generates, for example, a UI for identifying a store in a virtual space (hereinafter referred to as "store identification UI"), a UI for checking a product (hereinafter referred to as "product confirmation UI"), a UI for sorting products in a cart (hereinafter referred to as "product sorting UI"), a wallet UI for executing a payment method, a UI for checking purchased products (hereinafter referred to as "purchased product confirmation UI"), and a UI related to a menu (hereinafter referred to as "menu UI"). The information regarding the UI generated by the UI information providing unit 130 is transmitted to the virtual space management server 200. Thereby, the virtual space management server 200 can provide a UI corresponding to the user to the user terminal 300 in the virtual space. Hereinafter, for convenience, when referring to "the user's hand", it refers to the hand of the avatar corresponding to the user in the virtual reality space and the user's real hand in the augmented reality space.

[0039] First, with reference to FIG. 7, an example of a store identification UI will be described. FIG. 7 is a diagram showing an example of a store identification UI in a virtual space. As shown in FIG. 7, the store identification UI includes an identification object U1 that is displayed for stores in the virtual space that can be settled by a settlement means. The identification object U1 is displayed, for example, when the user approaches a store to a predetermined distance (here, "L"). The identification object U1 is an object that indicates, for example, a logo, the type of settlement means, and the like. Thereby, the in-virtual-space payment system 10 can easily find stores that can use the settlement means each user has, thus improving the convenience for the user.

[0040] Next, with reference to FIG. 8, an example of the product confirmation UI will be described. FIG. 8 is a diagram showing an example of the product confirmation UI in the virtual space. As shown in FIG. 8, the product confirmation UI may include an instruction object U2 that is displayed in the virtual space according to the movement of the user's hand recognized by the user terminal 300. Here, the instruction object U2 is, for example, an object that operates according to the movement of at least a part of the user's hand. Specifically, the instruction object U2 may be, for example, an object that linearly extends from the user's index finger. Also, the instruction object U2 may be, for example, an object that linearly extends from the part where the index finger and thumb of one of the user's hands overlap. Also, the linearly extending object may be displayed so as to penetrate the product. Also, the instruction object U2 may be, for example, a dot-shaped object indicating the intersection point between the virtual extension line indicated by the user's index finger of one hand and the product. Also, the instruction object U2 may be, for example, an object that indicates the intersection point between the virtual extension line indicated by the user's index finger of one hand and the product with a mark such as an arrow. Thereby, the payment system 10 in the virtual space can select the product with an intuitive operation without approaching the product, so that the convenience of the user can be improved. Also, when the instruction object U2 overlaps or intersects with the product, and the user makes an operation of pinching an object by overlapping the index finger and thumb of the user's hand (hereinafter referred to as "pinch operation"), the product confirmation UI brings the product closer to the user and displays the product rotatable and deformable according to the movement of the user's hand. Thereby, the payment system 10 in the virtual space can allow the user to view the product from various angles at hand without approaching the product, so that the convenience of the user can be improved. Also, the product confirmation UI may include a price display object U3 that displays the price, specifications, etc. of the product in a state where the product is specified by the instruction object U2. Thereby, the payment system 10 in the virtual space can select a product on the virtual space by a gesture more similar to the product selection behavior in the real world, so that by realizing a product purchase experience with a more intuitive operation in the virtual space, a deeper sense of immersion can be provided to the user.Furthermore, the virtual space payment system 10 allows users to easily check product prices, specifications, etc., thereby improving user convenience.

[0041] Also, as shown in FIG. 8, the product confirmation UI may include a display object U4 that displays, in a predetermined area, each of a plurality of products related to the product designated by the user. For example, when the product designated by the user is "bag A", the related products are bags of the same brand, bags of a similar shape, bags of a similar color, bags of a similar use, etc., in relation to the bag A. The display object U4 is a spherical object that displays three-dimensional image data of the product in an inner area of ​​the spherical shape. Each of the plurality of display objects U4 may be displayed in a line in a direction away from the user so that at least a portion of them overlap. This can improve the visibility of the product to the user in the virtual space. Also, the product confirmation UI may be displayed to move in response to the movement of the user's hand on the plurality of display objects U4. This allows the user to view the product by an intuitive operation.

[0042] Next, with reference to FIG. 9, an example of a product selection UI will be described. FIG. 9 is a diagram showing an example of a product selection UI in a virtual space. As shown in FIG. 9, the product selection UI includes, for example, a cart object U5 and a product display object U6 that displays the selected product. The cart object U5 is a three-dimensionally displayed object for storing the selected products. The cart object U5 is displayed so as to cover one palm of the user and moves in accordance with the movement of the palm. The product display object U6 is an object that displays the product reduced within a predetermined area when the product is moved to the cart object U5. The product display object U6 is an object that displays the three-dimensional image data of the product in the inner spherical area in response to the operation of pinching the product by the other hand of the user. The product display object U6 is displayed so as to be movable by the user while accepting the pinching operation by the user. This enables the user to select products by an intuitive operation.

[0043] Next, referring to FIG. 10, another example of the product selection UI will be described. FIG. 10 is a diagram showing another example of the product selection UI in a virtual space. As shown in FIG. 10, the product selection UI includes, for example, a product display object U7 that displays a plurality of products added to the cart by the user. The product display object U7 is, for example, an area that reduces and displays each of the products moved to the cart object U5 shown in FIG. 9 within a predetermined area. The product selection UI may, for example, display a plurality of product display objects U7 in a row so as to surround the upper body (torso) of the user's avatar. This makes it easier for the user to view the selected products, thus improving the user's convenience. Also, the product selection UI cancels the selection state of the product in response to a hand movement of the user that removes it from the row of the product display objects U7 connected in a row. This enables the user to cancel the selection state of the product by an intuitive operation. Further, the product selection UI moves the product along the row in response to a hand movement that moves the product display objects U7 connected in a row along that row. This enables the user to view the selected products by an intuitive operation.

[0044] Next, with reference to FIG. 11, an example of the menu UI will be described. FIG. 11 is a diagram showing an example of the menu UI in the virtual space. As shown in FIG. 11, the menu UI includes, for example, an icon U8 arranged so as to surround the user's hand with the thumb protruding forward. To surround the hand, for example, it includes arranging the icon U8 around the wrist once or in an arc. The icon U8 is an item in the menu (for example, an item for transitioning to checkout, cart, inventory). Thereby, since the user can easily grasp the items included in the menu, the convenience of the user can be improved. Further, the menu UI may include an instruction object U9 for instructing any one of the plurality of icons U8. The instruction object U9 is displayed, for example, so as to indicate the direction indicated by the tip of the user's thumb. The menu UI may change the icon U8 indicated by the instruction object U9, for example, in response to an operation of twisting in one direction of the user's wrist. This enables the user to select menu items by an intuitive operation.

[0045] Next, with reference to FIG. 12, an example of the first wallet UI will be described. FIG. 12 is a diagram showing an example of the first wallet UI in a virtual space. As shown in FIG. 12, the first wallet UI includes a card-shaped object U10 corresponding to a payment means in the real space in the virtual space. The first wallet UI is displayed in a row in one direction such that a part of a plurality of card-shaped objects U10 overlaps. This enables the user to select a payment means by an intuitive operation. Also, the display position of the first wallet UI may be changed according to the usage frequency of the user. Specifically, the first wallet UI may display the card-shaped objects U10 stacked on top of each other in order of decreasing usage frequency (for example, in FIG. 12, "Credit Card A" has the highest usage frequency). Thereby, the user can easily perform a selection operation on a frequently used payment means. Also, the first wallet UI may display a limit amount display object U11 including the usage limit amount of each payment means. Thereby, since the user can easily grasp the available amount of each payment means, the convenience of the user can be improved.

[0046] Next, with reference to FIG. 13, an example of the second wallet UI will be described. FIG. 13 is a diagram showing an example of the second wallet UI in a virtual space. As shown in FIG. 13, the second wallet UI includes an object U12 having a shape corresponding to the type of assets related to the payment means in the virtual space. The second wallet UI may include, for example, a wallet-shaped object corresponding to a regular account, an object shaped like a piggy bank corresponding to a savings account, a three-dimensional polygonal object corresponding to a cryptocurrency, and the like. This enables the user to select a payment means by an intuitive operation.

[0047] Next, referring to FIG. 14, an example of the third wallet UI will be described. FIG. 14 is a diagram showing an example of the third wallet UI in a virtual space. As shown in FIG. 14, the third wallet UI includes an input object U13 for allowing a user to input secret information in the virtual space. The input object U13 is, for example, a dial lock. The input object U13 is an object that accepts the input of secret information in response to an operation of turning the dial lock by the user's hand. When the validity of the secret information is confirmed, the third wallet UI causes the wallet providing server 100 to transmit settlement execution information for executing a settlement. Thereby, it is possible to cause the user to execute a settlement by an intuitive operation. Note that the third wallet UI may be an input object indicating a keypad, a signature pad, or a keyboard.

[0048] Next, referring to FIG. 15, an example of the fourth wallet UI will be described. FIG. 15 is a diagram showing an example of the fourth wallet UI in a virtual space. As shown in FIG. 15, the fourth wallet UI includes a slide object U14 that slides in response to the operation of the user's hand. The slide object U14 may be provided on a plate-shaped object U15. Further, the fourth wallet UI may display the slide object U14 on one palm of the user. The fourth wallet UI transmits settlement execution information in response to an operation of sliding the slide object U14 by the user's hand. Thereby, it is possible to cause the user to execute a settlement by an intuitive operation.

[0049] Next, referring to FIG. 16, another example of the fifth wallet UI will be described. FIG. 16 is a diagram showing another example of the fifth wallet UI in a virtual space. As shown in FIG. 16, the fifth wallet UI may include a card-shaped slide object U16 and a gate object U17 through which the slide object U16 passes. In this case, the fifth wallet UI transmits settlement execution information according to the movement of the user's hand that slides the slide object U16. Thereby, the user can execute settlement by an intuitive operation.

[0050] Next, referring to FIG. 17, an example of a purchased item confirmation UI will be described. FIG. 17 is a diagram showing an example of a purchased item confirmation UI in a virtual space. As shown in FIG. 17, the purchased item confirmation UI includes an icon U18 (for example, in the form of a folder) corresponding to the category of the item (for example, clothing, games, etc.) classified by the item classification unit described later. The purchased item confirmation UI may display the item associated with the icon U18 selected by the user, for example, in a predetermined format. Thereby, the operability of the user can be improved. The user can easily find the 3D image data of the purchased item. Also, the predetermined format may include, for example, a format in which the 3D image data of the purchased clothing is hung and displayed in a wardrobe U19 in the case of an icon U18 corresponding to the clothing category. Thereby, since the purchased item can be browsed as if it were being confirmed in the real space, the convenience of the user can be improved. Also, the purchased item confirmation UI may display, for example, a combination of a plurality of purchased items. Specifically, as shown in FIG. 17, the purchased item confirmation UI may display the clothing hung in the wardrobe U19 and applied to a humanoid object U20, which is the clothing that has received the user's selection operation. Thereby, when shopping, the user can check the coordination with the clothes they currently have, so the convenience of the user can be improved. Also, for large and highly customizable items such as cars, motorcycles, and bicycles, combinations can be considered at the time of purchase and functions can be previewed, so the convenience of the user can be improved.

[0051] The display processing unit 140 executes a process of transmitting information regarding various UIs to the virtual space management server 200 so that the virtual space management server 200 can display the various UIs generated by the UI information providing unit 130 on the display unit 1007 of the user terminal 300. Note that the display processing unit 140 of the wallet providing server 100 may directly transmit information regarding various UIs to the user terminal 300 so as to display the various UIs on the display unit 1007 of the user terminal 300. The information regarding various UIs may include a program for causing the display unit 1007 of the user terminal 300 to display the various UIs.

[0052] The settlement unit 150 completes an operation related to the settlement of a product selected by the user in the virtual space, for example, in response to a selection operation by the user on a user interface related to settlement means (for example, the first to fourth wallet UIs). Specifically, the user terminal 300 transmits settlement execution information to the virtual space management server 200 based on a selection operation by the user on, for example, the first and second wallet UIs related to settlement means, or an operation by the user on, for example, the third, fourth, and fifth wallet UIs. The settlement unit 150 executes a settlement process based on the purchase information transmitted from the virtual space management server 200 based on the settlement execution information. The settlement process is, for example, a process of transmitting purchase information including information regarding a product purchased by the user to the e-commerce server 500. When the settlement unit 150 executes the settlement process, the settlement by the user in the virtual space is completed.

[0053] The product classification unit 160 classifies information regarding a purchased product, which is a product settled by the settlement unit 150, into one of a plurality of categories. The product classification unit 160 stores the information regarding the classified purchased product in the user information D111. ===Configuration of Virtual Space Management Server 200===

[0054] Next, with reference to FIG. 18, the configuration of the virtual space management server 200 will be described. FIG. 18 is a diagram showing an example of the configuration of the virtual space management server 200. As shown in FIG. 18, the virtual space management server 200 includes, for example, functional units such as a storage unit 210, an acquisition unit 220, a display processing unit 230, and a purchase information transmission unit 240. Each functional unit is, for example, a function realized by the processor 1001 reading out a program stored in the memory 1002.

[0055] The storage unit 110 stores, for example, store space information D211, product arrangement information D212, product information D213, action information D214, UI information D215, and user information D216. Note that since the UI information D215 and the user information D216 are the same as the UI information D113 and the user information D111, their descriptions are omitted.

[0056] First, with reference to FIG. 19, the store space information D211 will be described. FIG. 19 is a diagram showing an example of the store space information D211. The store space information D211 is a database that stores three-dimensional data of a store in the virtual space. The three-dimensional data of the store includes data related to the three-dimensional shape of a room composed of walls, floors, ceilings, product display shelves, and the like. As shown in FIG. 19, the store space information D211 may include, for example, items such as [store space ID], [store space name], and [update history]. The [store space ID] stores a store space ID that can uniquely identify a store in the virtual space. The [store space name] stores the name of the store space. The name of the point p space may be, for example, the store name. The [update history] stores the history of updates to the store space.

[0057] Next, referring to FIG. 20, the merchandise arrangement information D212 will be described. FIG. 20 is a diagram showing an example of the merchandise arrangement information D212. The merchandise arrangement information D212 stores the arrangement of objects. The object is an object within the virtual space that the user pays attention to. As shown in FIG. 20, the merchandise arrangement information D212 may include, for example, items such as [store space ID], [object ID], [3D coordinates], [scale], and [update date and time]. The [store space ID] stores the store space ID. The [object ID] stores the object ID of the object arranged within the virtual space. The [3D coordinates] stores the coordinates (x, y, z) indicating the position of the object arranged in the store within the virtual space. The [scale] stores the scale when displaying the object within the virtual space. The [update date and time] stores the update date and time of each record.

[0058] Next, referring to FIG. 21, the merchandise information D213 will be described. FIG. 21 is a diagram showing an example of the merchandise information D213. The merchandise information D213 stores attributes of objects and the like. As shown in FIG. 21, the merchandise information D213 may include, for example, items such as [object ID], [product name], [price], and [update date and time]. The [object ID] stores the object ID of the corresponding object. The [product name] stores the product name. The [price] stores the price of the product. The [update date and time] stores the update date and time of each record.

[0059] Next, referring to FIG. 22, action information D214 will be described. FIG. 22 is a diagram showing an example of the action information D214. The action information D214 stores actions corresponding to user operations. As shown in FIG. 22, the action information D214 may include items such as [Action ID], [Action Name], [Recognized Action], [UIID], and [Execution Date and Time], for example. [Action ID] stores an action ID that can uniquely identify the type of action. [Action Name] stores the name of the action. [Recognized Action] stores information corresponding to the user's operation recognized by the user terminal 300. Specifically, the recognized user operation includes, for example, the movements of the user's hand shown in FIGS. 8, 9, 10, 14, 15, 16, and 11. [UIID] stores a UIID that can uniquely identify various UIs. [Execution Date and Time] stores the date and time when the action was executed by the user.

[0060] The acquisition unit 220 acquires information from other devices such as the wallet providing server 100 and the user terminal 300, for example. The various information acquired by the acquisition unit 220 is stored in the storage unit 210.

[0061] The display processing unit 230 executes a process of displaying, for example, three-dimensional image data related to the virtual space stored in the storage unit 210 on the display unit 1007 of the user terminal 300. Specifically, the display processing unit 230 displays a virtual space including a UI as shown in FIGS. 7 to 17 on the display unit 1007 of the user terminal 300.

[0062] When the purchase information transmission unit 240 acquires settlement execution information from the user terminal 300, for example, it identifies the product that is the target of the purchase operation. The purchase information transmission unit 240 transmits purchase information related to the product that is the target of the purchase operation to the wallet providing server 100. ===User Terminal 300===

[0063] Next, with reference to FIG. 23, the configuration of the user terminal 300 will be described. FIG. 23 is a diagram showing an example of the configuration of the user terminal 300. As shown in FIG. 23, the user terminal 300 includes functional units such as a storage unit 310, an acquisition unit 320, an action recognition unit 330, and a display processing unit 340. Each functional unit is a function realized, for example, by the processor 1001 reading out a program stored in the memory 1002.

[0064] The storage unit 310 stores, for example, user information D311 and action information D312. Since the user information D311 is the same as the user information D111, its description will be omitted. Since the action information D312 is the same as the action information D214, its description will be omitted. Note that the storage unit 310 may store information similar to the UI information D215. The acquisition unit 320 acquires information from other devices such as the wallet providing server 100 and the virtual space management server 200, for example. The various types of information acquired by the acquisition unit 320 are stored in the storage unit 310. The action recognition unit 330 recognizes the user's action based on the information acquired by the sensor group 1008 provided in the user terminal 300, for example. Specifically, the action recognition unit 330 refers to the action information D312 and generates information related to the action (hereinafter referred to as "action information"). The user terminal 300 transmits the action information to the virtual space management server 200. Thereby, a UI corresponding to the user's action can be displayed on the display unit 1007. The display processing unit 340 causes the display unit 1007 to display, for example, three-dimensional image data related to the virtual space acquired from the virtual space management server 200 and three-dimensional image data related to various UIs. ===Processing of the Virtual Space Payment System 10===

[0065] Referring to FIG. 24, an example of the processing of the in-virtual-space payment system 10 will be described. FIG. 24 is a flowchart showing an example of the processing of the in-virtual-space payment system 10. Hereinafter, since a retail store opens a store in the virtual space, an example of the processing flow from when a user finds the retail store in the virtual space, purchases clothes at the retail store, and completes the settlement process will be described. Also, hereinafter, reference will be made to FIGS. 7 to 17 as appropriate.

[0066] In step S100, the retail store terminal 400 of the retail store transmits store information and product information to the wallet providing server 100. The wallet providing server 100 may create 3D image data of the store and products based on the store information and product information. The wallet providing server 100 associates the store ID with the store information, product information, and 3D image data of the store and products and stores them in the storage unit 110.

[0067] Next, in step S101, the wallet providing server 100 transmits the store information, product information, and 3D image data of the store and products to the virtual space management server 200. At this time, the wallet providing server 100 may transmit the store information, product information, and 3D image data of the store and products together with a program for applying them to the virtual space provided by the virtual space management server 200. That is, the wallet providing server 100 may transmit the store information, product information, and 3D image data of the store and products to the virtual space management server 200 by means of an SDK. Thereby, the retail store can easily open a store in the virtual space.

[0068] Next, in step S102, the virtual space management server 200 stores the store information, product information, and 3D image data of the store and products in the storage unit 210. The virtual space management server 200 reflects the store and products in the virtual space.

[0069] Next, in step S200, the wallet providing server 100 transmits a wallet APP for enabling various UIs in the virtual space in response to the request of the user terminal 300. Also, the virtual space management server 200 transmits a virtual space APP for displaying the virtual space in response to the request of the user terminal 300.

[0070] Next, in step S201, the user terminal 300 receives the user's operation input and transmits user information and payment means information to the wallet providing server 100. The wallet providing server 100 stores the user information and payment means information in the storage unit 110.

[0071] Next, in step S202, the wallet providing server 100 associates UI information corresponding to the payment means information with the user information and transmits it to the virtual space management server 200. Thereby, the virtual space management server 200 can provide a wallet UI to the user. In step S203, the virtual space management server 200 associates the wallet UI with the user information and stores it in the storage unit 210.

[0072] Next, in step S300, the user terminal 300 transmits authentication information to the virtual space management server 200. In step S301, the virtual space management server 200 transmits information regarding the virtual space to the user terminal 300. In step S302, the user terminal 300 displays the virtual space on the display unit 1007. The user terminal 300 transmits operation information to the virtual space management server 200 by recognizing the user's operation with reference to the action information D312. In step S303, the virtual space management server 200 transmits UI information regarding various UIs corresponding to the user's operation based on the operation information to the user terminal 300. Next, in step S304, the user terminal 300 displays various UIs on the display unit 1007.

[0073] The following describes the process in which the user terminal 300 displays various UIs. In the following, it is described that the virtual space management server 200 refers to the UI information 215 to cause the display unit 1007 of the user terminal 300 to display various UIs. However, for example, when the UI information is stored in the user terminal 300, the user terminal 300 may refer to the UI information to cause the display unit 1007 to display various UIs.

[0074] As shown in FIG. 7, when the virtual space management server 200 determines that the distance between the avatar and the store in the virtual space has reached a predetermined distance, it refers to the UI information D215 and causes the store identification UI to be displayed on the display unit 1007. Thereby, since the user can easily recognize the stores where the registered payment means can be used, the convenience of the user can be improved.

[0075] As shown in FIG. 8, when the user checks the products in the store, the virtual space management server 200 refers to the UI information D215 and causes the product confirmation UI (here, the instruction object U2) for pulling the product to the user's hand to be displayed on the display unit 1007 of the user terminal 300. The virtual space management server 200 obtains the motion information regarding the movement of the user's hand recognized by referring to the action information D312 on the user terminal 300, so that the product pulled to the user's hand can be rotated, reduced, or enlarged. Further, the virtual space management server 200 refers to the product information D213 to identify the products related to the product specified by the instruction object U2. The virtual space management server 200 displays each of the 3D image data of the identified plurality of products within the area of the spherical display object U4. The virtual space management server 200 moves the spherical display object U4 in accordance with the movement of the hand based on the motion information regarding the movement of the user's hand recognized by the user terminal 300. Thereby, the user can check the related products in a list, and the in-virtual-space payment system 10 enables the user to browse the products by an intuitive operation.

[0076] As shown in FIG. 9, the virtual space management server 200 causes the display unit 1007 of the user terminal 300 to display a product selection UI (here, the cart object U5 and the product display object U6) for the user to add products in the store to the cart. The virtual space management server 200 acquires operation information regarding the operation of the user's left hand palm protruding as recognized by referring to the action information D312 on the user terminal 300. The virtual space management server 200 causes the cart object U5 to be displayed on the display unit 1007 of the user terminal 300 based on the UI information D215. Thereby, since the in-virtual-space payment system 10 can display the cart by an intuitive operation, the convenience of the user can be improved. The virtual space management server 200 acquires operation information regarding the operation of the user pinching the product as recognized by referring to the action information D312 on the user terminal 300. The virtual space management server 200 causes the product display object U6 including the three-dimensional image data of the product to be displayed on the display unit 1007 of the user terminal 300 by referring to the UI information D215. When the virtual space management server 200 acquires operation information regarding the operation of moving the product display object U6 to the cart object U5, it adds the product to a list for purchase (hereinafter referred to as the "purchase list"). The purchase list is, for example, a list stored in the storage unit 210 of the virtual space management server 200. Thereby, since the in-virtual-space payment system 10 can add products to the cart by an intuitive operation, the convenience of the user can be improved.

[0077] Also, as shown in FIG. 10, the virtual space management server 200 causes the display unit 1007 of the user terminal 300 to display a product selection UI (here, the product display object U7) for viewing the products stored in the purchase list. When the virtual space management server 200 acquires operation information regarding an operation in which, for example, an icon U8 indicating a cart of the menu UI is selected from the user terminal 300, the virtual space management server 200 causes the product display object U7 to be displayed on the display unit 1007. When the virtual space management server 200 acquires information regarding an operation of removing the product display object U7 from the list, the virtual space management server 200 excludes the product displayed on the product display object U7 from the purchase list. As a result, the in-virtual-space payment system 10 can easily view the products in the cart and can exclude products from the cart by an intuitive operation, thereby improving the convenience for the user.

[0078] As shown in FIG. 11, the virtual space management server 200 causes the display unit 1007 of the user terminal 300 to display a menu UI for allowing an item to be selected. The virtual space management server 200 acquires operation information regarding an operation in which, with one hand held with the thumb facing forward, the hand is thrusted forward, which is recognized by referring to action information D312 on the user terminal 300. The virtual space management server 200 causes the icon U8 included in the menu UI to be displayed on the display unit 1007 of the user terminal 300 so as to surround the vicinity of the thrusted hand of the user. As a result, the user can easily check the menu items, thereby improving the convenience for the user. When the virtual space management server 200 acquires operation information regarding an operation in which a predetermined icon U8 of the menu UI is selected from the user terminal 300, the virtual space management server 200 causes an object corresponding to the icon U8 to be displayed on the display unit 1007 of the user terminal 300.

[0079] As shown in FIGS. 12 and 13, the virtual space management server 200 causes the display unit 1007 of the user terminal 300 to display a wallet UI (here, the first wallet UI or the second wallet UI) for the user to select a payment method when purchasing a product. For example, when the virtual space management server 200 obtains operation information regarding an operation for settling a product in the purchase list recognized by referring to the action information D312 on the user terminal 300, the virtual space management server 200 causes the first wallet UI or the second wallet UI to be displayed on the display unit 1007. Based on a purchase operation for the payment method (for example, the credit card U10 or the wallet object U12) selected by the user from the first wallet UI or the second wallet UI, the virtual space management server 200 generates purchase information.

[0080] Also, as shown in FIGS. 14, 15, and 16, the virtual space management server 200 causes the display unit 1007 of the user terminal 300 to display a wallet UI (here, the third wallet UI, the fourth wallet UI, or the fifth wallet UI) for receiving an operation for finalizing a purchase when performing a payment with a predetermined payment method. For example, when the virtual space management server 200 obtains operation information regarding an operation of selecting the payment method shown in FIGS. 12 and 13, the virtual space management server 200 causes the third wallet UI, the fourth wallet UI, or the fifth wallet UI to be displayed on the display unit 1007 of the user terminal 300. When the user terminal 300 recognizes an operation of causing the user to execute a payment for the third wallet UI, the fourth wallet UI, or the fifth wallet UI by referring to the action information D312, the user terminal 300 transmits settlement execution information for executing the settlement to the virtual space management server 200.

[0081] Next, in step S305, purchase information is generated based on the settlement execution information, and the purchase information is transmitted to the wallet providing server 100. Next, in step S306, the wallet providing server 100 transmits the purchase information to the e-commerce server 500 in order to settle the product that is the target of the settlement. As a result, the user can complete an operation related to settlement in the virtual space without detaching the user terminal 300 by operating the user terminal 300.

[0082] Next, in step S306, the wallet providing server 100 identifies the e-commerce server 500 corresponding to the purchased product based on the purchase information and transmits the purchase information. As a result, the e-commerce server 500 can execute the payment process to the retailer associated with the purchase information. Thereby, the user can complete the transaction, for example, within a VR (Virtual Reality) SNS (Social Networking Service).

[0083] Next, in step S307, the user terminal 300 receives a payment availability notification from the wallet providing server 100 regarding whether payment is possible at the e-commerce server 500. Thereby, since the user can know whether payment in the virtual space can be executed, the convenience of the user can be improved.

[0084] Next, in step S308, the virtual space management server 200 stores the purchased product information in the user information D216. As shown in FIG. 17, the virtual space management server 200 may display, on the display unit 1007 of the user terminal 300, a purchase product confirmation UI that refers to the user information D216 and displays 3D image data of the product for which the purchase procedure has been completed as a 3D image. ===Modification Example===

[0085] The virtual space payment system 10 may further be configured to issue a token number encrypted (obfuscated) based on, for example, the secret information of a credit card (such as the card number) in the payment means. Specifically, for example, after step S6 in FIG. 2, the user terminal 300 transmits information regarding the card number to the service provider that issues the token number by the user's operation input. The user terminal 300 acquires a token number corresponding to the card number. The user terminal 300 stores the token number in cooperation with the virtual space APP. When the user terminal 300 receives an operation to purchase a product from the user in the virtual space, it transmits the payment execution information to the virtual space management server 200. The virtual space management server 200 transmits the purchase information including the token number to the wallet providing server 100 (step S9 in FIG. 2). Thereby, a secure purchase process can be provided to the user.

[0086] In addition, the virtual space payment system 10 may manage the 3D data regarding the purchased product on the blockchain infrastructure. Specifically, a purchase certificate is issued to the user for the purchased product, and the 3D data regarding the product is arranged in a distributed storage. Also, the transaction history of the product is stored on the blockchain infrastructure. Thereby, it can be proved that the user is a legitimate purchaser of the product, and the data can be used even if the service of the content distribution provider is stopped. ===Hardware Configuration ===

[0087] The wallet providing server 100 and the virtual space management server 200 that realize the functions related to the above-described embodiment may be, for example, a server computer, a personal computer (such as a desktop, laptop, tablet, etc.), a media computer platform (such as a cable, satellite set-top box, digital video recorder), a handheld computer device (such as a PDA, an email client, etc.), or another type of computer, or a communication platform.

[0088] The user terminal 300 and the retail store terminal 400 that implement the functions related to the above-described embodiments may be, for example, a smartphone, a mobile phone (feature phone), a personal computer (e.g., desktop, laptop, tablet, etc.), a media computer platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computer device (e.g., PDA (Personal Digital Assistant), email client, etc.), a wearable terminal (e.g., glasses-type device, watch-type device, etc.), other types of computers, or a communication platform. Note that the user terminal 300 is preferably a transparent head-mounted display or a non-transparent head-mounted display.

[0089] Referring to FIG. 25, an example of the hardware configuration when the wallet providing server 100, the virtual space management server 200, the user terminal 300, and the retail store terminal 400 are implemented by a computer 1000 will be described. Note that the various functions of the wallet providing server 100, the virtual space management server 200, the user terminal 300, and the retail store terminal 400 can be realized by dividing them among a plurality of devices.

[0090] FIG. 25 is a diagram showing an example of the hardware configuration of a computer. As shown in FIG. 25, the computer 1000 includes, for example, a processor 101, a memory 1002, a storage device 1003, an input I / F unit 1004, a data I / F unit 1005, a communication I / F unit 1006, a display unit 1007, and a sensor group 1008. Note that the sensor group 1008 is hardware provided in the user terminal 300 and may not be provided in the wallet providing server 100, the virtual space management server 200, the user terminal 300, and the retail store terminal 400.

[0091] The processor 1001 is a control unit that controls various processes in the computer 1000 by executing programs stored in the memory 1002.

[0092] The memory 1002 is a storage medium such as a RAM (Random Access Memory). The memory 1002 temporarily stores the program code of the program executed by the processor 1001 and the data required during the execution of the program.

[0093] The storage device 1003 is a non-volatile storage medium such as a hard disk drive (HDD) or a flash memory. The storage device 1003 stores the operating system and various programs for implementing the above-described respective configurations.

[0094] The input I / F unit 1004 is a device for receiving an input from a user. Specific examples of the input I / F unit 1004 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, etc. The input I / F unit 1004 may be connected to the computer 1000 via an interface such as USB (Universal Serial Bus).

[0095] The data I / F unit 1005 is a device for inputting data from the outside of the computer 1000. Specific examples of the data I / F unit 1005 include a drive device for reading data stored in various storage media. The data I / F unit 1005 may be provided outside the computer 1000. In that case, the data I / F unit 1005 is connected to the computer 1000 via an interface such as USB.

[0096] The communication I / F unit 1006 is a device for performing data communication via the Internet N, either wired or wirelessly, with a device outside the computer 1000. The communication I / F unit 1006 may be provided outside the computer 1000. In that case, the communication I / F unit 1006 is connected to the computer 1000 via an interface such as USB.

[0097] The display unit 1007 is a device for displaying various types of information. Specific examples of the display unit 1007 include, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display unit 1007 may be provided outside the computer 1000. In that case, the display unit 1007 is connected to the computer 1000 via, for example, a display cable or the like. Also, when a touch panel is adopted as the input I / F unit 1004, the display unit 1007 can be configured integrally with the input I / F unit 1004.

[0098] Note that in the above, when referring to the display unit 1007 of the user terminal 300, the display unit 1007 includes a liquid crystal display panel. And when the user terminal 300 is a transmissive head-mounted display, the real space is not displayed on the display unit 1007. Also, when the user terminal 300 is a non-transmissive head-mounted display or a smartphone, an image in which the photographed real space image and the virtual reality space image are superimposed may be generated and displayed on the display unit 1007. When the user terminal 300 is a transmissive head-mounted display or a non-transmissive head-mounted display, the display unit 1007 may include two display panels and display a stereo image.

[0099] The sensor group 1008 includes various sensors. The sensor group 1008 includes sensors for grasping the user's surrounding space, sensors for grasping the user's position, posture, and hand movement, and sensors for detecting the user's viewpoint and line of sight. Sensors for grasping the user's surrounding space are, for example, RGB image sensors, depth sensors, and the like. Sensors for detecting the user's position, posture, hand movement, and the user's line of sight are gyro sensors, digital compasses, acceleration sensors, proximity sensors, motion tracking sensors, and the like. ===Summary===

[0100] The virtual space payment system 10 in this embodiment executes a process for displaying an instruction object U2 (for example, an object extending linearly) for instructing a product object three-dimensionally displayed in the augmented reality space or the virtual reality space, which moves according to the operation of the user's hand wearing the user terminal 300 (display device) that displays the virtual reality space, which is an augmented reality space or a virtual three-dimensional space, on the display unit 1007 of the user terminal 300 (display device). When there is a designated product object that is a product (product object) that overlaps or intersects with the instruction object U2, the display processing unit 140 executes a process for displaying the designated product object closer to the user by the user's pinch operation. The virtual space payment system 10 in this embodiment includes a settlement unit 150 that completes an operation related to the settlement of a selected product in response to a selection operation by the user on a user interface related to a settlement means for settling a selected product that is a product corresponding to the product (designated product object) selected by the user. Thereby, since product selection in the virtual space can be performed by a gesture more similar to the product selection behavior in the real world, by realizing a product purchase experience with a more intuitive operation in the virtual space, a deeper sense of immersion can be provided to the user.

[0101] Also, in the virtual space payment system 10 in this embodiment, the display processing unit 140 executes a process for displaying the instruction object U2 as an object extending linearly from at least a part of the user's hand. This enables the user to select products by an intuitive operation.

[0102] Also, in the virtual space payment system 10 in this embodiment, the settlement unit 150 completes an operation related to the settlement of a selected product by the user in the augmented reality space or the virtual reality space based on a token number (identification information) obtained by anonymizing an identification code for identifying the settlement means. This can provide a secure purchase process for the user.

[0103] Also, in the in-virtual-space payment system 10 in this embodiment, the display processing unit 140 executes processing for causing an identification object U1 (a predetermined identification mark) to be displayed for stores that can be settled by a settlement means in an augmented reality space or a virtual reality space. Thereby, since stores where each user's settlement means can be used can be easily found, the convenience for the user can be improved.

[0104] Also, in the in-virtual-space payment system 10 in this embodiment, the display processing unit 140, in an augmented reality space or a virtual reality space, has a plurality of related products related to the product of the designated product object, and a display object U4 for displaying each of the plurality of related products in a predetermined display area. The plurality of display objects U4 are displayed in a row in a direction away from the user such that at least a part of each of the plurality of display objects U4 overlaps, and executes processing for moving the plurality of display objects U4 in accordance with the movement of the user's hand with respect to the plurality of display objects U4. Thereby, the visibility of the user's product in the virtual space can be improved. Also, it enables the user to browse products by an intuitive operation.

[0105] Also, in the in-virtual-space payment system 10 in this embodiment, the display processing unit 140, in an augmented reality space or a virtual reality space, executes processing for reducing and displaying a selected product object within a predetermined area (here, a product display object U6) in response to an operation such that the user pinches the selected product object as recognized by the user terminal 300 (display device). Thereby, it enables the user to select products by an intuitive operation.

[0106] Also, in the in-virtual-space payment system 10 in the present embodiment, the display processing unit 140 causes a plurality of selected products set as purchase candidates by a user's selection operation on the selected product object to be displayed on the product display object U6 in the augmented reality space or the virtual reality space. Each of the plurality of product display objects U7 is displayed in a row so as to surround the body of the user or the user's avatar. A process for moving the plurality of product display objects U7 is executed according to the movement of the user's hand with respect to the plurality of product display objects U7. Thereby, since it becomes easier for the user to view the selected products, the convenience of the user can be improved.

[0107] Also, in the in-virtual-space payment system 10 in the present embodiment, the display processing unit 140 executes a process for making the product display object U7 on which the selected product is displayed invisible so that the setting of the selected product as a purchase candidate is removed in response to an operation in which the user moves the selected product displayed on the product display object U7, which is recognized by the user terminal 300 (display device), outside a predetermined range in the augmented reality space or the virtual reality space. Thereby, the user can cancel the selection state of the product by an intuitive operation.

[0108] Also, in the in-virtual-space payment system 10 in the present embodiment, the display processing unit 140 executes a process for displaying a plurality of icons U8 that constitute a menu UI (menu object) displayed so as to surround at least one hand of the user recognized by the user terminal 300 (display device) in the augmented reality space or the virtual reality space. Thereby, since the user can easily grasp the items included in the menu, the convenience of the user can be improved.

[0109] Also, in the virtual space payment system 10 in the present embodiment, the display processing unit 140 displays an instruction object U9 that indicates one of the icons U8 included in the menu UI (menu object) in the augmented reality space or the virtual reality space, and in response to an operation of twisting the user's wrist recognized by the user terminal 300 (display device) in one direction, executes a process for changing the icon U8 indicated by the instruction object U9. This enables the user to select menu items by an intuitive operation.

[0110] Also, in the virtual space payment system 10 in the present embodiment, the display processing unit 140 displays a user interface related to payment means in the shape of a card corresponding to the payment means in the real space in the augmented reality space or the virtual reality space, and executes a process for displaying them in a row in one direction so that a part of a plurality of card-shaped objects U10 (user interface) overlaps. This enables the user to select payment means by an intuitive operation.

[0111] Also, in the virtual space payment system 10 in the present embodiment, the display processing unit 140 executes a process for displaying a user interface related to payment means in a shape (here, object U12) according to the type of asset in the augmented reality space or the virtual reality space. This enables the user to select payment means by an intuitive operation.

[0112] Also, in the virtual space payment system 10 in the present embodiment, the display processing unit 140 executes a process for displaying an input object U13 indicating a dial lock for the user to input secret information in the augmented reality space or the virtual reality space, and the settlement unit 150 executes the user's settlement based on the information input to the input object in response to the movement of the user's hand recognized by the display device. This enables the user to execute a settlement by an intuitive operation.

[0113] Also, in the virtual space payment system 10 in the present embodiment, the display processing unit 140 executes a process for displaying a slide object U16 that slides in accordance with the operation of the user's hand recognized by the user terminal 300 (display device) in the augmented reality space or the virtual reality space, and the settlement unit 150 executes the user's settlement based on the information input to the slide object U16 in accordance with the operation of the user's hand recognized by the user terminal 300 (display device). Thereby, it is possible to cause the user to execute a settlement by an intuitive operation.

[0114] Also, in the virtual space payment system 10 in the present embodiment, the display processing unit 140 executes a process for displaying a slide object U14 on one palm of the user recognized by the user terminal 300 (display device) in the augmented reality space or the virtual reality space. Thereby, it is possible to cause the user to execute a settlement by an intuitive operation.

[0115] Also, the virtual space payment system 10 in the present embodiment further includes a product classification unit 160 that classifies information on purchased products, which are products settled by the settlement unit 150, into any one of a plurality of categories. The display processing unit 140 displays a plurality of folders corresponding to the plurality of categories in the augmented reality space or the virtual reality space, and in response to a user's selection operation on one of the plurality of folders, executes a process for displaying a purchased product object (here, 3D image data of clothing), which is a purchased product classified into the category corresponding to the folder that received the selection operation. Thereby, the operability of the user can be improved.

[0116] Also, in the in-virtual-space payment system 10 in the present embodiment, the display processing unit 140 executes processing for combining and displaying a plurality of purchased product objects (here, 3D image data of clothing) according to the category of the product based on the information regarding the purchased product object (here, 3D image data of clothing) stored in the product classification unit 160 in the augmented reality space or the virtual reality space. Thereby, since the purchased products can be browsed as if they are being confirmed in the real space, the convenience for the user can be improved.

[0117] Also, in the in-virtual-space payment system 10 in the present embodiment, the product classification unit 160 stores information regarding the user's clothes that are the purchased products, and the display processing unit 140 executes processing for suspending and displaying a clothing object that is the clothes visible to the user on the wardrobe U19 in the augmented reality space or the virtual reality space. Thereby, since the user can shop while checking the coordination with the clothes they currently have during shopping, the convenience for the user can be improved.

Description of Reference Numerals

[0118] 10... In-virtual-space payment system, 100... Wallet providing server, 200... Virtual space management server, 300... User terminal, 400... Retailer terminal.

Claims

1. In an augmented reality space or a virtual reality space which is a virtual three-dimensional space, a display processing unit that executes a process for causing an instruction object for instructing a product object three-dimensionally displayed in the augmented reality space or the virtual reality space to move in accordance with a movement of a user's hand to be displayed on a display unit of a display device; A settlement unit that accepts a selection operation by the user on a user interface related to a settlement means for settling a selected product which is a designated product object that is the product object indicated by the instruction object, and completes an operation related to the settlement of the selected product in response to the selection operation; Comprising: The display processing unit: In the augmented reality space or the virtual reality space, a plurality of related products related to the product of the designated product object are each displayed as a plurality of display objects in a predetermined display area; The plurality of display objects are arranged in a direction away from the user so that at least a part of each of the plurality of display objects overlaps, and a process for moving the plurality of display objects is executed in response to a movement of the user's hand with respect to the plurality of display objects; An information processing system.

2. The display processing unit executes a process for causing the instruction object to be displayed as an object linearly extending from at least a part of the user's hand. The information processing system according to claim 1.

3. In the augmented reality space or the virtual reality space, the display processing unit executes a process for reducing and displaying the selected product object within a predetermined area in response to an operation of the user pinching the selected product object recognized by the display device to indicate the selected product. The information processing system according to claim 1 or claim 2.

4. In the augmented reality space or the virtual reality space, the display processing unit causes a plurality of selected products set as purchase candidates by a selection operation by the user on the selected product object indicating the selected product to be displayed as product display objects, and each of the plurality of product display objects is displayed continuously so as to surround the body of the user or the avatar of the user. Execute processing for moving the plurality of product display objects in accordance with the movement of the user's hand with respect to the plurality of product display objects. The information processing system according to any one of claims 1 to 3.

5. A display processing unit that executes processing for causing a display unit of a display device to display an instruction object for instructing a product object three-dimensionally displayed in an augmented reality space or a virtual reality space that moves in accordance with the movement of a user's hand in the augmented reality space or the virtual reality space, which is a virtual three-dimensional space. A settlement unit that accepts a selection operation by the user on a user interface related to settlement means for settling a selected product, which is a designated product object that is the product object indicated by the instruction object, and completes an operation related to the settlement of the selected product in accordance with the selection operation. Comprising In the augmented reality space or the virtual reality space, the display processing unit displays a plurality of selected products set as purchase candidates by a selection operation by the user on a selection product object indicating the selected product on a product display object, and each of the plurality of product display objects is displayed continuously so as to surround the body of the user or the user's avatar. Execute processing for moving the plurality of product display objects in accordance with the movement of the user's hand with respect to the plurality of product display objects. In the augmented reality space or the virtual reality space, the display processing unit executes processing for making the product display object on which the selected product is displayed non-displayed so that the setting of the selected product as a purchase candidate is removed in accordance with an operation in which the user moves the selected product displayed on the product display object recognized by the display device outside a predetermined range. Information processing system.

6. The computer Execute processing for causing a display unit of a display device to display an instruction object for instructing a product object three-dimensionally displayed in an augmented reality space or a virtual reality space that moves in accordance with the movement of a user's hand in the augmented reality space or the virtual reality space, which is a virtual three-dimensional space. Accepting a selection operation by the user on a user interface for settlement means for settling a selected product, which is a product corresponding to the designated product object that is the product object indicated by the instruction object, and completing an operation related to the settlement of the selected product according to the selection operation; In the augmented reality space or the virtual reality space, displaying a plurality of display objects in a predetermined display area for each of a plurality of related products related to the product of the designated product object; Displaying the plurality of display objects in a row in a direction away from the user so that at least a part of each of the plurality of display objects overlaps, and moving the plurality of display objects according to the movement of the user's hand with respect to the plurality of display objects; An information processing method for executing.

7. On a computer, Executing a process for causing a display unit of a display device to display an instruction object for instructing a product object three-dimensionally displayed in the augmented reality space or the virtual reality space, which is an augmented reality space or a virtual three-dimensional space and moves according to the movement of the user's hand; Accepting a selection operation by the user on a user interface for settlement means for settling a selected product, which is a product corresponding to the designated product object that is the product object indicated by the instruction object, and completing an operation related to the settlement of the selected product according to the selection operation; In the augmented reality space or the virtual reality space, displaying a plurality of display objects in a predetermined display area for each of a plurality of related products related to the product of the designated product object; Displaying the plurality of display objects in a row in a direction away from the user so that at least a part of each of the plurality of display objects overlaps, and moving the plurality of display objects according to the movement of the user's hand with respect to the plurality of display objects; A program for causing the above to be executed.

Images