System and method for controlling the system

Abstract

To provide a system for providing a virtual object in accordance with information on a user.SOLUTION: A system for managing a virtual object includes an anchor management unit 311 which manages feature quantities in the real world for displaying the virtual object in association with the real world, in association with identification information corresponding to the virtual object. The anchor management unit 311 further manages the identification information in association with conditions for determining a method for providing a user with the virtual object.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a system for managing virtual objects.

Background Art

[0002] Technologies that create spaces that provide simulated experiences by fusing the real and virtual worlds, such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality), are attracting attention. XR is the general term for these. In recent years, mechanisms for displaying a single virtual object at the same location in the real world on multiple terminals have been implemented on platforms provided by various companies. For example, there exists a cloud system that associates and manages virtual objects that should be placed in the real world with real-world feature values captured by a camera or the like. By capturing a real world that matches a feature value already managed by the system with the camera of any terminal, it becomes possible to view the virtual object managed in association with the feature value on that arbitrary terminal. Patent Document 1 discloses a technique for switching the display of a specific virtual object using user behavior information and physical environment information. For example, in the initial state, a virtual object of a simple blue spherical globe is displayed, and when the user performs an action such as gazing at the globe, the globe switches to one representing continents.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] However, when providing XR services to users, there are cases where multiple virtual objects are placed in the same location by the virtual object provider. For example, if the virtual object provider wants to change the object displayed depending on the user's age, gender, contract details with the user, or the user's situation, multiple virtual objects may be placed in the same location. When multiple virtual objects are placed in the same location, the user terminal will receive multiple anchor information from the virtual object management server, making it impossible to determine which object to display to the user.

[0005] The present invention aims to provide a system that provides virtual objects according to user information. [Means for solving the problem]

[0006] To solve the above problems, the present invention provides a system comprising a terminal capable of projecting virtual objects into the real world, and a management means for managing feature quantities in the real world for displaying virtual objects linked to the real world, in association with identification information corresponding to the virtual objects, The management means can further associate and manage identification information corresponding to multiple different virtual objects for the same feature quantity, and can associate and manage different conditions for each of the identification information corresponding to the multiple different virtual objects as conditions for determining how to provide the virtual objects to the user. The aforementioned different conditions include conditions relating to whether or not a contract is agreed to in the service that provides the virtual object, the aforementioned multiple different virtual objects include a first virtual object provided to a user who has not agreed to the contract, and a second virtual object provided to a user who has agreed to the contract, the management means manages to associate a condition indicating that the user has not agreed to the contract with a first identification information corresponding to the first virtual object, and manages to associate a condition indicating that the user has agreed to the contract with a second identification information corresponding to the second virtual object, The terminal executes a request using real-world features that can be the target of projection of virtual objects, and the request includes conditions corresponding to one or more pieces of identification information associated with the features included in the request, and user information of the user using the terminal. Whether or not consent to the aforementioned contract is identified from Provided based on The first virtual object or the second The virtual object is projected onto the real world. [Effects of the Invention]

[0007] According to the present invention, a system can be provided that provides virtual objects according to user information. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of a virtual object management system. [Figure 2] This diagram shows the hardware configuration of the client terminal. [Figure 3] This diagram shows the hardware configuration of the virtual object management server. [Figure 4] This diagram shows the software configuration of the virtual object management system. [Figure 5] This figure shows an example of what the screen of a client terminal looks like. [Figure 6] This is a sequence diagram showing the process flow for registering and rendering virtual objects. [Figure 7] This flowchart shows the process for determining the virtual object to be provided to the client terminal. [Modes for carrying out the invention]

[0009] Figure 1 shows the overall configuration of a system for managing virtual objects. The virtual object management system includes a virtual object management server 121 that provides virtual objects, and client terminals that can project the virtual objects provided by the virtual object management server 121 into the real world. In this embodiment, an example is described in which client terminals 131 to 133 are connected to the virtual object management server 121.

[0010] Client terminal 131 is connected to virtual object management server 121 via network 100 and network 101. Client terminals 132 and 133 are connected to virtual object management server 121 via network 100 and network 101. Network 100 is the internet, and networks 101 and 102 are the internet, networks within homes, companies, schools, wireless LANs set up in towns, etc. Networks 100 to 102 can be any so-called communication networks, such as LANs, WANs, telephone lines, dedicated digital lines, ATM or frame relay lines, cable television lines, wireless lines for data broadcasting, etc. Networks 100 to 102 only need to be capable of sending and receiving data.

[0011] Client terminals 131-133 are terminals capable of capturing images of the real world, displaying virtual objects, and communicating with the virtual object management server 121 in order to project virtual objects into the real world. Client terminals 131-133 include, for example, dedicated hardware that supports the rendering of virtual objects handled in XR, such as HMDs (head-mounted displays) and smart glasses, and communication devices with a built-in program execution environment, such as smartphones. If client terminals 131-133 are not dedicated hardware that supports the rendering of virtual objects, such as smartphones, they will use APIs provided by a web browser or OS to render virtual objects. Client terminals 131-133 are equipped with cameras to capture images of the surroundings and displays to display virtual objects. By capturing images of the surroundings through the cameras and projecting virtual objects onto the displays that capture images of the real world, client terminals 131-133 provide users with a simulated experience that fuses the real and virtual worlds.

[0012] The virtual object management server 121 provides a service that provides virtual objects to external terminals. The virtual object management server 121 is constructed, for example, using a server computer. In this embodiment, an example is described in which the virtual object provision service is provided by the virtual object management server 121, but it is not limited to this. The services and functions provided by the virtual object management server 121 may be realized by one or more information processing devices, virtual machines (cloud services) that utilize resources provided by a data center including information processing devices, or a combination of these.

[0013] Furthermore, as part of the virtual object provision service, the virtual object management server 121 manages virtual objects placed in the real world by associating them with real-world feature quantities captured by cameras, etc. In this embodiment, the virtual object management server 121 manages virtual objects and real-world feature quantities captured by cameras, etc. by associating them and using anchors. An anchor includes the virtual object, real-world feature quantities for displaying the virtual object in association with the real world, as well as an identifier and session ID for identifying the anchor itself. In addition, the anchor in this embodiment includes property information including conditions (various parameters) that define how the virtual object is provided to the user. The anchor manages at least three pieces of information as feature quantities for displaying the virtual object in association with the real world: the feature quantities in Table 1 described later, location information, and sensor information. The virtual object management server 121 receives anchor registration requests from client terminals 131 to 133 and manages the registered anchors. The virtual object management server 121 also returns an anchor that matches the conditions from among the managed anchors in response to an anchor acquisition request from client terminals 131 to 133. Furthermore, the virtual object management server 121 also manages user information for client terminals 131-133. The virtual object management server 121 receives user login / logout requests from client terminals 131-133 and performs login / logout processing.

[0014] Figure 2 is a diagram illustrating the hardware configuration of client terminals 131 to 133. The client terminals 131 to 133 include a CPU 202, a GPU 210, a RAM 203, a ROM 204, an HDD 205, a NIC 209, a camera 207, a display 206, and an interface 208. These components are connected to a system bus 201.

[0015] The CPU (Central Processing Unit) 202 controls the entire terminal. The GPU (Graphics Processing Unit) 210 performs arithmetic processing required for rendering virtual objects in real time. The RAM (Random Access Memory) 203 is a temporary storage unit that functions as the main memory and work area for the CPU 202 and the GPU 210. The ROM (Read Only Memory) 204 is a data read-only memory that stores various data such as basic I / O programs. The HDD (Hard Disc Drive) 205 is a large-capacity memory that stores application programs such as a web browser, an operating system (OS), related programs, and various types of data. Note that the HDD 205 is an example of a storage device, and may be a memory such as a solid state drive (SSD) or an external storage device. The CPU 202 loads a program stored in the memory (the ROM 204 or the HDD 205) into the RAM 203 and executes it, thereby comprehensively controlling each unit connected to the system bus 201.

[0016] The display 206 is a display unit that displays virtual objects, information required for operations, and the like. When the client terminal is a smartphone, a tablet, or the like, the display 206 may be a touch panel in which a display unit and an input unit are integrated. By associating input coordinates and display coordinates on the touch panel, it is possible to configure a GUI that allows a user to operate the screen displayed on the touch panel as if directly manipulating it.

[0017] Camera 207 includes an outward-facing camera that captures images of the surroundings and an inward-facing camera that primarily captures images of the user. By analyzing the images captured by Camera 207 with an application program stored in HDD 205, it becomes possible to overlay virtual objects onto the real world and calculate features of the real world. Furthermore, if client terminals 131-133 are dedicated XR devices such as HMDs, it is possible to manipulate the virtual objects displayed on display 206 using the user's finger movements recognized by Camera 207. If client terminals 131-133 are not dedicated XR devices such as smartphones, it is possible to manipulate the virtual objects displayed on display 206 by operating the touch panel or other controls on display 206.

[0018] The NIC (Network Interface Card) 209 is a network interface that exchanges data with external devices such as the virtual object management server 121 via networks 101 and 102. Interface 208 is an interface with external devices and connects peripheral devices such as various external sensors. Note that the configuration in Figure 2 is just an example, and the configuration of client terminals 131 to 133 is not limited to this. For example, the storage location of data and programs can be changed to ROM 204, RAM 203, HDD 205, etc., depending on their characteristics.

[0019] Figure 3 is a diagram showing the hardware configuration of the virtual object management server 121. The virtual object management server 121 includes a CPU 222, a RAM 223, a ROM 224, an HDD 225, a NIC 229, and an interface 228. These are connected to a system bus 221. The CPU 222 controls the virtual object management server 121. The RAM 223 is a temporary storage unit, and functions as the main memory, work area, or the like of the CPU 222. The ROM 224 is a read-only memory, and stores various data such as basic I / O programs. The HDD 225 is a large-capacity memory, and stores programs of the service server group, an operating system (OS), related programs, various types of data, and the like. The CPU 222 comprehensively controls each unit connected to the system bus 221 by loading a program stored in the memory (the ROM 224 or the HDD 225) into the RAM 203 and executing the program.

[0020] The NIC 229 is a network interface that exchanges data with external devices such as the client terminals 131 to 133 via the network 100. The interface 228 is an interface with an external device, and connects peripheral devices. Note that the configuration in Figure 3 is an example, and the configuration of the virtual object management server 121 is not limited thereto.

[0021] Figure 4 is a diagram showing the software configuration of the virtual object management system according to the present embodiment. The software configuration of the client terminals 131 to 133 shown in Figure 4 is implemented by the CPU 202 and the GPU 210 executing processing based on programs stored in a memory (the ROM 204 or the HDD 205). Similarly, the software configuration of the virtual object management server 121 shown in Figure 4 is implemented by the CPU 222 executing processing based on a program stored in a memory (the ROM 224 or the HDD 225).

[0022] Client terminals 131-133 each have a virtual object data management unit 301, an anchor generation unit 302, an anchor acquisition unit 303, an anchor drawing unit 304, a login unit 305, a local anchor management unit 306, and a virtual object display control unit 307. The virtual object data management unit 301 manages the 3D data of virtual objects. The 3D data in various formats stored in the virtual object data management unit 301 are virtual objects that users can freely place on top of the real world.

[0023] The anchor generation unit 302 creates anchors based on user operations. The user selects a 3D model stored in the virtual object data management unit 301 via the anchor generation unit 302 and places the virtual object in the real world using hand movements captured by the camera 207 or touch panel operations on the display 206. An example of the display of a virtual object on a client terminal is explained using Figure 5. Figure 5(A) shows an image (video) displayed on the display 206 of an HMD-type client terminal 131. The desk 1001 is a real-world desk captured by the camera 207. The cylindrical virtual object 1002 is a cylindrical virtual object stored in the virtual object data management unit 301 of the client terminal 131. The user places the cylindrical virtual object 1002 on the real-world desk 1001, for example, by manipulating it with gestures.

[0024] When a virtual object is placed, the anchor generation unit 302 analyzes the image to extract feature quantities from the area around the placed virtual object, associates them with the virtual object, and stores them in the local anchor management unit 306. The anchor generation unit 302 also uses the GPS sensor connected via the interface 208 to identify the virtual object's location information and associates it with the anchor. Users can associate anchors with sensors through the anchor generation unit 302. The local anchor management unit 306 manages anchors on each client terminal. The local anchor management unit 306 stores and manages anchors generated on each client terminal and anchors obtained from the virtual object management server 121.

[0025] The anchor acquisition unit 303 acquires anchors from the virtual object management server 121. Specifically, the anchor acquisition unit 303 sends an anchor acquisition request to the virtual object management server 121 and receives the anchors from the virtual object management server 121 as a response to the anchor acquisition request. The anchors acquired from the virtual object management server 121 are then stored in the local anchor management unit 306.

[0026] The anchor drawing unit 304 places virtual objects contained in an anchor in the real world based on the feature quantities contained in the anchor. The anchor drawing unit 304 compares the feature quantities contained in each anchor stored in the local anchor management unit 306 with the real world image captured by the camera 207, and places the virtual objects contained in the anchor in the area of ​​the real space where the feature quantities match. Figure 5(B) is a diagram showing the image (video) displayed on the display 206 of the HMD-type client terminal 133. Figure 5(B) shows how a cylindrical virtual object 1002 placed by the user on the desk 1001 on the client terminal 131 is projected as a cylindrical virtual object 1032 on the desk 1031 with the same feature quantities on the client terminal 133. By acquiring the anchor corresponding to Figure 5(A), which was generated on the client terminal 131 and provided to the virtual object management server 121, on the client terminal 133 and drawing virtual objects based on the anchor, Figure 5(B) can be displayed.

[0027] The login unit 305 performs user authentication. Authentication is performed, for example, by a combination of username and password. The login unit 305 displays the login screen on the terminal's display 206. Figure 5(C) shows an example of the login screen. When the image code 501 is read by the camera 207 of the client terminal 132, the login screen 502 is displayed. The login unit 305 sends the username and password entered on the login screen to the login processing unit 315 of the virtual object management server 121. The username and password are entered, for example, by the user's finger movements captured by the camera 207, by the operation of the touch panel on the display 206, or by a keyboard connected to the interface 208. User authentication is performed in the login processing unit 315, and if the user is able to log in to the service that provides virtual objects, the virtual object 503 becomes available for display. Note that the login method is not limited to a combination of username and password. For example, biometric authentication such as facial recognition using a face image captured by the camera 207, iris recognition using the iris, or fingerprint recognition using a fingerprint sensor connected to the interface 208 may also be used. The virtual object display control unit 307 controls various editing and operations performed by the user on the virtual object displayed on the display 206 in the client terminal.

[0028] The virtual object management server 121 includes an anchor management unit 311, an anchor receiving unit 312, an anchor providing unit 313, a user information management unit 314, and a login processing unit 315. The anchor management unit 311 manages anchors. Table 1 is an example of an anchor management table managed by the anchor management unit 311. An anchor includes anchor ID, session ID, virtual object data, feature quantities, location information, sensor information, gender, time, etc. [Table 1]

[0029] The anchor ID is identification information used to uniquely identify an anchor, and also serves as identification information corresponding to a virtual object. The anchor ID is assigned when the anchor receiving unit 312 saves a record to the anchor management table. The session ID is an identifier used to link multiple anchors as a single group. Anchors belonging to the same session will be assigned the same session ID. By linking multiple anchors to a single session ID, multiple anchors with the same session ID can be presented to the user simultaneously. In other words, a provision method defined by the session ID can provide a different virtual object from a given virtual object.

[0030] Virtual object data is data of a 3D model of a virtual object in any format. Feature quantities, location information, and sensor information are features used to display the virtual object in relation to the real world. Feature quantities, for example, represent 3D information of the real world obtained by analyzing data captured by camera 207 around where an anchor is placed. Location information indicates the 3D position of the virtual object in the real world. Sensor information includes location information (GPS coordinates) where the anchor is placed, the Beacon and Wi-Fi ID to which the anchor is associated, etc. The location where the virtual object is installed can be identified by the Beacon and Wi-Fi ID to which the anchor is associated. Gender and time are examples of property information that determine how the virtual object is provided to the user. Property information is referenced when determining the anchor to be provided to the client terminal, and the virtual object management server 121 determines the virtual object data to be returned based on the gender of the user who requested the virtual object and the time of the request. Note that property information is not limited to gender and time. Property information may include, for example, settings based on the user's progress in virtual object provision services such as stages, settings based on the user's personal attributes such as gender, age, and membership rank, and settings based on the environment to which the user belongs, such as season and date and time. Settings based on the user's progress in virtual object provision services may include whether or not the user has agreed to the EULA.

[0031] The anchor receiving unit 312 receives anchor registration requests from client terminals 131 to 133. The anchor receiving unit 312 then stores the anchors included in the received anchor registration requests in the anchor management unit 311. The anchor providing unit 313 receives anchor acquisition requests from client terminals 131 to 133, searches the anchor management unit 311 for anchors that match the specified conditions, and returns them to the client terminals 131 to 133.

[0032] The User Information Management Unit 314 is a user management means for managing user information that uses the virtual object management system. Tables 2 and 3 are examples of user information management tables managed by the User Information Management Unit 314. Table 2 is an example of a user attribute information management table. Table 3 is an example of a user progress information management table. In this embodiment, an example is described in which the table for managing user attribute information and the user's progress status within the virtual object management system are managed in separate tables, but it may be managed in one table or in multiple tables. [Table 2] [Table 3]

[0033] The user attribute information management table manages user attribute information such as user ID, username, password, age, gender, and creator. The user ID is information used to uniquely identify a user. The username is the name set by the user. The password is the password used for login authentication, etc., set by the user. Age is the age set by the user, and gender is the gender set by the user. The creator indicates whether the user is the creator of the virtual object.

[0034] The user progress information management table manages user IDs and EULA agreement status. The user ID is information used to uniquely identify a user managed in the user attribute information management table. The EULA agreement status indicates whether or not the user has agreed to the EULA. Agreement to the EULA is required to use the services provided by the virtual object management system, and the EULA agreement status is managed as progress information within the service, linked to the user ID.

[0035] The login processing unit 315 receives login requests from client terminals 131 to 133, compares them with user information managed by the user information management unit 314, and returns the login processing result to the client terminals 131 to 133. If the information in the login request matches the user information managed by the user information management unit 314, the login processing unit 315 returns the login processing result as successful to the client terminals 131 to 133.

[0036] Figure 6 is a sequence diagram showing the process flow for registering and rendering virtual objects. It shows the process from registering an anchor generated by client terminal 131 with the virtual object management system 111, to retrieving and displaying the anchor stored in the virtual object management system 111 by client terminal 133. The user operating client terminal 131 is denoted as "X," and the user operating client terminal 133 is denoted as "A." User "X" is the user with user ID U024 in Table 2, and user "A" is the user with user ID U001 in Table 2. The processing performed on the client terminal shown in Figure 6 is achieved by the CPU 202 or GPU 210 of the client terminal reading the program stored in memory into RAM 203 and executing it. The processing performed on the virtual object management server 121 shown in Figure 6 is achieved by the CPU 222 of the virtual object management server 121 reading the program stored in memory into RAM 223 and executing it.

[0037] First, from S601 to S605, we will explain the sequence in which a client terminal 131 operated by user "X" registers an anchor with the virtual object management server 121. In S601, the login unit 305 of the client terminal 131 sends the user ID and password entered by the user to the login processing unit 315 of the virtual object management server 121. In S602, the login processing unit 315 of the virtual object management server 121 verifies the user information received from the client terminal 131 and returns the result of the user authentication process for login to the client terminal 131. For example, if the login processing unit 315 confirms that the user information received from the client terminal 131 matches the user ID and password managed by the user information management unit 314 for user "X", it determines that the login was successful. Then, the login processing unit 315 returns a login success result to the client terminal 131.

[0038] In S603, the anchor generation unit 302 of the client terminal 131 generates an anchor for a virtual object stored in the virtual object data management unit 301 deployed by user "X", and stores it in the local anchor management unit 306. When generating an anchor, user "X" can set conditions that define how the virtual object is provided to the user. That is, the method of providing the virtual object to the user can be set by the owner who created the virtual object. In S604, the anchor generation unit 302 of the client terminal 131 sends an anchor registration request for the anchor generated in S603 to the anchor receiving unit 312 of the virtual object management server 121. In S605, the anchor receiving unit 312 of the virtual object management server 121 registers the anchor received from the client terminal 131 with the anchor management unit 311 and returns the registration result to the anchor generation unit 302 of the client terminal 131. If multiple anchors are to be registered with the same session ID, the series of processes shown in S611 (S603~S605) are repeated.

[0039] Next, from S621 to S632, we will describe the sequence in which a client terminal 133 operated by user "A" retrieves and displays a virtual object from the virtual object management server 121. In S621, the login unit 305 of the client terminal 133 sends the user ID and password entered by the user to the login processing unit 315 of the virtual object management server 121. In S622, the login processing unit 315 of the virtual object management server 121 verifies the user information received from the client terminal 131 and returns the result of the user authentication process for login to the client terminal 133. For example, if the login processing unit 315 confirms that the user information received from the client terminal 133 matches the user ID and password managed by the user information management unit 314, it returns the login result to the client terminal 131 as a successful login.

[0040] In S623, the anchor acquisition unit 303 of the client terminal 133 acquires sensor information. The anchor acquisition unit 303 acquires a signal from the Beacon terminal as sensor information, for example, via a sensor that detects Bluetooth signals connected to the client terminal 133 via interface 208. The sensor information to be acquired may also be information about the Wi-Fi to which the client terminal 133 is connected, or information read from an image code via camera 207. If sensor information cannot be acquired, as shown in S641, the anchor acquisition unit 303 repeats the process in S623 until sensor information can be acquired.

[0041] In S624, the anchor acquisition unit 303 of the client terminal 133 sends a request for provision of a virtual object (anchor search request) to the anchor provision unit 313 of the virtual object management server 121. The anchor search request includes anchor identification information (i.e., identification information corresponding to the virtual object), at least one real-world feature for displaying the virtual object, and user information. For example, the anchor search request includes user information and sensor information acquired in S623. For example, if S623 detects a signal from a Beacon terminal with id=123, the anchor acquisition unit 303 sends an anchor search request associated with the Beacon terminal with id=123 to the anchor provision unit 313 along with user information for user "A". In this embodiment, in addition to user information, an example has been described in which sensor information, which is one of the real-world feature for displaying the virtual object, is acquired in S623, and this is used to request the provision of a virtual object from the virtual object management server 121. However, this is not the only way to do so. The information acquired in S623 and sent to the virtual object management server 121 in S624 may also be other information that indicates feature quantities, or it may be identification information corresponding to a virtual object.

[0042] In S625, the anchor provision unit 313 of the virtual object management server 121 searches for an anchor in the anchor management unit 311 based on an anchor search request received from the client terminal 133, and determines which anchor to return to the client terminal 133. For example, if the received request contains identification information corresponding to a virtual object, the anchor provision unit 313 determines that the information (anchor) of the virtual object corresponding to the identification information should be returned. Also, if the received request contains a feature, the anchor provision unit 313 narrows down the anchor to be returned based on the feature and the anchor information managed by the anchor management unit 311. For example, the anchor provision unit 313 searches for the anchor associated with the Beacon terminal with id=123 in the anchor management unit 311 by referring to the user information management table, etc., and determines which anchor to return. Furthermore, the anchor provision unit 313 narrows down the anchor to be returned according to the provision method determined based on the conditions for providing virtual objects to the user as defined by the user information and the anchor information managed by the anchor management unit 311. The processing in S625 based on the conditions will be explained in detail in the flowchart in Figure 7, which will be described later. In S626, the anchor provision unit 313 of the virtual object management server 121 returns the anchor determined in S625 to the anchor acquisition unit 303.

[0043] In S627, the anchor acquisition unit 303 of the client terminal 133 stores the anchor acquired from the virtual object management server 121 in the local anchor management unit 306. In S628, the anchor acquisition unit 303 of the client terminal 133 sends a search request to the anchor provision unit 313 of the virtual object management server 121 for anchors in the same session as the anchor acquired in S627. For example, if the anchor acquisition unit 303 acquired an anchor with anchor ID "a" in S627, it sends a search request to the anchor provision unit 313 for the same anchor as session ID 111, which has anchor ID "a".

[0044] In S629, the anchor provision unit 313 of the virtual object management server 121 searches the anchor management unit 311 for an anchor different from the one provided by the same session ID, based on the session ID received from the client terminal 133. Then, in S630, the anchor provision unit 313 of the virtual object management server 121 returns the search result from S629 to the client terminal 133. If an anchor for the same session is found in the search in S629, the anchor provision unit 313 returns the anchor found in S629 to the anchor acquisition unit 303 of the client terminal 133. For example, the anchor provision unit 313 searches for an anchor with session ID=111 and sends the anchor with session ID=111 and anchor ID "d" to the client terminal 133. On the other hand, if no anchor for the same session is found in the search in S629, it sends a message to the client terminal 133 indicating that there are no anchors belonging to the same session. Through the processing in S628-S630, the anchor provisioning unit 313 can provide the client terminal 133 with a different virtual object from the one previously provided to the client terminal 133, using a provisioning method determined by the session ID.

[0045] In S631, the anchor acquisition unit 303 of the client terminal 133 stores the anchor acquired from the anchor provision unit 313 in S630 in the local anchor management unit 306. In S642, the anchor drawing unit 304 of the client terminal 133 draws a virtual object based on the anchor acquired from the virtual object management server 121 and stored in the local anchor management unit 306 in S627 and S631. If there are multiple anchors stored in the local anchor management unit 306 in S627 and S631, the anchor drawing unit 304 repeats the virtual object drawing process in S642 as many times as there are stored anchors, as shown in S642.

[0046] Figure 7 is a flowchart showing the process of determining the virtual object to be provided to the client terminal. It is a flowchart detailing the process in which the virtual object management server 121 in S625 receives a request for anchor provision from the client terminal 133, searches for the anchor to be returned, and determines it. Here, we will explain an example in which the anchor management table managed by the anchor management unit 311 is shown in Table 4, the user attribute information management table managed by the user information management unit 314 is shown in Table 5, and the user progress information management table managed by the user information management unit 314 is shown in Table 6. Furthermore, it is assumed that user "C" uses a service in which agreement to the EULA is required to use the service, and that the virtual object displayed differs depending on the user's attribute information and progress status. [Table 4] [Table 5] [Table 6]

[0047] The anchor management table shown in Table 4 includes the information described in Table 1, as well as EULA agreement status, stage, and membership rank. Similarly, the user attribute information management table shown in Table 5 includes the information described in Table 2, as well as membership rank, and the user progress information management table shown in Table 6 includes the information described in Table 3, as well as stage. EULA agreement status indicates whether EULA agreement is required to use the anchor. Stage indicates the progress of the service providing the virtual object. Membership rank indicates the user's membership rank (e.g., Bronze, Gold, Platinum, Guest, etc.) in the service providing the virtual object.

[0048] The EULA agreement status, stage, membership rank, and gender managed in the anchor information management table (Table 4) are conditions that determine how virtual objects are provided to users. The method of providing virtual objects to user terminals can be changed based on conditions such as whether the user has agreed to the EULA regarding service use, the user's age, the date and time when the user requested a virtual object from the terminal, and the user's membership rank. For example, for guest users who have not agreed to the EULA (e.g., U001), a different virtual object with a different EULA than the one provided to service members can be provided.

[0049] Each process shown in Figure 7 is realized by the CPU 222 of the virtual object management server 121 reading a program stored in memory into RAM 223 and executing it. In S701, the anchor provision unit 313 receives a virtual object provision request (search request) from the anchor acquisition unit 303 of the client terminal 133. The virtual object provision request received from the client terminal 133 includes identification information of the anchor corresponding to the virtual object, at least one of the real-world features, and user information.

[0050] In steps S702-S705, the anchor provision unit 313 determines which virtual objects to provide based on the user information included in the virtual object provision request and the conditions that define how virtual objects managed by the virtual object management server 121 are provided to the user. In S702, the anchor provision unit 313 refers to the user information management table (Tables 5 and 6) to obtain the settings based on the progress of the virtual object provision service for the user corresponding to the user information included in the virtual object provision request. Then, based on the settings based on the user's progress in the virtual object provision service, the anchor provision unit 313 refers to the anchor management table (Table 4) to narrow down the virtual objects to provide to the client terminal 133. If the virtual object provision request includes user information for user "C", the anchor provision unit 313 obtains the settings based on the progress of the virtual object provision service for the user whose username is "C" in the user information management table. Specifically, the anchor provision unit 313 obtains that the user whose username is "C", i.e., user ID "U003", has agreed to the EULA and has progressed to stage "2" of the virtual object provision service. The anchor provider unit 313 refers to the anchor management table and narrows down the anchors of virtual objects whose EULA agreement status is "○" and whose stage is "2" to anchors with anchor IDs "c" and "d".

[0051] In S703, the anchor provider 313 determines whether the virtual object provided to the client terminal 133 differs depending on the user's attributes, based on the virtual object narrowed down in S702. For example, the anchor provider 313 determines whether the virtual object provided differs depending on the user's attributes by checking whether the values ​​indicating user attributes such as "gender" and "member rank" in the anchor information corresponding to the narrowed-down virtual object differ. If the values ​​indicating user attributes such as "gender" and "member rank" in the anchor information corresponding to the narrowed-down virtual object differ, it determines that the virtual object provided to the client terminal 133 differs depending on the user's attributes and proceeds to the process in S704. On the other hand, if the values ​​indicating user attributes such as "gender" and "member rank" in the anchor information corresponding to the narrowed-down virtual object are the same, it determines that the virtual object returned does not differ depending on the user's attributes and proceeds to the process in step S706.

[0052] In S704, the anchor provisioning unit 313 refers to the user information management table (Tables 5 and 6) and obtains settings based on the user's attributes (user attribute information) corresponding to the user information included in the virtual object provision request. Settings based on user attributes include, for example, settings based on the user's personal attributes such as gender, age, membership rank, and affiliated organization, as well as settings based on the environment to which the user belongs (the situation in which the user is placed), such as time and season. If user information corresponding to user "C" has been received from the client terminal 133, the anchor provisioning unit 313 obtains the settings based on the attributes of the user whose username is "C" in the user information management table. Specifically, the anchor provisioning unit 313 obtains information that the user whose username is "C," that is, the user ID "U003," has a membership rank of "Platinum" and is "female." In S705, based on the settings based on the user's attributes (user attribute information) obtained in S704, the anchor provisioning unit 313 further narrows down the virtual objects to be provided to the client terminal 133 from the virtual objects narrowed down in S702. Specifically, the anchor provider unit 313, based on the fact that user "C" has a membership rank of "Platinum" and is female, refers to the anchor management table and narrows down the anchors to those with anchor ID "d" among anchors with anchor IDs "c" and "d". It also obtains that the EULA has been agreed to and that the virtual object provision service has progressed to stage "2". The anchor provider unit 313 refers to the anchor management table and narrows down the anchors of virtual objects with an EULA agreement status of "○" and stage "2" to anchors with anchor IDs "c" and "d". In S706, the anchor provider unit 313 responds to the client terminal 133 with the narrowed-down anchors, thereby responding to the client terminal 133 with the virtual object corresponding to those anchors.

[0053] Through the above process, even when multiple virtual objects are placed in the same location, the appropriate virtual object can be provided and displayed to the user's terminal according to the user's attribute information, the user's current situation, and the progress within the service.

[0054] In this embodiment, the filtering of virtual objects to be returned to the client terminal 133 was performed by the virtual object management server 121, but this could also be done on the client terminal 133 side. By storing user information (for example, Tables 5 and 6) within the client terminal 133, it is also possible to filter the virtual objects displayed on the client terminal 133 according to conditions based on the user information.

[0055] Furthermore, if the creator explicitly instructs the provision of a virtual object to a target user via push sharing or similar means, it is not necessary to verify the user's user information in order to provide the virtual object. In other words, it is possible to skip the process of determining the virtual object to be provided based on user information conditions (S702-S705) and provide the anchor specified by the owner.

[0056] As described above, according to this embodiment, even when multiple virtual objects are placed in the same location, an appropriate virtual object can be provided according to the user's contract details, attribute information such as the user's age, gender, and membership rank, and the environment to which the user belongs, such as the date and time. This makes it possible to provide and display a virtual object that corresponds to the user's information, even when the feature quantities are similar.

[0057] The disclosure of this embodiment includes the following configuration of the information processing device. (Configuration 1) A system for managing virtual objects, comprising a management means for managing real-world feature quantities for displaying virtual objects linked to the real world, in association with identification information corresponding to the virtual objects, wherein the management means further manages the identification information in association with conditions that define how the virtual objects are provided to users. (Configuration 2) The system according to Configuration 1, wherein the system includes a terminal capable of projecting a virtual object into the real world, and in response to a request using the identification information and at least one of the features in the real world from the terminal and user information, the management means responds to the terminal with information of a virtual object managed by the identification information and at least one of the features in the real world, in accordance with a provision method determined based on the user information and the conditions, and the terminal projects the virtual object into the real world. (Configuration 3) The system according to Configuration 1 or 2, characterized in that the method of providing includes providing a different virtual object from the virtual object. (Configuration 4) The system according to any one of Configurations 1 to 3, characterized in that the conditions include settings based on the progress of the user corresponding to the user information in the service that provides the virtual object. (Configuration 5) The system according to Configuration 4, characterized in that the settings based on the user's progress include whether or not the contract has been agreed upon. (Configuration 6) The system according to any one of Configurations 1 to 5, characterized in that the conditions include settings based on the user's personal attributes corresponding to the user information. (Configuration 7) The system according to any one of Configurations 1 to 6, characterized in that the conditions include settings based on the environment to which the user corresponding to the user information belongs. (Configuration 8) The system according to any one of Configurations 1 to 7, further comprising a user management means for managing a link between user identification information and settings based on the user's progress in the service or settings based on the user's personal attributes. (Configuration 9) The system according to any one of Configurations 1 to 8, characterized in that the method for providing the virtual object to the user is set by the owner of the virtual object. (Configuration 10) The system according to Configuration 9, characterized in that when the owner of the virtual object instructs the sharing of the virtual object to a user, the virtual object is provided without confirming the conditions. (Configuration 11) The system according to any one of Configurations 1 to 10, characterized in that the terminal includes a head-mounted display.

[0058] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0059] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist.

Claims

1. A terminal that can project virtual objects into the real world, A system including a management means for managing real-world feature quantities for displaying virtual objects linked to the real world, by associating them with identification information corresponding to the virtual objects, The aforementioned management means further, For the same feature, it is possible to associate and manage identification information corresponding to multiple different virtual objects. As conditions for determining how to provide the virtual object to the user, different conditions can be associated and managed for each of the identification information corresponding to the multiple different virtual objects. The aforementioned different conditions include conditions regarding whether or not there is an agreement to the contract in the service that provides the virtual object, The aforementioned multiple different virtual objects include a first virtual object provided to a user who has not agreed to the agreement, and a second virtual object provided to a user who has agreed to the agreement. The management means manages a condition indicating that the user has not agreed to the contract in association with the first identification information corresponding to the first virtual object, and manages a condition indicating that the user has agreed to the contract in association with the second identification information corresponding to the second virtual object. The terminal executes a request using features in the real world that can be the target of a virtual object projection, and projects the first virtual object or the second virtual object onto the real world, which is provided based on conditions corresponding to one or more pieces of identification information associated with the features included in the request and whether or not the contract has been agreed upon, as identified from the user information of the user using the terminal. A system characterized by the following features.

2. The management means can manage each of the identification information corresponding to the different multiple virtual objects by associating conditions based on the personal attributes of the user corresponding to the user information, in addition to conditions regarding whether or not the contract has been agreed upon. The system according to claim 1, wherein the terminal projects onto the real world, when the conditions based on the user's personal attributes differ for the candidate virtual object selected from the plurality of different virtual objects associated with the feature quantities included in the request, based on whether or not the user has agreed to the contract and the conditions relating to whether or not the user has agreed to the contract, based on the user's personal attributes identified from the user information and the conditions based on the user's personal attributes.

3. The management means can manage each of the identification information corresponding to the different multiple virtual objects by associating conditions based on the environment to which the user corresponding to the user information belongs, in addition to conditions regarding whether or not the contract has been agreed upon. The system according to claim 1, wherein the terminal projects, onto the real world, a virtual object provided based on the environment to which the user belongs and the conditions based on the environment to which the user belongs, when the conditions based on the environment to which the user belongs differ for a candidate virtual object selected from a plurality of different virtual objects associated with the feature quantities included in the request, based on whether or not the user has agreed to the contract and the conditions related to whether or not the user has agreed to the contract, as identified from the user information.

4. The system according to claim 1, characterized in that the method for providing the virtual object to the user is set by the owner of the virtual object.

5. The system according to claim 4, characterized in that if the owner of the virtual object instructs the sharing of the virtual object to a user, the virtual object is provided to the terminal without confirming the conditions.

6. The system according to claim 1, characterized in that the terminal includes a head-mounted display.

7. A terminal that can project virtual objects into the real world, A control method for a system including a management means for managing real-world feature quantities for displaying virtual objects linked to the real world, by associating them with identification information corresponding to the virtual objects, In the aforementioned management means, The process involves associating and managing identification information corresponding to multiple different virtual objects for the same feature quantity, The method for providing the virtual object to the user includes a step of managing different conditions associated with each of the identification information corresponding to the multiple different virtual objects, as a condition for providing the virtual object to the user. The aforementioned different conditions include conditions regarding whether or not there is an agreement to the contract in the service that provides the virtual object, The aforementioned multiple different virtual objects include a first virtual object provided to a user who has not agreed to the agreement, and a second virtual object provided to a user who has agreed to the agreement. In the process of managing the aforementioned conditions in association, a condition indicating that the person has not agreed to the contract is associated and managed with the first identification information corresponding to the first virtual object, and a condition indicating that the person has agreed to the contract is associated and managed with the second identification information corresponding to the second virtual object. In the aforementioned terminal, The process involves executing a request using features in the real world that can be the target of projection of a virtual object, and projecting the first or second virtual object onto the real world based on conditions corresponding to one or more pieces of identification information associated with the features included in the request and whether or not the contract has been agreed upon, as identified from the user information of the user using the terminal. A method for controlling a system characterized by the following features.

Images