System and method for controlling the system

Abstract

To provide a system that can ascertain an actual browsing state of a shared virtual object.SOLUTION: A system that manages a virtual object includes: a browsing state receiving unit 321 which receives, from one or more terminals, data regarding display of the virtual object in each terminal based on information which is provided from a service for managing feature quantities in the real world for displaying the virtual object in association with the real world, in association with identification information; and a browsing state providing unit which provides information on the virtual object obtained based on the acquired data.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a system for managing virtual objects.

Background Art

[0002] Technologies for creating a space that combines the real and virtual worlds, such as VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality), to provide a pseudo-experience, have attracted attention. XR is a general term for these. In recent years, a mechanism for displaying a single virtual object on the same location in the real world on multiple terminals has been realized on the platforms provided by each company. For example, there is a cloud system that associates and manages virtual objects to be placed in the real world with feature amounts of the real world captured by a camera or the like. By capturing the real world that matches the feature amounts managed by the system with the camera of an arbitrary terminal, it becomes possible to view the virtual objects associated and managed with the feature amounts on that arbitrary terminal. Further, virtual objects can be shared among multiple users in the same space. Patent Document 1 discloses an information processing apparatus that displays identification information for identifying whether a shared user is set to be permitted access to a virtual object in the vicinity of the shared user on the screen. In Patent Document 1, it is identified whether a shared user is permitted access to a virtual object according to the access right of the user to the virtual object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, while the technology described in Patent Document 1 can identify users who can access virtual objects, it is difficult to determine which users are actually viewing the virtual objects because the virtual objects do not exist in the real world. For example, even if a user is given access to a virtual object, they may not actually be able to view it if their device is unable to properly recognize real-world feature points. In AR-based meetings or classes, users sharing virtual objects may want to confirm whether they are displaying the virtual object correctly to the users receiving it. Furthermore, at exhibitions and events, there is a need to understand which virtual objects are actually being viewed most frequently by users who have access to the venue and have permission to view them.

[0005] The present invention aims to provide a system that can grasp the actual viewing status of shared virtual objects. [Means for solving the problem]

[0006] To solve the above problems, the present invention provides a system for managing virtual objects, A browsing information detection means for detecting whether a virtual object is being viewed on each terminal, and a browsing status transmission means for transmitting the browsing status detected by the browsing information detection means on each terminal. Based on information provided by a service that manages real-world features associated with identification information for displaying virtual objects linked to the real world from one or more devices. including the data of the browsing status detected by the browsing information detection means Acquisition means for acquiring data relating to the display of the virtual object on each terminal, and the virtual object obtained based on the acquired data Viewing status It has a means of providing information relating to the present invention. [Effects of the Invention]

[0007] According to the present invention, it is possible to understand the actual viewing status of a shared virtual object. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of a virtual object management system. [Figure 2] This diagram shows the hardware configuration of the client terminal. [Figure 3] This diagram shows the hardware configuration of the virtual object management server. [Figure 4] This diagram shows the software configuration of the virtual object management system. [Figure 5] This diagram shows the field of view of users registering and viewing shared virtual objects. [Figure 6] This is a sequence diagram showing the process flow for registering and rendering shared virtual objects. [Figure 7] This is a flowchart showing the anchor generation process. [Figure 8] This is a flowchart showing the virtual object rendering process on the viewer terminal. [Figure 9] This is a flowchart showing the browsing status display process on the owner's terminal. [Figure 10] This figure shows an example of how the viewing status of a virtual object is displayed in Example 1. [Figure 11] This figure shows an example of how the viewing status of a virtual object is displayed in Example 2. [Figure 12] This figure shows an example of statistical information for a virtual object in Example 2. [Figure 13] This flowchart shows the process for checking the viewing status of a shared virtual object. [Figure 14] This figure shows an example of how the viewing status of a virtual object is displayed in Example 3. [Modes for carrying out the invention]

[0009] (Example 1) Figure 1 shows the overall configuration of a system for managing virtual objects. The virtual object management system includes a virtual object management server 111 that provides virtual objects, a statistics server 121, and client terminals that can project the virtual objects provided by the virtual object management server 111 into the real world. In this embodiment, an example is described in which client terminals 131 to 133 are connected to the virtual object management server 111.

[0010] The virtual object management server 111 and the statistics server 121 are connected to each other via network 100. Client terminal 131 is connected to the virtual object management server 111 and the statistics server 121 via network 100 and network 101. Client terminals 132 and 133 are connected to the virtual object management server 111 and the statistics server 121 via network 100 and network 101. Network 100 is the internet, and networks 101 and 102 are the internet, networks within homes, companies, schools, wireless LANs set up in towns, etc. Networks 100 to 102 can be any so-called communication networks, such as LANs, WANs, telephone lines, dedicated digital lines, ATM or frame relay lines, cable television lines, wireless lines for data broadcasting, etc. Networks 100 to 102 only need to be capable of sending and receiving data.

[0011] Client terminals 131 to 133 are terminals capable of photographing the real world, displaying virtual objects, and communicating with the virtual object management server 111 in order to project virtual objects into the real world. The client terminals 131 to 133 are, for example, dedicated hardware corresponding to the rendering of virtual objects handled by XR such as HMD (Head-Mounted Display), smart glasses, or communication devices with a built-in program execution environment such as a smartphone. When the client terminals 131 to 133 are not dedicated hardware corresponding to the rendering of virtual objects, such as a smartphone, they perform the rendering of virtual objects using APIs provided by a web browser, an OS, or the like. The client terminals 131 to 133 include a camera for photographing the surroundings and a display for displaying virtual objects. The client terminals 131 to 133 photograph the surroundings through the camera and project and display virtual objects onto the real world photographed by the camera on the display, thereby providing the user with a pseudo-experience that combines the real and virtual worlds.

[0012] The virtual object management server 111 manages by associating feature amounts in the real world for associating virtual objects with the real world with identification information, and provides a service that provides virtual objects to external terminals. The virtual object management server 111 is constructed using, for example, a server computer. In addition, in this embodiment, an example in which the virtual object providing service is provided by the virtual object management server 111 will be described, but it is not limited to this. The services and functions provided by the virtual object management server 111 may also be realized by a virtual machine (cloud service) using resources provided by a data center including an information processing apparatus in addition to one or a plurality of information processing apparatuses, or a combination thereof.

[0013] Furthermore, as part of the virtual object provision service, the virtual object management server 111 manages virtual objects placed in the real world by associating them with real-world feature quantities captured by cameras, etc. In this embodiment, the virtual object management server 111 manages virtual objects and real-world feature quantities captured by cameras, etc. by associating them and managing them using anchors. An anchor includes the virtual object, real-world feature quantities for displaying the virtual object in association with the real world, as well as an identifier and session ID for identifying the anchor itself. The anchor manages at least three pieces of information as feature quantities for displaying the virtual object in association with the real world: the feature quantities shown in Table 2 (described later), location information, and sensor information. The virtual object management server 111 receives anchor registration requests from client terminals 131 to 133 and manages the registered anchors. The virtual object management server 111 also returns anchors that match the conditions from among the managed anchors in response to anchor acquisition requests from client terminals 131 to 133. The virtual object management server 111 also manages user information for client terminals 131 to 133. The virtual object management server 111 receives user login / logout requests from client terminals 131-133 and performs login / logout processing.

[0014] The statistics server 121 is a server that provides a service that understands and provides the viewing status of virtual objects. The statistics server 121 obtains the viewing status of virtual objects from one or more client terminals and manages the viewing status of virtual objects for each user. In this embodiment, an example is described in which the service for managing the viewing status of virtual objects is provided by the statistics server 121, but it is not limited to this. The services and functions provided by the statistics server 121 may be realized by one or more information processing devices, a virtual machine (cloud service) that utilizes resources provided by a data center including information processing devices, or a combination of these. Furthermore, the functions of the virtual object management server 111 and the statistics server 121 may be realized by a single server.

[0015] FIG. 2 is a diagram showing the hardware configuration of client terminals 131 to 133. The client terminals 131 to 133 include a CPU 202, a GPU 210, a RAM 203, a ROM 204, a HDD 205, a NIC 209, a camera 207, a display 206, and an interface 208. These are connected to a system bus 201.

[0016] The CPU (Central Processing Unit) 202 controls the entire terminal. The GPU (Graphics Processing Unit) 210 performs arithmetic operations necessary for rendering virtual objects in real time. The RAM (Random Access Memory) 203 is a temporary storage means and functions as the main memory, work area, etc. of the CPU 202 and the GPU 210. The ROM (Read Only Memory) 204 is a memory dedicated to data reading and stores various data such as a basic I / O program. The HDD (Hard Disc Drive) 205 is a large-capacity memory that stores application programs such as a web browser, an operating system (OS), related programs, various data, etc. Note that the HDD 205 is an example of a storage device, and it may be a memory such as a solid state drive (SSD) or an external storage device. The CPU 202 comprehensively controls each unit connected to the system bus 201 by loading and executing a program stored in a memory (ROM 204 or HDD 205) into the RAM 203.

[0017] The display 206 is a display means for displaying virtual objects, information necessary for operations, etc. When the client terminal is a smartphone, a tablet, or the like, the display 206 may be a touch panel in which the display means and the input means are integrated. By associating the input coordinates and the display coordinates on the touch panel, a GUI can be configured as if the user can directly operate the screen displayed on the touch panel.

[0018] Camera 207 includes an outward-facing camera that captures images of the surroundings and an inward-facing camera that primarily captures images of the user. By analyzing the images captured by Camera 207 with an application program stored in HDD 205, it becomes possible to overlay virtual objects onto the real world and calculate features of the real world. Furthermore, if client terminals 131-133 are dedicated XR devices such as HMDs, it is possible to manipulate the virtual objects displayed on display 206 using the user's finger movements recognized by Camera 207. If client terminals 131-133 are not dedicated XR devices such as smartphones, it is possible to manipulate the virtual objects displayed on display 206 by operating the touch panel or other controls on display 206.

[0019] The NIC (Network Interface Card) 209 is a network interface that exchanges data with external devices such as the virtual object management server 111 via networks 101 and 102. Interface 208 is an interface with external devices and connects peripheral devices such as various external sensors. Note that the configuration in Figure 2 is just an example, and the configuration of client terminals 131 to 133 is not limited to this. For example, the storage location of data and programs can be changed to ROM 204, RAM 203, HDD 205, etc., depending on their characteristics.

[0020] Figure 3 shows the hardware configuration of the virtual object management server 111 and the statistics server 121. The virtual object management server 111 and the statistics server 121 have a CPU 222, RAM 223, ROM 224, HDD 225, NIC 229, and interface 228. These are connected to the system bus 221. The CPU 222 controls the virtual object management server 111. The RAM 223 is a temporary storage means and functions as the main memory, work area, etc., of the CPU 222. The ROM 224 is a memory dedicated to data reading and stores various data such as basic I / O programs. The HDD 225 is a large-capacity memory that stores the service server group programs, operating system (OS), related programs, various data, etc. The CPU 222 comprehensively controls each unit connected to the system bus 221 by loading programs stored in memory (ROM 224 or HDD 225) into RAM 203 and executing them.

[0021] NIC229 is a network interface that exchanges data with external devices such as client terminals 131-133 and other servers via network 100. Interface 228 is an interface to external devices and connects peripheral devices. Note that the configuration in Figure 3 is an example, and the configuration of the virtual object management server 111 and statistics server 121 is not limited to this.

[0022] Figure 4 shows the software configuration of the virtual object management system according to this embodiment. The software configuration of the client terminals 131 to 133 shown in Figure 4 is realized by the CPU 202 and GPU 210 executing processing based on programs stored in memory (ROM 204 or HDD 205). Similarly, the software configuration of the virtual object management server 111 and statistics server 121 shown in Figure 4 is realized by the CPU 222 of each server executing processing based on programs stored in memory (ROM 224 or HDD 225).

[0023] The virtual object management server 111 includes an anchor management unit 311, an anchor receiving unit 312, an anchor providing unit 313, a user management unit 314, and a login processing unit 315. The user management unit 314 manages user information for users of the virtual object management system. Table 1 is an example of a user information management table managed by the user management unit 314. [Table 1]

[0024] The user attribute information management table manages user ID, password, user privileges, and login expiration date. The user attribute information management table may also manage user attribute information such as username, age, and gender. The user ID is information used to uniquely identify a user. The password is the password used for login authentication (basic authentication) set by the user. The privilege column indicates the privileges of each user. In this embodiment, among client terminals 131-133, client terminals logged in with "viewer" privileges are called viewer terminals, and client terminals logged in with "owner" privileges are called owner terminals. Users logged in with "viewer" privileges are considered viewers. Viewers can view virtual objects placed by the owner. Users logged in with "owner" privileges are considered owners. Owners have the privilege to place and view virtual objects. The login expiration date indicates the expiration date of the logged-in user's authentication status. The user attribute information management table may also include a column for confidential attributes indicating whether or not to disclose the viewing status.

[0025] The login processing unit 315 receives login requests from client terminals 131-133, compares them with user information managed by the user management unit 314, and returns the login processing result to client terminals 131-133. The login processing unit 315 compares the combination of user ID and password included in the login request with the user information (Table 1) managed by the user management unit 314. If the information in the login request matches the user information managed by the user management unit 314, the login processing unit 315 returns the login processing result as successful to client terminals 131-133.

[0026] The anchor management unit 311 manages anchor information. Table 1 is an example of an anchor management table managed by the anchor management unit 311. An anchor includes anchor ID, session ID, virtual object data, feature quantities, location information, sensor information, and manufacturer. [Table 2]

[0027] The anchor ID is identification information used to uniquely identify an anchor, and also serves as identification information corresponding to a virtual object. The anchor ID is assigned when the anchor receiving unit 312 saves a record to the anchor management table. The session ID is an identifier used to link multiple anchors as a single group. Anchors belonging to the same session will be assigned the same session ID. By linking multiple anchors to a single session ID, multiple anchors with the same session ID can be presented to the user simultaneously. In other words, a provision method defined by the session ID can provide a different virtual object from a given virtual object.

[0028] Virtual object data is data of a 3D model of a virtual object in any format. Feature vectors, location information, and sensor information are features used to display the virtual object in relation to the real world. Feature vectors, for example, represent 3D information of the real world obtained by analyzing data captured by camera 207 around where the anchor is placed. Location information indicates the 3D position of the virtual object in the real world. Sensor information includes location information of the anchor (GPS coordinates), the Beacon and Wi-Fi ID to which the anchor is associated, etc. The location where the virtual object is placed can be identified by the Beacon and Wi-Fi ID to which the anchor is associated. The creator is the user ID of the user who created the anchor, and one of the values ​​in the User ID column of Table 1 is stored there.

[0029] The anchor receiving unit 312 receives anchor registration requests from client terminals 131 to 133. The anchor receiving unit 312 then stores the anchors included in the received anchor registration requests in the anchor management unit 311. The anchor providing unit 313 receives anchor acquisition requests from client terminals 131 to 133, searches for anchors that match the conditions in the anchor management unit 311, and returns them to client terminals 131 to 133. The anchor providing unit 313 can return one anchor associated with a specific anchor ID in response to an anchor acquisition request from client terminals 131 to 133, or it can return multiple anchors associated with the same session ID or the same sensor ID. In addition, the anchor acquisition requests from client terminals 131 to 133 include information about the logged-in user (such as the user ID). Therefore, the anchor providing unit 313 can also control whether or not to provide an anchor based on the logged-in user information included in the anchor acquisition request.

[0030] Client terminals 131 to 133 each have a virtual object data management unit 301, an anchor generation unit 302, an anchor acquisition unit 303, an anchor drawing unit 304, a login unit 305, and a local anchor management unit 306. Furthermore, client terminals 131 to 133 are equipped with a viewing status detection unit 307, a viewing status transmission unit 308, a viewing status acquisition unit 309, and a local viewing status management unit 310 for managing the viewing status of shared virtual objects.

[0031] The virtual object data management unit 301 manages the 3D data of virtual objects. The 3D data in various formats stored in the virtual object data management unit 301 are virtual objects that users can freely place on top of the real world. The anchor generation unit 302 creates anchors based on user operations. The user selects a 3D model stored in the virtual object data management unit 301 through the anchor generation unit 302 and places the virtual object in the real world using hand movements captured by the camera 207 or touch panel operations on the display 206.

[0032] When a virtual object is placed, the anchor generation unit 302 analyzes the image to extract feature quantities from the area around the virtual object, associates them with the virtual object, and stores them in the local anchor management unit 306. The anchor generation unit 302 also uses the GPS sensor connected via interface 208 to identify the virtual object's location information and associates it with the anchor. The user can associate anchors with sensors through the anchor generation unit 302. The anchor generation unit 302 stores the generated anchors in the local anchor management unit 306 and also transmits them to the anchor receiver unit 312.

[0033] The anchor acquisition unit 303 acquires anchors (virtual objects) from the virtual object management server 111. Specifically, the anchor acquisition unit 303 sends an anchor acquisition request to the virtual object management server 111 and receives anchors from the virtual object management server 111 as a response to the anchor acquisition request. The anchors acquired from the virtual object management server 111 are then stored in the local anchor management unit 306. The anchor acquisition unit 303 includes identification information of the virtual object or feature quantities for displaying the virtual object linked to the real world in the anchor acquisition request sent to the virtual object management server 111. These feature quantities include feature quantities of the real world captured and extracted by the camera 207, location information such as GPS signals, Wi-Fi and Beacon information, and sensor information such as signals detected by a sensor that detects Bluetooth signals connected to the interface 208. Alternatively, the feature quantities may be information read from image codes via the camera 207. By making an anchor acquisition request using these feature quantities, it is possible to acquire anchors near the current location based on GPS signals, or anchors linked to detected Wi-Fi or Beacon devices.

[0034] The anchor drawing unit 304 places virtual objects contained in an anchor in the real world based on the features contained in the anchor. The anchor drawing unit 304 compares the features contained in each anchor stored in the local anchor management unit 306 with the real world image captured by the camera 207, and places the virtual objects contained in the anchor in the area of ​​real space where the features of the real space region match. The local anchor management unit 306 manages anchors on each client terminal. The local anchor management unit 306 stores and manages anchors generated on each client terminal and anchors obtained from the virtual object management server 111.

[0035] Here, an example of displaying virtual objects on a client terminal will be explained using Figure 5. Figure 5(A) shows an image displayed on the display 206 of an HMD-type client terminal 131. In this embodiment, client terminal 131 is an owner terminal operated by an owner who has logged in with "owner" privileges. The owner has the authority to place and view virtual objects, and can place virtual objects on the owner terminal. The owner uses hand gestures 403 on client terminal 131 to manipulate a cylindrical virtual object 402 stored in the virtual object data management unit 301 of client terminal 131 and place it on a desk 401 in the real world. Note that the operation using hand gestures 403 is merely an example, and the operation method is not limited to this.

[0036] Figures 5(B) and 5(C) show images displayed on the display 206 of an HMD-type client terminal 133. In this embodiment, client terminal 133 is a viewer terminal operated by a viewer logged in with "viewer" privileges. The viewer can view virtual objects placed by the owner on the viewer terminal. Note that the configuration of client terminals 131 to 133 is the same whether it is the owner terminal or the viewer terminal; only the logged-in user is different. The anchor generation unit 302 extracts feature quantities by analyzing the video of the surrounding area where the virtual object is placed, captured via the camera 207, associates them with the virtual object, and saves them in the local anchor management unit 306. In Figure 5(B), the viewer is viewing a cylindrical virtual object 422 on desk 421 with the same feature quantities on client terminal 133, where the owner placed a cylindrical virtual object 402 on desk 401 using client terminal 131. On the other hand, in Figure 5(C), although the desk 421 is partially within the field of view, the field of view does not reach the position where the virtual object 422 should be displayed. Therefore, the anchor drawing unit 304 does not display the virtual object 422.

[0037] The login unit 305 sends the username and password to the login processing unit 315 of the virtual object management server 111 to authenticate the user. Authentication is performed, for example, by a combination of username and password. The username and password are entered, for example, by the user's finger movements captured by the camera 207, by the operation of the touch panel on the display 206, or by a keyboard connected to the interface 208. User authentication is performed in the login processing unit 315 of the virtual object management server 111, and if the user is able to log in to the service that provides virtual objects, the virtual objects become available for display. Note that the login method is not limited to a combination of username and password. For example, biometric authentication such as facial recognition using a face image captured by the camera 207, iris recognition using the iris, or fingerprint authentication using a fingerprint sensor connected to the interface 208 may also be used.

[0038] Next, we will describe the configuration for the owner terminal, client terminal 131, to understand the viewing status of virtual objects on the viewer terminal. Client terminals 131 to 133 each have a viewing status detection unit 307, a viewing status transmission unit 308, a viewing status acquisition unit 309, and a local viewing status management unit 310. Of these, the viewer terminal performs processing using the viewing status detection unit 307, the viewing status transmission unit 308, and the local viewing status management unit 310. On the other hand, the owner terminal performs processing using the viewing status acquisition unit 309 and the local viewing status management unit 310.

[0039] The viewing status detection unit 307 and the viewing status transmission unit 308 are software modules that function on the client terminal 133, which is a viewer terminal. The viewing status detection unit 307 detects the viewing status of the virtual object on the client terminal 133, that is, whether the virtual object is actually being viewed on the client terminal 133. The viewing status detection unit 307 detects the viewing status of the virtual object on the client terminal 133 by determining whether the anchor drawing unit 304 is displaying the virtual object on the display 206. Here, the state of whether or not the virtual object 422 is actually being displayed on the client terminal 133, which is a viewer terminal, is defined as "viewing status". Below is an example of viewing status data managed by the viewer terminal in JSON format. "visible" is a viewing status flag that indicates the viewing status. { "session": { "sessionId": "111", "sessionName": "Lesson 1", "users": [ { "userId": "userA", "displayUserName": "User A", "anchors": [ { "anchorId": "a", "anchorName": "Cylinder A", "visible": true / / Virtual object is being viewed } ] } ] } }

[0040] For example, in the state shown in Figure 5(B), the anchor drawing unit 304 of the client terminal 133 is displaying the virtual object 422, so the viewing state detection unit 307 determines that the virtual object 422 is in a viewing state. When it is determined that it is in a viewing state, the viewing state detection unit 307 sets "visible" in the data to true. On the other hand, in the state shown in Figure 5(C), although the desk 421 is partially in the field of view, the field of view has not reached the position where the virtual object 422 should be displayed, so the anchor drawing unit 304 is not displaying the virtual object 422. Therefore, the viewing state detection unit 307 determines that the virtual object 422 is not in a viewing state and sets "visible" in the data to false. Note that moving the client terminal 133 to the right from the field of view in Figure 5(C) results in the field of view in Figure 5(B). When the field of view is as shown in Figure 5(B) and the anchor drawing unit 304 displays the virtual object 422, the viewing state detection unit 307 determines that the virtual object 422 is in a viewing state and updates "visible" in the data from false to true. The viewing state detection unit 307 also saves the detected viewing state to the local viewing state management unit 310.

[0041] The browsing status transmission unit 308 transmits the browsing status of the virtual object 422 on the client terminal 133, as detected by the browsing status detection unit 307, to the statistics server 121. In response to changes in the browsing status, the browsing status transmission unit 308 transmits the browsing status of the virtual object 422 on the client terminal 133 to the statistics server 121.

[0042] The browsing status acquisition unit 309 is a software module that functions on the client terminal 131, which is the owner terminal. The browsing status acquisition unit 309 acquires the browsing status of each user's virtual object 422 from the browsing status provision unit 322 of the statistics server 121. The client terminal 131, which is the owner terminal, can understand the actual browsing status of users by acquiring the browsing status via the browsing status acquisition unit 309. The browsing status acquisition unit 309 also saves the browsing status of the virtual object acquired from the statistics server 121 to the local browsing status management unit 310.

[0043] The local browsing status management unit 310 saves the browsing status of the virtual object 422 detected by the browsing status detection unit 307 and the browsing status of the virtual object 422 for each viewer acquired by the browsing status acquisition unit 309.

[0044] The statistics server 121 includes a viewing status receiving unit 321, a viewing status providing unit 322, a viewing status management unit 323, and a viewing status aggregation unit 324. The viewing status receiving unit 321 receives the viewing status of the virtual object 422 as data related to the display of the virtual object 422 on the client terminal 133, which is a viewer terminal, from the viewing status transmission unit 308 of the client terminal 133. The viewing status receiving unit 321 stores the viewing status of the virtual object received from the viewer terminal in the viewing status management unit 323.

[0045] The browsing status provision unit 322 provides the browsing status for each user, managed by the browsing status management unit 323 of the statistics server 121, in response to a request from the browsing status acquisition unit 309 of the client terminal 131, which is the owner terminal. The browsing status management unit 323 stores the browsing status received by the browsing status receiving unit 321 and the aggregated browsing status aggregated by the browsing status aggregation unit 324. The browsing status aggregation unit 324 aggregates the browsing status stored in the browsing status management unit 323 in a way that is easy for the owner terminal to handle, and saves it again to the browsing status management unit 323 as aggregated browsing status. In this way, the browsing status of a user can be understood by detecting the browsing status on the client terminal 133, which is the viewer terminal, and sending it to the statistics server 121, and by the client terminal 131, which is the owner terminal, obtaining the browsing status from the statistics server 121.

[0046] First, we will explain the process from registering an anchor generated by client terminal 131 with the virtual object management server 111, to the client terminal 133 retrieving and displaying the anchor stored in the virtual object management server 111. Figure 6 is a sequence diagram showing the flow of processing for registering and rendering virtual objects. The user who is the owner operating client terminal 131 is referred to as "userX" in Table 1, and the user who is the viewer operating client terminal 133 is referred to as "userA" in Table 1. The anchor that userX registers by operating the client terminal is the anchor represented by anchor ID "a" in Table 2. Furthermore, the processing performed on the client terminal shown in Figure 6 is realized by the client terminal's CPU 202 or GPU 210 reading the program stored in memory into RAM 203 and executing it. The processing performed on the virtual object management server 111 shown in Figure 6 is realized by the virtual object management server 111's CPU 222 reading the program stored in memory into RAM 223 and executing it.

[0047] First, S601 to S605 describe the sequence in which the client terminal 131 operated by userX registers an anchor with the virtual object management server 111. In S601, the login unit 305 of the client terminal 131 sends the user ID and password entered by the user to the login processing unit 315 of the virtual object management server 111. In S602, the login processing unit 315 of the virtual object management server 111 verifies the user information received from the client terminal 131 and returns the result of the user authentication process for login to the client terminal 131. For example, if the login processing unit 315 confirms that the user information received from the client terminal 131 matches the user ID and password managed by the user management unit 314 for userX, it returns the login result to the client terminal 131 as a successful login.

[0048] In S603, the anchor generation unit 302 of the client terminal 131 generates anchors for virtual objects stored in the virtual object data management unit 301 deployed by userX and stores them in the local anchor management unit 306. In S604, the anchor generation unit 302 of the client terminal 131 sends an anchor registration request for the anchors generated in S603 to the anchor receiving unit 312 of the virtual object management server 111. In S605, the anchor receiving unit 312 of the virtual object management server 111 registers the anchors received from the client terminal 131 with the anchor management unit 311 and returns the registration result to the anchor generation unit 302 of the client terminal 131. If multiple anchors are to be registered with the same session ID, the series of processes shown in S611 (S603-S605) are repeated.

[0049] Here, the details of the anchor generation process performed by the anchor generation unit 302 of the owner terminal in S603 will be explained using Figure 7. Figure 7 is a flowchart of the anchor generation process. Each process shown in Figure 7 is realized by the CPU 202 or GPU 210 of the client terminal 131 reading the program stored in memory into RAM 203 and executing it.

[0050] In S701, the anchor generation unit 302 places a virtual object in real-world space based on user input and determines its position and orientation. Next, in S702, the anchor generation unit 302 captures the surroundings through the camera 207 and acquires three-dimensional features of the space. In S703, the anchor generation unit 302 determines whether sufficient features have been collected in S702. If it determines that sufficient features have not been collected, it repeats the process in S702. On the other hand, if it determines that sufficient features have been collected, it proceeds to the process in S704.

[0051] In S704, the anchor generation unit 302 sets the feature quantities acquired in S702, as well as the expiration date and other necessary properties for the anchor. The expiration date and property information to be set for the anchor may be set by user operation, or default settings may be automatically assigned. In S705, the anchor generation unit 302 determines whether or not to set sensor information for the anchor. The anchor generation unit 302 determines whether or not to set sensor information for the anchor based on whether or not sensor information that can be set for the anchor, such as GPS information, has been detected on the client terminal 131. Alternatively, the anchor generation unit 302 may display a screen to the user to confirm whether or not to set sensor information, and determine whether or not to set sensor information for the anchor based on the user's operation. If it is determined that sensor information should be set for the anchor, the process in S706 is performed. On the other hand, if it is determined that sensor information should not be set for the anchor, this process is terminated.

[0052] In S706, the anchor generation unit 302 sets sensor information for the anchor. For example, if sensor information that can be set for an anchor is detected on the client terminal 131, the anchor generation unit 302 sets the detected sensor information for the anchor. Alternatively, the anchor generation unit 302 may accept settings according to the type of sensor specified by the user and set the sensor information for the anchor. For example, the anchor generation unit 302 associates the anchor information with the Beacon with id=123 according to the user's operation. In this way, an anchor is generated for a virtual object created on the owner terminal and placed in the real space, with property information such as spatial features and expiration dates, and sensor information set, and registered with the virtual object management server 111.

[0053] Next, from S621 to S632, we will describe the sequence in which the client terminal 133 operated by userA retrieves and displays virtual objects from the virtual object management server 111. In S621, the login unit 305 of the client terminal 133 sends the user ID and password entered by the user to the login processing unit 315 of the virtual object management server 111. In S622, the login processing unit 315 of the virtual object management server 111 verifies the user information received from the client terminal 131 and returns the result of the user authentication process for login to the client terminal 133. For example, if the login processing unit 315 confirms that the user information received from the client terminal 133 matches the user ID and password managed by the user management unit 314 for userA, it returns the login result to the client terminal 131 as a successful login.

[0054] In S623, the anchor acquisition unit 303 of the client terminal 133 acquires sensor information. The anchor acquisition unit 303 acquires a signal from the Beacon terminal as sensor information, for example, via a sensor that detects Bluetooth signals connected to the client terminal 133 via interface 208. The sensor information to be acquired may also be information about the Wi-Fi to which the client terminal 133 is connected, or information read from an image code via camera 207. If sensor information cannot be acquired, as shown in S641, the anchor acquisition unit 303 repeats the process in S623 until sensor information can be acquired.

[0055] In S624, the anchor acquisition unit 303 of the client terminal 133 sends a request for provision of a virtual object (anchor search request) to the anchor provision unit 313 of the virtual object management server 111. The anchor search request includes anchor identification information (i.e., identification information corresponding to the virtual object), at least one real-world feature for displaying the virtual object, and user information. For example, the anchor search request includes user information and sensor information acquired in S623. For example, if S623 detects a signal from the Beacon terminal with id=123, the anchor acquisition unit 303 sends an anchor search request associated with the Beacon terminal with id=123 to the anchor provision unit 313 along with user information for userA. In this embodiment, in addition to user information, an example has been described in which sensor information, which is one of the real-world feature for displaying the virtual object, is acquired in S623, and this is used to request the provision of a virtual object from the virtual object management server 111. However, this is not the only way to do so. The information acquired in S623 and sent to the virtual object management server 111 in S624 may also be other information that indicates feature quantities, or it may be identification information corresponding to a virtual object.

[0056] In S625, the anchor provision unit 313 of the virtual object management server 111 searches for an anchor in the anchor management unit 311 based on an anchor search request received from the client terminal 133, and determines which anchor to return to the client terminal 133. For example, if the received request contains identification information corresponding to a virtual object, the anchor provision unit 313 determines that the information (anchor) of the virtual object corresponding to the identification information should be returned. Also, if the received request contains a feature, the anchor provision unit 313 narrows down the anchor to be returned based on the feature and the anchor information managed by the anchor management unit 311. For example, the anchor provision unit 313 searches for the anchor associated with the Beacon terminal with id=123 in the anchor management unit 311 by referring to the user information management table, etc., and determines which anchor to return.

[0057] In S627, the anchor acquisition unit 303 of the client terminal 133 stores the anchor acquired from the virtual object management server 111 in the local anchor management unit 306. In S628, the anchor acquisition unit 303 of the client terminal 133 sends a search request to the anchor provision unit 313 of the virtual object management server 111 for anchors in the same session as the anchor acquired in S627. For example, if the anchor acquisition unit 303 acquires an anchor with anchor ID "a" in S627, it sends a search request to the anchor provision unit 313 for the same anchor as session ID 111, which has anchor ID "a". If the anchor acquisition unit 303 acquires anchors with anchor IDs "a" and "b" in S627, it sends a search request to the anchor provision unit 313 for session IDs "111" and "222".

[0058] In S629, the anchor provision unit 313 of the virtual object management server 111 searches for an anchor from the anchor management unit 311 that is different from the anchor provided by the same session ID, based on the session ID received from the client terminal 133. Then, in S630, the anchor provision unit 313 of the virtual object management server 111 returns the search result from S629 to the client terminal 133. If an anchor for the same session is found in the search in S629, the anchor provision unit 313 returns the anchor found in S629 to the anchor acquisition unit 303 of the client terminal 133. For example, the anchor provision unit 313 searches for an anchor with session ID=111 and sends the anchor with session ID=111 and anchor ID "d" to the client terminal 133. On the other hand, if no anchor for the same session is found in the search in S629, it sends a message to the client terminal 133 indicating that there are no anchors belonging to the same session.

[0059] In S631, the anchor acquisition unit 303 of the client terminal 133 stores the anchor acquired from the anchor provision unit 313 in S630 in the local anchor management unit 306. In S642, the anchor drawing unit 304 of the client terminal 133 draws a virtual object based on the anchor acquired from the virtual object management server 111 and stored in the local anchor management unit 306 in S627 and S631. If there are multiple anchors stored in the local anchor management unit 306 in S627 and S631, the anchor drawing unit 304 repeats the virtual object drawing process in S642 for the number of stored anchors, as shown in S642. In this embodiment, when the anchor drawing unit 304 of the client terminal 133 draws a virtual object, the viewing status detection unit 307 detects the viewing status of the virtual object and sends the viewing status to the statistics server 121.

[0060] Here, we will explain the details of the anchor drawing process performed by the anchor drawing unit 304 in S632. Figure 8 is a flowchart showing the anchor drawing process in the viewer terminal. Each process shown in Figure 8 is realized when the CPU 202 or GPU 210 of the client terminal 133 reads a program stored in memory into RAM 203 and executes it.

[0061] In S801, the anchor drawing unit 304 acquires feature quantities of real-world regions from the video captured by the camera 207. In S802, the anchor drawing unit 304 determines whether the feature quantities of real-world regions acquired in S801 match the feature quantities of the anchors. If the feature quantities of real-world regions acquired in S801 match the feature quantities of the anchors, the process in S803 is performed. On the other hand, if the feature quantities of real-world regions acquired in S801 do not match the feature quantities of the anchors, the process in S805 is performed.

[0062] In S803, the anchor drawing unit 304 draws a virtual object in real space and displays it on the display 206. Then, in S804, the viewing status detection unit 307 sets the viewing status flag of the viewer for the anchor to "true" and saves the viewing status to the local viewing status management unit 310.

[0063] In S805, the viewing status detection unit 307 saves the viewing status flag of the viewer for the anchor to the local viewing status management unit 310 as "false". In S807, the viewing status detection unit 307 determines whether or not there has been a change in the viewing status flag of the virtual object. If it determines that there has been a change in the viewing status flag of the virtual object, it proceeds to S807. On the other hand, if it determines that there has been no change in the viewing status flag of the virtual object, it returns to the process of S801. In S807, the viewing status transmission unit 308 transmits the viewing status of the virtual object to the viewing status reception unit 321 of the statistics server 121 and returns to the process of S801. The viewing status reception unit 321, which has acquired the viewing status of the virtual object as data related to the display of the virtual object on the viewer terminal from the viewing status transmission unit 308 of the viewer terminal, saves the acquired viewing status to the viewing status management unit 323. The above describes the process of displaying the virtual object associated with the anchor generated by the owner terminal, client terminal 131, on the viewer terminal, client terminal 133, and sending the viewing status to the statistics server 121. While the application is running, the viewer terminal, client terminal 133, repeatedly performs the process shown in Figure 8, drawing the virtual object and sending the viewing status to the statistics server 121.

[0064] Furthermore, the timing and conditions for the browsing status transmission unit 308 to transmit the browsing status of virtual objects to the statistics server 121 can be further controlled. For example, if changes in the browsing status occur continuously in a short period of time, it may overload the statistics server 121. Therefore, the browsing status transmission unit 308 may be controlled to transmit the browsing status of virtual objects only after a certain interval has elapsed. The timing at which the browsing status transmission unit 308 transmits the browsing status of virtual objects and the statistics server 121 acquires it will be after a predetermined interval, thereby reducing the load on the statistics server 121.

[0065] Next, the rendering process of virtual objects and the acquisition of the viewing status on the owner terminal, client terminal 131, as well as an example of using the acquired viewing status, will be explained using Figures 9 and 10. Figure 9 is a flowchart showing the rendering process of virtual objects on the owner terminal. Each process shown in Figure 9 is realized by the CPU 202 or GPU 210 of client terminal 131 reading the program stored in memory into RAM 203 and executing it.

[0066] The process of drawing virtual objects on the owner terminal, client terminal 131, in S801 to S803 is the same as the process of drawing virtual objects on the viewer terminal, client terminal 133, so the explanation is omitted. In S901, the viewing status acquisition unit 309 acquires the viewing status associated with the anchor of the virtual object as information about the virtual object being drawn in S803 from the viewing status provision unit 322 of the statistics server 121. The viewing status provision unit 322 provides the viewing status of the virtual object to the owner terminal at predetermined intervals, for example, when there is a change in the viewing status of the virtual object or when the owner terminal requests the viewing status. The viewing status acquisition unit 309 then saves the viewing status acquired from the statistics server 121 to the local viewing status management unit 310. In S902, the anchor drawing unit 304 draws the viewing status of the virtual object as a virtual object near the virtual object drawn in S803, based on the viewing status acquired in S908, and returns to the process of S801.

[0067] Here, we will describe an example of the display on the display 206 of the client terminal 131, which is the owner terminal, when the anchor drawing unit 304 executes the process in S902. Figure 10 is a diagram showing an example of the display of the viewing status in Embodiment 1. Virtual objects 1001 and 1002 are displayed near the shared virtual object 402. Virtual object 1001 is a virtual object that indicates the viewing status of the shared virtual object 402. Virtual object 1001 is displayed as a result of the anchor drawing unit 304 executing the process in S902. Virtual object 1001 shows, for example, the user ID of the user who is currently viewing virtual object 402 (the same as virtual object 422). Virtual object 1002 also shows the number of users who are viewing virtual object 402.

[0068] The viewer terminal repeatedly detects changes in the viewing status of virtual objects and sends them to the statistics server 121, as shown in Figure 8. The owner terminal also repeatedly obtains the viewing status of shared virtual objects and renders the viewing status of said virtual objects as virtual objects, as shown in Figure 9, making it easy to understand the viewing status of shared virtual objects. Note that the process by which the owner terminal in S901 obtains the viewing status of virtual objects from the statistics server 121 may be performed after a predetermined interval. This can reduce the amount of communication and the processing load on the owner terminal and the statistics server 121.

[0069] Furthermore, if the application is terminated midway through the process on the viewer terminal, the viewing status (viewing status flag) of the virtual object may remain in the "visible" state ("visible": true). Therefore, the viewer terminal can send the viewing status of virtual objects that have been set to "not viewed" ("visible": false) to the statistics server 121 before the application is terminated. In addition, the user management unit 314 can update the viewing status by sending the "not viewed" flag to the viewing status receiving unit 321 so that the viewing status of all viewers becomes "not viewed" when the user's login expiration date shown in Table 1 has passed.

[0070] As described above, according to this embodiment, the viewing status of a shared virtual object on a viewer terminal can be acquired by the statistics server 121, and information about the virtual object obtained based on the acquired viewing status can be provided to the owner terminal. The owner terminal can display the viewing status of the virtual object based on the information about the virtual object provided by the statistics server 121. This makes it possible for the owner to understand the actual viewing status of the shared virtual object.

[0071] (Example 2) In Example 1, as shown in Figure 10, an example of displaying the viewing status was described, which shows a list of users and the number of users viewing the shared virtual object near the virtual object. In Example 2, an example of checking the user's viewing status in a different way than in Example 1 is described. For example, by displaying the viewing status of the virtual object acquired by the viewing status acquisition unit 309 of the owner terminal client terminal 131 on the display 206 with an arbitrary display specification, a different, more detailed method of understanding can be realized. Furthermore, by processing and aggregating the aforementioned JSON format data in the viewing status aggregation unit 324 of the statistics server 121, the viewing status provision unit 322 can provide viewing status with more information.

[0072] Figures 11 and 12 show an example of how the viewing status of a virtual object is displayed in Embodiment 2. In Embodiment 2, the client terminal 132 is described as the owner terminal. The owner terminal does not necessarily have to be a head-mounted display; the client terminal 132 may be a smartphone, PC, tablet, or other device. The viewing status of the shared virtual object is displayed on the display 206 of the client terminal 132, which is the owner terminal.

[0073] UI1100 is the user interface (UI) of an application for checking the viewing status of shared virtual objects. The UI is provided by an application running on client terminal 132. This application has the software configuration of the client terminal described in Figure 3 (virtual object data management unit 301 to local viewing status management unit 310).

[0074] The browsing status acquisition unit 309 of the client terminal 132 acquires the browsing status, which has been aggregated by the browsing status aggregation unit 324 of the statistics server 121, from the browsing status provision unit 322, and displays the UI 1100 on the display 206 based on the aggregated browsing status. The thumbnail 1101 is a thumbnail of a virtual object whose browsing status can be checked from the client terminal 132. The description field 1102 is a description field for the browsing status of the virtual object in the thumbnail 1101. The button 1103 is a button for displaying statistical information about the virtual object in the thumbnail 1101. When the button 1103 is detected to have been pressed by the user, the client terminal 132 displays the statistical information of the virtual object shown in Figure 12.

[0075] Figure 12 shows an example of statistical information for a virtual object. UI 1200 is a UI for displaying statistical information for the virtual object of thumbnail 1101. Graph 1201 shows statistical information regarding the viewing status of the virtual object corresponding to thumbnail 1101. An example of the statistical data required to display Graph 1201 is shown below. The statistical data is generated by the viewing status aggregation unit 324 of the statistics server 121 and stored in the viewing status management unit 323. { "session": { "sessionId": "999", "sessionName": "Exhibition Hall 1" }, "anchor": { "an chorId": "aaa", "sessionName": "Box A" }, "statistics": { "metrics": "total", "unit": "hour", "period": "", "data": [ {"2022 / 12 / 31 9:00:00": 9}, {"2022 / 12 / 31 10:00:00": 16}, {"2022 / 12 / 31 9:00:00": 25}, ... {"2022 / 12 / 31 21:00:00": 0} ] } }

[0076] Statistical data is generated when the viewing status of virtual objects, which are sent from the viewer terminal to the statistics server 121 and stored in the viewing status management unit 323, is aggregated by the viewing status aggregation unit 324 of the statistics server 121. Alternatively, the viewing status acquisition unit 309 may specify query conditions when sending an acquisition request to the viewing status provision unit 322, allowing the viewing status provision unit 322 to acquire and process the viewing status from the viewing status management unit 323. The viewing status acquisition unit 309 of the client terminal 132 acquires statistical data from the statistics server 121 and displays a graph 1201 based on the statistical data in the UI 1200.

[0077] Here, we will describe the process by which the viewing status acquisition unit 309 acquires the viewing status of a virtual object, including statistical data, and displays UI1100 and UI1200. Figure 13 is a flowchart of the process for displaying the viewing status of a virtual object. Each process shown in Figure 13 is realized by the CPU 202 or GPU 210 of the client terminal 132 reading the program stored in memory into RAM 203 and executing it.

[0078] In S1301, the browsing status acquisition unit 309 of the client terminal 132 acquires the browsing status of the virtual object, including statistical data, from the browsing status provision unit 322 of the statistics server 121. Then, in S1302, the browsing status acquisition unit 309 updates the UI, such as UI1100, to the latest browsing status based on the browsing status acquired in S1301, and returns to S1301. By periodically repeating the process shown in Figure 13, the owner can check the browsing status of the virtual object in real time. In this embodiment, the process shown in Figure 13 is described as a process in which the browsing status acquisition unit 309 periodically acquires and displays the browsing status, but the browsing status acquisition unit 309 may also acquire the browsing status when UI1100 or UI1200 is opened.

[0079] Thus, according to this embodiment, the owner can check the viewing status of virtual objects and statistical information related to viewing. Furthermore, the owner can check the detailed viewing status of virtual objects in real time even if they are not in the same space as the viewer.

[0080] (Example 3) Examples 1 and 2 described how to check the viewing status of virtual objects on a viewer terminal from the owner terminal. However, there are cases where the viewer terminal also wants to know the viewing status of virtual objects. Example 3 describes a method for checking the viewing status of virtual objects from the viewer terminal under certain conditions. Although it is possible to understand the viewing status of virtual objects on the viewer terminal by granting the viewer the same "owner" privileges as the owner, extending user privileges is not desirable from an operational standpoint.

[0081] Furthermore, it is conceivable that some viewers may not want their usernames to be known when viewing a virtual object that is accessed by an unspecified number of viewers. In Example 3, a hidden attribute ("hidden" attribute) is added to the viewing status data of the virtual object shown in Example 1 to prevent other viewers from knowing the viewing status. The hidden attribute ("hidden" attribute) is a setting that prevents other viewers from knowing the viewing status by not displaying the username, etc., at the destination where the viewing status information of the virtual object is provided (for example, other viewer terminals). Below is an example of the viewing status data in Example 3, which is shown in JSON format. { "session": { "sessionId": "111", "sessionName": "Lesson 1", "users": [ { "userId": "userA", "displayUserName": "User A", "hidden": true, / / Do not inform other viewers of the viewing status. "anchors": [ { "anchorId": "a", "anchorName": "Cylinder A", "visible": true / / Virtual object is being viewed } ] } ] } }

[0082] Similar to Example 1, the client terminal 133, which is a viewer terminal, sends its viewing status to the statistics server 121 in response to changes in the viewing status of the virtual object. In addition, in order for the viewer terminal to understand the viewing status of other viewers, the viewing status acquisition unit 309 of the client terminal 133 is configured to obtain the viewing status of the virtual object from the viewing status provision unit 322. However, if the user logged into the client terminal 133 has the privilege of "viewer", the viewing status provision unit 322 will not return the viewing status of users for whom "hidden" is true.

[0083] When the viewer terminal, client terminal 133, obtains and displays the viewing status of a virtual object, it performs the same processing as the owner terminal (Figure 9). That is, the anchor drawing unit 304 of client terminal 133 obtains the viewing status of the shared virtual object from the statistics server 121. Then, as shown in Figure 10, it draws the viewing status of the shared virtual object 402 (same as virtual object 422) as virtual objects (virtual object 1001 and virtual object 1002).

[0084] Here, we will explain an example of how to display the viewing status of a virtual object when there is a viewer with the hidden attribute enabled. We will explain using the example where userA, userB, userC, userD, etc. are all viewing a virtual object linked to the same anchor, and only userA has the "hidden" attribute enabled ("hidden" is true). Figure 14 is a diagram showing an example of how to display the viewing status of a virtual object in Example 3.

[0085] Figure 14(A) shows client terminal 133 when logged in as userA. Figure 14(B) shows client terminal 133 when logged in as userB. Figure 14(C) shows client terminal 133 when logged in as userC. The shading of usernames within virtual objects 1400-1402 indicates that it is the user's own username.

[0086] From userA's client terminal 133, virtual object 1400 can be viewed by userB, userC, and others because no other users have the "hidden" attribute enabled ("hidden" is true). On the other hand, from userB's client terminal 133, virtual object 1401 cannot be viewed because userA's hidden attribute is enabled ("hidden" is true). Similarly, from userC's client terminal 133, virtual object 1402 cannot be viewed because userA's "hidden" attribute is enabled ("hidden" is true).

[0087] As described above, according to this embodiment, the viewing status of shared virtual objects can also be displayed on the viewer terminal. Furthermore, by including a privacy attribute in the data that the viewing status transmission unit 308 sends to the viewing status reception unit 321, users who do not want their viewing status to be known can be hidden. In this way, the viewing status of virtual objects can be known from the viewer terminal while respecting the user's wish not to let others know their viewing status by setting a privacy attribute.

[0088] (Example 4) Even if the anchor acquisition unit 303 of the viewer terminal successfully acquires anchor information from the anchor provision unit 313 of the virtual object management server 111, there are cases where the virtual object associated with the anchor cannot be displayed for some reason. For example, if there is a problem with the camera 207 of the client terminal 133, which is the viewer terminal, and it is not able to properly capture features of the real world, or if features of the real world cannot be detected due to the shooting angle, the virtual object cannot be drawn. In this case, the owner terminal is made able to identify viewer terminals where the anchor acquisition unit 303 has successfully acquired the anchor from the virtual object management server 111, but the virtual object associated with the anchor cannot be displayed. By the owner being able to identify viewer terminals that have acquired virtual objects but cannot display them, the owner can support the viewer so that it can properly view the virtual objects.

[0089] The anchor management unit 311 of the virtual object management server 111 manages the provision of anchors to client terminals by the anchor provision unit 313 as the anchor acquisition status of the client terminal. Table 3 is an anchor acquisition status management table that shows the anchor acquisition status of the client terminal. [Table 3]

[0090] The anchor acquisition status management table includes, for example, the anchor ID, session ID, user ID, and anchor acquisition date and time. The anchor ID, session ID, and user ID columns are the same as those described in Tables 1 and 2. The anchor acquisition date and time column indicates the date and time when each user acquired the target anchor. For example, Table 3 shows the state in which the anchor acquisition unit 303 of the client terminal to which each user is logged has acquired an anchor with anchor ID "a" from the anchor provision unit 313.

[0091] Let's take the example of a situation where the anchor acquisition unit 303 of the client terminal 133, where viewer userD is logged in, has acquired an anchor with anchor ID "a" from the anchor provision unit 313, but is unable to display the virtual object associated with that anchor.

[0092] The browsing status management unit 323 of the statistics server 121 obtains the anchor acquisition status of client terminals from the anchor management unit 311 of the virtual object management server 111, as shown in Table 3. This process of obtaining the anchor acquisition status of client terminals from the virtual object management server 111, executed by the browsing status management unit 323, is performed periodically. The browsing status management unit 323 then compares the obtained anchor acquisition status of the client terminals with the virtual object browsing status based on information obtained from the browsing status transmission unit 308 of each client terminal, which is stored within the browsing status management unit 323 itself. If the virtual object is displayed correctly on each client terminal, the virtual object browsing status is transmitted from each client terminal to the statistics server 121. Therefore, the user ID shown in the anchor acquisition status and the user ID included in the virtual object browsing status match. On the other hand, if there are client terminals that are unable to display the acquired virtual object, the user ID corresponding to the user of that client terminal is included in the anchor acquisition status, but not in the virtual object browsing status. Therefore, the viewing status management unit 323 can identify viewers that are unable to display the virtual object by comparing the anchor acquisition status of the client terminal 133 with the viewing status of the virtual object stored in the viewing status management unit 323.

[0093] When the viewing status management unit 323 identifies a viewer that is unable to display a virtual object, it saves the viewing status of the virtual object for each user to itself. The following is an example of the viewing status of a virtual object, which is stored by the viewing status management unit 323 in JSON format and includes information to identify hidden terminals. { "session": { "sessionId": "111", "sessionName": "Lesson 1", "users": [ { "userId": "userA", "displayUserName": "User A", "anchors": [ { "anchorId": "a", "anchorName": "Cylinder A", "visible": true / / Virtual object is being viewed } ] }, ... { "userId": "userD", "displayUserName": "User D", "anchors": [ { "anchorId": "a", "anchorName": "Cylinder A", "visible": false, / / Virtual object not viewable "missing": true / / A certain amount of time has passed since the anchor was obtained. } ] } ] } }

[0094] The client terminals to which viewers userA and userD log in have acquired the anchor with anchor ID "a". The viewing status flag visible for userA, who can view the virtual object associated with the anchor, is "true". On the other hand, the viewing status flag visible for userD, who cannot view the virtual object associated with the anchor, is "false". The attribute "missing" is assigned to userD, who cannot view the virtual object associated with the anchor. In other words, the attribute "missing" is assigned when the viewing status flag visible is "false". The "true" missing attribute is assigned to viewers who have not had a viewing status of the virtual object for a certain period of time. The time during which the viewing status of the virtual object has not been viewed is calculated based on the anchor acquisition time and the current time, which are included in the anchor acquisition status of the client terminal shown in Table 3, which the viewing status management unit 323 periodically obtains from the anchor management unit 311.

[0095] The viewing status acquisition unit 309 of the owner terminal, client terminal 131, acquires the viewing status of a virtual object from the viewing status provision unit 322, which includes information to identify the hidden terminal, as shown in the above JSON format data. The viewing status acquisition unit 309 of client terminal 131 determines that the virtual object cannot be viewed on client terminal 133, where userD's viewer is logged in, because the attribute "missing:true" is attached to userD's viewing status. The anchor drawing unit 304 of client terminal 131 then notifies the owner by displaying the user who cannot view the virtual object as a virtual object, similar to the method used to display viewing users in Example 1.

[0096] This embodiment includes the following information processing device configuration. (Configuration 1) A system for managing virtual objects, comprising: an acquisition means for acquiring data relating to the display of virtual objects on each terminal from one or more terminals, based on information provided by a service that manages real-world feature quantities for displaying virtual objects linked to the real world in association with identification information; and a provision means for providing information relating to the virtual objects obtained based on the acquired data. (Configuration 2) The system according to Configuration 1, characterized in that the information relating to the virtual object indicates the number of users who are viewing the virtual object. (Configuration 3) The system according to Configuration 1 or 2, characterized in that the information relating to the virtual object indicates the username of the user displaying the virtual object. (Configuration 4) The system according to any one of Configurations 1 to 3, characterized in that the providing means provides information about the virtual object to the owner of the virtual object. (Configuration 5) The system according to any one of Configurations 1 to 4, characterized in that when the terminal terminates the application that displays the virtual object, it transmits data indicating that the virtual object is hidden to the acquisition means. (Configuration 6) The system according to any one of Configurations 1 to 5, characterized in that the service transmits data to the acquisition means indicating that the virtual object is not visible on the terminal of a user using the terminal when the login expiration date of the user using the terminal has passed. (Configuration 7) The system according to any one of Configurations 1 to 6, characterized in that the data can be configured with a privacy attribute to prevent the username from being displayed at the recipient of the information regarding the virtual object. (Configuration 8) The system according to any one of Configurations 1 to 7, characterized in that the information relating to the virtual object indicates a username that is provided with information by the service but does not display the virtual object corresponding to said information. (Configuration 9) The system according to any one of Configurations 1 to 8, characterized in that the terminal includes a head-mounted display.

[0097] As described above, according to this embodiment, by comparing the anchor acquisition status with the viewing status notified from the client terminal, the owner can identify viewers who have acquired anchor information but are unable to view the virtual object.

[0098] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0099] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist.

Claims

1. A system for managing virtual objects, A browsing information detection means for detecting whether a virtual object is being viewed on each terminal, A browsing status transmission means that transmits the browsing status detected by the browsing information detection means at each terminal, An acquisition means for acquiring data relating to the display of a virtual object on each terminal, including data on the browsing state detected by the browsing information detection means, based on information provided from a service that manages real-world feature quantities for displaying virtual objects linked to the real world in association with identification information, from one or more terminals. A system characterized by having a means for providing information regarding the viewing status of the virtual object obtained based on the acquired data.

2. The system according to claim 1, characterized in that the information regarding the viewing status of the virtual object indicates the number of users who are viewing the virtual object.

3. The system according to claim 1, characterized in that the information regarding the viewing status of the virtual object indicates the name of the user who is displaying the virtual object.

4. The providing means provides the owner of the virtual object's terminal with information regarding the viewing status of the virtual object. The system according to claim 1, characterized in that the owner's terminal projects visual information based on information regarding the viewing status of the virtual object.

5. The browsing information detection means determines that the user is browsing a virtual object when the virtual object is displayed on the terminal's display unit. The browsing status transmission means transmits the browsing status data to the acquisition means when there is a change in the browsing status detected by the browsing information detection means. The system according to claim 1, characterized in that the viewing status transmission means transmits data indicating that the virtual object is hidden to the acquisition means when the application displaying the virtual object is terminated.

6. The system according to claim 1, characterized in that the service transmits data indicating that a virtual object is not visible on the user's terminal when the login expiration date of the user using the terminal has passed.

7. The system according to claim 1, characterized in that the data can be configured with a privacy attribute to prevent the username from being displayed at the recipient of information regarding the viewing status of the virtual object.

8. The system according to claim 1, characterized in that the information regarding the viewing status of the virtual object indicates a user name that is provided with information by the service but does not display the virtual object corresponding to said information.

9. The system according to any one of claims 1 to 8, characterized in that the terminal includes a head-mounted display.

10. A method for controlling a system that manages virtual objects, A step to detect the viewing status of whether a virtual object is being viewed on each terminal, The steps include: transmitting the browsing status detected on each terminal; A step of acquiring data relating to the display of the virtual object on each terminal, including detected viewing status data, based on information provided by a service that manages real-world features for displaying virtual objects linked to the real world in association with identification information, from one or more terminals. A method for controlling a system, characterized by comprising the step of providing information regarding the viewing status of the virtual object obtained based on the acquired data.

Images