System, method, and program

Abstract

To publish a content (virtual object) with appropriate control of a use method of the content (content browsing, saving, and the like) by a creator of an anchor who links the content to a real world.SOLUTION: A system further manages a setting of an authority for using a virtual object associated with identification information when managing a feature amount in a real world for displaying the virtual object linked to the real world in associated with the identification information.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technology for managing virtual objects assuming technologies such as AR (Augumented Reality). [Background technology]

[0002] "XR" is a general term for technologies such as VR (Virtual Reality) and AR (Augumented Reality) that create a space that provides a simulated experience by fusing the real world with the virtual world, and various standardization efforts are being made. In recent years, mechanisms for displaying one virtual object on multiple devices at the same location in the real world have been realized on platforms provided by various companies. For example, there is a system that links and manages a virtual object to be placed in the real world with features of the real world captured by a camera or the like. After that, by capturing a real world that matches the features managed by the system with the camera of any terminal, it becomes possible to view the virtual object managed by linking the features on that terminal.

[0003] In Patent Document 1, the display of a specific virtual object is switched using information on the user's behavior and physical environment. For example, a simple blue globe virtual object is displayed at first, but when the user approaches or looks at the object, it switches to one that depicts a detailed topography. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-118578 A Summary of the Invention [Problem to be solved by the invention]

[0005] Users who place virtual objects in the real world that can be displayed on multiple devices may want to set various permissions for the virtual objects, such as the scope of their visibility, depending on their purpose. The technique described in Patent Document 1 switches the virtual object to be displayed based on user behavior information and physical environment information, making it difficult for the user who places the virtual object to exercise control such as making it available only to intended users.

[0006] An object of the present invention is to provide a mechanism that reflects the authority to be set for a virtual object when the virtual object is linked to a feature of the real world and made public. [Means for solving the problem]

[0007] The system of the present invention is a system for managing virtual objects, and has a management means for managing features in the real world in order to link the virtual object to the real world and display it, in association with identification information, and the management means manages the identification information and further manages settings of authority for use of the virtual object associated with the identification information. Effect of the Invention

[0008] According to the present invention, when a virtual object is linked to features of the real world and made public, a mechanism can be provided in which the authority to be set for the virtual object is reflected, for example, in a manner that reflects the intention of the user who makes the object public. [Brief description of the drawings]

[0009] [Figure 1] 1 illustrates an overall configuration of a virtual object management system according to a first embodiment. [Diagram 2] FIG. 2 is a hardware configuration diagram of a server computer and a client terminal that constitute the virtual object management system in the first embodiment. [Diagram 3] FIG. 2 is a software configuration diagram of a virtual object management system according to the first embodiment. [Figure 4] 1 illustrates a process flow for sharing a virtual object among a plurality of terminals in the virtual object management system according to the first embodiment. [Diagram 5] 11 is a flowchart showing a process flow for generating an anchor in the first embodiment. [Figure 6] 11 is a flowchart showing a process flow in which the virtual object management system in the first embodiment searches for an anchor. [Figure 7] 11 is a flowchart illustrating a process flow in which the client terminal in the first embodiment renders a virtual object. [Figure 8] 13 is a flowchart showing a process flow in which the virtual object management system in the second embodiment searches for an anchor. [Figure 9] 13 is a flowchart illustrating a process flow in which a client terminal in a second embodiment renders a virtual object. [Figure 10] 13 is an example of a screen of a client terminal in the first embodiment. [Figure 11] 13 is an example of a screen of a client terminal in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

[0011] Example 1 FIG. 1 is a diagram showing the overall configuration of a virtual object management system according to an embodiment of the present invention.

[0012] 1, a virtual object management system 121, client terminals 131 to 133 are connected via networks 100 to 102. The networks 100 to 102 are so-called communication networks realized by, for example, a LAN such as the Internet, a WAN, a telephone line, a dedicated digital line, an ATM or frame relay line, a cable television line, a wireless line for data broadcasting, or the like. The networks 100 to 102 may be any network capable of transmitting and receiving data. In the present invention, the network 100 is the Internet, and the networks 101 and 102 are the Internet, a general home or office network, a wireless LAN set up in a city, or the like.

[0013] The client terminals 131-133 are, for example, dedicated hardware such as an HMD (head mounted display) or smart glasses that supports the rendering of virtual objects handled in XR, or a mobile phone with a built-in program execution environment such as a smartphone. The client terminals 131-133 also include a camera for capturing images of the surroundings and a display for displaying virtual objects. The client terminals 131-133 capture images of the surroundings through a camera or the like, and project virtual objects onto the display in a superimposed manner on the real world, thereby providing the user with a simulated experience that combines the real world and the virtual world. If the client terminals 131-133 are not dedicated hardware such as a smartphone, the virtual objects are rendered using an API provided by a web browser, an OS, or the like.

[0014] The virtual object management system 121 is a system for providing a service for linking and managing virtual objects to be placed in the real world with features of the real world captured by a camera or the like, and a service for providing the virtual objects to an external terminal. The virtual object management system 121 is constructed using a server computer. It can be constructed by adopting cloud computing technology or the like. In this embodiment, association information that links a virtual object managed by the virtual object management system 121 with a feature amount of the real world captured by a camera or the like is hereinafter referred to as an anchor. In addition to the above information, the association information includes property information including an identifier for identifying the anchor itself, a session ID, and various parameters, as described later.

[0015] The virtual object management system 121 receives anchor registration requests from the client terminals 131 to 133 and manages the registered anchors. The virtual object management system 121 also receives anchor acquisition requests from the client terminals 131 to 133 and returns anchors that meet the conditions from among the anchors it manages.

[0016] The virtual object management system 121 also manages users who use the client terminals 131 to 133. That is, the virtual object management system 121 receives login / logout requests from the client terminals 131 to 133 and performs login / logout processing.

[0017] Hereinafter, the server functions described in this invention may be realized by a single server or a single virtual server, or may be realized by multiple servers or multiple virtual servers, or multiple virtual servers may be executed on a single server.

[0018] FIG. 2 is a hardware configuration diagram of the virtual object management system 121 and the client terminals 131 to 133 according to the embodiment of the present invention.

[0019] In FIG. 2, a central processing unit (CPU) 202 controls the entire device. The CPU 202 executes application programs, an OS, etc. stored in a hard disk drive (HDD) 205, and controls temporary storage of information, files, etc. required for executing the programs in a random access memory (RAM) 203. A graphics processing unit (GPU) 210 performs calculation processing required for rendering virtual objects in real time. A read only memory (ROM) 204 is a storage means, and stores various data such as basic I / O programs inside. The RAM 203 is a temporary storage means, and functions as the main memory, work area, etc. of the CPU 202 and the GPU 210. The HDD 205 is one of the external storage means, and functions as a large-capacity memory, and stores application programs such as a web browser, programs of a group of service servers, an OS, related programs, etc. The display 206 is a display means, and displays virtual objects, information required for operations, etc. The interface 208 is an external device I / F, and connects peripheral devices such as various external sensors.

[0020] The camera 207 is an out-camera in the client terminals 131 to 133 that captures surrounding images, an in-camera that mainly captures oneself, etc. By analyzing the image captured by the camera 207, especially the out-camera, with an application program stored in the HDD 205, it becomes possible to arrange virtual objects by superimposing them on the real world, and to calculate the feature amount of the real world. In addition, if the client terminals 131 to 133 are dedicated terminals for XR such as HMDs, it becomes possible to operate virtual objects displayed on the display 206 with the user's fingers recognized by the camera 207. In the case where the client terminals 131 to 133 are not dedicated terminals for XR such as smartphones, it is possible to operate virtual objects displayed on the display 206 by operating a touch panel of the display, etc. The virtual object management system 121 does not necessarily require the camera 207.

[0021] The system bus 201 controls the flow of data within the device. The network interface card (NIC) 209 exchanges data with external devices via the interface 209 and the networks 100 to 102. The above computer configuration is an example, and is not limited to the configuration example of FIG. 2. For example, the storage destination of data and programs can be changed to the ROM 204, the RAM 203, the HDD 205, etc. depending on the characteristics of the data and programs. In addition, the CPU 202 and the GPU 210 execute processing based on the programs stored in the HDD 205, thereby realizing processing in the software configuration as shown in FIG. 3.

[0022] Next, the software configuration of the virtual object management system according to the present embodiment will be described with reference to FIGS.

[0023] FIG. 3 illustrates a software configuration in which functions related to the present invention are extracted from the virtual object management system according to the present embodiment.

[0024] The virtual object management system 121 includes an anchor management unit 311 , an anchor receiving unit 312 , an anchor providing unit 313 , a user management unit 314 , and a login processing unit 315 .

[0025] The login processing unit 315 accepts login requests from the client terminals 131 to 133, collates the information with the information in the user management unit 314, and returns the login processing result to the client terminals 131 to 133. Table A shows an example of data managed by the user management unit.

[0026] [Table 1]

[0027] The user ID column is an ID that uniquely identifies a user. The password column is a password for basic authentication used when logging in with the user ID. The login processing unit 315 checks the combination of user ID and password included in the login request from the client terminals 131 to 133 against table A, and if they match, returns the login processing result to the client terminals 131 to 133 as a success. The group list column describes group information to which a user uniquely identified by the user ID column belongs, separated by commas. For example, a user whose user ID column is userW belongs to two groups, group T and group S. Also, a user whose user ID column is userY does not belong to any group, as the group list column is empty.

[0028] When the anchor receiving unit 312 receives an anchor registration request from the client terminals 131-133, it stores the received anchor information in the anchor management unit 311. Furthermore, when the anchor providing unit 313 receives an anchor acquisition request from the client terminals 131-133, it searches the anchor management unit 311 for an anchor that meets the conditions, and returns the anchor to the client terminals 131-133.

[0029] Table B shows an example of data managed by anchor management unit 311.

[0030] [Table 2]

[0031] When the anchor receiving unit 312 receives an anchor registration request from the client terminals 131 to 133, it stores in the anchor management unit 311 a record of the corresponding anchor.

[0032] The anchor ID column is a unique ID (identification information) for identifying an anchor, which is assigned when the anchor receiving unit 312 saves a record in Table B. The same ID is added to the session ID column for the same session. By linking multiple anchors to one session ID, multiple anchors with the same session ID can be presented to the user at the same time. The virtual object data column is data of a 3D model in any format.

[0033] The features for displaying a virtual object represented by an anchor in the present invention in association with the real world are managed by three pieces of information in Table 2: (1) feature column, (2) position information column, and (3) sensor information column.

[0034] (1) The feature sequence is a three-dimensional feature of the real world obtained by analyzing data (two-dimensional image, three-dimensional image, etc.) captured by the camera 207 around the anchor. (2) The position information sequence represents the three-dimensional position of the virtual object in the real world. (3) The sensor information sequence is information about the location where the anchor is placed (GPS coordinates), the beacon or Wi-Fi ID to which the anchor is linked, etc.

[0035] In the present invention, the features indicated by the anchors are managed using three pieces of information in Table 2: (1) feature string, (2) location information string, and (3) sensor information string. However, it is also possible to manage the features of each anchor using at least one of these, or a combination of multiple of these, in combination with other information.

[0036] The anchor providing unit 313 can return one anchor linked to a specific anchor ID in response to an anchor acquisition request from the client terminals 131 to 133, and can also return multiple anchors linked to the same session ID or the same sensor ID. The Owner column is the user ID of the user who created the anchor, and one of the values ​​in the User ID column of Table A is stored. The Publication Range column is the publication range of the anchor, and is assumed to be selected by the user who created the anchor from three levels, such as Public (public to everyone), Conditional Publication (public to users belonging to group S), and Private (not public to anyone). The anchor acquisition request from the client terminals 131 to 133 includes information on the login user (such as user ID). The anchor providing unit 313 returns to the client terminals 131 to 133 only anchors whose publication range is included in the login user included in the anchor acquisition request. The Storage Permission Range column is the range of users who are permitted to locally save the data of the 3D model stored in the virtual object data column. When the 3D model of the virtual object data string can be stored locally, that is, in the virtual object data management unit 301, the user can link the 3D model of the virtual object data string to another anchor and register it elsewhere. As with the disclosure range string, the storage permission range string is assumed to be selected by the anchor generating user from three levels, such as Public (permitted to everyone), conditional permission (permitted to users belonging to group S), and Private (not permitted to anyone). In this embodiment, the anchor providing unit 313 compares whether the login user included in the anchor acquisition request from the client terminals 131 to 133 is included in the disclosure range, but this may also be performed by the anchor acquisition unit 303 (described later) of the client terminals 131 to 133.

[0037] The client terminals 131 to 133 each include a virtual object data management unit 301 , an anchor generation unit 302 , an anchor acquisition unit 303 , an anchor drawing unit 304 , a login unit 305 , and a local anchor management unit 306 .

[0038] The login unit 305 transmits a user name and password inputted by a finger photographed by the camera 207, by operating the touch panel of the display 206, or by a keyboard connected to the interface 208 to the login processing unit 315. Note that the authentication method for the virtual object management system 121 may be face authentication using a face image photographed by the camera 207, iris authentication using an iris, fingerprint authentication using a fingerprint sensor connected to the interface 208, or the like.

[0039] 3D model data in various formats is stored in the virtual object data management unit 301. The 3D data stored in the virtual object data management unit 301 is a virtual object that the user can freely arrange by overlaying it on the real world.

[0040] The anchor generation unit 302 is responsible for processing to create anchors by user operations. The user can select a 3D model stored in the virtual object data management unit 301 through the anchor generation unit 302, and place a virtual object in the real world by using a finger captured by the camera 207 or by operating the touch panel of the display 206. Figures 10(a), (b), (c), and (d) show images seen on the display 206 of the HMD-type client terminals 131 and 133, respectively.

[0041] 10(a), the user can operate the cylindrical virtual object 1002 stored in the virtual object data management unit 301 of the client terminal 131 in the above-mentioned manner and place it on a desk 1001 in the real world. Next, the anchor generation unit 302 analyzes the video of the surroundings where the virtual object is placed, captured via the camera 207, extracts features, associates them with the virtual object, and stores them in the local anchor management unit 306. The anchor generation unit 302 also identifies the position information of the anchor using a GPS sensor connected via the interface 208, and associates them with the anchor. The user also associates the anchor with the sensor via the anchor generation unit 302. Furthermore, the user sets the disclosure range and the permitted storage range via the anchor generation unit 302.

[0042] Fig. 10(b) shows how the user sets a disclosure range 1029 and a storage permission range 1022 for an anchor by operating a dialog 1021 displayed on the display 206 of the client terminal 131. In Fig. 10(b), the user selects Public 1026 (disclosed to everyone) as the disclosure range and Public 1023 (permitted to everyone) as the storage permission range. The anchor generation unit 302 transmits the anchor created as described above and stored in the local anchor management unit 306 to the anchor receiving unit 312.

[0043] The anchor acquisition unit 303 makes an anchor acquisition request to the anchor providing unit 313 based on sensor information connected to the interface 208, and stores the anchor acquired from the anchor providing unit 313 in the local anchor management unit 306. The anchor acquisition unit 303 makes an acquisition request to the anchor providing unit 313 for an anchor near the current location based on, for example, a GPS signal, or an anchor associated with a Wifi or Beacon signal when the anchor acquisition unit 303 detects the Wifi or Beacon signal.

[0044] The anchor drawing unit 304 compares the features contained in each anchor stored in the local anchor management unit 306 with the image of the real world captured by the camera 207, and places the virtual object contained in the anchor in the part where the features match.

[0045] FIG. 10C illustrates a state in which the cylindrical virtual object 1002, which the user places on a desk 1001 using the client terminal 131 as shown in FIG. 10A, is projected as a cylindrical virtual object 1032 onto a desk 1031 having the same features on the client terminal 133. If the user who operates the client terminal 133 is included in the range of anchors permitted to be saved for the cylindrical virtual object 1032, when the user selects the cylindrical virtual object 1032, the anchor drawing unit 304 draws a save dialogue 1043 (FIG. 10D). When the user selects the save dialogue, the anchor drawing unit 304 saves the cylindrical virtual object 1042 as 3D model data in the virtual object data management unit 301 of the client terminal 133. As described above, the 3D model data managed by the virtual object data management unit 301 can be freely placed in the real world. On the other hand, if the user operating the client terminal 133 is not included in the save permission range of the anchor of the cylindrical virtual object 1032, the anchor drawing unit 304 does not display the save dialog 1043 even if the user selects the cylindrical virtual object 1032.

[0046] The public range and the allowed saving range of Table B may be set so that they can be switched depending on whether or not the user is logged in, such as when not logged in or when a guest user is only able to view the table, but when logged in, they can also save the table. Also, the settings may be such that the table can be viewed only by users who follow a specific user or store on SNS, etc.

[0047] As described above, the user can freely place the 3D model data stored in the virtual object data management unit 301 over the real world, but cannot manipulate the virtual objects linked to the anchors stored in the local anchor management unit 306.

[0048] Next, a series of steps up to displaying an anchor generated by a client terminal 131 on another client terminal 133 will be described with reference to FIGS.

[0049] 4 shows a sequence from when an anchor generated by client terminal 131 is registered in virtual object management system 121 to when client terminal 133 acquires and displays the anchor stored in virtual object management system 121. The user who operates client terminal 131 is assumed to be userX in Table A, and the user who operates client terminal 133 is assumed to be userY in Table A. Also, it is assumed that the anchor that userX operates the client terminal to register is an anchor whose anchor ID in Table B is represented by a.

[0050] First, a sequence in which userX operates the client terminal 131 to register an anchor will be described from S401 to S405 in Fig. 4. UserX operates the client terminal 131 and transmits a user ID and a password to the login processing unit 315 via the login unit 305 (S401). The login processing unit 315 confirms that the user ID and the password managed by the user management unit 314 match those of userX, and if they match, returns a login result indicating successful login (S402). UserX places a virtual object stored in the virtual object data management unit 301 via the anchor generation unit 302 to generate an anchor, and stores the anchor in the local anchor management unit 306 (S403). The anchor generation unit 302 transmits an anchor registration request for the generated anchor to the anchor reception unit 312 (S404). The anchor reception unit 312 that receives the anchor registration request registers the received anchor in the anchor management unit 311, and then returns the registration result to the anchor generation unit 302. If it is desired to register multiple anchors with the same session ID, steps S403 to S405 are repeated (S411).

[0051] A detailed flow of the anchor generation process S403 performed by the anchor generation unit 302 in FIG. 4 will be described with reference to FIG. 5. The anchor generation unit 302 places a virtual object in a space in the real world according to a user's operation, and determines its position and orientation (S502). Next, the anchor generation unit 302 captures an image of the surroundings using the camera 207 in S503, and acquires three-dimensional features of the space. If the anchor generation unit 302 determines in S504 that the features have been sufficiently collected, it performs acquisition of the features in S503 again. If the anchor generation unit 302 determines in S504 that the features have been sufficiently collected, it displays a dialog as shown in FIG. 10(b) on the display 206 to the user, so that the user can set a disclosure range and a storage permission range (S505, S506). For example, as shown in FIG. 10(b), the disclosure range is set to Public (public to everyone), and the storage permission range is set to Public (permitted to everyone). Furthermore, the anchor generation unit 302 has the user set an expiration date and other necessary properties for the anchor in S507, and ends the process if sensor information is not to be set for the anchor in S508. If sensor information is to be set for the anchor in S508, the anchor generation unit 302 has the user set a setting according to the type of sensor (S509), and ends the process. For example, the anchor generation unit 302 links the anchor information to the Beacon with id=123 in accordance with the user's operation in S509.

[0052] The above is the sequence in which userX operates the client terminal 131 to register an anchor in the virtual object management system 121.

[0053] Next, a sequence in which userY operates the client terminal 133 to obtain and draw an anchor from S421 to S442 in FIG. 4 will be described.

[0054] UserY operates client terminal 133 and sends a user ID and password to login processing unit 315 via login unit 305 (S421). Login processing unit 315 checks whether the user ID and password managed by user management unit 314 match those of userY, and if they match, returns a login result indicating that the login was successful (S422).

[0055] The anchor acquisition unit 303 acquires the sensor information in S423. If the sensor information cannot be acquired, the anchor acquisition unit 303 repeats S423 (S441). The sensor information is acquired by the anchor acquisition unit 303 via a sensor that detects a Bluetooth signal and is connected to the client terminal 133 via the interface 208.

[0056] For example, when a signal from a Beacon terminal with id=123 is detected, the anchor acquisition unit 303 sends a search request for anchors linked to the Beacon terminal with id=123 to the anchor provision unit 313 together with userY, the logged-in user (S424). The anchor provision unit 313 searches the anchor management unit 311 for anchors linked to the Beacon terminal with id=123, filters out only those whose public range includes userY in S425, and returns them to the anchor acquisition unit 303 (S426). There are two anchors linked to the Beacon with id=123, anchor ID=a and anchor ID=b. Of these, anchor ID=b has a public range for users belonging to Group:S, but userY does not belong to Group:S according to Table A, so it cannot be acquired. On the other hand, anchor ID=a can be acquired because its public range is Public (open to everyone).

[0057] Next, the anchor acquisition unit 303 stores the anchors returned by the anchor providing unit 313 in the local anchor management unit 306 (S427), and further sends a search request for anchors in the same session to the anchor providing unit 313 for each anchor (S428). That is, the anchor acquisition unit 303 makes a search request to the anchor providing unit 313 for an anchor that is the same as 111, which is the session ID of anchor ID=a (S428). The anchor providing unit 313 searches for an anchor with session ID=111 from the anchor management unit 311 (S429), filters out only those whose public range includes userY, and returns them to the anchor acquisition unit 303 (S430). That is, the anchor with session ID-=111 has anchor ID=d, and its public range is also Public (public to everyone), so the anchor providing unit 313 returns this to the anchor acquisition unit 303 (S430).

[0058] Next, the anchor acquisition unit 303 stores the anchor returned by the anchor provision unit 313 in the local anchor management unit 306 (S431). Finally, the anchor drawing unit 304 repeats the anchor drawing process S432 as many times as the number of anchors stored in the local anchor management unit 306 (S442).

[0059] Here, a detailed flow of the anchor search process S425 associated with a sensor and the anchor search process S429 associated with the same session performed by the anchor providing unit 313 will be described with reference to the flowchart of FIG.

[0060] FIG. 6A is a flowchart of the anchor search process S425 performed by the anchor providing unit 313 to search for an anchor associated with a sensor.

[0061] The anchor providing unit 313 checks whether an anchor matching the sensor included in the anchor acquisition request from the anchor acquisition unit 303 still exists in table A managed by the anchor management unit 311 (S602). If no anchor exists in S602, the anchor providing unit 313 ends the process. If an anchor exists in S602, the anchor providing unit 313 acquires the corresponding anchor as a response to the request from the anchor management unit 311 (S603), performs a viewing authority check process (S604) described later in FIG. 6(c), and returns to S602.

[0062] FIG. 6B is a flowchart of anchor search processing S429 performed by the anchor providing unit 313 to search for anchors associated with the same session.

[0063] The anchor providing unit 313 checks whether an anchor matching the session ID included in the anchor acquisition request from the anchor acquisition unit 303 still exists in table A managed by the anchor management unit 311 (S612). If no anchor exists in S612, the anchor providing unit 313 ends the processing. If an anchor exists in S612, the anchor providing unit 313 acquires an anchor from the anchor management unit 311 (S613), performs a viewing authority check process (S614) described later in FIG. 6(c), and returns to S612.

[0064] FIG. 6C is a flowchart of the browsing authority check processes S604 and S614 which are performed in common in the flowcharts of FIGS. 6A and 6B.

[0065] The anchor providing unit 313 checks whether the user who made the anchor acquisition request is included in the publication range of the found anchor (S622), and if so, holds the acquired anchor in the anchor list (S623). If not included in S622, the anchor providing unit 313 ends the browsing authority check process S604 and S614. In both the anchor search process S425 linked to the sensor and the anchor search process S429 linked to the same session, the anchor providing unit 313 returns to the anchor acquiring unit 303 only the anchors held in the anchor list in S623.

[0066] FIG. 7 is a flowchart of the anchor drawing process S432 performed by the anchor drawing unit 304.

[0067] The anchor drawing unit 304 acquires the feature amount of the area in real space from the image captured by the camera 207 (S702), and checks whether it matches the feature amount of the anchor (S703). If there is no match in S703, the anchor drawing unit 304 ends the anchor drawing process S432.

[0068] If there is a match in S703, the anchor drawing unit 304 checks whether the anchor's permitted save range includes the logged-in user. If the anchor's permitted save range includes the logged-in user in S704, a save dialog is associated (S705), and if not, a save dialog is not associated (S707), and a virtual object is drawn on the display 206.

[0069] When the anchor drawing unit 304 has drawn the virtual object with a save dialog linked in S705, a save dialog 1043 for the virtual object appears when the user selects a virtual object displayed on the display 206, as shown in Fig. 10(d). On the other hand, when the anchor drawing unit 304 has drawn the virtual object without linking the save dialog in S707, the save dialog 1043 for the virtual object does not appear even if the user selects a virtual object displayed on the display 206.

[0070] One possible way that users can use virtual objects that is unique to XR is to copy a virtual object that is linked to the real world and place it again in another location (another anchor). In this case, the user who places the virtual object may want to set the storage permission range to restrict the users who can save the copy, similar to the disclosure range.

[0071] Examples of virtual objects that you may want to limit to the users who can save them and prevent from being placed in other locations include virtual objects that contain coupon information and are placed in front of a physical store as a bonus for visiting the store, or virtual objects that you simply want to prohibit from being reproduced for copyright reasons.

[0072] According to the present invention, in addition to the public range for viewing a virtual object, the permitted range for saving is also managed in association with an anchor, so that it is possible to handle such an operation example.

[0073] By using the above-described method, a registered user of an anchor can realize control such that the virtual object is made public only to intended users, and only the intended users can save the virtual object.

[0074] Example 2 In the first embodiment, when an anchor with conditions set for the public range or the allowed saving range (such as displaying or saving only to users who are in a group or who follow the anchor) cannot be displayed or saved, there may be a case where the user wants to be prompted to display or save the anchor by presenting the conditions to the user. For example, a store manager may want to build a connection with customers by displaying discount coupons only to users who follow the store, or by allowing only users in a group who handle products in a certain genre to save new products in that genre.

[0075] The method described in Example 1 allows the setting of the range of disclosure and the range of permitted saving, but does not allow the expansion of the number of users who can display and save by instructing users to do so, which is inconvenient for store managers and others who want to use the method in the manner described above.

[0076] In this embodiment, for an anchor that has conditions set for the public range or the allowed saving range, if the anchor cannot be displayed because the user does not meet the conditions, the user is notified, and if the anchor cannot be saved, a registration dialog is displayed on the display 206 instead of the save dialog (described later).

[0077] Since this embodiment has many overlapping parts with the first embodiment, only the differences from the first embodiment will be described.

[0078] Table C shows the data managed by the anchor management unit 311 in this embodiment.

[0079] [Table 3]

[0080] Except for the additional message column and URL column in Table C, the definitions of the corresponding columns are the same as those in Table B. Note that the session ID column, virtual object data column, feature column, location information column, sensor information column, and Owner column are omitted in Table C. The notification message column stores a character string that notifies the user when an anchor is not displayed, and the URL column stores a URL that performs the procedures required to display and save the anchor.

[0081] FIG. 11 shows images displayed on the displays 206 of the client terminals 131 to 133 in this embodiment.

[0082] As shown in FIG. 11(a), if anchor B cannot be displayed because the user who is trying to display it does not belong to Group:S, a cubic virtual object 1102 placed on a desk 1101 is not projected onto the display 206 and is invisible to the user.

[0083] However, in this embodiment, when a virtual object such as a cylindrical virtual object 1102 in FIG. 11(a) is not displayed but can be displayed by satisfying certain conditions, the anchor drawing unit 304 generates a notification area 1103 and notifies the user of a character string (1104) of a notification message string. When the user selects the notification area 1103, a transition to the URL stored in the URL column occurs. The notification area 1103 may be generated near the location where the cylindrical virtual object 1102 is placed, or may be generated in a standard area of ​​the client terminals 131 to 133.

[0084] The notification method may be projecting on the display 206 of the client terminals 131 to 133 as shown in Fig. 11(a), or the user may be notified by other means such as audio. Also, if the notification message column is empty, the user is not notified even if the anchor cannot be displayed. For example, anchor C, like anchor B, cannot be displayed unless the anchor belongs to Group:S. However, since the notification message string is empty, the anchor drawing unit 304 does not generate the notification area 1103 even if the user attempting to display anchor C does not belong to Group:S.

[0085] Furthermore, if anchor D cannot be saved because the user who attempts to save it does not belong to Group: T, the anchor drawing unit 304 draws the virtual object of anchor D with a registration dialog (1113). That is, when a user who does not belong to Group: T selects the virtual object of anchor D, a registration dialog 1113 is generated as shown in Fig. 11(b). When the user selects the registration dialog 1113 in Fig. 11(b), the URL stored in the URL column is displayed.

[0086] FIG. 8 is a flowchart showing the flow of a browsing authority check process in this embodiment, that is, a flowchart corresponding to the process in FIG. 6(c) in the first embodiment.

[0087] The anchor providing unit 313 checks whether the user who made the anchor acquisition request is included in the disclosure range of the found anchor (S802), and if so, holds the virtual object and metadata in the anchor list (S803) (same as in the first embodiment). If not included in S802, only the metadata is held in the anchor list (S805). The metadata here is all information other than the virtual object data string in Table C. That is, the difference from the first embodiment is that the anchor providing unit 313 returns only the metadata to the anchor acquiring unit 303 even for anchors that are not included in the disclosure range.

[0088] FIG. 9 is a flowchart of the anchor drawing process S432 performed by the anchor drawing unit 304 in this embodiment.

[0089] The anchor drawing unit 304 acquires the feature amount of the area in real space from the image captured by the camera 207 (S902), and checks whether it matches the feature amount of the anchor (S903). If there is no match in S903, the anchor drawing unit 304 ends the anchor drawing process S432. If there is a match in S903, the anchor drawing unit 304 checks whether the anchor includes information of a virtual object (S904).

[0090] If no virtual object is included in S904, the user does not have viewing authority, so the anchor drawing unit 304 checks whether it is necessary to notify the user that the user does not have viewing authority (S910). That is, in S910, the anchor drawing unit 304 checks whether a value is set in the notification message column of Table C for the anchor, and if a value is set, in S905, generates a notification area 1103 like the one shown in Figure 11(a) described above, and then ends the processing.

[0091] If no value is set in the notification message column of Table C for the anchor in S910, the anchor drawing unit 304 ends the process. If a virtual object is included in S904, the user has viewing authority, so the anchor drawing unit 304 checks whether the anchor's save permission range includes the logged-in user (S906). If the anchor's save permission range includes the logged-in user in S906, a save dialog is associated with the virtual object and the virtual object is drawn on the display 206 (S907).

[0092] If the anchor saving permission range does not include the login user in S906, the anchor drawing unit 304 checks in S911 whether it is necessary to notify the user that the user does not have saving authority. That is, in S911, the anchor drawing unit 304 checks whether a value is set in the URL column of Table C for the anchor, and if a value is set, links a registration dialog 1113 like that shown in FIG. 11(b) and draws a virtual object on the display 206 (S908).

[0093] If no value is set in the URL column of Table C for the anchor in S911, the anchor drawing unit 304 draws the virtual object on the display 206 without linking it to either the save dialog or the registration dialog (S912). If the anchor drawing unit 304 draws the virtual object with the save dialog linked in S907, when the user selects the virtual object displayed on the display 206, a save dialog 1043 of the virtual object appears as shown in FIG. 10(d). If the anchor drawing unit 304 draws the virtual object with the registration dialog linked in S908, when the user selects the virtual object displayed on the display 206, a registration dialog 1113 as shown in FIG. 11(b) appears. If the anchor drawing unit 304 draws the virtual object without linking it to either the save dialog or the registration dialog in S912, when the user selects the virtual object displayed on the display 206, no dialog appears.

[0094] In this embodiment, the timing of the notification to the user is the timing when the anchor feature amount in S903 matches the feature amount in the real world. However, even if the feature amounts do not match, the notification may be the timing when the user looks in the direction in which the cylindrical virtual object 1102 is to be placed, or when the user approaches the place where the object is to be placed.

[0095] In this embodiment, the conditions set in the disclosure range column and storage permission range column of Table C are assumed to be whether or not the user belongs to a group in the group list column of Table A in the first embodiment. However, for example, by linking with an existing SNS, it is possible to configure settings such as whether or not the user is registered with an existing SNS, whether or not the user is participating in a specific group, and whether or not the user is following a specific user. In this case, the URL column of Table C contains the URL of the registration page, ID linking page, participation page for a specific group, and page of a specific user of the existing SNS. Note that the method of linking with an existing SNS will be omitted.

[0096] By guiding users to view and save using this embodiment, it is possible to narrow down the users by setting the range of disclosure and the range of allowed saving, while gradually allowing more users to view and save.

[0097] (Other Examples) The present invention also includes an apparatus or system configured by appropriately combining the above-described embodiments, and a method thereof.

[0098] Here, the present invention is a device or system that executes one or more pieces of software (programs) that realize the functions of the above-mentioned embodiments. Also, a method for realizing the above-mentioned embodiments executed by the device or system is also one aspect of the present invention. Also, the program is supplied to the system or device via a network or various storage media, and the program is read into one or more memories by one or more computers (CPU, MPU, etc.) of the system or device and executed. In other words, as one aspect of the present invention, the program itself, or various storage media that store the program and can be read by a computer, are also included. Also, the present invention can be realized by a circuit (e.g., ASIC) that realizes the functions of the above-mentioned embodiments. [Explanation of symbols]

[0099] 121 Virtual Object Management System 131 Client terminal (head mounted display) 132 Client terminal (smartphone)

Claims

1. A system for managing virtual objects, comprising: a management means for managing a feature amount in the real world by associating the feature amount with identification information of an anchor for linking the virtual object to the real world and displaying the virtual object; The system is characterized in that the management means further manages, in association with the identification information, information indicating the user who created the anchor and user information indicating a target of disclosure.

2. The system described in Claim 1, characterized in that the management means further includes settings for determining the range of users who can save copies of the virtual object.

3. The system described in Claim 2, characterized in that the range of users who can save copies of the virtual object can be determined using group information.

4. The system includes a terminal capable of projecting virtual objects into a real world; responding to a request from the terminal using at least one of identification information and a feature amount in the real world by the management means with a virtual object managed by the identification information and at least one of the feature amount; 4. The system according to claim 1, wherein in the terminal, the responded virtual object is projected into the real world.

5. The system described in Claim 4, characterized in that the terminal includes a head-mounted display.

6. The system includes a management system and a client, 6. The system according to claim 1, wherein the identification information is assigned in the management system in response to receiving a request from the client.

7. A method in a system for managing virtual objects, comprising: a management step of managing a feature amount in the real world by associating the virtual object with identification information of an anchor for linking the virtual object to the real world and displaying the anchor; A method according to claim 1, wherein in the management step, information indicating the user who created the anchor and user information indicating a target of disclosure are further managed in association with the identification information.

8. For managing virtual objects, A program for causing a computer to execute a management step of managing a feature amount in the real world by associating a virtual object with identification information of an anchor for linking the virtual object to the real world and displaying the anchor, The management step further manages, in association with the identification information, information indicating a user who created the anchor and user information indicating a target of disclosure.

Images