Systems, methods, and programs

Abstract

To publish a content (virtual object) with appropriate control of a use method of the content (content browsing, saving, and the like) by a creator of an anchor who links the content to a real world.SOLUTION: A system further manages a setting of an authority for using a virtual object associated with identification information when managing a feature amount in a real world for displaying the virtual object linked to the real world in associated with the identification information.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a technology for managing virtual objects assuming technologies such as AR (Augumented Reality).

Background Art

[0002] "XR" has attracted attention as a general term for technologies that create a space for providing a pseudo-experience by integrating the real and virtual worlds such as VR (Virtual Reality) and AR (Augumented Reality), and various standardization efforts are underway. In recent years, a mechanism for displaying a single virtual object at the same location in the real world on multiple terminals has been realized on the platforms provided by each company. For example, there is a system that manages by linking a virtual object to be placed in the real world and the feature amount of the real world captured by a camera or the like. Thereafter, by capturing the real world that matches the feature amount managed by the system with the camera of an arbitrary terminal, it becomes possible to view the virtual object managed in association with the feature amount on that arbitrary terminal.

[0003] Patent Document 1 switches the display of a specific virtual object using the behavior information and physical environment information of the user. For example, initially, a virtual object of a globe that is simply a blue sphere is displayed, but when approaching or staring at it, it switches to one that expresses detailed terrain.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Users who place virtual objects that can be displayed on multiple devices into the real world may want to set various permissions for those virtual objects, such as their visibility settings, depending on their intended use. The method described in Patent Document 1 switches the virtual objects displayed based on user behavior information and physical environment information, making it difficult to control whether the virtual objects are made available to users intended by the user who sets them up.

[0006] The present invention aims to provide a mechanism that reflects the permissions that should be set on a virtual object when it is made public in association with the characteristics of the real world. [Means for solving the problem]

[0007] The system in the present invention is a system for managing virtual objects, and has a management means for managing feature quantities in the real world in association with identification information of anchors for displaying virtual objects linked to the real world, and the management means further associates with the identification information information indicating the user who created the anchor The report, and settings for determining the range of users who can save copies of the virtual object. It is characterized by its ability to manage. [Effects of the Invention]

[0008] According to the present invention, when a virtual object is published in association with characteristics of the real world, a mechanism is provided that reflects the permissions that should be set on that virtual object, for example, the intention of the user publishing it. [Brief explanation of the drawing]

[0009] [Figure 1] The overall configuration of the virtual object management system in Example 1 is shown. [Figure 2] This is a hardware configuration diagram of the server computer and client terminals that constitute the virtual object management system in Example 1. [Figure 3] This is a software configuration diagram of the virtual object management system in Example 1. [Figure 4] This illustrates the process flow for sharing virtual objects across multiple terminals in the virtual object management system in Example 1. [Figure 5] This is a flowchart illustrating the process for generating anchors in Example 1. [Figure 6] This flowchart illustrates the process by which the virtual object management system in Example 1 searches for an anchor. [Figure 7] This flowchart illustrates the process flow in which the client terminal renders a virtual object in Example 1. [Figure 8] This flowchart illustrates the process by which the virtual object management system in Example 2 searches for an anchor. [Figure 9] This flowchart illustrates the process flow in which the client terminal renders a virtual object in Example 2. [Figure 10] This is an example of the client terminal screen in Example 1. [Figure 11] This is an example of the client terminal screen in Example 2. [Modes for carrying out the invention]

[0010] The best mode for carrying out the present invention will be described below with reference to the drawings.

[0011] (Example 1) Figure 1 is a diagram showing the overall configuration of a virtual object management system according to an embodiment of the present invention.

[0012] In Figure 1, the virtual object management system 121, client terminals 131-133, and networks 100-102 are connected. Networks 100-102 are so-called communication networks, such as LANs, WANs, telephone lines, dedicated digital lines, ATM or frame relay lines, cable television lines, and wireless lines for data broadcasting. Networks 100-102 only need to be capable of transmitting and receiving data. In this invention, network 100 is the internet, and networks 101 and 102 are the internet, networks within ordinary homes or companies, and wireless LANs set up in urban areas.

[0013] Client terminals 131-133 include, for example, dedicated hardware that supports the rendering of virtual objects handled in XR, such as HMDs (head-mounted displays) and smart glasses, or mobile phones with a built-in program execution environment, such as smartphones. Client terminals 131-133 also have cameras for capturing the surroundings and displays for displaying virtual objects. By capturing the surroundings through the cameras and projecting virtual objects onto the display overlaid on the real world, client terminals 131-133 provide users with a simulated experience that blends the real and virtual worlds. If client terminals 131-133 are not dedicated hardware, such as smartphones, they utilize APIs provided by web browsers or the operating system to render virtual objects.

[0014] The virtual object management system 121 is a system that provides services such as linking virtual objects to be placed in the real world with feature quantities of the real world captured by cameras, etc., and providing said virtual objects to external terminals. The virtual object management system 121 is constructed using a server computer. It can be constructed by adopting cloud computing technology and the like. In this embodiment, hereinafter, the association information that associates a virtual object managed by the virtual object management system 121 with a feature amount of the real world captured by a camera or the like is referred to as an anchor. In addition to those pieces of information, the association information includes property information including an identifier for identifying the anchor itself, a session ID, and various parameters, as will be described later.

[0015] The virtual object management system 121 receives anchor registration requests from the client terminals 131 to 133 and manages the registered anchors. Further, the virtual object management system 121 receives anchor acquisition requests from the client terminals 131 to 133 and returns an anchor that matches the conditions from the managed anchors.

[0016] The virtual object management system 121 also manages users who use the client terminals 131 to 133. That is, the virtual object management system 121 receives login / logout requests from the client terminals 131 to 133 and performs login / logout processing.

[0017] Hereinafter, the functions of the server described in the present invention may be realized by a single server or a single virtual server, or may be realized by a plurality of servers or a plurality of virtual servers. Alternatively, a plurality of virtual servers may be executed on a single server.

[0018] FIG. 2 is a hardware configuration diagram of the virtual object management system 121 and the client terminals 131 to 133 according to an embodiment of the present invention.

[0019] In Figure 2, the Central Processing Unit (CPU) 202 controls the entire system. The CPU 202 executes application programs, the OS, etc., stored in the Hard Disk Drive (HDD) 205, and controls the temporary storage of information and files necessary for program execution in the Random Access Memory (RAM) 203. The Graphics Processing Unit (GPU) 210 performs the calculations necessary to render virtual objects in real time. The Read Only Memory (ROM) 204 is a storage means and stores various data such as basic I / O programs. The RAM 203 is a temporary storage means and functions as the main memory and work area for the CPU 202 and GPU 210. The HDD 205 is one of the external storage means and functions as a large-capacity memory, storing application programs such as web browsers, service server programs, the OS, and related programs. The display 206 is a display means that displays virtual objects and information necessary for operation. The interface 208 is an external device I / F and connects peripheral devices such as various external sensors.

[0020] Camera 207 includes rear cameras on client terminals 131-133 that capture images of the surroundings, and front cameras that primarily capture images of the user. By analyzing the images captured by camera 207, especially the rear cameras, with an application program stored in HDD 205, it becomes possible to place virtual objects overlaid on the real world and calculate features of the real world. Furthermore, if client terminals 131-133 are dedicated XR terminals such as HMDs, it becomes possible to manipulate the virtual objects displayed on display 206 using the user's fingers recognized by camera 207. If client terminals 131-133 are not dedicated XR terminals such as smartphones, it is possible to manipulate the virtual objects displayed on display 206 by operating the touch panel of the display. The virtual object management system 121 does not necessarily require camera 207.

[0021] The system bus 201 controls the flow of data within the device. The Network Interface Card (NIC) 209 exchanges data with external devices via the interface 209 and networks 100-102. Note that the above computer configuration is just one example and is not limited to the configuration example in Figure 2. For example, the storage location of data and programs can be changed to ROM 204, RAM 203, HDD 205, etc., depending on their characteristics. In addition, the CPU 202 and GPU 210 execute processing based on the programs stored in HDD 205, thereby realizing the processing in the software configuration shown in Figure 3.

[0022] Next, the software configuration of the virtual object management system according to this embodiment will be explained using Figures 3 and 10.

[0023] Figure 3 illustrates a software configuration in which the functions related to the present invention are extracted from the virtual object management system according to this embodiment.

[0024] The virtual object management system 121 consists of an anchor management unit 311, an anchor receiving unit 312, an anchor providing unit 313, a user management unit 314, and a login processing unit 315.

[0025] The login processing unit 315 receives login requests from client terminals 131-133, compares them with the information in the user management unit 314, and returns the login processing result to the client terminals 131-133. Table A shows an example of data managed by the user management unit.

[0026] [Table 1]

[0027] The User ID column is an ID that uniquely identifies a user. The Password column is the password used for basic authentication when logging in with the User ID. The Login Processing Unit 315 compares the User ID and password combination included in the login request from client terminals 131-133 with Table A, and if they match, returns the login result as successful to client terminals 131-133. The Groups List column lists the group information to which the user uniquely identified by the User ID column belongs, separated by commas. For example, a user with User ID W belongs to two groups, Group T and Group S. Also, a user with User ID Y does not belong to any group, as the Groups List column is empty.

[0028] When the anchor receiving unit 312 receives an anchor registration request from client terminals 131 to 133, it stores the received anchor information in the anchor management unit 311. Similarly, when the anchor providing unit 313 receives an anchor acquisition request from client terminals 131 to 133, it searches the anchor management unit 311 for an anchor that matches the specified criteria and returns it to the client terminals 131 to 133.

[0029] Table B shows an example of data managed by the anchor management unit 311.

[0030] [Table 2]

[0031] When the anchor receiving unit 312 receives an anchor registration request from client terminals 131 to 133, it saves the corresponding anchor record to the anchor management unit 311.

[0032] The Anchor ID column is a unique ID (identification information) assigned by the anchor receiver 312 when saving a record to Table B, used to identify the anchor. The Session ID column assigns the same ID to anchors within the same session. By associating multiple anchors with a single Session ID, multiple anchors with the same Session ID can be presented to the user simultaneously. The Virtual Object Data column contains data of a 3D model in any format.

[0033] The features used to link and display virtual objects indicated by anchors in the present invention to the real world are managed by three pieces of information in Table 2: (1) the feature column, (2) the location information column, and (3) the sensor information column.

[0034] (1) The feature column consists of three-dimensional features of the real world obtained by analyzing data (2D images, 3D images, etc.) taken by camera 207 around where the anchor is placed. (2) The location information column represents the three-dimensional position of the virtual object in the real world. (3) The sensor information column consists of location information where the anchor is placed (GPS coordinates), the ID of the Beacon or Wi-Fi associated with the anchor, etc.

[0035] In this invention, the features indicated by anchors are managed using three pieces of information: (1) the feature column, (2) the location information column, and (3) the sensor information column, as shown in Table 2. However, it is also possible to manage the features of each anchor using at least one or more of these, or a combination of them, and other pieces of information.

[0036] The anchor provider 313 can return one anchor associated with a specific anchor ID in response to an anchor acquisition request from client terminals 131-133, and can also return multiple anchors associated with the same session ID or the same sensor ID. The Owner column is the user ID of the user who created the anchor, and stores one of the values ​​in the User ID column of Table A. The Public Scope column is the public scope of the anchor, and it is assumed that the user who created the anchor will select from three levels, such as Public (public to everyone), Conditional Public (public to users belonging to group S), and Private (not public to anyone). The anchor acquisition request from client terminals 131-133 includes information of the logged-in user (such as the user ID). The anchor provider 313 returns only the anchors whose public scope includes the logged-in user included in the anchor acquisition request to client terminals 131-133. The Save Permission Scope column is the range of users who are permitted to save the 3D model data stored in the Virtual Object Data column locally. If the 3D model of the virtual object data column can be saved locally, that is, saved in the virtual object data management unit 301, the user can link the 3D model of the virtual object data column to another anchor and register it elsewhere. The save permission range column, like the public access range column, is expected to be selected by the anchor generating user from three levels, such as Public (allowed to everyone), Conditional access (allowed to users belonging to group S), and Private (not allowed to anyone). In this embodiment, the anchor provision unit 313 compares whether the logged-in users included in the anchor acquisition requests from client terminals 131 to 133 are included in the public access range, but this may also be done by the anchor acquisition unit 303 (described later) of client terminals 131 to 133.

[0037] Client terminals 131-133 consist of a virtual object data management unit 301, an anchor generation unit 302, an anchor acquisition unit 303, an anchor drawing unit 304, a login unit 305, and a local anchor management unit 306.

[0038] The login unit 305 transmits the username and password entered via a finger captured by the camera 207, operation of the touch panel on the display 206, or a keyboard connected to the interface 208 to the login processing unit 315. The authentication method for the virtual object management system 121 may also be facial recognition using a facial image captured by the camera 207, iris recognition using the iris, or fingerprint authentication using a fingerprint sensor connected to the interface 208.

[0039] The virtual object data management unit 301 stores 3D model data in various formats. The 3D data stored in the virtual object data management unit 301 are virtual objects that users can freely place on top of the real world.

[0040] The anchor generation unit 302 is responsible for creating anchors based on user operations. The user can select a 3D model stored in the virtual object data management unit 301 via the anchor generation unit 302 and place the virtual object in the real world using their fingers captured by the camera 207 or by operating the touch panel on the display 206. Figures 10(a)(b) and (c)(d) show the images that appear on the displays 206 of the HMD-type client terminals 131 and 133, respectively.

[0041] As shown in Figure 10(a), the user can manipulate the cylindrical virtual object 1002 stored in the virtual object data management unit 301 of the client terminal 131 in the manner described above and place it on the real-world desk 1001. Subsequently, the anchor generation unit 302 analyzes the video of the surrounding area where the virtual object is placed, captured via the camera 207, extracts feature quantities, links them to the virtual object, and stores them in the local anchor management unit 306. The anchor generation unit 302 also uses the GPS sensor connected via the interface 208 to identify the anchor's location information and links it to the anchor. The user also links the anchor to the sensor through the anchor generation unit 302. Furthermore, the user sets the public access range and the storage permission range through the anchor generation unit 302.

[0042] Figure 10(b) shows how a user sets the public access range 1029 and the save permission range 1022 for an anchor by operating a dialog box 1021 displayed on the display 206 of the client terminal 131. In Figure 10(b), the user has selected Public 1026 (public to everyone) as the public access range and Public 1023 (allowed to everyone) as the save permission range. The anchor generation unit 302 sends the anchor created as described above and stored in the local anchor management unit 306 to the anchor receiving unit 312.

[0043] The anchor acquisition unit 303 requests anchor acquisition from the anchor provision unit 313 based on sensor information connected to the interface 208, and stores the acquired anchors in the local anchor management unit 306. For example, if the anchor acquisition unit 303 detects an anchor near the current location based on a GPS signal, or a Wi-Fi or Beacon signal, it requests the anchor provision unit 313 to acquire the anchor associated with that Wi-Fi or Beacon.

[0044] The anchor drawing unit 304 compares the feature quantities contained in each anchor stored in the local anchor management unit 306 with the real-world images captured by the camera 207, and places the virtual objects contained in the anchor in the areas where the features match.

[0045] Figure 10(c) shows how a cylindrical virtual object 1002, placed on a desk 1001 by a user on client terminal 131 as shown in Figure 10(a), is projected as a cylindrical virtual object 1032 onto a desk 1031 with the same feature quantities on client terminal 133. If the user operating client terminal 133 is within the range of permitted anchor saving for the cylindrical virtual object 1032, and the user selects the cylindrical virtual object 1032, the anchor drawing unit 304 draws a save dialog 1043 (Figure 10(d)). When the user selects the save dialog, the anchor drawing unit 304 saves the cylindrical virtual object 1042 as 3D model data to the virtual object data management unit 301 of client terminal 133. As described above, the 3D model data managed by the virtual object data management unit 301 can be freely placed in the real world. On the other hand, if the user operating the client terminal 133 is not included in the range of permitted anchors for the cylindrical virtual object 1032, the anchor drawing unit 304 will not display the save dialog 1043 even if the user selects the cylindrical virtual object 1032.

[0046] The visibility and save permission settings for Table B could be configured to switch based on login status; for example, users who are not logged in or are guests could only view the table, but logged-in users could save it. Alternatively, it could be configured so that only users who follow specific users or stores on social media can view it.

[0047] As mentioned above, the 3D model data stored in the virtual object data management unit 301 can be freely placed by the user overlaid on the real world, but the virtual objects linked to the anchors held in the local anchor management unit 306 cannot be manipulated.

[0048] Next, we will explain the sequence of events from generating an anchor on client terminal 131 to displaying it on another client terminal 133, using Figures 4, 5, 6, and 7.

[0049] Figure 4 illustrates the sequence from the registration of an anchor generated by client terminal 131 to the virtual object management system 121, to the retrieval and display of the anchor stored in the virtual object management system 121 by client terminal 133. The user operating client terminal 131 is referred to as userX in Table A, and the user operating client terminal 133 is referred to as userY in Table A. Furthermore, it is assumed that the anchor registered by userX when operating the client terminal is the anchor represented by anchor ID a in Table B.

[0050] First, the sequence in Figure 4 from S401 to S405 in which userX operates the client terminal 131 to register an anchor will be explained. userX operates the client terminal 131 and sends the user ID and password to the login processing unit 315 via the login unit 305 (S401). The login processing unit 315 checks that the user ID and password managed by the user management unit 314 match those of userX, and if they match, it returns a login result indicating successful login (S402). userX generates an anchor by placing a virtual object stored in the virtual object data management unit 301 via the anchor generation unit 302 and saves it to the local anchor management unit 306 (S403). The anchor generation unit 302 sends an anchor registration request for the generated anchor to the anchor receiving unit 312 (S404). The anchor receiving unit 312, having received the anchor registration request, registers the received anchor with the anchor management unit 311 and then returns the registration result to the anchor generation unit 302. If you want to register multiple anchors with the same session ID, repeat steps S403-S405 (S411).

[0051] Using Figure 5, the detailed flow of the anchor generation process S403 performed by the anchor generation unit 302 in Figure 4 will be explained. The anchor generation unit 302 places a virtual object in the real world space based on user operation and determines its position and orientation (S502). Next, in S503, the anchor generation unit 302 takes a picture of the surroundings through the camera 207 and acquires three-dimensional features of the space. If the anchor generation unit 302 determines in S504 that it has not collected enough features, it performs feature acquisition again in S503. If the anchor generation unit 302 determines in S504 that it has collected enough features, it displays a dialog box like the one in Figure 10(b) to the user via the display 206, allowing them to set the scope of public access and the scope of save permission (S505, S506). For example, as shown in Figure 10(b), the scope of public access can be set to Public (public to everyone) and the scope of save permission to Public (allowed to everyone). Furthermore, in S507, the anchor generation unit 302 prompts the user to set an expiration date and other necessary properties for the anchor, and in S508, if no sensor information is set for the anchor, it terminates the process. If sensor information is set for the anchor in S508, the anchor generation unit 302 prompts the user to set the appropriate settings according to the type of sensor (S509), and then terminates the process. For example, in S509, the anchor generation unit 302 associates the anchor information with the Beacon with id=123 according to the user's operation.

[0052] The above is the sequence in which userX operates the client terminal 131 to register the anchor with the virtual object management system 121.

[0053] Next, we will explain the sequence in S421 to S442 in Figure 4 in which userY operates the client terminal 133 to acquire and draw the anchor.

[0054] User Y operates the client terminal 133 and sends the user ID and password to the login processing unit 315 via the login unit 305 (S421). The login processing unit 315 checks if the user ID and password managed by the user management unit 314 match those of user Y, and if they match, it returns a login result indicating successful login (S422).

[0055] The anchor acquisition unit 303 acquires sensor information in S423. If sensor information cannot be acquired, the anchor acquisition unit 303 repeats S423 (S441). Sensor information is acquired by the anchor acquisition unit 303 via a sensor that detects Bluetooth signals connected to the client terminal 133 via interface 208.

[0056] For example, if a signal is detected from the Beacon terminal with id=123, the anchor acquisition unit 303 sends a search request for the anchor associated with the Beacon terminal with id=123 to the anchor provision unit 313, along with the logged-in user, userY (S424). The anchor provision unit 313 searches the anchors associated with the Beacon terminal with id=123 from the anchor management unit 311, filters them in S425 to include only those whose public access range includes userY, and returns them to the anchor acquisition unit 303 (S426). There are two anchors associated with the Beacon with id=123: anchor ID=a and anchor ID=b. Of these, anchor ID=b is associated with a user whose public access range belongs to Group:S, but userY does not belong to Group:S according to Table A, so it cannot be acquired. On the other hand, anchor ID=a can be acquired because its public access range is Public (public to everyone).

[0057] Next, the anchor acquisition unit 303 stores the anchors returned by the anchor provision unit 313 in the local anchor management unit 306 (S427), and then sends a search request for anchors in the same session to the anchor provision unit 313 for each anchor (S428). In other words, the anchor acquisition unit 303 makes a search request to the anchor provision unit 313 for the same anchor as session ID 111, which is anchor ID=a (S428). The anchor provision unit 313 searches the anchor management unit 311 for the anchor with session ID=111 (S429), filters it to include only those whose public access scope includes userY, and returns them to the anchor acquisition unit 303 (S430). In other words, the anchor with session ID=111 has anchor ID=d and its public access scope is Public (public to everyone), so the anchor provision unit 313 returns this to the anchor acquisition unit 303 (S430).

[0058] Next, the anchor acquisition unit 303 saves the anchors returned by the anchor provision unit 313 to the local anchor management unit 306 (S431). Finally, the anchor drawing unit 304 repeats the anchor drawing process S432 for the number of anchors saved in the local anchor management unit 306 (S442).

[0059] Here, the detailed flow of the anchor search process S425 linked to the sensor and the anchor search process S429 linked to the same session, performed by the anchor provisioning unit 313, will be explained using the flowchart in Figure 6.

[0060] Figure 6(a) is a flowchart of the anchor search process S425, which is performed by the anchor supply unit 313 and is linked to the sensor.

[0061] The anchor provisioning unit 313 checks whether an anchor matching the sensor included in the anchor acquisition request from the anchor acquisition unit 303 still exists in Table A managed by the anchor management unit 311 (S602). If no anchor exists in S602, the anchor provisioning unit 313 terminates processing. If an anchor exists in S602, the anchor provisioning unit 313 acquires the corresponding anchor from the anchor management unit 311 as a response to the request (S603), performs the access permission check process described later in Figure 6(c) (S604), and returns to S602.

[0062] Figure 6(b) is a flowchart of the anchor search process S429 performed by the anchor provisioning unit 313, which is linked to the same session.

[0063] The anchor provider unit 313 checks whether an anchor matching the session ID included in the anchor acquisition request from the anchor acquisition unit 303 still exists in Table A managed by the anchor management unit 311 (S612). If the anchor does not exist in S612, the anchor provider unit 313 terminates processing. If the anchor exists in S612, the anchor provider unit 313 acquires the anchor from the anchor management unit 311 (S613), performs the access permission check process described later in Figure 6(c) (S614), and returns to S612.

[0064] Figure 6(c) is a flowchart of the access permission check processes S604 and S614, which are common to the flowcharts in Figures 6(a) and (b).

[0065] The anchor provider unit 313 checks whether the user who requested the anchor acquisition is included in the public access scope of the found anchor (S622), and if so, it stores the acquired anchor in the anchor list (S623). If it is not included in S622, the anchor provider unit 313 terminates the access permission check process S604 and S614. In both the anchor search process S425 associated with the sensor and the anchor search process S429 associated with the same session, the only anchors that the anchor provider unit 313 returns to the anchor acquisition unit 303 are those stored in the anchor list in S623.

[0066] Figure 7 is a flowchart of the anchor drawing process S432 performed by the anchor drawing unit 304.

[0067] The anchor drawing unit 304 acquires feature quantities of real-world regions from the video captured by the camera 207 (S702) and checks whether they match the anchor feature quantities (S703). If they do not match in S703, the anchor drawing unit 304 terminates the anchor drawing process S432.

[0068] If a match is found in S703, the anchor drawing unit 304 checks whether the anchor's save permission range includes the logged-in user. In S704, if the anchor's save permission range includes the logged-in user, a save dialog is associated (S705); otherwise, the save dialog is not associated (S707), and the virtual object is drawn on the display 206.

[0069] If the anchor drawing unit 304 draws with a save dialog linked in S705, when the user selects a virtual object displayed on the display 206, the save dialog 1043 for the virtual object will appear as shown in Figure 10(d). On the other hand, if the anchor drawing unit 304 draws without linking a save dialog in S707, the save dialog 1043 for the virtual object will not appear even if the user selects a virtual object displayed on the display 206.

[0070] Furthermore, one anticipated use case for virtual objects unique to XR is the ability to copy virtual objects linked to the real world and then place them again in a different location (a different anchor). In this scenario, users who place virtual objects would likely want to set save permission settings, similar to the sharing settings, to restrict who can save them as copies.

[0071] Examples of virtual objects that you want to restrict which users can save and prevent from being placed in other locations include virtual objects containing coupon information placed in front of a physical store as a perk for visiting the store, and virtual objects that you simply want to prohibit from being reproduced for copyright reasons.

[0072] According to the present invention, in addition to the public access range for viewing virtual objects, the permitted access range for saving is also managed in conjunction with the anchor, making it possible to accommodate such operational examples.

[0073] Through the methods described above, registered anchor users can implement control so that virtual objects are exposed only to intended users, and only intended users can save virtual objects.

[0074] (Example 2) In Example 1, there are cases where it would be desirable to control the display and saving of anchors that have conditions set for their visibility and save permission (such as only displaying and saving them to users in a group or users who follow the store) by presenting the conditions to the user if they are not displayed or saved. For example, this could be done when a store owner wants to maintain a connection with customers by displaying discount coupons only to users who follow the store, or by allowing only users in a group that handles a certain type of product to save new products in that category.

[0075] In the method described in Example 1, while it is possible to set the scope of publication and the scope of saving permission, it is not possible to expand the number of users who can view and save by guiding users to do so. Therefore, it is inconvenient for store owners and others who want to use the content in the manner described above.

[0076] In this embodiment, for anchors with conditions set for publication range and save permission range, if the user cannot view them because the conditions are not met, the user is notified, and if saving is not possible, a registration dialog is displayed on display 206 instead of a save dialog (described later).

[0077] Since this embodiment overlaps considerably with Embodiment 1, only the differences from Embodiment 1 will be explained.

[0078] Table C shows the data managed by the anchor management unit 311 in this embodiment.

[0079] [Table 3]

[0080] Except for the additional message and URL columns in Table C, the definitions of the columns are the same as those of the corresponding columns in Table B. Note that the session ID, virtual object data, feature, location information, sensor information, and Owner columns are omitted in Table C. The notification message column stores a string that notifies the user if the anchor is not displayed, and the URL column stores the URL that performs the procedure necessary to display and save the anchor.

[0081] Figure 11 shows the images displayed on the displays 206 of client terminals 131 to 133 in this embodiment.

[0082] As shown in Figure 11(a), if the user attempting to view anchor B is unable to do so because they do not belong to Group:S, the cube virtual object 1102 placed on the desk 1101 will not be projected onto the display 206 and will not be visible to the user.

[0083] However, in this embodiment, if an object is not displayed but can be displayed by fulfilling certain conditions, such as the cylindrical virtual object 1102 in Figure 11(a), the anchor drawing unit 304 generates a notification area 1103 and notifies the user of the string (1104) of the notification message sequence. When the user selects notification area 1103, the system will navigate to the URL stored in the URL column. The notification area 1103 may be generated near where the cylindrical virtual object 1102 is placed, or it may be generated in the standard area of ​​client terminals 131-133.

[0084] The notification method may be projected onto the displays 206 of client terminals 131-133 as shown in Figure 11(a), or the user may be notified by other means such as audio. Also, if the notification message column is empty, the user will not be notified even if the anchor cannot be displayed. For example, anchor C, like anchor B, cannot be displayed unless it belongs to Group:S, but because the notification message column is empty, the anchor drawing unit 304 does not generate the notification area 1103 even if the user who tried to display anchor C does not belong to Group:S.

[0085] Furthermore, if the user attempting to save anchor D is unable to do so because they do not belong to Group:T, the anchor drawing unit 304 draws the virtual object of anchor D with a registration dialog (1113) enabled. That is, if a user who does not belong to Group:T selects the virtual object of anchor D, a registration dialog 1113 is generated as shown in Figure 11(b). In Figure 11(b), if the user selects the registration dialog 1113, the system transitions to the URL stored in the URL column.

[0086] Figure 8 is a flowchart showing the flow of the access permission check process in this embodiment, that is, a flowchart corresponding to the process in Figure 6(c) in Embodiment 1.

[0087] The anchor provider 313 checks whether the user who requested the anchor acquisition is included in the public access scope of the found anchor (S802), and if so, it stores the virtual object + metadata in the anchor list (S803) (same as in Example 1). If it is not included in S802, only the metadata is stored in the anchor list (S805). Here, metadata refers to all information other than the virtual object data column in Table C. In other words, the difference from Example 1 is that the anchor provider 313 returns only the metadata to the anchor acquisition 303 even for anchors that are not included in the public access scope.

[0088] Figure 9 is a flowchart of the anchor drawing process S432 performed by the anchor drawing unit 304 in this embodiment.

[0089] The anchor drawing unit 304 acquires feature quantities of real-world regions from the video captured by the camera 207 (S902) and checks whether they match the anchor feature quantities (S903). If they do not match in S903, the anchor drawing unit 304 terminates the anchor drawing process S432. If they do match in S903, the anchor drawing unit 304 checks whether the anchor contains information about a virtual object (S904).

[0090] If no virtual object is included in S904, the anchor drawing unit 304 checks whether it needs to notify the user that it does not have viewing permission, as there is no viewing permission (S910). In other words, the anchor drawing unit 304 checks in S910 whether a value is set in the notification message column of Table C for the anchor, and if it is set, it generates a notification area 1103 as shown in Figure 11(a) above in S905 and terminates the process.

[0091] If no value is set in the notification message column of table C for the anchor in S910, the anchor drawing unit 304 terminates processing. If a virtual object is included in S904, and viewing permission is granted, the anchor drawing unit 304 checks whether the anchor's save permission range includes the logged-in user (S906). If the anchor's save permission range includes the logged-in user in S906, a save dialog is linked and the virtual object is drawn on the display 206 (S907).

[0092] If the scope of anchor saving permission in S906 does not include the logged-in user, the anchor drawing unit 304 checks in S911 whether it is necessary to notify the user that they do not have saving permission. Specifically, in S911, the anchor drawing unit 304 checks whether a value is set in the URL column of Table C for the anchor. If a value is set, it associates a registration dialog 1113 as shown in Figure 11(b) and draws the virtual object on the display 206 (S908).

[0093] If no value is set in the URL column of Table C for the anchor in S911, the anchor drawing unit 304 draws the virtual object on the display 206 without associating it with either the save dialog or the registration dialog (S912). If the anchor drawing unit 304 draws with the save dialog associated in S907, when the user selects the virtual object displayed on the display 206, the save dialog 1043 for the virtual object appears as shown in Figure 10(d). Also, if the anchor drawing unit 304 draws with the registration dialog associated in S908, when the user selects the virtual object displayed on the display 206, the registration dialog 1113 appears as shown in Figure 11(b). Furthermore, if the anchor drawing unit 304 draws without associating it with either the save dialog or the registration dialog in S912, no dialog will appear even if the user selects the virtual object displayed on the display 206.

[0094] In this embodiment, the timing of notification to the user was set to the moment when the anchor feature of S903 and the feature of the real world matched. However, even if the feature amounts do not match, the notification could also be set to the moment the user looks in the direction in which the cylindrical virtual object 1102 is placed, or when the user approaches the place where it will be placed.

[0095] In this embodiment, the conditions set in the Public Access Scope and Save Permission Scope columns of Table C are assumed to be whether or not the user belongs to a group listed in the Groups List column of Table A in Embodiment 1. However, it is also possible to configure the system to set whether or not the user is registered with an existing SNS, whether or not they are a member of a specific group, or whether or not they follow a specific user, for example, by linking with an existing SNS. In that case, the URL column of Table C would contain URLs such as the registration page or ID linking page of the existing SNS, the page for joining a specific group, or the page of a specific user. The method of linking with an existing SNS is omitted here.

[0096] By guiding users to view and save the content using this embodiment, it is possible to control the process so that the content is gradually viewed and saved by setting the scope of publication and the scope of save permission, while still limiting the number of users.

[0097] (Other examples) The present invention also includes apparatuses, systems, and methods configured by appropriately combining the embodiments described above.

[0098] Here, the present invention is a device or system that is the main body for executing one or more software programs that realize the functions of the embodiments described above. Furthermore, a method for realizing the embodiments described above that are executed on that device or system is also part of the present invention. In addition, the program is supplied to the system or device via a network or various storage media, and the program is read into one or more memories by one or more computers (CPU, MPU, etc.) of the system or device and executed. In other words, as part of the present invention, the program itself or various storage media that can be read by the computer storing the program are also included. Furthermore, the present invention can also be realized by a circuit (e.g., ASIC) that realizes the functions of the embodiments described above. [Explanation of symbols]

[0099] 121 Virtual Object Management System 131 Client terminal (head-mounted display) 132 Client terminals (smartphones)

Claims

1. A system for managing virtual objects, It has a management means for managing features in the real world, associated with anchor identification information for displaying virtual objects linked to the real world, The system is characterized in that the management means further manages, in association with the identification information, information indicating the user who created the anchor and settings for determining the range of users who can save copies of the virtual object.

2. The system according to claim 1, wherein the management means further manages user information indicating the target of disclosure of the anchor in association with the identification information.

3. The system according to claim 1, characterized in that the range of users who can save copies of the virtual object can be determined using group information.

4. The system includes a terminal capable of projecting virtual objects into the real world. In response to a request using identification information from a terminal and at least one of the features in the real world, the management means responds with a virtual object managed by the identification information and at least one of the features. The system according to any one of claims 1 to 3, characterized in that the virtual object that is responded to is projected into the real world at the terminal.

5. The system according to claim 4, characterized in that the terminal includes a head-mounted display.

6. The aforementioned system includes a management system and a client. The system according to any one of claims 1 to 5, characterized in that the identification information is allocated in the management system in response to the receipt of a request from the client.

7. A method in a system for managing virtual objects, It has a management process for managing features in the real world, associated with anchor identification information for displaying virtual objects linked to the real world. The management step is characterized in that, in association with the identification information, information indicating the user who created the anchor and settings for determining the range of users who can save copies of the virtual object are also managed.

8. To manage virtual objects, Associate virtual objects with anchor identification information to display them linked to the real world. This is a program that allows a computer to perform the management process of managing features in the real world. And, The program is characterized in that, in the management step, information indicating the user who created the anchor and settings for determining the range of users who can save copies of the virtual object are also managed in association with the identification information.

Images