Information processing apparatus, information processing method, and program

Abstract

To reduce time and effort of work of a user.SOLUTION: An information processing apparatus sets a display attribute indicating permission or non-permission of display in a space experienced by a second user to a first virtual object for which shared display is permitted by a first user in a space in which virtual objects are arranged. Then, the information processing apparatus controls display of the first virtual object in the space experienced by the second user on the basis of the display attribute set for the first virtual object.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing technique for shared display of a virtual object. [Background technology]

[0002] In recent years, there has been progress in the development of technology that utilizes mixed reality (MR) to allow multiple people to share and display virtual objects placed in a mixed reality space (MR space). By sharing and displaying virtual objects, everyone experiencing the MR space can have the same experience through the shared virtual objects, enabling smooth communication. Patent document 1 discloses a technology that enables the owner of a virtual object to control whether the virtual object is shared or hidden on the terminals of other users experiencing the MR space by setting sharing permission or non-permission for the virtual object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-168646 Summary of the Invention [Problem to be solved by the invention]

[0004] In the case of the technology described in Patent Document 1, for example, if another user wants to place another virtual object in the location of a virtual object that is being shared, the other user must request the user who owns the shared virtual object to change the shared display to unauthorized, which is a time-consuming and cumbersome task for both users experiencing MR.

[0005] Therefore, an object of the present invention is to reduce the user's effort. [Means for solving the problem]

[0006] The information processing device of the present invention is characterized by having an attribute setting means for setting a display attribute indicating whether or not a first virtual object that has been permitted for shared display by a first user within a space in which the virtual objects are placed is permitted to be displayed in the space experienced by a second user, and a display control means for controlling the display of the first virtual object within the space experienced by the second user based on the display attribute set for the first virtual object. [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce the burden on the user. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an HMD system. [Figure 2] FIG. 1 is a diagram illustrating an example of the configuration of an HMD. [Figure 3] FIG. 1 illustrates an example of the configuration of an information processing device. [Figure 4] 1 is a diagram illustrating an example of a functional configuration of an information processing device according to a first embodiment. [Figure 5] 4 is a flowchart of information processing according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a display attribute setting UI. [Figure 7] 10 is a flowchart of a virtual object display process according to a display attribute. [Figure 8] FIG. 10 is a diagram illustrating an example of a functional configuration of an information processing device according to a second embodiment. [Figure 9] FIG. 10 is an explanatory diagram of an example of overlapping virtual objects. [Figure 10] 10 is a flowchart of information processing according to the second embodiment. [Figure 11] 10 is a flowchart of an overlap determination process. [Figure 12] FIG. 10 is a diagram illustrating an example of a functional configuration of an information processing device according to a third embodiment. [Figure 13]10 is a flowchart of information processing according to the third embodiment. [Figure 14] 10 is a flowchart of an occlusion determination process. [Figure 15] FIG. 10 is a diagram illustrating an example of a functional configuration of an information processing device according to a fourth embodiment. [Figure 16] FIG. 10 is an explanatory diagram of a display example of a viewing area. [Figure 17] 10 is a flowchart of information processing according to the fourth embodiment. [Figure 18] FIG. 10 is an explanatory diagram of an area where a virtual plane and a viewing area overlap. [Figure 19] 10 is a flowchart of a viewing area setting process. [Figure 20] FIG. 10 is an explanatory diagram of non-permission control of shared display based on a non-display area. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the present embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). Furthermore, the present invention may be configured by appropriately combining parts of the embodiments and modified examples described below. In the following embodiments, redundant descriptions of the same hardware configurations, functional configurations, and processing processes (processing steps) will be omitted.

[0010] In the following embodiments, an information processing system (HMD system) is taken as an example in which a head mounted display (hereinafter referred to as HMD) used by each user is connected to an information processing device. The HMD system in the following embodiments is a system in which two or more users can wear an HMD and experience MR through a mixed reality space (MR space).

[0011] First Embodiment 1 is a diagram showing an example of the configuration of an HMD system in which multiple HMDs are connected to an information processing device, as an application example of an embodiment. In FIG. 1, an example is given in which two HMDs, HMD 101a and HMD 101b, are present as the multiple HMDs, and these HMDs 101a and 101b are connected to an information processing device 102. Assume that the HMD 101a is an HMD worn on the head of a certain user, and the HMD 101b is an HMD worn on the head of another user different from the user wearing the HMD 101a. Furthermore, the information processing device 102 is connected to input devices (not shown), such as a controller and a keyboard, for receiving input from each of the users.

[0012] The HMD 101a and the information processing device 102, and the HMD 101b and the information processing device 102, are connected via a video signal line such as an HDMI (registered trademark) cable or a data signal line such as a USB cable, and image data, control signals, and the like are communicated. The connection between each HMD and the information processing device may be a wireless connection such as a wireless LAN. The connection between the information processing device 102 and an input device (not shown) is assumed to be a wired connection such as a USB cable or a wireless connection such as Bluetooth (registered trademark). An information processing device 102 may exist for each of the HMDs, HMD 101a and HMD 101b. In this case, the information processing devices present in each of the HMDs are connected via a network cable or the like to exchange information. As another example in which an information processing device exists for each of the HMDs, HMD 101a and HMD 101b, an integrated HMD system may be configured in which the HMD 101a includes an information processing device therein, and similarly, the HMD 101b also includes an information processing device therein. As yet another example, the information processing device may be included in either one of the HMDs 101a and 101b.

[0013] In the following description, when it is not necessary to distinguish between the HMD 101a and HMD 101b, the alphabet at the end of the reference numeral will be omitted and they will be collectively referred to as the HMD 101. This also applies to the internal hardware configuration of the HMD 101 in Fig. 2, which will be described below. In the following description, image data handled by each HMD or information processing device will be simply referred to as an image, unless otherwise specified.

[0014] FIG. 2 is a diagram showing an example of the internal hardware configuration of the HMD 101. The HMD 101 has multiple RGB cameras 201 and IMUs (Inertial Measurement Units) such as gyro sensors and acceleration sensors (not shown) to realize position tracking of the HMD. In Fig. 2, an example is given in which two RGB cameras 201L and 201R are provided as the multiple RGB cameras 201. Furthermore, the HMD 101 also has a distance sensor 202 such as a LiDAR (Light Detection And Ranging) for acquiring depth information in the depth direction.

[0015] The HMD 101 also includes a left-eye display 203L and a right-eye display 203R, each configured with a display panel such as a liquid crystal panel or an organic EL panel, for displaying images corresponding to the user's left eye 200L and right eye 200R. The HMD 101 also includes a left-eye eyepiece 204R disposed at a position corresponding to the user's left eye 200L, and a right-eye eyepiece 204R disposed at a position corresponding to the user's left eye 200L. As a result, the user sees an enlarged virtual image of the left-eye display image displayed on the display 203L through the eyepiece 204L, and also sees an enlarged virtual image of the left-eye display image displayed on the display 203R through the eyepiece 204L. At this time, by providing an appropriate parallax between the left-eye display image displayed on the left-eye display 203L and the right-eye display image displayed on the right-eye display 203R, it is possible to provide the user with a sense of depth.

[0016] FIG. 3 is a diagram showing an example of the hardware configuration of the information processing device 102. As shown in FIG. The CPU 301 is a processor that controls all the components in the information processing device and executes various types of arithmetic processing. The GPU 302 receives instructions from the CPU 301, renders virtual objects, and creates a display image to be displayed on the display 203 by superimposing the virtual objects on a real image captured by the RGB camera 201 of the HMD 101.

[0017] The RAM 303 functions as a main memory, work area, etc. for the CPU 301. The ROM 304 stores a group of programs executed by the CPU 301. The mass storage unit 305 stores the information processing program according to this embodiment executed by the CPU 301, various data used in the information processing described below, etc. Note that the mass storage unit 305 may be, for example, an HDD or SSD. The general-purpose I / F 306 is a serial bus interface such as USB or IEEE 1394, and is connected to an IMU and distance sensor provided in the HMD 101. This allows the information processing device 102 to acquire position and orientation information, a depth image of a target object, and the like from the HMD 101. The general-purpose I / F 306 is also used to acquire real images from the RGB camera 201 of the HMD 101. The output I / F 307 is an interface such as HDMI or DisplayPort, and is used to display images on the display 203 of the HMD 101. A network I / F 308 communicates via a network such as a LAN or the Internet under the control of the CPU 301. A system bus 310 controls the flow of data within the device. Note that the information processing device 102 includes other components, but a description of these components will be omitted here.

[0018] The functional configuration of the information processing device 102 according to the first embodiment will be described below. The information processing device 102 of this embodiment is a device that executes information processing such as placing one or more virtual objects in an MR space and displaying them on the displays 203 of HMDs 101 worn by multiple users. The information processing device 102 of this embodiment can also execute information processing such that a user experiencing MR places a virtual object in the MR space and moves or erases the virtual object in the MR space in response to an instruction from the user. Furthermore, the information processing device 102 of this embodiment can also execute information processing for setting whether shared display of a virtual object is permitted or not in response to an input from the user.

[0019] In the description of this embodiment, among multiple users experiencing MR using the HMD system, an HMD used by a first user is referred to as a first HMD 101, and a virtual object set by the first user is referred to as a first virtual object. Furthermore, in this embodiment, a user other than the first user is referred to as a second user, an HMD used by the second user is referred to as a second HMD 101, and a virtual object set by the second user is referred to as a second virtual object. While this embodiment illustrates an example in which two users, the first user and the second user, experience MR, three or more users may be involved. Furthermore, the first user and the second user in this embodiment are not limited to being one user each; the first user may be two or more users, and the second user may be two or more users. Furthermore, the first virtual object set by the first user may be one or more, and similarly, the second virtual object set by the second user may be one or more, and may be one or more. When a first user sets multiple first virtual objects, the first user can set whether each of the first virtual objects can be moved, deleted, or shared within the MR space. Similarly, when a second user sets multiple second virtual objects, the second user can set whether each of the second virtual objects can be moved, deleted, or shared within the MR space. In this embodiment, it is assumed that the positions in the MR space experienced by the first and second users are represented using the same coordinate system.

[0020] In the following embodiment, an example will be described in which the user who has set permission or non-permission to share a virtual object is the first user. The information processing device 102 of this embodiment determines whether to display a first virtual object set by a first user on a second HMD worn by a second user, based on the setting by the first user of whether shared display is permitted or not. For example, if the first user has set shared display of the first virtual object to permitted, the information processing device 102 displays the first virtual object in the MR space of the second HMD 101. On the other hand, if the first user has set shared display of the first virtual object to not permitted, the information processing device 102 does not display the second virtual object in the MR space of the second HMD 101.

[0021] Furthermore, when the first user has permitted shared display of a first virtual object, the information processing device 102 of this embodiment determines whether to display the first virtual object on the second HMD based on the setting of display permission or non-permission by the second user. That is, the information processing device 102 of this embodiment controls the display of the first virtual object on the second HMD based on the display attribute of display permission or non-permission set by the second user for the first virtual object permitted shared display by the first user. For example, when the first user has permitted shared display of the first virtual object, the information processing device 102 displays the first virtual object in the MR space of the second HMD 101 if the second user has set a display attribute that permits display of the first virtual object. On the other hand, even if shared display of the first virtual object is permitted, when the second user has set a display attribute that prohibits display of the first virtual object, the information processing device 102 controls not to display the first virtual object in the MR space of the second HMD 101. In this way, in this embodiment, the second user can arbitrarily set whether or not to display the first virtual object that the first user has permitted to be shared and displayed on the second HMD.

[0022] As described above, the information processing device 102 performs display control of virtual objects based on a setting of permission / prohibition of shared display for the virtual objects and a setting of display attributes of permission / prohibition of display for virtual objects that are permitted for shared display. Note that display control of virtual objects based only on a setting of permission / prohibition of shared display for virtual objects is generally similar to existing technology, and therefore a description thereof will be omitted. In this embodiment, an example will be described in which a first user permits shared display of a first virtual object, and a second user sets a display attribute of permission / prohibition for the first virtual object. Hereinafter, a second user sets a display attribute of permission / prohibition for the first virtual object that is permitted for shared display by the first user, and the information processing device 102 performs display control of whether to display the first virtual object on the second HMD based on the display attribute.

[0023] FIG. 4 is a diagram showing an example of the functional configuration of the information processing device 102 according to the first embodiment that performs the display control described above. As described above, the object information acquisition unit 401 acquires information about a first virtual object that the first user has permitted to be displayed in a shared manner, and information about a second virtual object that the second user has set. Details of the information about the first virtual object and the second virtual object will be described later. Hereinafter, the information about these virtual objects will be referred to as virtual object information. The display attribute setting unit 402 acquires display attributes that the second user has set to allow or disallow display in the MR space of the second HMD for the first virtual object that the first user has permitted to be displayed in a shared manner.

[0024] The display control unit 403 performs display control to display a second virtual object set by the second user in the MR space of the second HMD. When displaying a first virtual object for which shared display is set by the first user to be permitted in the MR space of the second HMD, the display control unit 403 performs display control based on the display attribute of permitting or prohibiting display set by the second user for the first virtual object. When the display attribute setting unit 402 sets the first virtual object to be not permitted to be displayed (display denial), the notification unit 404 generates notification information indicating this and notifies the first HMD used by the first user who set the first virtual object.

[0025] Fig. 5 is a flowchart showing the flow of information processing performed by each functional unit shown in Fig. 4 in the information processing device 102 according to this embodiment. In each of the following flowcharts, the symbol S represents a processing step (process). First, in the process of S501, the object information acquisition unit 401 acquires virtual object information of a first virtual object permitted for shared display by a first user and virtual object information of a second virtual object set by a second user. The virtual object information includes identification information of the HMD used by the user who set the virtual object, a shared setting flag indicating whether shared display is permitted or not, the position and orientation of the virtual object in the MR space, mesh data indicating the shape of the virtual object, texture data indicating the pattern, etc.

[0026] Next, in the processing of S502, the display attribute setting unit 402 acquires the display attribute set by the second user to allow or disallow display in the MR space of the second HMD for the first virtual object that has been permitted for shared display by the first user. In this embodiment, the setting of display attributes that permit or prohibit the display of a virtual object for which shared display is permitted is performed, for example, by a user instruction input via a GUI (graphical user interface). Figures 6(a) and 6(b) are diagrams showing examples of GUIs that are displayed when a user sets display attributes. In this embodiment, these GUIs are displayed on the display screen of the second HMD or the screen of the information processing device 102 when a second user sets display attributes.

[0027] FIG. 6A is a diagram showing an example of a GUI 601 including a list of first virtual objects for which a first user has permitted shared display and a check box corresponding to each of the first virtual objects. FIG. 6A shows an example in which two first users, user A and user B, are first users, a virtual window exists as a virtual object for which user A has set and permitted shared display, and a robot exists as a virtual object for which user B has set and permitted shared display. The GUI 601 includes check boxes 602 and 603 corresponding to each virtual object for which shared display is permitted. The information processing apparatus 102 of this embodiment assigns a display attribute flag to a virtual object for which a check box has been checked by a second user, indicating that the second user has permitted display. On the other hand, the information processing apparatus 102 assigns a display attribute flag to a virtual object for which a check box has not been checked, indicating that the second user has prohibited display.

[0028] FIG. 6(b) is a diagram illustrating an example of a GUI 610 in which first virtual objects permitted for shared display by a first user are placed in an MR space, and displays and check boxes for setting display attributes are placed near the first virtual objects. In the example of FIG. 6(b), as in the example of FIG. 6(a), two first users, user A and user B, are assumed. In the example of the GUI 610 of FIG. 6(b), a virtual window 611 permitted for shared display by user A and a virtual robot 612 permitted for shared display by user B are placed in the MR space. Furthermore, in FIG. 6(b), a check box 613 is placed near the virtual window 611, and a check box 614 is placed near the virtual robot 612. In the example of the GUI 610 of FIG. 6(b), the information processing apparatus 102 assigns a display attribute flag indicating that the second user has permission to display the virtual object whose check box is checked by the second user. Similarly, the information processing device 102 assigns a display attribute flag to virtual objects for which check boxes have not been checked, indicating that the second user has prohibited the display of the virtual objects. Note that the method for setting the display attributes is not limited to the method using the GUI exemplified in Figures 6(a) and 6(b).

[0029] Returning to the explanation of the flowchart in FIG. Next, in the process of S503, the display control unit 403 performs display control to display the second virtual object set by the second user in the MR space of the second HMD as described above. Furthermore, the display control unit 403 controls whether or not to display the first virtual object, for which shared display is permitted by the first user, in the MR space of the second HMD, based on the display attribute flag (permit / prohibit) set by the second user. Details of the display control performed by the display control unit 403 in S503 will be described later.

[0030] Thereafter, in the process of S504, if a display attribute that prohibits display (denies display) is set for the first virtual object by the second user, the notification unit 404 notifies the first HMD of the first user who set the first virtual object of notification information indicating that fact. Note that if a display attribute that permits display of the first virtual object is set by the second user, the notification unit 404 may, but need not, notify the first HMD of the first user of notification information indicating that fact.

[0031] FIG. 7 is a detailed flowchart of the display control performed by the display control unit 403 in S503. In S503, the display control unit 403 executes the processes from S701 to S706 in Fig. 7 for all virtual objects present in the MR space of the second HMD used by the second user. That is, the processes from S701 to S706 are repeatedly executed for each virtual object. In the repeated process from S701 to S706, first, in the process of S702, the display control unit 403 determines whether or not the virtual object being the target of the repeated process is a virtual object set by the second user. In the case of this embodiment, the display control unit 403 determines whether or not the virtual object is a second virtual object set by the second user, for example, based on the identification information of the HMD included in the virtual object information. Then, if it is determined in S702 that the virtual object is a second virtual object set by the second user, the process of the display control unit 403 proceeds to S705. On the other hand, if it is determined that the virtual object is not a second virtual object set by the second user, that is, if it is a first virtual object set by the first user, the process of the display control unit 403 proceeds to S703.

[0032] When the process proceeds to S703, the display control unit 403 determines whether or not a display attribute that allows the second user to display the first virtual object set by the first user in the MR space of the second HMD has been set. In this embodiment, the display control unit 403 makes this determination based on whether or not a display attribute flag indicating that the second user has permitted the first virtual object to be displayed has been assigned to the first virtual object. Then, when it is determined in S703 that the first virtual object is the one permitted to be displayed by the second user, the process of the display control unit 403 proceeds to S705. On the other hand, when it is determined that the first virtual object is not the one permitted to be displayed by the second user, the process of the display control unit 403 proceeds to S704.

[0033] If the process proceeds to S705, the display control unit 403 displays the virtual object in the MR space of the second HMD used by the second user. That is, in this case, the display control unit 403 displays the virtual object generated based on the position and orientation of the virtual object in the MR space, mesh data indicating the shape of the virtual object, and texture data indicating the pattern, etc., included in the virtual object information, in the MR space of the second HMD. On the other hand, if the process proceeds to S704, the display control unit 403 does not display the virtual object in the MR space of the second HMD used by the second user. After S704 or S705, the process of the display control unit 403 returns to S702, and the display control unit 403 performs the processes from S702 onwards on the next virtual object to be processed.

[0034] As described above, according to the information processing device 102 of the first embodiment, a second user can arbitrarily set whether or not to display a first virtual object that a first user has permitted to be displayed in the MR space of a second HMD. That is, in this embodiment, if a second user wants to place a second virtual object at the position of a first virtual object that the first user has permitted to be displayed in a shared manner, the second user does not need to request the first user to disable the shared display of the first virtual object. This reduces the time and effort required for both users experiencing MR, and does not disrupt the MR experience for both users.

[0035] <Second embodiment> In the first embodiment described above, if the second user does not want to display a first virtual object that has been permitted for shared display by a first user in the MR space of a second HMD, the second user can set the display of the first virtual object to not be permitted. In contrast, in the second embodiment, an example is described in which the information processing device 102 automatically sets whether or not to display a first virtual object that a first user has permission to share and display in the MR space of a second HMD used by a second user. Here, the following situations can be considered as situations in which the second user does not want the first virtual object permitted for shared display to be displayed in the MR space of the second HMD. For example, a case in which the second user has already placed the second virtual object at the position where the first virtual object is to be displayed, or a case in which the second user wants to newly place the second virtual object at the position where the first virtual object is already displayed, etc. In other words, when the second virtual object set by the second user and the first virtual object permitted for shared display by the first user are positioned to overlap in the MR space of the second HMD, it is considered that the second user does not want the first virtual object to be displayed.

[0036] Therefore, in the second embodiment, the information processing device 102 performs an overlap determination to determine whether a first virtual object permitted for shared display by a first user and a second virtual object set by a second user overlap in the MR space of a second HMD. Based on the result of the overlap determination, the information processing device 102 automatically sets whether the first virtual object permitted for shared display by the first user is permitted or not permitted to be displayed in the MR space of the second HMD of the second user. In the second embodiment, virtual object overlap refers not only to complete overlap but also to at least partial overlap. In other words, virtual object overlap refers to a situation in which the virtual objects come into contact with or interfere with each other when placed in an MR space. In the second embodiment, the configuration of the HMD system is the same as that shown in FIG. 1 , the configuration of the HMD 101 is the same as that shown in FIG. 2 , and the configuration of the information processing device 102 is the same as that shown in FIG. 3 . Therefore, illustrations and descriptions of these are omitted. The following describes the configuration and processing that are different from the first embodiment.

[0037] Fig. 8 is a diagram showing an example of the functional configuration of an information processing device 102 according to the second embodiment. In the functional configuration of Fig. 8, an object information acquisition unit 801, a display control unit 804, and a notification unit 805 are generally similar to the corresponding object information acquisition unit 401, display control unit 403, and notification unit 404 in Fig. 4, and therefore descriptions thereof will be omitted.

[0038] In the information processing device 102 of the second embodiment, an overlap determination unit 802 determines whether a second virtual object set by a second user overlaps with a first virtual object permitted for shared display by a first user. The display attribute setting unit 803 sets, for the first virtual object, a display attribute that permits or prohibits display in the MR space of the second HMD, based on the overlap determination result by the overlap determination unit 802.

[0039] 9(a) to 9(c) are diagrams showing an example of overlapping virtual objects. FIG. 9(a) shows a first virtual object 901 set by a first user and permitted for shared display, and a second virtual object 902 set by a second user. 9(b) shows an example in which a first virtual object 901 and a second virtual object 902 are placed in the MR space based on the position and orientation information included in the respective virtual object information. That is, if the first virtual object 901 and the second virtual object 902 are placed in the MR space based on the position and orientation information included in the virtual object information, the virtual objects will overlap each other. In this way, when the first virtual object 901 and the second virtual object 902 overlap in the MR space, the display attribute setting unit 803 of the second embodiment sets the display attribute of the first virtual object 901 to unauthorized so that the first virtual object 901 is not displayed in the MR space of the second HMD. FIG. 9(c) shows an example in which the display attribute of the first virtual object 901 is set to not permitted, so that the first virtual object 901 is controlled to be hidden and only the second virtual object 902 is displayed.

[0040] Fig. 10 is a flowchart showing the flow of information processing performed by the functional units shown in Fig. 8 in the information processing device 102 according to the second embodiment. Note that in the flowchart of Fig. 10, S1001, S1004, and S1005 are generally similar to the corresponding S501, S503, and S504 in Fig. 5, and therefore a description thereof will be omitted.

[0041] In the second embodiment, after acquiring virtual object information in S1001, the process of the information processing device 102 proceeds to S1002, which is performed by the overlap determination unit 802. In S1002, the overlap determination unit 802 determines whether the second virtual object set by the second user overlaps with the first virtual object permitted for shared display by the first user. Details of the overlap determination process in S1002 will be described later.

[0042] Next, in S1003, the display attribute setting unit 803 sets a display attribute for the first virtual object to permit or prohibit display in the MR space of the second HMD based on the result of the overlap determination by the overlap determination unit 802. If the first virtual object and the second virtual object overlap, the display attribute setting unit 803 sets the display attribute for the first virtual object to prohibit, and if the first virtual object and the second virtual object do not overlap, the display attribute for the first virtual object to permit. Then, after S1003, the processing of the information processing device 102 proceeds to S1004.

[0043] FIG. 11 is a detailed flowchart of the overlap determination process performed by the overlap determination unit 802 in S1002 of FIG. In S1002, the overlap determination unit 802 executes the processes from S1101 to S1111 in Fig. 11 for all first virtual objects permitted for shared display by the first user. That is, the processes from S1101 to S1111 are repeatedly executed for each first virtual object permitted for shared display. 11 for all second virtual objects set by the second user. That is, the processes from S1102 to S1180 are repeatedly executed for each second virtual object set by the second user. 11 for all vertices of the first virtual object for which shared display is permitted by the first user. That is, the processes from S1104 to S1107 are repeatedly executed for each vertex of the first virtual object for which shared display is permitted.

[0044] In the repeated processing of S1102 to S1108, the overlap determination unit 802 calculates the center of gravity positions and normal vectors for all faces of the second virtual object being repeatedly processed as processing in S1103. Here, the normal vectors are vectors that are perpendicular to the faces and point inward of the virtual object.

[0045] Furthermore, overlap determination unit 802 determines whether or not the vertex that is the target of the repeated processing of S1104 to S1107, among all the vertices of the first virtual object that is the target of the repeated processing of S1101 to S1111, is inside the second virtual object. Here, the inside / outside determination process for determining whether a vertex of the first virtual object is inside the second virtual object can use, for example, the following process. The overlap determination unit 802 calculates a vector connecting the center of gravity position and the target vertex of the first virtual object for each surface constituting the second virtual object. Then, the overlap determination unit 802 calculates the angle between the vector and the normal vector acquired in S1103. To calculate this angle, for example, a method such as calculating the dot product of vectors can be used. If the angle formed by the vectors is within 90 degrees, the overlap determination unit 802 determines that the target vertex of the first virtual object exists inside the surface of the second virtual object. The overlap determination unit 802 performs this process for all surfaces constituting the second virtual object. If the overlap determination unit 802 determines that the target vertex of the first virtual object is inside all surfaces constituting the second virtual object, it determines that the vertex exists inside the second virtual object.

[0046] Next, in S1106, the overlap determination unit 802 branches the process depending on the inside / outside determination process of S1105. For example, if it is determined in S1105 that a vertex of the first virtual object is inside the second virtual object, the process of the overlap determination unit 802 proceeds to S1110. On the other hand, if it is determined in S1105 that a vertex of the first virtual object is not inside the second virtual object, the overlap determination unit 802 performs the inside / outside determination process of S1105 with the next vertex of the first virtual object as the processing target. Then, if all vertices of the first virtual object to be processed are not inside the second virtual object, the process of the overlap determination unit 802 proceeds to S1109.

[0047] In S1110, the overlap determination unit 802 sets the first virtual object, whose vertex is determined to exist inside the second virtual object, to not be permitted to be displayed in the MR space of the second HMD. On the other hand, if the process proceeds to S1109, the overlap determination unit 802 sets the first virtual object, all of whose vertices are determined not to exist inside the second virtual object, to be permitted to be displayed in the MR space of the second HMD.

[0048] As described above, the information processing device 102 according to the second embodiment can automatically set a first virtual object that has been set by a first user to be permitted for shared display not to be displayed on a second HMD when the first virtual object overlaps with a second virtual object. That is, in the second embodiment, when a first virtual object that has been set by a first user to be permitted for shared display overlaps with a second virtual object, the second user does not need to set the display attributes to prevent the first virtual object from being displayed on the second HMD.

[0049] <Third embodiment> In addition to the overlapping described in the second embodiment, a situation in which a second user does not want to display a first virtual object that a first user has permitted to be shared may also occur when the first virtual object occludes the second virtual object in the MR space of the second HMD. That is, a situation in which the first virtual object occludes the second virtual object in the MR space of the second HMD, making the second virtual object invisible to the second user (interfering with viewing) may also occur. In the third embodiment, occlusion of a virtual object may include not only a case in which the virtual object is completely occluded, but also a case in which the virtual object is at least partially occluded.

[0050] Therefore, the information processing device 102 of the third embodiment performs an occlusion determination to determine whether a second virtual object set by a second user is occluded by a first virtual object permitted for shared display by a first user. Based on the result of the occlusion determination, the information processing device 102 of the third embodiment sets whether the first virtual object permitted for shared display by the first user is permitted to be displayed in the MR space of the second HMD of the second user. In the third embodiment, occlusion by the first virtual object may include a case where the second virtual object is only partially occluded, in addition to a case where the second virtual object is completely occluded. Note that in the third embodiment, the configuration of the HMD system is the same as the example in FIG. 1, the configuration of the HMD 101 is the same as the example in FIG. 2, and the configuration of the information processing device 102 is the same as the example in FIG. 3, and therefore illustration and description thereof will be omitted. Below, configurations and processes different from those of the first embodiment will be described.

[0051] Fig. 12 is a diagram showing an example of the functional configuration of an information processing device 102 according to the third embodiment. In the functional configuration of Fig. 12, an object information acquisition unit 1201, a display control unit 1205, and a notification unit 1206 are generally similar to the corresponding object information acquisition unit 401, display control unit 403, and notification unit 404 in Fig. 4, and therefore descriptions thereof will be omitted.

[0052] In the information processing device 102 of the third embodiment, an HMD information acquisition unit 1202 acquires information of a second HMD used by a second user, such as the position and orientation in the MR space, the display angle of view, and the display resolution. In this embodiment, the information of the HMD, such as the position and orientation in the MR space, the display angle of view, and the display resolution, is referred to as HMD information.

[0053] The occlusion determination unit 1203 performs occlusion determination based on the virtual object information of the first virtual object and the second virtual object acquired by the object information acquisition unit 1201 and the HMD information acquired by the HMD information acquisition unit 1202. Details of the occlusion determination process performed by the occlusion determination unit 1203 will be described later. The display attribute setting unit 1204 sets, for the first virtual object, a display attribute that allows or prohibits display in the MR space of the second HMD, based on the occlusion determination result by the occlusion determination unit 1203.

[0054] Fig. 13 is a flowchart showing the flow of information processing performed by the functional units shown in Fig. 12 in the information processing device 102 according to the third embodiment. Note that in the flowchart of Fig. 13, S1301, S1305, and S1306 are generally similar to the corresponding S501, S503, and S504 in Fig. 5, and therefore description thereof will be omitted.

[0055] In the third embodiment, after acquiring the virtual object information in S1301, the processing of the information processing device 102 proceeds to S1302, which is performed by the HMD information acquisition unit 1202. In S1302, the HMD information acquisition unit 1202 acquires HMD information including information such as the position and orientation of the second HMD in the MR space, the display angle of view, and the display resolution.

[0056] Next, in S1303, the occlusion determination unit 1203 determines whether the second virtual object is occluded by the first virtual object permitted for shared display by the first user in the MR space of the second HMD used by the second user. Details of the occlusion determination process in S1302 will be described later.

[0057] Next, in S1304, the display attribute setting unit 803 sets a display attribute for the first virtual object to permit or prohibit display in the MR space of the second HMD based on the result of the occlusion determination by the occlusion determination unit 1203. That is, if the second virtual object is occluded by the first virtual object, the display attribute setting unit 803 sets the display attribute for the first virtual object to prohibit, and if the second virtual object is not occluded, the display attribute for the first virtual object to permit. Then, after S1304, the processing of the information processing device 102 proceeds to S1305.

[0058] FIG. 14 is a detailed flowchart of the occlusion determination process performed by the occlusion determination unit 1203 in S1302 of FIG. First, in step S1401, the occlusion determination unit 1203 generates a depth image when a second virtual object is projected into the MR space of the second HMD used by the second user. The depth image is an image that is rendered according to the distance in the MR space, and can be generated by executing a known rendering process. Note that the HMD information in step S1302 is used in the coordinate conversion included in the rendering process.

[0059] Next, in S1402, the occlusion determination unit 1203 generates a depth image when the first virtual object, for which shared display is permitted by the first user, is projected into the MR space of the second HMD. The depth image in this case can also be generated by executing a known rendering process similar to that in S1401.

[0060] Next, the occlusion determination unit 1203 executes the processes from S1403 to S1407 in Fig. 14 for all pixels on the display screen of the second HMD used by the second user. That is, the occlusion determination unit 1203 repeats the processes from S1403 to S1407 for each pixel on the display screen of the second HMD.

[0061] In the repeated processing of S1403 to S1407, the occlusion determination unit 1203 branches the processing in S1404 depending on whether or not a first virtual object permitted for shared display by the first user is present at the pixel being processed repeatedly. If the occlusion determination unit 1203 determines that a first virtual object permitted for shared display is present at the pixel being processed, the processing proceeds to S1405. On the other hand, if the occlusion determination unit 1203 determines that a first virtual object permitted for shared display is not present at the pixel being processed repeatedly, the occlusion determination unit 1203 makes the next pixel the target of the repeated processing. Here, the processing of determining whether or not a first virtual object permitted for shared display by the first user is present is performed based on the depth image generated from the first virtual object in S1402. That is, if the pixel being processed is present in the depth image generated from the first virtual image and the pixel value of the pixel being processed is not a value of the far plane, the occlusion determination unit 1203 determines that a first virtual object is present at the pixel being processed.

[0062] If the process proceeds to S1405, the occlusion determination unit 1203 branches the process depending on whether or not a second virtual object set by the second user exists for the pixel to be processed. If the second virtual object exists, the process proceeds to S1406; if the second virtual object does not exist, the process proceeds to S1407. If the occlusion determination unit 1203 determines that a second virtual object exists for the pixel to be processed, the process proceeds to S1406. On the other hand, if the occlusion determination unit 1203 determines that a second virtual object does not exist, the process repeats the process for the next pixel. The process of determining whether or not a second virtual object exists is performed based on the depth image generated from the second virtual object in S1401. If the pixel to be processed is present in the depth image generated from the second virtual image and the pixel value of the pixel to be processed is not a far plane value indicating a far distance that is not visible in the MR space, the occlusion determination unit 1203 determines that a second virtual object exists for the pixel to be processed.

[0063] If the process proceeds to S1406, the occlusion determination unit 1203 compares the depth value of the depth image of the first virtual object with the depth value of the depth image of the second virtual object at the corresponding pixel position for the pixel to be processed. Then, based on the comparison result of the depth values ​​for each corresponding pixel position, the occlusion determination unit 1203 determines which is the closer side in the MR space of the second HMD. If it is determined that the depth value of the first virtual object is the closer side, the process proceeds to S1409. On the other hand, if it is determined that the depth value of the second virtual object is not the closer side, the occlusion determination unit 1203 iteratively processes the next pixel.

[0064] Then, in S1409, the occlusion determination unit 1203 determines that the first virtual object occludes the second virtual object in the MR space of the second HMD used by the second user. On the other hand, if the process of S1403 to S1407 is repeated for all pixels on the display screen of the second HMD, and it is determined in S1406 that the depth value of the second virtual object is not in the foreground, the occlusion determination unit 1203 proceeds to S1408. In S1408, the occlusion determination unit 1203 determines that the first virtual object does not occlude the second virtual object in the MR space of the second HMD used by the second user.

[0065] As described above, the information processing device 102 according to the third embodiment can automatically set the second HMD not to display a first virtual object when the second virtual object is occluded by the first virtual object that the first user has permitted to be displayed in a shared manner. In other words, in the third embodiment, when the second virtual object is occluded by the first virtual object that the first user has permitted to be displayed in a shared manner, the second user does not need to set the display attributes to prevent the first virtual object from being displayed in the second HMD.

[0066] <Fourth embodiment> In the first to third embodiments described above, an example was described in which whether or not to display a first virtual object permitted for shared display by a first user in the MR space of a second HMD used by a second user was manually or automatically controlled. When the first virtual object permitted for shared display by a first user is controlled not to be displayed in this manner, the second user will naturally lose track of where the first virtual object is located in the MR space of the second HMD. Therefore, the second user may accidentally place a real object, not a virtual object, at the position of the first virtual object permitted for shared display by the first user in the MR space. In this case, the real object will obstruct the first user's viewing of the first virtual object in the MR space experienced by the first user using the first HMD.

[0067] Therefore, in the fourth embodiment, when the display attribute of a first virtual object permitted for shared display by a first user is set to hidden, an area in which the first user is viewing the first virtual object using the first HMD is presented to the second HMD of the second user. In this embodiment, the area in which the first user is viewing the first virtual object using the first HMD is referred to as the viewing area. The information processing device 102 of the fourth embodiment sets the viewing area based on the position and orientation of the first HMD in the MR space and the position of the first virtual object in the MR space. Note that in the fourth embodiment, the configuration of the HMD system is the same as the example in FIG. 1, and the configuration of the HMD 101 is the same as the example in FIG. 2, and the configuration of the information processing device 102 is the same as the example in FIG. 3. Therefore, illustration and description thereof will be omitted. Below, configurations and processes different from the above-described embodiments will be described.

[0068] 15 is a diagram showing an example of the functional configuration of an information processing device 102 according to the fourth embodiment. Note that the object information acquisition unit 1501, the display attribute setting unit 1502, the display control unit 1503, and the notification unit 1507 are generally similar to the object information acquisition unit 401, the display attribute setting unit 402, the display control unit 403, and the notification unit 404 corresponding to FIG. 4, and therefore description thereof will be omitted.

[0069] In the information processing device 102 of the fourth embodiment, the HMD information acquisition unit 1504 acquires information such as the position and orientation in the MR space, the display angle of view, and the display resolution of the first HMD used by the first user who has permitted shared display of the first virtual object. As in the example of the third embodiment described above, in the fourth embodiment, information such as the position and orientation in the MR space of the HMD, the display angle of view, and the display resolution is referred to as HMD information. However, while the HMD information described in the third embodiment is information related to the second HMD used by the second user, the HMD information in the fourth embodiment is information related to the first HMD used by the first user.

[0070] The area setting unit 1505 sets a viewing area based on virtual object information of a first virtual object permitted for shared display by the first user, the display attributes set for the first virtual object, and the HMD information acquired by the HMD information acquisition unit 1504. When the display attributes of the first virtual object permitted for shared display are set to unauthorized, the area setting unit 1505 sets an area between the first HMD and the first virtual object in the MR space as the viewing area. Details of the viewing area setting process by the area setting unit 1505 will be described later. Note that the setting of the display attributes of the first virtual object permitted for shared display to unauthorized may be performed by the second user as in the first embodiment described above, or may be automatically set by the information processing device 102 as in the second to fourth embodiments.

[0071] The area display unit 1506 presents (displays) in the MR space of the second HMD used by the second user the viewing area set by the area setting unit 1505. That is, the area display unit 1506 presents in the MR space of the second HMD the viewing area when the first user is viewing the first virtual object in the MR space of the first HMD.

[0072] 16 is a schematic diagram showing an overhead view of an MR space experienced by a first user 1601 using a first HMD (not shown) and a second user 1611 using a second HMD (not shown). In the MR space, there are two real objects, desks 1602 and 1612, and multiple virtual monitors 1603, 1604, 1605, 1613, 1614, and 1615 are placed as virtual objects. Of these virtual monitors, the virtual monitors 1603, 1604, and 1605 are virtual objects set by the first user 1601, while the virtual monitors 1613, 1614, and 1615 are virtual objects set by the second user 1611. It is also assumed that the first user 1601 has permission to share the virtual monitors 1603, 1604, and 1605.

[0073] Here, assume that the display attribute of the virtual monitor 1605, for which the first user 1601 has permitted shared display, is set to hidden because it interferes with the second user 1611's viewing of the virtual monitor 1615. In this case, the region setting unit 1505 sets a viewing region 1606 for the first user 1601 based on the positional relationship between the positions of the HMD and virtual monitor 1605 of the first user 1601 and the position of the HMD of the second user 1611. Note that in the example of FIG. 6, for convenience of illustration, the viewing region 1606 is shown as a two-dimensional region without a height direction; however, because the MR space is a three-dimensional space, it is actually calculated as a three-dimensional region including a height direction.

[0074] Fig. 17 is a flowchart showing the flow of information processing performed by the functional units shown in Fig. 15 in the information processing device 102 according to the fourth embodiment. Note that in the flowchart of Fig. 17, S1701, S1702, S1703, and S1707 are generally similar to the corresponding S501, S502, S503, and S504 in Fig. 5, and therefore description thereof will be omitted.

[0075] In the fourth embodiment, the process of the information processing apparatus 102 proceeds to S1704, which is performed by the HMD information acquisition unit 1504, after S1703. Proceeding to S1704, the HMD information acquisition unit 1504 acquires HMD information such as the position and orientation in the MR space, the display angle of view, and the display resolution of the first HMD used by the first user who has permitted the shared display of the first virtual object.

[0076] Next, in S1705, the area setting unit 1505 sets a viewing area based on the position of the first virtual object for which shared display is permitted, the display attributes set for the first virtual object, and the HMD information acquired by the HMD information acquisition unit 1504. Details of the viewing area setting process in S1705 will be described later.

[0077] Next, in S1706, the area display unit 1506 presents the viewing area set in S1705 in the MR space of the second HMD used by the second user. The viewing area for the second HMD may be presented, for example, by filling the interior of the viewing area with a semitransparent color. Alternatively, if a second virtual object exists within the viewing area, the viewing area may be displayed in a manner that does not interfere with viewing of the second virtual object. For example, a virtual plane may be defined at the height of the shortest vertex of the second virtual object existing within the viewing area, and the overlapping area between the virtual plane and the viewing area may be displayed by filling it with a predetermined pattern with a transparent background color. FIG. 18 is a diagram showing an example in which the overlapping area between the virtual plane and the viewing area is displayed by filling it with a predetermined pattern with a transparent background color. 18, when a second virtual object 1801 exists inside a viewing area 1802, the area display unit 1506 displays the viewing area 1802 at the height of the shortest vertex among the vertices that make up the second virtual object 1801. Of course, the method of presenting (displaying) the viewing area is not limited to these examples. After S1706, the processing of the information processing device 102 proceeds to S1707, which is the same as S504.

[0078] FIG. 19 is a flowchart of the viewing area setting process performed in the area setting unit 1505 in S1705 of FIG. First, in the process of S1901, the area setting unit 1505 calculates a direction vector connecting the position of the first HMD acquired in S1704 and the position of the first virtual object acquired in S1701.

[0079] Next, in the process of S1902, the area setting unit 1505 calculates the coordinates of each vertex of the first virtual object when the first virtual object is placed in the MR space based on the virtual object information acquired in S1701. Next, in the process of S1903, the region setting unit 1505 uses the direction vector calculated in S1901 and the coordinates of each vertex calculated in S1902 to determine each vertex that will be the endpoint of the first virtual object. Each vertex that will be the endpoint of the first virtual object can be determined, for example, as follows. First, the region setting unit 1505 determines the positional relationship between a vertex of interest and the direction vector. For example, the region setting unit 1505 calculates the cross product of the direction vector and a vector connecting the vertex of interest and the position of the first HMD, and determines whether the vertex is on the left or right side of the direction vector based on the sign of the cross product, and calculates the distance between the direction vector and the vertex of interest. The region setting unit 1505 performs this process for all vertices and calculates the vertex with the greatest distance on both the left and right sides of the direction vector. This vertex becomes the endpoint of the first virtual object.

[0080] Next, in the processing of S1904, the area setting unit 1505 sets the area enclosed by the line connecting the end point determined in S1903 and the position of the first HMD obtained in S1704 as a triangular prism-shaped area extended in the height direction of the first virtual object in the MR space as the viewing area.

[0081] As described above, the information processing device 102 according to the fourth embodiment sets the area between the first user and the first virtual object that the first user has permitted to be shared and displayed as a viewing area, and displays the area in the MR space of the second HMD used by the second user. This allows the second user to understand the viewing area when the first user views the first virtual object, even if the display attribute of the first virtual object is set to unauthorized, and to know that it is desirable to place, for example, a real object in the viewing area.

[0082] <First Modification> In the above-described embodiments, when the display attribute of a first virtual object that is permitted for shared display is changed to "disabled," the first virtual object is not displayed in the MR space of the second HMD. In contrast, as a first modification, the first virtual object may be displayed using a different rendering method so as not to interfere with the second user's viewing. For example, a rendering method that reduces the transparency of the first virtual object or a rendering method that renders only the shape using a wireframe may be used.

[0083] <Second Modification> In the second embodiment described above, the overlap between the first virtual object permitted for shared display and the second virtual object set by the second user is determined, but the overlap determination is not limited to this example. In the second modified example, an area where the second virtual object may be placed is set in advance as a non-display area, and the overlap determination unit 802 determines whether at least a part of the first virtual object overlaps (is in contact with or interferes with) the non-display area.

[0084] Fig. 20 is a diagram used to explain the second modified example. In the example of Fig. 20, for example, a rectangular parallelepiped area having eight vertices is preset as a non-display area 2002 on a table 2001, which is a physical object existing in the MR space. Then, the overlap determination unit 802 of the second modified example determines whether this preset non-display area 2002 overlaps with a first virtual object 2003 that has been permitted for shared display by a first user. In the example of Fig. 20, because the first virtual object 2003 and the preset non-display area 2002 overlap, the display attribute setting unit 803 sets the display attribute of the first virtual object 2003 to "unpermitted."

[0085] Furthermore, the above-described embodiments and modifications may be combined as appropriate. For example, when the first embodiment is combined with the second embodiment or the third embodiment, the display attributes may be set by the information processing device 102 in addition to the setting of the display attributes by the second user. For example, when the second embodiment is combined with the third embodiment, the display attributes of a virtual object may be set based on the results of both the overlap determination and the occlusion determination of the virtual object. Furthermore, the fourth embodiment, the first modification, or the second modification may be applied to the first to third embodiments.

[0086] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific implementations of the present invention, and should not be construed as limiting the technical scope of the present invention. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0087] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) an attribute setting means for setting a display attribute indicating whether or not a first virtual object permitted for shared display by a first user is to be displayed in a space experienced by a second user in the space where the virtual object is placed; a display control means for controlling the display of the first virtual object in the space experienced by the second user based on the display attribute set for the first virtual object; An information processing device comprising: (Configuration 2) The information processing device according to configuration 1, wherein the display control means controls the display of the first virtual object so as not to display the first virtual object in the space experienced by the second user when the display attribute set for the first virtual object is the unauthorized one. (Configuration 3) The information processing device according to configuration 1, characterized in that, when the display attribute set for the first virtual object is the unauthorized one, the display control means changes the drawing method of the first virtual object and displays it in the space experienced by the second user. (Configuration 4) 4. The information processing device according to configuration 3, wherein the method of rendering the first virtual object when the display attribute is not permitted includes a method of reducing transparency of the first virtual object. (Configuration 5) 4. The information processing device according to configuration 3, wherein the method for drawing the first virtual object when the display attribute is the unauthorized one includes a method for drawing a wire frame corresponding to the first virtual object. (Configuration 6) 6. The information processing apparatus according to any one of configurations 1 to 5, wherein the attribute setting means sets the display attributes based on an instruction input from the second user. (Configuration 7) further comprising overlap determination means for determining an overlap between the first virtual object and a second virtual object set by the second user, The information processing device according to any one of configurations 1 to 6, wherein the attribute setting means sets the display attribute for the first virtual object to prohibit display in the space experienced by the second user when the overlap determination means determines that at least a portion of the first virtual object and the second virtual object overlap. (Configuration 8) The information processing device according to configuration 7, wherein the overlap determination means determines that the first virtual object and the second virtual object overlap when at least a part of the first virtual object is included inside the second virtual object. (Configuration 9) The information processing device according to configuration 8, wherein the overlap determination means acquires a center of gravity and a normal vector for each face of the second virtual object, calculates an angle formed by a vector connecting the center of gravity of a face of the second virtual object to a vertex of the first virtual object and the normal vector for all faces of the second virtual object, and determines that the first virtual object and the second virtual object overlap when the calculated angle is within 90 degrees for all faces of the second virtual object. (Configuration 10) further comprising overlap determination means for determining an overlap between a non-display area set in the space experienced by the second user and the first virtual object; The information processing device described in any one of configurations 1 to 9, characterized in that, when the overlap determination means determines that at least a portion of the non-display area and the first virtual object overlap, the attribute setting means sets the display attribute for the first virtual object to prohibit display in the space experienced by the second user. (Configuration 11) further comprising occlusion determination means for determining whether a second virtual object set by the second user is occluded by the first virtual object; The information processing device described in any one of configurations 1 to 10, characterized in that, when the occlusion determination means determines that at least a portion of the second virtual object is occluded by the first virtual object, the attribute setting means sets the display attribute for the first virtual object to prohibit display in the space experienced by the second user. (Configuration 12) The information processing device according to configuration 11, wherein the occlusion determination means determines whether the second virtual object is occluded by the first virtual object based on distances from the second user to the second virtual object and the first virtual object in the space experienced by the second user. (Configuration 13) The information processing device described in configuration 12, characterized in that the occlusion determination means determines whether the second virtual object is occluded by the first virtual object based on a result of comparing depth values ​​for each pixel at corresponding positions in a depth image projected onto the second virtual object and a depth image projected onto the first virtual object in the space experienced by the second user. (Configuration 14) an area setting means for setting a viewing area in which the first user views the first virtual object in the space experienced by the first user; 14. The information processing device according to any one of configurations 1 to 13, further comprising: area display means for displaying the viewing area within the space experienced by the second user. (Configuration 15) The information processing device described in configuration 14, characterized in that the area setting means sets, as the viewing area, an area surrounded by lines connecting the position of the first user in the space experienced by the first user to each vertex that is the endpoint of the first virtual object, and an area expanded in the height direction of the first virtual object in the space. (Configuration 16) The information processing device according to configuration 15, wherein the area setting means determines in which direction each vertex of the first virtual object exists with respect to the direction vector based on the cross product of a direction vector connecting the position of the first user to the first virtual object in the space experienced by the first user and a vector connecting each vertex of the first virtual object with the position of the first user, calculates a distance of the first virtual object with respect to the direction vector for each direction in which each vertex of the first virtual object exists, and sets the vertex with the greatest distance as an end point of the first virtual object. (Configuration 17) 17. The information processing device according to any one of configurations 1 to 16, wherein the display control means further comprises a notification means for notifying the first user that the display attribute set for the first virtual object is not permitted, when the display attribute is not permitted. (Configuration 18) 18. The information processing device according to any one of configurations 1 to 17, wherein the space experienced by the user is a mixed reality space, and the user experiences the mixed reality space using a head-mounted display device. (Method 1) an attribute setting step of setting a display attribute for a first virtual object permitted for shared display by a first user in a space in which the virtual object is placed, the display attribute indicating whether or not the first virtual object is permitted for display in the space experienced by a second user; a display control step of controlling display of the first virtual object in the space experienced by the second user based on the display attributes set for the first virtual object; An information processing method comprising: (Program 1) A program that causes a computer to function as the information processing device according to any one of configurations 1 to 18. [Explanation of symbols]

[0088] 401: Object information acquisition unit, 402: Display attribute setting unit, 403: Display control unit, 404: Notification unit

Claims

1. an attribute setting means for setting a display attribute indicating whether or not a first virtual object permitted for shared display by a first user in a space where the virtual object is placed is permitted to be displayed in the space experienced by a second user; a display control means for controlling the display of the first virtual object in the space experienced by the second user based on the display attribute set for the first virtual object; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the display control means controls the display of the first virtual object so that the first virtual object is not displayed in the space experienced by the second user when the display attribute set for the first virtual object is not permitted.

3. 2. The information processing device according to claim 1, wherein, when the display attribute set for the first virtual object is the unauthorized attribute, the display control means changes the drawing method of the first virtual object and displays it in the space experienced by the second user.

4. 4. The information processing apparatus according to claim 3, wherein the method of rendering the first virtual object when the display attribute is the non-permitted state includes a method of reducing transparency of the first virtual object.

5. 4. The information processing apparatus according to claim 3, wherein the method of drawing the first virtual object when the display attribute is the non-permitted state includes a method of drawing a wire frame corresponding to the first virtual object.

6. 2. The information processing apparatus according to claim 1, wherein said attribute setting means sets said display attributes based on an instruction input from said second user.

7. further comprising overlap determination means for determining an overlap between the first virtual object and a second virtual object set by the second user, 2. The information processing device according to claim 1, wherein, when the overlap determination means determines that at least a portion of the first virtual object and the second virtual object overlap, the attribute setting means sets the display attribute for the first virtual object to prohibit display in the space experienced by the second user.

8. 8. The information processing apparatus according to claim 7, wherein the overlap determination means determines that the first virtual object and the second virtual object overlap when at least a part of the first virtual object is included inside the second virtual object.

9. 9. The information processing device according to claim 8, wherein the overlap determination means acquires a center of gravity and a normal vector for each face of the second virtual object, calculates an angle formed by a vector connecting the center of gravity of a face of the second virtual object to a vertex of the first virtual object and the normal vector for all faces of the second virtual object, and determines that the first virtual object and the second virtual object overlap if the calculated angle is within 90 degrees for all faces of the second virtual object.

10. further comprising overlap determination means for determining an overlap between a non-display area set in the space experienced by the second user and the first virtual object; 2. The information processing device according to claim 1, wherein, when the overlap determination means determines that at least a portion of the non-display area and the first virtual object overlap, the attribute setting means sets the display attribute for the first virtual object to prohibit display in the space experienced by the second user.

11. the display device further includes an occlusion determination means for determining whether a second virtual object set by the second user is occluded by the first virtual object, 2. The information processing device according to claim 1, wherein, when the occlusion determination means determines that at least a portion of the second virtual object is occluded by the first virtual object, the attribute setting means sets the display attribute for the first virtual object to prohibit display in the space experienced by the second user.

12. 12. The information processing device according to claim 11, wherein the occlusion determination means determines whether the second virtual object is occluded by the first virtual object based on distances from the second user to the second virtual object and the first virtual object in the space experienced by the second user.

13. The information processing device according to claim 12, characterized in that the occlusion determination means determines whether the second virtual object is occluded by the first virtual object based on a result of comparing depth values ​​for each pixel at corresponding positions in a depth image projected onto the second virtual object and a depth image projected onto the first virtual object in the space experienced by the second user.

14. an area setting means for setting a viewing area in which the first user views the first virtual object in the space experienced by the first user; 2. The information processing apparatus according to claim 1, further comprising: area display means for displaying the viewing area within the space experienced by the second user.

15. 15. The information processing device according to claim 14, wherein the area setting means sets, as the viewing area, an area surrounded by lines connecting the position of the first user in the space experienced by the first user to each vertex that is an endpoint of the first virtual object, and extending the area in the height direction of the first virtual object in the space.

16. 16. The information processing apparatus according to claim 15, wherein the area setting means determines, based on a cross product of a direction vector connecting the position of the first user to the first virtual object in the space experienced by the first user and a vector connecting each vertex of the first virtual object and the position of the first user, in which direction the vertices of the first virtual object exist with respect to the direction vector; calculates a distance of the first virtual object with respect to the direction vector for each direction in which each vertex of the first virtual object exists; and sets the vertex with the greatest distance as an end point of the first virtual object.

17. 2. The information processing device according to claim 1, wherein the display control means further comprises a notification means for notifying the first user that the display attribute set for the first virtual object is the unauthorized one when the display attribute is the unauthorized one.

18. 2. The information processing apparatus according to claim 1, wherein the space experienced by the user is a mixed reality space, and the user experiences the mixed reality space using a head-mounted display device.

19. an attribute setting step of setting, for a first virtual object permitted for shared display by a first user in a space in which the virtual object is placed, a display attribute indicating whether or not the first virtual object is permitted for display in the space experienced by a second user; a display control step of controlling display of the first virtual object in the space experienced by the second user based on the display attributes set for the first virtual object; An information processing method comprising:

20. Computer, an attribute setting means for setting a display attribute indicating whether or not a first virtual object permitted for shared display by a first user in a space where the virtual object is placed is permitted to be displayed in the space experienced by a second user; a display control means for controlling the display of the first virtual object in the space experienced by the second user based on the display attribute set for the first virtual object; A program that causes the device to function as an information processing device having the above.

Images