Information processing device, information processing method, and program

What is AI technical title?
AI technical title is built by Patsnap AI team. It summarizes the technical point description of the patent document.
The information processing device adjusts virtual viewpoints based on real-space measurements and coordinate systems to allow multiple participants with varying experience spaces to observe virtual objects from the same perspective, addressing communication limitations in XR systems.

JP2025182448APending Publication Date: 2025-12-15CANON KK

View PDF 2 Cites 0 Cited by

Patent Information

Application Number: JP2024090009
Authority / Receiving Office: JP · JP
Patent Type: Applications
Current Assignee / Owner
Filing Date: 2024-06-03
Publication Date: 2025-12-15

AI Technical Summary

Technical Problem

Existing XR systems face challenges in allowing multiple participants to observe virtual objects from the same viewpoint, particularly when participants have different-sized experience spaces, leading to communication hindrances and limited interaction.

Method used

An information processing device that measures the position and posture of users in real space, detects their experiential spaces, sets a reference coordinate system, and adjusts the virtual viewpoint to ensure all participants can see the virtual object from the same perspective, using methods like Visual SLAM and Structure from Motion.

Benefits of technology

Enables multiple users to observe virtual objects from the same viewpoint, regardless of their individual experience space sizes, enhancing communication and interaction in shared XR experiences.

✦ Generated by Eureka AI based on patent content.

Smart Images

Figure 2025182448000001_ABST

Patent Text Reader

Abstract

To provide an information processing device that multiple experienced people can observe a virtual object from the same viewpoint.SOLUTION: An information processing device comprises an experience space detection unit 102 to detect an experience space in which a first experienced person can move in a real space, a second position receiving unit 103 to receive the position (a second position) of a second experienced person in a virtual space, a coordinate system setting unit 104 to set a reference coordinate system when a virtual object is placed in the experience space detected by the experience space detection unit 102 and in the virtual space based on the second position received by the second position receiving unit 103, a virtual viewpoint setting unit 105 to set the virtual viewpoint of the first experienced person in the virtual space based on the reference coordinate system set by the position (the first position) of the first experienced person in the real space measured by a first position measuring unit 101 and by the coordinate system setting unit 104, and a scene generation unit 106 to generate the virtual space image including the virtual object based on the virtual viewpoint of the first experienced person set by the virtual viewpoint setting unit 105.SELECTED DRAWING: Figure 1

Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, XR systems have been developed that use virtual reality (VR) and mixed reality (MR) technologies to present users with images that make them feel as if virtual objects are actually in front of them. XR systems can present virtual objects at life-size scales by using, for example, a head-mounted display (HMD). Therefore, if you have design data such as shapes and designs created with a CAD (computer-aided design) system, you can conduct a similar review without having to prepare a physical object.

[0003] In XR systems, the virtual viewpoint in the virtual space moves in accordance with the movement of the viewer's viewpoint in the real space, creating images that make it appear as if the virtual objects are fixed in the real space. Therefore, if the range in which the viewer can move in the real space (the experience space within the real space) is limited, the range in which the viewer can move in the virtual space (the experience space within the virtual space) is also limited. To address this limitation, a method that does not rely on control input from the viewer is sometimes used, by having the viewer use a controller or gestures to move the viewer's viewpoint. However, to create an XR system that is easy to use even for users who are unfamiliar with the operation, a method that does not rely on control input from the user is desirable.

[0004] Even in remote experience sharing situations where multiple participants simultaneously observe the same virtual object using an XR system in different locations in real space, the limited experience space within the virtual space can be a problem. In remote experience sharing, one participant may have a small experience space while the other has a large experience space. In this case, a virtual object that can be observed by the other participant with a large experience space may not be observable by the other participant with a small experience space. This can hinder communication between participants.

[0005] For example, in Patent Document 1, in such a situation, if the position of the other participant is outside the participant's own experiential space, the other participant's avatar is displayed in a different display format than usual. Furthermore, in Patent Document 1, the other participant is notified of such a situation by an icon. This method in Patent Document 1 has the effect of avoiding misunderstandings in communication between participants, since multiple participants can know that they cannot observe a virtual object from the same viewpoint. However, the method in Patent Document 1 still has the limitation that multiple participants cannot observe a virtual object from the same viewpoint.

[0006] Furthermore, in Patent Document 2, the movement range and movement trajectory of a virtual object are adjusted so that the virtual object is displayed within a range where the user can interact with it. This method in Patent Document 2 processes the placement of virtual objects, which are content in a virtual space, and when there are multiple users, the placement locations of the virtual objects may differ depending on the size of each user's experience space. Therefore, even with the method in Patent Document 2, there is still a possibility that multiple users may not be able to observe the virtual object from the same viewpoint. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent No. 6920057 [Patent Document 2] Patent No. 6403843 [Non-patent literature]

[0008] [Non-Patent Document 1] G. Klein and D. Murray: "Parallel Tracking and Mapping for Small AR Workspaces", Proceedings of Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality, 2007. Summary of the Invention [Problem to be solved by the invention]

[0009] In the above-mentioned Patent Documents 1 and 2, when multiple participants use the XR system simultaneously in experience spaces of different sizes, there is a problem that the multiple participants cannot observe the virtual objects from the same viewpoint depending on the size and arrangement of the virtual objects.

[0010] The present invention has been made in view of the above-mentioned problems, and has as its object to enable a plurality of participants to observe a virtual object from the same viewpoint. [Means for solving the problem]

[0011] The information processing device of the present invention has a measurement means for measuring at least one state of a position and posture of a first experiencer in a real space, a first acquisition means for acquiring a space in the real space in which the first experiencer can move as an experiential space, a second acquisition means for acquiring at least one state of a position and posture of a second experiencer in a virtual space, a first setting means for setting a reference coordinate system for placing a virtual object in the virtual space based on the experiential space acquired by the first acquisition means and the at least one state of the second experiencer acquired by the second acquisition means, a second setting means for setting a virtual viewpoint of the first experiencer in the virtual space based on the at least one state of the first experiencer measured by the measurement means and the reference coordinate system set by the first setting means, and a generation means for generating an image of the virtual space including the virtual object based on the virtual viewpoint of the first experiencer set by the second setting means. [Effects of the Invention]

[0012] According to the present invention, a plurality of participants can observe a virtual object from the same viewpoint. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of an information processing system according to a first embodiment. [Figure 2] 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to a first embodiment. [Figure 3] FIG. 2 is a diagram for explaining processing performed by the information processing device according to the first embodiment. [Figure 4] FIG. 2 is a diagram for explaining processing performed by the information processing device according to the first embodiment. [Figure 5] 5 is a flowchart showing an example of a processing procedure of an information processing method performed by the information processing device according to the first embodiment. [Figure 6] 10 is a flowchart illustrating an example of a processing procedure of an information processing method performed by an information processing device according to a first modification of the first embodiment. [Figure 7] FIG. 10 is a diagram for explaining a process performed by an information processing device according to a third modification of the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a schematic configuration of an information processing system according to a second embodiment. [Figure 9] 10 is a flowchart showing an example of a processing procedure of an information processing method performed by an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0015] (First embodiment) First, the first embodiment will be described.

[0016] Fig. 1 is a diagram showing an example of a schematic configuration of an information processing system 10-1 according to the first embodiment. The above-described XR system can be applied to the information processing system 10-1 shown in Fig. 1. As shown in Fig. 1, the information processing system 10-1 includes an information processing device 100, a camera 210 which is an imaging device, and a display device 220.

[0017] The information processing device 100 is a device for generating virtual reality images and mixed reality images. As shown in FIG. 1, the information processing device 100 according to the first embodiment has the functional components of a first position measurement unit 101, an experiential space detection unit 102, a second position reception unit 103, a coordinate system setting unit 104, a virtual viewpoint setting unit 105, a scene generation unit 106, and an output unit 107. Furthermore, as necessary, the information processing device 100 has the functional components of an event detection unit 108 indicated by a dotted line block. Each functional component (101 to 108) of this information processing device will be described later.

[0018] In this embodiment, it is assumed that the camera 210 and the display device 220 are attached to a head-mounted display (HMD), but the present invention is not limited to this. Also, in this embodiment, it is assumed that the camera 210 and the display device 220 are provided in a stereo configuration with two cameras 210 and two display devices 220 in the HMD, but a single or multiple cameras 210 or display devices 220 may be used.

[0019] 2 is a diagram showing an example of a hardware configuration of the information processing device 100 according to the first embodiment. As shown in FIG. 2, the information processing device 100 has a hardware configuration including a CPU 111, a RAM 112, a ROM 113, an input I / F 114, an output I / F 115, and a bus 116.

[0020] The CPU 111 comprehensively controls the operation of the information processing device 100 and performs various processes. The RAM 112 is used as a work area when the CPU 111 performs processes while controlling each component. The RAM 112 also stores various information (including data) obtained by the CPU 111 performing various controls and processes. The ROM 113 stores control programs, various application programs, various information (including data), and the like. The input I / F 114 inputs an image from the camera 210 as an input signal in a format processable by the information processing device 100. The output I / F 115 outputs the video from the output unit 107 as an output signal in a format processable by the display device 220. The bus 116 connects the CPU 111, RAM 112, ROM 113, input I / F 114, and output I / F 115 so that they can communicate with each other.

[0021] In this embodiment, the CPU 111 loads a control program stored in the ROM 113 into the RAM 112 and executes it, thereby realizing the respective functional components (101 to 108) of the information processing device shown in Fig. 1. However, at least some of the functional components (101 to 108) of the information processing device shown in Fig. 1 may be realized by dedicated hardware or a GPU (not shown).

[0022] FIG. 3 is a diagram for explaining processing performed by the information processing device 100 according to the first embodiment. Specifically, FIG. 3(a) is a diagram illustrating the relationship between real space and a virtual space presented to a first user by the information processing device 100. FIG. 3(b) is a diagram illustrating a situation in which a second user is in a different location from the first user and shares the same virtual space as the first user. In FIG. 3, the virtual space shared by the first user and the second user is a space in which a virtual object 301 is placed with reference to a reference coordinate system 302. In this case, there may be multiple virtual objects 301 with reference to the reference coordinate system 302.

[0023] 3(a) and 3(b) show a virtual object 301 placed in a virtual space, and a reference coordinate system 302 that serves as a reference when placing the virtual object 301 in the virtual space.

[0024] In FIG. 3(a), a first experiential space 310 indicates a space in real space in which a first user can move. In FIG. 3(a), a viewpoint 311 indicates the position of the viewpoint (first position) in real space of the first user wearing the HMD. The information processing device 100 can present virtual reality images or mixed reality images to the first user, for example, by performing processing so that the reference coordinate system 302 is fixed to the first experiential space 310. Also, in FIG. 3(a), a viewpoint 312 indicates the position of the viewpoint (second position) of the second user in the virtual space.

[0025] In FIG. 3(b), a second experiential space 320 indicates a space in real space in which a second user can move. In FIG. 3(b), a viewpoint 322 indicates the position of the viewpoint in real space of the second user wearing the HMD. The information processing device 100 can present virtual reality images or mixed reality images to the second user, for example, by performing processing so that the reference coordinate system 302 is fixed to the second experiential space 320. In FIG. 3(b), a viewpoint 321 indicates the position of the viewpoint of the first user in the virtual space.

[0026] In this embodiment, the second user experiences a larger real space than the first user, or can operate the virtual viewpoint using a controller, gestures, or the like. Therefore, in this embodiment, the position (second position) of the viewpoint 312 of the second user in the virtual space can be freely moved around the virtual object 301. However, in this embodiment, it is sufficient to set the position (second position) of the viewpoint 312 of the second user in the virtual space based on the reference coordinate system 302, and the setting method is not limited to the above-mentioned method.

[0027] 3(a), if the virtual object 301 is larger than the first experiential space 310, the first experiencer may not be able to observe the virtual object 301 from the same viewpoint as the second experiencer simply by moving in the real space. In this embodiment, this problem is solved by, for example, moving the position of the reference coordinate system 302 in the real space according to the position (second position) of the viewpoint 312 of the second experiencer in the virtual space. Specific processing by the information processing device 100 for this purpose will be described with reference to FIG. 1.

[0028] The first position measurement unit 101 shown in FIG. 1 measures the position (first position) of the viewpoint 311 of the first user in real space using an image input from the camera 210. A specific method is, for example, a method called Visual SLAM as disclosed in Non-Patent Document 1. This method tracks the position and orientation of the camera 210 in real time by tracking changes in the appearance of the input image. Other methods may be used, such as detecting a marker placed in real space from the image and determining the position and orientation of the camera 210. Furthermore, the first position measurement unit 101 in this embodiment may measure the position (first position) of the viewpoint 311 of the first user in real space using a sensor other than the camera 210. For example, the first position may be measured using a motion capture sensor installed in real space and a marker attached to an object worn by the first user, or may be measured using an inertial sensor (Inertial Measurement Unit). Furthermore, the position (first position) of the viewpoint 311 of the first user in real space may be measured using these sensors in combination with the camera 210.

[0029] The first position measurement unit 101 in this embodiment measures at least the position (first position) of the first user's viewpoint 311 in real space, but in practice it is desirable to also measure the direction (posture) of the viewpoint 311. Therefore, in this embodiment, the first position measurement unit 101 measures the position (first position) and posture (first posture) of the first user's viewpoint 311 in real space, and the posture obtained here is also referenced in the virtual viewpoint setting unit 105, which will be described later, and the like.

[0030] The experiential space detection unit 102 shown in FIG. 1 detects and acquires a space in real space in which the first experiencer can move (a space in which the viewpoint 311 can be moved) as the first experiential space 310. Specifically, the experiential space detection unit 102 can automatically detect the first experiential space 310 by restoring the three-dimensional shape of the captured real space from an image captured by the camera 210. For example, the experiential space detection unit 102 first restores the shape of the real space around the first experiencer using a method such as Structure from Motion, and then estimates a floor surface without obstacles using planar estimation. Then, based on the estimation result, the experiential space detection unit 102 may detect a space at a certain height from the floor surface as the first experiential space 310 in which the experiencer can move. However, the method for detecting the first experiential space 310 in this embodiment is not limited to the method described above. For example, the first experiential space 310 may be detected based on detection information from a sensor capable of measuring depth, or the first experiential space 310 may be detected based on an instruction input from the first experiencer.

[0031] The second position receiving unit 103 shown in FIG. 1 receives and acquires the position (second position) of the viewpoint 312 of the second experiencer in the virtual space. In this embodiment, the second experiencer shares the same virtual space experience as the first experiencer. Furthermore, as shown in the first experience space 310 in FIG. 3(a) and the second experience space 320 in FIG. 3(b), the second experiencer can move his / her viewpoint in a wider range of the virtual space than the first experiencer. In this case, the second experiencer may be a user simultaneously experiencing an XR system (information processing system 10-1 in FIG. 1) connected to a network in a different location from the first experiencer, or may be an avatar of a user whose movements and voice are pre-recorded. For example, the second experiencer also wears an HMD like the first experiencer, and the information processing system 10-1 of the second experiencer measures the position of the viewpoint 322 of the second experiencer in the second experience space 320 using the method described for the first position measuring unit 101. Then, in the information processing system 10-1 of the second experiencer, the position of the viewpoint 322 is projected into the virtual space using the reference coordinate system 302 in the second experiential space 320, thereby calculating the position (second position) of the viewpoint 312 of the second experiencer in the virtual space. The position (second position) of the viewpoint 312 of the second experiencer calculated by the information processing system 10-1 of the second experiencer is transmitted to the information processing system 10-1 of the first experiencer, and received by the second position receiving unit 103 of the information processing system 10-1 of the first experiencer. However, in this embodiment, the method of receiving and acquiring the position (second position) of the viewpoint 312 of the second experiencer in the virtual space in the second position receiving unit 103 of the information processing system 10-1 of the first experiencer is not limited to the method described above.

[0032] 1 sets a reference coordinate system 302 in real space based on the first experiential space 310 detected by the experiential space detection unit 102 and the position (second position) of the viewpoint 312 of the second experiencer in the virtual space received by the second position receiving unit 103. The reference coordinate system 302 is a coordinate system that serves as a reference for rendering the virtual space including the virtual object 301. For example, the virtual object 301 and the position (second position) of the viewpoint 312 of the second experiencer in the virtual space are recorded as relative positions and orientations from the reference coordinate system 302.

[0033] Furthermore, the reference coordinate system 302 can be expressed as a transformation matrix M for projecting a point X in the real space onto a corresponding point Y in the virtual space. In three-dimensional space, it is expressed by the following equation (1).

number

[0034] Furthermore, by moving the position of the reference coordinate system 302 in the real space, it is possible to control where in the real space the virtual object 301 and the position (second position) of the second user's viewpoint 312 in the virtual space are projected. If the position (second position) of the second user's viewpoint 312 in the virtual space (as in FIG. 3(a)) is projected outside the first user's experience space 310, the first user cannot reach the second position just by moving in the real space. In other words, in this case, the first user will not be able to observe the virtual object 301 from the same viewpoint as the second user.

[0035] Therefore, in this embodiment, when the second position described above is located at a position projected outside the first experiential space 310, the coordinate system setting unit 104 moves and sets the reference coordinate system 302 so that the second position described above is projected inside the first experiential space 310. An example of this specific method will be described with reference to FIG. 4.

[0036] Fig. 4 is a diagram for explaining the processing performed by the information processing device 100 according to the first embodiment. In Fig. 4, the same components as those shown in Fig. 3 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0037] First, the coordinate system setting unit 104 uses the reference coordinate system 302 shown in FIG. 4(a) to calculate a coordinate point P (point 401 in FIG. 4(a)) obtained by projecting the position (second position) of the viewpoint 312 of the second experiencer in the virtual space onto the real space. Next, as shown in FIG. 4(a), if the coordinate point P (401) is outside the first experience space 310, the coordinate system setting unit 104 calculates a coordinate point Q (point 402 in FIG. 4(a)) of the closest point in the first experience space 310 to the coordinate point P (401). Furthermore, the coordinate system setting unit 104 calculates a movement vector (arrow 403 in FIG. 4(a)) V = QP, and moves the position of the reference coordinate system 302 according to the movement vector V. FIG. 4(b) is a diagram showing the state after the position of the reference coordinate system 302 has been moved according to the movement vector V (arrow 403 in FIG. 4(a)). As shown in Figure 4(b), when the position of the viewpoint 312 of the second user in the virtual space (second position) is projected onto the real space using the reference coordinate system 302 after the movement, it is located at coordinate point Q in the first experiential space 310. In the case shown in Figure 4(b), the first user can reach the position of coordinate point Q (402) simply by moving within the first experiential space 310, and therefore can observe the virtual object 301 from the same viewpoint as the second user.

[0038] On the other hand, when the coordinate point P(401) is inside the first experiential space 310, the coordinate system setting unit 104 does not move the position of the reference coordinate system 302. This allows the first user to observe the virtual object 301 from any viewpoint within the first experiential space 310. In this embodiment, for example, a constraint may be imposed so that the reference coordinate system 302 moves only in the horizontal direction. However, the method of moving the reference coordinate system 302 in this embodiment is not limited to this as long as the coordinate point P(401) shown in FIG. 4(a) is within the first experiential space 310. In addition, the coordinate system setting unit 104 may gradually move the reference coordinate system 302 using animation or the like to prevent the first user from suddenly moving the viewpoint.

[0039] Here, we return to the explanation of Figure 1 again. 1 sets the virtual viewpoint of the first experiencer in the virtual space based on the first position (and also the first orientation) of the first experiencer in the real space measured by the first position measurement unit 101 and the reference coordinate system 302 set by the coordinate system setting unit 104. Specifically, for example, the virtual viewpoint setting unit 105 projects the first position and first orientation of the first experiencer in the real space measured by the first position measurement unit 101 into the virtual space in accordance with the reference coordinate system 302, and sets the position and orientation of the virtual viewpoint of the first experiencer.

[0040] 1 renders a virtual space based on the position (and even posture) of the virtual viewpoint of the first user in the virtual space set by the virtual viewpoint setting unit 105, and generates an image of the virtual space including the virtual object 301 to be presented to the first user. Here, the image generated by the scene generation unit 106 may be a virtual reality image, or may be a mixed reality image generated by superimposing it with an image of the real space input from the camera 210 or the like.

[0041] The output unit 107 shown in FIG. 1 outputs the video generated by the scene generation unit 106 to the display device 220, and presents the video to the first experiencer.

[0042] FIG. 5 is a flowchart showing an example of a processing procedure of an information processing method by the information processing device 100 according to the first embodiment.

[0043] First, in step S101 of FIG. 5, the information processing device 100 (specifically, the first position measurement unit 101 and the experiential space detection unit 102) acquires an image captured by the camera 210.

[0044] Next, in step S102 of FIG. 5, the first position measurement unit 101 measures the position (first position) of the viewpoint 311 of the first experiencer in real space using the image from the camera 210 acquired in step S101. At this time, as described above, it is desirable that the first position measurement unit 101 also measure the orientation (first orientation) of the viewpoint 311 of the first experiencer in real space. In this case, for example, the first position measurement unit 101 measures the first position and first orientation of the first experiencer in real space by estimating the position and orientation of the HMD to which the camera 210 is attached using the image from the camera 210 acquired in step S101. If this first orientation is also measured, the first orientation is also taken into consideration in processing by the virtual viewpoint setting unit 105 in step S107 of FIG. 5, which will be described later.

[0045] 5, the experiential space detection unit 102 uses the image from the camera 210 acquired in step S101 to detect and acquire a space in real space in which the first experiencer can move as the first experiential space 310. Specifically, for example, the experiential space detection unit 102 detects the first experiential space 310 by restoring the three-dimensional shape of the real space around the first experiencer from the image from the camera 210 acquired in step S101.

[0046] Next, in step S104 of FIG. 5, the second position receiving unit 103 receives and acquires the position (second position) of the viewpoint 312 of the second experiencer in the virtual space.

[0047] 5, the coordinate system setting unit 104 determines whether the position (second position) of the viewpoint 312 of the second experiencer in the virtual space received in step S104 is outside the first experience space 310 detected in step S103. Specifically, in step S105, the coordinate system setting unit 104 projects the second position received in step S104 onto a position in the real space based on the reference coordinate system 302, and determines whether the projected position is outside the first experience space 310.

[0048] As a result of the determination in step S105 of FIG. 5, if the second position received in step S104 is outside the first experiential space 310 detected in step S103 (S105 / Yes), the process proceeds to step S106. When the process proceeds to step S106 in FIG. 5, the coordinate system setting unit 104 moves and sets the reference coordinate system 302 so that the second position described above is projected inside the first experiential space 310.

[0049] 5 is completed, the process proceeds to step S107. Also, as a result of the determination in step S105 in FIG. 5, if the second position received in step S104 is not outside the first experiential space 310 detected in step S103 (S105 / No), the process proceeds to step S107. Proceeding to step S107 in FIG. 5, the virtual viewpoint setting unit 105 sets the virtual viewpoint of the first user in the virtual space based on the first position of the first user measured in step S102 and the reference coordinate system 302 set by the coordinate system setting unit 104.

[0050] 5, the scene generation unit 106 renders the virtual space based on the virtual viewpoint of the first user in the virtual space set in step S107, and generates an image of the virtual space including the virtual object 301 to be presented to the first user. Thereafter, the output unit 107 outputs the image generated by the scene generation unit 106 to the display device 220, and presents the image to the first user.

[0051] 5, the information processing device 100 determines whether or not there is a request to end the process. If there is no request to end the process (S109 / No), the process returns to step S101 and repeats the process from step S101 onwards.

[0052] Also, as a result of the determination in step S109 in FIG. 5, if there is a request to end the process (S109 / Yes), the process of the flowchart shown in FIG. 5 ends.

[0053] The information processing device 100 according to the first embodiment described above has the following configuration. The information processing device 100 includes a first position measurement unit 101, which corresponds to a measurement means for measuring the position of a first experiencer in real space (and, if necessary, the posture, or at least one state of the position and posture). The information processing device 100 also includes an experiential space detection unit 102, which corresponds to a first acquisition means for acquiring a space in real space in which the first experiencer can move as an experiential space. The information processing device 100 also includes a second position receiving unit 103, which corresponds to a second acquisition means for acquiring the position of a second experiencer in virtual space (and, if necessary, the posture, or at least one state of the position and posture). The information processing device 100 also includes a coordinate system setting unit 104, which sets a reference coordinate system 302 for placing a virtual object 301 in the virtual space based on the experiential space acquired by the experiential space detection unit 102 and the position of the second experiencer acquired by the second position receiving unit 103. The coordinate system setting unit 104 for setting the reference coordinate system 302 corresponds to a first setting means. The information processing device 100 also includes a virtual viewpoint setting unit 105 that sets the virtual viewpoint of the first experiencer in the virtual space based on the position of the first experiencer measured by the first position measurement unit 101 and the reference coordinate system 302 set by the coordinate system setting unit 104. The virtual viewpoint setting unit 105 that sets the virtual viewpoint of the first experiencer corresponds to second setting means. The information processing device 100 also includes a scene generation unit 106 that corresponds to generation means that generates an image of the virtual space including the virtual object 301 based on the virtual viewpoint of the first experiencer set by the virtual viewpoint setting unit 105. The information processing device 100 also includes an output unit 107 that corresponds to output means that outputs the image generated by the scene generation unit 106 to the display device 220 and presents the image to the first experiencer. According to the configuration of the information processing device 100 according to the first embodiment, multiple users, namely, a first user and a second user, can observe the virtual object 301 from the same viewpoint. For example, even if the virtual object 301 is larger than the room in which the first user is located, the first user can observe the virtual object 301 from the same viewpoint as the second user who is located in a remote location, simply by moving within the room. Furthermore, even if multiple users use the XR system simultaneously in areas of different sizes, the multiple users can observe the virtual object 301 from the same viewpoint.

[0054] <Variation 1> Next, a first modification of the first embodiment will be described.

[0055] In the first embodiment described above, when the second position is located at a position projected outside the first experiential space 310, the coordinate system setting unit 104 moves the reference coordinate system 302 so that the second position is projected inside the first experiential space 310. However, depending on the situation in the virtual space, the relative position and orientation (hereinafter referred to as "alignment") between the virtual object 301 and the real object may be important. For example, there may be a case where it is desired to give the first user the sensation of sitting in the chair of the virtual object 301 by overlaying the chair of the virtual object 301 at the same position as the chair of the real object. In this case, if the alignment is shifted by moving the reference coordinate system 302, the first user may look at the chair of the virtual object 301 and try to sit in a place where the chair of the real object does not exist. Therefore, in variant example 1, the coordinate system setting unit 104 sets the reference coordinate system 302 to a predetermined alignment or sets the reference coordinate system 302 not to move while a predetermined event is occurring by the first experiencer.

[0056] In Modification 1, the predetermined event refers to an event that can be determined based on information obtained from the first position measurement unit 101 or the experiential space detection unit 102, an event that can be determined based on an image from the camera 210, an event that can be determined from information included in the virtual space, etc. For example, an event may be defined as an event in which a first experiencer enters a specific range in real space. This can be determined, for example, from the position of the first experiencer obtained by the first position measurement unit 101. Alternatively, an event may be defined as an event in which the first experiencer makes a seated motion. This can be determined, for example, by detecting a chair from the viewpoint of the first experiencer or from an image from the camera 210. Furthermore, an event that can be detected based on a positional relationship with a virtual object 301 included in the virtual space may be defined as an event. Here, information included in the virtual space includes information shared by a remote experiencer, information on operation inputs by the experiencer via gestures or a controller, etc.

[0057] The information processing device 100 according to the first modification of the first embodiment further includes an event detection unit 108, which is indicated by a dotted-line block in FIG. 1. The event detection unit 108 is a detection unit that detects the occurrence of an event predetermined by the first experiencer. The event detection unit 108 receives, as necessary, information obtained from the camera 210, the first position measurement unit 101, the experiential space detection unit 102, the second position reception unit 103, and the like, and detects the occurrence of the event predetermined by the first experiencer. The event detection result by the event detection unit 108 is then input to the coordinate system setting unit 104 shown in FIG. 1.

[0058] For example, when the virtual object 301 is a car, an event is defined in which the first user is near a real chair. When this event is being detected by the event detection unit 108, the coordinate system setting unit 104 sets the reference coordinate system 302 to a predetermined alignment so that, for example, the real chair and the seat of the virtual car coincide with each other. This provides the same effect as in the first embodiment, that is, when the first user is away from the real chair, the first user can observe the virtual car from the same viewpoint as the second user. At the same time, when the first user sits in the real chair, the first user can observe the virtual car from a viewpoint as if they were sitting in the car seat.

[0059] Furthermore, an event that can be determined from information included in the virtual space may be defined as, for example, an event in which the viewpoint in the virtual space of the first user is near the driver's seat of the car of virtual object 301. When this event is being detected by event detection unit 108, coordinate system setting unit 104 sets reference coordinate system 302 not to move, regardless of whether the position (second position) of viewpoint 312 of the second user in the virtual space is inside or outside first experiential space 310. This provides the same effect as in the first embodiment, that is, when the virtual viewpoint of the first user is away from the driver's seat of the car of virtual object 301, the first user can observe the virtual car from the same viewpoint as the second user. At the same time, when the virtual viewpoint of the first user is near the driver's seat of the car of virtual object 301, the reference coordinate system 302 does not move, and therefore the second user can observe the car of virtual object 301 regardless of the viewpoint position.

[0060] Fig. 6 is a flowchart showing an example of the processing procedure of an information processing method by the information processing device 100 according to Modification 1 of the first embodiment. In Fig. 6, the same processing steps as those shown in Fig. 5 are assigned the same step numbers, and detailed descriptions thereof will be omitted.

[0061] In the first modification, first, the same processes as steps S101 to S104 in FIG. 5 are performed. 6, the event detection unit 108 performs a process of detecting a predetermined event. At this time, the event detection unit 108 may use the information obtained in steps S101 to S104, or may use other information, for the process of detecting the event.

[0062] 6, the coordinate system setting unit 104 determines whether or not a predetermined event has been detected in step S201. If the result of this determination is that a predetermined event has not been detected in step S201 (S202 / No), the process proceeds to step S105, and the processes from step S105 onward are performed in the same manner as in the flowchart shown in FIG.

[0063] Also, as a result of the determination in step S202 in FIG. 6, if a predetermined event is detected in step S201 (S202 / Yes), the process proceeds to step S203. 6, the coordinate system setting unit 104 sets the reference coordinate system 302 to a predetermined alignment. Then, in step S107 of FIG. 6, the virtual viewpoint setting unit 105 sets the virtual viewpoint of the first user in the virtual space based on the first position of the first user measured in step S102 and the reference coordinate system 302 set in step S203. Thereafter, the processes from step S108 onwards are carried out in the same manner as in the flowchart shown in FIG. 5.

[0064] In the first modification, the coordinate system setting unit 104 may also adopt a configuration in which, when a predetermined event is detected in step S201 (S202 / Yes), the coordinate system setting unit 104 sets the reference coordinate system 302 not to move. When this configuration is adopted, the process proceeds to step S107 without performing the process of step S203 in FIG. 6.

[0065] <Variation 2> Next, a second modification of the first embodiment will be described.

[0066] In the first embodiment described above, the coordinate system setting unit 104 moves the position of the reference coordinate system 302 in real space. When viewed from the viewpoint of the first user, this appears to have moved the position of the virtual object 301 in real space. This may cause the first user to feel that the virtual object 301 has suddenly started moving, which may cause discomfort to the first user. To reduce this possibility, in the second modification, when the reference coordinate system 302 is moved (or is being moved) according to the second position, the scene generation unit 106 generates an image including a notification form that notifies the first user of this in the image viewed by the first user. Specific notification forms include at least one of a display that adds at least one of a message, an icon, and a confirmation button, and a display that changes the display style of the virtual object 301 so that the virtual object 301 is not noticeable. Examples of changing the display style of the virtual object 301 include displaying the virtual object 301 semi-transparently, or displaying only the edge portion of the virtual object 301.

[0067] To make it easier for the first user to recognize the notification, a time lag may be provided between the issuance of the notification and the start of movement of the reference coordinate system 302, or the system may wait for the confirmation button to be pressed. Furthermore, the notification method may be determined taking into consideration the magnitude of the impact on the field of view of the first user, such as the distance between the first user and the virtual object 301 and the extent to which the virtual object 301 occupies within the field of view of the first user. However, the notification method when moving the reference coordinate system 302 is not limited to the form described here.

[0068] The display for notifying the first participant may or may not be shared with a second participant who shares the same virtual space. The second participant may be notified differently from the first participant, for example, by an additional display that only notifies the first participant that the reference coordinate system 302 of the first participant has moved.

[0069] <Variation 3> Next, a third modification of the first embodiment will be described.

[0070] In the first embodiment described above, the coordinate system setting unit 104 sets the reference coordinate system 302 so that the coordinate point P (point 401 in FIG. 4(a)) indicating the position of the second user is located inside the first experiential space 310. In the first embodiment, the virtual object 301 moves in conjunction with the movement of the reference coordinate system 302, which may cause the virtual object 301 to overlap with the viewpoint of the first user. In this case, the first user may lose sight of the situation in which he or she is placed due to an obstruction of the view. Therefore, in the third modification, the reference coordinate system 302 is set by moving it in consideration of the fact that the viewpoint of the first user in the virtual space does not overlap with the virtual object 301.

[0071] Fig. 7 is a diagram for explaining the processing performed by the information processing device 100 according to Modification 3 of the first embodiment. In Fig. 7, the same components as those shown in Fig. 3 and Fig. 4 are denoted by the same reference numerals, and detailed description thereof will be omitted.

[0072] As shown in Figure 7(a), when the coordinate point P (point 701 in Figure 7(a)) obtained by projecting the second position described above onto real space is outside the first experiential space 310, the coordinate system setting unit 104 sets the reference coordinate system 302 by moving it according to the following procedure.

[0073] First, the coordinate system setting unit 104 calculates a path 702 starting from the virtual viewpoint of the first user (expressed as the same point as viewpoint 311 in FIG. 7(a)) and reaching coordinate point P (point 701 in FIG. 7(a)) while avoiding virtual object 301 as an obstacle. Here, the A* (A-star) search algorithm is a well-known method for finding the shortest path between two points while avoiding obstacles.

[0074] Next, the coordinate system setting unit 104 moves the reference coordinate system 302 little by little so that the virtual viewpoint of the first experiencer moves along this path 702. Then, as shown in FIG. 7(b), the coordinate system setting unit 104 finishes moving the reference coordinate system 302 when the coordinate point P (point 701 in FIG. 7(b)) enters the first experience space 310.

[0075] As described above, in the third modification, the coordinate system setting unit 104 sets the reference coordinate system 302 so that the virtual viewpoint of the first user and the virtual object 301 do not overlap.

[0076] This provides the effect that the first user can observe virtual object 301 from the same viewpoint as the second user without overlapping virtual object 301, simply by moving the remaining path within first experiential space 310. However, the method of moving reference coordinate system 302 in Modification 3 is not limited to the method described here.

[0077] Furthermore, depending on the situation, it may be impossible to obtain a path 702 that leads to coordinate point P associated with the second position while avoiding virtual object 301, which is an obstacle, or the path 702 may be long, resulting in a long time for moving reference coordinate system 302. In such cases, reference coordinate system 302 may be moved so that the viewpoint of the first user in the virtual space is located near coordinate point P associated with the second position. Since such a movement of reference coordinate system 302 may result in a large change in the viewpoint of the first user, a display may be displayed to notify the first user of the change in scene, such as by temporarily darkening the image generated by scene generation unit 106.

[0078] (Second embodiment) Next, a second embodiment will be described. In the following description of the second embodiment, matters common to the first embodiment will be omitted, and only matters different from the first embodiment will be described.

[0079] In the first embodiment described above, a first user can move within the first experiential space 310, and the first user's position (first position) is mainly measured by the first position measurement unit 101. However, the method of the first embodiment cannot be used in cases where the first user is unable to move because he or she is sitting in a chair, or where the user's position cannot be measured because the user is using an HMD that can measure only the user's posture. Therefore, in the second embodiment, a process of the information processing device 100 that enables the virtual object 301 to be observed from the same viewpoint as the second user even in such cases will be described.

[0080] FIG. 8 is a diagram showing an example of a schematic configuration of an information processing system 10-2 according to the second embodiment. The above-described XR system can be applied to the information processing system 10-2 shown in FIG. 8. In FIG. 8, the same components as those shown in FIG. 1 are denoted by the same reference numerals, and detailed description thereof will be omitted. As shown in FIG. 8, the information processing system 10-2 includes an information processing device 100, an attitude sensor 230, and a display device 220.

[0081] In the second embodiment, the orientation sensor 230 is attached to the HMD worn by the first user. For example, an inertial sensor may be used as the orientation sensor 230, but this embodiment is not limited to this inertial sensor, and multiple sensors may also be used in combination.

[0082] 8, the information processing device 100 according to the second embodiment has the functional configuration of a first orientation measurement unit 201, an experiential space setting unit 202, a second position and orientation receiving unit 203, a coordinate system setting unit 204, a virtual viewpoint setting unit 105, a scene generation unit 106, and an output unit 107. Furthermore, if necessary, the information processing device 100 according to the second embodiment may have the functional configuration of an event detection unit 108 indicated by the dotted line block in FIG.

[0083] 8 measures the orientation (orientation) of the viewpoint of the first user in real space using information input from the orientation sensor 230. Note that if the virtual viewpoint setting unit 105 does not receive input of the viewpoint position (first position) of the first user in real space, it sets the position of the virtual viewpoint of the first user to an arbitrary position. For example, the position of the virtual viewpoint of the first user is set to the origin position of the virtual space, a position where the virtual object 301 can be suitably observed, or near the viewpoint position of the second user received from the second position and orientation receiving unit 203.

[0084] The experiential space setting unit 202 shown in FIG. 8 sets, for example, a space surrounding the viewpoint of the first experiencer in the virtual space as a first experiential space 310.

[0085] A second position and orientation receiving unit 203 shown in FIG. 8 receives the position (second position) and orientation (second orientation) of the second user in the virtual space.

[0086] 8 first determines whether the second position is inside or outside the first experiential space 310 based on the first experiential space 310 set by the experiential space setting unit 202 and the second position received by the second position and orientation receiving unit 203. If the second position is outside the first experiential space 310, the coordinate system setting unit 204 moves and sets the reference coordinate system 302 so that the second position is inside the first experiential space 310. Note that in this embodiment, the initial position of the reference coordinate system 302 in real space is not necessary for movement calculations and may be set at any position.

[0087] 8 may set the reference coordinate system 302 by moving it so that the distance between the position of the second experiencer (second position) and the position of the virtual viewpoint of the first experiencer is less than a threshold value. Note that the distance calculation process here may be performed by projecting the position of the first experiencer in real space (first position) onto the virtual space, or by projecting the second position in the virtual space onto the real space.

[0088] Furthermore, it may be difficult for the first user to turn around if the first user is sitting in a chair, etc. To address this, the coordinate system setting unit 204 shown in Fig. 8 may set the reference coordinate system 302 by rotating it in accordance with the second posture so that the difference between the posture of the second user (second posture) and the direction of the virtual viewpoint (posture) of the first user is less than a threshold value.

[0089] Fig. 9 is a flowchart showing an example of the processing procedure of the information processing method by the information processing device 100 according to the second embodiment. In Fig. 9, the same processing steps as those shown in Fig. 5 and Fig. 6 are assigned the same step numbers, and detailed descriptions thereof will be omitted.

[0090] First, in step S301 of FIG. 9, the first orientation measurement unit 201 uses information input from the orientation sensor 230 to measure the orientation of the first user in real space.

[0091] Next, in step S302 of FIG. 9, the experiential space setting unit 202 sets, for example, a space surrounding the viewpoint of the first experiencer in the virtual space as the first experiential space 310.

[0092] Subsequently, in step S303 of FIG. 9, the second position and orientation receiving unit 203 receives the position (second position) and orientation (second orientation) of the second user in the virtual space.

[0093] Next, in step S304 of FIG. 9, the coordinate system setting unit 204 determines whether the position (second position) of the second experiencer in the virtual space received in step S303 is outside the first experience space 310 set in step S302.

[0094] As a result of the determination in step S304 of FIG. 9, if the second position received in step S303 is outside the first experiential space 310 set in step S302 (S304 / Yes), the process proceeds to step S305. When the process proceeds to step S305 in FIG. 9, the coordinate system setting unit 204 moves and sets the reference coordinate system 302 so that the second position described above is projected inside the first experiential space 310.

[0095] Also, if the result of the determination in step S304 in FIG. 9 is that the second position received in step S303 is not outside the first experiential space 310 set in step S302 (S304 / No), the process proceeds to step S306. When the process proceeds to step S306 in FIG. 9, the coordinate system setting unit 204 determines whether or not the posture difference, which is the difference between the second posture received in step S303 and the direction (posture) of the virtual viewpoint of the first user, is greater than a threshold value.

[0096] As a result of the determination in step S306 of FIG. 9, if the posture difference, which is the difference between the second posture received in step S303 and the direction (posture) of the virtual viewpoint of the first user, is greater than the threshold value (S306 / Yes), the process proceeds to step S307. When the process proceeds to step S307 in FIG. 9, the coordinate system setting unit 204 rotates and sets the reference coordinate system 302 so that the attitude difference is equal to or smaller than a threshold value (may be less than the threshold value).

[0097] 9, if the posture difference, which is the difference between the second posture received in step S303 and the orientation (posture) of the virtual viewpoint of the first user, is not greater than the threshold (S306 / No), the process proceeds to step S107. Furthermore, if the process of step S305 in FIG. 9 has ended, or if the process of step S307 in FIG. 9 has ended, the process proceeds to step S107. 9, the virtual viewpoint setting unit 105 sets the virtual viewpoint of the first user in the virtual space based on the first posture, etc. of the first user measured in step S301 and the reference coordinate system 302 set by the coordinate system setting unit 104. Thereafter, the processes from step S108 onwards are carried out in the same manner as in the flowchart shown in FIG.

[0098] The information processing device 100 according to the second embodiment described above has the following configuration. The information processing device 100 includes a first orientation measurement unit 201, which corresponds to a measurement means for measuring the orientation of a first experiencer in real space (and, if necessary, the position, or at least one state of the orientation and the position). The information processing device 100 also includes an experiential space setting unit 202, which corresponds to a first acquisition means for acquiring a space in real space in which the first experiencer can move as an experiential space. The information processing device 100 also includes a second position and orientation receiving unit 203, which corresponds to a second acquisition means for acquiring the position and orientation of a second experiencer in virtual space. The information processing device 100 also includes a coordinate system setting unit 204, which sets a reference coordinate system 302 for placing a virtual object 301 in virtual space, based on the experiential space acquired by the experiential space setting unit 202 and the position and orientation of the second experiencer acquired by the second position and orientation receiving unit 203. The coordinate system setting unit 204, which sets the reference coordinate system 302, corresponds to a first setting means. The information processing device 100 also includes a virtual viewpoint setting unit 105 that sets a virtual viewpoint of the first user in the virtual space based on the posture, etc., of the first user measured by the first posture measurement unit 201 and a reference coordinate system 302 set by the coordinate system setting unit 204. The virtual viewpoint setting unit 105 that sets the virtual viewpoint of the first user corresponds to second setting means. The information processing device 100 also includes a scene generation unit 106 that corresponds to generation means that generates an image of the virtual space including a virtual object 301 based on the virtual viewpoint of the first user set by the virtual viewpoint setting unit 105. The information processing device 100 also includes an output unit 107 that corresponds to output means that outputs the image generated by the scene generation unit 106 to the display device 220 and presents the image to the first user. According to the configuration of the information processing device 100 according to the second embodiment, similarly to the first embodiment described above, multiple participants, namely, a first participant and a second participant, can observe the virtual object 301 from the same viewpoint. Furthermore, even if the first participant is unable to move because he / she is sitting in a chair or is wearing an HMD that can measure only his / her posture, he / she can observe the virtual object 301 from the same viewpoint as the second participant.

[0099] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. This program and a computer-readable storage medium storing the program are included in the present invention.

[0100] It should be noted that the above-described embodiments of the present invention are merely illustrative examples of the implementation of the present invention, and the technical scope of the present invention should not be construed as being limited by these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0101] The disclosure of this embodiment includes the following configuration, method, and program. [Configuration 1] a measuring means for measuring at least one of the position and posture of the first user in real space; a first acquisition means for acquiring a space in the real space in which the first experiencer can move as an experience space; a second acquisition means for acquiring at least one of a position and a posture of a second user in the virtual space; a first setting means for setting a reference coordinate system for arranging a virtual object in the virtual space based on the experiential space acquired by the first acquisition means and the at least one state of the second experiencer acquired by the second acquisition means; a second setting means for setting a virtual viewpoint of the first experiencer in the virtual space based on the at least one state of the first experiencer measured by the measurement means and the reference coordinate system set by the first setting means; a generating means for generating an image of the virtual space including the virtual object based on the virtual viewpoint of the first user set by the second setting means; An information processing device comprising: [Configuration 2] The first setting means sets the reference coordinate system so that the position of the second experiencer is inside the experience space. 2. The information processing device according to configuration 1, [Configuration 3] The first setting means sets the reference coordinate system so that a distance between the position of the second experiencer and the position of the virtual viewpoint of the first experiencer is less than a threshold value. 2. The information processing device according to configuration 1, [Configuration 4] The first setting means sets the reference coordinate system so that a difference between the posture of the second experiencer and the posture of the virtual viewpoint of the first experiencer is less than a threshold value. 2. The information processing device according to configuration 1, [Configuration 5] The first setting means sets the reference coordinate system so that the virtual viewpoint of the first user does not overlap with the virtual object. 2. The information processing device according to configuration 1, [Configuration 6] further comprising a detection means for detecting that a predetermined event has occurred by the first experiencer, The first setting means sets the reference coordinate system not to move or sets the reference coordinate system to a predetermined alignment when the detection means detects that the event has occurred. 6. The information processing device according to any one of configurations 1 to 5. [Configuration 7] When the first setting means performs setting to move the reference coordinate system, the generating means generates the image including a notification form to notify the fact. 7. The information processing device according to any one of configurations 1 to 6. [Configuration 8] The notification form is at least one of a display in which at least one display element of a message, an icon, and a confirmation button is added, and a display in which the display style of the virtual object is changed. 8. The information processing device according to configuration 7. [Method 1] a measuring step of measuring at least one of a position and a posture of a first user in real space; a first acquisition step of acquiring a space in which the first experiencer can move in the real space as an experience space; a second acquisition step of acquiring at least one state of a position and a posture of a second user in the virtual space; a first setting step of setting a reference coordinate system for arranging a virtual object in the virtual space based on the experiential space acquired in the first acquisition step and the at least one state of the second experiencer acquired in the second acquisition step; a second setting step of setting a virtual viewpoint of the first user in the virtual space based on the at least one state of the first user measured in the measuring step and the reference coordinate system set in the first setting step; a generating step of generating an image of the virtual space including the virtual object based on the virtual viewpoint of the first user set in the second setting step; An information processing method comprising: [Program 1] A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 8. [Explanation of symbols]

[0102] 10: Information processing system, 100: Information processing device, 101: First position measurement unit, 102: Experience space detection unit, 103: Second position reception unit, 104: Coordinate system setting unit, 105: Virtual viewpoint setting unit, 106: Scene generation unit, 107: Output unit, 108: Event detection unit, 111: CPU, 112: RAM, 113: ROM, 114: Input I / F, 115: Output I / F, 116: Bus, 201: First orientation measurement unit, 202: Experience space setting unit, 203: Second position and orientation reception unit, 204: Coordinate system setting unit, 210: Camera, 220: Display device, 230: Orientation sensor, 301: Virtual object, 302: Reference coordinate system, 310: First experience space, 320: Second experience space

Claims

1. a measuring means for measuring at least one of the position and posture of the first user in real space; a first acquisition means for acquiring a space in the real space in which the first experiencer can move as an experience space; a second acquisition means for acquiring at least one of a position and a posture of a second user in the virtual space; a first setting means for setting a reference coordinate system for arranging a virtual object in the virtual space based on the experiential space acquired by the first acquisition means and the at least one state of the second experiencer acquired by the second acquisition means; a second setting means for setting a virtual viewpoint of the first experiencer in the virtual space based on the at least one state of the first experiencer measured by the measurement means and the reference coordinate system set by the first setting means; a generating means for generating an image of the virtual space including the virtual object based on the virtual viewpoint of the first user set by the second setting means; An information processing device comprising:

2. The first setting means sets the reference coordinate system so that the position of the second experiencer is inside the experience space.