Information processing device, program, and information processing system

Abstract

[Problem] To provide a technique capable of appropriately reducing positional misalignment of a user in a virtual space according to situations. [Solution] Provided is an information processing device provided with a control unit that: sets, as a shared object of a first user and a second user, a virtual object that is located within a first range based on the first user and the second user is located within a second range based on the virtual object in a virtual space; transmitting relative information indicating a relative position relationship between the shared object and the first user to a user terminal associated with the second user; and switching the setting of the shared object in association with a change in at least any one of the location of the first user, the location of the shared object, and the location of the second user in the virtual space.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, a program, and an information processing system. Background Art

[0002] In recent years, technologies that enable users to communicate with each other while sharing virtual objects in a virtual space, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), have been considered.

[0003] In such a technology, data such as the position, posture, or voice of each user is transmitted to other users sharing a virtual space. On the receiving side, a virtual object representing the user who transmitted the data is drawn at a specified position in the virtual space based on the received data. Performing such a drawing process on the transmitting and receiving sides enables communication between users in the virtual space.

[0004] For example, Patent Document 1 listed below discloses a technology for determining the display position of a virtual object shared by users based on the relative positional relationship between each user and a specific virtual object.

[0005] [Citation List]

[0006] [Patent Document]

[0007] [Patent Document 1]

[0008] U.S. Patent Application Publication No. 2014 / 368534 (Specification) Summary of the invention

[0009] Technical issues

[0010] However, when a virtual object representing a corresponding user is displayed in a virtual space, the position of each user drawn in the virtual space may be misplaced due to differences in the environment in which the position, posture, etc. of each user are measured, resulting in a significant lack of realism. In the technology disclosed in Patent Document 1, only the positional relationship between a specific virtual object and each user is used to determine the display position, and positional misplacement of the user relative to an object other than the specific virtual object may occur.

[0011] Therefore, the present disclosure provides a technology capable of appropriately reducing the positional misalignment of a user in a virtual space according to the situation.

[0012] [Solution to the problem]

[0013] In order to solve the above-mentioned problem, according to one aspect of the present disclosure, an information processing device is provided, including a control unit, which: sets the following virtual object as a shared object of a first user and a second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object; sends relative information indicating the relative position relationship between the shared object and the first user to a user terminal associated with the second user; and switches the setting of the shared object in response to a change in at least any one of the position of the first user, the position of the shared object, and the position of the second user in the virtual space.

[0014] According to the present disclosure, there is provided a program for causing a computer to function as a control unit, the control unit: setting a virtual object as a shared object for a first user and a second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object; sending relative information indicating the relative positional relationship between the shared object and the first user to a user terminal associated with the second user; and switching the setting of the shared object in response to a change in at least any one of the position of the first user in the virtual space, the position of the virtual object set as the shared object, and the position of the second user.

[0015] According to the present disclosure, an information processing system is provided, including: a first user terminal associated with a first user; a second user terminal associated with a second user; and an information processing device, including a control unit, the control unit: setting the following virtual object as a shared object for the first user and the second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object; sending relative information indicating the relative position relationship between the shared object and the first user to the second user terminal; and switching the setting of the shared object in response to a change in at least any one of the position of the first user in the virtual space, the position of the virtual object set as the shared object, and the position of the second user, wherein the second user terminal displays a user object representing the first user at a position in the virtual space calculated based on the relative information received from the information processing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] [ Figure 1 ] Figure 1 is an explanatory diagram showing an overview and a functional configuration example of an information processing system according to the present disclosure.

[0017] [ Figure 2 ] Figure 2 It is an explanatory diagram showing the positional displacement of the user avatar displayed in the virtual space.

[0018] [ Figure 3 ] Figure 3 2 is a block diagram showing the functions of the control unit 220 of the user terminal 20 in more detail.

[0019] [ Figure 4 ] Figure 4 2 is an explanatory diagram showing an example in which the shared object setting unit 225 sets and cancels a shared object.

[0020] [ Figure 5 ] Figure 5 2 is an explanatory diagram showing another example of setting and canceling a shared object by the shared object setting unit 225 .

[0021] [ Figure 6 ] Figure 6 2 is an explanatory diagram showing another example of setting and canceling a shared object by the shared object setting unit 225 .

[0022] [ Figure 7 ] Figure 7 2 is an explanatory diagram showing an example in which the shared object setting unit 225 switches a shared object.

[0023] [ Figure 8 ] Figure 8 2 is an explanatory diagram showing another example in which the shared object setting unit 225 switches the shared object.

[0024] [ Fig. 9 ] Fig. 9 2 is an explanatory diagram showing another example in which the shared object setting unit 225 switches the shared object.

[0025] [ Fig.10 ] Fig.10 2 is an explanatory diagram showing another example in which the shared object setting unit 225 switches the shared object.

[0026] [ Fig.11 ] Fig.11 is an explanatory diagram showing an example in which the shared object setting unit 225 sets a shared object for each of different other users.

[0027] [ Fig.12 ] Fig.12 is an explanatory diagram showing another example in which the shared object setting unit 225 sets a shared object for each of different other users.

[0028] [ Fig.13 ] Fig.13 is a sequence diagram illustrating an example of the operation of the information processing system according to an embodiment of the present disclosure.

[0029] [ Fig.14 ] Fig.14 is a first flowchart showing the operation flow of the position information acquisition process performed by the user terminal 20 .

[0030] [ Fig.15 ] Fig.15 is a second flowchart showing the operation flow of the position information acquisition process performed by the user terminal 20 .

[0031] [ Fig.16 ] Fig.16 : is a flowchart showing an example of the operation of a process in which the user terminal 20 displays an avatar based on received position information.

[0032] [ Fig.17 ] Fig.17 1 is a sequence diagram showing an example of the operation process of the information processing system when the server 10 executes the position information acquisition process, the shared object setting process, and the display control process.

[0033] [ Fig.18 ] Fig.18 : is a sequence diagram showing an example of the operation process of the information processing system when the user terminals 20 directly communicate with each other.

[0034] [ Fig.19 ] Fig.19 is a block diagram showing an example of the hardware configuration 90 . DETAILED DESCRIPTION

[0035] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In addition, in this specification and the drawings, components having substantially the same functional configuration will be denoted by the same reference numerals, and thus repeated description thereof will be omitted.

[0036] In the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by appending different numbers after the same reference numerals. However, in the case where it is not necessary to particularly distinguish each of the plurality of components having substantially the same functional configuration, each of the plurality of components is simply labeled with the same reference numeral.

[0037] The description will be given in the following order.

[0038] 1. Overview of an information processing system according to an embodiment of the present disclosure

[0039] 2. Functional Configuration Example

[0040] 2-1. Server 10

[0041] 2-2. User Terminal 20

[0042] 3. Operation examples

[0043] 4. Modify the example

[0044] 5. Hardware Configuration Example

[0045] 6. Supplement

[0046] <1. Overview of Information Processing System According to One Embodiment of the Present Disclosure>

[0047] The present disclosure relates to a technology that can appropriately reduce the position dislocation of users in a virtual space according to the situation. An application suitable for the present disclosure is a technology that uses VR or AR technology to display a two-dimensional or three-dimensional virtual space built in a computer or on a computer network to a user. Using such technology, the avatar of each user who allows the user to operate it is displayed in the virtual space, so that the user can operate their avatar to communicate with each other. Such a virtual space is sometimes referred to as a metaverse.

[0048] Figure 1 1 is an explanatory diagram showing an overview and a functional configuration example of an information processing system according to the present disclosure. Figure 1 As shown, the information processing system according to the present disclosure includes a server 10 and a user terminal 20. The server 10 and the user terminal 20 are configured to be able to communicate with each other via a network 5.

[0049] The user terminal 20 is a client terminal that senses the position or posture of the user U in the real space, and acquires position information indicating the position and posture of the user U in the virtual space based on the sensing data. The user terminal 20 receives the position information of other users U in the virtual space from the server 10, and performs display control processing on virtual objects (hereinafter also referred to as user objects or avatars) representing other users U based on the received position information. The user terminal 20 is implemented by an information processing terminal such as a personal computer or a smart phone. The information processing system according to the present disclosure includes a plurality of user terminals 20. Figure 1 , two user terminals 20 are shown as an example: a user terminal 20 a used by a user U1 and a user terminal 20 b used by a user U2 . However, the present disclosure is not limited to this example, and the information processing system may include three or more user terminals 20 .

[0050] like Figure 1As shown, the user terminal 20 is connected to the HMD 250 (which is a head mounted display (HMD) that can be worn on the head of the user U), the camera 240, the camera 241, and the camera 242. The user terminal 20 acquires position information indicating the position and posture of the user U in the virtual space based on the sensing data acquired by the HMD 250, the camera 240, the camera 241, and the camera 242. The acquired sensing data includes, for example, the angular velocity and acceleration of the head of the user U wearing the HMD 250, the first-person perspective motion image of the user U, and the motion image of the user U. The user terminal 20 transmits the acquired position information of the user U to the server 10.

[0051] The HMD 250 is an example of a display device that displays a virtual space according to the control of the control unit 220. The display device may be a CRT display device, a liquid crystal display (LCD), an OLED device, or may also be a TV device, a projector, a smartphone, a tablet terminal, a personal computer (PC), etc. instead of or in addition to the HMD.

[0052] HMD 250 is implemented by an optical transmissive display implementing AR technology or MR technology, which allows virtual objects as information about the virtual space to be superimposed on the real space, while the real space is directly visible to the user's eyes. In this case, HMD 250 can be a glasses-type terminal or a goggles-type terminal. HMD 250 can be implemented by using a non-transmissive display that covers the user's field of view with a display unit. In this case, the user terminal 20 can display virtual objects to the user U via HMD 250 using VR technology, which allows the user to view the virtual space in which a 3D model, etc. is placed from any viewpoint.

[0053] The number of cameras connected to each user terminal 20 is not limited to three, and the number of cameras can be appropriately set according to the number required to obtain the position information of the user U. The camera 240, the camera 241, and the camera 242 are examples of sensors for obtaining the position, posture, etc. of the user U. Each sensor may be, for example, a visible light camera, an infrared camera, a depth camera, or an event-based camera that outputs only a portion of a subject whose brightness value has changed. The sensor may be a distance sensor, an ultrasonic sensor, etc.

[0054] The server 10 is an information processing device having a function as a relay server and a function of transmitting the position information of each user received from the corresponding user terminal 20 to other user terminals 20. When each of the user terminals 20 receives the position information of other users U from the server 10, the user terminal 20 displays a user object representing the other user in the virtual space based on the position information. The user object may be a real person avatar generated based on a real person picture of the user, or may be a fictitious virtual object image such as a character. In the following description and drawings, for the sake of convenience of explanation, a virtual object is sometimes referred to as a virtual body.

[0055] (Question summary)

[0056] As described above, when users are displayed in a virtual space, the position of each user drawn in the virtual space may be misplaced due to differences in the environment in which the position, posture, etc. of each user are measured. When the distance between the user and the virtual object is far enough, the user is unlikely to notice the misalignment of the display position between the virtual object and the user. However, when the user is positioned close enough to the virtual object, the misalignment of the display position of the user will produce more discomfort. In particular, when there is a common virtual object at a position where users can view each other in the virtual space, the positional relationship between the virtual object and each user is highly perceptible. For example, in communication between users in the metaverse, they can have a conversation or discussion while sharing a virtual object. For example, there is a possible use case in which a virtual object of a product under development is shared between multiple users and communicated about the virtual object. There is also another possible use case in which multiple users sit side by side in a vehicle represented by a virtual object and go for a ride. In this case, the positional misalignment between the virtual object and each user will produce discomfort and communication barriers.

[0057] Figure 2 It is an explanatory diagram showing the positional displacement of the user avatar displayed in the virtual space. Figure 2 The virtual space V1 shown in the upper part of FIG. 1 includes a virtual object O1 representing a passenger car, a user object UO1, and a user object UO2. In the virtual space V1, two users, the user object UO1 and the user object UO2, exist inside the virtual object O1 representing the passenger car. The user object UO1 is Figure 1 The user object corresponding to the user U1 is shown. Similarly, the user object UO2 is the user object corresponding to the user U2.

[0058] In the virtual space V1, the user object UO1 is displayed at the position of the left front seat in the virtual object O1. On the other hand, the user object UO2, which should be displayed at the position of the right front seat in the virtual object O1, is displayed at a position shifted forward from the correct position of the seat and hidden in the instrument panel. Such misalignment of the display position may occur due to factors such as the distance measurement accuracy of a sensor (e.g., a camera device) for acquiring position information of the user U1 and the user U2, performance differences between HMDs worn by users, differences in installation positions between sensors, or differences in calibration accuracy.

[0059] Figure 2 The first-person perspective image FPV1 shown in the lower part of is an image viewed from the position of the user object UO1 shown in the virtual space V1 in the upper part toward the right front seat inside the virtual object O1, that is, a first-person perspective image of the user U1 in the virtual space. As shown in the first-person perspective image FPV1, when viewed by the user U1, the user object UO2 is displayed as shifted from its original ideal display position indicated by the dotted line L1 (the position of the right front seat inside the vehicle) to the position indicated by the dotted line L2 in front of the passenger car (virtual object O1). The user object UO2 may be outside the field of vision of the user U1, or the user object UO2 may be hidden in the dashboard on the front side of the virtual object O1, or may be displayed outside the virtual object O1. Such positional misalignment of the display position of the user object reduces the sense of reality in the virtual space, thereby causing discomfort and communication barriers.

[0060] In order to eliminate such positional misalignment, for example, Patent Document 1 discloses using the relative positional relationship between a specific virtual object and each user, but when there are multiple virtual objects, this is not effective for objects other than the specific virtual object. Figure 2 In the described example, if the virtual object O1 representing the passenger car is not a specific virtual object, there is a possibility that the misalignment of the display position of the user object UO2 cannot be eliminated.

[0061] Therefore, in one embodiment of the present disclosure, the positional dislocation of the user in the virtual space can be appropriately reduced according to the situation. More specifically, according to an embodiment of the present disclosure, the positional dislocation is reduced by using relative information indicating the relative positional relationship between the virtual object in the virtual space and each user as the display position of each user in the virtual space. Here, by appropriately switching the virtual object used as the base point of the relative positional relationship with each user to one of the multiple virtual objects in the virtual space that meets the predetermined conditions, the positional dislocation of the user can be appropriately reduced according to the situation. Such an embodiment of the present disclosure will be described in detail below.

[0062] <2. Functional Configuration Example>

[0063] <2-1. Server 10>

[0064] First, refer again Figure 1 , a functional configuration example of the server 10 according to the present embodiment will be described in detail. Figure 1 As shown, the server 10 includes a storage unit 110 , a control unit 120 , and a communication unit 130 .

[0065] (Storage Unit 110)

[0066] The storage unit 110 is a storage device capable of storing programs and data for operating the control unit 120. The storage unit 110 may also temporarily store various types of data required during the operation of the control unit 120. For example, the storage device may be a nonvolatile storage device.

[0067] (Control Unit 120)

[0068] The control unit 120 has a function of controlling the overall operation of the server 10. The control unit 120 includes a central processing unit (CPU) and the like, and implements its functions by loading a program stored in the storage unit 110 into a random access memory (RAM) and executing it through the CPU. In this case, a computer-readable recording medium having the program recorded therein may also be provided. Alternatively, the control unit 120 may be configured by dedicated hardware or by a combination of a plurality of hardware. The control unit 120 configured in this way controls the communication unit 130 described later to send the location information of the user U received from the user terminal 20 to other user terminals 20. In Figure 1 In the example shown, the control unit 120 causes the communication unit 130 to transmit the location information of the user U1 received from the user terminal 20a to the user terminal 20b. In addition, the control unit 120 causes the communication unit 130 to transmit the location information of the user U2 received from the user terminal 20b to the user terminal 20a.

[0069] (Communication unit 130)

[0070] The communication unit 130 communicates with the user terminal 20 via the network 5 under the control of the control unit 120. For example, the communication unit 130 transmits the location information of each user U received from the corresponding user terminal 20 to other user terminals 20 under the control of the control unit 120.

[0071] <2-2. User Terminal 20>

[0072] Next, a functional configuration example of the user terminal 20 according to the present embodiment will be described. Figure 1As shown, the user terminal 20 includes a storage unit 210, a control unit 220, and a communication unit 230. The user terminal 20 is communicatively connected to a camera 240, a camera 241, a camera 242, and an HMD 250. The user terminal 20, the camera 240, the camera 241, the camera 242, and the HMD 250 may be configured to communicate via a wired connection, or may be configured to communicate via wireless communication.

[0073] (Storage unit 210)

[0074] The storage unit 210 is a storage device capable of storing programs and data for operating the control unit 220. The storage unit 210 may also temporarily store various types of data required during the operation of the control unit 220. For example, the storage device may be a nonvolatile storage device.

[0075] (Control Unit 220)

[0076] The control unit 220 has a function of controlling the overall operation of the user terminal 20. The control unit 220 includes a central processing unit (CPU) and the like, and realizes its function by loading the program stored in the storage unit 210 into a random access memory (RAM) and executing it through the CPU. In this case, a computer-readable recording medium having the program recorded therein may also be provided. Alternatively, the control unit 220 may be configured by dedicated hardware or by a combination of multiple hardware.

[0077] The control unit 220 configured in this way acquires position information indicating the position and posture of the user U in the virtual space based on the sensing data of the user U. The control unit 220 controls the communication unit 230 to transmit the acquired position information of the user U to the server 10. Here, as described below, the control unit 220 has a function of appropriately generating position information to be transmitted to the server 10 according to the situation of the positional relationship between the user U, other users, and virtual objects in the virtual space.

[0078] More specifically, the control unit 220 may use the coordinates of the absolute position in the virtual space, or may use the relative position relationship with a specific virtual object, and may use relative information indicating the relative position relationship with a virtual object set as a shared object with other users U, as the position information of the user U. A shared object is a virtual object in the virtual space used as a base point for calculating relative information, and a virtual object that satisfies predetermined conditions regarding the positions of the user U and other users U is set as a shared object by the control unit 220. The control unit 220 may appropriately switch the virtual object set as a shared object in response to changes in circumstances such as movement of the user U. Thus, the user terminal 20 that receives the position information can reduce the misalignment of the display position of the user object UO representing the user U. This will be referred to later. Figures 3 to 12 The details of the process in which the control unit 220 configured in this way acquires the position information of the user U and sets the shared object will be described in more detail.

[0079] (Communication unit 230)

[0080] The communication unit 230 has a function of communicating with the server 10 under the control of the control unit 220. For example, the communication unit 230 transmits the location information of the user U to the server 10 under the control of the control unit 220. The communication unit 230 also receives the location information of each of the other users U from the server 10.

[0081] Reference above Figure 1 The functional configuration example of the user terminal 20 has been described. Figures 3 to 12 The process in which the control unit 220 of the user terminal 20 acquires the location information of the user U and sets a shared object is described in more detail.

[0082] Figure 3 2 is a block diagram showing the functions of the control unit 220 of the user terminal 20 in more detail. Figure 3 As shown, the control unit 220 has the functions of a sensor data acquisition unit 221 , a coordinate calculation unit 223 , a shared object setting unit 225 , a relative information calculation unit 227 , and a display control unit 229 .

[0083] The sensor data acquisition unit 221 has a function of acquiring sensing data from the camera 240, the camera 241, the camera 242, and the HMD 250. For example, the sensor data acquisition unit 221 acquires a moving image of the user U for calculating position information from the camera 240, the camera 241, and the camera 242. The sensor data acquisition unit 221 also acquires data such as the voice of the user U, the angular velocity or acceleration indicating the posture and orientation of the head of the user U from the HMD 250. Herein, three cameras are used as examples of sensors for acquiring sensing data of the user U, but as described above, each camera is an example of a sensor. The sensor may be, for example, a visible light camera or an infrared camera, or may be an event-based camera that outputs only a portion of a subject whose brightness value has changed.

[0084] The coordinate calculation unit 223 has a function of calculating the position of the user U in the virtual space based on the sensing data acquired by the sensor data acquisition unit 221. For example, the coordinate calculation unit 223 analyzes the motion images of the user U acquired from the camera 240, the camera 241, and the camera 242 to observe the motion of the user U in the real space, and estimates the position and posture of the user U in the real space. Specifically, the coordinate calculation unit 223 can estimate the 3D coordinates based on the 2D coordinates calculated from the plurality of motion images. At this time, the coordinate calculation unit 223 can also calculate the 3D coordinates indicating the position of the part of the body of the user U in association with the center position of the body of the user U. The algorithm of the method for calculating the position coordinates of the user U is not particularly limited. In this article, a motion image is used as an example, but the data is not limited thereto, and sensing data from various types of sensors attached to the user U can also be used. Then, the coordinate calculation unit 223 calculates the coordinates of the absolute position of the user U in the virtual space, or the relative position relationship between the user U and a specific virtual object as a preset reference based on the above-mentioned estimation results of the position and posture of the user U. The coordinate calculation unit 223 may calculate the position of the user U in the virtual space according to the operation input information (movement instruction) of the user, without being limited to the sensing data obtained by sensing the user.

[0085] The shared object setting unit 225 has a function of setting a shared object to be shared between the user U and other users U. More specifically, the shared object setting unit 225 identifies a virtual object that is located within a predetermined range (hereinafter, a first range) whose base point is the position of the user U1 in the virtual space calculated by the coordinate calculation unit 223, other users that are located within a predetermined range (hereinafter, a second range) whose base point is the virtual object, and sets the virtual object as a shared object of the user U and the other users U.

[0086] The shared object setting unit 225 also has a function of canceling the setting of a virtual object that has been set as a shared object. In addition, the shared object setting unit 225 has a function of switching a shared object by canceling the setting of a shared object and then setting a new different virtual object as a shared object. Figures 4 to 12 The process of setting, canceling and switching the shared object by the shared object setting unit 225 will be described in more detail. Figure 1 An example of acquiring position information of the user U1 in the virtual space by using the function of the shared object setting unit 225 included in the control unit 220a of the user terminal 20a is shown.

[0087] (Example 1 of setting and canceling a shared object)

[0088] Figure 4 2 is an explanatory diagram showing an example in which the shared object setting unit 225 sets and cancels a shared object. Figure 4 The virtual space V2 shown in the upper part of includes a user object UO1, a user object UO2, and a virtual object O2. Figure 4 The virtual space V3 shown in the lower part of is a virtual space in a state where the positional relationship among the user object UO1, the user object UO2, and the virtual object O2 is changed from the state of the virtual space V2 due to the movement of the user object UO1.

[0089] In the virtual space V2, the virtual object O2 is located between the user object UO1 and the user object UO2. In the process of setting the shared object, the shared object setting unit 225 of the user terminal 20 first identifies the virtual objects located within the first range whose base point is the user object UO1. Figure 4 In the virtual space V2 shown, a virtual object O2 located within the first range R1 is identified. Next, the shared object setting unit 225 of the user terminal 20 sets the following virtual objects located within the identified first range R1 as shared objects: other users other than the user object UO1 exist within the second range whose base point is the virtual object. In the virtual space V2, since the user object UO2 exists within the second range R2 whose base point is the virtual object O2, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2.

[0090] When the shared object is set by the shared object setting unit 225, the relative information calculation unit 227 of the user terminal 20 calculates relative information indicating the relative position relationship between the shared object and the user object UO1. The relative information also includes identification information of the virtual object set as the shared object. Next, the communication unit 230 sends the calculated relative information as the position information of the user U to the server 10 under the control of the control unit 220.

[0091] Next, assume that the state of the virtual space V2 changes to the state of the virtual space V3. Figure 4 As shown, in the virtual space V3, the user object UO1 moves in a direction away from the user object UO2 and the virtual object O2. In addition, due to the movement of the user object UO1, the virtual object O2 is located outside the first range R1. In this case, since the virtual object located within the first range R1 whose base point is the user object UO1 is no longer detected, the shared object setting unit 225 cancels the setting of the shared object for the virtual object O2.

[0092] In the case where the shared object setting unit 225 cancels the setting of the shared object and no other virtual object is set as a shared object, the control unit 220 of the user terminal 20 sets the coordinates of the absolute position of the user U in the virtual space calculated by the coordinate calculation unit 223, or the relative position with respect to a specific virtual object, as the position information of the user U to be sent to the server 10. The communication unit 230 sends the above position information to the server 10 under the control of the control unit 220.

[0093] An example of setting and canceling a shared object by the shared object setting unit 225 has been described above. Therefore, in an information processing system according to the contents of the present disclosure, a relative position relative to a virtual object (shared object) to be shared with other users U is sent as position information of each user U in a virtual space. At this time, virtual objects within a predetermined range, i.e., a first range, whose base point is user U, can also be set as shared objects. Thus, only virtual objects relatively close to user U, rather than virtual objects too far from user U, are used as base points for relative positions relative to user U, thereby being able to more effectively reduce the misalignment of the display position of user U.

[0094] In addition, in the information processing system according to the contents of the present disclosure, only when other users U exist within the second range whose base point is the virtual object, the virtual object located in the first range is set as a shared object of user U and other users U. Thus, only when other users U exist relatively close to the target virtual object, the position information of user U sent to other users U becomes relative information. When any other users U do not exist within the second range, the coordinates of the absolute position of user U or the relative position relative to a specific virtual object are sent to other users U as position information. Therefore, when viewed from the perspective of other users U, the user object representing user U is displayed based on the relative position of user U relative to the virtual object closer to other users U, thereby more effectively reducing the misalignment of the display position of user U.

[0095] (Example 2 of setting and canceling a shared object)

[0096] Next, we will refer to Figure 5 Another example in which the shared object setting unit 225 sets and cancels a shared object is described. Figure 5 2 is an explanatory diagram showing another example of setting and canceling a shared object by the shared object setting unit 225 . Figure 5 The virtual space V2 shown in the upper part is as shown in FIG. Figure 4 As described in 6, the description is omitted here. Figure 5 The virtual space V4 shown in the lower part of is a virtual space in a state where the positional relationship between the user object UO1, the user object UO2, and the virtual object O2 is changed from the state of the virtual space V2 due to the movement of the position of the virtual object O2.

[0097] As reference Figure 4 As described, in the state of the virtual space V2, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2.

[0098] Next, assume that the state changes from virtual space V2 to virtual space V4. Figure 5 As shown, in the virtual space V4, the position of the virtual object O2 changes from the position within the first range R1 to the position outside the first range R1. In this case, since any virtual object located within the first range R1 whose base point is the user object UO1 is no longer detected, the shared object setting unit 225 cancels the setting of the shared object for the virtual object O2.

[0099] Another example in which the shared object setting unit 225 sets and cancels a shared object has been described above.

[0100] (Example 3 of setting and canceling a shared object)

[0101] Next, we will refer to Figure 6 Another example in which the shared object setting unit 225 sets and cancels a shared object is described. Figure 6 2 is an explanatory diagram showing another example of setting and canceling a shared object by the shared object setting unit 225 . Figure 6 The virtual space V2 shown in the upper part is as shown in FIG. Figure 4 described, so the description is omitted here. Figure 6 The virtual space V5 shown in the lower part is a virtual space in a state where the positional relationship among the user object UO1, the user object UO2, and the virtual object O2 is changed from the state of the virtual space V2 due to the movement of the position of the user object UO2.

[0102] As reference Figure 4 As described, in the state of the virtual space V2, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2.

[0103] Next, assume that the state changes from virtual space V2 to virtual space V5. Figure 6 As shown, in the virtual space V5, the position of the user object UO2 changes to a position outside the second range R2. In this case, since any other user located within the second range R2 whose base point is the virtual object O2 is no longer detected, the shared object setting unit 225 cancels the setting of the shared object for the virtual object O2.

[0104] Another example in which the shared object setting unit 225 sets and cancels a shared object has been described above.

[0105] (Example 1 of setting and switching shared objects)

[0106] Next, we will refer to Figures 7 to 10 The process of switching the shared object by the shared object setting unit 225 is described in detail. After setting a shared object for a certain virtual object, the shared object setting unit 225 cancels the setting of the shared object for the virtual object, and sets a new virtual object as the shared object, thereby switching the setting of the shared object. The cancellation of the shared object and the setting of the new shared object can be performed sequentially or simultaneously. In the case where there are multiple virtual objects that can each be set as a shared object within the first range whose base point is the user U, the shared object setting unit 225 selects a virtual object and sets it as the shared object. At this time, the shared object setting unit 225 can select a virtual object that is closest to the user U from the multiple virtual objects.

[0107] Figure 7 2 is an explanatory diagram showing an example in which the shared object setting unit 225 switches the shared object. Figure 7In the illustrated example, setting of a shared object by the shared object setting unit 225 in a case where a plurality of virtual objects are located within the first range R1 whose base point is the user object UO1 will be described.

[0108] Figure 7 The virtual space V6 shown in the upper part of indicates a virtual space including the user object UO1 , the user object UO2 , the virtual object O2 , and the virtual object O3 . Figure 7 The virtual space V7 shown in the lower part is a virtual space in which the positional relationship among the user object UO1, the user object UO2, the virtual object O2, and the virtual object O3 is changed from the state of the virtual space V6 due to the movement of the position of the virtual object O2. In the virtual space V6, the virtual object O2 is located closer to the user object UO1 than the virtual object O3.

[0109] like Figure 7 As shown, in the virtual space V6, multiple virtual objects including virtual object O2 and virtual object O3 are located in a first range R1 whose base point is user object UO1. User object UO2 exists in a second range R2 whose base point is virtual object O2. User object UO2 also exists in a second range R3 whose base point is virtual object O3.

[0110] In the state of the virtual space V6, the shared object setting unit 225 of the user terminal 20 first identifies virtual objects located within a first range whose base point is the user object UO1. Figure 7 In the example of the virtual space V6 shown, multiple virtual objects, namely virtual object O2 and virtual object O3, are identified. In the case where multiple virtual objects are located within a first range R1 whose base point is the user object UO1, and other users exist within any second range of a second range whose base point is each virtual object, the shared object setting unit 225 selects one virtual object from the multiple virtual objects and sets it as a shared object. At this time, the shared object setting unit 225 can select the virtual object closest to the user object UO1 from the multiple virtual objects. Figure 7 In the example shown in the virtual space V6 in FIG. 2 , the shared object setting unit 225 sets the virtual object O2 , which is the virtual object closest to the user object UO1 among the virtual objects O2 and O3 , as the shared object.

[0111] Next, assume that the state changes from virtual space V6 to virtual space V7. Figure 7As shown, in the virtual space V7, the position of the virtual object O2 has moved outside the first range R1. In this case, the shared object setting unit 225 cancels the setting of the shared object for the virtual object O2 in response to the change in the position of the virtual object O2 that has been set as a shared object and in response to the virtual object O2 being outside the first range R1. In addition, in the virtual space V7, since the virtual object O3, which is a virtual object other than the virtual object O2, is located within the first range R1, and the user object UO2 exists in the second range R3 whose base point is the virtual object O3, the shared object setting unit 225 sets the virtual object O3 as a shared object. In this way, the shared object setting unit 225 switches the shared object.

[0112] As described above, the shared object setting unit 225 switches the virtual object set as the shared object in response to the change in the positional relationship between the virtual object set as the shared object and the user object UO1 and the user object UO2. With this configuration, even if the positional relationship between the user U, other users U, and the virtual object set as the shared object changes, the relative information about the new virtual object closest to the user U is sent from the user terminal 20 to the server 10 as the positional information of the user U. Therefore, even if the positional relationship between the user U, other users U, and the shared object changes in the virtual space, the misalignment of the display position of the user U viewed by other users on the position information receiving side can be more effectively reduced.

[0113] In addition, in the case where a plurality of virtual objects are located within the first range, and the same other user U exists within the second range whose base point is each virtual object, the shared object setting unit 225 selects one virtual object from the plurality of virtual objects and sets it as a shared object. At this time, the shared object setting unit 225 selects the virtual object closest to the user U. Thus, the relative information about the user U becomes the information with the virtual object closest to the user U as the base point, and therefore, compared with the case where the relative information with respect to the virtual object located at a position far from the user U is used as the position information of the user U, the misalignment of the display position of the user U can be further reduced.

[0114] (Example 2 of setting and switching shared objects)

[0115] Figure 8 2 is an explanatory diagram showing another example in which the shared object setting unit 225 switches the shared object. Figure 8 The virtual space V8 in the upper part includes a user object UO1, a user object UO2, a user object UO3, a virtual object O2, and a virtual object O3.

[0116] exist Figure 8In the example shown, in the virtual space V8, the virtual object O2 is located within the first range R1 whose base point is the user object UO1. The user object UO2 exists within the second range R2 whose base point is the virtual object O2. On the other hand, the virtual object O3 is located outside the first range R1. In this case, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2.

[0117] Next, suppose the state of the virtual space V8 changes to Figure 8 The state of the virtual space V9 is shown in the lower part. Figure 8 As shown, in the virtual space V9, due to the change in the position of the user object UO1, the virtual object O2 is located outside the first range R1. In addition, in the virtual space V9, the virtual object O3 is located within the first range R1. The user object UO3 exists in the second range R3 whose base point is the virtual object O3.

[0118] The shared object setting unit 225 cancels the setting of the shared object in response to the change in the position of the user object UO1 and in response to the position of the virtual object O2 set as the shared object being outside the first range R1. In addition, with respect to the shared object setting unit 225, since in the virtual space V9, the virtual object O3 is located within the first range R1, and the user object UO3 exists within the second range R3 whose base point is the virtual object O3, the shared object setting unit 225 sets the virtual object O3 as the shared object.

[0119] (Example 3 of setting and switching shared objects)

[0120] Fig. 9 2 is an explanatory diagram showing another example in which the shared object setting unit 225 switches the shared object. Fig. 9 The virtual space V10 in the upper portion includes a user object UO1 , a user object UO2 , a virtual object O2 , and a virtual object O3 .

[0121] exist Fig. 9 In the example shown, it can be seen that in the virtual space V10, the virtual object O2 and the virtual object O3 are located within the first range R1 whose base point is the user object UO1. It can also be seen that the user object UO2 exists within the second range R2 whose base point is the virtual object O2. In this case, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2.

[0122] Next, suppose the state of the virtual space V10 changes to Fig. 9 The state of the virtual space V11 is shown in the lower part. Fig. 9As shown, it can be seen that the position of the user object UO2 has changed in the virtual space V11. In addition, it can be seen that due to the change in the position of the user object UO2, the user object UO2 is now located in the second range R3 whose base point is the virtual object O3.

[0123] Due to the change in the position of the user object UO2, the shared object setting unit 225 cancels the setting of the shared object for the virtual object O2 in response to the position of the user object UO2 being outside the second range R2 whose base point is the virtual object O2 (which is set as a shared object). In addition, the shared object setting unit 225 newly sets the following virtual object O3 as a shared object: the virtual object O3 is located in the first range whose base point is the user object UO1 in the virtual space V11, and the user object UO2 exists in the second range R3 whose base point is the virtual object O3.

[0124] (Example 4 of setting and switching shared objects)

[0125] Fig.10 2 is an explanatory diagram showing another example in which the shared object setting unit 225 switches the shared object. Fig.10 The virtual space V12 in the upper portion includes a user object UO1 , a user object UO2 , a virtual object O2 , and a virtual object O3 .

[0126] exist Fig.10 In the example shown, in the virtual space V12, the virtual object O2 is located within a first range R1 whose base point is the user object UO1. It can also be seen that the user object UO2 exists within a second range R2 whose base point is the virtual object O2. In this case, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2.

[0127] Next, it is assumed that the state of the virtual space V12 is changed to the state of the virtual space V13. Fig.10 As shown, the position of the user object UO1 has changed in the virtual space V13. In addition, due to the change in the position of the user object UO1, the virtual object O2 is located outside the first range R1. On the other hand, the virtual object O3 is located within the first range R1. The user object UO2 is located within the second range R3 whose base point is the virtual object O3.

[0128] The shared object setting unit 225 cancels the setting of the shared object for the virtual object O2 in response to the change in the position of the user object UO1 and in response to the position of the virtual object O2 set as the shared object being outside the first range R1. In addition, the shared object setting unit 225 sets the following virtual object O3 as the shared object: the virtual object O3 is located in the first range R1 whose base point is the user object UO1 in the virtual space V13, and the user object UO2 exists in the second range R3 whose base point is the virtual object O3.

[0129] (Example 1 of setting shared objects for each of the different other users)

[0130] Fig.11 is an explanatory diagram showing an example in which the shared object setting unit 225 sets a shared object for each of different other users. Fig.11 The illustrated virtual space V14 includes a user object UO1 , a user object UO2 , a user object UO3 , a virtual object O2 , and a virtual object O4 .

[0131] exist Fig.11 In the example shown, in the virtual space V14, virtual objects O2 and O4 are located in a first range R1 whose base point is the user object UO1. User object UO2 exists in a second range R2 whose base point is the virtual object O2. User object UO3 exists in a second range R3 whose base point is the virtual object O4.

[0132] In this case, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2. The shared object setting unit 225 also sets the virtual object O4 as a shared object of the user object UO1 and the user object UO3.

[0133] In this way, the shared object setting unit 225 according to the present embodiment sets each virtual object as a shared object to be shared between the user object UO1 and each of the other users for one or more virtual objects located within the first range R1 based on the user object UO1. With this configuration, a shared object as a reference for the relative position of the user object UO1 is individually set for each of the different users. Therefore, it is possible to reduce the misalignment of the display position of the user object UO1 viewed by each of the other users.

[0134] (Example 2 of setting shared objects for each of the different other users)

[0135] Fig.12 is an explanatory diagram showing another example in which the shared object setting unit 225 sets a shared object for each of different other users. Fig.12The illustrated virtual space V15 includes a user object UO1 , a user object UO2 , a user object UO3 , and a virtual object O2 .

[0136] It can be seen that in the virtual space V15, the virtual object O2 is located in the first range R1 whose base point is the user object UO1. It can also be seen that the user objects UO2 and UO3 are located in the second range R2 whose base point is the virtual object O2.

[0137] In this case, the shared object setting unit 225 sets the virtual object O2 as a shared object of the user object UO1 and the user object UO2. The shared object setting unit 225 also sets the virtual object O2 as a shared object of the user object UO1 and the user object UO3. In this way, in the case where a plurality of other users exist within the second range of the virtual object, the shared object setting unit 225 according to the present embodiment sets the virtual object as a shared object to be shared between the user object UO1 and the other users.

[0138] In this way, the shared object setting unit 225 according to the present embodiment can set the same virtual object as a shared object for multiple other users according to the positional relationship between each of the user U and other users and the virtual object, or can set different virtual objects for each user.

[0139] Reference above Figures 4 to 12 The details of the process of setting, canceling, and switching a shared object by the shared object setting unit 225 are described.

[0140] Back to Figure 3 , the functions of the control unit 220 of the user terminal 20 will be described below. The relative information calculation unit 227 has the following functions: when a shared object is set by the shared object setting unit 225, calculates relative information indicating the relative positional relationship between the shared object and the user U. More specifically, the relative information calculation unit 227 takes the shared object as a base point and calculates the coordinates of the user U in the virtual space with the shared object as the origin as relative information. The relative information also includes identification information of the virtual object set as the shared object.

[0141] In the case where the shared object setting unit 225 has set a shared object, the control unit 220 sets the relative information about the shared object as a base point as the position information to be sent to the other user terminals 20. In the case where the shared object setting unit 225 has not set a shared object or has canceled the setting of the shared object, the control unit 220 sets the coordinates of the absolute position of the user U in the virtual space calculated by the coordinate calculation unit 223 or the relative position of the user U relative to a specific virtual object set in advance as a reference as the position information to be sent to the other user terminals 20. Alternatively, the control unit 220 sets the relative position of a specific virtual body in the virtual space relative to the user U as the position information to be sent to the other user terminals 20.

[0142] The display control unit 229 performs display control of the display unit of the HMD 250. For example, the display control unit 229 has a function of controlling the generation and display of a first-person perspective image of the user U in the virtual space. At this time, the display control unit 229 displays the user object of the other user U at the position in the virtual space indicated by the position information of the other user U received from the other user terminal 20. The display device that displays the image of the virtual space is not limited to the HMD 250, and may also be other display devices. For example, the display device may be a CRT display device, a liquid crystal display (LCD) or an OLED device, or may be a TV device, a projector, a smart phone, a tablet terminal, a PC, etc.

[0143] <3. Operation examples>

[0144] Next, we will refer to Figures 13 to 16 An example of the operation of the information processing system according to the present embodiment is described.

[0145] Fig.13 is a sequence diagram showing an example of the operation of an information processing system according to an embodiment of the present disclosure. The information processing system repeats at a predetermined update interval Fig.13 A series of operation processing is shown.

[0146] First, the user terminal 20a performs position information acquisition processing (S103). Herein, the position information acquisition processing performed by the user terminal 20 refers to a process of acquiring position information indicating the position of the user U wearing the HMD 250 connected to the user terminal 20 in the virtual space.

[0147] Next, the user terminal 20b similarly performs the position information acquisition process (S106). The processes of S103 and S106 may also be performed separately at timings independent of each other. The user terminal 20a and the user terminal 20b may continue to perform the processes of S103 and S106, respectively.

[0148] Next, the user terminal 20a sends the acquired location information to the server 10 (S109). The server 10 sends the location information received from the user terminal 20a to the user terminal 20b (S112). Similarly, the user terminal 20b sends the acquired location information of the user U2 to the server 10 (S115). The server 10 sends the location information of the user U2 received from the user terminal 20b to the user terminal 20a (S118).

[0149] Next, the user terminal 20a displays a user avatar (user object) representing the user U2 in the virtual space displayed on the HMD 250a based on the position information of the user U2 received from the server 10 (S121). Similarly, the user terminal 20b displays a user avatar of the user U1 in the virtual space displayed on the HMD 250b based on the position information of the user U1 received from the server 10 (S124).

[0150] Referenced above Fig.13 An example of the operation of the information processing system according to the present embodiment is described. Fig.13 The operation of the information processing system described above can update the position of each user in the virtual space in real time. Fig.14 and Fig.15 describe Fig.13 The processing flow of the position information acquisition processing in S103 and S106 in the sequence diagram of FIG.

[0151] Fig.14 20 is a first flowchart showing the operation flow of the position information acquisition processing performed by the user terminal 20. First, the control unit 220 of the user terminal 20 determines whether there is any virtual object located within the first range whose base point is the position of the user U in the virtual space (S203). In the case where there is no virtual object within the first range (S203 / No), the control unit 220 sets the coordinates of the absolute position of the user U, or the relative position of the user U relative to a specific virtual body predetermined as a reference, as the position information to be sent to the server 10 (S209), and then ends the position information acquisition processing.

[0152] On the other hand, if there is a virtual object in the first range (S203 / Yes), the control unit 220 determines whether there is another user U in any second range of the second range whose base point is each virtual object located in the first range (S206). If there is no other user in any second range of the second range (S206 / No), the process proceeds to S209.

[0153] If there are other users in any of the second ranges (S206 / Yes), the process proceeds to Fig.15 S212 in the flowchart shown.

[0154] Fig.15 2 is a second flowchart showing the operation flow of the position information acquisition process performed by the user terminal 20. The relative information calculation unit 227 acquires relative information about virtual objects located within a first range whose base point is the user U (S212). The relative information includes identification information for uniquely identifying each virtual object and a relative position indicating a relative positional relationship between each virtual object and the user U.

[0155] The relative information calculation unit 227 repeats the process of S212 until the relative information about all virtual objects located within the first range is obtained (S215 / No). When the relative information calculation unit 227 obtains the relative information about all virtual objects located within the first range (S215 / Yes), the process proceeds to S218.

[0156] Next, the control unit 220 obtains user information of other users U in each second range of the second range whose base point is all virtual objects located in the first range (S218). The user information is information for uniquely identifying each user. For example, the user information may be a user ID.

[0157] The control unit 220 repeats the process of S218 until the user information of all other users U in the second range is acquired (S221 / No). When the control unit 220 acquires the user information of all other users U in the second range (S221 / Yes), the process proceeds to S224.

[0158] Next, the shared object setting unit 225 sets each of all virtual objects located within the first range whose relative information is acquired in S215 as a shared object of the user U and other users U existing within the second range whose base point is the corresponding virtual object ( S224 ).

[0159] Next, the control unit 220 sets each relative information about the shared object as the base point of each of the user U and other users as the position information to be transmitted to each of the other users (S227), and the process returns to Fig.14 , ending the location information acquisition process.

[0160] Referenced above Fig.14 and Fig.15 Describes Fig.13 This is an example of the operation of the position information acquisition process performed by each of the user terminals 20 in S103 and S106 in FIG.

[0161] Next, we will refer to Fig.16 describe Fig.13 The operation flow of the avatar display based on the received position information in S121 and S124 is shown. Fig.16 : is a flowchart showing an example of the operation of a process in which the user terminal 20 displays an avatar based on received position information.

[0162] First, the communication unit 230 of the user terminal 20 receives the position information of the other user U (also referred to as the counterpart user) from the server 10. If the received position information is relative information about the other user U and the virtual object set as a shared object (S303 / Yes), the display control unit 229 of the user terminal 20 displays the counterpart user's avatar at a relative position relative to the shared object set on the other user U side based on the relative information (S306). The relative information includes identification information of the virtual object set as a shared object on the other user U side and relative position information indicating the positional relationship of the other user U relative to the shared object.

[0163] On the other hand, if the received position information is not relative information (S303 / No), specifically, if the received position information is an absolute position in the virtual space or a relative position relative to a specific virtual body pre-defined as a reference, the display control unit 229 displays the avatar of the other user U at the received coordinates of the absolute position of the other user U or at the relative position relative to the specific virtual body (S309).

[0164] <4. Modification example>

[0165] An example of the functional configuration and an example of the operation of the information processing system according to an embodiment of the present disclosure have been described above. In the above embodiment, the relative information calculation unit 227 of the user terminal 20 obtains the relative position relationship between the position of the user object UO1 and the shared object as the relative position. The present disclosure is not limited to this, and the relative position relationship between the position of each part of the body of the user object UO1 and the shared object may also be obtained as the relative position. In the case where the shared object is a collection of small virtual objects, the relative information calculation unit 227 may obtain the relative position relationship between the position of each part of the body of the user object UO1 and at least one of the virtual objects forming the collection as the shared object as the relative position. For example, in the case of remote training for performing medical surgery, assembly work, etc. using VR or AR, such a modified example may be utilized. For example, the user terminal 20 may set the relative position relationship between the virtual object of a part of an organ or body forming a shared object and the hand, finger, etc. of the user object as a relative position. According to such a modified example, the misalignment of the display position between a collection of small virtual objects such as the internal organs of the human body and the user object can be prevented.

[0166] In the above embodiment, when multiple virtual objects are located within a first range whose base point is the user U, and the same other users exist within a second range whose base point is each virtual object, the shared object setting unit 225 selects the virtual object closest to the user object UO1 from the multiple virtual objects and sets it as a shared object. The present disclosure is not limited to this, and the shared object setting unit 225 can select the virtual object to be set as the shared object according to the priority preset for each of the multiple virtual objects. For example, in an example of applying the present disclosure to the case of training for remote surgery on the human body, the shared object setting unit 225 can set the priority of the virtual object representing a specific organ as a surgical object among the virtual objects representing human organs to the highest level, and can set the priority of the virtual object of the organ other than the specific organ to a low level. According to such a modified example, the shared object setting unit 225 can set the shared object according to the priority, thereby more effectively avoiding the misalignment of the display position relative to the user object.

[0167] In the above-mentioned embodiment, the display control unit 229 of the user terminal 20 performs a process of displaying an avatar (user object) representing the other user U (the other user) based on the position information of the other user U (the other user) received from the server 10. At this time, the user terminal 20 may further perform the following process. The display control unit 229 of the user terminal 20 detects whether the shared object of the user U and the other user U has been switched based on the identification information of the shared object included in the received position information of the other user U. When the display control unit 229 detects that the shared object has been switched, the display control unit 229 calculates the distance between the display positions of the other user U calculated based on each of the shared objects before and after the switching. If the calculated distance is equal to or greater than the threshold value, the display control unit 229 performs a display control depicting the avatar of the other user U moving from the display position before the switching to the display position after the switching at a predetermined speed. According to such a modified example, even if the display position of the other user U changes significantly before and after the switching of the shared object, it is possible to prevent the other user U from appearing to move suddenly and causing discomfort.

[0168] In the above embodiment, the control unit 220 of the user terminal 20 has the functions of a coordinate calculation unit 223, a shared object setting unit 225, a relative information calculation unit 227, and a display control unit 229. The present disclosure is not limited thereto, and the control unit 120 of the server 10 may also have the functions of a coordinate calculation unit 223, a shared object setting unit 225, a relative information calculation unit 227, and a display control unit 229. The control unit 120 of the server 10 may also perform the following processing: obtaining the location information of the user U, setting, canceling, switching shared objects, and display control. Fig.17 Provide a detailed description.

[0169] Fig.17 1 is a sequence diagram showing an example of the operation process of the information processing system when the server 10 performs the position information acquisition process, the shared object setting process, and the display control process. First, the user terminal 20a and the user terminal 20b transmit the respectively acquired sensing data of the user U1 and the user U2 to the server 10 (S403, S406).

[0170] Next, the server 10 performs a position information acquisition process based on the received sensing data of each user U (S409). In S409, the server 10 performs the same process as the reference. Fig.13 The sequence diagram describes the same processing of S103 and S106.

[0171] The server 10 sends the acquired location information to each user terminal 20 (S412, S415). Specifically, the server 10 sends the location information of the user U1 acquired based on the sensing data of the user terminal 20a to the user terminal 20b. The server 10 also sends the location information of the user U2 acquired based on the sensing data of the user terminal 20b to the user terminal 20a. Next, the user terminal 20a displays the avatar of the user U2 based on the received location information of the user U2 (S418). The user terminal 20b displays the avatar of the user U1 based on the received location information of the user U1 (S421).

[0172] Reference above Fig.17 An example of the operation process in which the server 10 performs the position information acquisition process, the shared object setting process, and the display control process is described.

[0173] Furthermore, as another modification example of the information processing system according to the present disclosure, the user terminals 20 may directly communicate with each other and transmit and receive the location information of each user U without passing through the server 10 . Fig.18 2 is a sequence diagram showing an example of the operation process of the information processing system when the user terminals 20 directly communicate with each other.

[0174] First, the user terminal 20 a and the user terminal 20 b execute position information acquisition processing for the user U1 and the user U2 , respectively ( S503 , S506 ).

[0175] Next, the user terminal 20a sends the acquired position information of the user U1 to the user terminal 20b (S509). Similarly, the user terminal 20b sends the acquired position information of the user U2 to the user terminal 20a (S512).

[0176] The user terminal 20a and the user terminal 20b display the avatar of the user U1 or the user U2 based on the received position information, respectively (S515, S518).

[0177] <5. Hardware Configuration Example>

[0178] The embodiments of the present disclosure have been described above. The above information processing, such as the setting, cancellation and switching of shared objects and the calculation of relevant information about the shared objects as the base point, is realized through the collaboration between software and hardware. The hardware configuration example that can be applied to the server 10 and the user terminal 20 will be described below.

[0179] Fig.19 90. The hardware configuration example of the hardware configuration 90 described below is merely one example of the hardware configuration of the server 10 and the user terminal 20. Therefore, each of the server 10 and the user terminal 20 does not necessarily include Fig.19 All hardware components shown in . Fig.19 Some of the illustrated hardware components may not exist in the server 10 and the user terminal 20 .

[0180] like Fig.19 As shown, the hardware configuration 90 includes a CPU 901, a read-only memory (ROM) 903, and a RAM 905. The hardware configuration 90 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 923, and a communication device 925. Instead of or in addition to the CPU 901, the hardware configuration 90 may have a processing circuit such as a processing circuit called a graphics processing unit (GPU), a digital signal processor (DSP), or an application-specific integrated circuit (ASIC).

[0181] The CPU 901 is used as an operation processing device and a control device, and controls all or part of the operations in the hardware configuration 90 according to various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927. The ROM 903 stores programs and calculation parameters used by the CPU 901. The RAM 905 temporarily stores programs used in the execution of the CPU 901, parameters that change appropriately during execution, and the like. The CPU 901, the ROM 903, and the RAM 905 are interconnected by a host bus 907, and the host bus 907 is configured by an internal bus such as a CPU bus. In addition, the host bus 907 is connected to an external bus 911 such as a peripheral component interconnect / interface (PCI) bus via a bridge 909.

[0182] The input device 915 is, for example, a device operated by a user, such as a button. The input device 915 may include a mouse, a keyboard, a touch panel, a switch, a lever, etc. In addition, the input device 915 may also include a microphone for detecting the user's voice. The input device 915 may be, for example, a remote control device using infrared or other radio waves, or may be an external connection device 929 such as a mobile phone corresponding to the operation of the hardware configuration 90. The input device 915 includes an input control circuit that generates an input signal based on information input by the user and outputs the input signal to the CPU 901. The user inputs various types of data to the hardware configuration 90 and instructs processing operations by operating the input device 915.

[0183] The input device 915 may also include an imaging device and a sensor. The imaging device is a device that captures an image of a real space and generates a captured image using an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) and various types of components such as a lens for controlling the formation of a subject image on the imaging element. The imaging device may be an imaging device that captures a still image or an imaging device that captures a moving image.

[0184] The sensor may include various types of sensors, such as, for example, a distance measurement sensor, an acceleration sensor, a gyroscope sensor, a geomagnetic sensor, a vibration sensor, a light sensor, and a sound sensor. The sensor acquires information about the state of the hardware configuration 90 itself, such as the posture of the housing of the hardware configuration 90, or information about the surrounding environment of the hardware configuration 90, such as the brightness or noise around the hardware configuration 90. The sensor may also include a global positioning system (GPS) sensor that receives a GPS signal to measure the latitude, longitude, and altitude of the device.

[0185] The output device 917 is composed of a device that can visually or auditorily notify the user of the acquired information. The output device 917 can be, for example, a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display, or a sound output device such as a speaker or earphone. In addition, the output device 917 may include a plasma display panel (PDP), a projector, a hologram, a printer device, etc. The output device 917 outputs the result obtained by the processing of the hardware configuration 90 as text or a video such as an image, or as a sound such as a voice or an acoustic sound. The output device 917 may also include a lighting device for illuminating the surroundings, etc.

[0186] The storage device 919 is a data storage device configured as an example of a storage unit of the hardware configuration 90. The storage device 919 is configured by, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores programs executed by the CPU 901, various data, various data acquired from the outside, and the like.

[0187] The drive 921 is a reader and writer for a removable recording medium 927 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built in or externally attached to the hardware configuration 90. The drive 921 reads information recorded on the attached removable recording medium 927, and outputs the information to the RAM 905. The drive 921 also writes records to the attached removable recording medium 927.

[0188] The connection port 923 is a port for directly connecting a device to the hardware configuration 90. The connection port 923 may be, for example, a universal serial bus (USB) port, an IEEE1394 port, or a small computer system interface (SCSI) port. The connection port 923 may also be an RS-232C port, an optical audio terminal, an HDMI (registered trademark) (High Definition Multimedia Interface) port, etc. The connection of the external connection device 929 to the connection port 923 enables various types of data to be exchanged between the hardware configuration 90 and the external connection device 929.

[0189] The communication device 925 is, for example, a communication interface consisting of a communication device for connecting to a local network, or a communication network with a base station for wireless communication, etc. The communication device 925 can be, for example, a communication card for a wired or wireless local area network (LAN), Bluetooth (registered trademark), Wi-Fi (registered trademark) or wireless USB (WUSB). The communication device 925 can also be a router for optical communication, a router for an asymmetric digital subscriber line (ADSL) or any modem for communication. The communication device 925, for example, uses a predetermined protocol such as TCP / IP to send a signal to the Internet or another communication device or receives a signal from the Internet or another communication device, etc. The local network connected to the communication device 925 or the communication network with a base station is a wired or wireless network, such as the Internet, a home LAN, infrared communication, radio wave communication or satellite communication.

[0190] <6.Supplement>

[0191] Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings as described above, the technical scope of the present disclosure is not limited to such examples. It is obvious that a person of ordinary skill in the technical field of the present disclosure can conceive various modified examples or variation examples within the scope of the technical ideas set forth in the claims, and it should be understood that these naturally also fall within the technical scope of the present disclosure.

[0192] For example, in the above-mentioned embodiment, the user terminal 20 acquires the position information of the user U based on the sensing data acquired by the camera 240, the camera 241, the camera 240 and the HMD 250. As a method for tracking the position of the user U wearing the HMD, this type of method is generally referred to as an outside-in method. However, the information processing system according to the present disclosure is not limited to this example. For example, as another example of the outside-in method, a base station that radially emits laser light may also be installed instead of the cameras 240, 241, and 242. The user terminal 20 may acquire the position information of the user U wearing the HMD 250 based on the reception time of the laser light received by the HMD 250, the angle of the laser light at the reception point, and the time difference between the emission time of the laser light and the reception time of the laser light. Alternatively, as another example, the user terminal 20 may acquire the position information of the user U by a method using a geomagnetic sensor instead of the cameras 240, 241, and 242.

[0193] In addition, the user terminal 20 may also obtain the position information of the user U through an inside-out method, which is a method of tracking the position of the HMD 250 based on the surrounding image acquired by the camera equipped by the HMD 250 itself. For example, the user terminal 20 may obtain the position information of the user U through simultaneous localization and mapping (SLAM), which creates an environment map of the surrounding conditions of the user U using the camera equipped by the HMD 250, and estimates the own position of the HMD 250 worn by the user U.

[0194] The steps in the processing of the operation of the server 10 and the user terminal 20 according to the present embodiment do not necessarily need to be processed in chronological order according to the order described in the explanatory diagram. For example, each step in the processing of the operation of the server 10 and the user terminal 20 may be processed in an order different from the order described in the explanatory diagram, or may be processed in parallel.

[0195] It is also possible to create one or more computer programs for causing hardware such as CPU, ROM, RAM built into the above-mentioned server 10 and user terminal 20 to implement the functions of the information processing system according to the present embodiment. In addition, a computer-readable storage medium having one or more computer programs stored therein is provided.

[0196] In addition, the effects described herein are merely illustrative or exemplary and are not intended to be limiting. In other words, the technology according to the present disclosure may exhibit other effects that are apparent to those skilled in the art from the description herein in addition to or in place of the above-described effects.

[0197] The following configurations also belong to the technical scope of the present disclosure.

[0198] (1) An information processing device includes a control unit, the control unit:

[0199] Setting a virtual object as a shared object between a first user and a second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object;

[0200] sending relative information indicating a relative positional relationship between the shared object and the first user to a user terminal associated with the second user; and

[0201] The setting of the shared object is switched in response to a change in at least any one of a position of the first user, a position of the shared object, and a position of the second user in the virtual space.

[0202] (2) The information processing device according to (1), wherein the control unit switches the setting of the shared object by canceling the setting of the shared object for the virtual object and setting a new virtual object as the shared object.

[0203] (3) The information processing device according to (2), wherein:

[0204] Control unit:

[0205] In response to a change in the position of the first user in the virtual space and in response to the position of the shared object being outside the first range, canceling the setting of the shared object; and

[0206] A new virtual object is set as a shared object: the new virtual object is located within a first range whose base point is the changed position, and the other user exists within a second range whose base point is the new virtual object.

[0207] (4) The information processing device according to (2), wherein:

[0208] Control unit:

[0209] In response to a change in the position of the shared object in the virtual space and in response to the position of the shared object being outside the first range, canceling the setting of the shared object; and

[0210] A new virtual object is set as a shared object: the new virtual object is located within a first range whose base point is the first user, and other users exist within a second range whose base point is the new virtual object.

[0211] (5) The information processing device according to (2), wherein:

[0212] Control unit:

[0213] In response to the second user's position being outside a second range whose base point is the shared object due to a change in the second user's position in the virtual space, canceling the setting of the shared object; and

[0214] A new virtual object is set as a shared object: the new virtual object is located within a first range whose base point is the first user, and other users exist within a second range whose base point is the new virtual object.

[0215] (6) The information processing device according to any one of (3) to (5), wherein the control unit transmits relative information indicating a relative positional relationship between the newly set shared object and the first user to user terminals associated with other users.

[0216] (7) An information processing device according to (1) or (2), wherein, when a virtual object located within a first range whose base point is a first user is not detected, or when a second user is not detected within a second range whose base point is a shared object, the control unit cancels the setting of a virtual object that has been set as a shared object.

[0217] (8) An information processing device according to (6), wherein, when the virtual object is not within the first range, the control unit sends coordinate information indicating the position of the first user in the virtual space or relative information indicating the positional relationship between the first user and a preset virtual object to a user terminal associated with the second user.

[0218] (9) The information processing device according to any one of (1) to (8), wherein when a plurality of virtual objects are located within a first range whose base point is the first user, and a second user exists within a second range whose base point is each of the plurality of virtual objects,

[0219] The control unit selects one virtual object from among the plurality of virtual objects and sets the one virtual object as a shared object.

[0220] (10) The information processing device according to (8), wherein the control unit selects a virtual object located closest to the first user in the virtual space from among the plurality of virtual objects.

[0221] (11) The information processing device according to (8), wherein the control unit selects the virtual object to be set as the shared object from among the plurality of virtual objects according to a priority preset for each of the plurality of virtual objects.

[0222] (12) The information processing device according to any one of (1) to (11), wherein:

[0223] For one or more virtual objects located within a first range whose base point is the first user, when another user exists within each second range whose base point is each virtual object among the one or more virtual objects, the control unit:

[0224] setting each of the one or more virtual objects as a shared object to be shared between the first user and each of the other users; and

[0225] Relative information indicating a relative positional relationship between each of the set shared objects and the first user is sent to each user terminal associated with each other user.

[0226] (13) The information processing device according to any one of (1) to (12), wherein:

[0227] When a plurality of other users exist within the second range, the control unit:

[0228] setting the virtual object as a shared object to be shared between the first user and each of the other users; and

[0229] Relative information indicating a relative positional relationship between the first user and the shared object is sent to each user terminal associated with each of the other users.

[0230] (14) An information processing device according to (13), wherein the control unit calculates a relative positional relationship between a position of each part of a body of the first user included in a first user object representing the first user in a virtual space and the shared object as relative information.

[0231] (15) An information processing device according to any one of (1) to (14), wherein, when identification information of a shared object set with the second user as a base point and relative information indicating the relative position relationship of the second user with respect to the shared object are received from a user terminal associated with the second user as position information of the second user, the control unit performs control of displaying a second user object representing the second user at a position calculated based on the relative information in the virtual space.

[0232] (16) The information processing device according to (15), wherein:

[0233] The relative information includes identification information for uniquely identifying the virtual object set as a shared object, and

[0234] When it is detected that the shared object set based on the second user has been switched based on the identification information of the shared object received from the user terminal associated with the second user,

[0235] Control unit:

[0236] calculating a distance between display positions of the second user calculated based on each of the shared objects before and after the switching; and

[0237] When the calculated distance is equal to or greater than a threshold value, control is performed to depict a motion of the second user object moving from a display position before switching to a display position after switching at a predetermined speed.

[0238] (17) The information processing device according to any one of (1) to (16), further including a communication unit that receives location information of each user from a user terminal associated with the first user and a user terminal associated with the second user.

[0239] (18) The information processing device according to any one of (1) to (16), further comprising a communication unit that communicates with a user terminal or a server associated with the second user,

[0240] Among them, the control unit:

[0241] displaying the virtual space and the virtual object within the field of view of the first user in the virtual space on the display unit;

[0242] transmitting, from the communication unit, relative information indicating a relative positional relationship between the shared object and the first user in the virtual space; and

[0243] Based on the relative information received from the user terminal or the server and indicating the relative positional relationship between the second user and the shared object set with the second user as a base point, a user object representing the second user is displayed.

[0244] (19) A program for causing a computer to function as a control unit, the control unit:

[0245] Setting a virtual object as a shared object for the first user and the second user: in the virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object;

[0246] sending relative information indicating a relative positional relationship between the shared object and the first user to a user terminal associated with the second user; and

[0247] The setting of the shared object is switched in response to a change in at least any one of a position of the first user in the virtual space, a position of a virtual object set as a shared object, and a position of the second user.

[0248] (20) An information processing system comprising:

[0249] a first user terminal associated with a first user;

[0250] a second user terminal associated with a second user; and

[0251] An information processing device comprises a control unit, the control unit:

[0252] Setting a virtual object as a shared object between the first user and the second user: in the virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object;

[0253] sending relative information indicating a relative position relationship between the shared object and the first user to a second user terminal; and

[0254] switching the setting of the shared object in response to a change in at least any one of a position of the first user in the virtual space, a position of the virtual object set as the shared object, and a position of the second user,

[0255] in,

[0256] The second user terminal displays a user object representing the first user at a position in the virtual space calculated based on the relative information received from the information processing device.

[0257] [reference numerals list]

[0258] 10 servers

[0259] 110 storage units

[0260] 120 control unit

[0261] 130 communication units

[0262] 20 user terminals

[0263] 210 storage units

[0264] 220 control unit

[0265] 221 sensor data acquisition unit

[0266] 223 coordinate calculation unit

[0267] 225 Shared Object Setup Unit

[0268] 227 Relative Information Calculation Unit

[0269] 229 Display control unit

[0270] 230 communication unit

[0271] 240 Camera Device

[0272] 241 Camera Device

[0273] 242 Camera Device

[0274] 250HMD

Claims

1. An information processing device, comprising a control unit, wherein: Setting a virtual object as a shared object for a first user and a second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object; sending relative information indicating a relative position relationship between the shared object and the first user to a user terminal associated with the second user; as well as The setting of the shared object is switched in response to a change in at least any one of a position of the first user, a position of the shared object, and a position of the second user in the virtual space.

2. The information processing device according to claim 1, wherein: The control unit switches the setting of the shared object by canceling the setting of the shared object for the virtual object and setting a new virtual object as the shared object.

3. The information processing device according to claim 2, wherein: The control unit: In response to a change in the position of the first user in the virtual space and in response to the position of the shared object being outside the first range, canceling the setting of the shared object; and A new virtual object is set as the shared object: the new virtual object is located within the first range whose base point is the changed position, and other users exist within a second range whose base point is the new virtual object.

4. The information processing device according to claim 2, wherein: The control unit: In response to a change in the position of the shared object in the virtual space and in response to the position of the shared object being outside the first range, canceling the setting of the shared object; as well as A new virtual object is set as the shared object: the new virtual object is located within a first range whose base point is the first user, and other users exist within a second range whose base point is the new virtual object.

5. The information processing device according to claim 2, wherein: The control unit: In response to the position of the second user being outside the second range whose base point is the shared object due to a change in the position of the second user in the virtual space, canceling the setting of the shared object; as well as A new virtual object is set as the shared object: the new virtual object is located within a first range whose base point is the first user, and other users exist within a second range whose base point is the new virtual object.

6. The information processing device according to claim 3, wherein: The control unit transmits relative information indicating a relative positional relationship between the newly set shared object and the first user to a user terminal associated with the other user.

7. The information processing device according to claim 1, wherein: When a virtual object located within a first range whose base point is the first user is not detected, or when the second user is not detected to exist within a second range whose base point is the shared object, the control unit cancels the setting of the virtual object that has been set as the shared object.

8. The information processing device according to claim 6, wherein: When the virtual object is not within the first range, the control unit sends coordinate information indicating the position of the first user in the virtual space or relative information indicating the positional relationship between the first user and a preset virtual object to the user terminal associated with the second user.

9. The information processing device according to claim 1, wherein: When a plurality of virtual objects are located within the first range whose base point is the first user, and the second user exists within the second range whose base point is each of the plurality of virtual objects, The control unit selects one virtual object from among the plurality of virtual objects and sets the one virtual object as the shared object.

10. The information processing device according to claim 9, wherein: The control unit selects a virtual object located closest to the first user in the virtual space from among the plurality of virtual objects.

11. The information processing device according to claim 9, wherein: The control unit selects a virtual object to be set as the shared object from among the plurality of virtual objects according to a priority preset for each of the plurality of virtual objects.

12. The information processing device according to claim 1, wherein: For one or more virtual objects located within the first range whose base point is the first user, when another user exists within each second range whose base point is each virtual object among the one or more virtual objects, the control unit: setting each of the one or more virtual objects as the shared object to be shared between the first user and each other user; and Relative information indicating a relative positional relationship between each of the set shared objects and the first user is sent to each user terminal associated with each other user.

13. The information processing device according to claim 1, wherein: When a plurality of other users exist within the second range, the control unit: setting the virtual object as the shared object to be shared between the first user and each of the other users; as well as Relative information indicating a relative positional relationship between the first user and the shared object is sent to each user terminal associated with each of the other users.

14. The information processing device according to claim 13, wherein: The control unit calculates, as the relative information, a relative positional relationship between a position of each part of a body of the first user included in a first user object representing the first user in the virtual space and the shared object.

15. The information processing device according to claim 1, wherein: When identification information of a shared object set with the second user as a base point and relative information indicating the relative position relationship of the second user with respect to the shared object are received from a user terminal associated with the second user as position information of the second user, the control unit performs control of displaying a second user object representing the second user at a position calculated based on the relative information in the virtual space.

16. The information processing device according to claim 15, wherein: The relative information includes identification information for uniquely identifying the virtual object set as the shared object, and When it is detected that the shared object set based on the second user has been switched based on the identification information of the shared object received from the user terminal associated with the second user, The control unit: calculating a distance between display positions of the second user calculated based on each of the shared objects before and after the switching; as well as When the calculated distance is equal to or greater than a threshold value, control is performed to depict a motion of the second user object moving from a display position before switching to a display position after switching at a predetermined speed. 17 . The information processing device according to claim 1 , further comprising a communication unit that receives location information of each user from a user terminal associated with the first user and a user terminal associated with the second user.

18. The information processing device according to claim 1, further comprising a communication unit that communicates with a user terminal or a server associated with the second user, in, The control unit: displaying the virtual space and the virtual object within the field of view of the first user in the virtual space on a display unit; transmitting, from the communication unit, the relative information indicating a relative positional relationship between the shared object and the first user in the virtual space; as well as Based on the relative information indicating the relative positional relationship between the second user and the shared object set with the second user as a base point, which is received from the user terminal or the server, a user object representing the second user is displayed.

19. A program for causing a computer to function as: A control unit, the control unit: Setting a virtual object as a shared object for a first user and a second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object; sending relative information indicating a relative position relationship between the shared object and the first user to a user terminal associated with the second user; as well as The setting of the shared object is switched in response to a change in at least any one of a position of the first user in the virtual space, a position of a virtual object set as the shared object, and a position of the second user.

20. An information processing system comprising: a first user terminal associated with a first user; a second user terminal associated with a second user; as well as An information processing device comprises a control unit, wherein: setting the following virtual object as a shared object of the first user and the second user: in a virtual space, the virtual object is located within a first range whose base point is the first user, and the second user exists within a second range whose base point is the virtual object; sending relative information indicating a relative position relationship between the shared object and the first user to the second user terminal; as well as switching the setting of the shared object in response to a change in at least any one of a position of the first user in the virtual space, a position of a virtual object set as the shared object, and a position of the second user, in, The second user terminal displays a user object representing the first user at a position in a virtual space calculated based on the relative information received from the information processing device.

Images