Information processing device, information processing method, and program

Abstract

To provide a technology capable of appropriately arranging each avatar within a non-real space even in the case that a positional relation of a plurality of participants is not predetermined in a real space in a service for enabling communication within the non-real space.SOLUTION: According to the present invention, an information processing device includes determination means for determining arrangement within a non-real space of a plurality of avatars respectively corresponding to a plurality of participants of a service for enabling communication within the non-real space, and acquisition means for acquiring information in a positional relation between a first participant and a second participant within a real space in the case of detecting that the second participant of the service exists around the first participant who newly participates in the service within the real space. The determination means determines arrangement of the plurality of avatars on the basis of the information.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and a storage medium, and more particularly to a service that enables communication in a non-real space. [Background technology]

[0002] Systems have been proposed for online conferences in unreal spaces such as virtual reality spaces (VR spaces) or mixed reality spaces (MR spaces). In such systems, participants in an online conference are represented by virtual avatars. For example, participants wear a head-mounted display (HMD) on their heads. The HMD displays a conference room, which is a three-dimensional unreal space, and displays the avatars of other participants in the conference room. The voices of the other participants are output from the speakers of the HMD so that they can be heard from the direction of the avatars of the other participants. This allows participants to experience the feeling of actually having a conference with the other participants.

[0003] Patent Document 1 discloses a technology that displays on the HMD of a user sitting in a car seat an avatar of another user who is located away from the car, as if the avatar were sitting in another seat in the car.

[0004] Patent Document 2 discloses a technique for determining the arrangement of a plurality of avatars (a plurality of avatars corresponding to a plurality of users, respectively) in a virtual reality space based on the positional relationships of the plurality of users in the real space. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-149607 [Patent Document 2] International Publication No. 2018 / 198777 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in a conventional system for online conferences in an unreal space, the relative positions of multiple participants who are close to each other in the real space may differ from the relative positions of multiple avatars (multiple avatars corresponding to multiple participants) in the unreal space. In such a case, the direction from which the voices of other participants can be heard through the HMD speaker differs from the direction from which the voices of the other participants can be heard without going through the HMD speaker, which makes the participants feel uncomfortable and reduces their sense of immersion.

[0007] Such problems are not limited to systems for online conferences in unreal spaces, but may occur in various services that enable communication in unreal spaces. For example, in the technology disclosed in Patent Document 1, the avatars of two users aligned left and right in real space may end up aligned front to back in a car. In the technology disclosed in Patent Document 2, a predetermined fixed positional relationship is taken into consideration as the positional relationship of multiple users in real space, but the above problem occurs when the positional relationship of multiple users in real space is not predetermined.

[0008] The present invention aims to provide a technology that can appropriately determine the positioning of each avatar in a non-real space in a service that enables communication in the non-real space, even when the relative positions of multiple participants in the real space are not predetermined. [Means for solving the problem]

[0009] A first aspect of the present invention is an information processing device comprising: a determination means for determining the placement of a number of avatars in a non-real space, each avatar corresponding to a number of participants of a service that enables communication in the non-real space; and an acquisition means for acquiring information regarding the positional relationship between the first participant and the second participant in the real space when the presence of a second participant of the service is detected around a first participant who has newly joined the service in the real space, wherein the determination means determines the placement of the number of avatars based on the information.

[0010] A second aspect of the present invention is an information processing method comprising a determination step of determining an arrangement in an unreal space of multiple avatars corresponding to multiple participants of a service that enables communication in the unreal space, and an acquisition step of acquiring information regarding a positional relationship between the first participant and the second participant in the real space when the presence of a second participant of the service is detected around a first participant who has newly joined the service in the real space, wherein the determination step determines an arrangement of the multiple avatars based on the information.

[0011] A third aspect of the present invention is a program for causing a computer to function as each of the means of the information processing device described above.A fourth aspect of the present invention is a computer-readable storage medium storing a program for causing a computer to function as each of the means of the information processing device described above. Effect of the Invention

[0012] According to the present invention, in a service that enables communication in an unreal space, the position of each avatar in the unreal space can be suitably determined even if the positional relationships between multiple participants in the real space are not predetermined. [Brief description of the drawings]

[0013] [Figure 1]FIG. 1 is a diagram showing a configuration of an online conference system according to first to third embodiments. [Diagram 2] FIG. 1 is a diagram showing the configuration of an HMD system according to first to third embodiments. [Diagram 3] FIG. 1 shows a configuration of a server device according to first to third embodiments. [Figure 4] FIG. 1 is a diagram showing the arrangement of an HMD system according to a first embodiment; [Diagram 5] FIG. 1 is a diagram showing a processing flow of an online conference system according to a first embodiment. [Figure 6] FIG. 1 is a diagram showing a processing flow of the HMD system according to the first embodiment; [Figure 7] FIG. 1 is a diagram showing a processing flow of a server device according to a first embodiment; [Figure 8] FIG. 13 is a diagram showing the arrangement of an HMD system according to a second embodiment. [Figure 9] FIG. 13 is a diagram showing a processing flow of an online conference system according to a second embodiment. [Figure 10] FIG. 11 is a diagram showing a processing flow of the HMD system according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing a processing flow of a server device according to a second embodiment; [Figure 12] FIG. 11 is a diagram showing a processing flow of an HMD system according to a third embodiment. [Figure 13] FIG. 13 is a diagram showing a processing flow of a server device according to a third embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that each embodiment described below is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of each embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of each embodiment.

[0015] [First embodiment] <System configuration> A first embodiment of the present invention will be described. FIG. 1 is a schematic diagram showing a rough configuration of a system according to the first embodiment. The system in FIG. 1 is a system for communication in an unreal space such as a virtual reality space (VR space) or a mixed reality space (MR space), for example, a system for an online conference in an unreal space (online conference system). The VR space is a three-dimensional space (world) created with CG (Computer Graphics). The MR space is a three-dimensional space similar to the real space. A virtual object (e.g., an avatar) can be placed in the MR space, but a virtual object cannot be placed in the real space. In other words, when the real space is viewed without using a display device that displays virtual objects, the real space is not an MR space, and when the real space is viewed using a display device that displays virtual objects, the real space is an MR space. The MR space can also be regarded as an augmented reality (AR) space. The avatar is a character (user character) that is an alter ego of the user (user of the above system) in the unreal space, and is created with CG.

[0016] The system in FIG. 1 includes HMD systems 100, 400, 500, and 600, a server device 200, and a network 300.

[0017] Each of the HMD systems 100, 400, 500, and 600 is used by a participant of a service that enables communication in a non-real space (e.g., a participant of an online conference in a non-real space). Each of the HMD systems 100, 400, 500, and 600 has a head-mounted display (HMD) that is a head-mounted display device, and a client device that is an information processing device. The four HMD systems 100, 400, 500, and 600 correspond to four participants, respectively. Each of the HMD systems 100, 400, 500, and 600 is communicably connected to the server device 200 via the network 300.

[0018] In the first embodiment, an example where the client device and the HMD are separate devices is described, but the client device and the HMD may be integrally configured. Also, the number of participants is not particularly limited and may be more than 4 or less than 4. The connection between the HMD systems 100, 400, 500, 600 and the server device 200 via the network 300 may be a wired connection or a wireless connection.

[0019] The server device 200 is a server device (information processing device) that provides a service that enables communication in a virtual space. Note that the number of server devices is not particularly limited, and the above service may be provided by a plurality of server devices performing processing in parallel.

[0020] In the first embodiment, an example of an online meeting in a virtual space is described. When the virtual space is a VR space, in each of the HMD systems 100, 400, 500, 600, an image in which an avatar is placed in a CG conference room is displayed. When the virtual space is an MR space, in each of the HMD systems 100, 400, 500, 600, the avatar is displayed so that the avatar is placed in the real conference room. When the virtual space is an MR space, in each of the HMD systems 100, 400, 500, 600, the HMD may function as a video see-through type HMD or an optical see-through type HMD. In the first embodiment, an example where the HMD functions as a video see-through type HMD is described. When the virtual space is an MR space, a participant can see other participants (real participants, not avatars) who are near the participant in the real space. In the HMD system (HMD) of the participant, the avatars of other participants who are near the participant in the real space are not displayed, and only the avatars of other participants who are at a location away from the participant in the real space are displayed.

[0021] <Configuration of the HMD System> FIG. 2 is a block diagram showing the configuration of the HMD system 100. The HMD system 100 includes an HMD 101, a client device 102, and an input device 103. The HMD 101 and the client device 102 are connected to each other in a wired or wireless manner so as to be communicable, and the client device 102 and the input device 103 are connected to each other in a wired or wireless manner so as to be communicable. The HMD systems 400, 500, and 600 have the same configuration as the HMD system 100. In the first embodiment, an example in which the HMD 101, the client device 102, and the input device 103 are separate devices will be described, but at least two of these three devices may be integrally configured.

[0022] <Configuration of HMD> The HMD 101 includes an imaging unit 111, a display unit 112, a position and orientation detection unit 113, an audio input unit 114, and an audio output unit 115.

[0023] The imaging unit 111 images the external world (real space). For example, the imaging unit 111 includes an image sensor that converts light into an electrical signal and an objective optical system that guides the light from the external world to the image sensor.

[0024] The display unit 112 displays an image (an image corresponding to the field of view of the user wearing the HMD 101) generated by the client device 102. Note that the HMD 101 may include a display unit for the right eye and a display unit for the left eye. Then, a stereoscopic image (an image for the right eye and an image for the left eye) that enables stereoscopic viewing may be displayed by the two display units.

[0025] The position and orientation detection unit 113 detects the position and orientation (position and orientation) of the HMD 101 (the user wearing the HMD 101) in the real space. For example, the position and orientation detection unit 113 has various sensors such as an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, and a GPS sensor. The detection result of the position and orientation detection unit 113 is used to control the display by the display unit 112 and to control the orientation of an avatar corresponding to the user (the user wearing the HMD 101). For example, a part of the unreal space to be displayed on the display unit 112 is changed according to a change in the position and orientation of the user. This makes it possible to give the user a sense of being in the unreal space. Also, according to a change in the user's posture, the posture of the avatar corresponding to the user is changed. This enables communication using posture.

[0026] The audio input unit 114 is a microphone that acquires audio, and may be an input terminal to which an audio input device such as a microphone is connected.

[0027] The audio output unit 115 is a speaker that outputs audio, and may be an output terminal to which an audio output device such as a speaker, a headphone, or an earphone is connected.

[0028] It should be noted that a tablet terminal or other display device may be used instead of the HMD 101. Any display device may be used as long as it is a (portable) display device that can display an image corresponding to the user's field of view.

[0029] <Input device configuration> The input device 103 includes an operation unit 170. The operation unit 170 receives a user operation (operation by a user) and outputs an operation signal corresponding to the user operation. In response to an operation (operation signal), for example, an avatar corresponding to the user is controlled to perform a specific action (gesture or movement). The user can also give an instruction to switch the unreal space between the VR space and the MR space. The input device 103 is, for example, a joystick that outputs an operation signal according to the direction and magnitude of the tilt of a lever. Note that the input device 103 is not limited to a joystick, and may be, for example, a mouse, a keyboard, or a game pad.

[0030] <Client device configuration> The client device 102 has a CPU 120, a ROM 130, a RAM 140, a communication unit 150, and a participant detection unit 160. The CPU 120 realizes an image acquisition unit 121, a position and orientation extraction unit 122, a control unit 123, an audio acquisition unit 124, an audio transmission unit 125, and an input processing unit 126. The client device 102 is an information processing device having excellent arithmetic processing functions and graphic drawing functions, and is, for example, a smartphone, a personal computer, or a workstation.

[0031] The CPU 120 performs various processes (various controls) of the client device 102. The ROM 130 is a non-volatile memory. The ROM 130 stores an OS (operating system) program, various application programs, and other information. For example, the ROM 130 stores a program for an application for conducting an online conference (online conference execution application). The RAM 140 is a volatile memory, and is used, for example, as a working area for the CPU 120.

[0032] The image acquisition section 121 acquires a real image obtained by the imaging section 111 (an image obtained by capturing an image of a real space).

[0033] The position and orientation extraction unit 122 acquires the position and orientation (position and orientation information) detected by the position and orientation detection unit 113. The position and orientation extraction unit 122 may detect markers arranged in the real space from the real image acquired by the image acquisition unit 121. Then, the position and orientation extraction unit 122 may detect the position and orientation of the HMD 101 (user) in the real space based on the detection result of the markers.

[0034] The control unit 123 executes an online conference execution application (a program of the online conference execution application) stored in the ROM 130, and executes various processes for conducting an online conference. For example, the control unit 123 transmits the position and orientation information acquired by the position and orientation extraction unit 122 to the server device 200 in order to share it with other participants of the online conference. In order to conduct an online conference, the control unit 123 receives avatar images (three-dimensional images of avatars) of the other participants and a CG conference room image (three-dimensional image of a CG conference room) from the server device 200, and stores them in the RAM 140. The control unit 123 also receives position and orientation information of the other avatars (other participants) and seat information of each avatar from the server device 200. The control unit 123 controls the communication unit 150 to communicate with the server device 200.

[0035] When the unreal space is a VR space, the control unit 123 generates a video (video data) of the online conference based on the avatar images of the other participants, the CG conference room image, the position and orientation information of the other avatars, the seat information of each avatar, and the position and orientation information of the user. When the unreal space is an MR space, the control unit 123 does not receive the CG conference room image, but generates a video (video data) of the online conference using a real image acquired by the image acquisition unit 121 instead of the CG conference room image. Then, the control unit 123 controls the generated video to be displayed on the display unit 112. For example, the control unit 123 generates a video in which the avatar image is superimposed on the CG conference room image or the real image. The control unit 123 determines the orientation of each avatar as the position and orientation. The system controls the position of each avatar based on the seat information (information indicating the seat). Note that the generated image is from a first-person perspective, and the user's avatar is not displayed.

[0036] The voice acquisition unit 124 acquires voice (voice data) acquired by the voice input unit 114. The control unit 123 transmits the voice (voice data) acquired by the voice acquisition unit 124 to the server device 200 in order to convey it to the other participants. The control unit 123 receives voice data of the other participants (voice data acquired by the HMD (voice input unit) of the other participants) from the server device 200. Then, the control unit 123 generates voice (voice data) to be output from the voice output unit 115 based on the received voice data. The voice transmission unit 125 transmits the voice data generated by the control unit 123 to the voice output unit 115, and the voice output unit 115 outputs voice in accordance with the voice data transmitted from the voice transmission unit 125. As a result, the voice is output from the voice output unit 115 so that the voice of the other participants can be heard from the direction of the avatars of the other participants.

[0037] The communication unit 150 is a communication interface that communicates with an external device in a wired or wireless manner. The communication method (communication protocol) is not particularly limited, but for example, the communication unit 150 communicates with the external device via a so-called wireless LAN in accordance with the IEEE802.11 standard.

[0038] The participant detection unit 160 detects other participants from around the user in the real space. For example, the participant detection unit 160 detects other client devices (client devices of other participants) by Bluetooth (registered trademark). The detection of other client devices can also be regarded as the detection of other HMD systems (HMD systems of other participants). When other participants exist around the user in the real space, the participant detection unit 160 receives a beacon signal emitted from the other HMD systems (other client devices). Then, the control unit 123 detects the identification number of the other HMD system, the direction in which the other HMD system exists, and the distance to the other HMD system from the beacon signal received by the participant detection unit 160. When multiple other participants exist around the user in the real space, multiple combinations of the identification number, direction, and distance are detected. In addition, the participant detection unit 160 emits a beacon signal so that the other HMD systems (participant detection units) can detect the HMD system 100. The control unit 123 transmits the detected information to the server device 200. The method of detecting other participants is not particularly limited. For example, other HMD systems may be detected from images acquired by the image acquisition unit 121.

[0039] The input processing unit 126 acquires an operation signal output from the operation unit 170. The control unit 123 controls the behavior of an avatar corresponding to the user in response to the operation signal acquired by the input processing unit 126. The control unit 123 transmits the operation signal acquired by the input processing unit 126 to the server device 200 in order to share it with other participants of the online conference. The control unit 123 receives the operation signal of the other participant (operation signal obtained by the input device (operation unit) of the other participant) from the server device 200. Then, the control unit 123 controls the behavior of the other avatar (the avatar corresponding to the other participant) in response to the received operation signal. The control unit 123 may switch the unreal space between the VR space and the MR space in response to the operation signal acquired by the input processing unit 126.

[0040] <Server device configuration> 3 is a block diagram showing the configuration of the server device 200. The server device 200 has a CPU 201, a ROM 202, a RAM 203, and a communication unit 204. A control unit 211 and a participant management unit 212 are realized by the CPU 201.

[0041] The CPU 201 performs various processes (various controls) of the server device 200. ROM 202 is a non-volatile memory. An OS (operating system) program, various application programs, and other information are stored in ROM 202. For example, ROM 202 stores a program of an application (online conference providing application) for providing online conference services to each HMD system, and participant information of the online conference. The participant information is, for example, a list indicating the identification number of the HMD system for each participant. RAM 203 is a volatile memory, and is used, for example, as a working area for CPU 201.

[0042] The control unit 211 executes an online conference providing application (a program of the online conference providing application) stored in the ROM 202, and executes various processes for providing an online conference service to each HMD system. For example, the control unit 211 transmits avatar images of other participants and a CG conference room image to each HMD system (each client device). The control unit 211 also transmits and receives position and orientation information and operation information to and from each HMD system (each client device) so that the position and orientation information and operation information of each participant are shared by a plurality of participants. The control unit 211 transmits seat information of each avatar to each HMD system (each client device). The control unit 211 transmits and receives audio data to and from the HMD system (client device) in order to convey the audio data of the participant to the other participants. For example, the control unit 211 transmits audio data received from the HMD system 100 to the HMD systems 400, 500, and 600. The control unit 211 controls the communication unit 204 to communicate with each HMD system (each client device).

[0043] The participant management unit 212 manages, for each participant in the online conference, the identification number of the HMD system and the seat in which the avatar sits in the online conference room.

[0044] The communication unit 204 is a communication interface that communicates with an external device in a wired or wireless manner. The communication method (communication protocol) is not particularly limited, but for example, the communication unit 204 communicates with the external device via a so-called wireless LAN in accordance with the IEEE802.11 standard.

[0045] <Avatar seat> The participant management unit 212 determines the seat of each avatar in the online conference. The determination of the seat of each avatar in the online conference can be regarded as the determination of the arrangement of each avatar in the unreal space. In a conventional online conference system, the positional relationship of multiple participants (multiple HMD systems) who are close to each other in the real space may differ from the positional relationship of multiple avatars (multiple avatars corresponding to multiple participants, respectively) in the unreal space. In that case, the direction from which the voice of the other participants can be heard through the speaker of the HMD differs from the direction from which the voice of the other participants can be heard without going through the speaker of the HMD, which makes the participants feel uncomfortable and impairs their immersion.

[0046] FIG. 4(A) is a schematic diagram showing the arrangement of HMD systems (participants) in real space. In FIG. 4(A), the participant using the HMD system 400 is located to the right of the participant using the HMD system 100. FIG. 4(B) is a schematic diagram showing the arrangement of avatars in a non-real space. In FIG. 4(B), the avatar of the participant using the HMD system 400 is located in front of the avatar of the participant using the HMD system 100. In this case, the participant using the HMD system 100 hears the voice of the participant using the HMD system 400 from two directions, the right direction and the front direction. The voice of the participant using the HMD system 400 is heard from the right direction without going through the speaker of the HMD system 100, and is heard from the front direction via the speaker of the HMD system 100.

[0047] Therefore, in the first embodiment, as shown in FIG. 4C, an HMD system 400 is used. The avatar of the participant using the HMD system 100 is placed to the right (next to the right) of the avatar of the participant using the HMD system 100. In other words, for multiple participants who are close in the real space, the positional relationship of the multiple avatars in the unreal space is made to approximately match the positional relationship of the multiple participants in the real space. In this way, the participant using the HMD system 100 can hear the voice of the participant using the HMD system 400 only from the right, which reduces the sense of discomfort and improves the sense of immersion.

[0048] In addition, the positional relationship between the multiple participants in the real space is not necessarily determined in advance. Therefore, in the first embodiment, even if the positional relationship between the multiple participants in the real space is not determined in advance, the arrangement of each avatar in the unreal space can be suitably determined.

[0049] <Processing flow of online conference system> Fig. 5 is a sequence diagram showing a processing flow of the system (online conference system) of Fig. 1. Fig. 5 shows a processing flow when determining the seat (arrangement) of an avatar corresponding to a participant who newly joins an online conference. Here, it is assumed that a user of the HMD system 100 participates in the online conference.

[0050] In step S501, the HMD system 100 transmits a request to participate in an online conference and the identification number of the HMD system 100 to the server device 200.

[0051] In step S502, the server device 200 transmits to the HMD system 100 a confirmation request as to whether or not other HMD systems (other participants) exist around the HMD system 100 (the user of the HMD system 100) in real space.

[0052] In step S503, the HMD system 400 emits a beacon signal to the surroundings. The HMD system 400 may emit a beacon signal constantly or periodically, regardless of the timing of step S503. The HMD systems 100, 500, and 600 may also emit a beacon signal.

[0053] In step S504, the HMD system 100 checks (determines) whether or not another HMD system exists around the HMD system 100 in the real space.

[0054] Here, it is assumed that the HMD system 400 (the user of the HMD system 400) exists around the HMD system 100 (the user of the HMD system 100) in the real space. Therefore, in step S505, the HMD system 100 receives a beacon signal of the HMD system 400. As a result, the presence of the HMD system 400 around the HMD system 100 in the real space is detected. Then, the HMD system 100 detects the identification number of the HMD system 400 from the received beacon signal. Furthermore, the HMD system 100 detects the direction in which the HMD system 400 exists relative to the HMD system 100 in the real space and the distance between the HMD system 100 and the HMD system 400 in the real space from the received beacon signal.

[0055] In step S506, the HMD system 100 transmits the identification number of the HMD system 400 to the server device 200. This process is also a process of inquiring whether the HMD system 400 is participating in an online conference (the same online conference as the online conference in which the HMD system 100 is participating) or not.

[0056] In step S507, the server device 200 confirms (determines) from the received identification number of the HMD system 400 whether or not the HMD system 400 is participating in an online conference (the same online conference in which the HMD system 100 is participating).

[0057] Here, it is assumed that the HMD systems 400, 500, and 600 are participating in the online conference. Since the HMD system 400 is participating in the online conference, in step S508, the server device 200 notifies the HMD system 100 that the HMD system 400 is participating in the online conference.

[0058] In step S509, the HMD system 100 generates information (positional relationship information) regarding the positional relationship between the HMD system 100 and the HMD system 400 in real space from the direction and distance detected in step S505. For example, information indicating the relative position of the HMD system 400 with respect to the HMD system 100 is generated. The relative position is indicated, for example, by the direction and distance detected in step S505. Then, the HMD system 100 transmits the generated positional relationship information to the server device 200. Note that the format of the positional relationship information is not particularly limited, and for example, information indicating the relative position of the HMD system 100 with respect to the HMD system 400 may be generated.

[0059] In step S510, the server device 200 determines a seat for each avatar. Here, the seat for each avatar is determined based on the positional relationship information (the positional relationship between the HMD system 100 and the HMD system 400 in the real space) transmitted in step S509. For example, the seat for each avatar is determined so that the direction in which the avatar of the HMD system 400 exists relative to the avatar of the HMD system 100 in the unreal space is approximately the same as the direction in which the HMD system 400 exists relative to the HMD system 100 in the real space. This allows the latest seat information for each avatar to be obtained.

[0060] In step S511, the server device 200 transmits necessary information to the HMD systems 100, 400, 500, and 600. For example, the avatar images, CG conference room images, and latest seating information of the HMD systems 400, 500, and 600 are transmitted to the HMD system 100. The avatar images and latest seating information of the HMD system 100 are transmitted to the HMD systems 400, 500, and 600. This starts an online conference using the HMD systems 100, 400, 500, and 600. Then, position and orientation information, operation information, and audio data are transmitted and received as necessary. Note that the method of registering the avatars is not particularly limited, but it is assumed that the avatars of the HMD systems 100, 400, 500, and 600 are registered in advance in the server device 200.

[0061] In addition, when there are no other HMD systems around the HMD system 100 in the real space, the processes of steps S505 to S509 are omitted. The HMD system 100 transmits to the server device 200 that there are no other HMD systems around the HMD system 100 in the real space. Then, in step S510, the server device 200 determines the seats of the respective avatars without basing on the above positional relationship information.

[0062] Also, when the HMD system 400 is not participating in the online meeting, the processes of steps S508 and S509 are omitted. The server device 200 notifies the HMD system 100 that the HMD system 400 is not participating in the online meeting. Then, in step S510, the server device 200 determines the seats of the respective avatars without basing on the above positional relationship information.

[0063] <Processing Flow of HMD System> FIG. 6 is a flowchart showing the processing flow of the HMD system 100. FIG. 6 shows the processing flow when the HMD system 100 newly participates in an online meeting.

[0064] In step S601, the control unit 123 controls the communication unit 150 to transmit a request to participate in the online meeting and the identification number of the HMD system 100 to the server device 200. In step S602, the control unit 123 controls the communication unit 150 to receive from the server device 200 a confirmation request as to whether there are other HMD systems around the HMD system 100 in the real space.

[0065]

[0066] ​In step S603, the control unit 123 controls the participant detection unit 160 to check whether or not another HMD system exists around the HMD system 100 in the real space. If the participant detection unit 160 receives a beacon signal (if it is detected that another HMD system exists around the HMD system 100 in the real space), the process proceeds to step S605. If the participant detection unit 160 does not receive a beacon signal (if it is not detected that another HMD system exists around the HMD system 100 in the real space), the process proceeds to step S604.

[0067] In step S604, the control unit 123 controls the communication unit 150 to notify the server device 200 that no other HMD systems exist around the HMD system 100 in the real space.

[0068] In step S605, the control unit 123 detects the identification number of the discovered HMD system, the direction in which the discovered HMD system exists, and the distance to the discovered HMD system from the beacon signal received in step S603.

[0069] In step S606, the control unit 123 controls the communication unit 150 to transmit the identification number detected in step S605 (the identification number of the discovered HMD system) to the server device 200. This process is also a process of inquiring whether the discovered HMD system is participating in an online conference (the same online conference as the online conference in which the HMD system 100 is participating) or not. If there is a response to the inquiry that the discovered HMD system is participating in the online conference, the process proceeds to step S607. If there is a response to the inquiry that the discovered HMD system is not participating in the online conference, the process of FIG. 6 ends.

[0070] In step S607, the control unit 123 generates information (positional relationship information) regarding the positional relationship between the HMD system 100 and the discovered HMD system in real space, from the direction and distance detected in step S605.

[0071] In step S608, control unit 123 transmits to server device 200 the positional relationship information generated in step S607.

[0072] <Server device processing flow> Fig. 7 is a flowchart showing a processing flow of the server device 200. Fig. 7 shows a processing flow when determining a seat for an avatar corresponding to a participant who newly joins an online conference. It is assumed that a user of the HMD system 100 participates in the online conference.

[0073] In step S701, the control unit 211 controls the communication unit 204 to receive a request to participate in an online conference and the identification number of the HMD system 100 from the HMD system 100.

[0074] In step S702, the control unit 211 controls the communication unit 204 to issue a confirmation request as to whether or not another HMD system exists around the HMD system 100 in the real space. is transmitted to the HMD system 100.

[0075] In step S703, the control unit 211 controls the communication unit 204 to receive a response to the confirmation request in step S701 from the HMD system 100. The control unit 211 then confirms (determines) whether or not another HMD system exists around the HMD system 100 in the real space from the response to the confirmation request. If another HMD system is found (detected) around the HMD system 100 in the real space, an identification number of the found HMD system is received as a response to the confirmation request. If another HMD system is not found around the HMD system 100 in the real space, information indicating that another HMD system does not exist around the HMD system 100 in the real space is received as a response to the confirmation request. If an HMD system is found, the process proceeds to step S705, and if an HMD system is not found, the process proceeds to step S704.

[0076] In step S704, the control unit 211 determines the seat of each avatar, regardless of the positional relationship of the multiple HMD systems in the real space.

[0077] In step S705, the control unit 211 controls the participant management unit 212 to confirm (determine) whether or not the HMD system discovered by the HMD system 100 is participating in an online conference, based on the identification number received in step S703. Here, the online conference refers to an online conference in which the HMD system 100 is participating. If the discovered HMD system is participating in an online conference, the process proceeds to step S708, and if the discovered HMD system is not participating in an online conference, the process proceeds to step S706.

[0078] In step S706, the control unit 211 controls the communication unit 204 to notify the HMD system 100 that the HMD system discovered by the HMD system 100 is not participating in the online conference.

[0079] In step S707, the control unit 211 determines the seat of each avatar, regardless of the positional relationship of the multiple HMD systems in the real space.

[0080] In step S708, the control unit 211 controls the communication unit 204 to notify the HMD system 100 that the HMD system discovered by the HMD system 100 is participating in the online conference.

[0081] In step S709, the control unit 211 controls the communication unit 204 to receive, from the HMD system 100, information (positional relationship information) regarding the positional relationship between the HMD system 100 in real space and the discovered HMD system.

[0082] In step S710, the control unit 211 determines the seat of each avatar based on the positional relationship information received in step S709.

[0083] As described above, according to the first embodiment, when it is detected that another participant is present around a new participant in real space, information on the positional relationship of the participants in real space is acquired, and the placement of each avatar is determined based on the information. In this way, in a service that enables communication in a non-real space, even if the positional relationship of multiple participants in the real space is not predetermined, the placement of each avatar in the non-real space can be suitably determined.

[0084] Although an example has been described in which the HMD system 100 checks whether or not another HMD system exists around the HMD system 100 in real space in response to a check request transmitted from the server device 200, the present invention is not limited to this. For example, the server device 200 does not need to transmit a check request to the HMD system 100. The HMD system 100 may always or periodically check whether or not another HMD system exists around the HMD system 100 in real space.

[0085] Also, an example has been described in which the HMD system 100 inquires of the server device 200 as to whether or not the discovered HMD system is participating in an online conference, but the present invention is not limited to this. For example, when the HMD system 100 discovers another HMD system, the HMD system 100 may transmit positional relationship information to the server device 200 without making an inquiry. If the discovered HMD system is participating in an online conference, the server device 200 determines the seats of each avatar based on the positional relationship information. If the discovered HMD system is not participating in an online conference, the server device 200 determines the seats of each avatar without based on the positional relationship information.

[0086] Furthermore, the HMD system 100 (client device 102) may have the functionality of the server device 200 and provide online conference services to other HMD systems.

[0087] Also, an example in which a new participant (HMD system 100) searches for other participants (other HMD systems) from around the participant in real space has been described, but the present invention is not limited to this. For example, the existing participants (HMD systems 400, 500, 600) may be notified that a new participant has been added, and may search for other participants from around the participant in real space.

[0088] [Second embodiment] A second embodiment of the present invention will be described below. Note that, in the following, a description of the same points as in the first embodiment (for example, the same configuration and processing as in the first embodiment) will be omitted, and only points different from the first embodiment will be described.

[0089] In an online conference, a participant may instruct the movement of an avatar. If the avatar can move anywhere, the positional relationship of multiple avatars in the unreal space may deviate from the positional relationship of multiple participants in the real space for multiple participants who are close to each other in the real space. When the positional relationship is deviated, the direction in which the voice of another participant can be heard through the speaker of the HMD differs from the direction in which the voice of the other participant can be heard without going through the speaker of the HMD, causing the participant to feel uncomfortable and losing the sense of immersion. Therefore, in the second embodiment, the movement of the avatar is restricted based on the positional relationship information, and a predetermined notification is given to the participant who instructed the movement.

[0090] <Seat change restrictions> FIG. 8(A) is a schematic diagram showing the arrangement of HMD systems (participants) in real space. The participant using the HMD system 100 is located to the left of the participant using the HMD system 400. FIG. 8(B) is a schematic diagram showing the arrangement of avatars in unreal space. The avatar of the participant using the HMD system 100 is located to the left of the avatar of the participant using the HMD system 400. In other words, the positional relationship of the HMD systems 100 and 400 (avatars) in the unreal space matches the positional relationship of the HMD systems 100 and 400 (participants) in the real space. In this situation, assume that the participant using the HMD system 100 issues an instruction to move his / her avatar to the right of the avatar of the participant using the HMD system 400. If such a movement is permitted, the positional relationship of the HMD systems 100 and 400 (avatars) in the unreal space will not match (will not match substantially) the positional relationship of the HMD systems 100 and 400 (participants) in the real space. In the first embodiment, such a movement instruction is not followed, and a predetermined notice (for example, a notice that the avatar cannot be moved to the specified position (seat)) is given to the participant who instructed the movement. For example, a warning screen as shown in FIG. 8(C) is displayed on the display unit 112 of the HMD system 100.

[0091] <Processing flow of online conference system> Fig. 9 is a sequence diagram showing a processing flow of the online conference system according to the second embodiment. Fig. 9 shows a processing flow when an instruction to move an avatar is given during an online conference, but the instruction is not accepted and a predetermined notification is made. Here, it is assumed that a user of the HMD system 100 gives an instruction to move his / her avatar.

[0092] Input Devices In step S901, the HMD system 100 receives an instruction to move an avatar of the HMD system 100 from a user of the HMD system 100 (a seat change operation on the operation unit 170).

[0093] In step S902, the HMD system 100 transmits a seat change request to the server device 200. The seat change request includes information about a seat change operation. For example, the seat change request indicates a position after movement designated by a user of the HMD system 100.

[0094] In step S903, the server device 200 determines whether or not to move the avatar according to the seat change request in step S902 (seat change operation in step S901) based on the positional relationship information related to the HMD system 100. The server device 200 checks (determines) whether or not the movement of the avatar to the specified position will cause the above-mentioned inconsistency (inconsistency between the positional relationship in the real space and the positional relationship in the unreal space). If no inconsistency occurs, the server device 200 determines to move the avatar, and if a inconsistency occurs, determines not to move the avatar. If the positional relationship information related to the HMD system 100 has not been acquired in advance, there is no other HMD system around the HMD system 100 in the real space, so the server device 200 determines to move the avatar.

[0095] For example, assume that in the real space, an HMD system 400 exists around the HMD system 100. In the non-real space, the direction in which the avatar of the HMD system 400 exists with respect to the avatar of the HMD system 100 is defined as the first direction, and in the real space, the direction in which the HMD system 400 exists with respect to the HMD system 100 is defined as the second direction. The server device 200 checks (determines) whether the first direction substantially coincides with the second direction even when moving the avatar according to the seat change request. Then, when the first direction substantially coincides with the second direction, the server device 200 determines to move the avatar, and when the first direction no longer substantially coincides with the second direction, the server device 200 determines not to move the avatar.

[0096] Here, assume that it is determined not to move the avatar. Therefore, in step S904, the server device 200 notifies the HMD system 100 that the avatar cannot be moved. When it is determined that the avatar is to be moved, the server device 200 does not notify the HMD system 100 and moves the avatar according to the seat change request. When the server device 200 updates the seat information by moving the avatar, it transmits the latest seat information to each HMD system.

[0097] In step S905, the HMD system 100 displays a warning screen indicating that the avatar cannot be moved to the specified position on the display unit 112.

[0098] <Processing Flow of HMD System> FIG. 10 is a flowchart showing the processing flow of the HMD system 100. FIG. 10 shows the processing flow when a seat change operation is performed on the HMD system 100.

[0099] When a seat change operation is performed on the HMD system 100, in step S1001, the control unit 123 transmits a seat change request to the server device 200 by controlling the communication unit 150.

[0100] In step S1002, the control unit 123 checks (determines) whether or not a response to the seat change request (notification that the avatar cannot be moved) has been received. If a response to the seat change request has been received, the process proceeds to step S1003, and if a response to the seat change request has not been received, the process of FIG. 10 ends.

[0101] In step S1003, control unit 123 displays on display unit 112 a warning screen indicating that the avatar cannot be moved to the specified position.

[0102] <Server device processing flow> Fig. 11 is a flowchart showing a processing flow of the server device 200. Fig. 11 shows a processing flow when a seat change operation is performed on the HMD system 100. It is assumed that positional relationship information related to the HMD system 100 has been acquired in advance.

[0103] When a seat change operation is performed on the HMD system 100, the control unit 211 controls the communication unit 204 to receive a seat change request from the HMD system 100 in step S1101.

[0104] In step S1102, the control unit 211 determines whether or not to move the avatar in accordance with the seat change request (seat change operation) in step S1101. This process corresponds to step S903. If the avatar is to be moved, the process proceeds to step S1103, and if the avatar is not to be moved, the process proceeds to step S1104.

[0105] In step S1103, the control unit 211 moves the avatar in accordance with the seat change request in step S1101.

[0106] In step S1104, the control unit 211 controls the communication unit 204 to notify the HMD system 100 that the avatar cannot be moved.

[0107] As described above, according to the second embodiment, the movement of the avatar is restricted, and a predetermined notification is given to the participant who instructed the movement. By restricting the movement of the avatar, it is possible to suppress a mismatch between the positional relationship of the multiple avatars in the unreal space and the positional relationship of the multiple participants in the real space for multiple participants who are close to each other in the real space. And, by giving a predetermined notification, it is possible to suppress the occurrence of a sense of incongruity and discomfort caused by the restriction of the movement of the avatar.

[0108] Although an example of restricting the movement of the avatar has been described, it is also possible to not restrict the movement of the avatar, and to send a predetermined notification (for example, a notification that the sense of realism or immersion will be reduced) to the participant who instructed the movement. This will prevent situations in which a participant (avatar) cannot get close to other participants (other avatars) with whom he or she is close. Also, each participant can participate in the online conference from a viewpoint of their choice. For multiple participants who are close to each other in real space, the positional relationship of multiple avatars in the unreal space can be calculated by the positional relationship in real space. Participants who wish to align their avatars with the other participants can be prompted to cancel the movement of their avatars.

[0109] [Third embodiment] A third embodiment of the present invention will be described below. Note that, in the following, a description of the same points as in the first embodiment (for example, the same configuration and processing as in the first embodiment) will be omitted, and only points different from the first embodiment will be described.

[0110] When the unreal space is a VR space, the avatars are also displayed in the participant's HMD system as other participants who are close to the participant in the real space. In this case, for multiple participants who are close to each other in the real space, even if the positional relationship of multiple avatars in the unreal space is slightly different from the positional relationship of multiple participants in the real space, there is little sense of incongruity. For example, if the direction in which multiple avatars are lined up in the unreal space is approximately the same as the direction in which multiple participants are lined up in the real space, there is little sense of incongruity.

[0111] When the unreal space is an MR space, the avatars of other participants who are near the participant in the real space are not displayed in the participant's HMD system. In the case of the optical see-through method, the participant can directly see the other participants who are near the participant. In the case of the video see-through method, a real image (an image of the real space) capturing an area including other participants who are near the participant is displayed in the participant's HMD system. Therefore, the participant can see other participants who are near the participant in the real space in the real image. In this case, if the above-mentioned two positional relationships (positional relationship in the real space and positional relationship in the unreal space) are different, the avatars of participants in remote locations may overlap with other participants who are near the participant in the real space.

[0112] Therefore, in the third embodiment, the method of determining the arrangement of each avatar is different between when the unreal space is a VR space and when the unreal space is an MR space. For example, when the unreal space is a VR space, the arrangement of each avatar is determined based on the direction in which multiple participants who are close to each other line up in the real space. When the unreal space is an MR space, the seat (arrangement) of each avatar is determined based on the direction in which multiple participants who are close to each other line up in the real space and the distance (spacing) between the multiple participants.

[0113] <Avatar seat> Fig. 12(A) is a schematic diagram showing the arrangement of avatars in a virtual space, in which users of HMD systems 400, 500, and 600 are participating in an online conference.

[0114] Fig. 12(B) is a schematic diagram showing the arrangement of the HMD system in real space. In Fig. 12(B), the user of the HMD system 100 is located to the right of the user of the HMD system 400. It is shown that the user using the HMD system 100 is about to participate in an online conference.

[0115] FIG. 12(C) is a schematic diagram showing the arrangement of avatars in an unreal space. Suppose the unreal space is a VR space, and the user of the HMD system 100 attempts to participate in an online conference in the situations of FIGS. 12(A) and 12(B). In this case, the seat of the avatar of the HMD system 100 may be determined based on the direction (rightward) in which the HMD system 100 (the user of the HMD system 100) is present relative to the HMD system 400 (the user of the HMD system 400) in the real space. Therefore, the avatar of the HMD system 100 is arranged to the right of the avatar of the HMD system 400. For example, the avatar of the HMD system 100 is arranged within the range surrounded by the dashed line in FIG. 12(C). In the example of FIG. 12(C), The avatar of the HMD system 100 is placed in an empty seat, avoiding the seat of the avatar of the HMD system 600.

[0116] Fig. 12(D) is a schematic diagram showing the arrangement of avatars in an unreal space. Assume that the unreal space is an MR space, and the user of the HMD system 100 attempts to participate in an online conference in the situation of Figs. 12(A) and 12(B). In this case, the seat of the avatar of the HMD system 100 may be determined based on the direction in which the HMD system 100 exists relative to the HMD system 400 in the real space, and the distance between the HMD system 400 and the HMD system 100 in the real space. Therefore, the avatar of the HMD system 100 is arranged to the right of the avatar of the HMD system 400. Then, the avatar of the HMD system 600 that was located to the right of the avatar of the HMD system 400 is moved to another seat.

[0117] <Server device processing flow> Fig. 13 is a flowchart showing a processing flow of the server device 200. Here, it is assumed that the processing of Fig. 13 is performed in step S710 of Fig. 7. It is assumed that the users of the HMD systems 400, 500, and 600 are participating in an online conference, and that the user of the HMD system 100 newly participates in the online conference. It is also assumed that the HMD system 400 (the user of the HMD system 400) exists around the HMD system 100 (the user of the HMD system 100) in the real space.

[0118] In step S1301, the control unit 211 confirms (determines) whether the unreal space is a VR space or an MR space. If the unreal space is a VR space, the process proceeds to step S1302, and if the unreal space is an MR space, the process proceeds to step S1303.

[0119] In step S1302, the control unit 211 determines a seat for each avatar based on the direction in real space in which the HMD system 100 exists relative to the HMD system 400. For example, the location of each avatar is determined so that the direction in unreal space in which the avatar of the HMD system 100 exists relative to the avatar of the HMD system 400 is approximately the same as the direction in real space in which the HMD system 100 exists relative to the HMD system 400.

[0120] In step S1303, the control unit 211 determines the seat of each avatar based on the direction in which the HMD system 100 exists relative to the HMD system 400 in the real space and the distance between the HMD system 400 and the HMD system 100 in the real space. For example, the direction in which the avatar of the HMD system 100 exists relative to the avatar of the HMD system 400 in the unreal space is set as a first direction, and the direction in which the HMD system 100 exists relative to the HMD system 400 in the real space is set as a second direction. The distance between the avatar of the HMD system 400 and the avatar of the HMD system 100 in the unreal space is set as a first distance, and the distance between the HMD system 400 and the HMD system 100 in the real space is set as a second distance. Then, the arrangement of each avatar is determined so that the first direction approximately matches the second direction and the first distance approximately matches the second distance.

[0121] As described above, according to the third embodiment, the method of determining the location of each avatar is switched so that the location of each avatar is determined by different methods when the unreal space is a VR space and when the unreal space is an MR space. This makes it possible to more appropriately determine the location of each avatar in each of the VR space and the MR space.

[0122] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It may also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0123] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) a determination means for determining, in a non-real space, a placement of a plurality of avatars corresponding to a plurality of participants in a service that enables communication in the non-real space; an acquisition means for acquiring information on a positional relationship between a first participant and a second participant in a real space when it is detected that a second participant of the service exists around a first participant who newly participates in the service in the real space; having The determining means determines a location of the plurality of avatars based on the information. 23. An information processing apparatus comprising: (Configuration 2) The information processing device is a server device to which a plurality of client devices, each of which is used by one of the participants, is connected. 2. The information processing device according to configuration 1. (Configuration 3) The information processing device is an information processing device used by any one of the plurality of participants. 2. The information processing device according to configuration 1. (Configuration 4) A detection means for detecting other participants of the service from around the participant using the information processing device in the real space. Further having 4. The information processing device according to configuration 3. (Configuration 5) The determining means determines the arrangement of the plurality of avatars such that a direction in which the avatar of the second participant exists relative to the avatar of the first participant in the unreal space is approximately the same as a direction in which the second participant exists relative to the first participant in the real space. 5. The information processing device according to any one of configurations 1 to 4. (Configuration 6) The system further includes a control means for controlling, when the first participant issues an instruction to move the avatar of the first participant, to give a predetermined notification to the first participant based on the information. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) the determining means determines a layout of the plurality of avatars such that a first direction in which an avatar of the second participant exists relative to an avatar of the first participant in the unreal space substantially coincides with a second direction in which the second participant exists relative to the first participant in the real space; The control means performs control so as to give the predetermined notification to the first participant when the first direction does not substantially match the second direction as a result of moving the avatar of the first participant in accordance with the instruction of the first participant. 7. The information processing device according to configuration 6. (Configuration 8) The non-real space is a virtual reality space or a mixed reality space. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) The determining means determines the arrangement of the plurality of avatars in different ways depending on whether the unreal space is the virtual reality space or the mixed reality space. 9. The information processing device according to configuration 8. (Configuration 10) The determining means is If the non-real space is the virtual reality space, determining an arrangement of the plurality of avatars based on an orientation of the second participant relative to the first participant in the real space; If the unreal space is the mixed reality space, determining a location of the plurality of avatars based on a direction in which the second participant is present relative to the first participant in the real space and a distance between the first participant and the second participant in the real space. 10. The information processing device according to configuration 9. (Configuration 11) The determining means is if the unreal space is the virtual reality space, determining an arrangement of the plurality of avatars such that a first direction in which an avatar of the second participant exists relative to an avatar of the first participant in the unreal space substantially coincides with a second direction in which the second participant exists relative to the first participant in the real space; If the unreal space is the mixed reality space, determining a layout of the plurality of avatars such that the first direction is approximately equal to the second direction and a distance between an avatar of the first participant and an avatar of the second participant in the unreal space is approximately equal to a distance between the first participant and the second participant in the real space. 11. The information processing device according to configuration 10. (method) a determining step of determining, in the unreal space, placement of a plurality of avatars respectively corresponding to a plurality of participants of a service enabling communication in the unreal space; an acquisition step of acquiring information on a positional relationship between a first participant and the second participant in the real space when it is detected that a second participant of the service exists around a first participant who has newly joined the service in the real space; having In the determining step, a layout of the plurality of avatars is determined based on the information. 23. An information processing method comprising: (program) 12. A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 11. (medium) 12. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 11. [Explanation of symbols]

[0124] 102: client device 120: CPU 200: Server device 201: CPU

Claims

1. a determining means for determining, in the unreal space, the placement of a plurality of avatars corresponding to a plurality of participants in a service that enables communication in the unreal space; an acquisition means for acquiring information regarding a positional relationship between a first participant and a second participant in the real space when it is detected that a second participant of the service is present around the first participant who has newly joined the service in the real space; a control means for controlling, when the first participant issues an instruction to move the avatar of the first participant, to issue a predetermined notification to the first participant based on the information; and The determining means determines the arrangement of the plurality of avatars based on the information.

1. An information processing device comprising:

2. The information processing device is a server device to which a plurality of client devices, each of which is used by the plurality of participants, are connected.

2. The information processing apparatus according to claim 1, wherein:

3. The information processing device is an information processing device used by any one of the plurality of participants.

2. The information processing apparatus according to claim 1, wherein:

4. A detection means for detecting other participants of the service from around the participant using the information processing device in the real space. Further having 4. The information processing apparatus according to claim 3,

5. The determining means determines the arrangement of the plurality of avatars so that a direction in which the avatar of the second participant exists relative to the avatar of the first participant in the unreal space is approximately the same as a direction in which the second participant exists relative to the first participant in the real space.

2. The information processing apparatus according to claim 1, wherein:

6. the determining means determines the arrangement of the plurality of avatars so that a first direction in which an avatar of the second participant exists relative to an avatar of the first participant in the unreal space substantially coincides with a second direction in which the second participant exists relative to the first participant in the real space; The control means controls the system so as to issue the predetermined notification to the first participant when the first direction does not substantially coincide with the second direction as a result of moving the avatar of the first participant in accordance with the instruction from the first participant.

2. The information processing apparatus according to claim 1, wherein:

7. The unreal space is a virtual reality space or a mixed reality space.

2. The information processing apparatus according to claim 1, wherein:

8. The determining means determines the arrangement of the plurality of avatars in different ways depending on whether the unreal space is the virtual reality space or the mixed reality space.

8. The information processing apparatus according to claim 7,

9. The determining means If the unreal space is the virtual reality space, determining a location of the plurality of avatars based on an orientation of the second participant relative to the first participant in the real space; If the unreal space is the mixed reality space, the arrangement of the plurality of avatars is determined based on the direction in which the second participant is present relative to the first participant in the real space and the distance between the first participant and the second participant in the real space.

9. The information processing apparatus according to claim 8,

10. The determining means If the unreal space is the virtual reality space, determining the arrangement of the plurality of avatars so that a first direction in which an avatar of the second participant exists relative to an avatar of the first participant in the unreal space substantially coincides with a second direction in which the second participant exists relative to the first participant in the real space; If the unreal space is the mixed reality space, the arrangement of the multiple avatars is determined so that the first direction substantially coincides with the second direction, and the distance between the avatar of the first participant and the avatar of the second participant in the unreal space substantially coincides with the distance between the first participant and the second participant in the real space.

10. The information processing apparatus according to claim 9,

11. a determining step of determining, in the unreal space, the placement of a plurality of avatars corresponding to a plurality of participants in a service that enables communication in the unreal space; an acquisition step of acquiring information regarding a positional relationship between a first participant and a second participant in the real space when it is detected that a second participant of the service is present around the first participant who has newly joined the service in the real space; a control step of performing control so that a predetermined notification is given to the first participant based on the information when the first participant gives an instruction to move the avatar of the first participant; and In the determining step, a layout of the plurality of avatars is determined based on the information.

1. An information processing method comprising:

12. A computer as each of the means of the information processing device according to any one of claims 1 to 10. Program to make it work.

Images