Information processing device, information processing method, and program

Abstract

To provide a technology capable of appropriately arranging each avatar within a non-real space even in the case that a positional relation of a plurality of participants is not predetermined in a real space in a service for enabling communication within the non-real space.SOLUTION: According to the present invention, an information processing device includes determination means for determining arrangement within a non-real space of a plurality of avatars respectively corresponding to a plurality of participants of a service for enabling communication within the non-real space, and acquisition means for acquiring information in a positional relation between a first participant and a second participant within a real space in the case of detecting that the second participant of the service exists around the first participant who newly participates in the service within the real space. The determination means determines arrangement of the plurality of avatars on the basis of the information.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and a storage medium, and particularly to a service that enables communication in an unreal space.

Background Art

[0002] A system for online meetings in an unreal space such as a virtual reality space (VR space) or a mixed reality space (MR space) has been proposed. In such a system, participants in an online meeting are represented by virtual avatars. For example, a participant wears a head-mounted display (HMD) on their head. The HMD displays a meeting room that is a three-dimensional unreal space, and avatars of other participants are displayed in the meeting room. Then, the voices of other participants are output from the speakers of the HMD so that they can be heard from the direction where the avatars of the other participants are located. Thereby, a participant can experience as if actually having a meeting with other participants.

[0003] Patent Document 1 discloses a technique for displaying, on the HMD of a user sitting in a car seat, an avatar of another user who is at a location away from the car as if sitting in another seat of the car.

[0004] Patent Document 2 discloses a technique for determining the arrangement of a plurality of avatars (a plurality of avatars corresponding to a plurality of users respectively) in a virtual reality space based on the positional relationship of a plurality of users in the real space.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

[0006] However, in conventional systems for online meetings in a virtual space, the relative positions of multiple participants who are close together in the real world may differ from the relative positions of multiple avatars (each corresponding to a different participant) in the virtual space. In such cases, the direction from which a participant's voice is heard through the HMD's speakers may differ from the direction from which their voice is heard without the HMD's speakers, causing participants to feel uncomfortable and diminishing their sense of immersion.

[0007] Such problems are not limited to systems for online meetings in a non-realistic space, but can arise in various services that enable communication in a non-realistic space. For example, in the technology disclosed in Patent Document 1, the avatars of two users who are side by side in the real world may end up side by side in a car. In the technology disclosed in Patent Document 2, a predetermined fixed positional relationship is considered for the positional relationship of multiple users in the real world, but the above problem arises when the positional relationship of multiple users in the real world is not predetermined.

[0008] The present invention aims to provide a technology that enables communication in a non-real space, which allows for the appropriate determination of the placement of each avatar in a non-real space, even when the relative positions of multiple participants in the real space are not predetermined. [Means for solving the problem]

[0009] A first aspect of the present invention includes a determination means for determining the placement of a plurality of avatars in a non-real space, each corresponding to a plurality of participants in a service that enables communication in a non-real space, and an acquisition means for acquiring information regarding the positional relationship between the first participant and the second participant in the real space when it is detected that a second participant of the service is present around a first participant who has newly joined the service in the real space. and a control means that controls the first participant to give a predetermined notification to the first participant based on the information when the first participant gives an instruction to move the first participant's avatar.The information processing device has a determination means that determines the arrangement of the plurality of avatars based on the information.

[0010] A second aspect of the present invention includes a determination step of determining the placement of a plurality of avatars in a non-real space, each corresponding to a plurality of participants in a service that enables communication in a non-real space, and an acquisition step of obtaining information regarding the positional relationship between the first participant and the second participant in the real space when it is detected that a second participant of the service is present around a first participant who has newly joined the service in the real space. and a control step which controls the first participant to give a predetermined notification to the first participant based on the information when the first participant gives an instruction to move the first participant's avatar. The information processing method is characterized by having the above information, and in the decision step, determining the arrangement of the plurality of avatars based on the above information.

[0011] A third aspect of the present invention is a program for causing a computer to function as each of the means of the information processing apparatus described above. A fourth aspect of the present invention is a computer-readable storage medium that stores a program for causing a computer to function as each of the means of the information processing apparatus described above. [Effects of the Invention]

[0012] According to the present invention, in a service that enables communication in a non-real space, even if the relative positions of multiple participants in the real space are not predetermined, the placement of each avatar in the non-real space can be suitably determined. [Brief explanation of the drawing]

[0013] [Figure 1] Diagram showing the configuration of the online meeting system according to the first to third embodiments. [Figure 2] Diagram showing the configuration of the HMD system according to the first to third embodiments. [Figure 3] Diagram showing the configuration of the server device according to the first to third embodiments. [Figure 4] A diagram showing the arrangement of the HMD system according to the first embodiment. [Figure 5] Figure showing the processing flow of the online conferencing system according to the first embodiment [Figure 6] Figure showing the processing flow of the HMD system according to the first embodiment [Figure 7] Figure showing the processing flow of the server device according to the first embodiment [Figure 8] Figure showing the arrangement etc. of the HMD system according to the second embodiment [Figure 9] Figure showing the processing flow of the online conferencing system according to the second embodiment [Figure 10] Figure showing the processing flow of the HMD system according to the second embodiment [Figure 11] Figure showing the processing flow of the server device according to the second embodiment [Figure 12] Figure showing the processing flow of the HMD system according to the third embodiment [Figure 13] Figure showing the processing flow of the server device according to the third embodiment

MODE FOR CARRYING OUT THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described in detail based on the accompanying drawings. Note that each of the embodiments described below is merely an example, and configurations obtained by appropriately modifying or changing the configurations of each embodiment within the scope of the gist of the present invention are also included in the present invention. Configurations obtained by appropriately combining the configurations of each embodiment are also included in the present invention.

[0015] [First Embodiment] [Configuration of the System] A first embodiment of the present invention will now be described. Figure 1 is a schematic diagram showing the general configuration of a system according to the first embodiment. The system in Figure 1 is a system for communication in a non-real space such as a virtual reality space (VR space) or a mixed reality space (MR space), and is, for example, a system for online meetings in a non-real space (online meeting system). A VR space is a three-dimensional space (world) created with computer graphics (CG). An MR space is a three-dimensional space similar to the real space. Virtual objects (e.g., avatars) can be placed in an MR space, but virtual objects cannot be placed in the real space. In other words, when viewing the real space without using a display device that displays virtual objects, the real space does not become an MR space, and when viewing the real space using a display device that displays virtual objects, the real space becomes an MR space. An MR space can also be considered an augmented reality (AR) space. An avatar is a character (user character) that represents the user (user of the above system) in a non-real space, and is created with computer graphics.

[0016] The system in Figure 1 includes HMD systems 100, 400, 500, and 600, a server device 200, and a network 300.

[0017] Each of the HMD systems 100, 400, 500, and 600 is used by participants in a service that enables communication in a non-real space (for example, participants in an online meeting in a non-real space). Each of the HMD systems 100, 400, 500, and 600 has a head-mounted display (HMD), which is a head-worn display device, and a client device, which is an information processing device. The four HMD systems 100, 400, 500, and 600 correspond to four participants, respectively. Each of the HMD systems 100, 400, 500, and 600 is communicably connected to a server device 200 via a network 300.

[0018] In the first embodiment, an example where the client device and the HMD are separate devices is described, but the client device and the HMD may be integrally configured. Also, the number of participants is not particularly limited and may be more than or less than four. The connection between the HMD systems 100, 400, 500, 600 and the server device 200 via the network 300 may be a wired connection or a wireless connection.

[0019] The server device 200 is a server device (information processing device) that provides a service that enables communication in a virtual space. Note that the number of server devices is not particularly limited, and the above service may be provided by a plurality of server devices performing processing in parallel.

[0020] In the first embodiment, an example of an online meeting in a virtual space is described. When the virtual space is a VR space, in each of the HMD systems 100, 400, 500, 600, an image in which an avatar is placed in a CG conference room is displayed. When the virtual space is an MR space, in each of the HMD systems 100, 400, 500, 600, the avatar is displayed so that the avatar is placed in the actual conference room. When the virtual space is an MR space, in each of the HMD systems 100, 400, 500, 600, the HMD may function as a video see-through type HMD or an optical see-through type HMD. In the first embodiment, an example where the HMD functions as a video see-through type HMD is described. When the virtual space is an MR space, a participant can see other participants (actual participants, not avatars) near the participant in the real space. In the HMD system (HMD) of a participant, the avatar of another participant near the participant in the real space is not displayed, and only the avatar of another participant at a location away from the participant in the real space is displayed.

[0021] <Configuration of HMD System> FIG. 2 is a block diagram showing the configuration of the HMD system 100. The HMD system 100 includes an HMD 101, a client device 102, and an input device 103. The HMD 101 and the client device 102 are connected to each other by wire or wirelessly so as to be communicable, and the client device 102 and the input device 103 are connected to each other by wire or wirelessly so as to be communicable. The HMD systems 400, 500, and 600 have the same configuration as the HMD system 100. In the first embodiment, an example in which the HMD 101, the client device 102, and the input device 103 are separate devices will be described, but at least two of these three devices may be integrally configured.

[0022] <Configuration of HMD> The HMD 101 includes an imaging unit 111, a display unit 112, a position and orientation detection unit 113, an audio input unit 114, and an audio output unit 115.

[0023] The imaging unit 111 images the external world (real space). For example, the imaging unit 111 includes an image sensor that converts light into an electrical signal, and an objective optical system that guides the light of the external world to the image sensor.

[0024] The display unit 112 displays the video (video corresponding to the field of view of the user wearing the HMD 101) generated by the client device 102. Note that the HMD 101 may include a display unit for the right eye and a display unit for the left eye. Then, a stereoscopic video (video for the right eye and video for the left eye) that enables stereoscopic viewing may be displayed by the two display units.

[0025] The position and orientation detection unit 113 detects the position and orientation (position and posture) of the HMD 101 (the user wearing the HMD 101) in real space. For example, the position and orientation detection unit 113 has various sensors such as an accelerometer, an angular velocity sensor, a geomagnetic sensor, and a GPS sensor. The detection results of the position and orientation detection unit 113 are used to control the display by the display unit 112 and to control the posture of the avatar corresponding to the user (the user wearing the HMD 101). For example, the part of the non-real space displayed on the display unit 112 is changed according to the change in the user's position and orientation. This gives the user the feeling of being in a non-real space. Also, the posture of the avatar corresponding to the user is changed according to the change in the user's posture. This enables communication using posture.

[0026] The audio input unit 114 is a microphone that acquires sound. The audio input unit 114 may also be an input terminal to which an audio input device such as a microphone is connected.

[0027] The audio output unit 115 is a speaker that outputs sound. The audio output unit 115 may also be an output terminal to which an audio output device such as a speaker, headphones, or earphones is connected.

[0028] Alternatively, a tablet device or other display device may be used instead of the HMD101. Any portable display device capable of displaying images corresponding to the user's field of view may be used.

[0029] <Configuration of input devices> The input device 103 includes an operation unit 170. The operation unit 170 receives user operations (operations by the user) and outputs an operation signal corresponding to the user operation performed. In response to the operation (operation signal), for example, the avatar corresponding to the user is controlled to perform a specific action (gesture or movement). The user can also give instructions to switch between the VR space and the MR space. The input device 103 is, for example, a joystick that outputs an operation signal according to the direction and magnitude of the lever's tilt. Note that the input device 103 is not limited to a joystick, but may be, for example, a mouse, a keyboard, or a gamepad.

[0030] <Client device configuration> The client device 102 includes a CPU 120, ROM 130, RAM 140, a communication unit 150, and a participant detection unit 160. The CPU 120 enables the implementation of an image acquisition unit 121, a position and orientation extraction unit 122, a control unit 123, an audio acquisition unit 124, an audio transmission unit 125, and an input processing unit 126. The client device 102 is an information processing device with excellent arithmetic processing and graphics rendering capabilities, such as a smartphone, a personal computer, or a workstation.

[0031] The CPU 120 performs various processes (various controls) on the client device 102. The ROM 130 is non-volatile memory. The ROM 130 stores the OS (operating system) program, various application programs, and other information. For example, the ROM 130 stores the program for an application to conduct online meetings (online meeting execution application). The RAM 140 is volatile memory and is used, for example, as a workspace for the CPU 120.

[0032] The image acquisition unit 121 acquires the real-world image (an image of the real space) obtained by the imaging unit 111.

[0033] The position and orientation extraction unit 122 acquires the position and orientation (position and orientation information) detected by the position and orientation detection unit 113. The position and orientation extraction unit 122 may also detect markers placed in real space from the real image acquired by the image acquisition unit 121. Then, based on the marker detection results, the position and orientation extraction unit 122 may detect the position and orientation of the HMD 101 (user) in real space.

[0034] The control unit 123 executes the online meeting execution application (the program for the online meeting execution application) stored in the ROM 130 and performs various processes for conducting an online meeting. For example, the control unit 123 transmits the position and orientation information acquired by the position and orientation extraction unit 122 to the server device 200 to share it with other participants in the online meeting. To conduct the online meeting, the control unit 123 receives avatar images (3D images of avatars) and CG conference room images (3D images of CG conference rooms) of other participants from the server device 200 and stores them in the RAM 140. The control unit 123 also receives position and orientation information of other avatars (other participants) and seat information for each avatar from the server device 200. The control unit 123 communicates with the server device 200 by controlling the communication unit 150.

[0035] If the non-real space is a VR space, the control unit 123 generates online meeting video (video data) based on the avatar images of other participants, the CG conference room image, the position and orientation information of other avatars, the seating information of each avatar, and the user's position and orientation information. If the non-real space is an MR space, the control unit 123 does not receive the CG conference room image, and instead generates online meeting video (video data) using real-world images acquired by the image acquisition unit 121. The control unit 123 then controls the display unit 112 to display the generated video. For example, the control unit 123 generates video by overlaying avatar images on the CG conference room image or real-world image. The control unit 123 uses the position and orientation information of each avatar. The system is controlled based on information, with each avatar's position controlled based on seat information (information indicating the seat). The generated video is a first-person perspective video, and the user's avatar is not displayed.

[0036] The voice acquisition unit 124 acquires the voice (voice data) obtained by the voice input unit 114. The control unit 123 transmits the voice (voice data) obtained by the voice acquisition unit 124 to the server device 200 to communicate it to other participants. The control unit 123 receives the voice data of other participants (voice data obtained by other participants' HMDs (voice input units)) from the server device 200. Then, the control unit 123 generates voice (voice data) to be output from the voice output unit 115 based on the received voice data. The voice transmission unit 125 transmits the voice data generated by the control unit 123 to the voice output unit 115, and the voice output unit 115 outputs voice according to the voice data transmitted from the voice transmission unit 125. As a result, the voice output unit 115 outputs voice so that the voices of other participants can be heard from the direction in which the other participants' avatars are located.

[0037] The communication unit 150 is a communication interface that communicates with external devices via wired or wireless means. The communication method (communication protocol) is not particularly limited, but for example, the communication unit 150 communicates with external devices using a so-called wireless LAN in accordance with the IEEE 802.11 standard.

[0038] The participant detection unit 160 detects other participants from around the user in real space. For example, the participant detection unit 160 detects other client devices (client devices of other participants) using Bluetooth®. The detection of other client devices can also be considered as the detection of other HMD systems (HMD systems of other participants). When other participants are present around the user in real space, the participant detection unit 160 receives beacon signals emitted from other HMD systems (other client devices). The control unit 123 then detects the identification number of the other HMD system, the direction in which the other HMD system is located, and the distance to the other HMD system from the beacon signals received by the participant detection unit 160. If there are multiple other participants around the user in real space, multiple combinations of identification number, direction, and distance are detected. The participant detection unit 160 also emits beacon signals so that other HMD systems (participant detection units) can detect the HMD system 100. The control unit 123 transmits the detected information to the server device 200. Furthermore, the method for detecting other participants is not particularly limited; for example, other HMD systems may be detected from images acquired by the image acquisition unit 121.

[0039] The input processing unit 126 acquires the operation signal output from the operation unit 170. The control unit 123 controls the operation of the avatar corresponding to the user in accordance with the operation signal acquired by the input processing unit 126. The control unit 123 transmits the operation signal acquired by the input processing unit 126 to the server device 200 to share it with other participants of the online meeting. The control unit 123 receives operation signals from other participants (operation signals obtained by the input devices (operation units) of other participants) from the server device 200. Then, the control unit 123 controls the operation of other avatars (avatars corresponding to other participants) in accordance with the received operation signals. The control unit 123 may switch between the VR space and the MR space in accordance with the operation signal acquired by the input processing unit 126.

[0040] <Server device configuration> Figure 3 is a block diagram showing the configuration of the server device 200. The server device 200 includes a CPU 201, ROM 202, RAM 203, and a communication unit 204. The CPU 201 enables the control unit 211 and the participant management unit 212.

[0041] The CPU 201 performs various processes (various controls) of the server device 200. The ROM 202 is ROM202 is non-volatile memory. ROM202 stores the OS (operating system) program, various application programs, and other information. For example, ROM202 stores the application program for providing online meeting services to each HMD system (online meeting provision application), and online meeting participant information. Participant information is, for example, a list showing the HMD system identification number for each participant. RAM203 is volatile memory and is used, for example, as a workspace for CPU201.

[0042] The control unit 211 executes the online meeting provision application (the program for the online meeting provision application) stored in the ROM 202 and performs various processes to provide online meeting services to each HMD system. For example, the control unit 211 transmits avatar images of other participants and CG conference room images to each HMD system (each client device). The control unit 211 also transmits and receives position and orientation information and operation information to and from each HMD system (each client device) so that the position and orientation information and operation information of each participant are shared among multiple participants. The control unit 211 transmits seat information for each avatar to each HMD system (each client device). The control unit 211 transmits and receives audio data to and from HMD systems (client devices) in order to transmit participants' audio data to other participants. For example, the control unit 211 transmits audio data received from HMD system 100 to HMD systems 400, 500, and 600. The control unit 211 communicates with each HMD system (each client device) by controlling the communication unit 204.

[0043] The participant management unit 212 manages the HMD system identification number and the seat where the avatar sits in the online meeting room for each participant in the online meeting.

[0044] The communication unit 204 is a communication interface that communicates with external devices via wired or wireless connection. The communication method (communication protocol) is not particularly limited, but for example, the communication unit 204 communicates with external devices using a so-called wireless LAN in accordance with the IEEE 802.11 standard.

[0045] <Avatar's seating arrangement> The participant management unit 212 determines the seating arrangement for each avatar in the online meeting. Determining the seating arrangement for each avatar in the online meeting can also be seen as determining the placement of each avatar in the non-real space. In conventional online meeting systems, the relative positions of multiple participants (multiple HMD systems) who are close to each other in the real space may differ from the relative positions of multiple avatars (multiple avatars corresponding to each participant) in the non-real space. In such cases, the direction from which the voices of other participants are heard through the HMD speakers may differ from the direction from which their voices are heard without the HMD speakers, causing participants to feel uncomfortable and diminishing their sense of immersion.

[0046] Figure 4(A) is a schematic diagram showing the placement of HMD systems (participants) in real space. In Figure 4(A), the participant using HMD system 400 is to the right of the participant using HMD system 100. Figure 4(B) is a schematic diagram showing the placement of avatars in non-real space. In Figure 4(B), the avatar of the participant using HMD system 400 is in front of the avatar of the participant using HMD system 100. In this case, the participant using HMD system 100 will hear the voice of the participant using HMD system 400 from two directions: to the right and in front. The voice of the participant using HMD system 400 is heard from the right without going through the speaker of HMD system 100, and from the front via the speaker of HMD system 100.

[0047] Therefore, in the first embodiment, as shown in Figure 4(C), the HMD system 400 is used The avatar of a participant using the HMD system 100 is positioned to the right (next to) the avatar of the participant using the HMD system 100. In other words, for multiple participants who are close together in real space, the relative positions of their avatars in the non-real space will roughly match the relative positions of the participants in real space. By doing this, the participant using the HMD system 100 will hear the voices of the participant using the HMD system 400 only from the right, resulting in reduced discomfort and improved immersion.

[0048] Furthermore, the relative positions of multiple participants in real space are not necessarily predetermined. Therefore, in the first embodiment, even when the relative positions of multiple participants in real space are not predetermined, the placement of each avatar in the non-real space can be suitably determined.

[0049] <Processing flow of online meeting system> Figure 5 is a sequence diagram showing the processing flow of the system (online meeting system) in Figure 1. Figure 5 shows the processing flow for determining the seat (placement) of the avatar corresponding to a newly joined participant in an online meeting. Here, we assume that a user of the HMD system 100 is joining the online meeting.

[0050] In step S501, the HMD system 100 sends a request to join the online meeting and its identification number to the server device 200.

[0051] In step S502, the server device 200 sends a request to the HMD system 100 to confirm whether or not other HMD systems (other participants) exist around the HMD system 100 (the user of the HMD system 100) in real space.

[0052] In step S503, the HMD system 400 emits a beacon signal to its surroundings. The HMD system 400 may emit beacon signals continuously or periodically, not limited to the timing of step S503. The HMD systems 100, 500, and 600 may also emit beacon signals.

[0053] In step S504, the HMD system 100 checks (determines) whether or not other HMD systems exist around it in real space.

[0054] Here, we assume that HMD system 400 (user of HMD system 400) is located around HMD system 100 (user of HMD system 100) in real space. Therefore, in step S505, HMD system 100 receives a beacon signal from HMD system 400. This allows HMD system 100 to detect the presence of HMD system 400 in real space. HMD system 100 then detects the identification number of HMD system 400 from the received beacon signal. Furthermore, HMD system 100 detects the direction in which HMD system 400 is located relative to HMD system 100 in real space, and the distance between HMD system 100 and HMD system 400 in real space, from the received beacon signal.

[0055] In step S506, the HMD system 100 sends the identification number of the HMD system 400 to the server device 200. This process also queries whether the HMD system 400 is participating in an online meeting (the same online meeting that the HMD system 100 is participating in).

[0056] In step S507, the server device 200 checks (determines) from the received identification number of the HMD system 400 whether the HMD system 400 is participating in an online meeting (the same online meeting in which the HMD system 100 is participating).

[0057] Here, we assume that HMD systems 400, 500, and 600 are participating in an online meeting. Since HMD system 400 is participating in the online meeting, in step S508, the server device 200 notifies HMD system 100 that HMD system 400 is participating in the online meeting.

[0058] In step S509, the HMD system 100 generates information regarding the positional relationship between the HMD system 100 and the HMD system 400 in real space (positional relationship information) based on the direction and distance detected in step S505. For example, information indicating the relative position of the HMD system 400 to the HMD system 100 is generated. The relative position is indicated, for example, by the direction and distance detected in step S505. The HMD system 100 then transmits the generated positional relationship information to the server device 200. The format of the positional relationship information is not particularly limited; for example, information indicating the relative position of the HMD system 100 to the HMD system 400 may be generated.

[0059] In step S510, the server device 200 determines the seat for each avatar. Here, the seat for each avatar is determined based on the positional relationship information transmitted in step S509 (the positional relationship between HMD system 100 and HMD system 400 in real space). For example, the seat for each avatar is determined such that the direction in which the avatar of HMD system 400 is located relative to the avatar of HMD system 100 in non-real space approximately matches the direction in which HMD system 400 is located relative to HMD system 100 in real space. This provides the latest seat information for each avatar.

[0060] In step S511, the server device 200 transmits necessary information to the HMD systems 100, 400, 500, and 600. For example, the avatar images of the HMD systems 400, 500, and 600, the CG conference room image, and the latest seating information are transmitted to the HMD system 100. The avatar images of the HMD system 100 and the latest seating information are transmitted to the HMD systems 400, 500, and 600. This initiates the online meeting using the HMD systems 100, 400, 500, and 600. Then, as needed, position and orientation information, operation information, and audio data are transmitted and received. The method of registering avatars is not particularly limited, but it is assumed that the avatars of the HMD systems 100, 400, 500, and 600 are pre-registered with the server device 200.

[0061] In addition, when there are no other HMD systems around the HMD system 100 in the real space, the processes of steps S505 to S509 are omitted. The HMD system 100 transmits to the server device 200 that there are no other HMD systems around the HMD system 100 in the real space. Then, in step S510, the server device 200 determines the seats of each avatar without relying on the above positional relationship information.

[0062] Also, when the HMD system 400 is not participating in the online meeting, the processes of steps S508 and S509 are omitted. The server device 200 notifies the HMD system 100 that the HMD system 400 is not participating in the online meeting. Then, in step S510, the server device 200 determines the seats of each avatar without relying on the above positional relationship information.

[0063] <Processing Flow of HMD System> FIG. 6 is a flowchart showing the processing flow of the HMD system 100. FIG. 6 shows the processing flow when the HMD system 100 newly participates in an online meeting.

[0064] In step S601, the control unit 123 controls the communication unit 150 to transmit a request to participate in the online meeting and the identification number of the HMD system 100 to the server device 200. In step S602, the control unit 123 controls the communication unit 150 to receive from the server device 200 a confirmation request as to whether there are other HMD systems around the HMD system 100 in the real space.

[0065]

[0066] ​In step S603, the control unit 123 controls the participant detection unit 160 to check whether or not other HMD systems are present around the HMD system 100 in real space. If the participant detection unit 160 receives a beacon signal (if it is detected that other HMD systems are present around the HMD system 100 in real space), the process proceeds to step S605. If the participant detection unit 160 does not receive a beacon signal (if it is not detected that other HMD systems are present around the HMD system 100 in real space), the process proceeds to step S604.

[0067] In step S604, the control unit 123 controls the communication unit 150 to notify the server device 200 that there are no other HMD systems around the HMD system 100 in real space.

[0068] In step S605, the control unit 123 detects the identification number of the discovered HMD system, the direction in which the discovered HMD system is located, and the distance to the discovered HMD system from the beacon signal received in step S603.

[0069] In step S606, the control unit 123 controls the communication unit 150 to transmit the identification number (identification number of the discovered HMD system) detected in step S605 to the server device 200. This process also queries whether the discovered HMD system is participating in an online meeting (the same online meeting that the HMD system 100 is participating in). If the query is answered with a response indicating that the discovered HMD system is participating in an online meeting, the process proceeds to step S607. If the query is answered with a response indicating that the discovered HMD system is not participating in an online meeting, the process shown in Figure 6 ends.

[0070] In step S607, the control unit 123 generates information (positional relationship information) regarding the positional relationship between the HMD system 100 and the discovered HMD system in real space, based on the direction and distance detected in step S605.

[0071] In step S608, the control unit 123 transmits the positional relationship information generated in step S607 to the server device 200.

[0072] <Server device processing flow> Figure 7 is a flowchart showing the processing flow of the server device 200. Figure 7 shows the processing flow for determining the seat of an avatar corresponding to a participant who has newly joined an online meeting. Assume that a user of the HMD system 100 is joining the online meeting.

[0073] In step S701, the control unit 211 receives a request to join the online meeting and the identification number of the HMD system 100 from the HMD system 100 by controlling the communication unit 204.

[0074] In step S702, the control unit 211 controls the communication unit 204 to request confirmation of whether or not other HMD systems exist around the HMD system 100 in real space. This is sent to the HMD system 100.

[0075] In step S703, the control unit 211 receives a response from the HMD system 100 to the confirmation request in step S701 by controlling the communication unit 204. The control unit 211 then confirms (determines) from the response to the confirmation request whether or not other HMD systems exist around the HMD system 100 in real space. If other HMD systems are found (detected) around the HMD system 100 in real space, the identification number of the found HMD system is received as a response to the confirmation request. If no other HMD systems are found around the HMD system 100 in real space, information indicating that no other HMD systems exist around the HMD system 100 in real space is received as a response to the confirmation request. If an HMD system is found, the process proceeds to step S705; if no HMD system is found, the process proceeds to step S704.

[0076] In step S704, the control unit 211 determines the seating arrangement for each avatar without considering the relative positions of multiple HMD systems in real space.

[0077] In step S705, the control unit 211 controls the participant management unit 212 to determine whether the HMD system discovered by the HMD system 100 is participating in an online meeting, based on the identification number received in step S703. Here, "online meeting" refers to an online meeting in which the HMD system 100 is participating. If the discovered HMD system is participating in an online meeting, the process proceeds to step S708; if the discovered HMD system is not participating in an online meeting, the process proceeds to step S706.

[0078] In step S706, the control unit 211 controls the communication unit 204 to notify the HMD system 100 that the HMD system detected by the HMD system 100 is not participating in the online meeting.

[0079] In step S707, the control unit 211 determines the seating arrangement for each avatar without considering the relative positions of multiple HMD systems in real space.

[0080] In step S708, the control unit 211 controls the communication unit 204 to notify the HMD system 100 that the HMD system discovered by the HMD system 100 is participating in an online meeting.

[0081] In step S709, the control unit 211 controls the communication unit 204 to receive information (positional relationship information) regarding the positional relationship between the HMD system 100 and the discovered HMD system in real space from the HMD system 100.

[0082] In step S710, the control unit 211 determines the seat for each avatar based on the positional relationship information received in step S709.

[0083] As described above, according to the first embodiment, when the presence of other participants is detected around a new participant in real space, information regarding the relative positions of those participants in real space is acquired, and the placement of each avatar is determined based on that information. In this way, in a service that enables communication in a non-real space, even if the relative positions of multiple participants in real space are not predetermined, the placement of each avatar in the non-real space can be suitably determined.

[0084] The example described shows that the HMD system 100 checks whether other HMD systems exist around it in real space in response to a confirmation request sent from the server device 200, but this is not limited to this example. For example, the server device 200 does not need to send a confirmation request to the HMD system 100. The HMD system 100 may also check whether other HMD systems exist around it in real space, either continuously or periodically.

[0085] Furthermore, while an example has been described in which the HMD system 100 queries the server device 200 to determine whether the discovered HMD system is participating in an online meeting, the system is not limited to this example. For instance, if the HMD system 100 discovers another HMD system, it may send positional information to the server device 200 without making a query. If the discovered HMD system is participating in an online meeting, the server device 200 determines the seating arrangement for each avatar based on the positional information. If the discovered HMD system is not participating in an online meeting, the server device 200 determines the seating arrangement for each avatar without relying on the positional information.

[0086] Furthermore, the HMD system 100 (client device 102) may also have the functionality of a server device 200 and provide online meeting services to other HMD systems.

[0087] Furthermore, while we have described an example where a new participant (HMD system 100) searches for other participants (other HMD systems) in its surroundings within the real world, the explanation is not limited to this. For example, existing participants (HMD systems 400, 500, 600) may be notified that a new participant has been added, and the existing participants may then search for the other participants in their surroundings within the real world.

[0088] [Second Embodiment] A second embodiment of the present invention will now be described. Note that the same aspects as the first embodiment (for example, the same configuration and processing as the first embodiment) will be omitted in the following description, and only the differences from the first embodiment will be described.

[0089] In online meetings, participants may instruct their avatars to move. If avatars can move anywhere, the relative positions of multiple avatars in the virtual space may differ from the relative positions of multiple participants in the real space, even if they are close together in the real space. When this positional discrepancy occurs, the direction from which a participant's voice is heard through the HMD's speakers may differ from the direction from which their voice is heard without the HMD's speakers, causing participants to feel uncomfortable and diminishing their sense of immersion. Therefore, in the second embodiment, the movement of avatars is restricted based on positional information, and a predetermined notification is given to the participant who instructed the movement.

[0090] <Seat change restrictions> Figure 8(A) is a schematic diagram showing the placement of HMD systems (participants) in real space. A participant using HMD system 100 is to the left of a participant using HMD system 400. Figure 8(B) is a schematic diagram showing the placement of avatars in non-real space. The avatar of a participant using HMD system 100 is to the left of the avatar of a participant using HMD system 400. In other words, the relative positions of HMD systems 100 and 400 (avatars) in non-real space correspond to the relative positions of HMD systems 100 and 400 (participants) in real space. Now, suppose a participant using HMD system 100 gives an instruction to move their avatar to the right of the avatar of a participant using HMD system 400. If such a movement is allowed, the relative positions of HMD systems 100 and 400 (avatars) in non-real space will not match (or will almost not match) the relative positions of HMD systems 100 and 400 (participants) in real space. In the first embodiment, the participant who issued the instruction to move is not given a predetermined notification (for example, a notification that the avatar cannot be moved to the designated position (seat)). For example, the warning screen shown in Figure 8(C) is displayed on the display unit 112 of the HMD system 100.

[0091] <Processing flow of online meeting system> Figure 9 is a sequence diagram showing the processing flow of an online meeting system according to the second embodiment. Figure 9 shows the processing flow when an avatar is instructed to move during an online meeting, but the instruction to move is not accepted, and a predetermined notification is sent. Here, it is assumed that the user of the HMD system 100 instructs their avatar to move.

[0092] Input device In step S901, the HMD system 100 receives an instruction from the user of the HMD system 100 to move the avatar of the HMD system 100 (a seat change operation to the control unit 170).

[0093] In step S902, the HMD system 100 sends a seat change request to the server device 200. The seat change request includes information about the seat change operation. For example, the seat change request indicates the new position specified by the user of the HMD system 100.

[0094] In step S903, the server device 200 determines whether or not to move the avatar according to the seat change request in step S902 (seat change operation in step S901) based on the positional relationship information related to the HMD system 100. The server device 200 checks (determines) whether or not the aforementioned discrepancy (discrepancy between the positional relationship in real space and the positional relationship in non-real space) occurs as a result of moving the avatar to the specified position. The server device 200 then determines to move the avatar if no discrepancy occurs, and not to move the avatar if a discrepancy occurs. If positional relationship information related to the HMD system 100 has not been acquired in advance, the server device 200 determines to move the avatar because there are no other HMD systems around the HMD system 100 in real space.

[0095] For example, assume that an HMD system 400 exists around the HMD system 100 in the real space. In the non-real space, the direction in which the avatar of the HMD system 400 exists with respect to the avatar of the HMD system 100 is defined as the first direction, and in the real space, the direction in which the HMD system 400 exists with respect to the HMD system 100 is defined as the second direction. The server device 200 checks (determines) whether the first direction substantially coincides with the second direction even when moving the avatar according to the seat change request. Then, when the first direction substantially coincides with the second direction, the server device 200 determines to move the avatar, and when the first direction no longer substantially coincides with the second direction, the server device 200 determines not to move the avatar.

[0096] Here, assume that it is determined not to move the avatar. Therefore, in step S904, the server device 200 notifies the HMD system 100 that the avatar cannot be moved. When it is determined to move the avatar, the server device 200 does not notify the HMD system 100 and moves the avatar according to the seat change request. When the server device 200 updates the seat information by moving the avatar, it transmits the latest seat information to each HMD system.

[0097] In step S905, the HMD system 100 displays a warning screen indicating that the avatar cannot be moved to the designated position on the display unit 112.

[0098] <Processing Flow of HMD System> FIG. 10 is a flowchart showing the processing flow of the HMD system 100. FIG. 10 shows the processing flow when a seat change operation is performed on the HMD system 100.

[0099] When a seat change operation is performed on the HMD system 100, in step S1001, the control unit 123 transmits a seat change request to the server device 200 by controlling the communication unit 150.

[0100] In step S1002, the control unit 123 checks (determines) whether or not it has received a response to the seat change request (notification that the avatar cannot be moved). If a response to the seat change request is received, the process proceeds to step S1003; if no response to the seat change request is received, the process shown in Figure 10 ends.

[0101] In step S1003, the control unit 123 displays a warning screen on the display unit 112 indicating that the avatar cannot be moved to the specified position.

[0102] <Server device processing flow> Figure 11 is a flowchart showing the processing flow of the server device 200. Figure 11 shows the processing flow when a seat change operation is performed on the HMD system 100. It is assumed that positional relationship information related to the HMD system 100 has been acquired in advance.

[0103] When a seat change operation is performed on the HMD system 100, in step S1101, the control unit 211 receives a seat change request from the HMD system 100 by controlling the communication unit 204.

[0104] In step S1102, the control unit 211 determines whether or not to move the avatar in accordance with the seat change request (seat change operation) in step S1101. This process corresponds to step S903. If the avatar is to be moved, the process proceeds to step S1103; if the avatar is not to be moved, the process proceeds to step S1104.

[0105] In step S1103, the control unit 211 moves the avatar in accordance with the seat change request in step S1101.

[0106] In step S1104, the control unit 211 controls the communication unit 204 to notify the HMD system 100 that the avatar cannot move.

[0107] As described above, according to the second embodiment, the movement of the avatar is restricted, and a predetermined notification is given to the participant who instructed the movement. By restricting the movement of the avatar, it is possible to suppress the discrepancy between the relative positions of multiple avatars in the non-real space and the relative positions of multiple participants in the real space for multiple participants who are close to each other in the real space. Furthermore, by giving a predetermined notification, it is possible to suppress the occurrence of discomfort and unpleasantness caused by the restriction of avatar movement.

[0108] While an example of restricting avatar movement has been explained, it is also possible to not restrict avatar movement and instead provide a notification (for example, a notification that the sense of presence or immersion will be reduced) to the participant who issued the movement instruction. This prevents situations where a participant (avatar) is unable to approach other participants (other avatars) who are close to them. In addition, each participant can participate in the online meeting from their preferred viewpoint. For multiple participants who are close to each other in the real world, the relative positions of multiple avatars in the non-real world are shown in the real world. For participants whose positions need to be aligned with those of multiple other participants, it is possible to prompt them to cancel their avatar's movement.

[0109] [Third Embodiment] A third embodiment of the present invention will now be described. Note that the same aspects as the first embodiment (for example, the same configuration and processing as the first embodiment) will be omitted in the following description, and only the differences from the first embodiment will be described.

[0110] When the non-real space is a VR space, the participant's HMD system displays avatars as other participants who are near that participant in the real space. In this case, even if the relative positions of multiple avatars in the non-real space differ slightly from the relative positions of multiple participants in the real space, the sense of incongruity is minimal. For example, if the direction in which multiple avatars are lined up in the non-real space roughly matches the direction in which multiple participants are lined up in the real space, the sense of incongruity is minimal.

[0111] If the non-real space is an MR space, the participant's HMD system will not display avatars of other participants who are near the participant in the real space. In the case of optical see-through, the participant can directly see other participants who are near them. In the case of video see-through, the participant's HMD system displays a real image (an image of the real space) that includes the area where other participants are near the participant. Therefore, the participant can see other participants who are near them in the real space in the real image. In this case, if the two positional relationships described above (positional relationship in the real space and positional relationship in the non-real space) are different, the avatar of a participant in a distant location may overlap with that of another participant who is near the participant in the real space.

[0112] Therefore, in the third embodiment, the method for determining the placement of each avatar differs depending on whether the non-real space is a VR space or an MR space. For example, if the non-real space is a VR space, the placement of each avatar is determined based on the direction in which multiple participants who are close to each other in the real space are lined up. If the non-real space is an MR space, the seat (placement) of each avatar is determined based on the direction in which multiple participants who are close to each other in the real space are lined up and the distance (spacing) between those multiple participants.

[0113] <Avatar's seating arrangement> Figure 12(A) is a schematic diagram showing the placement of avatars in a non-realistic space. In Figure 12(A), users of HMD systems 400, 500, and 600 are participating in an online meeting.

[0114] Figure 12(B) is a schematic diagram showing the placement of HMD systems in real space. In Figure 12(B), the user of HMD system 100 is to the right of the user of HMD system 400. This indicates that the user using HMD system 100 is about to join an online meeting.

[0115] Figure 12(C) is a schematic diagram showing the placement of avatars in a non-real space. Suppose the non-real space is a VR space, and a user of HMD system 100 attempts to participate in an online meeting in the situations shown in Figures 12(A) and 12(B). In this case, the seating position of the avatar on HMD system 100 should be determined based on the direction (to the right) in which HMD system 100 (the user of HMD system 100) is located relative to HMD system 400 (the user of HMD system 400) in the real space. Therefore, the avatar of HMD system 100 is positioned to the right of the avatar of HMD system 400. For example, the avatar of HMD system 100 is positioned within the area enclosed by the dashed line in Figure 12(C). In the example in Figure 12(C), The avatar of HMD system 100 is placed in an empty seat, avoiding the seat of the avatar of HMD system 600.

[0116] Figure 12(D) is a schematic diagram showing the placement of avatars in a non-real space. Suppose the non-real space is an MR space, and a user of HMD system 100 attempts to participate in an online meeting in the situation shown in Figures 12(A) and 12(B). In this case, the seating position of the avatar on HMD system 100 should be determined based on the direction in which HMD system 100 is located relative to HMD system 400 in the real space, and the distance between HMD system 400 and HMD system 100 in the real space. Therefore, the avatar on HMD system 100 is positioned to the right of the avatar on HMD system 400. The avatar on HMD system 600, which was to the right of the avatar on HMD system 400, is then moved to a different seat.

[0117] <Server device processing flow> Figure 13 is a flowchart of the processing flow of server device 200. Here, we assume that the processing in Figure 13 is performed in step S710 of Figure 7. Assume that users of HMD systems 400, 500, and 600 are participating in an online meeting, and that a user of HMD system 100 is newly joining the online meeting. Furthermore, assume that in the real world, HMD system 400 (user of HMD system 400) is present around HMD system 100 (user of HMD system 100).

[0118] In step S1301, the control unit 211 checks (determines) whether the unreal space is a VR space or an MR space. If the unreal space is a VR space, the process proceeds to step S1302; if the unreal space is an MR space, the process proceeds to step S1303.

[0119] In step S1302, the control unit 211 determines the seating arrangement for each avatar based on the orientation of the HMD system 100 relative to the HMD system 400 in real space. For example, the arrangement of each avatar is determined such that the orientation of the avatars of the HMD system 100 relative to the avatars of the HMD system 400 in non-real space approximately coincides with the orientation of the HMD system 100 relative to the HMD system 400 in real space.

[0120] In step S1303, the control unit 211 determines the seating arrangement for each avatar based on the direction in which the HMD system 100 is located relative to the HMD system 400 in real space, and the distance between the HMD system 400 and the HMD system 100 in real space. For example, the direction in which the avatar of the HMD system 100 is located relative to the avatar of the HMD system 400 in non-real space is defined as the first direction, and the direction in which the HMD system 100 is located relative to the HMD system 400 in real space is defined as the second direction. The distance between the avatar of the HMD system 400 and the avatar of the HMD system 100 in non-real space is defined as the first distance, and the distance between the HMD system 400 and the HMD system 100 in real space is defined as the second distance. The arrangement of each avatar is then determined such that the first direction approximately coincides with the second direction and the first distance approximately coincides with the second distance.

[0121] As described above, according to the third embodiment, the method for determining the placement of each avatar is switched so that the placement of each avatar is determined in a different way depending on whether the non-real space is a VR space or an MR space. This makes it possible to more favorably determine the placement of each avatar in both the VR space and the MR space.

[0122] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. This can also be achieved by a circuit that performs one or more functions (for example, an ASIC).

[0123] This embodiment includes the following configurations, methods, programs, and media. (Composition 1) A determination means for determining the placement of multiple avatars in a non-real space, each corresponding to a multiple participant in a service that enables communication in a non-real space, When it is detected that a second participant of the service is present around a first participant who has newly joined the service in real space, an acquisition means obtains information regarding the positional relationship between the first participant and the second participant in real space. It has, The determination means determines the arrangement of the plurality of avatars based on the information. An information processing device characterized by the following: (Configuration 2) The information processing device is a server device to which multiple client devices, each used by one of the multiple participants, are connected. The information processing device according to configuration 1, characterized by the above. (Composition 3) The information processing device is an information processing device used by any of the multiple participants. The information processing device according to configuration 1, characterized by the above. (Composition 4) Detection means for detecting other participants of the service from around the participant using the information processing device in the aforementioned real space. It further possesses The information processing apparatus according to configuration 3, characterized by the features described herein. (Composition 5) The determination means determines the arrangement of the multiple avatars such that the direction in which the second participant's avatar exists relative to the first participant's avatar in the non-real space substantially coincides with the direction in which the second participant exists relative to the first participant in the real space. An information processing device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The system further includes control means for controlling the system to send a predetermined notification to the first participant based on the information when the first participant gives an instruction to move the first participant's avatar. An information processing device according to any one of configurations 1 to 5, characterized by the above. (Composition 7) The determination means determines the arrangement of the plurality of avatars such that the first direction in which the second participant's avatar exists relative to the first participant's avatar in the non-real space substantially coincides with the second direction in which the second participant exists relative to the first participant in the real space. The control means controls the first participant to give the predetermined notification when the first direction no longer substantially coincides with the second direction due to the movement of the first participant's avatar in accordance with the first participant's instructions. The information processing device according to configuration 6, characterized by the features described therein. (Composition 8) The aforementioned non-real space is a virtual reality space or a mixed reality space. An information processing device according to any one of configurations 1 to 7, characterized by the above. (Composition 9) The determination means causes the method for determining the arrangement of the multiple avatars to differ depending on whether the non-real space is the virtual reality space or the mixed reality space. The information processing apparatus according to configuration 8, characterized by the above. (Composition 10) The aforementioned determination means is If the non-real space is the virtual reality space, the arrangement of the multiple avatars is determined based on the direction in which the second participant is located relative to the first participant in the real space. If the non-real space is the mixed reality space, the arrangement of the multiple avatars is determined based on the direction in which the second participant is located relative to the first participant in the real space, and the distance between the first participant and the second participant in the real space. The information processing apparatus according to configuration 9, characterized by the features described therein. (Composition 11) The aforementioned determination means is If the non-real space is the virtual reality space, the arrangement of the plurality of avatars is determined such that the first direction in which the second participant's avatar exists relative to the first participant's avatar in the non-real space substantially coincides with the second direction in which the second participant exists relative to the first participant in the real space. If the non-real space is the mixed reality space, the arrangement of the plurality of avatars is determined such that the first direction substantially coincides with the second direction, and the distance between the avatar of the first participant and the avatar of the second participant in the non-real space substantially coincides with the distance between the first participant and the second participant in the real space. The information processing apparatus according to configuration 10, characterized by the above. (method) A decision step of determining the placement of multiple avatars in the aforementioned non-real space, each corresponding to multiple participants in a service that enables communication within a non-real space, When it is detected that a second participant of the service is present around a first participant who has newly joined the service in real space, an acquisition step is taken to obtain information regarding the positional relationship between the first participant and the second participant in real space. It has, In the aforementioned decision step, the arrangement of the multiple avatars is determined based on the information. An information processing method characterized by the following: (program) A program for causing a computer to function as one of the means of the information processing device described in any one of the configurations 1 to 11. (medium) A computer-readable storage medium containing a program that causes the computer to function as one of the means of the information processing device described in any one of items 1 to 11 of the configuration. [Explanation of Symbols]

[0124] 102: Client device 120: CPU 200: Server device 201: CPU

Claims

1. A determination means for determining the placement of multiple avatars in a non-real space, each corresponding to a multiple participant in a service that enables communication in a non-real space, When it is detected that a second participant of the service is present around a first participant who has newly joined the service in real space, an acquisition means for acquiring information regarding the positional relationship between the first participant and the second participant in real space; Control means that controls the first participant to give a predetermined notification to the first participant based on the information when the first participant gives an instruction to move the first participant's avatar. It has, The determination means determines the arrangement of the plurality of avatars based on the information. An information processing device characterized by the following:

2. The information processing device is a server device to which multiple client devices, each used by one of the multiple participants, are connected. The information processing apparatus according to feature 1.

3. The information processing device is an information processing device used by any of the multiple participants. The information processing apparatus according to feature 1.

4. Detection means for detecting other participants of the service from around the participant using the information processing device in the aforementioned real space. Furthermore, it has The information processing apparatus according to claim 3.

5. The determination means determines the arrangement of the plurality of avatars such that the direction in which the second participant's avatar exists relative to the first participant's avatar in the non-real space substantially coincides with the direction in which the second participant exists relative to the first participant in the real space. The information processing apparatus according to feature 1.

6. The determination means determines the arrangement of the plurality of avatars such that the first direction in which the second participant's avatar exists relative to the first participant's avatar in the non-real space substantially coincides with the second direction in which the second participant exists relative to the first participant in the real space. The control means controls the first participant to give the predetermined notification when the first direction no longer substantially coincides with the second direction due to the movement of the first participant's avatar in accordance with the first participant's instructions. The information processing apparatus according to feature 1.

7. The aforementioned non-real space is a virtual reality space or a mixed reality space. The information processing apparatus according to feature 1.

8. The determination means causes the method for determining the arrangement of the multiple avatars to differ depending on whether the non-real space is the virtual reality space or the mixed reality space. The information processing apparatus according to feature 7.

9. The aforementioned determination means is If the non-real space is the virtual reality space, the arrangement of the multiple avatars is determined based on the direction in which the second participant is located relative to the first participant in the real space. If the non-real space is the mixed reality space, the arrangement of the multiple avatars is determined based on the direction in which the second participant is located relative to the first participant in the real space, and the distance between the first participant and the second participant in the real space. The information processing apparatus according to feature 8.

10. The aforementioned determination means is If the non-real space is the virtual reality space, the arrangement of the plurality of avatars is determined such that the first direction in which the second participant's avatar exists relative to the first participant's avatar in the non-real space substantially coincides with the second direction in which the second participant exists relative to the first participant in the real space. If the non-real space is the mixed reality space, the arrangement of the multiple avatars is determined such that the first direction substantially coincides with the second direction, and the distance between the avatar of the first participant and the avatar of the second participant in the non-real space substantially coincides with the distance between the first participant and the second participant in the real space. The information processing apparatus according to feature 9.

11. A decision step of determining the placement of multiple avatars in the aforementioned non-real space, each corresponding to multiple participants in a service that enables communication within a non-real space, When it is detected that a second participant of the service is present around a first participant who has newly joined the service in real space, the acquisition step involves acquiring information regarding the positional relationship between the first participant and the second participant in real space. A control step that controls the system to send a predetermined notification to the first participant based on the information when the first participant gives an instruction to move the first participant's avatar. It has, In the aforementioned decision step, the arrangement of the plurality of avatars is determined based on the information. An information processing method characterized by the following:

12. The computer is used as one of the means in the information processing apparatus described in any one of claims 1 to 10. A program to make something work.

Images