Virtual space sharing system, virtual space sharing method and virtual space sharing program

Abstract

The present disclosure relates to a virtual space sharing system, a virtual space sharing method, and a virtual space sharing program. A virtual space sharing system capable of outputting a virtual image of a virtual space shared by multiple users in mutual directions is implemented. A virtual space sharing system of one aspect of the present disclosure comprises: an action acquisition unit that acquires the actions of users existing in each of multiple spaces; an image generation unit that generates a virtual image of a virtual image corresponding to each user performing an action in a virtual space shared by multiple spaces based on the action of the user; and an output unit that outputs a virtual image of another user's virtual image performing an action in the virtual space to a display unit configured in each space.

Description

Technical Field

[0001] The present disclosure relates to a virtual space sharing system, a virtual space sharing method, and a virtual space sharing program. Background Art

[0002] In recent years, the technology of making an avatar corresponding to a user move in a virtual space has been put into practical use in animation distribution, etc. Such an animation distribution system, for example, as disclosed in Japanese Patent Application Laid-Open No. 2019-204244, simultaneously distributes virtual images of an avatar corresponding to a user moving in a virtual space to multiple viewing users. Summary of the invention

[0003] The applicant has discovered the following problem. The animation distribution system of Japanese Patent Publication No. 2019-204244 is a one-way distribution system that distributes virtual images to audio-visual users. Therefore, the animation distribution system of Japanese Patent Publication No. 2019-204244 is not configured to distribute (output) virtual images of a virtual space shared by multiple users in mutual directions.

[0004] The present disclosure realizes a virtual space sharing system, a virtual space sharing method, and a virtual space sharing program capable of outputting a virtual image of a virtual space shared by a plurality of users in mutual directions.

[0005] A virtual space sharing system of a technical solution of the present disclosure comprises:

[0006] an action acquisition unit, which acquires actions of users in each of the plurality of spaces;

[0007] an image generating unit that generates a virtual image in which an avatar corresponding to each of the users moves in a virtual space shared by the plurality of spaces based on the movement of the user; and

[0008] The output unit outputs a virtual image of another user's avatar moving in the virtual space to a display unit arranged in each space.

[0009] The virtual space sharing system preferably includes a space setting unit that sets the virtual space based on a space request input from the outside.

[0010] An initial position for configuring the virtual image of another user is determined in the virtual space according to the virtual space.

[0011] The virtual space sharing system preferably includes a placement setting unit that sets the placement of the avatar of each other user for each of the initial positions determined in the virtual space based on a placement request of the avatar input from the outside.

[0012] The above-mentioned virtual space sharing system preferably comprises:

[0013] A sight line acquisition unit, which acquires the sight line direction of the user; and

[0014] The attention part acquisition unit acquires the part that the user pays attention to in the virtual image based on the sight line direction of the user,

[0015] The image generation unit generates the virtual image based on a portion of the virtual image that the user pays attention to.

[0016] The virtual space sharing system preferably includes a voice acquisition unit for acquiring the voice of the user.

[0017] When the virtual space is a demonstration space, the image generating unit generates the virtual image in such a manner that a virtual image of a user performing a demonstration using data loaded into the demonstration space and the data do not interfere with virtual images of other users and are displayed larger than the virtual images of the other users.

[0018] The virtual space sharing system preferably includes a voice acquisition unit for acquiring the voice of the user.

[0019] When the virtual space is a meeting space, the image generation unit generates the virtual image such that virtual images of other users are arranged side by side in the meeting space.

[0020] The above-mentioned virtual space sharing system preferably comprises:

[0021] A sight line acquisition unit, which acquires the sight line direction of the user; and

[0022] The attention part acquisition unit acquires the part that the user pays attention to in the virtual image based on the sight line direction of the user,

[0023] The image generation unit generates the virtual image based on the portion of the virtual image that the user is paying attention to, such that virtual images of other users who are having a conversation are having a face-to-face conversation with each other in the conference space.

[0024] In the virtual space sharing system, it is preferable that the image generating unit generates the virtual image within a preset field of view range in a field of view direction of the avatar of the user in the virtual space.

[0025] A virtual space sharing method of a technical solution of the present disclosure includes the following steps:

[0026] Obtaining actions of users in each of the plurality of spaces in time series;

[0027] generating a virtual image in which a virtual avatar corresponding to each of the users moves in a virtual space shared by the plurality of spaces based on the time-series movements of the user; and

[0028] A virtual image of another user's avatar moving in the virtual space is output to a display unit arranged in each of the spaces.

[0029] The virtual space sharing program of a technical solution of the present disclosure enables a computer to perform the following processing:

[0030] Obtaining actions of users in each of the plurality of spaces in time series;

[0031] generating a virtual image in which a virtual avatar corresponding to each of the users moves in a virtual space shared by the plurality of spaces based on the time-series movements of the user; and

[0032] A virtual image of another user's avatar operating in the virtual space is output to a display unit arranged in each of the spaces.

[0033] According to the present disclosure, a virtual space sharing system, a virtual space sharing method, and a virtual space sharing program capable of outputting a virtual image of a virtual space shared by a plurality of users in mutual directions can be realized.

[0034] The above and other objects, features and advantages of the present disclosure will be more fully understood from the detailed description given below and the accompanying drawings, which are given by way of illustration only and thus should not be construed as limiting the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a block diagram showing the virtual space sharing system and peripheral configuration according to the first embodiment.

[0036] Figure 2 This is a diagram showing a state of a first space when a conference is held using the virtual space sharing system according to the first embodiment.

[0037] Figure 3 This is a diagram showing a state of a second space when a conference is held using the virtual space sharing system according to the first embodiment.

[0038] Figure 4 This is a diagram showing a state of the third space when a conference is held using the virtual space sharing system according to the first embodiment.

[0039] Figure 5 This is a diagram showing a state of the fourth space when a conference is held using the virtual space sharing system according to the first embodiment.

[0040] Figure 6This is a diagram showing a state of a first space when a conference is held using the virtual space sharing system according to the second embodiment.

[0041] Figure 7 This is a diagram showing a state of a second space when a conference is held using the virtual space sharing system according to the second embodiment.

[0042] Figure 8 This is a diagram showing a state of a space where a user performing a presentation exists when a presentation is performed using the virtual space sharing system according to the third embodiment.

[0043] Fig. 9 This is a diagram showing a state of a space in which users who listen to a presentation using the virtual space sharing system according to the third embodiment exist.

[0044] Fig.10 This is a diagram showing a state of space when the virtual space is changed to a meeting space during a presentation.

[0045] Fig.11 This is a diagram showing the state of a space where a teacher exists when a dance lesson is conducted using the virtual space sharing system according to the fourth embodiment.

[0046] Fig.12 This is a diagram showing a state of a space in which students exist when a dance lesson is conducted using the virtual space sharing system according to the fourth embodiment.

[0047] Fig.13 This is a diagram showing a state of a space where a first user exists when the first user approaches a second user and talks to the second user.

[0048] Fig.14 This is a diagram showing a state of a space where a third user exists when a first user approaches a second user and talks.

[0049] Fig.15 This is a diagram showing a state of a space where a first user exists when a second user approaches a first user who is speaking and speaks.

[0050] Fig.16 This is a diagram showing a state of a space where a third user exists when a second user approaches and speaks to a first user who is speaking.

[0051] Fig.17 This is a diagram showing an example of a hardware configuration included in the virtual space sharing system. DETAILED DESCRIPTION

[0052] Hereinafter, with reference to the accompanying drawings, a specific embodiment to which the present disclosure is applied is described in detail. However, the present disclosure is not limited to the following embodiments. In addition, in order to make the description clear, the following description and the accompanying drawings are appropriately simplified.

[0053] <Implementation Method 1>

[0054] First, the structure of the virtual space sharing system of this embodiment is described. The virtual space sharing system of this embodiment is suitable when users in each of multiple spaces use the virtual images corresponding to the users to communicate in a common virtual space. Here, the space can be any space such as a living room, office, or studio where users who use the virtual space sharing system to communicate with other users exist.

[0055] Figure 1 is a block diagram showing the virtual space sharing system and surrounding structures of this embodiment. Figure 1 As shown, the virtual space sharing system 1 includes an action acquisition unit 2, a line of sight acquisition unit 3, a sound acquisition unit 4, a space setting unit 5, an avatar setting unit 6, a configuration setting unit 7, a focus area acquisition unit 8, an image generation unit 9, an output unit 10 and a database unit 11.

[0056] Furthermore, the virtual space sharing system 1 is connected to the motion capture device 12, the eye tracking device 13, the microphone 14, the display unit 15, and the speaker 16 arranged in each of the plurality of spaces S via the network 17. Furthermore, the virtual space sharing system 1 is connected to an input device 18 such as a portable terminal owned by a user existing in each space via the network 17.

[0057] The motion acquisition unit 2 acquires the motion of the user in each of the plurality of spaces S. For example, the motion acquisition unit 2 acquires the motion of the user from the motion capture device 12 disposed in each space S.

[0058] In detail, the motion capture device 12 is arranged in each space S in such a manner that it can detect the user's motion in each space S, and is equipped with an RGBD camera such as a kinect. In addition, the motion capture device 12 detects the user's skeletal coordinates in time series as the user's motion, and outputs information indicating the detected user's skeletal coordinates in time series to the motion acquisition unit 2 via the network 17.

[0059] Thus, the motion acquisition unit 2 can obtain the user's posture changes in time series, the direction of the user's face, the user's position, etc. based on the user's time series skeletal coordinates represented by the information input from the motion capture device 12 in each space S. Here, the user's skeletal coordinates can be obtained, for example, through the world coordinates in the space S.

[0060] However, the motion acquisition unit 2 may also be composed of a motion capture device disposed in each space S. In other words, the motion acquisition unit 2 may acquire the user's motion by the motion acquisition unit 2 itself or from an external motion capture device. In short, the motion acquisition unit 2 is not limited to any means as long as it can acquire the user's motion.

[0061] The sight line acquisition unit 3 acquires the sight line direction of the user present in each space S. For example, the sight line acquisition unit 3 acquires the sight line direction of the user from the eye tracking device 13 disposed in each space S.

[0062] Specifically, the eye tracking device 13 is disposed in each space S so as to detect the user's gaze direction. The eye tracking device 13 includes, for example, an infrared light source and an infrared camera, and detects the user's gaze direction in time series based on the positional relationship between the pupil center and Purkinje's spot.

[0063] The eye tracking device 13 outputs information indicating the detected time-series gaze direction of the user to the gaze acquisition unit 3 via the network 17. Thus, the gaze acquisition unit 3 can acquire the gaze direction of the user. Here, the gaze direction of the user can be acquired by, for example, the world coordinates in the space S.

[0064] However, the sight acquisition unit 3 may also be composed of an eye tracking device arranged in each space S. That is, the sight acquisition unit 3 may acquire the sight direction of the user by the sight acquisition unit 3 itself, or may acquire the sight direction of the user from an external eye tracking device. In addition, in the case where the sight direction of the user can be detected by the motion capture device 12, the eye tracking device 13 may also be composed of the motion capture device 12. In short, as long as the sight acquisition unit 3 can acquire the sight direction of the user, the means are not limited.

[0065] The sound acquisition unit 4 acquires the voice of the user present in each space S. For example, the sound acquisition unit 4 acquires the voice of the user from the microphone 14 arranged in each space S. Specifically, the microphone 14 is arranged in each space S in such a manner as to be able to detect the voice of the user, and outputs information indicating the detected voice of the user to the sound acquisition unit 4 via the network 17.

[0066] Thereby, the voice acquisition unit 4 can acquire the voice of the user. Here, the voice acquisition unit 4 can acquire the direction of the voice based on the voice of the user detected by the microphone 14.

[0067] However, the voice acquisition unit 4 may also be composed of microphones arranged in each space S. That is, the voice acquisition unit 4 may acquire the user's voice by the voice acquisition unit 4 itself or from an external microphone. In short, the voice acquisition unit 4 is not limited to any means as long as it can acquire the user's voice.

[0068] The space setting unit 5 sets a virtual space shared by a plurality of spaces S based on a space request input from the outside. As the virtual space, for example, a meeting space, a presentation space, or an office space may be prepared. The virtual space may be two-dimensional or three-dimensional and have coordinates within the virtual space.

[0069] The space setting unit 5 displays a selection image prompting the user to select a plurality of virtual space candidates on the display unit of the input device 18 owned by the user, for example. Then, when at least one user selects a virtual space candidate displayed on the display unit of the user's input device 18, information indicating a space request is input to the space setting unit 5. The space setting unit 5 sets the virtual space based on the space request indicated by the input information.

[0070] However, the means by which the user inputs the space request is not limited, and it is sufficient as long as information indicating the space request input by the user is input to the space setting unit 5. In addition, it is sufficient as long as the information indicating the space request is input to the space setting unit 5 from the outside, for example, it is not limited to the user, and the space request may be input by another person via an input device. Furthermore, in the case where the virtual space is limited to one type in advance, the space setting unit 5 may be omitted.

[0071] The avatar setting unit 6 sets an avatar corresponding to the user that appears in the virtual space based on an avatar setting request input from the outside. The avatar moves in the virtual space in a manner corresponding to the user's motion acquired by the motion acquisition unit 2, as will be described in detail later.

[0072] As the avatar, for example, a CG (Computer Graphics) character, a CG character with a facial image of a user acquired by the motion acquisition unit 2 or acquired in advance embedded in the facial part, a costumed character with a user image acquired by the motion acquisition unit 2 but with clothes changed, etc. can be prepared. In this case, the avatar can also be two-dimensional or three-dimensional and have coordinates in the virtual space. However, the avatar is not limited to the above as long as it moves in the virtual space in accordance with the user.

[0073] As will be described in detail later, the avatar setting unit 6 displays an image prompting the user to select a plurality of character candidates and to perform detailed settings for the selected character on the display unit of the input device 18 owned by the user. In addition, when the user selects a character candidate displayed on the display unit of the user's input device 18, information indicating a request for avatar setting is input to the avatar setting unit 6.

[0074] The avatar setting unit 6 sets an avatar corresponding to the user based on the avatar setting request represented by the input information. However, the means by which the user inputs the avatar setting request is not limited, and the information representing the avatar setting request input by the user can be input to the avatar setting unit 6.

[0075] In addition, information indicating an avatar setting request may be input from outside to the avatar setting unit 6. For example, not only the user but also another person may input the avatar setting request via an input device. Furthermore, if an avatar corresponding to the user is pre-assigned, the avatar setting unit 6 may be omitted.

[0076] As described later, the placement setting unit 7 sets the initial position of the avatar generated by the image generating unit 9 in the virtual space based on the avatar placement request input from the outside. Here, the details will be described later, and the initial position of the avatar of other users is pre-determined in the virtual space according to the virtual space.

[0077] The arrangement setting unit 7 displays a selection image prompting the user to select an arrangement position candidate on the display unit of the input device 18 owned by the user, for example. The user selects an arrangement position candidate displayed on the display unit of the user's input device 18 for each avatar of another user, thereby inputting information indicating an arrangement request to the arrangement setting unit 7.

[0078] The configuration setting unit 7 sets the initial position of each avatar of another user in the virtual space based on the configuration request represented by the input information. However, the means by which the user inputs the configuration request of the avatar of another user is not limited, and it is sufficient as long as the information representing the configuration request of each avatar of another user input by the user is input to the configuration setting unit 7.

[0079] In addition, information indicating a virtual image setting request may be input from outside to the configuration setting unit 7. For example, not limited to the user, other persons may input the virtual image setting request via an input device. Furthermore, when the other user to be communicated with is one person in advance and the initial position of the virtual image of the other user is set in advance, the configuration setting unit 7 may be omitted.

[0080] The attention part acquisition unit 8 displays the user's line of sight direction indicated by the information acquired by the line of sight acquisition unit 3 on the screen 19 (see FIG. 1 ) projected from the display unit 15 disposed in the space S where the user exists. Figure 2 ) to obtain the part that the user is concerned about in the virtual image.

[0081] Specifically, the display unit 15 is a device capable of projecting images, such as a display or a projector. For example, when the display unit 15 is a projector and projects the virtual image generated by the image generating unit 9 onto the wall surface of the space S, the wall surface of the space S constitutes the screen 19 .

[0082] At this time, the coordinates in the virtual image projected on the screen 19 are associated with the coordinates in the virtual image generated by the image generating unit 9. Here, the coordinates in the virtual image projected on the screen 19 can be acquired through the world coordinates in the space S based on the positional relationship between the display unit 15 and the screen 19, for example.

[0083] The coordinates in the virtual image generated by the image generation unit 9 are set for each pixel. Thus, the area in the screen 19 corresponding to each pixel can be acquired based on the positional relationship between the display unit 15 and the screen 19. In addition, the coordinates of the user in the space S (for example, the standing position of the user in the space S, the position of the user's eyes, etc.) can be acquired based on the skeletal coordinates of the user.

[0084] Therefore, the attention part acquisition unit 8 can acquire the user's attention part in the virtual image based on the user's coordinates, the user's line of sight direction, and the area in the screen 19 corresponding to each pixel. At this time, if the attention part acquisition unit 8 further refers to the user's facial orientation, it can acquire the user's attention part in the virtual image with good accuracy.

[0085] The image generation unit 9 generates a virtual image in which the avatar of each user moves in the virtual space based on the action of the user. At this time, as will be described in detail later, the image generation unit 9 generates a virtual image for each user based on the set virtual space, the avatar, and the initial positions of the avatars of other users in the virtual space, the actions of each user, and the parts that each user pays attention to in the virtual image, and the virtual images of all users share one virtual space and the avatars of other users start to move from the state configured at the initial position.

[0086] Here, the image generation unit 9 generates a virtual image such that, for example, when the user moves forward or backward relative to the screen 19 in the space S, the virtual image in the virtual space moves forward or backward, and when the user moves left or right relative to the screen 19, the virtual image in the virtual space moves left or right. In addition, the image generation unit 9 generates a virtual image such that, for example, the hands and feet of the virtual image in the virtual space move in a manner corresponding to the movements of the hands and feet of the user.

[0087] It should be noted that "shared virtual space" includes not only the situation where the positional relationship of each virtual image in the virtual space is shared, but also the state where the positional relationship of each virtual image in the virtual space is modified. In other words, "shared virtual space" is not limited to the situation where a virtual space is shared, but also includes the situation where the status of the virtual space is shared.

[0088] The output unit 10 outputs information indicating the virtual image for each user generated by the image generating unit 9 to the display unit 15 of each space S via the network 17 , or outputs information indicating the voice of each user acquired by the voice acquiring unit 4 to the speaker 16 of each space S via the network 17 .

[0089] The database unit 11 stores information indicating virtual space candidates, information indicating placement position candidates of avatars of other users in the virtual space, multiple character candidates, information on detailed settings of each character, and the positional relationship between the display unit 15 and the screen 19.

[0090] Next, the flow of a conference using the virtual space sharing system 1 of the present embodiment will be described. First, the flow of a user setting the conference space, the avatar, and the initial positions of the avatars of other users will be described.

[0091] Here, in this embodiment, a case where a first user in a first space, a second user in a second space, a third user in a third space, and a fourth user in a fourth space hold a meeting in a common meeting space is described as an example.

[0092] Figure 2 This is a diagram showing a state of a first space when a conference is held using the virtual space sharing system according to the present embodiment. Figure 3 This is a diagram showing a state of the second space when a conference is held using the virtual space sharing system according to the present embodiment. Figure 4 This is a diagram showing a state of the third space when a conference is held using the virtual space sharing system according to the present embodiment. Figure 5 This is a diagram showing a state of the fourth space when a conference is held using the virtual space sharing system according to the present embodiment.

[0093] First, the input device 18 owned by each of the first user U1 , the second user U2 , the third user U3 , and the fourth user U4 is connected to the virtual space sharing system 1 , and information indicating virtual space candidates is acquired through the input device 18 .

[0094] Then, the first user U1, the second user U2, the third user U3, and the fourth user U4 select a meeting space from a plurality of virtual space candidates via the input device 18. As a result, the space setting unit 5 sets the virtual space as the meeting space.

[0095] However, even if each user U1, U2, U3, U4 does not set a virtual space, for example, by setting a virtual space and setting other users attending the meeting, each user U1, U2, U3, U4 can share a meeting space.

[0096] Next, the input device 18 owned by each of the first user U1, the second user U2, the third user U3, and the fourth user U4 is connected to the virtual space sharing system 1, and information for setting the avatar is obtained from the input device 18. Then, the first user U1, the second user U2, the third user U3, and the fourth user U4 select a desired role from a plurality of role candidates via the input device 18.

[0097] For example, the first user U1, the second user U2, the third user U3, and the fourth user U4 select any one of the character candidates from among a CG character, a CG character in which the facial image of the user acquired by the motion acquisition unit 2 is embedded in the facial part, or a character in which the clothes of the image of the user acquired by the motion acquisition unit 2 are changed, etc., through the input device 18. As a result, the avatar setting unit 6 acquires the selected character.

[0098] Furthermore, the avatar setting unit 6 further displays an image prompting the user to set detailed settings for each selected character on the display unit of the input device 18. Specifically, for example, when a CG character is selected, a plurality of CG characters are displayed as candidate images. When a CG character with a user's facial image embedded is selected, a plurality of CG characters with a user's facial image embedded are displayed as candidate images. When a costumed character is selected, a plurality of clothes, decorations, etc. are displayed as selection images.

[0099] The first user U1, the second user U2, the third user U3, and the fourth user U4 complete desired avatars by selecting candidate images via the input device 18. Thus, the avatar setting unit 6 sets avatars A1, A2, A3, A4 corresponding to the respective users U1, U2, U3, U4.

[0100] At this time, the avatars A1, A2, A3, A4 of the users U1, U2, U3, U4 may be stored in the database unit 11 so that the completed avatars can be reused the next time the virtual space sharing system 1 is used for a meeting or the like.

[0101] Thus, the next time the virtual space sharing system 1 is used for a meeting or the like, the user can omit the need to set the avatar again by calling the avatar stored in the database unit 11 .

[0102] Here, the technique of embedding the facial image of the user acquired by the motion acquisition unit 2 into the facial part of the CG character or changing the clothes of the image of the user acquired by the motion acquisition unit 2 can use common means. At this time, based on the image acquired by the motion acquisition unit 2 or acquired in advance, the virtual image can be formed in a manner corresponding to the balance of the user's body and head.

[0103] Next, the input device 18 owned by each of the first user U1 , the second user U2 , the third user U3 , and the fourth user U4 is connected to the virtual space sharing system 1 , and the input device 18 acquires information indicating candidate locations of avatars of other users in the selected meeting space.

[0104] For example, when the conference space is set as a virtual space, the initial position of the virtual image is determined in such a way that the virtual images of other users are arranged horizontally in the conference space. In this case, the initial position of the virtual image can be determined such that when the virtual image is projected onto the screen 19, the virtual image arranged at the initial position is within the user's field of view.

[0105] like Figure 2 As shown, the first user U1 selects placement position candidates via the input device 18 so that the avatar A2 of the second user U2 , the avatar A3 of the third user U3 , and the avatar A4 of the fourth user U4 are arranged from left to right on the screen 19 .

[0106] like Figure 3 As shown, the second user U2 selects arrangement position candidates via the input device 18 so that the avatar A1 of the first user U1 , the avatar A3 of the third user U3 , and the avatar A4 of the fourth user U4 are arranged from left to right on the screen 19 .

[0107] like Figure 4 As shown, the third user U3 selects arrangement position candidates via the input device 18 so that the avatar A1 of the first user U1 , the avatar A2 of the second user U2 , and the avatar A4 of the fourth user U4 are arranged from left to right on the screen 19 .

[0108] like Figure 5As shown, the fourth user U4 selects arrangement position candidates via the input device 18 so that the avatar A1 of the first user U1 , the avatar A2 of the second user U2 , and the avatar A3 of the third user U3 are arranged from left to right on the screen 19 .

[0109] Thus, the arrangement setting unit 7 sets the initial positions of other users in the conference space that appear on the virtual image displayed in each space S. In this way, each user can appropriately set the initial positions of other users in the conference space.

[0110] Next, the process of generating a virtual image when a meeting is held using the virtual space sharing system 1 of the present embodiment will be described. Here, the following state will be described as an example: a virtual image of a meeting space is projected onto a screen 19 of each space S1, S2, S3, and S4, wherein in the meeting space, virtual images of other users are arranged toward the users existing in each space S1, S2, S3, and S4 as described above, and in such a state, the first user U1 and the second user U2 start a conversation, and the third user U3 and the fourth user U4 watch the conversation between the first user U1 and the second user U2.

[0111] The motion capture devices 12 of the first space S1, the second space S2, the third space S3, and the fourth space S4 input information indicating the motion of the first user U1, the second user U2, the third user U3, and the fourth user U4 to the virtual space sharing system 1. Thus, the motion acquisition unit 2 acquires the motion of each user U1, U2, U3, and U4.

[0112] At the same time, the eye tracking devices 13 of the first space S1, the second space S2, the third space S3, and the fourth space S4 input information indicating the sight lines of the first user U1, the second user U2, the third user U3, and the fourth user U4 to the virtual space sharing system 1. Thus, the sight line acquisition unit 3 acquires the sight line directions of the respective users U1, U2, U3, and U4.

[0113] In addition, information indicating the voices of the first user U1, the second user U2, the third user U3, and the fourth user U4 is input from the microphones 14 of the first space S1, the second space S2, the third space S3, and the fourth space S4 to the virtual space sharing system 1. As a result, the voice acquisition unit 4 acquires the voices of the respective users U1, U2, U3, and U4. The voice acquisition unit 4 outputs the information indicating the acquired voices of the respective users U1, U2, U3, and U4 to the output unit 10.

[0114] Furthermore, the focus area acquisition unit 8 acquires the focus area of ​​each user U1, U2, U3, U4 in the virtual image currently projected on the screen 19 based on the coordinates of the first user U1, the second user U2, the third user U3, and the fourth user U4, the sight lines of the first user U1, the second user U2, the third user U3, and the fourth user U4, and the area in the screen 19 corresponding to each pixel of the display unit 15 of the first space S1, the second space S2, the third space S3, and the fourth space S4.

[0115] The image generation unit 9 generates a virtual image based on the set meeting space, the avatar, and the initial positions of the avatars of other users in the meeting space, the actions of each user, and the parts of the virtual image that each user focuses on. Here, when the meeting space is set as a virtual space, the image generation unit 9 generates a virtual image in a manner in which the avatars of other users having a conversation are talking face to face in the meeting space.

[0116] Specifically, the attention area acquisition unit 8 acquires the area where the avatar A2 of the second user U2 appears in the virtual image projected on the screen 19 in the first space S1 as the attention area of ​​the first user U1. In addition, the voice acquisition unit 4 acquires the voice of the first user U1.

[0117] Meanwhile, the attention area acquisition unit 8 acquires the area where the avatar A1 of the first user U1 appears in the virtual image projected on the screen 19 in the second space S2 as the attention area of ​​the second user U2. In addition, the voice acquisition unit 4 acquires the voice of the second user U2.

[0118] At this time, the image generation unit 9 identifies, for example, which user's avatar is arranged in which pixel region in the virtual image generated by the image generation unit 9 based on the set meeting space, initial positions of other users' avatars, and the like.

[0119] Thus, the image generation unit 9 can recognize that the first user U1 and the second user U2 are having a conversation. Therefore, the image generation unit 9 generates a virtual image in a manner such that the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2 are having a face-to-face conversation in the conference space, and outputs the virtual image to the display unit 15 of the third space S3 and the fourth space S4. At this time, if the image generation unit 9 also refers to the direction of the sound, it can accurately identify the users who are having a conversation.

[0120] Furthermore, the attention area acquisition unit 8 acquires the area where the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2 are talking in the virtual image projected on the screen 19 in the third space S3 as the attention area of ​​the third user U3. In addition, the sound acquisition unit 4 does not acquire the sound of the third user U3.

[0121] On the other hand, the attention area acquisition unit 8 acquires the area where the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2 are talking in the virtual image projected on the screen 19 in the fourth space S4 as the attention area of ​​the fourth user U4. In addition, the sound acquisition unit 4 does not acquire the sound of the fourth user U4.

[0122] Thus, the image generator 9 can recognize that the third user U3 and the fourth user U4 are not participating in the conversation between the first user U1 and the second user U2 but are listening to the conversation between the first user U1 and the second user U2.

[0123] As a result, the image generating unit 9 generates a first virtual image as a virtual image output to the display unit 15 of the first space S1. In the first virtual image, when the virtual image A2 of the second user U2, the virtual image A3 of the third user U3, and the virtual image A4 of the fourth user U4 are arranged in the meeting space, the virtual image A2 of the second user U2, the virtual image A3 of the third user U3, and the virtual image A4 of the fourth user U4 move toward the first user U1 existing in the first space S1, and move based on the actions of each user U2, U3, and U4.

[0124] The image generating unit 9 generates a second virtual image as a virtual image output to the display unit 15 of the second space S2. In the second virtual image, when the virtual image A1 of the first user U1, the virtual image A3 of the third user U3, and the virtual image A4 of the fourth user U4 are arranged in the meeting space, the virtual image A1 of the first user U1, the virtual image A3 of the third user U3, and the virtual image A4 of the fourth user U4 move toward the second user U2 existing in the second space S2, and move based on the actions of each user U1, U3, and U4.

[0125] As a virtual image output to the display unit 15 of the third space S3, the image generating unit 9 generates a third virtual image in which, when the virtual image A1 of the first user U1, the virtual image A2 of the second user U2, and the virtual image A4 of the fourth user U4 are arranged in the meeting space, the third virtual image operates based on the actions of each user U1, U2, and U4 so that the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2 have a face-to-face conversation, and the virtual image A4 of the fourth user U4 watches the conversation between the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2.

[0126] As a virtual image output to the display unit 15 of the fourth space S4, the image generating unit 9 generates a fourth virtual image in which, when the virtual image A1 of the first user U1, the virtual image A2 of the second user U2, and the virtual image A3 of the third user U3 are arranged in the meeting space, the fourth virtual image operates based on the actions of each user U1, U2, and U3 so that the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2 have a face-to-face conversation, and the virtual image A3 of the third user U3 watches the conversation between the virtual image A1 of the first user U1 and the virtual image A2 of the second user U2.

[0127] By the way, when the third user U3 and the fourth user U4 are not watching the conversation between the virtual image A1 of the first user U1 and the virtual image S2 of the second user U2, the image generating unit 9 generates a virtual image of viewing other parts in the virtual space based on the parts and facial orientations that the third user U3 and the fourth user U4 are paying attention to.

[0128] The image generation unit 9 outputs information indicating the first to fourth virtual images generated as described above to the output unit 10. The output unit 10 outputs information indicating the first virtual image to the display unit 15 of the first space S1, and outputs information indicating the voices of the other users U2, U3, and U4 to the speaker 16 of the first space S1. In addition, the output unit 10 outputs information indicating the second virtual image to the display unit 15 of the second space S2, and outputs information indicating the voices of the other users U1, U3, and U4 to the speaker 16 of the second space S2.

[0129] Furthermore, the output unit 10 outputs information representing the third virtual image to the display unit 15 of the third space S3, and outputs information representing the voices of the other users U1, U2, and U4 to the speaker 16 of the third space S3. Furthermore, the output unit 10 outputs information representing the fourth virtual image to the display unit 15 of the fourth space S4, and outputs information representing the voices of the other users U1, U2, and U3 to the speaker 16 of the fourth space S4.

[0130] Then, the display unit 15 of each space S1, S2, S3, and S4 projects the input virtual image onto the screen 19, and the speaker 16 outputs the voice of other users. Each user U1, U2, U3, and U4 performs actions or speaks on the projected virtual image, so that the image generation unit 9 generates virtual images that are continuously output to other users as animation.

[0131] As described above, the virtual space sharing system 1 and the virtual space sharing method of the present embodiment can output virtual images accompanying the user's movements in the mutual direction. Therefore, using the virtual space sharing system 1 and the virtual space sharing method of the present embodiment, users can achieve good communication with each other.

[0132] In addition, when the conference space is selected as the virtual space, the virtual images are generated in such a manner that the virtual images of other users are arranged horizontally in the conference space regardless of the positions of the users in the space S, so that it is easy to visually recognize the virtual images of other users and communicate with other users. In addition, since the virtual images are generated in such a manner that the virtual images of the users having a conversation in the conference space are talking face to face with each other, a sense of presence can be given to the conference space.

[0133] <Implementation Method 2>

[0134] When a conference is held with other users, the image generation unit 9 may generate a virtual image so that the virtual image of the other user having the conversation is arranged in front of the user existing in the space. Figure 6 This is a diagram showing a state of a first space when a conference is held using the virtual space sharing system according to the present embodiment. Figure 7 This is a diagram showing a state of the second space when a conference is held using the virtual space sharing system according to the present embodiment.

[0135] Here, in this embodiment, a case where a first user U11 in a first space S11 and a second user U12 in a second space S12 hold a meeting is described as an example. At this time, it is assumed that the avatar A11 of the first user U11 selects a costumed character, and the second avatar A12 of the second user U12 selects a costumed character.

[0136] In this state, when the first user U11 existing in the first space S11 speaks to the avatar A12 of the second user U12 projected on the screen 19 of the first space S11, Figure 6 As shown, the image generation unit 9 generates a virtual image in which the avatar A12 of the second user U12 is arranged substantially in front of the first user U11 in the meeting space.

[0137] In addition, when the second user U12 existing in the second space S12 speaks to the virtual image A11 of the first user U11 projected on the screen 19 of the second space S12, as shown in FIG. Figure 7 As shown, the image generation unit 9 generates a virtual image in which the avatar A11 of the first user U11 is arranged substantially in front of the second user U12 in the meeting space.

[0138] It should be noted that, when there are multiple other users participating in the conference, the avatars of other users who are not participating in the conversation may be arranged to avoid the avatars of other users who are participating in the conversation.

[0139] <Implementation Method 3>

[0140] In the first embodiment, the meeting space is selected as the virtual space. However, even when the presentation space is selected as the virtual space, the virtual space sharing system 1 of the first embodiment can be used to implement the same.

[0141] Figure 8 This is a diagram showing a state of a space where a user performing a presentation exists when a presentation is performed using the virtual space sharing system according to the present embodiment. Fig. 9 This is a diagram showing a state of a space in which users who listen to a presentation using the virtual space sharing system according to the present embodiment exist.

[0142] The following describes a flow when a user selects a demonstration space as a virtual space and performs a demonstration. Here, in this embodiment, a case where the user U21 performs a demonstration using the material D to a plurality of users U22 is described as an example.

[0143] First, the process of the user setting the presentation space and the initial positions of the avatars of other users will be described. It should be noted that the process of each user selecting an avatar is the same as that in the first embodiment, so the description thereof will be omitted.

[0144] The input device 18 owned by each user U21, U22 is connected to the virtual space sharing system 1, and information indicating virtual space candidates is obtained through the input device 18. Then, each U21, U22 selects a presentation space from a plurality of virtual space candidates via the input device 18. At this time, the user U21 can select a presentation space via the input device 18, and can input that the user U21 himself is the host for presentation.

[0145] Thus, the space setting unit 5 sets the virtual space as the demonstration space and sets the user U21 as the user who performs the demonstration. However, even if the individual users U21 and U22 do not set the demonstration space, for example, by setting the demonstration space on behalf of the user or others and setting the users who attend the demonstration, the demonstration space can be shared by the users U21 and U22.

[0146] Next, the input device 18 owned by each of the users U21 and U22 is connected to the virtual space sharing system 1 to acquire information indicating the arrangement position candidates of the avatars of other users in the presentation space.

[0147] For example, when the presentation space is set as a virtual space, in the presentation space of the virtual image projected on the screen 19 of the space S21 where the user U21 performing the presentation exists, an area for loading presentation material D is determined at the upper right side of the screen 19. In the area for loading material D, for example, the user U21 can load the material D by selecting a file of the material D through the input device 18.

[0148] Then, in the presentation space, the initial position of the avatar A22 of the other user U22 is determined so that the avatar A22 of the other user U22 is arranged on the left side of the screen 19 facing the seated user U21 existing in the space S21 .

[0149] Therefore, if Figure 8 As shown, the user U21 selects arrangement position candidates so that the avatars A22 of each other user U22 sit at each seat via the input device 18. At this time, the seat can be selected so that the avatars A22 of the other users U22 important for the presentation sit at the front row.

[0150] It should be noted that the avatar A22 of the other user U22 may be configured in a manner that does not interfere with the data D. In addition, the initial position of the avatar A22 of the other user U22 may be determined so that when the virtual image is projected onto the screen 19 , the avatar A22 configured at the initial position is within the field of view of the user U21 .

[0151] On the other hand, in the presentation space represented by the virtual image of the screen 19 projected onto the space S22 where the user U22 listening to the presentation exists, an area where the data D and the avatar A21 of the other user U21 are arranged is determined approximately in the center of the presentation space.

[0152] In addition, in the presentation space, the initial position of the avatar A22 of the other user U22 is determined so that the avatar A22 of the other user U22 avoids the document D and the avatar A21 of the other user U21 and is arranged to sit on a seat facing the document D.

[0153] Therefore, if Fig. 9 As shown, the user U22 present in the space S22 selects the configuration position candidate in such a manner that the avatar A22 of each other user U22 is seated at each seat via the input device 18. At this time, the avatar A21 of the other user U21 is configured to face the inside of the space S22 so as to be presented to the user U22 present in the space S22 as if next to the document D.

[0154] Thus, the configuration setting unit 7 sets the initial positions of other users U21 and U22 in the demonstration space that appear in the virtual image projected in each space S22. At this time, since the user U21 is set as the user who performs the demonstration as described above, the configuration setting unit 7 can set the virtual image A21 of the user U21 in the area.

[0155] Next, the process of generating a virtual image when the user U21 uses the demonstration space to demonstrate to the user U22 will be described. Here, the following state is used as an example for description: in the state where the material D and the virtual image of the other user U22 are configured in the demonstration space as described above and are projected onto the screen 19 of the space S21 where the user U21 is present, the user U21 points at the material D in the projected virtual image and demonstrates toward the virtual image A22 of the other user U22, and in the state where the virtual images A21 and A22 of the other users U21 and U22 are configured in the demonstration space as described above and are projected onto the screen 19 of the space S22 where the user U22 is present, the user U22 listens to the demonstration toward the virtual image A21 of the other user U21 and the material D in the projected virtual image.

[0156] In this state, information indicating the motion of the users U21 and U22 is input from the motion capture devices 12 in the spaces S21 and S22 to the virtual space sharing system 1. Thus, the motion acquisition unit 2 acquires the motion of the users U21 and U22.

[0157] Specifically, the action acquisition unit 2 acquires the action of the user U21 who points at the material D in the virtual image projected on the screen 19 of the space S21 and makes a presentation toward the avatar A22 of the other user U22 .

[0158] Furthermore, the action acquisition unit 2 acquires the action of the user U22 present in the space S22 who listens to the presentation toward the avatar A21 of another user U21 while referring to the material D in the virtual image projected on the screen 19 of the space S22 .

[0159] At the same time, information indicating the sight lines of the users U21 and U22 is input from the eye tracking device 13 in the spaces S21 and S22 to the virtual space sharing system 1. Thus, the sight line acquisition unit 3 acquires the sight line directions of the users U21 and U22.

[0160] Specifically, the sight line acquisition unit 3 acquires the sight line direction of the user U21 who is presenting toward the avatar A22 of the other user U22 while referring to the data D in the virtual image projected on the screen 19 of the space S21 .

[0161] Furthermore, the sight line acquisition unit 3 acquires the sight line direction of the user U22 present in the space S22 who is listening to the presentation toward the avatar A21 of another user U21 while referring to the data D in the virtual image projected on the screen 19 of the space S22 .

[0162] Furthermore, information indicating the voices of the users U21 and U22 is input from the microphones 14 of the spaces S21 and S22 to the virtual space sharing system 1. Thus, the voice acquisition unit 4 acquires the voices of the users U21 and U22.

[0163] Specifically, the voice acquisition unit 4 acquires the voice of the user U21 who is giving a presentation in the space S21. In addition, the voice acquisition unit 4 acquires the voice of the user U22 who is listening to the presentation in the space S22 and appropriately inquiring the user U21.

[0164] Then, the focus area acquisition unit 8 acquires the area that users U21 and U22 are focused on in the virtual image currently projected onto the screen 19 based on the coordinates of users U21 and U22, the line of sight directions of users U21 and U22, and the area in the screen 19 corresponding to each pixel of the display unit 15 of the spaces S21 and S22.

[0165] Specifically, the attention site acquisition unit 8 acquires, as the attention site of the user U21 , a region where the avatar A22 of another user U22 and the data D are reflected in the virtual image projected on the screen 19 in the space S21 .

[0166] Furthermore, the attention site acquisition unit 8 acquires, as the attention site of the user U22 in the space S22 , a region where the avatar A21 of another user U21 and the data D are reflected in the virtual image projected on the screen 19 in the space S22 .

[0167] The image generation unit 9 generates a virtual image based on the set presentation space, the avatar, the initial positions of the avatars of other users in the presentation space, the actions of each user, and the part of the virtual image that each user pays attention to.

[0168] Specifically, as a virtual image output to the display unit 15 of the space S21, the image generating unit 9 Figure 8 As shown, the first virtual image is generated, in which the data D is loaded in the upper right corner of the demonstration space, and the virtual image A22 of the other user U22 faces the user U21 in the space S21 sitting on the seat on the left side of the demonstration space, and the virtual image A22 of the other user U22 moves based on the action of the other user U22.

[0169] In addition, as a virtual image to be output to the display unit 15 of the space S22, the image generating unit 9 generates a second virtual image in which Fig. 9 As shown, the material D is loaded in the approximate center of the demonstration space, and, next to the material D, the virtual image A21 of the other user U21 moves based on the movement of the other user U21 in a manner of indicating the material D toward the user U22 existing in the space S22 while performing the demonstration, and, in the demonstration space, the virtual image A22 of the other user U22 avoids the material D and the virtual image A21 of the other user U21 and sits facing the material D, and the virtual image A22 of the other user U22 moves based on the movement of the other user U22.

[0170] At this time, the image generating unit 9 may generate the second virtual image in such a manner that the position of the finger of the user U21 pointing to the document D in the space S21 corresponds to the position of the finger of the avatar A21 of the user U21 pointing to the document D in the second virtual image.

[0171] Furthermore, the image generating unit 9 may generate a virtual image in such a manner that the document D is shifted to a previous or next page or the document D is enlarged or reduced based on the hand movement (e.g., up and down movement) of the user U21 or the like in the area where the document D is displayed.

[0172] Furthermore, the image generating unit 9 may generate the second virtual image such that the position of the user U21 relative to the document D in the space S21 corresponds to the position of the avatar A21 of the user U21 relative to the document D in the presentation space.

[0173] The image generation unit 9 outputs information indicating the first and second virtual images generated as described above to the output unit 10. The output unit 10 outputs information indicating the first virtual image to the display unit 15 of the space S21, and outputs information indicating the voice of the other user U22 to the speaker 16 of the space S21. In addition, the output unit 10 outputs information indicating the second virtual image to the display unit 15 of the space S22, and outputs information indicating the voices of the other users U21 and U22 to the speaker 16 of the space S22.

[0174] Furthermore, the display unit 15 of the spaces S21 and S22 projects the virtual image onto the screen 19, and the speaker 16 outputs the voices of other users. The users U21 and U22 perform actions or speak in response to the projected virtual image, whereby the image generation unit 9 generates a virtual image output to other users as an animation.

[0175] Here, when the user U22 approaches the data D in the space S22 and makes additional explanations or inquiries next to the data D, if the user U22 passes a predetermined period within the predetermined range of the data D, the image generating unit 9 can generate a virtual image by swapping the position of the virtual image A21 of the user U21 and the position of the virtual image A22 of the user U22 in the demonstration space.

[0176] Furthermore, when it is determined that one of the users U21 and U22 is more suitable for holding a meeting during the presentation, one of the users U21 and U22 may change the virtual space to a meeting space during the presentation via the input device 18 .

[0177] Fig.10 1 is a diagram showing a space situation when the virtual space is changed to a meeting space in a demonstration. When the space is changed to the meeting space, the image generation unit 9 performs the same operation as in the first embodiment. Fig.10 As shown, a virtual image in which other users are arranged side by side in the conference space is generated.

[0178] At this time, when the presentation space is changed to the meeting space, the image generation unit 9 may generate a virtual image in which the user existing in the space is the center and the virtual images of other users are arranged in the meeting space. It should be noted that, when the presentation space is changed to the meeting space, a virtual image in which the virtual images of other users are arranged sitting in seats in the meeting space may also be generated.

[0179] When the demonstration space is selected as the virtual space in this way, a virtual image configured with the virtual image A22 of the other user U22 is generated in such a way that the user U22 existing in the space S22 can easily see the data D in the demonstration space and the virtual image A21 of the other user U21 regardless of the positions of the users U21 and U22 in the spaces S21 and S22, thereby making it easy for the user U22 existing in the space S22 to listen to the demonstration.

[0180] Furthermore, since the virtual image is generated so that the avatar A21 of the user U21 performing the presentation in the presentation space is arranged next to the material D toward the user U22 existing in the space S22 , a sense of presence can be provided in the presentation space.

[0181] <Implementation Method 4>

[0182] In the first embodiment, the meeting space is selected as the virtual space. However, even if the class space is selected as the virtual space, the virtual space sharing system 1 of the first embodiment can be used to implement the same.

[0183] Here, in this embodiment, a case where a teacher and students conduct a dance lesson using a public lesson space is described as an example. Fig.11 This is a diagram showing a state of a space where a teacher exists when a dance lesson is conducted using the virtual space sharing system according to the present embodiment. Fig.12 This is a diagram showing a state of a space in which students exist when a dance lesson is conducted using the virtual space sharing system according to the present embodiment.

[0184] When the course space is selected as the virtual space, for example, as a virtual image output to the display unit 15 of the space S31 where the teacher U31 exists, Fig.11 As shown, the image generation unit 9 generates a first virtual image in which the virtual images A32 of each student U32 are arranged horizontally in the course space and move based on the movement of the student U32. At this time, the arrangement of the virtual images A32 of the students U32 can be set to a predetermined initial position by the teacher U31 via the input device 18, similarly to the first embodiment.

[0185] In addition, as a virtual image output to the display unit 15 of the space S32 where the student U32 exists, Fig.12 As shown, the image generation unit 9 generates a second virtual image that moves based on the actions of the other students U32 and the teacher U31 in a state where the virtual images A32 of the other students U32 and the virtual image A31 of the teacher U31 are arranged in a horizontal arrangement in the course space. At this time, the arrangement of the virtual images A32 of the other students U32 and the virtual image A31 of the teacher U31 can be set to a predetermined initial position by the student U32 existing in the space S32 via the input device 18, similarly to the first embodiment.

[0186] It should be noted that in the present embodiment, virtual images of the teacher U31's virtual image A31 and the student U32's virtual image A32 are generated and arranged horizontally in the course space, but the configuration of the teacher U31's virtual image A31 and the student U32's virtual image A32 can be appropriately determined according to the content of the course.

[0187] <Implementation method 5>

[0188] In the first embodiment, the meeting space is selected as the virtual space. However, even when the office space is selected as the virtual space, the virtual space sharing system 1 of the first embodiment can be used to implement the same.

[0189] Here, the office space is, for example, a three-dimensional space that reproduces an actual company office, and a plurality of desks arranged in the three-dimensional space are determined as candidate locations for the user's avatar. When each user selects a candidate location through his / her input device 18, the location setting unit 7 sets the desk in the office space selected by the user as the initial location of the user's avatar.

[0190] The image generation unit 9 generates a virtual image such that the virtual image corresponds to the user's movements, and when the user is seated, the virtual image of the user is seated at the desk in the office space, and when the user moves relative to the screen 19, the virtual image of the user moves from the desk in the office space. Furthermore, the image generation unit 9 generates the virtual image within a preset field of view range in the field of view direction of the virtual image of each user in the office space.

[0191] The following describes the process of generating a virtual image when users share an office space. Here, the following state is used as an example for description: the third user in the third space sees the first user in the first space approaching the virtual image of the second user and talking, while the second user in the second space approaches the virtual image of the first user and talks.

[0192] Fig.13 This is a diagram showing a state of a space where a first user exists when the first user approaches a second user and talks to the second user. Fig.14 This is a diagram showing a state of a space where a third user exists when a first user approaches a second user and talks. Fig.15 This is a diagram showing a state of a space where a first user exists when a second user approaches a first user who is speaking and speaks. Fig.16 This is a diagram showing a state of a space where a third user exists when a second user approaches and speaks to a first user who is speaking.

[0193] First, if Fig.13 As shown, the following virtual images are projected onto the screen 19 of the first space S41: in the field of view of the virtual image A41 of the first user U41 in the office space, the virtual image A42 of the second user U42 and the virtual image A43 of the third user U43 are arranged at initial positions and sit on their desks.

[0194] In this state, if Fig.13As shown, when the first user U41 approaches the avatar A42 of the second user U42 in the virtual image projected on the screen 19 of the first space S41 and speaks, the image generation unit 9 generates, as a virtual image output to the display unit 15 of the second space S42, a virtual image in which the avatar A41 of the first user U41 approaches the second user U42 existing in the second space S42 and speaks within the field of view of the avatar A42 of the second user U42 in the office space, based on the actions of the respective users U41, U42, and U43 acquired by the action acquisition unit 2. At this time, if the avatar A43 of the third user U43 exists in the field of view of the avatar A42 of the second user U42, the avatar A43 of the third user U43 is also included.

[0195] In addition, as a virtual image output to the display unit 15 of the third space S43, Fig.14 As shown, the image generation unit 9 generates a virtual image in which the avatar A41 of the first user U41 approaches and talks to the avatar A42 of the second user U42 sitting at a desk in an office space based on the actions of each user U41, U42, and U43 acquired by the action acquisition unit 2.

[0196] Furthermore, when the second user U42 approaches the virtual image A41 of the first user U41 in the virtual image projected on the screen 19 of the second space S42 and speaks, the virtual image output to the display unit 15 of the first space S41 is as follows: Fig.15 As shown, the image generation unit 9 generates a virtual image in which the avatar A42 of the second user U42 approaches and talks to the first user U41 existing in the first space S41 within the field of view of the avatar A41 of the first user U41 in the office space, based on the actions of the users U41, U42, and U43 acquired by the action acquisition unit 2. At this time, if the avatar A43 of the third user U43 exists in the field of view of the avatar A41 of the first user U41, the avatar A43 of the third user U43 is also included.

[0197] In addition, as a virtual image output to the display unit 15 of the third space S43, Fig.16 As shown, the image generation unit 9 generates a virtual image in which the avatar A41 of the first user U41 and the avatar A42 of the second user U42 are talking in a close state in the office space based on the actions of the users U41, U42, and U43 acquired by the action acquisition unit 2.

[0198] As described above, in the present embodiment, a virtual image is generated in which the avatar moves based on the movements of each user within the field of view of the avatar of the user in the office space, thereby realizing a virtual office with a sense of presence.

[0199] Here, the input device 18 may be operable so that the conversation between the first user U41 and the second user U42 is not heard by the third user U43. This allows a conversation that is not to be heard by a third person to be conducted in an office space.

[0200] In addition, the volume of the conversation between the first user U41 and the second user U42 output in the third space S43 can be adjusted based on the distance between the avatar A43 of the third user U43 and the avatar A41 of the first user U41 and the avatar A42 of the second user U42 in the office space.

[0201] Furthermore, for example, when other users do not want to hear the sound in the space, the microphone 14 can be muted via the input device 18. In addition, for example, the volume of the conversation between other users can be adjusted via the input device 18.

[0202] <Other Implementation Methods>

[0203] In the above-mentioned Embodiments 1 to 5, the present disclosure is described as a hardware configuration, but the present disclosure is not limited thereto. The present disclosure can also realize the processing of each component by causing a CPU (Central Processing Unit) to execute a computer program.

[0204] For example, the virtual space sharing system 1 of the above-mentioned embodiment can have the following hardware configuration. Fig.17 This is a diagram showing an example of a hardware configuration included in the virtual space sharing system 1 .

[0205] Fig.17 The device 51 shown includes a processor 53, a memory 54, and an interface 52. The virtual space sharing system 1 described in the above embodiment is realized by the processor 53 reading and executing a program stored in the memory 54. That is, the program is a program for causing the processor 53 to function as the virtual space sharing system 1 of the above embodiment.

[0206] The above-mentioned program can be stored using various types of non-transitory computer readable media and supplied to a computer (including a computer of an information notification device). Non-volatile computer readable media include various types of tangible recording media (tangible storage medium). Examples of non-volatile computer readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives), optical magnetic recording media (e.g., magneto-optical disks). Moreover, this example includes CD-ROM (Read Only Memory), CD-R, CD-R / W. Moreover, this example includes semiconductor memory (e.g., mask ROM, PROM, EPROM, flash ROM, RAM). In addition, the program can also be supplied to a computer by various types of volatile computer readable media (transitory computer readable medium). Examples of volatile computer readable media include electrical signals, optical signals, and electromagnetic waves. Volatile computer readable media can supply programs to a computer via wired communication paths such as wires and optical fibers or wireless communication paths.

[0207] The present disclosure is not limited to the above-described embodiments, and appropriate changes can be made without departing from the spirit and scope of the present disclosure.

[0208] For example, a plurality of speakers 16 may be configured in the space, and the sound may be output from the speakers 16 based on the direction of the sound emitted by the user. That is, in the case where the virtual image of another user speaks toward the user existing in the space, the speakers 16 may be configured and controlled in such a way that the voice of the other user is output from the screen 19 side toward the user. In addition, in the case where other users have a face-to-face conversation in the virtual space, the speakers 16 may be configured and controlled in such a way that the voice of the other user is output in the direction of the sound emitted by the virtual image of the other user. In other words, the direction of the face of the virtual image of the user and the direction in which the voice of the user is output in the space may be roughly the same. Thus, a virtual space with a sense of presence can be realized.

[0209] Furthermore, the sound acquisition unit 4 may acquire the sound in the space via the microphone 14, and output the sound from the speaker 16 based on the direction of the sound in the space. Thus, a real-time space with a sense of presence can be realized.

[0210] For example, the image generation unit 9 may not reflect part of the user's actions in the actions of the user's virtual image in the virtual space according to the selected virtual space. For example, in the demonstration of Embodiment 3, a virtual image may be generated in which the user U22's virtual image A22 is seated even if the user U22 is standing or sitting. Thus, the user U22 in the space S22 can focus on the demonstration. It should be noted that part of the user's actions may not be reflected in the actions of the user's virtual image in the virtual space by inputting the actions that are not desired to be reflected from the outside via an input device.

[0211] From the disclosure thus described, it is obvious that the embodiments of the disclosure may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the disclosure, and all such modifications obvious to those skilled in the art are intended to be included within the scope of the following claims.

Claims

1. A virtual space sharing system, comprising: an action acquisition unit, which acquires actions of users in each of the plurality of spaces; an image generating unit for generating a virtual image in which a virtual avatar corresponding to each of the users moves in a virtual space shared by the plurality of spaces based on the movement of the user; an output unit that outputs a virtual image of avatars of users other than the user himself moving in the virtual space to a display unit arranged in the space of the user; A sight line acquisition unit, which acquires the sight line direction of the user; as well as The attention part acquisition unit acquires the part that the user pays attention to in the virtual image based on the sight line direction of the user, Among them, the image generating unit generates the virtual image in a manner that the virtual images of other users who are having conversations are having face-to-face conversations with each other in a conference space based on the part that the user is paying attention to in the virtual image, and in a manner that the other users who are not having conversations are watching the other users who are having face-to-face conversations in the conference space when the part that the other users are paying attention to is the other users who are having conversations.

2. The virtual space sharing system according to claim 1, in, have: a space setting unit configured to set the virtual space based on a space request input from the outside, An initial position for configuring the virtual image of another user is determined in the virtual space according to the virtual space.

3. The virtual space sharing system according to claim 2, in, have: The arrangement setting unit sets an arrangement of an avatar of each other user for each of the initial positions determined in the virtual space based on an avatar arrangement request input from the outside.

4. The virtual space sharing system according to any one of claims 1 to 3, in, have: a voice acquisition unit, the voice acquisition unit acquiring the voice of the user, When the virtual space is a demonstration space, the image generating unit generates the virtual image in such a manner that a virtual image of a user performing a demonstration using data loaded into the demonstration space and the data do not interfere with virtual images of other users and are displayed larger than the virtual images of the other users.

5. The virtual space sharing system according to any one of claims 1 to 3, in, have: a voice acquisition unit, the voice acquisition unit acquiring the voice of the user, When the virtual space is the meeting space, the image generation unit generates the virtual image such that virtual images of other users are arranged side by side in the meeting space.

6. The virtual space sharing system according to claim 1, in, The image generation unit generates the virtual image in a preset field of view range in a field of view direction of the avatar of the user in the virtual space.

7. A virtual space sharing method, comprising the following steps: Obtaining actions of users in each of the plurality of spaces in time series; generating a virtual image in which a virtual avatar corresponding to each of the users moves in a virtual space shared by the plurality of spaces based on the time-series movements of the user; and outputting a virtual image of avatars of users other than the user himself moving in the virtual space to a display unit arranged in the space of the user, When generating the virtual image, the user's line of sight direction is obtained, and based on the user's line of sight direction, the part that the user is paying attention to in the virtual image is obtained. Based on the part that the user is paying attention to in the virtual image, the virtual image is generated in a manner such that the virtual images of other users who are having a conversation are having a face-to-face conversation with each other in a conference space, and when the part that other users who are not having a conversation are paying attention to is the other users who are having a conversation, the virtual image of other users who are not having a conversation are generated in a manner such that the other users who are not having a conversation are watching the other users who are having a face-to-face conversation in the conference space.

8. A computer-readable recording medium having recorded thereon a virtual space sharing program for causing a computer to execute the following processing: Obtaining actions of users in each of the plurality of spaces in time series; generating a virtual image in which a virtual avatar corresponding to each of the users moves in a virtual space shared by the plurality of spaces based on the time-series movements of the user; and outputting a virtual image of avatars of users other than the user himself / herself moving in the virtual space to a display unit arranged in the space of the user, When generating the virtual image, the user's line of sight direction is obtained, and based on the user's line of sight direction, the part that the user is paying attention to in the virtual image is obtained. Based on the part that the user is paying attention to in the virtual image, the virtual image is generated in a manner such that the virtual images of other users who are having a conversation are having a face-to-face conversation with each other in a conference space, and when the part that other users who are not having a conversation are paying attention to is the other users who are having a conversation, the virtual image of other users who are not having a conversation are generated in a manner such that the other users who are not having a conversation are watching the other users who are having a face-to-face conversation in the conference space.

Images