Information processing device, information processing method, and program

Abstract

[Problem] To achieve natural interaction between an artist and an audience through a virtual space. [Solution] This information processing device comprises: a presentation unit which presents an image showing an audience avatar group disposed in at least one virtual space to a display area which can be viewed by the artist corresponding to the artist avatar disposed in the virtual space; and a control unit which controls the artist avatar in the at least one virtual space on the basis of an action of the artist on the audience avatar group shown in the image presented in the display area.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A virtual live concert in a virtual space that can be accessed simultaneously by many people makes it possible for the artist (performer) and the audience to be apart of the same location. In a live concert in a virtual space, audience avatars representing the audience and avatars representing the artist (performer avatar) are placed in the virtual space, and the audience recognizes the performer avatar placed in the virtual space as the artist. The performer avatar moves in sync with the actual movements of the artist in a studio, etc. In such a virtual live concert in a virtual space, it is possible to hold live concerts not only at a single venue, but also at multiple venues simultaneously.

[0003] Currently, in live performances where the artist and audience are not in the same place, the following types of artist-user interactions are possible. For example, in live viewing, the artist provides fan service by waving at the streaming camera. Also, in live streaming to devices, audience members can write comments that the artist reads.

[0004] In a virtual live performance in a virtual space, artists and audiences can share the same space, so it would be ideal to take advantage of this advantage and have a system in place that allows artists to observe and respond to the behavior of users (user avatars) at each venue, allowing for natural interaction between artists and audiences. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-37092 [Patent Document 2] Japanese Patent Publication No. 2020-042575 [Patent Document 3] Japanese Patent Application Laid-Open No. 2014-123376 Summary of the Invention [Problem to be solved by the invention]

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and aims to realize natural interaction between performers and audiences through a virtual space. [Means for solving the problem]

[0007] The information processing device of the present disclosure includes a presentation unit that presents images representing a group of audience avatars arranged in at least one virtual space in a display area visible to an actor corresponding to an actor avatar arranged in the virtual space, and a control unit that controls the actor avatar in the at least one virtual space based on the actor's actions toward the group of audience avatars represented in the images presented in the display area.

[0008] The information processing method disclosed herein presents images representing a group of audience avatars arranged in at least one virtual space in a display area visible to an actor corresponding to a performer avatar arranged in the virtual space, and controls the performer avatar in the at least one virtual space based on the actor's actions toward the group of audience avatars represented in the images presented in the display area.

[0009] The computer program of the present disclosure causes a computer to execute the steps of presenting images representing a group of audience avatars arranged in at least one virtual space in a display area visible to an actor corresponding to an actor avatar arranged in the virtual space, and controlling the actor avatar in the at least one virtual space based on the actor's actions toward the group of audience avatars represented in the images presented in the display area. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an information processing device according to the present disclosure. [Figure 2] A diagram of the system for capturing the performer's movements. [Figure 3] A diagram of modeling that creates content from captured movements. [Figure 4] FIG. 10 is a diagram illustrating an example of a display method for displaying a performer avatar. [Figure 5] FIG. 1 is a diagram illustrating an example of a presentation unit according to an embodiment of the present disclosure. [Figure 6] A diagram showing the development of seating in a virtual space. [Figure 7] A diagram showing how the performer's movements are reflected by the performer avatar. [Figure 8] A diagram showing the performer and performer avatar performing interaction actions. [Figure 9] A view of the performer's avatar from the audience. [Figure 10] A diagram showing the relative positions and appearances of audience avatars in the virtual space. [Figure 11] A diagram showing an actor interacting with an audience avatar in a virtual space. [Figure 12] FIG. 10 shows a view of a performer avatar from the perspective of an audience avatar in a virtual space that is not the target of interaction. [Figure 13] 10 is a flowchart illustrating an example of a processing flow of an information processing device according to an embodiment of the present disclosure. [Figure 14] 10 is a flowchart illustrating an example of the flow of a motion editing process of an information processing device according to an embodiment of the present disclosure. [Figure 15] 15 is a flowchart illustrating a modified example of FIG. 14. [Figure 16] 10 is a flowchart illustrating a process for editing the motion of a performer avatar in accordance with attributes of a virtual space. [Figure 17] FIG. 10 is a diagram illustrating an example of a modified example of a presentation unit according to an embodiment of the present disclosure. [Figure 18] FIG. 10 is a diagram showing an example of a modified example of a presentation unit of an HMD according to an embodiment of the present disclosure. [Figure 19] FIG. 18 shows an example of a display of a virtual space. [Figure 20]FIG. 1 is a diagram showing an example of the hardware configuration of a computer that executes a series of processes of an information processing device according to the present disclosure using a program. DETAILED DESCRIPTION OF THE INVENTION

[0011] 1 is a block diagram of an information processing system 100 according to an embodiment of the present disclosure. The information processing system 100 includes a performer-side device 300 and an audience-side device 400 as information processing devices according to the present embodiment.

[0012] Performer side device 300 (information processing device) includes motion acquisition unit 310, gaze acquisition unit 320, first positional relationship calculation unit 330, second positional relationship calculation unit 340, motion editing unit 350, video data generation unit 360, data distribution unit 370, data receiving unit 380, audience image synthesis unit 390, and presentation unit 200. The control unit according to this embodiment has at least one function of motion acquisition unit 310, gaze acquisition unit 320, first positional relationship calculation unit 330, second positional relationship calculation unit 340, motion editing unit 350, video data generation unit 360, data distribution unit 370, data receiving unit 380, and audience image synthesis unit 390.

[0013] The audience-side device 400 includes a data receiving unit 410, an audience presentation unit 420, an audience image generation unit 430, and an audience-side transmission unit 440. Although only one audience-side device 400 is shown, in reality, there may be multiple audience-side devices 400 depending on the number of audience members, audience groups, virtual spaces, etc. For example, there may be an audience-side device 400 for each audience member.

[0014] In this embodiment, performer avatars corresponding to performers and audience avatars corresponding to audience members are placed in a virtual space, i.e., a venue, to realize natural interactions between the performers and the audience in a situation where the performer avatars are performing a live performance. Here, the performer avatar is an object placed in the virtual space that reflects the movements of the performers. The virtual space and performer avatars will be described below.

[0015] (Virtual space) In the virtual space of an embodiment of the present disclosure, a venue where a live performance will be held is set up, and the venue is equipped with performer avatars, a group of audience avatars which is a collection of audience avatars, a stage on which the performer avatars perform, and audience seats where each audience avatar can view the performer avatars on stage.

[0016] One or more virtual spaces are provided, and a venue is set up for each virtual space. Attribute information may be set for each virtual space. The attribute information of a virtual space may be, for example, the attributes of a group of audience avatars placed in the virtual space (such as the language used or place of residence of the group of audience members). In the virtual space, services or actions may be performed according to the attribute information of the virtual space. For example, the language spoken by a performer may be converted into the language indicated by the attribute information of the virtual space. Furthermore, audience members may select a virtual space with the attribute information they desire and participate in a live performance in the selected virtual space.

[0017] (Performer avatar) Fig. 2 shows a shooting environment for shooting the movements (motions) of a performer 10 and generating a 3D model. Fig. 3 is a diagram illustrating modeling for generating a 3D model 12 from the acquired motion. Fig. 4 is a diagram showing an example in which the generated 3D model 12 is displayed as a performer avatar 11.

[0018] In FIG. 2, performer 10 performs a live performance in a shooting environment. Camera 321 captures images of performer 10 at a fixed sampling rate. There may be a single camera 321, or multiple cameras as shown in FIG. 2 to capture images of the performer from various angles. A display unit (presentation unit 200 in FIG. 1) that displays images of audience avatars in the virtual space is placed in front of or around performer 10. This allows performer 10 to perform while watching the status of audience avatars in each venue. A microphone may be placed in the shooting environment of FIG. 2 (for example, performer 10 may be made to hold the microphone) and the voice of performer 10 may be collected at a fixed sampling rate.

[0019] Next, from a collection of image data acquired by camera 321, a three-dimensional data object is generated as a three-dimensional model 12 as shown in Fig. 3. For generating the three-dimensional model, for example, a Visual Hull method or the like can be used.

[0020] Next, by rendering the generated 3D model 12 from an arbitrary viewpoint, the performer avatar 11 can be displayed on the audience device (audience presentation unit 420) as shown in FIG. 4. The performer avatar 11 may be transmitted to the audience device by rendering the 3D model 12 as video data. Motion data may be transmitted to the audience device, and the performer avatar 11 may be rendered on the audience device. The performer avatar 11 can be played on a display 401 as shown in FIG. 4(a) or on a head-mounted display 402 as shown in FIG. 4(b). In this way, the performer avatar 11 is presented to the audience member corresponding to the audience avatar. The display 401 in FIG. 4(a) can be viewed by a single audience member or multiple audience members (audience group). In the case of the head-mounted display 402 in FIG. 4(b), a head-mounted display 402 is provided for each audience member. When the performer's voice is collected by a microphone, the voice data is also sent to the device on the audience side along with the rendering data or motion data of the performer avatar 11, and the voice is output in synchronization with the movement of the performer avatar 11.

[0021] The information processing system 100 in FIG. 1 will be described in detail below. The audience-side device 400 may be a viewing device for each audience member, or a viewing device common to an audience group. The viewing device may be an HMD, monitor, TV, PC, or any other device. The following description assumes that the audience-side device 400 is a viewing device for each audience member. However, in a situation where audience groups simultaneously view the same display in each virtual space, one audience-side device 400 may be provided for each virtual space and common to the audience group. The spectator video generation unit 430 generates a video (image) representing an spectator avatar in the virtual space. For example, the spectator video generation unit 430 may use a video of a part of the body (e.g., the face) of the spectator watching the spectator-side device 400 as the video representing the spectator avatar. Alternatively, the spectator video generation unit 430 may generate a video representing the spectator avatar by rendering the spectator avatar in the virtual space. The video representing the spectator avatar is, for example, a video in which the spectator's viewpoint (the viewpoint of a virtual camera in the virtual space) is positioned toward the position of the performer avatar in the virtual space. In other words, a video of the spectator avatar as seen from the performer avatar is generated. In this way, the video (image) representing the spectator avatar may be an image of the spectator himself or an image of the spectator avatar (a character selected by the spectator).

[0022] The audience-side transmitter 440 transmits the video (image) representing the audience avatar generated by the audience video generator 430 to the performer-side device 300. The data receiver 380 of the performer-side device 300 receives the video representing the audience avatar from each audience-side device 400. The audience-side transmitter 440 is connected to the performer-side device 300 via a wired or wireless network. The wired or wireless network includes the Internet, a cellular communication network, a wireless LAN (Local Area Network), etc.

[0023] The audience image synthesizer 390 in the performer's device 300 synthesizes the video (image) representing the audience avatar received by the data receiver 380, and causes the presentation unit 200 to display the synthesized video (image).

[0024] The presentation unit 200 displays the images synthesized by the audience image synthesis unit 390 in a display area visible to the performer. As a result, images representing audience avatars in each virtual space (for example, images of audience members corresponding to the audience avatars or images of characters selected by the audience members) are displayed visible to the performer. The presentation unit 200 is disposed, for example, in front of the performer 10 in the studio. The presentation unit 200 is, for example, a display device such as a liquid crystal display device or an organic electroluminescence (EL) display device. In this case, the display area is the screen of the display device. The presentation unit 200 may include multiple displays corresponding to the multiple display areas, or the display area of ​​a single display may be divided into multiple areas. The presentation unit 200 may be a device that displays images in real space (a spatial image display device). In this case, the display area is a portion of the space where the image is displayed. The presentation unit 200 may also be a projector that projects images onto a screen. In this case, the display area is the projection area onto which the image from the projector is projected.

[0025] Fig. 5 shows an example of a group of audience avatars presented (displayed) by the presentation unit 200. The example in Fig. 5 assumes a situation in which a live performance is being held simultaneously in four virtual spaces (represented as virtual spaces 500_1, 500_2, 500_3, and 500_4, respectively). A group of audience avatars in the venue in each virtual space is displayed in a different display area. More details are as follows.

[0026] The presentation unit 200 shown in Fig. 5 has four tile-shaped display areas, each corresponding to one of the four virtual spaces. In each display area, a rectangular image including spectator avatars in the virtual space is displayed. The upper right display area displays spectator avatars 510_1 corresponding to virtual space 500_1, the upper left display area displays spectator avatars 510_2 corresponding to virtual space 500_2, the lower right display area displays spectator avatars 510_3 corresponding to virtual space 500_3, and the lower left display area displays spectator avatars 510_4 corresponding to virtual space 500_4.

[0027] The performer 10, who exists in an actual physical space (for example, in a film studio), can recognize the audience groups corresponding to the audience avatar groups 510_1 to 510_4 through the images of the audience avatar groups 510_1 to 510_4 presented in each display area. In the following description, any audience avatar or audience avatar group may be referred to as an audience avatar 510 or an audience avatar group 510.

[0028] The spectator video synthesis unit 390 may use any method for synthesizing video (images) representing spectator avatars for each virtual space. For example, the video of the spectator avatar in the virtual space may be placed at a random position within the display area corresponding to the virtual space. The video of the spectator avatar may be compressed and displayed within a rectangular display area while maintaining the relative positional relationship of the spectator avatar in the virtual space. Alternatively, the video may be synthesized using the method shown in FIG. 6, which will be described below.

[0029] FIG. 6 is a diagram illustrating an example of a composition method. In four virtual spaces 500 (500_1, 500_2, 500_3, 500_4), stages 520 (520_1, 520_2, 520_3, 520_4) and audience seats 530 (530_1, 530_2, 530_3, 530_4) are provided. The performer avatars on the stage 520 and the audience avatars in the audience seats are not shown. In this case, the audience seats 530 (530_1, 530_2, 530_3, 530_4) can be cut at appropriate positions as shown in FIG. 6 and expanded into rectangles 531_1, 531_2, 531_3, 531_4. The images representing the audience avatars are composed by placing the images representing the audience avatars in the display areas corresponding to the virtual spaces according to the positions of the audience avatars included in the rectangles.

[0030] The gaze acquisition unit 320 acquires gaze information of the performer 10 viewing the display area of ​​the presentation unit 200. One method of acquiring gaze information is to identify the gaze direction based on matching between a camera capturing an image of the performer's eyes and multiple images of the eyes prepared in advance. Alternatively, if the presentation unit 210 is a display unit of an HMD, the center of the image presented on the display unit of the HMD may be considered to be the gaze direction.

[0031] Based on the acquired gaze information, the gaze acquisition unit 320 detects which audience avatar or which audience avatar group the performer 10 is gazing at. For example, if four audience avatar groups contained in four virtual spaces are presented in the respective display areas, the gaze acquisition unit 320 detects which audience avatar group, or which audience avatar among the audience avatar groups, the performer 10 is gazing at. There may be one or more audience avatars to gaze at. There may be one or more audience avatar groups to gaze at. Alternatively, the gaze acquisition unit 320 may detect which of the four virtual spaces the performer is gazing at. The audience avatar group the performer is gazing at may span multiple virtual spaces. At least one audience avatar the performer is gazing at corresponds to the first audience avatar according to the present disclosure.

[0032] The gaze acquiring unit 320 provides information (gazing audience information) that identifies the audience avatar or audience avatars that the performer is gazing at to the motion editing unit 350 and the presentation unit 200. The gaze acquiring unit 320 may provide the gaze information of the performer 10 to the motion editing unit 350.

[0033] The presentation unit 200 may acquire information indicating the spectator avatar or group of spectator avatars that the performer 10 is gazing at (gazing spectator information) from the gaze acquisition unit 320, and may highlight and display the spectator or group of spectators that are being gazed at in the display area. For example, the color of the spectator or group of spectators that are being gazed at may be changed, and the spectator or group of spectators may be surrounded by a thick line (see FIG. 8, which will be described later). Alternatively, spectators who are not being gazed at (non-gazing spectator group) may be displayed in a blurred manner.

[0034] The motion acquisition unit 310 in the performer's device 300 acquires information about the movements of the performer (motion information). The motion information also includes information about the performer's position. For example, the motion acquisition unit 310 acquires motion information about the performer 10 based on image data of the performer 10 captured by a camera 321. Alternatively, the motion acquisition unit 310 may acquire position information from one or more sensors attached to the performer's body, and acquire the performer's motion information based on the acquired position information. The motion acquisition unit 310 provides the acquired motion information to the first positional relationship calculation unit 330 and the motion editing unit 350.

[0035] The first positional relationship calculation unit 330 calculates the positional relationship between the performer 10 and the audience avatar or audience avatars shown in the image (video) presented by the presentation unit 200, i.e., the positional relationship (first positional relationship) of the performer's viewpoint. The calculation of the positional relationship is performed, for example, based on motion information (including position information) of the performer 10, position information of the presentation unit 200 (e.g., the placement position and height of the display), and position information of the audience avatar or audience avatars within the display area of ​​the presentation unit 200. The first positional relationship includes, for example, information on the vector direction from the position of the performer 10 to the audience avatar or audience avatars that the performer 10 is gazing at in the image presented in the display area. The reference position of the performer 10 may be a predetermined position on the performer's body (e.g., the eyes or center of gravity), or a position determined by any other method.

[0036] Here, the positional relationship between the performer 10 and the audience avatars presented in the display area does not necessarily match the positional relationship between the performer avatar 11 and the audience avatars. For example, an audience avatar in the upper left display area of ​​FIG. 5 is located to the left and in front of the performer 10, but in the virtual space, the audience avatar may be located to the right of the performer avatar 11. For this reason, when reflecting the movement of the performer 10 in the performer avatar 11, it is necessary to take into account the positional relationship between the performer 10 and the audience avatars presented in the display area (first positional relationship) in order to achieve natural interaction with the audience. The first positional relationship calculation unit 330 transmits information indicating the calculated positional relationship between the performer 10 and the audience avatars (performer viewpoint positional relationship information) to the motion editing unit 350.

[0037] The second positional relationship calculation unit 340 calculates the positional relationship between the performer avatar 11 and the spectator avatar 510 as seen from the spectator avatar 510 (or the spectators), i.e., the positional relationship from the spectator's viewpoint (second positional relationship). To calculate the positional relationship, the spectator-side transmitter 440 in each spectator-side device transmits positional information of the spectator avatar in the virtual space, and the data receiver 380 in the performer-side device 300 receives the positional information. Based on this positional information, the second positional relationship calculation unit 340 calculates the positional relationship between the performer avatar and the spectator avatar from each spectator's viewpoint (second positional relationship). The position of the spectator avatar 510 may be any position as long as it allows the position of the spectator avatar 510 to be identified. For example, the position of the spectator avatar 510 may be the position of a virtual camera in the virtual space.

[0038] As described above, the positional relationship between the performer 10 and the audience avatar presented in the display area is not necessarily the same as the positional relationship between the performer avatar 11 and the audience avatar in the virtual space. Therefore, when reflecting the movement of the performer 10 in the performer avatar 11, it is necessary to take into consideration the positional relationship (second positional relationship) between the performer 10 and the audience avatar 510 in the virtual space in order to realize natural interaction with the audience.

[0039] The second positional relationship calculation unit 340 provides the motion editing unit 350 with information indicating the calculated positional relationship between the performer avatar 11 and the audience avatars in the virtual space (audience viewpoint positional relationship information).

[0040] The motion editing unit 350 edits the movement information (motion information) of the actual performer avatar 11 based on the movement of the performer 10 and outputs the edited information as the content 12.

[0041] The motion editing unit 350 receives positional relationship information of the performer's viewpoint, positional relationship information of the audience's viewpoint, performer motion information, and information on audience members watching the performer, and edits the performer's motion information. Editing of the motion information is performed, for example, for each audience member or for each audience group (for each virtual section).

[0042] For example, the real space in which the performer exists and the virtual space in which the performer avatar exists are associated with each other in coordinate systems, and a vector corresponding to the vector to the audience avatar (the audience avatar displayed in the display area) that the performer is gazing at is identified in the virtual space. The performer avatar in the virtual space faces the direction of the identified vector. The performer's movements in the motion information are edited by the difference between the direction (angle) in which the performer avatar is facing and the direction (angle) in which the audience avatar exists as seen from the performer avatar in the virtual space. For example, the angle of the performer's face, arms, or legs is rotated according to this difference. This makes it possible to realize the movement of the performer avatar in the virtual space gazing at the audience avatar (referred to as audience avatar A) that the performer in the real space is gazing at in the display area.

[0043] For other audience avatars (audience avatars not being gazed upon) existing in the same virtual space as audience avatar A, the motion information of the performer avatar is edited so that the performer avatar performs an action as if it is gazing at audience avatar A.

[0044] It is not necessary to edit the motion information for an audience avatar that exists in a virtual space other than audience avatar A. Alternatively, a screen object (such as a back screen) that shows the state of the virtual space in which audience avatar A exists may be placed in the virtual space, and the performer's motion information for the virtual space may be edited so that the performer avatar in the virtual space performs an action of gazing at the screen object or audience avatar A displayed on the screen object.

[0045] A specific example of an operation in which the movement of the performer avatar is changed for each audience member, audience group, or virtual space by editing the motion information will be described later.

[0046] Furthermore, the motion editing unit 350 may edit the motion information according to the attribute information of the virtual space, for example, the culture of the language area. For example, if an actor beckons by bending his wrist with his palm facing down, the motion information may be changed to bending his wrist with his palm facing up for a virtual space with certain attribute information.

[0047] The video data generator 360 generates data of a performer avatar (performer object) to be provided to the audience-side device based on the motion information edited by the motion editor 350. As an example, the performer avatar data is generated by generating the three-dimensional object (or two-dimensional object) described in FIGS. 2 to 4 and rendering the generated three-dimensional object (or two-dimensional object) as image data. In this case, the audience-side device 400 plays back the performer avatar as video based on the rendering data.

[0048] As another example, the data of the performer avatar may be edited motion information. In this case, the edited motion information is applied to the performer avatar in the audience device, and the performer avatar moves according to the edited motion information.

[0049] In this example, it is assumed that the rendering data of a three-dimensional object is data of a performer avatar.

[0050] The data distribution unit 370 transmits the data of the performer object generated by the video data generation unit 360 to the audience side device 400. The data distribution unit 370 is connected to the audience side device 400 via a wired or wireless network. The wired or wireless network includes the Internet, a cellular communication network, a wireless LAN (Local Area Network), or the like.

[0051] The data receiving unit 410 of the audience-side device 400 receives the performer object data distributed from the performer-side device 300. The data receiving unit 410 provides the received data to the audience presentation unit 420.

[0052] The audience presentation unit 420 displays video showing the virtual space and objects in the virtual space (such as performer avatars, audience objects, and various objects in the live venue). The audience wearing or viewing the audience-side device 400 may have their own objects (audience avatars) displayed or hidden. The audience presentation unit 420 generates video showing the state of the virtual space 500 from the viewpoint of a virtual camera set for the audience in the virtual space. The virtual camera is set, for example, at the head (e.g., at the eye position) of the audience avatar. The generated video (including performer avatars, other audience avatars, the stage, and audience seats) is viewed by the audience.

[0053] Hereinafter, the embodiments of the present disclosure will be described in detail using specific examples.

[0054] FIG. 7 is a diagram showing how the performer avatars 11 (11_1 to 11_4) in the virtual space 500 (500_1 to 500_4) move in response to the movement of the performer 10 in the real space. The virtual spaces 500_1 to 500_4 include the performer avatars 11_1 to 11_4, stages 520_1 to 520_4 on which the performer avatars 11_1 to 11_4 perform, and audience seats 530 (530_1 to 530_4) where the audience avatars watch the performer avatars. The performer avatars 11_1 to 11_4 move in the virtual space 500 in response to the movement of the performer 10. When the performer 10 moves his left arm up as shown in FIG. 7, the performer avatar 11 also moves his left arm up in a similar manner so as to synchronize with this movement. Note that in this example, the motion information is not edited.

[0055] 8(a) shows an example in which the performer 10 gazes at an audience avatar 511_1 among a group of audience avatars in a virtual space and performs a hand waving action. The audience avatar 511_1 is surrounded by a thick line or painted in a specific color, making it easy for the performer 10 to recognize that it is the audience avatar being gazed at. The direction of the performer 10's gaze is direction D1.

[0056] FIG. 8(b) shows the performer avatar 11_1 and a group of audience avatars in a virtual space in which an audience avatar 511_1 exists. The group of audience avatars includes the audience avatar 511_1 and another audience avatar 511_2. When the motion information of the performer 10 is not edited, that is, when the movement of the performer avatar 11_1 is not corrected, the performer avatar 11_1 looks in direction D1 and performs a waving motion. When viewed from the audience avatars 511_1 and 511_2, the performer avatar 11_1 appears to be waving in a different direction from the audience avatar 511_1. In contrast, in this embodiment, the motion information is edited according to the difference (angle difference) θ between the direction in which the performer avatar 11_1 looks and the direction in which the audience avatar 511_1 exists. As a result, the performer avatar 11_1 looks at the audience avatar 511_1 and performs a waving motion. To the audience member corresponding to the audience avatar 511_1, it appears as if the performer avatar 11_1 is waving to them. To the audience member corresponding to the audience avatar 511_2, it appears as if the performer avatar 11_1 is waving to the audience avatar 511_1. Note that it is also possible to configure the audience avatar 512_1 not to be corrected for motion information.

[0057] FIG. 9(a) shows an example of the view of the performer avatar 11_1 from the audience avatar 511_1 when the motion information is not edited. The performer avatar 11_1 is waving while looking in a different direction from the audience avatar 511_1. FIG. 9(b) shows an example of the view of the performer avatar 11_1 from the audience avatar 511_1 after the motion information has been edited. The performer avatar 11_1 is looking at the audience avatar 511_1 and waving. The audience of the other audience avatar 511_2 (see FIG. 8(b)) also sees that the performer avatar 11_1 is waving to the audience avatar 511_1.

[0058] FIG. 10 is a diagram illustrating the fields of view of an audience avatar 511_1 and an audience avatar 511_2. FIG. 10(a) shows the state inside the virtual space 500_1. FIG. 10(b) shows the field of view of the audience avatar 511_1, and FIG. 10(c) shows the field of view of the audience avatar 511_2. The audience avatar 511_1 can see the audience avatar 512_1 to the right of the direction facing the performer avatar 11_1. On the other hand, the audience avatar 512_1 can see the audience avatar 511_1 to the left of the direction facing the performer avatar 11_1. This allows the audience avatar to see the performer avatar 11_1 from the front no matter where they are in the audience seats 530_1, and maintains their positional relationship with other audience members.

[0059] The motion editing unit 350 may switch whether to edit the motion information (correct the movement of the performer avatar) depending on the angle difference θ (see FIG. 8 or FIG. 9). Specifically, when the angle difference θ is equal to or smaller than a threshold, the motion information is edited (the movement of the performer avatar is corrected) as shown in FIG. 8(b) or FIG. 9(b). When the angle difference θ is greater than the threshold, the motion information is not edited (the movement of the performer avatar is not corrected) as it would result in unnatural movement.

[0060] The above explanation has shown an example of correcting the movement of a performer avatar in a virtual space where an audience object that the performer is gazing at exists, but an example of correcting the movement of a performer avatar in another virtual space will now be described.

[0061] FIG. 11 shows the action of the performer 10 gazing at and waving his hand at an audience avatar 511_1 among an audience avatar group 510_1 in a virtual space 500_1 corresponding to the upper right display area among the plurality of display areas presented on the presentation unit 200.

[0062] 12(a) shows an example of correcting the movement of the performer avatar 11_1 in the virtual space 500_1 in which the audience avatar 511_1 exists. This example is similar to the above-mentioned FIG. 9(b), and therefore a description thereof will be omitted. FIG. 12(b) shows an example of correcting the movement of the performer avatar 11_2 in a virtual space (assumed to be virtual space 500_2) other than the virtual space 500_1. An audience member avatar 511_2 is an arbitrary audience member avatar in the virtual space 500_2. In the virtual space 500_2, screen objects (back screens, etc.) 551, 553, and 554 showing the state of the other virtual spaces 500_1, 500_3, and 500_4 are arranged. The screen objects 551, 553, and 554 display audience members 510_1, 510_3, and 510_4 in 500_1, 500_3, and 500_4. The performer's motion information is edited (the movement of the performer avatar 11_2 is corrected) so that the performer avatar 11_2 looks at the screen object 551 including the audience member avatar 511_1 that the performer is gazing at and performs a hand waving movement. That is, the motion information of the performer is edited (the movement of the performer avatar 11_2 is corrected) based on the positional relationship (third positional relationship) between the performer avatar 11_2 and the screen object 551. For example, the motion information of the performer is edited (the movement of the performer avatar 11_2 is corrected) so that the performer looks at the audience avatar 511_1 in the screen object 551 and waves his / her hand. In the other virtual spaces 500_3 and 500_4, the movements of the performer avatars may be corrected in a similar manner.

[0063] The motion editing unit 350 may switch whether to edit the motion information depending on the magnitude of the difference (angle difference) θ2 (see FIG. 12(b)) between the direction of the spectator avatar 511_2 and the direction of the screen object 551 or the direction of the spectator avatar 511_1 included in the screen object 551. For example, if the angle difference θ2 is equal to or smaller than a threshold, the motion information may be edited, and if the angle difference θ2 is greater than the threshold, the motion information may not be edited.

[0064] In the example of FIG. 12(b), there are three screen objects, but there may be one, two, or three or more. For example, when there is one screen object, the virtual space displayed on the screen object may be switched according to time. In this case, the movement of the performer avatar 11_2 may be corrected only when the audience avatar 511_1 that the performer is gazing at is displayed, and in other cases, the movement of the performer avatar 11_2 may not be corrected. Alternatively, when the audience avatar 511_1 is not displayed on the screen object, the virtual space in which the audience avatar 511_1 exists may be forcibly displayed on the screen object, and the movement of the performer avatar 11_2 may be corrected.

[0065] FIG. 13 is a flowchart illustrating an example of the processing flow of the information processing system 100 according to an embodiment of the present disclosure. First, the presentation unit 200 presents images of a group of audience avatars in the virtual space 500 to the performer 10 (S1001). At this time, the presentation unit 200 may synthesize and present images representing audience avatars for each virtual space using data received by the data receiving unit 380. The images representing audience avatars may be images of audience members or images of characters selected by the audience members.

[0066] Next, the gaze acquisition unit 320 acquires gaze information of the performer 10 and detects the gaze destination (S1002). If the gaze destination can be detected, the process proceeds to step S1003, and if not, the process returns to step S1001.

[0067] Next, the presentation unit 200 highlights and displays the performer's gazed area (S1003). Next, the motion editing unit 350 determines whether the performer is making a specific motion toward the audience (herein referred to as an interaction motion) based on the motion (motion) information of the performer 10 acquired by the motion acquisition unit 310 (S1004). Examples of interaction motions can be arbitrarily defined, such as waving, beckoning, winking, tilting the head, dancing, or spinning. The target of an interaction motion may be a specific audience avatar or a specific group of audience avatars. A group of audience avatars may exist not only within one virtual space but also across multiple virtual spaces. For example, a performer may wave across audience seats on the same floor in multiple virtual spaces. If the motion is an interaction motion, the process proceeds to step S1005; if it is not an interaction motion, the process returns to step S1001. Whether or not an interaction action has been performed may be determined, for example, by using predictions based on machine learning techniques based on the actor's movements from the past to the present in the vicinity.

[0068] Next, the motion editing unit 350 edits the performer's motion information for each spectator, each spectator group, or each virtual space based on the performer's gaze information, first positional relationship, second positional relationship, etc., and the video data generation unit 360 generates performer avatar data based on the corrected motion information. The data distribution unit 370 transmits the performer avatar data to each spectator side device 400 (S1005). If the spectator avatar groups that the performer is gazing at span multiple virtual spaces, the movements of the performer avatars described above can be corrected (motion information edited) for the spectator avatars or spectator avatar groups that the performer is gazing at in each of the multiple virtual spaces.

[0069] Next, the data receiving unit 380 receives the video images representing the audience avatars transmitted from the audience-side device 400, and presents the audience avatar group 510 to the performer 10 via the presentation unit 200 (S1006). This allows the performer 10 to see the audience's reactions to the interaction actions taken with the audience avatars. The audience-side device 400 reflects the audience's actions in the audience avatars. For example, if an audience member turns to the right, the audience avatar also turns to the right, and if an audience member waves, the audience avatar also waves.

[0070] Next, the motion editing unit 350 determines whether the interaction action has ended (S1007). If the interaction action is continuing, the process returns to step S1005. If the interaction action has ended, the process determines whether the performer's live performance has ended (S1008). If not, the process returns to step S1001. If the live performance has ended, the process of this flowchart ends.

[0071] 14 is a flowchart illustrating an example of the flow of a motion editing process of the information processing system 100 according to an embodiment of the present disclosure. In the description of the flowchart, a situation is assumed in which the performer 10 makes an interaction motion of pointing his / her hand toward an audience avatar 511_1 in a virtual space 500_1 out of four virtual spaces (see FIG. 11).

[0072] First, the motion editing unit 350 identifies in the virtual space 500_1 a direction corresponding to the direction in which the performer 10 points his / her hand toward the audience avatar 511_1 presented by the presentation unit 200 (the performer 10 is also gazing toward this direction). The motion editing unit 350 calculates the difference (angle difference) θ between the identified direction and the direction of the performer avatar 11_1 relative to the audience avatar 511_1 in the virtual space 500_1 (S1101).

[0073] Next, the motion editing unit 350 determines whether the difference θ is equal to or smaller than a threshold value (S1102). Here, the threshold value is a value for determining whether the motion of the performer avatar 11_1 will appear unnatural when viewed from the audience of the audience avatar 511_1 if the movement of the performer avatar 11_1 is corrected. If the difference θ is equal to or smaller than the threshold value, the process proceeds to step S1104; if it is greater than the threshold value, the process proceeds to step S1103.

[0074] Next, the motion editing unit 350 does not edit the motion information (correct the motion) (or cancels the editing of the motion information) (S1103).

[0075] Next, the motion editing unit 350 edits the motion information of the performer (corrects the movement of the performer avatar 11_1) in accordance with the magnitude of the difference θ (S1104).

[0076] Next, the video data generation unit 360 generates data of the performer avatar based on the corrected motion information (or uncorrected motion information), and the data distribution unit 370 transmits the data of the performer avatar to the audience device 400 (S1105). Here, the performer avatar data is assumed to be data indicating the motion of the performer avatar. The audience device 400 receives the data (motion data) of the performer avatar and causes the performer avatar 11_1 to move based on the motion data (S1106). The performer avatar 11_1 performing the movement is presented to the audience (S1106).

[0077] 15 is a flowchart illustrating another example of the flow of the motion editing process of the information processing system 100 according to an embodiment of the present disclosure. Steps S1201 to S1204 are the same as steps S1101 to S1104 in FIG.

[0078] The video data generation unit 360 corrects the 3D model (or 2D model) of the performer avatar based on the performer's motion information, and transmits video data obtained by rendering the corrected 3D model to the audience-side device 400 via the data distribution unit 370 (S1205). The audience-side device 400 receives the video data and plays it back to present the performer avatar to the audience (S1206).

[0079] 16 is a flowchart illustrating yet another example of the flow of the motion editing process of the information processing device according to an embodiment of the present disclosure. In this example, the motion of the performer avatar 11 is additionally edited according to the attribute information of the virtual space 500. Steps S1301 to S1304 are the same as S1101 to S1104 in FIG.

[0080] Next, the motion editing unit 350 determines whether the motion of the actor avatar 11 is to be edited based on the attribute information of the virtual space (S1305). If the motion is to be edited, the process proceeds to step S1306, and if the motion is not to be edited, the process proceeds to step S1307.

[0081] Next, the motion editing unit 350 edits the motion of the performer avatar 11 in accordance with the attribute information of the virtual space (S1306). For example, if the motion made by the performer 10 is a beckoning motion with the palm facing down, the motion of the performer avatar 11 is edited to a motion with the palm facing up and the wrist bent in accordance with the language culture of the virtual space 500.

[0082] Next, the motion editing unit 350 further edits the motion of the performer avatar 11 that is the editing target (S1306).

[0083] Steps S1307 and S1308 are the same as steps S1105 and S1106 in FIG.

[0084] As described above, according to the information processing device of the present disclosure, the intention of the performer's movements can be correctly reflected in the performer avatar 11 by controlling the performer avatar according to the positional relationship between the performer 10 and the audience avatar presented on the presentation unit 200.

[0085] (Variation) Next, a modified example of the presentation unit 200 will be described. FIG. 17 is a diagram illustrating a presentation unit 202 as a modified example of the presentation unit 200 according to an embodiment of the present disclosure. As shown in Fig. 17(a), the presentation unit 202 is configured by stacking multiple circular display panels 202_1, 202_2, 202_3, and 202_4. The display panels 202_1, 202_2, 202_3, and 202_4 each display the scene in the corresponding virtual space (such as an audience or audience seats). As shown in Fig. 17(b), when viewed from above, the presentation unit 202 has a roughly elliptical or circular shape surrounding the performer 10.

[0086] When the audience seats in each virtual space surround the stage, the performer 10 can view the audience avatars in the same positional relationship as in the virtual space, allowing the performer 10 to interact with the audience more intuitively.

[0087] Fig. 18 illustrates a presentation unit 203 as another modified example of the presentation unit 200 according to the embodiment of the present disclosure. The presentation unit 203 may be a head-mounted display (HMD) as shown in Fig. 18. The presentation unit 203 can freely arrange four virtual spaces within its field of view, allowing the performer 10 to visually recognize audience avatar groups 510_1 to 510_4 in the four virtual spaces.

[0088] 19 is a diagram showing another example of display of four virtual spaces presented by the presentation unit 203. Alternatively, the state of virtual space 500_1 including the audience avatar that the performer is gazing at may be displayed on a full screen, and the state of other virtual spaces may be displayed on a small screen, with the state of virtual space 500_1 being displayed overlapping the edge of the full screen.

[0089] [Application example] The following describes application examples of the information processing system 100. Note that the above-described information processing system 100 can also be applied to any of the following systems, devices, methods, etc. of the information processing system 100.

[0090] 20 shows an example of the hardware configuration of a computer that executes a series of processes of the information processing system 100 according to the present disclosure using a program. In the computer, a CPU 1001, a ROM 1002, and a RAM 1003 are connected to each other via a bus 1004.

[0091] An input / output interface 1005 is also connected to the bus 1004. To the input / output interface 1005, an input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected.

[0092] The input unit 1006 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 1007 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 1008 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 1009 includes, for example, a network interface. The drive drives removable media such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0093] In a computer configured as above, the CPU 1001 performs the above-described series of processes by, for example, loading a program stored in the storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004 and executing the program. The RAM 1003 also stores data necessary for the CPU 1001 to execute various processes as needed.

[0094] The program executed by the computer can be applied by recording it on removable media such as package media, for example. In this case, the program can be installed in the storage unit 1008 via the input / output interface 1005 by inserting the removable media into the drive 1010.

[0095] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 1009 and installed in the storage unit 1008.

[0096] The steps of the processes disclosed herein do not necessarily have to be performed in the order depicted in the flowcharts. For example, the steps may be performed in a different order than depicted in the flowcharts, or some of the steps depicted in the flowcharts may be performed in parallel.

[0097] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0098] Furthermore, the effects of the present disclosure described in this specification are merely examples, and other effects may also be present.

[0099] The present disclosure can also be configured as follows. [Item 1] The system comprises a presentation unit that presents a group of audience avatars arranged in at least one virtual space in a display area visible to an actor corresponding to an actor avatar arranged in the virtual space, and a control unit that controls the actor avatar in the at least one virtual space based on the actor's actions relative to the group of audience avatars represented in the image presented in the display area. [Item 2] The control unit controls the performer avatar in the at least one virtual space based on a first positional relationship between the performer and the group of audience avatars represented by the image presented in the display area and a second positional relationship between the performer avatar and the group of audience avatars in the at least one virtual space. Item 1. An information processing device according to item 1. [Item 3] the at least one virtual space includes a plurality of the virtual spaces; the presentation unit presents an image representing the audience avatar group for each of the virtual spaces in the display area; The control unit controls the performer avatar for each of the virtual spaces. Item 3. The information processing device according to item 1 or 2. [Item 4] an identification unit that identifies at least one first audience avatar that the performer gazes at among the audience avatars represented by the image presented in the display area; the first positional relationship is a positional relationship between the performer and the first audience avatar represented by the image, and the control unit controls the performer avatar in the first virtual space based on the first positional relationship and a second positional relationship between the first audience avatar and the performer avatar in a first virtual space that is the virtual space including the first audience avatar. Item 2. An information processing device according to item 2. [Item 5] The identification unit identifies the first audience avatar based on gaze information of the performer. Item 5. An information processing device according to item 4. [Item 6] The first spectator avatar includes at least one spectator avatar of the spectator avatar group in any one of the virtual spaces. Item 6. The information processing device according to item 4 or 5. [Item 7] The first spectator avatar includes at least one spectator avatar in each of the spectator avatar groups in the two or more virtual spaces. Item 7. The information processing device according to any one of items 4 to 6. [Item 8] A screen object including at least a part of the audience avatars in the first virtual space is displayed in a second virtual space that does not include the first audience avatar, and the control unit controls the performer avatar in the second virtual space based on a third positional relationship between the screen object and the performer avatar in the second virtual space and the first positional relationship. The information processing device according to any one of items 4 to 7. [Item 9] an acquisition unit for acquiring the movement of the performer; the control unit controls the performer avatar in the second virtual space based on the movement of the performer; The control unit corrects the movement of the performer avatar in the second virtual space in the direction of the screen object. Item 9. An information processing device according to item 8. [Item 10] The control unit determines whether the movement of the performer is an interaction movement, and corrects the movement of the performer avatar only if the movement is an interaction movement. Item 10. The information processing device according to item 8 or 9. [Item 11] The control unit corrects the movement of the performer avatar according to a difference between a direction in which the performer avatar faces and a direction from the performer avatar to the screen object. 12. The information processing device according to any one of items 8 to 11. [Item 12] The control unit corrects the movement of the performer avatar when the difference is equal to or smaller than a threshold, and does not correct the movement of the performer avatar when the difference is greater than the threshold. Item 12. The information processing device according to item 11. [Item 13] Attribute information is set for each of the virtual spaces, The control unit changes the behavior of the performer avatar according to attribute information of the virtual space. 13. The information processing device according to any one of items 3 to 12. [Item 14] The attribute information is based on the language used or place of residence of an audience group corresponding to an audience avatar group included in the virtual space. Item 14. The information processing device according to item 13. [Item 15] The control unit transmits motion data indicating the motion of the performer avatar to at least one terminal device that causes the performer avatar in the virtual space to move based on the motion data and provides a field of view image to an audience group corresponding to the audience avatar group based on a virtual viewpoint set in the virtual space. 15. The information processing device according to any one of items 1 to 14. [Item 16] The control unit transmits an image representing the performer avatar to at least one terminal device that places the performer avatar represented by the image in the virtual space and provides a view image to an audience group corresponding to the audience avatar group based on a virtual viewpoint set in the virtual space. 16. The information processing device according to any one of items 1 to 15. [Item 17] The presentation unit includes a display unit disposed in front of the performer, a display unit having a shape surrounding the performer, or a display unit of a head-mounted device worn by the performer. 17. The information processing device according to any one of items 1 to 16. [Item 18] the control unit receives image data of an audience group from at least one terminal device of the audience group corresponding to the audience avatar group; Item 18. The information processing device according to any one of items 1 to 17, wherein the presentation unit presents an image of the audience group indicated by the image data as the image representing the audience avatar group. [Item 19] An image representing a group of audience avatars arranged in at least one virtual space is displayed in a display area visible to an actor corresponding to the actor avatar arranged in the virtual space, and the actor avatar in the at least one virtual space is controlled based on the action of the actor relative to the group of audience avatars represented in the image displayed in the display area. Information processing methods. [Item 20] A computer program for causing a computer to execute the steps of: presenting images representing a group of audience avatars arranged in at least one virtual space in a display area visible to a performer corresponding to a performer avatar arranged in the virtual space; and controlling the performer avatar in the at least one virtual space based on the actions of the performer relative to the group of audience avatars represented in the images presented in the display area. [Explanation of symbols]

[0100] 10 Performers 11 Performer avatar 12 Contents 100 Information Processing Systems 200, 203 Presentation section 300 Performer's equipment 310 Motion Acquisition Unit 320 Gaze acquisition unit 321 Camera 330 First Positional Relationship Calculation Unit 340 Second positional relationship calculation unit 350 Motion Editorial Department 360 Video Data Generation Unit 370 Data Distribution Department 380, 410 Data receiving unit 390 Audience Image Synthesis Unit 400 Spectator equipment 401 Display 402 Head-mounted display 420 Audience Presentation Department 430 Audience Image Generation Unit 440 Spectator side transmitter 500 Virtual Space 510 Audience avatars 511, 512 Audience avatars 520 Stage 530, 531 audience seats 1001 CPU 1002 ROM 1003 RAM 1004 Bus 1005 Input / Output Interface 1006 Input section 1007 Output section 1008 Storage section 1009 Communications Department 1010 Drive

Claims

1. a presentation unit that presents an image representing a group of audience avatars arranged in at least one virtual space in a display area that is visible to a performer corresponding to a performer avatar arranged in the virtual space; a control unit that controls the performer avatar in the at least one virtual space based on the performer's actions relative to the audience avatars represented by the images presented in the display area; Equipped with the control unit controls the performer avatar in the at least one virtual space based on a first positional relationship between the performer and the group of audience avatars represented by the image presented in the display area and a second positional relationship between the performer avatar and the group of audience avatars in the at least one virtual space; an identification unit that identifies at least one first audience avatar that the performer gazes at among the audience avatars represented by the image presented in the display area; the first positional relationship is a positional relationship between the performer and the first audience avatar represented by the image, and the control unit controls the performer avatar in the first virtual space based on the first positional relationship and a second positional relationship between the first audience avatar and the performer avatar in a first virtual space that is the virtual space including the first audience avatar; a screen object including at least a part of the audience avatars in the first virtual space is displayed in a second virtual space that does not include the first audience avatar, and the control unit controls the performer avatar in the second virtual space based on a third positional relationship between the screen object and the performer avatar in the second virtual space and the first positional relationship; an acquisition unit for acquiring the movement of the performer; the control unit controls the performer avatar in the second virtual space based on a movement of the performer; the control unit corrects the movement of the actor avatar in the second virtual space in the direction of the screen object. Information processing device.

2. the at least one virtual space includes a plurality of the virtual spaces, the presentation unit presents an image representing the audience avatar group for each of the virtual spaces in the display area; The control unit controls the performer avatar for each of the virtual spaces. The information processing device according to claim 1 .

3. The identification unit identifies the first audience avatar based on gaze information of the performer. The information processing device according to claim 1 .

4. The first spectator avatar includes at least one spectator avatar of the spectator avatar group in any one of the virtual spaces. The information processing device according to claim 1 .

5. The first spectator avatar includes at least one spectator avatar in each of the spectator avatar groups in the two or more virtual spaces. The information processing device according to claim 1 .

6. The control unit determines whether the movement of the performer is an interaction movement, and corrects the movement of the performer avatar only if the movement is an interaction movement. The information processing device according to claim 1 .

7. The control unit corrects the movement of the performer avatar according to a difference between a direction in which the performer avatar faces and a direction from the performer avatar to the screen object. The information processing device according to claim 1 .

8. The control unit corrects the movement of the performer avatar when the difference is equal to or smaller than a threshold, and does not correct the movement of the performer avatar when the difference is greater than the threshold. The information processing device according to claim 7 .

9. Attribute information is set for each of the virtual spaces, The control unit changes the behavior of the performer avatar according to attribute information of the virtual space. The information processing device according to claim 2 .

10. The attribute information is based on the language used or place of residence of an audience group corresponding to an audience avatar group included in the virtual space. The information processing device according to claim 9 .

11. The control unit transmits movement data indicating the movement of the performer avatar to at least one terminal device that causes the performer avatar in the virtual space to move based on the movement data and provides a field of view image to an audience group corresponding to the audience avatar group based on a virtual viewpoint set in the virtual space. The information processing device according to claim 1 .

12. The control unit transmits an image representing the performer avatar to at least one terminal device that places the performer avatar represented by the image in the virtual space and provides a field of view image to an audience group corresponding to the audience avatar group based on a virtual viewpoint set in the virtual space. The information processing device according to claim 1 .

13. The presentation unit includes a display unit disposed in front of the performer, a display unit having a shape surrounding the performer, or a display unit of a head-mounted device worn by the performer. The information processing device according to claim 1 .

14. the control unit receives image data of the audience group from at least one terminal device of the audience group corresponding to the audience avatar group; The presentation unit presents an image of the audience group indicated by the image data as the image representing the audience avatar group. The information processing device according to claim 1 .

15. presenting an image representing a group of audience avatars arranged in at least one virtual space in a display area visible to a performer corresponding to a performer avatar arranged in the virtual space; controlling the performer avatar in the at least one virtual space based on the performer's actions relative to a group of audience avatars represented in the image presented in the display area; controlling the performer avatar in the at least one virtual space based on a first positional relationship between the performer and the group of audience avatars represented by the image presented in the display area and a second positional relationship between the performer avatar and the group of audience avatars in the at least one virtual space; identifying at least one first audience avatar that the performer gazes at among the audience avatars represented by the image presented in the display area; the first positional relationship is a positional relationship between the performer and the first audience avatar represented by the image, and the performer avatar in the first virtual space is controlled based on the first positional relationship and a second positional relationship between the first audience avatar and the performer avatar in a first virtual space that is the virtual space including the first audience avatar; a screen object including at least a part of the audience avatars in the first virtual space is displayed in a second virtual space that does not include the first audience avatar, and the performer avatar in the second virtual space is controlled based on a third positional relationship between the screen object and the performer avatar in the second virtual space and the first positional relationship; Acquire the performer's movements; controlling the performer avatar in the second virtual space based on the movement of the performer; correcting the movement of the actor avatar in the second virtual space to the direction of the screen object; Information processing methods.

16. presenting an image representing a group of audience avatars arranged in at least one virtual space in a display area visible to a performer corresponding to a performer avatar arranged in the virtual space; controlling the performer avatar in the at least one virtual space based on the performer's actions relative to the audience avatars shown in the image presented in the display area; controlling the performer avatar in the at least one virtual space based on a first positional relationship between the performer and the group of audience avatars represented by the image presented in the display area and a second positional relationship between the performer avatar and the group of audience avatars in the at least one virtual space; identifying at least one first audience avatar that the performer gazes at among the audience avatars represented by the image presented in the display area; the first positional relationship is a positional relationship between the performer and the first audience avatar represented by the image, and controlling the performer avatar in the first virtual space based on the first positional relationship and a second positional relationship between the first audience avatar and the performer avatar in a first virtual space that is the virtual space including the first audience avatar; a step of displaying a screen object including at least a part of the audience avatars in the first virtual space in a second virtual space that does not include the first audience avatar, and controlling the performer avatar in the second virtual space based on a third positional relationship between the screen object and the performer avatar in the second virtual space and the first positional relationship; acquiring the actor's movements; controlling the actor avatar in the second virtual space based on the actor's movement; correcting a movement of the actor avatar in the second virtual space to a direction of the screen object; A computer program for causing a computer to execute the above.

Images