Information processing device

Abstract

To provide a technology that a performer can effectively feel the presence of audiences in a performance held in a virtual space.SOLUTION: An information processing device 100 has an arrangement determination part 102 to determine the arrangement of a performer avatar and of multiple audience avatars in a virtual space, and a display information generating part 103 to generate the display information representing the virtual space from the perspective of the performer avatar. At least one of the audience avatars is provided with a virtual item in a part of his / her body. The display information generating part 103 generates the display information in which the display position of the virtual item is adjusted so that the hidden display of the virtual item decreases from before the adjustment of the virtual item display position.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an information processing device that processes information about a virtual space. [Background technology]

[0002] In recent years, live concerts using virtual spaces are becoming more common. In a live concert held in a real space, the performers and the audience are in the same real space, so it is easy for the performers to sense the presence of the audience. However, in a live concert held in a virtual space, the performers and the audience are not in the same real space, so it is not easy for the performers to sense the presence of the audience. In this regard, Patent Document 1 discloses a device that estimates the situation of the audience and outputs the estimation result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-285275 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology disclosed in Patent Document 1 is not intended for performances held in virtual space. For this reason, there is a demand for technology that allows performers to effectively feel the presence of the audience in performances held in virtual space.

[0005] Therefore, an object of the present disclosure is to provide technology that allows performers to effectively feel the presence of an audience in a performance held in a virtual space. [Means for solving the problem]

[0006] The information processing device according to the first aspect includes: a placement determination unit that determines placement of a performer avatar and a plurality of audience avatars in a virtual space; a display information generating unit that generates display information representing the virtual space from the viewpoint of the performer avatar; having At least one of the spectator avatars has a virtual item on a part of its body; The display information generation unit generates the display information in which a display position of the virtual item is adjusted so that hiding of the display of the virtual item is reduced compared to before the adjustment of the display position of the virtual item. Effect of the Invention

[0007] According to the present disclosure, it is possible to provide technology that allows performers to effectively feel the presence of an audience in a performance held in a virtual space. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of a configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing an example of a functional configuration of an information processing device according to a first embodiment; [Diagram 3] FIG. 2 is a plan view showing an example of a virtual space in which a performance will be held. [Figure 4] 11 is a schematic diagram illustrating an example of display information in which adjustment of the position of a virtual item is required. FIG. [Diagram 5] FIG. 13 is a schematic diagram showing an example in which the display position of a virtual item is adjusted by adjusting which hand of a spectator avatar holds the virtual item. [Figure 6] 11 is a schematic diagram showing an example in which the display position of a virtual item is adjusted by adjusting the position of a hand holding the virtual item. FIG. [Figure 7] 10A and 10B are schematic diagrams showing the movement of a virtual item within a predetermined movable range. [Figure 8] 10 is a flowchart showing an example of the flow of a process for generating display information for a performer in the first embodiment. [Figure 9] FIG. 11 is a block diagram showing an example of a configuration of an information processing system according to a second embodiment. [Figure 10] FIG. 11 is a block diagram showing an example of a functional configuration of an information processing device according to a second embodiment. [Figure 11] 13 is a flowchart showing an example of the flow of a process for generating display information for a performer in the second embodiment. [Figure 12] FIG. 2 is a block diagram showing the configuration of a computer that realizes processing of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described with reference to the drawings. In order to clarify the description, the following description and drawings are omitted and simplified as appropriate. In each drawing, the same elements are given the same reference numerals, and duplicated descriptions are omitted as necessary. In addition, the execution order of the processes shown in the flowchart or sequence chart is merely an example, and the execution order can be appropriately changed as long as no technical contradiction occurs. In addition, the features of each embodiment can be combined as long as no technical contradiction occurs.

[0010] <Embodiment 1> 1 is a block diagram showing an example of the configuration of an information processing system 10 according to the first embodiment. The information processing system 10 includes an information processing device 100, a performer device 200, and a plurality of audience devices 300. These devices are connected to a network 400 such as the Internet. In particular, the performer device 200 and the audience device 300 can communicate with the information processing device 100 via the network 400.

[0011] The information processing device 100 is, for example, a device such as a server, and is a device that transmits information about a performance such as a live concert held in a virtual space to the performer device 200 and the audience device 300. Details of the information processing device 100 will be described later.

[0012] In a performance held in a virtual space, unlike a performance held in a real space, the performers and the audience exist in different real spaces. In a performance held in a virtual space, the performer's avatar and the audience's avatar exist in the same virtual space. Hereinafter, the performer's avatar will also be referred to as the performer avatar, and the audience's avatar will also be referred to as the audience avatar. The audience watches the performance held in the virtual space using the audience device 300. The performer performs in the real space while watching the audience avatars existing in the virtual space using the performer device 200. The performer's performance in the real space is input to the information processing device 100, for example, via the performer device 200, and video or sound corresponding to the performance is distributed from the information processing device 100 to the audience device 300.

[0013] The spectator device 300 is a device used by spectators at a performance. In this embodiment, the spectator device 300 is specifically a head-mounted display. Spectators wear the spectator device 300, which is a head-mounted display, on their heads and watch a performance such as a live concert held in a virtual space. Note that the spectator device 300 does not necessarily have to be a head-mounted display. For example, the spectator device 300 may be a device equipped with a display and a speaker, and may be a smartphone, a tablet terminal, a personal computer, or the like.

[0014] When the spectator device 300 is configured with a head mounted display, the spectator device 300 includes, for example, a display arranged in front of the user's eyes, a speaker, and a motion sensor that detects the movement of the head mounted display (movement of the user's head). The motion sensor is, for example, an inertial measurement unit (IMU) such as a speed sensor or a gyroscope. This detects the movement of the head mounted display, such as the direction and inclination. When the spectator device 300 receives distribution information from the information processing device 100, a video reproduced according to the display information included in the distribution information is displayed on the display of the head mounted display. The display information is information for displaying a virtual space, and includes, for example, data constituting an image to be displayed on the display. When the spectator device 300 receives distribution information from the information processing device 100, a sound generated according to the sound data included in the distribution information is output from the speaker of the head mounted display.

[0015] The performer device 200 is a device used by a performer in a performance. The performer device 200 is a device equipped with at least a display. In this embodiment, the performer device 200 further has a microphone that functions as an input device for inputting the singing or playing sounds of a performer performing a musical performance. The performer device 200 may include a motion sensor that functions as an input device for inputting the movements of the performer. The microphone and motion sensor of the performer device 200 may be referred to as a performance input device. It should be noted that the performer device 200 may be a head-mounted display, similar to the spectator device 300.

[0016] Next, the details of the information processing device 100 will be described. In particular, the configuration and processing related to the output of information to the performer device 200 and the audience device 300 will be described. FIG. 2 is a block diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. As shown in FIG. 2, the information processing device 100 has an information storage unit 101, an arrangement determination unit 102, a display information generation unit 103, a display information output unit 104, a performance information acquisition unit 105, a distribution information generation unit 106, and a distribution unit 107.

[0017] The information storage unit 101 stores in advance audience information, which is information about the audience. For example, the information storage unit 101 stores in advance avatar data of each audience member as the audience information. The avatar data may include data indicating the appearance of the avatar. The information storage unit 101 may also store in advance information on virtual items held by each audience member as the audience information. The virtual items are items that the avatars can hold in their hands or wear. The virtual items are, for example, items for appealing to performers, specifically, penlights, wristbands, and the like. If an audience member holds a virtual item, an audience avatar equipped with the virtual item is placed in the virtual space for that audience member. Note that an audience member does not necessarily need to hold a virtual item. In this case, an audience avatar not equipped with a virtual item is placed in the virtual space for that audience member. The audience information may also include viewing position information that specifies the viewing position of the audience member in the virtual space where the performance is performed (i.e., the viewing position of the audience avatar corresponding to that audience member). The viewing position specified by the viewing position information may be, for example, the position of a seat provided in a virtual space that is designated in advance by the spectator, or a position determined according to the type of ticket purchased by the spectator. The spectator information stored in the information storage unit 101 is used for processing by the arrangement determination unit 102 and the display information generation unit 103, which will be described later, as necessary. The information storage unit 101 may further store information about performers in advance. For example, the information storage unit 101 may store data indicating the appearance of a performer avatar in advance. The performer avatar may be a two-dimensional or three-dimensional image composed of a video of the performer.

[0018] The placement determination unit 102 determines the placement of the performer avatar and multiple audience avatars in the virtual space. The placement determination unit 102 determines the positions of the performer avatar and each audience avatar in the virtual space represented by a virtual space model generated in advance. FIG. 3 is a plan view showing an example of the virtual space 50 in which a performance is performed. That is, FIG. 3 is a top view of the virtual space 50 in which a performance is performed. As shown in FIG. 3, the virtual space 50 has a stage area 51 and an appreciation area 52. The stage area 51 is an area in which the performer avatar 60 is placed. The placement determination unit 102 places the performer avatar 60 in a predetermined position (for example, the center, etc.) in the stage area 51, for example. Note that, when it is possible to transmit position information of the performer in the real space from the performer device 200, the placement determination unit 102 may determine the placement of the performer avatar 60 based on the position information of the performer in the real space. In this case, the placement of the performer avatar 60 may be determined in real time according to the position information of the performer. The viewing area 52 is an area in which the spectator avatars 70 are placed. The placement determination unit 102 determines the placement of each spectator avatar 70, for example, according to the above-mentioned viewing position information. The placement determination unit 102 may place each spectator avatar 70 randomly in the viewing area 52, or may place them according to a predetermined placement rule. In this embodiment, each spectator avatar 70 is assigned to a seat provided in the viewing area 52 of the virtual space 50 according to the determination of the placement determination unit 102, but the virtual space 50 does not necessarily have to include seats. For a spectator who possesses a virtual item, a spectator avatar 70 having the virtual item on a part of the avatar's body is placed in the virtual space 50 according to the determination of the placement determination unit 102.

[0019] The display information generating unit 103 generates display information that represents the virtual space in which a performance is performed from the viewpoint of a performer avatar. The display information generating unit 103 generates display information, i.e., video data, that represents the view of the virtual space seen from the viewpoint of the performer avatar, based on, for example, the virtual space represented by a virtual space model generated in advance and the arrangement of each avatar determined by the arrangement determining unit 102. Here, in this embodiment, the display information generating unit 103 adjusts the display positions of the virtual items provided to the audience avatars to positions that are easy to view from the performer avatar. Details of the adjustment of the display position by the display information generating unit 103 will be described later.

[0020] The display information output unit 104 outputs the display information generated by the display information generation unit 103 to the performer device 200. As described below, the display information generation unit 103 generates display information for displaying an image in which the virtual item is adjusted to a position that is easy for the performer avatar to see. Therefore, the display information output unit 104 outputs such display information to the performer device 200. The performer device 200 receives the display information and displays an image according to the display information on the display. This allows the performer to view the virtual space from the viewpoint of the performer avatar.

[0021] The performance information acquisition unit 105 acquires performance information that represents the performance of the performer. For example, sound data of the performer's singing sound or musical performance sound is acquired from the performer device 200 as the performance information.

[0022] The distribution information generating unit 106 generates display information that expresses the virtual space in which the performance is performed from the viewpoint of an audience avatar. The distribution information generating unit 106 generates display information for each audience member, i.e., for each audience avatar, but may generate display information common to each audience member. The distribution information generating unit 106 generates display information, i.e., video data, that expresses the view of the virtual space seen from the viewpoint of an audience avatar, based on, for example, the virtual space represented by a virtual space model generated in advance and the arrangement of each avatar determined by the arrangement determining unit 102. The distribution information generating unit 106 then integrates the display information and the performance information acquired by the performance information acquiring unit 105 to generate distribution information that is information to be distributed to the audience device 300.

[0023] The distribution unit 107 outputs the distribution information generated by the distribution information generation unit 106 to the audience device 300. When the distribution unit 107 generates distribution information for each audience member, it outputs the distribution information to the audience device 300 of the audience member corresponding to the generated distribution information. The audience device 300 that receives the distribution information displays an image according to the distribution information on the display. Also, when the distribution information includes sound data, the audience device 300 outputs sound. This allows the audience to enjoy the performance taking place in the virtual space.

[0024] Next, the display information generating unit 103 will be described in detail. As described above, in this embodiment, the display information generating unit 103 adjusts the display position of the virtual item provided in the audience avatar to a position that is easily visible from the performer avatar. Specifically, the display information generating unit 103 adjusts the position of the virtual item when at least a part of the virtual item cannot be displayed as an image viewed by the performer avatar when the display information of the viewpoint of the performer avatar is generated after placing the virtual item at a predetermined position with respect to the audience avatar. Here, the predetermined position is a position that is preset as a default position for the virtual item. For example, when the virtual item is a penlight, the position of the right hand (or left hand) of the audience avatar that is raised to a predetermined height is preset as the predetermined position. In other words, the display information generating unit 103 changes the position of the virtual item from the predetermined position when a part or the whole of the virtual item cannot be seen from the viewpoint of the performer avatar because an object existing on the line of sight connecting the virtual item and the viewpoint of the performer avatar obstructs the virtual item. In addition, the object present in the line of sight may be, for example, a spectator avatar placed in front of a spectator avatar with a hidden virtual item, or a virtual item held by a spectator avatar placed in front of a spectator avatar with a hidden virtual item.

[0025] FIG. 4 is a schematic diagram illustrating an example of display information that requires adjustment of the position of a virtual item. Specifically, FIG. 4 shows an example of a view of a virtual space seen from the viewpoint of a performer avatar. As shown in FIG. 4, a plurality of audience avatars may exist in the virtual space, each holding a virtual item in its hand. In the example shown in FIG. 4, the virtual item is an item that is held in the hand, such as a penlight. Here, in the example shown in FIG. 4, each audience avatar holds the same virtual item in its hand, but the virtual items held by each audience avatar may be different.

[0026] In the example shown in FIG. 4, a part of the virtual item 71b of the spectator avatar 70b in the third row from the front is hidden by the virtual item 71a of the spectator avatar 70a in the second row from the front. That is, a part of the virtual item 71b of the spectator avatar 70b in the third row is not visible from the viewpoint of the performer avatar. In other words, a part of the virtual item 71b is not displayed in the image from the viewpoint of the performer avatar. In such a case, the display information generating unit 103 adjusts the position of the virtual item, for example, as follows. For example, the display information generating unit 103 adjusts the display position of the virtual item by adjusting which hand of the spectator avatar holds the virtual item. That is, the display information generating unit 103 adjusts the display position of the virtual item by adjusting whether the spectator avatar holds the virtual item in its right hand or its left hand.

[0027] FIG. 5 is a schematic diagram showing an example in which the display position of a virtual item is adjusted by adjusting which hand of an audience avatar holds the virtual item. In the example shown in FIG. 4, the virtual item 71b is held in the right hand of the audience avatar 70b, but in the example shown in FIG. 5, the virtual item 71b is held in the left hand of the audience avatar 70b. As a result, the position of the hidden virtual item 71b is adjusted, and the virtual item 71b is no longer hidden. Therefore, according to the example shown in FIG. 5, the entire virtual item 71b can be seen from the viewpoint of the performer avatar. In other words, the entire virtual item 71b can be displayed in the video from the viewpoint of the performer avatar. In the example shown in FIG. 5, the virtual item 71b is completely hidden by adjusting the display position, but the display position does not necessarily have to be adjusted to completely remove the hidden area. In other words, it is sufficient that the position of the virtual item is adjusted so that the hidden area is reduced.

[0028] Furthermore, the display information generating unit 103 may adjust the position of the virtual item, for example, as follows. The display information generating unit 103 may adjust the display position of the virtual item by adjusting the position of the hand holding the virtual item (the hand holding the virtual item). FIG. 6 is a schematic diagram showing an example in which the display position of the virtual item is adjusted by adjusting the position of the hand holding the virtual item. In the example shown in FIG. 4, the virtual item 71b is held in the hand at a lower position (more specifically, a position lower than the shoulder of the spectator avatar 70b in FIG. 4), but in the example shown in FIG. 6, the virtual item 71b is held at an upper position (more specifically, a position higher than the shoulder in FIG. 6). As a result, the position of the hidden virtual item 71b is adjusted, and the virtual item 71b is no longer hidden. Note that, even in the example shown in FIG. 6, the virtual item 71b is completely hidden by adjusting the display position, but the display position does not necessarily have to be adjusted to completely remove the hidden state. Also, the position of the hand holding the virtual item may be adjustable in stages within a predetermined adjustment range. In this case, the position of the hand holding the virtual item may be adjusted to the position where the virtual item 71b is least hidden in the virtual space in the vertical direction, among the positions where the virtual item 71b is least hidden. In other words, if there is no difference in the degree of hiding of the virtual item 71b whether the spectator avatar 70b's hand is raised to the maximum or to a moderate height, the display information generating unit 103 adjusts the position of the hand holding the virtual item 71b to a moderate height position. In this way, it is possible to prevent the virtual item 71b of the spectator avatar 70b from overlapping with the virtual item of the spectator avatar in the row behind the spectator avatar 70b. In addition, changing the hand holding the virtual item and adjusting the position of the hand holding the virtual item may be used in combination. Even when both are used in combination, the position of the hand may be adjusted to the position where the virtual item 71b is least hidden in the virtual space in the vertical direction, among the positions where the virtual item 71b is least hidden.In this case, if the position where the vertical height of virtual item 71b in the virtual space is lowest among the positions where the display of virtual item 71b is least hidden is the same regardless of which hand the virtual item is held in, display information generation unit 103 may select the hand holding the virtual item before adjustment as the hand holding the virtual item.

[0029] In the above example, the display position of the virtual item does not change over time, but the display information generating unit 103 may generate display information in which the display position of the virtual item changes over time. Specifically, the display information generating unit 103 may generate display information in which the display position of the virtual item is variable within a predetermined movable range. This makes it possible to provide a more realistic image. Here, the movable range refers to a range in which movement is possible, and is also called a moving range. For example, the display information generating unit 103 may generate display information in which the virtual item 71 is displayed so as to move back and forth within a predetermined movable range 72 as shown in FIG. 7. In the example shown in FIG. 7, the swinging movement of the virtual item 71 is shown with the hand 75 of the spectator avatar 70 as a fulcrum, but the virtual item 71 may be displayed so as to move in a translational manner in a predetermined direction, such as up and down. In this way, the display information generating unit 103 may generate display information in which the display position of the virtual item changes over time. In this case, the display information generating unit 103 can adjust the position of the virtual item as exemplified in FIG. 5 or FIG. 6. In this case, it can also be said that the display information generating unit 103 adjusts the position of the movable range. In other words, when the display position of a virtual item is variable within a predetermined movable range, the display information generating unit 103 may generate display information in which the position of the movable range of the virtual item is adjusted so that the hiding of the movable range of the virtual item is reduced compared to before the adjustment of the position of the movable range of the virtual item.

[0030] The movement of the virtual item may be a reciprocating motion having a predetermined motion cycle. For example, the display information generating unit 103 may generate display information in which the swinging motion shown in FIG. 7 is performed at a predetermined cycle. In the example shown in FIG. 4, when the virtual item 71a and the virtual item 71b both swing at a predetermined motion cycle, a part of the virtual item 71b continues to be hidden by the virtual item 71a. However, if the motion cycle of the virtual item 71b deviates from the motion cycle of the virtual item 71a, the virtual item 71b is hidden momentarily. Therefore, the display information generating unit 103 may adjust the display position of the virtual item, that is, the display position at a certain timing, by adjusting the motion cycle of the virtual item. In other words, the display information generating unit 103 may adjust the display position of the virtual item so that the length of time of hiding is reduced. Also, instead of or in addition to adjusting the motion cycle, the initial position of the start of the motion of the virtual item may be adjusted. If the initial position of the start of the reciprocating motion is different, the hiding becomes instantaneous, and the virtual item can be prevented from being continuously hidden. Note that both the adjustment of the movement cycle and the adjustment of the initial position at the start of the movement result in a correction of the display position of the virtual item in each frame of the video, and therefore can be considered a type of adjustment of the display position.

[0031] As described above, in this embodiment, the display information generating unit 103 generates display information in which the display position of the virtual item is adjusted so that the amount of virtual item hidden is reduced compared to before the adjustment of the display position of the virtual item. The amount of virtual item hidden refers to the amount of virtual item hidden by an object in the virtual space (e.g., an audience avatar or another virtual item) that exists in front of the virtual item of interest. When the virtual item is hidden, the total number of virtual items perceived by the performer who sees the video from the viewpoint of the performer avatar becomes smaller than the actual number of virtual items that exist in the virtual space. Therefore, the performer cannot adequately feel the excitement of the performance. In contrast, in this embodiment, the amount of virtual item hidden in the display is reduced, so that the performer who sees the video from the viewpoint of the performer avatar can adequately feel the excitement of the performance. Therefore, the performer can effectively feel the presence of the audience in a performance held in a virtual space. Also, the audience can more effectively appeal to the performer about the virtual items.

[0032] Next, the flow of the process of generating display information for performers will be described. Fig. 8 is a flowchart showing an example of the flow of the process of generating display information for performers. Hereinafter, the flow of the process of generating display information in this embodiment will be described with reference to Fig. 8.

[0033] In step S100, the placement determination unit 102 determines the placement of the performer avatars and audience avatars in the virtual space. Next, in step S101, the display information generating unit 103 generates display information that expresses the virtual space in which the performance is performed from the viewpoint of the performer avatar. At this time, the display information output unit 104 generates display information in which the position of the virtual item is adjusted to a position that is easy to see from the performer avatar (performer) as described above. The display information generating unit 103 adjusts the position of the virtual item, for example, starting from the audience avatar in the front row. The reason for adjusting the position of the virtual item starting from the audience avatar in the front row is as follows. There is a possibility that moving another virtual item in the row behind this virtual item is necessary due to the movement of the virtual item. Therefore, if the position adjustment is performed starting from the back row, there is a possibility that the movement of the virtual item in the back row may be required multiple times, which is inefficient. Furthermore, if the display information generating unit 103 cannot adjust the position of the virtual item to a position that is easy to see, it may skip the processing for the virtual item. For example, if the display information generating unit 103 cannot adjust the amount of reduction in the hidden display compared to before the position adjustment to a predetermined threshold or more, it may skip the processing for the virtual item. Here, the threshold value is defined, for example, as a percentage of the area of ​​the virtual item visible from the performer's viewpoint. Then, in step S102, the display information output unit 104 outputs the display information generated in step S101 to the performer device 200. After step S102, the process returns to step S101, and the generation and output of the display information is repeated. Note that the positions of the performer avatars or the audience avatars may vary over time, in which case the process returns to step S100 after step S102.

[0034] <Embodiment 2> Next, a description will be given of embodiment 2. This embodiment differs from embodiment 1 in that the state of the spectators in the real space is represented in the virtual space.

[0035] Fig. 9 is a block diagram showing an example of the configuration of an information processing system 10a according to embodiment 2. The information processing system 10a differs from the information processing system 10 shown in Fig. 1 in that each audience device 300 is equipped with a microphone 310, and the information processing device 100 is replaced with an information processing device 100a. Below, configurations and processes that differ from embodiment 1 will be described, and overlapping descriptions will be omitted as appropriate.

[0036] The microphone 310 captures sounds generated by the audience. Here, the sounds generated by the audience may be, for example, the voices of the audience, the sounds of applause, or the sounds of hand clapping. Each audience device 300 transmits the sounds captured by the microphone 310 to the information processing device 100a.

[0037] Fig. 10 is a block diagram showing an example of a functional configuration of the information processing device 100a according to the present embodiment. As shown in Fig. 10, the information processing device 100a is different from the information processing device 100 shown in Fig. 2 in that an audience sound acquisition unit 108 is added and the display information generation unit 103 is replaced with a display information generation unit 103a.

[0038] The audience sound acquisition unit 108 acquires sounds generated by audience members corresponding to the audience avatars. Specifically, the audience sound acquisition unit 108 acquires sound data transmitted from the audience devices 300. The audience sound acquisition unit 108 acquires sounds for each audience member.

[0039] The display information generating unit 103a performs the same process as the display information generating unit 103, but differs from the display information generating unit 103 of the first embodiment in that it further has the following feature. The display information generating unit 103a generates display information that displays a virtual item with a visual corresponding to the sound acquired by the audience sound acquiring unit 108. More specifically, the display information generating unit 103a changes the visual of a virtual item provided in a spectator avatar corresponding to the spectator who generated the sound acquired by the audience sound acquiring unit 108 according to the sound. Specifically, the display information generating unit 103a displays a virtual item with a visual that is more noticeable to human perception as the volume of the sound increases. For example, the display information generating unit 103a may generate display information that displays a virtual item with a color corresponding to the volume of the sound, or may generate display information that displays a virtual item with a brightness corresponding to the volume of the sound. For example, if the virtual item is a virtual space simulating a light-emitting item such as a penlight, the color or brightness of the light emitted may be changed according to the volume. For example, the display information generating unit 103a may make the color of the virtual item closer to a specific color as the volume becomes louder, or may make the virtual item brighter as the volume becomes louder. The display information generating unit 103a may use the duration of the volume equal to or greater than a threshold together with or instead of the volume. That is, the display information generating unit 103a may generate display information that displays the virtual item in a visual that corresponds to the duration of the sound with a volume equal to or greater than a threshold. Specifically, the display information generating unit 103a displays the virtual item in a visual that is more noticeable to human perception as the duration of the sound with a volume equal to or greater than a threshold becomes longer.

[0040] Next, the flow of the process of generating display information for performers in this embodiment will be described. Fig. 11 is a flow chart showing an example of the flow of the process of generating display information for performers in embodiment 2. Hereinafter, the flow of the process of generating display information in this embodiment will be described with reference to Fig. 11.

[0041] In step S200, the placement determination unit 102 determines the placement of the performer avatars and audience avatars in the virtual space. Next, in step S201, the audience sound acquisition unit 108 acquires sounds generated by the audience. Next, in step S202, the display information generating unit 103a generates display information that expresses the virtual space in which the performance takes place from the viewpoint of the performer avatar. At this time, the display information generating unit 103a adjusts the display position of the virtual item, as in the first embodiment. Particularly, in this step, the display information generating unit 103 generates display information that displays the virtual item visually according to the sound acquired in step S201. Then, in step S203, the display information output unit 104 outputs the display information generated in step S202 to the performer device 200. After step S203, the process returns to step S201, and the acquisition of audience sounds and the generation and output of display information are repeated. Note that the positions of the performer avatars or the audience avatars may vary over time, in which case the process returns to step S200 after step S203.

[0042] The second embodiment has been described above. In general, audience members who are enjoying a performance make sounds by voices, clapping, hand clapping, and the like. The more the audience members enjoy the performance, the louder the volume of the sounds they make and the longer the duration of the sounds they make. In this embodiment, the visuals of the virtual items change according to the sounds made by the audience, so that the performers can easily perceive the reactions of the audience members who are in different locations in the real space. In particular, the visuals of the virtual items change according to the sounds, so that the performers can visually recognize the magnitude of the audience members' reactions. In this way, in this embodiment, the performers can effectively feel the presence of the audience members, and in particular, the audience members' reactions. In particular, in this embodiment, the acquired sounds of the audience members are not provided to the performers as they are, but the acquired sounds of the audience members are converted into visuals of virtual items to provide the audience members with their reactions. If the sounds made by the audience are provided to the performers, there is a risk that the performers will not be able to concentrate on the performance. However, in this embodiment, as described above, the audience members' reactions are provided to the performers as visual information, so that such a problem can be solved. In addition, since the visuals of the virtual items can be changed without the audience members having to operate a controller or the like, the audience members can also concentrate on watching the performance.

[0043] The above-described functions (processing) of the information processing device 100 and the information processing device 100a may be realized by a computer 500 having the following configuration, for example.

[0044] 12 is a block diagram showing a configuration of a computer 500 that realizes the processing of the information processing device 100 or the information processing device 100a. As shown in FIG.

[0045] The network interface 501 is used for communication over the network 400. The network interface 501 may include, for example, a network interface card (NIC).

[0046] The memory 502 is configured, for example, by a combination of a volatile memory and a non-volatile memory. The memory 502 is used to store programs executed by the processor 503, data used in processing, etc. The above-mentioned information storage unit 101 may be realized by a storage device such as the memory 502.

[0047] The processor 503 performs processing of the information processing device 100 or the information processing device 100a by reading and executing a program from the memory 502. The processing of the components of the information processing device 100 or the information processing device 100a shown in FIG. 2 or FIG. 10 is performed, for example, by the processor 503 reading and executing a program from the memory 502. The processor 503 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU (Central Processing Unit). The processor 503 may include multiple processors.

[0048] The program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0049] Although the embodiments have been described above, the present invention is not limited to the above embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, in the second embodiment, the configuration for changing the visual appearance of a virtual item based on the sound of the audience is combined with the configuration for adjusting the position of the virtual item to a position that is easily visible from the performer avatar, but the former configuration may be implemented without being combined with the latter configuration.

[0050] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes. (Appendix 1) a placement determination unit that determines placement of a performer avatar and a plurality of audience avatars in a virtual space; a display information generating unit that generates display information representing the virtual space from the viewpoint of the performer avatar; having At least one of the spectator avatars has a virtual item on a part of its body; The display information generating unit generates the display information in which the display position of the virtual item is adjusted so that the hiding of the display of the virtual item is reduced compared to before the adjustment of the display position of the virtual item. Information processing device. (Appendix 2) The spectator avatar holds the virtual item in his / her hand, The display information generating unit adjusts the display position of the virtual item by adjusting which hand of the spectator avatar holds the virtual item. 2. An information processing device according to claim 1. (Appendix 3) The spectator avatar holds the virtual item in his / her hand, The display information generating unit adjusts the display position of the virtual item by adjusting the position of the hand holding the virtual item. 3. The information processing device according to claim 1 or 2. (Appendix 4) When the display position of the virtual item is variable within a predetermined movable range, the display information generation unit generates the display information in which the position of the movable range of the virtual item is adjusted so that the hiding of the movable range of the virtual item is reduced compared to before the adjustment of the position of the movable range of the virtual item. 4. The information processing device according to claim 1 . (Appendix 5) The display information generating unit adjusts the display position of the virtual item by adjusting a motion period of the virtual item or an initial position at which the virtual item starts moving. 5. The information processing device according to claim 4. (Appendix 6) Further comprising an audience sound acquisition unit for acquiring sounds generated by audience members corresponding to the audience avatars, The display information generating unit generates the display information that displays the virtual item in a visual form corresponding to the sound. 6. An information processing device according to any one of claims 1 to 5. (Appendix 7) The display information generating unit generates the display information that displays the virtual item with a color or brightness corresponding to the sound. 7. The information processing device according to claim 6. (Appendix 8) determining a placement of the performer avatar and the multiple audience avatars in the virtual space; generating display information representing the virtual space from the viewpoint of the performer avatar; At least one of the spectator avatars has a virtual item on a part of its body; In generating the display information, the display information is generated in which the display position of the virtual item is adjusted so that the amount of hiding of the display of the virtual item is reduced compared to before the adjustment of the display position of the virtual item. Information processing methods. (Appendix 9) a placement determination step of determining placements of a performer avatar and a plurality of audience avatars in a virtual space; a display information generating step of generating display information representing the virtual space from the viewpoint of the performer avatar; on the computer, At least one of the spectator avatars has a virtual item on a part of its body; In the display information generating step, the display information is generated in which the display position of the virtual item is adjusted so that the amount of hiding of the display of the virtual item is reduced compared to before the adjustment of the display position of the virtual item. program. [Explanation of symbols]

[0051] 10. Information Processing Systems 10a Information Processing Systems 50 Virtual Space 51 Stage Area 52 Appreciation area 60 Performer Avatars 70 Spectator Avatar 70a Spectator Avatar 70b Spectator Avatar 71 Virtual Items 71a Virtual Items 71b Virtual Items 72 Range of motion 75 hands 100 Information processing device 100a Information processing device 101 Information storage section 102 Placement determination section 103 Display information generation section 103a Display information generation section 104 Display information output unit 105 Performance information acquisition unit 106 Distribution information generation unit 107 Distribution Department 108 Audience Sound Acquisition Section 200 Performer Equipment 300 Spectator Equipment 310 Mike 400 Network 500 Computers 501 Network Interface 502 Memory 503 Processor

Claims

1. a placement determination unit that determines placement of a performer avatar and a plurality of audience avatars in a virtual space; a display information generating unit that generates display information representing the virtual space from the viewpoint of the performer avatar; having At least one of the spectator avatars has a virtual item on a part of its body; The display information generating unit generates the display information in which the display position of the virtual item is adjusted so that the hiding of the display of the virtual item is reduced compared to before the adjustment of the display position of the virtual item. Information processing device.

2. The spectator avatar holds the virtual item in his / her hand, The display information generating unit adjusts the display position of the virtual item by adjusting which hand of the spectator avatar holds the virtual item. The information processing device according to claim 1 .

3. The spectator avatar holds the virtual item in his / her hand, The display information generating unit adjusts the display position of the virtual item by adjusting the position of the hand holding the virtual item.

3. The information processing device according to claim 1 or 2.

4. When the display position of the virtual item is variable within a predetermined movable range, the display information generation unit generates the display information in which the position of the movable range of the virtual item is adjusted so that the hiding of the movable range of the virtual item is reduced compared to before the adjustment of the position of the movable range of the virtual item. The information processing device according to claim 1 .

5. Further comprising an audience sound acquisition unit for acquiring sounds generated by audience members corresponding to the audience avatars, The display information generating unit generates the display information that displays the virtual item in a visual form corresponding to the sound. The information processing device according to claim 1 .

Images