Virtual space content delivery system, virtual space content delivery program, and virtual space content delivery method

Abstract

To provide a virtual space content delivery system for preventing decline in interest of a virtual space content.SOLUTION: A virtual space content delivery system includes: a performer terminal available for use by a performer; viewer user terminals enabling viewer users to a view virtual space content; and a server computer capable of executing processing relating to a virtual space and processing of delivering the virtual space content including an action content of a performer avatar, to the viewer user terminals, wherein the action content includes: a first action content in which action of the performer is reflected in the performer avatar in real time; and a second action content recorded in advance. When the first action content and the second action content are delivered in succession, the virtual space content delivery system executes specific processing of preventing the viewer users from recognizing that a posture of the performer avatar is not continuous at the time of switching over the action content.SELECTED DRAWING: Figure 40

Description

[Technical field]

[0001] The present invention relates to a virtual space content distribution system, a virtual space content distribution program, and a virtual space content distribution method capable of distributing virtual space content including at least action content in which a performer avatar moves in a virtual space. [Background technology]

[0002] Conventionally, there has been known a distribution system that generates animation of a character object (avatar) of a distribution user (performer) that moves in a virtual space in conjunction with the movements of the distribution user (performer), live-distributes a video including the animation of the character object (avatar), and also enables viewer users watching the video to virtually participate as spectators in the virtual space in which the avatar of the distribution user (performer) is placed, upon request from the viewer users (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-120098 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, as in Patent Document 1, when live streaming animation content of a character object (avatar) that reflects the movements of the streaming user (performer), the animation content of the character object (avatar) based on the movements of the streaming user (performer) is partially pre-recorded and distributed in order to reduce the processing load on the server computer or to prevent accidents such as an equipment malfunction causing the movements of the streaming user (performer) to not be properly reflected in the avatar. However, when only part of the content to be streamed is recorded and distributed in this way, there is a problem that when switching between the part where the movements of the streaming user (performer) are reflected in the character object (avatar) in real time and the recorded part, the posture of the character object (avatar) is different, which gives a sense of discomfort to the viewer and reduces the interest of the virtual space content.

[0005] The present invention has been made in response to such problems, and aims to provide a virtual space content distribution system, a virtual space content distribution program, and a virtual space content distribution method that can prevent a decline in interest in virtual space content. [Means for solving the problem]

[0006] The virtual space content delivery system of claim 1 comprises: A virtual space content delivery system capable of delivering virtual space content (e.g., a virtual live performance) including at least an action content in which a performer avatar acts in a virtual space, A performer terminal (e.g., performer terminal 200A, B) that can be used by a performer (e.g., performer user A, B) corresponding to the performer avatar (e.g., performer avatar 1A, 1B); A viewer user terminal (e.g., viewer terminal 300) on which a viewer user can view the virtual space content; A server computer (e.g., a distribution server computer 100) that is connected to the performer terminal and the viewer user terminal via a communication network and is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; Including, The action content includes a first action content (e.g., MC) in which the action of the performer inputted to the performer terminal when the action content is distributed is reflected in the performer avatar, and a second action content (e.g., a performance) that is recorded in advance before the action content is distributed, When the first action content and the second action content are delivered in succession, a process is performed to prevent a viewer user from recognizing that the pose of the performer avatar is not continuous when switching between action contents, the process including a blackout process for blacking out the action content displayed on the viewer user terminal (e.g., a pose assimilation process, a blackout effect reflection process, an object effect reflection process) is performed. It is characterized by the following. According to this feature, by executing a specific process, it is possible to prevent the viewer user from recognizing that the posture of the performer avatar is discontinuous when switching between action content, thereby preventing a decrease in interest in the virtual space content due to the viewer feeling uncomfortable due to differences in the posture of the performer avatar when switching between action content.

[0007] The virtual space content delivery system of claim 2 is the virtual space content delivery system of claim 1, When the second action content is delivered following the first action content, a first identification process (e.g., a posture assimilation process) is executed as the identification process; When the first operation content is distributed following the second operation content, a second specific process (for example, a blackout effect reflection process, an object effect reflection process) different from the first specific process is executed as the specific process. It is characterized by the following. According to this feature, different processing can be executed depending on the switching status of the action content, thereby more accurately preventing a decrease in interest in the virtual space content due to the viewer feeling uncomfortable due to differences in the posture of the performer avatar.

[0008] The virtual space content delivery system of claim 3 is the virtual space content delivery system according to claim 2, the first identification process includes a posture assimilation process (e.g., the posture assimilation process shown in FIG. 41 ) for causing the performer to assume an initial start posture of the performer avatar in the second action content at the end of the first action content; The second specific process includes a specific performance process (e.g., a blackout performance reflection process, an object performance reflection process) for executing a specific performance that makes it difficult for a viewer user to recognize the movement of the performer avatar, It is characterized by the following. According to this feature, by executing a specific performance that has a relatively low processing load as the second specific process, it is possible to reduce the overall processing load when switching between operation contents.

[0009] The virtual space content delivery system of claim 4 is the virtual space content delivery system according to claim 3, The start posture is stored in advance (for example, posture data of a target posture is stored in the distribution server computer 100), The performer terminal is equipped with a posture display means for displaying the starting posture so that the performer can visually confirm it (for example, as shown in FIG. 34 and FIG. 35, a portion equipped with a display device C212 capable of displaying pre-recorded poses). It is characterized by the following. This feature allows the performer to easily visually grasp the starting position.

[0010] The virtual space content delivery system of claim 5 is the virtual space content delivery system according to claim 2, the first specific process includes a specific performance process for executing a specific performance that makes it difficult for a viewer user to recognize the posture of the performer avatar (for example, a blackout performance reflection process for executing a blackout performance during a transition period from an MC to a program); The second specific process includes a posture assimilation process for making the performer take the final end posture of the performer avatar in the second action content at the start of distribution of the first action content (for example, a part that executes the posture assimilation process of FIG. 41 during a transition period from a performance to an MC). It is characterized by the following. According to this feature, by executing a specific performance that has a relatively low processing load as the first specific process, it is possible to reduce the overall processing load when switching between operation contents.

[0011] The virtual space content delivery system of claim 6 is the virtual space content delivery system according to claim 5, The end posture is stored in advance (for example, posture data of the target posture is stored in the distribution server computer 100), The performer terminal is equipped with a posture display means for displaying the end posture so that the performer can visually confirm it (for example, as shown in Figures 34 and 35, a portion equipped with a display device C212 capable of displaying pre-recorded poses). It is characterized by the following. This feature allows the performer to easily visually grasp the final posture.

[0012] The virtual space content delivery system of claim 7 is the virtual space content delivery system according to claim 4 or 6, The posture display means is capable of displaying the posture of the performer avatar as a viewpoint image from behind the performer avatar (for example, an image from behind as in FIGS. 34 and 35). It is characterized by the following. According to this feature, the performer can grasp the posture he or she should assume as an actual image that is the same on both sides, rather than as a mirror image with the left and right reversed, making it even easier for the performer to recognize the posture.

[0013] The virtual space content delivery system of claim 8 is the virtual space content delivery system according to claim 4 or 6, The posture display means is capable of displaying period information (for example, display of a remaining period display area 252) capable of specifying the period until the start of distribution of the second action content or the period until the start of distribution of the first action content. It is characterized by the following. This feature allows the performer to know exactly how much time is left before assuming a posture.

[0014] The virtual space content delivery system of claim 9 is the virtual space content delivery system according to claim 4 or 6, The posture assimilation process includes a process of identifying a difference between the posture displayed by the posture display means and the posture of the performer, and reflecting a correction movement corresponding to the identified difference in the movement of the performer avatar (e.g., a correction information transmission process of transmitting correction information). It is characterized by the following. According to this feature, even if the posture of the performer does not completely match the posture displayed by the posture display means, the corrective action is reflected, and it is possible to more accurately prevent a decrease in interest in the virtual space content caused by the viewer feeling uncomfortable due to differences in the posture of the performer avatar.

[0015] The virtual space content delivery system of claim 10 is the virtual space content delivery system according to claim 4 or 6, At a determination timing (e.g., posture determination timing) when the remaining period until the distribution start timing of the second action content or the remaining period until the distribution start timing of the first action content becomes a predetermined specific period, when the difference between the posture displayed by the posture display means and the posture of the performer is greater than a specified value (e.g., 5% corresponding to a degree of agreement of 95%), the fact that the difference is greater than the specified value is notified on the performer terminal (e.g., a portion notifying by displaying a yellow frame). It is characterized by the following. According to this feature, the performer can recognize that the difference between his / her posture and the posture displayed by the posture display means is larger than a specified value.

[0016] The virtual space content delivery system of claim 11 is the virtual space content delivery system according to claim 4 or 6, At a determination timing (e.g., posture determination timing) when the remaining period until the distribution start timing of the second action content or the remaining period until the distribution start timing of the first action content becomes a predetermined specific period, when a difference between the posture displayed by the posture display means and the posture of the performer is greater than a limit value (e.g., 40% corresponding to a degree of coincidence of 60%), a specific effect (e.g., blackout effect) is executed that makes it difficult for a viewer user to recognize the posture of the performer avatar. It is characterized by the following. According to this feature, if the difference between the performer's posture and the posture displayed by the posture display means is greater than a limit value, a specific effect is executed that makes it difficult for the viewer user to recognize the posture of the performer avatar, so that it becomes difficult for the viewer user to recognize the movements of the performer avatar. This avoids a decrease in interest due to the viewer feeling uncomfortable due to differences in the posture and movements of the performer avatar.

[0017] The virtual space content delivery program according to claim 12, A virtual space content distribution program for distributing virtual space content including at least an action content in which a performer avatar acts in a virtual space, comprising: The action content includes a first action content in which the action of the performer inputted into the performer terminal when the action content is distributed is reflected in the performer avatar, and a second action content recorded in advance before the action content is distributed, the first action content being provided from a server computer that is connected via a communication network to a performer terminal that can be used by the performer corresponding to the performer avatar and to a viewer user terminal on which a viewer user can view the virtual space content, and that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; a process to be executed when the first action content and the second action content are delivered successively, the process including a specific process for preventing a viewer user from recognizing that the posture of the performer avatar is not continuous when the action content is switched; The specific process includes a darkening process for darkening the motion content displayed on the viewer user terminal. It is characterized by the following. According to this feature, by executing a specific process, it is possible to prevent the viewer user from recognizing that the posture of the performer avatar is discontinuous when switching between action content, thereby preventing a decrease in interest in the virtual space content due to the viewer feeling uncomfortable due to differences in the posture of the performer avatar when switching between action content.

[0018] The virtual space content delivery method according to claim 13 comprises: A virtual space content distribution method for distributing virtual space content including at least an action content in which a performer avatar acts in a virtual space, comprising: The action content includes a first action content in which the action of the performer inputted into the performer terminal when the action content is distributed is reflected in the performer avatar, and a second action content recorded in advance before the action content is distributed, the first action content being provided from a server computer that is connected via a communication network to a performer terminal that can be used by the performer corresponding to the performer avatar and to a viewer user terminal on which a viewer user can view the virtual space content, and that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; a procedure to be executed when the first action content and the second action content are delivered successively, the procedure including a specific procedure for preventing a viewer user from recognizing that the posture of the performer avatar is not continuous when the action content is switched; The specific procedure includes a dimming procedure for dimming the motion content displayed at the viewer user terminal. It is characterized by the following. According to this feature, by executing a specific process, it is possible to prevent the viewer user from recognizing that the posture of the performer avatar is discontinuous when switching between action content, thereby preventing a decrease in interest in the virtual space content due to the viewer feeling uncomfortable due to differences in the posture of the performer avatar when switching between action content.

[0019] Furthermore, the present invention may have only the invention-specific matters described in the claims of the present invention, or may have the invention-specific matters described in the claims of the present invention as well as configurations other than the invention-specific matters. [Brief description of the drawings]

[0020] [Figure 1] 1 is a block diagram showing an example of a system configuration of a virtual space content delivery system according to a first embodiment of the present invention. [Diagram 2] 2 is a diagram illustrating an example of the configuration of a delivery server computer in the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Diagram 3]FIG. 2 is a diagram showing various data stored in a storage device of a delivery server computer according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a performer terminal in the virtual space content delivery system according to the first embodiment of the present invention. [Diagram 5] 2 is a diagram illustrating an example of the configuration of a viewer terminal in the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 6] 2 is a diagram illustrating a configuration example of an administrator terminal in the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 7] 3 is a diagram showing an example of the configuration of event data in the virtual space content delivery system according to the first embodiment of the present invention; FIG. [Figure 8] 2 is a diagram showing an example of the configuration of performer user data in the virtual space content delivery system according to the first embodiment of the present invention; FIG. [Figure 9] 3 is a diagram showing an example of the configuration of viewer user data in the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 10] 2 is an explanatory diagram showing data exchanged between each terminal and a server computer constituting the virtual space content delivery system according to the first embodiment of the present invention; FIG. [Figure 11] 1 is a schematic diagram showing the facilities of a studio used by a performer user in the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 12] FIG. 1 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content delivery system according to a first embodiment of the present invention. [Figure 13] 3 is a diagram showing a display example on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 14] 3 is a diagram showing a display example on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 15] 3A to 3C are diagrams illustrating examples of displays on a performer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 16]1A to 1C are diagrams illustrating an example of a display by a zoom operation on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 17] 1A to 1C are diagrams illustrating an example of a display by a zoom operation on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 18] 3 is a diagram showing a display example on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 19] FIG. 2 is a diagram showing an example of the display of a flying performance period on a viewer terminal constituting the virtual space content delivery system of the first embodiment of the present invention. [Figure 20] FIG. 2 is a flow diagram showing an example of a viewer viewpoint video control process executed in the viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 21] FIG. 11 is an explanatory diagram showing the contents of a virtual live performance delivered by the virtual space content delivery system according to the second embodiment of the present invention. [Figure 22] 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. FIG. [Figure 23] FIG. 11 is a diagram showing a participant user table used in a delivery server computer constituting a virtual space content delivery system according to a second embodiment of the present invention. [Figure 24] 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. FIG. [Diagram 25] 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. FIG. [Figure 26] 11 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. FIG. [Figure 27]FIG. 11 is an explanatory diagram showing the relationship between the period type of a virtual live show and the entity that generates the virtual camera viewpoint video in the virtual space content delivery system according to the second embodiment of the present invention. [Figure 28] FIG. 11 is an explanatory diagram showing the generation status of a virtual camera viewpoint video and a viewer viewpoint video in a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. [Figure 29] FIG. 11 is a flowchart showing a viewer viewpoint video generation process in a modified example of the second embodiment of the present invention. [Diagram 30] FIG. 13 is a diagram showing the contents of virtual space update data transmitted from a delivery server computer in a modified example of the second embodiment of the present invention. [Diagram 31] FIG. 11 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content distribution system according to a third embodiment of the present invention. [Diagram 32] FIG. 11 is a block diagram showing an example of a system configuration of a virtual space content delivery system according to a third embodiment of the present invention. [Diagram 33] FIG. 11 is a schematic diagram showing the facilities of studio B used by performer user B in a virtual space content delivery system according to a third embodiment of the present invention. [Diagram 34] FIG. 11 is a diagram showing a display example of a display device C installed in a studio A used by a performer user A in the third embodiment of the present invention. [Diagram 35] FIG. 11 is a diagram showing a display example of a display device C installed in a studio B used by a performer user B in the third embodiment of the present invention. [Diagram 36] FIG. 11 is an explanatory diagram showing the contents of a virtual live performance delivered by a virtual space content delivery system according to a third embodiment of the present invention. [Figure 37] FIG. 11 is an explanatory diagram showing the contents of a virtual live performance delivered by a virtual space content delivery system according to a third embodiment of the present invention. [Figure 38] FIG. 11 is a diagram showing an example of the configuration of time schedule data (TS data) stored in a delivery server computer constituting a virtual space content delivery system according to a third embodiment of the present invention. [Figure 39]11 is an explanatory diagram showing data exchanged between a delivery server computer and a performer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. FIG. [Diagram 40] 11 is an explanatory diagram showing data exchanged between a delivery server computer and a performer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. FIG. [Diagram 41] FIG. 11 is a flow diagram showing the contents of a posture assimilation process executed in a performer terminal constituting a virtual space content delivery system according to a third embodiment of the present invention. [Diagram 42] FIG. 13 is a diagram showing a transition period 2 of a virtual live broadcast in the third embodiment of the present invention. [Diagram 43] FIG. 11 is a diagram showing a transition period 4 of a virtual live broadcast in the third embodiment of the present invention. [Diagram 44] FIG. 11 is a diagram showing a transition period 10 of a virtual live broadcast in the third embodiment of the present invention. [Diagram 45] FIG. 13 is a diagram showing a branching situation in transition period 5 of a virtual live broadcast in the third embodiment of the present invention. [Figure 46] FIG. 11 is an explanatory diagram showing a correction operation generation principle in the posture assimilation process executed in the third embodiment of the present invention. [Figure 47] FIG. 13 is a diagram showing a transition period executed in a modified example 3-1 of the third embodiment of the present invention. [Figure 48] FIG. 13 is a diagram showing a transition period executed in a modified example 3-2 of the third embodiment of the present invention. [Figure 49] FIG. 13 is a diagram showing a transition period executed in a modified example 3-3 of the third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The following describes the form for implementing the virtual space content distribution system of the present invention based on the embodiments and with reference to the drawings. Note that the same or similar components in the drawings are given the same reference numerals, and duplicated explanations are omitted. The virtual space content distribution program according to the present invention may be a program used in the entire system consisting of the server computer and each terminal described later that constitute the virtual space content distribution system, or may be a program that operates only on a part of the devices that constitute the virtual space content distribution system, such as the server computer and each terminal, and each process described below is executed by a program stored in the device that performs the process. EXAMPLES

[0022] Fig. 1 is a block diagram showing an example of the system configuration of a virtual space content distribution system in Example 1 which is an embodiment of the present invention, and in the system of Example 1 shown in Fig. 1, a virtual live performance in which a performer avatar 1 corresponding to a performer user performs a live event in a virtual live venue shown in Fig. 12 is distributed as virtual space content. Note that in the following, distribution means transmission to multiple people and is synonymous with transmission.

[0023] As shown in FIG. 1, the virtual space content distribution system of this embodiment 1 is mainly composed of a distribution server computer 100 capable of executing various processes related to the virtual live venue shown in FIG. 11, which is a virtual space, and various processes related to the distribution of the virtual live event, an administrator terminal 150 used by an event manager M of an event management organization that operates and manages the virtual live event to manage the distribution of the virtual live event, a performer terminal 200 used by a performer user, and a viewer terminal 300 that can be used by a viewer user who virtually participates in and watches the virtual live event, and the distribution server computer 100, the administrator terminal 150, the performer terminal 200, and the viewer terminal 300 are connected to each other so that they can communicate data with each other via the Internet N, which is an open computer network.

[0024] In this embodiment 1, the administrator terminal 150 and the performer terminal 200 are also shown as being connected to the distribution server computer 100 via the Internet network N, but the present invention is not limited to this. For example, if the studio shown in FIG. 11 is a studio operated by an event management organization which provides the studio to a performer user, and the distribution server computer 100 is installed in this studio, the administrator terminal 150 and the performer terminal 200 may be locally connected to the distribution server computer 100 via data communication, or may be connected via a local area network LAN within the studio facility.

[0025] 1, there are multiple viewer users, such as viewer user A, viewer user B, viewer user C, etc., and viewer terminals 300 include viewer terminals 300a, 300b, 300c, etc. that can be used by each of viewer users A, B, C, etc. Note that there are usually multiple viewer users, but there may be one or four or more, and the number may be appropriately determined depending on the virtual space, such as a virtual live venue. Hereinafter, viewer user A, viewer user B, viewer user C, etc. may be collectively referred to as viewer users, and viewer terminals 300a, 300b, 300c, etc. may be collectively referred to as viewer terminals 300.

[0026] Furthermore, in this embodiment 1, an example is given of a form in which there is one performer user, but the present invention is not limited to this, and the virtual live performance may be a collaborative event in which multiple performer users working individually virtually participate, or a group event in which a group of multiple people working together virtually participate. In this way, when multiple performers appear, performer terminal 200a, performer terminal 200b, performer terminal 200c... may be provided for each performer user, or if a group of multiple people appears, one performer terminal 200 may be used by multiple people.

[0027] In addition, although FIG. 1 illustrates the distribution server computer 100 as a single unit, the present invention is not limited to this. These distribution server computers 100 may be configured with multiple server computers, such as a first server computer that mainly performs virtual space processing and a second server computer that mainly performs distribution processing, or may be configured as a cloud server, etc., so that the number of server computers can be changed appropriately depending on the scale of the event, the number of participating listener users, etc.

[0028] Similarly, although the administrator terminal 150 is illustrated as a single unit in FIG. 1, the present invention is not limited to this, and it goes without saying that there may be multiple administrator terminals 150 depending on the number of performer users and viewer users virtually participating in the live performance.

[0029] <Distribution server computer> Fig. 2 is a diagram showing the configuration of the distribution server computer 100 used in this embodiment 1. As described above, the distribution server computer 100 used in this embodiment 1 is a normal server computer with relatively excellent processing power installed in a studio operated by an event management organization, and as shown in Fig. 2, it has a processor (CPU) 101 connected to a data bus 108, a memory (RAM) 102, a storage 103 such as a hard disk, a communication interface (I / F) 104 capable of two-way communication by the TCP / IP protocol via the Internet, which is an open computer network, and an input / output interface (I / F) 105 to which an input device such as a keyboard can be connected.

[0030] In this embodiment 1, an input / output interface (I / F) 105 is provided to enable local operation, but if remote operation is possible via a communication interface (I / F) 104, the input / output interface (I / F) 105 may not be provided.

[0031] In addition to an operation system (server OS) for providing server functions (not shown), storage 103 of distribution server computer 100 stores various data and programs as shown in Fig. 3. Specifically, mainly stored are an event management program 110 for providing an event management function for managing live events, a virtual space generation program 120 for providing a virtual space of a virtual live venue, which is a virtual space, a performer user management program 130 for providing a performer user management function for managing performer users, and a viewer user management program 132 for providing a viewer user management function for managing viewer users.

[0032] The event management program 110 is a program for managing a virtual live event, which is an event held at a virtual live venue (see FIG. 12), in cooperation with the administrator terminal 150, and is capable of managing the progress of the virtual live event by using event data 111 (see FIG. 7), time schedule data (TS data) 112, and music data 113 stored in the storage 103 together with the event management program 110.

[0033] An example of the event data 111 used in the present embodiment 1 is shown in Fig. 7. The event data 111 stores information such as an event name, a file name of time schedule (TS) data, a performer ID uniquely assigned to a performer user who is a performer, a file name of a music list, a file name of a participating viewer list in which information of special viewer users who are permitted to participate virtually is registered in a third area of ​​the virtual audience area described later, and a file name of a participating viewer list in which information of viewer users who have reserved virtual participation is registered in a fourth area of ​​the virtual audience area described later, in association with an event ID uniquely assigned to the event.

[0034] The time schedule data (TS data) 112 is data in which the order of progress of the virtual live event to be held and the programs such as songs are described in chronological order, and the virtual live event is managed so as to progress based on the time schedule described in the time schedule data (TS data) 112. Moreover, the time schedule data (TS data) 112, event data 111, etc. can be changed, updated, or added by the event manager M operating the manager terminal 150.

[0035] Furthermore, the song data 113 is data for the performance parts (so-called karaoke) of each song sung by the performer user, and is played by the audio control program 124 (described later) based on the time schedule data (TS data) 112 by the event management program 110, and transmitted to the performer terminal 200 and the viewer terminal 300 (see FIG. 11).

[0036] The virtual space generation program 120 is a program for providing a virtual live venue, which is the virtual space shown in Figure 12, and has the function of providing a virtual live venue in which performer avatars and viewer avatars are virtually participating, using virtual space data 121 describing the structure of the virtual live venue, which is stored in storage 103 together with the virtual space generation program 120, object data 122 for placing various objects to be placed in the virtual live venue (including virtual placement object 43, which is a star decoration on the stage, and virtual aerial objects 50-52, which are star objects in the sky), etc., within the virtual space, and performer avatar data contained in performer user data 131 described later and viewer avatar data contained in viewer user data 133.

[0037] 12, viewpoint image control program 123 is a program for providing a function to generate viewpoint images from virtual cameras C1-C4 virtually set in the virtual live venue and an airborne virtual camera (not shown), and also to provide a function to display images from the viewpoints of viewer avatars 11-14 virtually participating in the third area of ​​the virtual live venue and viewer avatars 21-32 virtually participating in the fourth area on viewer terminal 300. Virtual camera C2 is a virtual camera set to provide an image from the viewpoint of performer avatar 1 on the stage of the virtual live venue, and the viewpoint image from virtual camera C2 is projected onto a screen S installed in the studio by projector 211 (described later) connected to performer terminal 200 in cooperation with performer terminal 200.

[0038] The audio control program 124 is a program for controlling the audio (including music) distributed during the virtual live event, and specifically provides a function for playing music data 113 and a function for synthesizing the audio produced by the performer user with the played music and distributing it to the administrator terminal 150 and the viewer terminal 300.

[0039] The comment control program 125, in cooperation with a viewer comment control program stored in the viewer terminal 300 described below, provides a function for displaying various comments entered by each viewer user at the viewer terminal 300 during a virtual live event in chronological order on the performer terminal 200 and the viewer terminal 300.

[0040] The gift control program 126 is a program for controlling gifts given from viewer users to performer users in cooperation with a viewer program stored in the viewer terminal 300 described below, and specifically provides a function for the viewer user to purchase gifts based on information on the unit purchase price set for various gifts stored in the gift data 134, a processing function for the viewer user to give a gift purchased using the gift object data included in the gift data 134 to the performer user (including processing for the presentation when giving the gift), and a processing function for changing the owner of a gift given from the viewer user to the performer user.

[0041] The performer user management program 130 provides a management function for information relating to performer users, an authentication function, and the like, based on performer user data 131 shown in FIG. 8, which is stored in the storage 103 together with the performer user management program 130 .

[0042] The performer user data 131 used in the present embodiment 1 is illustrated in Fig. 8. The performer user data 131 stores various information, such as the performer user's account (email address), name, authentication information, file name of avatar data of an avatar used in the virtual space, and file name of an item list in which owned items are registered, in association with a performer user ID uniquely assigned to the performer user. Although not illustrated in Fig. 8, information on virtual value such as points owned by each performer user may also be stored as the performer user data 131.

[0043] Performer users can be authenticated by matching their performer user ID, account, and authentication information, and each performer user can identify from an item list the items (gifts) that they own as a result of being given to them by viewers.

[0044] The viewer / user management program 132 provides functions such as management of information relating to viewer / users and authentication based on viewer / user data 133 shown in FIG. 9 stored in the storage 103 together with the program.

[0045] The viewer user data 133 used in the present embodiment 1 is illustrated in Fig. 9. The viewer user data 133 stores various information, such as the viewer user's account (email address), authentication information, the file name of the avatar data of the avatar to be used in the virtual space, the owned points which are the number of points that can be used in the virtual space, the file name of the item list in which the items (gifts) purchased using the points are registered, and personal information such as the name, date of birth, and telephone number, in association with the viewer user ID uniquely assigned to the viewer user. Although not shown in Fig. 9, a nickname (comment name) to be displayed together with the comment is also registered in the viewer user data 133, and the nickname (comment name) is displayed together with the comment. In addition, points can be increased by, for example, purchasing points from a predetermined operating company.

[0046] <Performer terminal> Fig. 4 is a diagram showing an example of the configuration of a performer terminal 200 in the virtual space content delivery system of this embodiment 1. In this embodiment 1, as shown in Fig. 11, the performer terminal 200 is installed in a control room adjacent to a studio where a performer user performs his / her acting motion, and uses a normal computer with relatively excellent processing power, and as shown in Fig. 4, has a processor (CPU) 201 connected to a data bus 208, a memory (RAM) 202, a storage 203 such as a hard disk, a communication interface (I / F) 204 capable of two-way communication by the TCP / IP protocol via the Internet, which is an open computer network, an image processing unit 206 including a graphics processing unit (GPU) to which display devices A210 to C212 are connected, and an input / output interface (I / F) 205 to which various input / output devices are connected.

[0047] The input / output interface (I / F) 205 is connected with input devices such as a motion sensor 220 including multiple wearable sensors 220C1-220C5 (see FIG. 11) worn by the performer user on the left and right limbs and waist, a facial expression input device 221 consisting of an imaging camera for inputting the performer's facial expressions, a voice input device 222 consisting of a sound collecting microphone worn on the performer user's head, and an operation input device 224 such as a keyboard or touch panel operable by an assistant operator O who assists the performer user. Note that by providing a controller that can be held by the performer user to perform various operations as these input devices, the performer himself may be able to perform various operations without the assistance of the assistant operator O.

[0048] In this embodiment 1, in order for the performer user to perform performance actions in accordance with the music (including musical actions such as singing and dancing), an assistant operator O assists with the operations during the performance. However, it is also possible to have multiple assistant operators O, or to set the operation content in advance in the performer terminal 200, for example, in a sequence program, thereby eliminating the need for an assistant operator O.

[0049] An audio output device 223 including high-performance earphones (in-ear monitors) worn by the performer user and speakers placed in the control room is connected to the input / output interface (I / F) 205 as an output device, so that the sound of the played music transmitted from the distribution server computer 100 is output to the performer user via the high-performance earphones (in-ear monitors), and the music audio is output from the speakers along with the performer's voice, allowing an assistant operator O, etc. to check the status of the music, including the voice produced by the performer user, in the control room.

[0050] In this embodiment 1, high-performance earphones (in-ear monitors) are used to avoid the inconvenience caused by the played music audio being input from the sound collection microphone worn by the performer user, but the present invention is not limited to this, and the music audio may be output from speakers in the studio, etc., as long as the inconvenience caused by the input of the music audio can be avoided.

[0051] In the first embodiment, the sound collecting microphones are worn by the performers, but these sound collecting microphones may be installed on the floor, wall, or ceiling of the studio.

[0052] Here, the motion sensor 220 used in the performer terminal 200 of the present embodiment 1 will be described. Any motion sensor can be used as long as it can properly detect (measure) the body movements (movements) of the performer user. In the present embodiment 1, in order to enable more accurate and short-period detection of the performer's movements, a number of wearable sensors 220C1 to 220C5 are used that are worn by the performer user on the body. Note that the motion sensor 220 may be of a type that does not require the performer to wear any equipment, such as LiDAR that uses laser light or the like.

[0053] In this embodiment 1, in order to reduce the burden of movement, particularly the head movement, caused by wearing sensors etc. when the performer user performs a performance movement, the head movement of the performer user is detected by image recognition using images captured by the imaging camera constituting the facial expression input device 221 as described below, and five wearable sensors 220C1 to 220C5 are used to detect movements other than the head. However, for example, wearable sensors may also be provided on the performer user's head, etc., or more wearable sensors (7 or more) may be worn to detect more detailed movements.

[0054] As shown in FIG. 11, the mounting sensors 220C1 to 220C5 detect their own positions and orientations in cooperation with base stations 220a and 220b installed in a studio room separated from the adjacent control room by a glass window.

[0055] Base station 220a and base station 220b may use, for example, a multi-axis laser emitter. Base station 220a may emit a flashing light for synchronization, and then scan the laser light around, for example, a vertical axis. Base station 220b may scan the laser light around, for example, a horizontal axis.

[0056] Each of the mounting sensors 220C1 to 220C5 may include a plurality of optical sensors that detect the incidence of blinking light and laser light from the base station 220a and the base station 220b.

[0057] Each of the wearing sensors 220C1 to 220C5 is capable of detecting its own position and orientation based on the time difference between the timing of the incidence of the flashing light and the timing of the incidence of the laser light, the light reception time at each optical sensor, the incidence angle of the laser light detected by each optical sensor, and other information as necessary. For example, the ViveTracker or base station provided by HTC CORPORATION can be suitably used.

[0058] Base station 220a and base station 220b emit blinking light and perform laser light scanning at regular intervals, so that the detection information of each of worn sensors 220C1-220C5 is updated at each interval. Detection information indicating the position and orientation of each motion sensor calculated in each of these worn sensors 220C1-220C5 is input to performer terminal 200 by short-range wireless communication and transmitted to distribution server computer 100 as performer avatar information together with facial motion information (see FIG. 10).

[0059] In this embodiment 1, an example is shown in which two base stations, base station 220a and base station 220b, but the present invention is not limited to this, and the number of base stations may be three or more.

[0060] In addition, the motion sensor 220 may be configured such that each of multiple motion sensors worn by the performer user is equipped with a number of infrared LEDs or visible light LEDs, and the position and orientation of each of the motion sensors is detected by detecting the light from these LEDs with an infrared camera installed on the floor or wall of the studio.

[0061] Also, in order to reduce the burden on the performer user, the motion sensor 220 may be replaced with a motion sensor using a lighter reflective marker instead of the wearable sensors 220C1-220C5. That is, the motion sensor may detect the motion of the performer user by capturing images of reflective markers attached to various parts of the performer user's body with adhesive tape or the like to generate image data, and then processing the captured image data to detect the positions and orientations of the reflective markers. In this case, a cameraman who captures the performer user may be placed in the studio to capture the performer user's motion from a direction suitable for detection, so that the motion of the performer user can be detected well.

[0062] In addition, the motion sensor 220 may be a suit with a built-in inertial sensor, such as a suit with a built-in inertial sensor for the MVN motion capture system marketed by Xsens.com, and the movements of the performer user may be detected by analyzing the sensor signal output from the inertial sensor.

[0063] Any facial expression input device 221 can be used as long as it can appropriately detect the head movement including the face of the performer user. In this embodiment 1, specifically, it is configured with an imaging camera (digital camera) arranged so as to be able to continuously capture head images including the face of the performer user, and the head movement and facial movement (facial expression) of the performer user captured by the imaging camera are detected by image recognition, and the facial motion information, which is the detected head movement and facial movement (facial expression), is transmitted to the delivery server computer 100 as performer avatar information together with body motion information, as shown in Fig. 10.

[0064] The facial expression input device 221 is not limited to the imaging camera (digital camera) used in the present embodiment 1, and may be, for example, a 3D camera capable of detecting the depth of a person's face, or may be a mobile terminal such as a smartphone equipped with a LiDER device. In this case, the performer user may wear such a mobile terminal.

[0065] As described above, the performer terminal 200 of this Example 1 has three display devices A210 to C212, where display device A210 is an LCD monitor or the like installed in a control room as shown in FIG. 11, display device B211 is a projector that projects an image onto a screen S installed in a studio, and display device C212 is a large vertical display installed adjacent to the screen S in the studio.

[0066] Display device A210, which is placed in the adjustment room of the studio, displays images from each viewpoint of virtual cameras C1 to C4, viewpoint images from a virtual camera moving in the air during the flying performance period described below, the contents of comments, and a message input window into which assistant operator O inputs any message he or she wishes to convey to the performer user.

[0067] Meanwhile, a screen S projected from a projector, which is a display device B211 installed in the studio, displays an image from the viewpoint of a virtual camera C2 (see FIG. 12), which is the viewpoint of the performer, for example, as shown in FIG. 15, including avatars of viewers who are virtually participating in the live performance in the third and fourth areas, thereby enabling the performer user to check the status of the viewer users who are virtually participating in the virtual live performance through the image.

[0068] In addition, comments from viewer users, as well as comments and messages entered by the assistant operator O, are displayed on a display device C212 installed in the studio, allowing the performer user to check the comments from viewer users and messages from the assistant operator O even during the performance.

[0069] In addition to an operating system (OS) for operating the computer that is the performer terminal 200, the storage 203 also stores performer programs that, when executed by the processor (CPU) 201 etc., work in cooperation with the distribution server computer 100 to provide various functions, including the function of allowing the performer user to control the movement of the performer avatar 1 in a virtual live performance, which is virtual space content.

[0070] As shown in FIG. 4, the performer programs include a performer avatar control program, a performer viewpoint image control program, a performer viewpoint image control program, a performer voice control program, a performer comment control program, performer authentication data, etc.

[0071] The performer avatar control program is a program that mainly provides the function of scanning the body motion of the performer user using the motion sensor 220 and scanning the facial motion (facial expression) of the performer user using the facial expression input device 221, generating performer avatar information for operating the performer avatar, and transmitting it to the distribution server computer 100.

[0072] The performer's viewpoint image control program is a program that provides a function for generating and outputting a viewpoint image of virtual camera C2, which is the performer's viewpoint, based on the virtual space update data distributed from distribution server computer 100.

[0073] The performer's audio control program is a program that provides the function of outputting the playback sound of a song based on the playback music data distributed from the distribution server computer 100 via the audio output device 223, converting the audio input from the audio input device 222 into data and transmitting it to the distribution server computer 100 as performer audio data, and generating and outputting monitor audio consisting of the playback sound of the song and the performer's audio.

[0074] The performer's comment control program is a program that provides a function for displaying each comment by an audience user that is distributed from the distribution server computer 100 .

[0075] The performer authentication data is data for authenticating the identity of the performer user in a communication connection with the distribution server computer 100 .

[0076] In addition, the performer avatar control program includes a motion parameter generation program capable of generating parameter information for the positions and rotation angles of the bones that make up the rig data (sometimes called "skeleton data") that indicates the skeleton of the performer avatar contained in the performer avatar data, as well as an image recognition processing program for image-recognizing the position of the performer user's head and facial expression from images captured by an imaging camera.

[0077] In addition, in this embodiment 1, the performer terminal 200 is exemplified as a form in which a computer serving as the performer terminal 200 installed in a studio operated by an event management organization is jointly used by multiple performer users to hold a live event, and each performer user can use it as their own performer terminal 200 by inputting performer authentication data. However, the present invention is not limited to this, and for example, when a performer user appears in a virtual live performance from their home, etc., the performer terminal 200 may be a computer installed in the performer user's home and used exclusively by the performer user, and in such cases, storage of performer authentication data is not necessarily required.

[0078] In this way, by using the performer terminal 200 constituting the virtual space content delivery system of the present embodiment 1, the performer user or the like can move the performer avatar 1 placed on the virtual stage G in the virtual live venue in conjunction with the performer user's own movements, and can reflect the performer user's own facial expression in the facial expression of the performer avatar 1. In addition, the performer user can deliver his or her own voice as the voice of the performer avatar 1 to the viewers via the delivery server computer 100.

[0079] <Viewer device> Fig. 5 is a diagram showing a configuration example of a viewer terminal 300 in the virtual space content delivery system of the present embodiment 1. In the present embodiment 1, the viewer terminal 300 is a smartphone P carried by a viewer user or a regular desktop computer (PC) installed at a viewer user's home or the like, and as shown in Fig. 5, has a processor (CPU) 301 connected to a data bus 308, a memory (RAM) 302, a storage 303 such as a hard disk or non-volatile memory, a communication interface (I / F) 304 capable of two-way communication by the TCP / IP protocol via the Internet, which is an open computer network, an image processing unit 306 including a graphics processing unit (GPU) to which a display device 310 is connected, and an input / output interface (I / F) 305 to which various input / output devices are connected.

[0080] The display device 310 may display each viewpoint video generated based on the virtual space data of the virtual live venue, which is a virtual space updated by the virtual space update data (see FIG. 10) distributed from the distribution server computer 100, by the viewer viewpoint video control program included in the viewer program stored in the storage 303, as described later, and the flying viewpoint video distributed from the distribution server computer 100 during the flying production period described later. If the viewer terminal 300 is a smartphone P, the display device 310 corresponds to the display device of the smartphone P, and if the viewer terminal 300 is a normal computer (PC), the display device 310 corresponds to the display device of the stationary type constituting the computer (PC). Note that these displays are not limited to those that display planar images (2D images), and may be, for example, those that can display stereoscopic images (3D images) by displaying right-eye images and left-eye images, such as a head-mounted display (HMD) that can be worn by the viewer.

[0081] The input / output interface (I / F) 305 is connected to an operation input device 321 consisting of one or more devices that allow the viewer to perform various operations as input devices, and an audio output device 322 such as earphones or speakers that can output live audio, etc., distributed from the distribution server computer 100.

[0082] It goes without saying that the audio output device 322 need not be an independent device, but may be integrated with the display device 310 like the above-mentioned head mounted display (HMD).

[0083] The operation input device 321 may be any device that allows the viewer to perform various operations such as motion operations including moving the viewer avatar, zoom operations, operations related to comments, operations related to gifts, etc., and may be composed of at least one of, for example, a transparent touch panel formed on the display surface of a smartphone, a keyboard constituting a computer, a game controller, etc.

[0084] In this embodiment 1, as described below, the virtual audience area in the virtual live venue is divided into a third area near the virtual stage G in which only viewer users who have been specially permitted by the performer user can virtually participate, and a fourth area around the third area in which general viewer users can virtually participate, as shown in Figure 12. As described above, a viewer user who virtually participates in the fourth area may have an operation input device 321 such as a touch panel, keyboard, game controller, etc., but a viewer user who virtually participates in the third area owns an operation input device 321 such as an MVN motion capture suit with the above-mentioned inertial sensor built in, and is a viewer user who can precisely move the viewer avatar with their own movements, just like a performer user.

[0085] However, even viewer users who are permitted to virtually participate in the third area cannot change the facial expression of the viewer avatar, but the present invention is not limited to this, and the facial expressions of these viewer avatars may be changed, for example, as described above, by the viewer user using the camera function of a smartphone or a camera connected to a computer to recognize the facial expression of the viewer user, even if it is to a different level than the level of change made by the performer user.

[0086] The various input / output devices connected to the input / output interface (I / F) 305 may be built-in or external, and in the case of external devices, the connection form may be either wired or wireless.

[0087] In addition to an operating system (OS) for operating the viewer terminal 300, which is a smartphone or computer, the storage 303 also stores a viewer program as a viewing application that, when executed by a processor (CPU) 301 or the like, works in cooperation with the distribution server computer 100 to provide various functions, including a live viewing function for viewer users in a virtual live, which is virtual space content.

[0088] As shown in FIG. 5, the viewer program includes a viewer viewpoint image control program which provides the function of generating each viewpoint image of the virtual live to be displayed on the display device 310, a viewer audio control program which provides the function of outputting live audio from the audio output device 322 based on the live audio (data) of the virtual live distributed from the distribution server computer 100, a viewer comment control program which displays comments based on comment data distributed from the distribution server computer 100 and provides functions related to comments such as comments entered by the viewer and sending them to the distribution server computer 100, and viewer authentication data for authenticating that the viewer user is the correct person in a communication connection with the distribution server computer 100.

[0089] In addition, although not shown in Figure 5, in addition to the above-mentioned programs, there are included a gift control program that performs control related to gifts, and an area designation program for reserving an area (position) in which a viewer avatar will virtually participate in the virtual live venue in advance before the start of a live event, as shown in variant example 3-1 (see Figures 42 and 43) described below, but other programs other than these may also be included.

[0090] In the first embodiment, the comments input by the viewer are short messages in text format, but are not limited to these short messages in text format, and may be, for example, still image messages, video messages, or any other electronic message format. The comments are displayed on the display device 310, superimposed on each viewpoint video.

[0091] In this way, by using the viewer terminal 300 that constitutes the virtual space content distribution system of this Example 1, a viewer user can move and operate the viewer avatar that is virtually participating in the virtual live venue, and by changing the viewpoint of the viewer avatar, the viewpoint image displayed on the display device 310 can be changed, and the zoom state (viewpoint state) can also be changed, allowing the viewer user to operate the viewer avatar to cheer on the performer avatar 1.

[0092] It should be noted that a viewer avatar who has virtually participated in the third area is only allowed to move within the third area, and cannot move to the fourth area. On the other hand, a viewer avatar who has virtually participated in the fourth area is only allowed to move within the fourth area, and cannot move to the third area. However, the present invention is not limited to this, and a viewer avatar who has virtually participated in three areas may be allowed to move to the fourth area. It should be noted that a viewer avatar cannot naturally move to the virtual stage G, etc., so the visibility of the performer avatar 1 is not obstructed by the viewer avatar moving onto the virtual stage G.

[0093] In addition, by displaying viewpoint images on these display devices 310 and outputting live audio, the user can enjoy the virtual live performance with the feeling that he or she is actually present at the virtual live performance venue.

[0094] Although detailed explanation is omitted, the comment function provided by the viewer comment control program allows viewers to enjoy the virtual live performance while checking comments entered by themselves and other viewer users, and the gift function provided by the gift control program allows viewers to liven up the virtual live performance by gifting items they own to the performer avatar 1.

[0095] <Administrator terminal> Fig. 6 is a diagram showing an example of the configuration of an administrator terminal 150 in the virtual space content delivery system of this embodiment 1. In this embodiment 1, the administrator terminal 150 uses a normal computer installed in a control room adjacent to the studio together with the performer terminal 200, and as shown in Fig. 6, has a processor (CPU) 151 connected to a data bus 158, a memory (RAM) 152, a storage 153 such as a hard disk, a communication interface (I / F) 154 capable of two-way communication by the TCP / IP protocol via the Internet, which is an open computer network, an image processing unit 156 including a graphics processing unit (GPU) to which a display device 160 is connected, and an input / output interface (I / F) 155 to which various input / output devices are connected.

[0096] As described below, the display device 160 is capable of individually displaying each viewpoint image of each of the virtual cameras C1 to C4 described below and each viewpoint image of the virtual camera moving in the air during the flying performance period, which are generated based on virtual space data of the virtual live venue, which is a virtual space updated by virtual space update data (see Figure 10) distributed from the distribution server computer 100, by an administrator viewpoint image control program included in the administrator program stored in storage 153, and is also capable of displaying comments based on comment data distributed from the distribution server computer 100, and is composed of one or more stationary displays.

[0097] The input / output interface (I / F) 155 is connected to an operation input device 161 consisting of one or more devices that enable the event manager M to perform various operations as input devices, and an audio output device 162 such as earphones, headphones, or speakers that can output live audio, etc., distributed from the distribution server computer 100.

[0098] The operation input device 161 can be any device that allows the event manager M to perform various operations such as input and settings related to the virtual live event, and may be composed of at least one of, for example, a keyboard that constitutes a computer, a transparent touch panel formed on the surface of a display, a viewpoint switching controller, etc.

[0099] The various input / output devices connected to the input / output interface (I / F) 155 may be connected in either a wired or wireless manner.

[0100] In addition to an operating system (OS) for operating the computer that is the administrator terminal 150, the storage 153 also stores an administrator program that is executed by a processor (CPU) 151, etc. to provide various functions in cooperation with the distribution server computer 100, including distribution management functions for the distribution of virtual live shows, which are virtual space content.

[0101] As shown in FIG. 6, the administrator programs include an administrator viewpoint video control program, an administrator audio control program, an administrator comment control program, an audience management program, a performer management program, and the like.

[0102] The viewpoint video control program for administrator is a program that provides a function for generating viewpoint videos from each virtual camera to be displayed on the display device 160, as well as a function for changing and switching each viewpoint.

[0103] The administrator audio control program is a program that provides a function for outputting live audio from the audio output device 162 based on the audio data of the virtual live event distributed from the distribution server computer 100 .

[0104] The administrator's comment control program is a program that displays comments based on comment data distributed from the distribution server computer 100, and provides comment-related functions such as prohibiting comments from being distributed and selecting viewers to prevent them from being distributed.

[0105] The viewer management program is a program for managing viewer users who virtually participate in the virtual live performance, and the performer management program is a program for managing performer users who virtually participate in the virtual live performance.

[0106] Although not shown in FIG. 6, the system includes an authentication program for authenticating whether the administrator who is the operator is the actual person, and a schedule program for editing the virtual live show time schedule and song order, etc., but may also include other programs.

[0107] In this way, by the event manager M operating the manager terminal 150 that constitutes the virtual space content distribution system of this embodiment 1, various settings for executing the event can be implemented, such as settings related to the program, order of songs, and performance, as well as the time schedule and the trajectory of the virtual camera moving in the air during the flying performance period, and the virtual live performance can be managed based on the settings made in this way.

[0108] <Virtual live venue> The virtual live venue, which is the virtual space used in Example 1, is shown in Fig. 12. As shown in Fig. 12, the virtual live venue, like a live venue in real space, is a virtual space having a virtual stage G where performer users appear as performer avatars 1 and a virtual audience area where viewer users who will be in the audience virtually participate as viewer avatars.

[0109] 12, virtual stage G has a relatively large trapezoidal floor surface, and performer avatar 1 can move on virtual stage G in the same way as in a live venue in real space. On the side of virtual stage G opposite the virtual audience seats, as shown in Fig. 12, a virtual stage wall is formed that is divided into three parts, a center part, a right part, and a left part, and virtual displays 40, 41, and 42 are provided in front of these virtual stage walls, and images and videos for performance are virtually displayed on these virtual displays 40, 41, and 42.

[0110] A star virtual arrangement object 43 related to the performer user is placed between the virtual displays 40, 41, 42, and large star virtual aerial objects 50-52 are placed in a second area, which is an area on the virtual stage G and is above a first area surrounded by the virtual stage walls, as shown in Fig. 12. The virtual aerial objects 50-52 are movable within the second area.

[0111] A virtual audience area where viewer users can virtually participate in the virtual live venue as viewer avatars is provided in front of the virtual stage G. As shown in Fig. 12, this virtual audience area is made up of a third area, which is a special area close to the center of the virtual stage G where the performer avatar 1 appears, and a fourth area, which is formed to surround the third area and is farther away from the center of the virtual stage G than the third area.

[0112] These third areas are available for virtual participation by special viewer users who are permitted to participate virtually, up to a predetermined upper limit. In this embodiment 1, as shown in FIG. 12, for example, viewer avatars 11 to 14 of four viewer users who have met a predetermined virtual participation record and have been permitted to participate virtually by the performer user are arranged.

[0113] In addition, the fourth area includes viewer avatars 21-32 of general viewer users who have reserved virtual participation before the start of the virtual live. Note that, in Fig. 12, the viewer avatars 21-32 are shown in a simplified manner for the sake of convenience, but are the same avatars as the viewer avatars 11-14.

[0114] Furthermore, each viewer avatar 11-14 placed in the third area can be moved within the third area by the viewer user operating the viewer terminal, and each viewer avatar 21-32 placed in the fourth area can be moved within the fourth area by the viewer user operating the viewer terminal.

[0115] Four virtual cameras C1 to C4 are virtually arranged (set) in the virtual live performance venue, as shown in Fig. 12. Virtual camera C1 is virtually arranged in front of performer avatar 1 so as to face the performer avatar 1, and the viewpoint image captured by virtual camera C1 is a viewpoint image viewed from a position close to the front of the performer avatar 1, as shown in Fig. 13(a).

[0116] Virtual camera C2 is a camera virtually placed (set) above the head of performer avatar 1, and the viewpoint image captured by virtual camera C2 is a performer viewpoint image captured by viewing the virtual audience area from performer avatar 1, as shown in FIG.

[0117] Virtual camera C3 is a virtual camera virtually placed on the virtual display 41 diagonally behind the performer avatar 1, and the viewpoint image from virtual camera C3 is a viewpoint image looking at the virtual audience area from diagonally behind the performer avatar 1, as shown in Figure 13(c).

[0118] Virtual camera C4 is a virtual camera virtually placed on the virtual display 42 diagonally behind the performer avatar 1, and the viewpoint image from virtual camera C4 is a viewpoint image looking at the virtual audience area from diagonally behind the performer avatar 1, as shown in Figure 13(b).

[0119] In addition to the fixedly placed (set) virtual cameras C1 to C4, in this embodiment 1, an air-moving virtual camera (not shown) is virtually placed so as to be movable in the air within the virtual live venue space, which is a virtual space. The air trajectories along which these air-moving virtual cameras move and the viewpoint directions (angles) at each position on the air trajectory are preset by the administrator terminal 150, and during the flying performance period described later, the performer avatar 1 virtually flies so as to follow the air trajectory of the air-moving virtual camera, thereby generating a viewpoint video of the virtually flying performer avatar 1 captured by the air-moving virtual camera.

[0120] In this embodiment 1, as shown in FIG. 12, an example is given in which four virtual cameras C1 to C4 are virtually arranged as fixedly arranged (set) virtual cameras, but the present invention is not limited to this, and the number of these fixedly arranged (set) virtual cameras may be five or more, or conversely, without providing fixedly arranged (set) virtual cameras, only the viewpoint images seen from the viewpoints corresponding to each avatar virtually participating in the virtual live venue may be displayed on the performer terminal 200 and the viewer terminal 300.

[0121] In addition, in this embodiment 1, an example is given of a configuration in which there is one virtual camera moving through the air, but the present invention is not limited to this, and a configuration in which there are multiple virtual cameras moving through the air may also be used, or conversely, a configuration in which no virtual camera moving through the air is virtually placed may also be used.

[0122] <Studio> 11 is a diagram showing the studio used in Example 1, and as described above, the studio is adjacent to the control room separated by a glass window, and the performer user performs each movement corresponding to singing in the studio, while the assistant operator O assists with the operation of the performer terminal 200 installed in the control room. Around the performer user wearing the wearable sensors 220C1 to 220C5, base stations 220a and 220b are mounted on a stand, and an imaging camera constituting the facial expression input device 221 is mounted on a stand in a position almost in front of the performer user.

[0123] A screen S is provided on the wall facing the performer user in the studio, and a projector (display device B211) installed on the ceiling of the studio projects the viewpoint image of virtual camera C2 onto screen S, so that the viewpoint image seen from performer avatar 1 as looking at the virtual audience area is displayed on screen S, as shown in Figure 15. This allows the performer user to proceed with the live performance while constantly keeping track of the behavior of the viewer avatars virtually participating in the virtual audience area (the viewer's reaction to the live performance).

[0124] A vertical large display is arranged to the side of the screen S as the display device C212, and comments from the audience and messages that the assistant operator O wants to convey to the performer users are displayed on the large display. The display device C212 may be configured to display information (performer support information) such as the lyrics of the song sung by the performer during the live performance and lines.

[0125] <Operation of virtual space content delivery system> FIG. 10 is an explanatory diagram for explaining the operation of the virtual space content delivery system of the first embodiment, specifically, the flow of delivery of video and audio of a virtual live performance.

[0126] First, the video system will be described. As described above, the distribution server computer 100 is capable of providing a virtual space of a virtual live venue based on the virtual space generation program 120 and the avatar data of the performer users and the viewer users, and the actions of the avatars of the performer users and the viewer users are reflected in the actions of the performer avatar 1 and the viewer avatar 1 actions (operations) of the viewer users are reflected in the actions of the viewer avatar by updating the virtual space data in which the state of the virtual space is described based on the performer avatar information transmitted from the performer terminal 200 and the viewer avatar information transmitted from the viewer terminal 300.

[0127] Although not shown in FIG. 10, when the performer terminal 200 and the viewer terminal 300 communicate with the distribution server computer 100 in order to virtually participate in the virtual live performance, virtual space data of the virtual live performance venue is delivered in advance from the distribution server computer 100 and stored in the performer terminal 200 and the viewer terminal 300. As described below, the virtual space data stored in advance in this manner is successively updated by virtual space update data delivered successively from the distribution server computer 100, allowing the performer terminal 200 and the viewer terminal 300 to identify the latest state of the virtual live performance venue.

[0128] Specifically, the movements and facial expressions of the performer user in the studio are scanned at predetermined time intervals by the body motion scan and face motion scan functions of the performer avatar control program described above, and performer avatar information is generated and transmitted to the distribution server computer 100.

[0129] On the other hand, viewer users who are permitted to virtually participate in the third area use motion sensors, just like the performers, and are scanned at specified intervals using the body motion scan and face motion scan functions, and viewer avatar information is generated and transmitted to the distribution server computer 100.

[0130] Furthermore, a viewer user permitted to virtually participate in the fourth area can use the touch panel, keyboard, or controller of the viewer terminal 300 to perform various operations such as movement, changing the viewpoint direction, changing the viewpoint state (zoom), raising hands, clapping hands, jumping, etc., to make the viewer avatar perform the corresponding action. Then, of these operations, viewer avatar information is generated based on the operation related to the viewer avatar's action and transmitted to the delivery server computer 100.

[0131] Furthermore, viewer users virtually participating in the third area can also use a controller or the like to perform various operations such as movement, changing the viewpoint (zoom), raising their hands, clapping their hands, jumping, etc. In this case, menu items corresponding to each action are displayed, and the menu items corresponding to each action can be selected using a controller or the like.

[0132] In this manner, an avatar action operation process is executed in the viewer terminal 300 to receive operations corresponding to each action to be performed by the viewer avatar. The avatar action operation process also includes processes such as displaying and erasing menu items corresponding to each of the above-mentioned actions.

[0133] In this way, based on the performer avatar information transmitted from the performer terminal 200 and the viewer avatar information transmitted from the viewer terminal 300, the distribution server computer 100 executes a virtual space update process that updates the virtual space data, thereby reflecting the actions (operations) of the performer user and the viewer user in each avatar virtually participating in the virtual live venue.

[0134] Then, after executing a virtual space update data generation process for generating virtual space update data based on the virtual space data before the update and the virtual space data after the update, the generated virtual space update data is distributed to the performer terminal 200 and the viewer terminal 300.

[0135] At the performer terminal 200 and viewer terminal 300 to which the virtual space update data has been delivered, as shown in FIG. 10, the pre-stored virtual space data is updated based on the delivered virtual space update data, and a performer perspective video generation process and a viewer perspective video generation process are executed using the updated virtual space data, thereby generating performer perspective video and viewer perspective video based on the updated virtual space data, and the generated performer perspective video and viewer perspective video are displayed on the performer terminal 200 and the viewer terminal 300 (viewer perspective video output process).

[0136] In this way, in this embodiment 1, the viewpoint video of each viewer user who is virtually participating in the virtual live venue through a viewer avatar is generated and displayed on the viewer terminal 300 of each viewer user, and by generating the viewer avatar viewpoint video of each viewer user on the distribution server computer 100, it is possible to prevent the processing load on the distribution server computer 100 from becoming significantly large, which would result in many viewer users being unable to participate in the virtual live, and it is also possible to prevent distribution from becoming difficult due to an increased processing load.

[0137] In this embodiment 1, a viewpoint image is generated and displayed on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 13 who is virtually participating in the third area close to the performer avatar 1 on the virtual stage G, in which a viewpoint image of almost the front of the performer avatar 1 is displayed large, as shown in Figure 13 (d), which is a viewpoint image seen from the viewer avatar 13 close to the performer avatar 1, while a viewpoint image is generated and displayed on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 28 who is virtually participating in the fourth area far from the performer avatar 1 on the virtual stage G, for example, at a position behind the viewer avatar 13, in which a viewpoint image of almost the front of the performer avatar 1 appears relatively small through the viewer avatar 13 and viewer avatar 14 who are virtually participating in the third area, as shown in Figure 13 (e).

[0138] In addition, the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 32 who is virtually participating at the edge position of the virtual stage G in the fourth area, which is set up as a U-shape when viewed from above, displays a viewpoint image seen from the viewer avatar 32 positioned diagonally forward of the performer avatar 1, as shown in Figure 13 (f), in which the performer avatar 1 appears relatively small through the viewer avatar 14 who is virtually participating in the third area.

[0139] Note that all of the display examples shown in Figure 13 are examples in which the viewer terminal 300 is a stationary computer (PC) and the display device 310 is a stationary display. However, for example, if the viewer terminal 300 is a smartphone P, tablet, etc., the viewpoint image of the virtual camera C1 is displayed as shown in Figure 14(a), and if the viewer terminal 300 is the viewer terminal 300 of the viewer avatar 13, it is displayed as shown in Figure 14(b).

[0140] Next, the audio system will be described. In the distribution server computer 100, music data 113 is played back by audio control program 124 based on time schedule data (TS data) 112, and transmitted to the performer terminal 200 as played music (data) (music playback process).

[0141] In the performer terminal 200, the music audio from the played music (data) transmitted from the distribution server computer 100 is output to the performer user through high-performance earphones (in-ear monitors) worn by the performer user (played music output process), and when the performer user sings along with the output played music, the performer voice input from the sound collection microphone (audio input device 222) is digitized and transmitted to the distribution server computer 100 as the performer voice (data).

[0142] The distribution server computer 100 generates distribution audio (data) from the performer audio (data) received from the performer terminal 200 and the played music played in the music playback process described above (distribution audio generation process), and distributes the generated distribution audio (data) to the viewer terminal 300 (generated audio distribution process).

[0143] In the viewer terminal 300, the distribution audio (data) distributed from the distribution server computer 100 is output from an audio output device 322 such as earphones or speakers, allowing the viewer to hear the live audio as if the performer avatar 1 were singing in the virtual live venue.

[0144] Furthermore, in the performer terminal 200, as described above, the performer's voice (data) is transmitted to the distribution server computer 100, and as shown in Figure 10, a monitor audio generation process and a monitor audio output process are executed, so that the music audio of the played music and the performer's voice are output from speakers placed in the control room, allowing an assistant operator O, etc. to check the singing status of the performer user in the control room.

[0145] <Viewer's viewpoint video control processing> Next, a viewer's viewpoint video control process executed in the viewer terminal 300 based on a viewer's viewpoint video control program included in the viewer's program will be described with reference to FIG.

[0146] In the viewer viewpoint video control process, first, it is determined whether or not the performer avatar 1 is in a flying performance period in which the performer avatar 1 is virtually flying, as shown in Fig. 19 (step S1). Whether or not the performer avatar 1 is in a flying performance period can be identified by performance status data indicating that the performer avatar 1 is in a flying performance period, which is sent from the delivery server computer 100. The delivery server computer 100 transmits performance status data indicating that the performer avatar 1 is in a flying performance period to the viewer terminal 300 when the performer avatar 1 is in a flying performance period based on the time schedule data.

[0147] If it is during the flying performance period (Y in step S1), proceed to step S21, whereas if it is not during the flying performance period (N in step S1), it is determined whether viewpoint-related operations including viewpoint movement operations (including changing the viewpoint by avatar movement operations and motion operations) and zoom operations are disabled (step S2).

[0148] If the viewpoint-related operation is invalid (Y in step S2), the process proceeds to step S7. If the viewpoint-related operation is not invalid (N in step S2), the process determines whether or not there is an operation that changes the viewpoint, such as an operation of moving an avatar or an operation of moving an avatar (step S3), and if there is an operation (Y in step S3), the viewer viewpoint is changed according to the operation, and if there is no operation (N in step S3), the process proceeds to step S5, and determines whether or not there is a zoom operation (step S5).

[0149] If a zoom operation is performed (Y in step S5), the image area displayed on the display device 310 is changed to an image area corresponding to the operation, and if a zoom operation is not performed (N in step S5), it is determined whether or not a motion tracking setting is present (step S7).

[0150] This motion tracking setting can be arbitrarily set by the viewer user on the viewer terminal 300, for example, before the start of a live performance or during a live performance, depending on the type of device used by the viewer user and his / her level of proficiency in the operation. Specifically, the motion tracking setting is set in cases where, for example, the viewer user is not familiar with operating the operation input device 321 such as a touch panel, keyboard, or game controller, or is not familiar with changing the viewpoint by operating the avatar to move or move, or, as will be described later, the virtual participation position of the viewer user's avatar is far from the performer avatar 1 and the viewer user is zoomed in and displayed in an enlarged manner, so that even a slight movement of the performer avatar 1 prevents the performer avatar 1 from being displayed appropriately, and so on, and the viewer user is unable to change the viewpoint in response to the movement or movement of the performer avatar 1.

[0151] If there is no motion following setting (N in step S7), the process proceeds to step S11. If there is a motion following setting (Y in step S7), it is further determined whether or not the motion following condition is satisfied (step S8).

[0152] In this embodiment 1, the motion tracking condition is a condition under which it is almost certain that the performer avatar 1 will no longer be displayed clearly on the display device 310. Specifically, when the head (or face) of the performer avatar 1 is within a specified range around the display area of ​​the display device 310, it is highly likely that the movement or motion of the performer avatar 1 will cause the head (face) to move outside the display area, causing it to no longer be displayed clearly, and therefore it is determined that the motion tracking condition is met.

[0153] In this way, in the first embodiment, by determining that the motion tracking condition is met before the head (face) of the performer avatar 1 disappears from the display area, it is possible to significantly reduce the occurrence of the head (face) of the performer avatar 1 disappearing from the display area, but the present invention is not limited to this, and these motion tracking conditions may be a condition that the head (face) moves out of the display area, or other conditions may be used as the motion tracking conditions. In other words, any conditions may be used as the motion tracking conditions as long as they can prevent the performer avatar 1 from disappearing from the display area in a good manner.

[0154] If the action tracking condition is met (Y in step S8), after disabling the viewpoint-related operations (step S12), the viewpoint image displayed in the display area of ​​display device 310 is automatically changed to a motion tracking viewpoint in which the head (face) of performer avatar 1 is within a non-metable range surrounded by a predetermined range around the display area, and the viewpoint image seen from the motion tracking viewpoint is displayed in the display area of ​​display device 310 (step S13).

[0155] On the other hand, if the action tracking condition is not met (N in step S8), it is further determined whether or not the viewpoint-related operation is disabled (step S9). If the viewpoint-related operation is disabled (Y in step S9), the disabled viewpoint-related operation is enabled (step S10) and then the process proceeds to step S11. If the viewpoint-related operation is not disabled (N in step S9), the process proceeds to step S11 without going through step S10.

[0156] In step 11, since the motion tracking setting is not set or the motion tracking condition is not satisfied, the viewpoint video of the viewer avatar is continuously displayed in the display area of ​​display device 310 (step S11).

[0157] Also, as mentioned above, if the flying performance period in which the performer avatar 1 virtually flies during a live performance has begun, the answer in step S1 is determined to be Y, and the process proceeds to step S21, where it is determined whether or not it is the start timing of the flying performance period (step S21).

[0158] If it is the start timing of the flying performance period (Y in step S21), the flying viewpoint image distributed from the distribution server computer 100 is displayed (step S22) in place of the viewer avatar's viewpoint image or action-tracking viewpoint image that was displayed in the display area of ​​the display device 310 before the start of the flying performance period, and then viewpoint-related operations are disabled (step S23) and the process returns to step S1.

[0159] In this embodiment 1, during the flying performance period, the same flying viewpoint video is displayed on the viewer terminals 300 of all viewer users. Instead of generating the flying viewpoint video on each viewer terminal 300 side, as in the case of viewer avatar viewpoint videos which are different for each viewer user, the flying viewpoint video is generated on the distribution server computer 100 side and distributed to the viewer terminals 300. This allows the flying viewpoint video, which involves movement of the viewpoint in the air and therefore imposes a large rendering load on the viewer terminals 300, to be displayed well even on viewer terminals 300 with low processing capabilities. However, the present invention is not limited to this, and these flying viewpoint videos may be generated on the individual viewer terminals 300 side, in the same way as viewer avatar viewpoint videos.

[0160] On the other hand, if it is not the start timing of the flying effect period (N in step S21), it is further determined whether it is the end timing of the flying effect period (step S24).

[0161] Whether or not it is the end timing of the flying performance period may be determined based on data indicating the end contained in the flying viewpoint video data transmitted from the distribution server computer 100, or may be determined based on the aforementioned performance status data that is different from the flying viewpoint video.

[0162] If it is not the end of the flying performance period (N in step S24), that is, if it is during the flying performance period, proceed to step S26, update the flying viewpoint video, and then return to step 1; on the other hand, if it is the end of the flying performance period (Y in step S24), enable viewpoint-related operations (step S25), proceed to step S11, display the viewer avatar's viewpoint video in the display area of ​​display device 310 in place of the flying viewpoint video that was displayed in the display area of ​​display device 310 during the flying performance period, and then return to step S1.

[0163] Thus, each viewpoint video displayed on the viewer terminal 300 by executing the viewer viewpoint video control process shown in FIG. 20 in the viewer terminal 300 of the first embodiment will be described with reference to FIGS. 16 to 19. FIG.

[0164] Fig. 16 is a diagram showing the display contents of the display device 310 when a zoom operation is performed on the viewer terminal 300 of the present embodiment 1. In the default state where no zoom operation is performed, as shown in Fig. 16(a), a viewpoint video of the video area centered on the upper body of the performer avatar 1 is displayed. In this default state, when the viewer user performs a zoom-up operation, for example, when the viewer terminal 300 is a smartphone P, by touching the "+" display (not shown), or when the viewer terminal 300 is a computer (PC), by operating the "+" key on the keyboard, as shown in Fig. 16(b), for example, a viewpoint video of the video area centered on the head (face) of the performer avatar 1 is displayed.

[0165] On the other hand, if the viewer user performs a zoom-down operation, for example, by touching the "- " display (not shown) if the viewer terminal 300 is a smartphone P, or by operating the "- " key on the keyboard if the viewer terminal 300 is a computer (PC), a viewpoint image of the image area including the entire body of the performer avatar 1 will be displayed, as shown in Figure 16 (c), for example.

[0166] In Figure 16, in order to make the zoom function easier to understand, the explanation was given using the viewpoint image of virtual camera C1 rather than the viewpoint of the viewer avatar, but as shown in Figure 17, zooming up and down can also be performed in the same way on the viewpoint of the viewer avatar.

[0167] Also, although not described in detail in the viewer viewpoint image control process of Figure 20, for example, when performer avatar 1 appears on virtual stage G for the first time when a live performance begins, the viewpoint of the viewer avatar will not be directed toward performer avatar 1, and there is a possibility that the appearing performer avatar 1 will not be displayed on the viewer terminal 300 for a long period of time. Therefore, the period when performer avatar 1 first appears may be designated as an appearance performance period, and during this performance period, the viewpoint image displayed on the viewer terminal 300 may be automatically switched to the viewpoint image of virtual camera C1, etc., instead of the viewpoint image of the viewer avatar, and individual viewer users may be allowed to zoom in and perform other operations on the viewpoint images of virtual camera C1, etc.

[0168] 17 shows, as an example of a viewpoint image of a viewer avatar, the viewpoint image of viewer avatar 28 who is virtually participating in the fourth area. As shown in Fig. 17, the viewpoint image of viewer avatar 28 includes the heads of viewer avatar 13 and viewer avatar 14 who are virtually participating in the third area, and the image shows performer avatar 1 seen through the heads of viewer avatar 13 and viewer avatar 14.

[0169] Therefore, the viewer user of viewer avatar 28 can enjoy the live performance by, for example, performing a zoom-in operation to change the image area so that the upper body of performer avatar 1 is displayed primarily, as shown in Figure 17(b), thereby preventing viewer avatar 13 and viewer avatar 14 from being displayed.

[0170] However, in the zoomed-in state as shown in Figure 17(b), for example, as the live performance progresses and the performer avatar 1 begins to move and make large movements, if the viewer user is unable to properly change the viewpoint of the viewer avatar 28 to match the movements of the performer avatar 1, the performer avatar 1 may not be displayed properly, as shown in Figures 18(d)-(f).If the viewer user has to pay attention to adjusting the viewpoint, it may become difficult to concentrate on the live performance and they may not be able to enjoy it well.

[0171] However, in this Example 1, when the viewer user sets the motion tracking setting as described above, as shown in Figures 18(a) to (c), the head of the performer avatar 1 enters within a specified peripheral area of ​​the display area, the motion tracking condition is met, the viewpoint is automatically changed to the motion tracking viewpoint, and the viewpoint image of the motion tracking viewpoint is displayed. Therefore, the viewer user does not need to pay attention to operating the viewpoint, and can concentrate on the live performance and enjoy it to the fullest.

[0172] Then, as the live performance progresses and the flying performance period begins, as shown in Figure 19, the viewpoint images of the individual viewer avatars are automatically changed to flying viewpoint images seen from a virtual camera moving in the air, without any operation by the viewer user.

[0173] Specifically, before the performer avatar 1 takes off in virtual flight, an image is displayed in which a wing object grows on the back of the performer avatar 1. Then, the performer avatar 1 looks up at a virtual aerial star object 50 virtually placed in the second area of ​​the virtual live venue, and the viewpoint of the viewer avatar moves in the same way. At this time, light effects 61 appear around the performer avatar 1.

[0174] Thereafter, when the performer avatar 1 enters virtual flight, the viewpoint image displayed on the display device 310 automatically switches from the viewpoint image of the viewer avatar in FIG. 19(b) to a flying viewpoint image seen from a virtual camera moving in the air, as shown in FIG. 19(c).

[0175] During the flying performance period, the aerial moving virtual cameras move within the second area based on a pre-set aerial trajectory as described above, and the flying viewpoint images seen from these aerial moving virtual cameras moving in the air are displayed on the display device 310 as shown in Figure 19(d).

[0176] <Modification of Example 1> In the above-mentioned first embodiment, the general action (first action) is an action of the performer avatar 1 standing without moving to the virtual stage G, the special action (second action) is an action of the performer avatar 1 flying virtually, and the specific action (third action) is an action of the performer avatar 1 moving rapidly in the lateral direction. However, the present invention is not limited to this, and the actions may be actions according to the content to be distributed. For example, the general action (first action) may be an action (normal action) that the performer avatar performs for the longest period in the distributed content, the special action (second action) may be an action that is performed in a special period that is the shortest period in the distributed content, or an action that is performed least frequently, and the specific action (third action) may be an action that is performed for a shorter period than the general action (first action) but for a longer period than the special action, or an operation that is performed less frequently than the general action (first action) but more frequently than the special action. Note that the special action (second action) may include the specific action (third action).

[0177] In addition, in the above-mentioned Example 1, an example was given of a form having both a function of switching the viewpoint corresponding to a special action (second action) and a function of following a specific action (third action), but the present invention is not limited to this, and the virtual space content distribution system may have only one of these functions.

[0178] In the above-mentioned Example 1, an example was given of a form in which the viewpoint of the viewer user is changed and followed in accordance with the movements of the performer avatar, but the present invention is not limited to this, and for example, the change and tracking of the viewpoint may be performed in accordance with specific effects such as smoke or fireworks (pillars of fire) devices, special lighting, deformation of the virtual stage G, etc., which are effects on the virtual stage G. EXAMPLES

[0179] Next, a virtual space content delivery system according to a second embodiment will be described below with reference to Fig. 21 to Fig. 28. Note that the same components as those in the first embodiment will be denoted by the same reference numerals and the description thereof will be omitted.

[0180] The configuration of the virtual space content distribution system of the second embodiment is the same as that of the first embodiment shown in FIG. 1. However, the progress of the virtual live performance is different, and therefore the period during which the viewpoint images from the virtual cameras C1 to C4 and the airborne moving virtual camera (hereinafter collectively referred to as virtual camera viewpoint images) are distributed and the entity that generates these virtual camera viewpoint images are different from those of the first embodiment, which is a characteristic feature of the second embodiment.

[0181] 21 is a diagram showing the progress of a virtual live performance delivered in a virtual space content delivery system of Example 2. In the virtual live performance of Example 2, a "live virtual space participation period" and a "pre-performance speech" period are set as periods before the start of the virtual live performance, and during the live virtual space participation period, viewer users can virtually participate in the virtual live performance venue, which is the target virtual space of the virtual live performance.

[0182] The period up to the timing for checking whether the pre-show download has been completed, which is set just before the end of the "pre-show speech" period, is the pre-show data download (DL) period, during which the virtual space data (basic data) of the virtual live venue, avatar data of performer avatar 1, and avatar data of other viewer users who have participated virtually are downloaded (DL) as pre-show data from the distribution server computer 100 to the viewer terminals 300 of viewer users who have participated virtually during the period during which they are able to participate in the live virtual space.

[0183] At the advance data DL completion check timing, which is the end timing of the advance data download (DL) period, it is checked whether the advance data download (DL) has been completed in each viewer terminal 300 or not.

[0184] If the download (DL) of the advance data is not completed by the time of the advance data DL completion check timing due to differences in the model, processing power, communication environment, etc. of the viewer terminal 300, the download (DL) of the advance data will continue. However, in this case, data that is highly important for the distribution of the virtual live, such as the virtual space data (basic data) of the virtual live venue required for the opening performance and the first MC, and the avatar data of the performer avatar 1, will be downloaded (DL) with priority.

[0185] Furthermore, in the second embodiment, as described above, an example is given of the data required to generate viewpoint video at the viewer terminal 300 during the distribution of a virtual live performance being downloaded (DL) as advance data, but the present invention is not limited to this, and these data, in particular special data required individually for each performance and each MC described below, may be downloaded (DL) sequentially each time at a timing corresponding to the timing at which each performance and each MC is distributed.

[0186] During the "pre-show speech" period, explanations about the virtual live show are given by video footage, etc. Therefore, by making the "pre-show speech" period a period during which advance data can be downloaded (DL), each viewer terminal 300 can also download (DL) from the distribution server computer 100 the avatar data of viewer avatars who virtually participated in the live virtual space just before the end of the participation period.

[0187] As shown in FIG. 21, the virtual live performance of Example 2 progresses as follows: "Opening performance" → "First MC" → "First performance" → "Second MC" → "Second performance" ... "Opening performance" is a performance in which the performer avatar 1 appears on the virtual stage G. In the "First MC", the performer avatar 1 appears on the virtual stage G, greets the audience through a talk, and introduces the first performance that will be performed first. The "First performance" is a performance that includes singing and dancing by the performer avatar 1 (performer), and also includes a flying performance in the middle of the performance, in which the performer avatar 1 virtually flies through the virtual live venue and is virtually captured by a virtual camera moving in the air. In the "Second MC", the performer avatar 1 (performer) mainly introduces the second performance through a talk. The "second performance" includes singing and dancing by the performer avatar 1 (performer), and also includes a costume change performance using footage virtually captured by the virtual camera C1 of the performer avatar 1 changing clothes, and a clone performance using footage virtually captured by the virtual camera C1 of the performer avatar 1 cloned into multiple people.

[0188] The processing load on the delivery server computer 100 during each period in which these effects and programs are performed is the lowest at "low" during the period in which participation in the live virtual space is possible, while it is the highest at "high" during the period in which the special effects "flying effect", "costume change effect", and "double-ego effect" are performed. The processing load on the delivery server computer 100 during the period of the "first performance" from the "introduction", "opening effect", "first MC", and "flying effect" is "medium", and the processing load is also "medium" during the period of the "first performance" after the "flying effect", the period of the "second performance" until the "second MC" and "costume change effect", the period of the "second performance" from the "costume change effect" to the "double-ego effect" is also "medium", and the period of the "second performance" after the "double-ego effect" is also "medium".

[0189] Next, regarding each period in the virtual live in Example 2, we will explain the viewpoint images that can be viewed on the viewer terminal 300. During the period during which participation in the live virtual space is possible, from the time of virtual participation until the start of the "pre-show speech," only the viewer avatar viewpoint images can be displayed.

[0190] Then, when the "pre-show speech" starts, only the virtual camera viewpoint video, specifically, the viewpoint video of virtual camera C1, can be displayed. In this way, the viewpoint video of the virtual camera is the only one used in the "pre-show speech" even though it is before the start of the virtual live performance. Since the pre-show speech includes an explanation of precautions regarding the virtual live performance, the purpose is to prevent a viewer user from not viewing the pre-show speech being performed on virtual stage G due to the viewer avatar viewpoint video being displayed on viewer terminal 300.

[0191] As described below, these virtual camera viewpoint images of the "introduction" are generated by the distribution server computer 100 and distributed to each viewer terminal 300, so that they can be displayed even on viewer terminals 300 that have not yet completed downloading the advance data.

[0192] For viewer terminals 300 for which the pre-data download has been completed at the above-mentioned pre-data download completion check timing, the viewer avatar viewpoint image can be displayed in addition to the virtual camera viewpoint image during the period of the "first performance" from the "first MC" to the "flying performance", as well as during the period after the "flying performance".

[0193] On the other hand, for viewer terminals 300 that have not completed downloading the advance data at the above-mentioned advance data download completion check timing, the viewer avatar viewpoint video cannot be generated accurately, so the virtual camera viewpoint video generated in the distribution server computer 100 is distributed, making it possible to display only the virtual camera viewpoint video during the entire virtual live broadcast, and it is not possible to display the viewer avatar viewpoint video.

[0194] Furthermore, among the images of the "Flying Performance," "Costume Change Performance," and "Double Ego Performance," which have a "high" processing load on the distribution server computer 100, the virtual camera viewpoint images of virtual camera C1 for the "Costume Change Performance" and "Double Ego Performance," as well as the virtual camera viewpoint images during the second performance period other than these "Costume Change Performance" and "Double Ego Performance," are generated on each viewer terminal 300 rather than on the distribution server computer 100, thereby making it possible to reduce the processing load on the distribution server computer 100 for generating the virtual camera viewpoint images of virtual camera C1 for these "Costume Change Performance" and "Double Ego Performance."

[0195] Furthermore, since the processing load for the "flying performance" period is high on the distribution server computer 100, it is possible to generate the "flying performance" at the viewer terminal 300, similar to the "costume change performance" and "doppelganger performance". In this case, detailed trajectory data of the movement trajectory of the virtual camera moving through the virtual live venue needs to be distributed in advance to each viewer terminal 300 to generate the video. Therefore, in Example 2, the video for the "flying performance" is generated and distributed at the distribution server computer 100, but the present invention is not limited to this, and the video for the "flying performance" may also be generated at each viewer terminal 300 in order to reduce the processing load on the distribution server computer 100.

[0196] Here, the participant user table used in the distribution server computer 100 of the second embodiment will be briefly described with reference to FIG.

[0197] The participant user table is a data table used to identify the participation status of viewer users in a virtual live event and the download status of pre-data for each virtual live event registered in the event data shown in Figure 7, and as shown in Figure 23, is a table that can store data such as participation identification data (participation identification flag) that can identify the participation status of the participant (viewer user) in correspondence with a participant ID (viewer user ID), and pre-data DL identification data (pre-data DL identification flag) that can identify the participation area of ​​the participant (viewer user) and the download status of the pre-data.

[0198] In addition, the participant ID field contains all the viewer user IDs of the prospective participants described in the file of each participant group stored in the event data corresponding to the virtual live.

[0199] The participation identification data (participation identification flag) is registered as "0" corresponding to a non-participating state when a participant ID (viewer user ID) is registered, and is updated to "1" indicating participation when the participation of each participant (viewer user) is permitted by authentication. Therefore, the participation identification data (participation identification flag) makes it possible to identify, among the viewer users who are scheduled to participate, those who are participating and those who could not participate for some reason.

[0200] The participation area data is data that can identify whether the participation area of ​​each participant (viewer user) is the third area or the fourth area, and is registered when the participation of each participant (viewer user) is permitted by authentication.

[0201] The pre-data DL identification data (pre-data DL identification flag) is updated from "0" indicating incomplete to "1" indicating complete if the download (DL) of the pre-data is completed based on a check as to whether or not the download (DL) of the pre-data is completed at the pre-data DL completion check timing described above. Therefore, by using this pre-data DL identification data (pre-data DL identification flag), it is possible to easily identify all participants (viewer users) who have not completed the download (DL) of the pre-data among the participants (viewer users) virtually participating.

[0202] Next, a flow of downloading advance data when a viewer user virtually participates in a virtual live concert venue in the second embodiment will be described with reference to FIG.

[0203] First, in a viewer program (viewing application) running on the viewer terminal 300, the viewer user performs an operation to participate in a virtual live event in which the viewer plans to virtually participate, for example, by selecting a participation menu or operating a specific key on the keyboard to which the participation operation is assigned.

[0204] By executing this participation operation, a participation request including the viewer user ID and authentication data for authenticating the viewer user is transmitted from the viewer terminal 300 to the distribution server computer 100.

[0205] The delivery server computer 100 that has received the participation request executes a participant authentication process on the condition that it is a period during which participation in the live virtual space is possible. In the participant authentication process, it is determined whether the viewer user is a prospective participant of the corresponding virtual live depending on whether the viewer user ID included in the received participation request is registered as a participant ID in the participant user database corresponding to the virtual live to be participated in, and the viewer user is authenticated by comparing the authentication data included in the participation request with the authentication data described in the authentication information file identified from the authentication information file data stored in association with the viewer user ID in the viewer user data shown in Fig. 9.

[0206] If the viewer user is a prospective participant and has been authenticated using the authentication data, the participant identification data (participation identification flag) in the participant user table is updated to "1" corresponding to "participation."

[0207] The delivery server computer 100 then transmits (responds to) advance data DL command information to the viewer terminal 300 that sent the participation request, to cause the viewer terminal 300 to download advance data corresponding to the virtual live event to be participated in. By receiving the advance data DL command information, the viewer terminal 300 can confirm that participation has been permitted.

[0208] In response to receiving the advance data DL instruction information, the viewer terminal 300 identifies advance data to be downloaded, and transmits to the delivery server computer 100 an advance data DL request requesting the download of the identified advance data.

[0209] In response to receiving this advance data DL request, the delivery server computer 100 executes an advance data transmission process.

[0210] In this advance data transmission process, the requested advance data is transmitted to the viewer terminal 300 that sent the advance data DL request. As described above, the advance data includes the virtual space data (basic data) of the virtual live venue, the avatar data of the performer avatar 1, and the avatar data of other viewer users who are already participating virtually.

[0211] In the advance data transmission process, the avatar data of the viewer user who has sent the advance data DL request is additionally distributed as advance data to the viewer terminals 300 other than the viewer terminal 300 that sent the advance data DL request and which are viewer terminals 300 of viewer users whose participation has already been confirmed by the participant data table. As a result, the avatar data of all viewer users who are virtually participating in the virtual live show are distributed in advance to and stored in the viewer terminals 300 of each viewer user who is virtually participating in the virtual live show, together with the virtual space data (basic data) and the avatar data of the performer avatars.

[0212] Then, at the pre-data download completion check timing described above (see FIG. 21), the pre-data stored in the viewer terminal 300 is checked to see whether all the pre-data has been downloaded, and if the pre-data download has been completed, the pre-data DL identification data (pre-data DL identification flag) in the participant user table is updated from "0", indicating incomplete, to "1", indicating complete.

[0213] Furthermore, for participants (viewer users) whose pre-data download identification data (pre-data DL identification flag) remains at "0", indicating incomplete download, because the pre-data download has not been completed, only the virtual camera viewpoint image can be displayed, as described above.

[0214] Next, a situation in which the subject from which various viewpoint videos displayed on the viewer terminal 300 are generated changes during each period of the virtual live will be described with reference to Figs. 24 to 27.

[0215] First, during the live virtual space participation period when only the viewer avatar viewpoint video can be displayed on the viewer terminal 300, the virtual space data already stored in the viewer terminal 300 is sequentially updated by receiving the advance data based on the virtual space update data including the performer avatar action data for generating the action of the performer avatar 1 that performs the action linked to the action of the performer user, which is transmitted from the distribution server computer 100, as shown in Fig. 24, and the viewer viewpoint video generation process is executed in the viewer terminal 300 based on the updated virtual space data to generate the viewer avatar viewpoint video, which is output (displayed) by the video output process. Note that, although the performer terminal 200 is omitted in Fig. 24, the virtual space update data is also transmitted to the performer terminal 200 as shown in Fig. 10 in the first embodiment.

[0216] Furthermore, during each of the periods of "introduction," "opening performance," and "flying performance" in which only virtual camera viewpoint video can be displayed on viewer terminals 300, as shown in Fig. 25, in the virtual space update data distribution process, distribution server computer 100 transmits virtual space update data only to performer terminals 200. Then, following the virtual space update data distribution process, a virtual camera viewpoint video generation process is executed in which a virtual camera viewpoint video is generated, and a virtual camera viewpoint video data distribution process is executed in which the video data of the virtual camera viewpoint video generated in the virtual camera viewpoint video generation process is distributed to all viewer terminals 300 participating in the virtual live.

[0217] The virtual camera viewpoint video data distributed from distribution server computer 100 in the virtual camera viewpoint video data distribution process is displayed by executing a video output process in viewer terminal 300 that receives the virtual camera viewpoint video data.

[0218] In other words, as shown in FIG. 27, for each period of “introduction”, “opening performance”, and “flying performance” when only virtual camera viewpoint images can be displayed, the virtual camera viewpoint images are generated in distribution server computer 100.

[0219] In addition, during the performance period excluding the MC period and the flying performance period, when either the viewer avatar viewpoint image or the virtual camera viewpoint image can be selected and displayed on the viewer terminal 300, as shown in FIG. 26, a virtual space update data distribution process is executed in the distribution server computer 100, and virtual space update data is distributed to the viewer terminal 300 of each viewer user participating in the virtual live, thereby executing the virtual space update process in the viewer terminal 300.

[0220] Then, in the viewer terminal 300, it is determined whether or not a viewer avatar viewpoint video has been selected as the video to be displayed. If a viewer avatar viewpoint video has been selected, a viewer avatar viewpoint video generation process is executed to generate a viewer avatar viewpoint video, similar to the flow shown in FIG. 24, and the generated viewer avatar viewpoint video is displayed by a video output process.

[0221] On the other hand, when the viewer avatar viewpoint video is not selected as the image to be displayed, i.e., when the virtual camera viewpoint video is selected as the image to be displayed, a virtual camera viewpoint video generation process is executed in the viewer terminal 300 to generate a virtual camera viewpoint video, and the generated virtual camera viewpoint video is displayed by a video output process.

[0222] Furthermore, the viewer user may be able to select which of the virtual cameras C1, C3, or C4 the virtual camera viewpoint image generated in this virtual camera viewpoint image should be, or the distribution server computer 100 may instruct all viewer terminals 300.

[0223] 27, during the performance period excluding the MC period and the flying performance period when either the virtual camera viewpoint video or the viewer avatar viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated on the viewer terminal 300. In this way, during the period when either the virtual camera viewpoint video or the viewer avatar viewpoint video can be selected and displayed on the viewer terminal 300, switching between the virtual camera viewpoint video and the viewer avatar viewpoint video is performed at the viewer user's selection, so that switching between these videos can be performed without problems such as video interruption and while preventing an increase in the processing load on the distribution server computer 100 that accompanies switching between these videos, as compared to when the virtual camera viewpoint video data is displayed by distribution from the distribution server computer 100.

[0224] On the other hand, during each of the periods of "introduction to the performance", "opening performance", and "flying performance", in which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only on the distribution server computer 100 and distributed to each viewer terminal 300. In this way, during the period in which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only on the distribution server computer 100, so that it is possible to prevent the process of generating the virtual camera viewpoint video from being executed redundantly on the viewer terminal 300, thereby preventing a needless increase in the processing load of the entire system.

[0225] Here, the situation when the viewpoint video is switched on the viewer terminal 300 will be described with reference to FIG. 28, taking as an example a period of "flying effect" during which only the virtual camera viewpoint video can be displayed.

[0226] For example, as shown in FIG. 28, in a viewer terminal 300 on which a viewer avatar viewpoint video is displayed during the "first MC" period and the first performance period up to the "flying performance", if the distribution server computer 100 starts distributing the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the start of the "flying performance" period in which only the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) can be displayed, the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will not be displayed on the viewer terminal 300 during the period from the start of these distributions until the virtual camera viewpoint video is displayed, which could result in a problem in which the viewer is unable to view the first video of the "flying performance".

[0227] For this reason, at the start of a "flying performance" in which the viewpoint image is forcibly changed from the viewer avatar viewpoint image to the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air), as shown in FIG. 28, the distribution server computer 100 of the second embodiment sends a BF advance notification to the viewer terminal 300 at a specific timing prior to the start of the "flying performance", thereby notifying the viewer terminal 300 of the distribution start (buffering start) timing for starting advance distribution of the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air) before the start of the flying performance period.

[0228] In response to receiving the BF advance notification, the viewer terminal 300 identifies the timing at which distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will begin from the distribution server computer 100, and begins buffering the received virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from that timing.

[0229] The timing at which distribution of these virtual camera viewpoint images (viewpoint images from virtual cameras moving in the air) begins, that is, the timing at which buffering begins, can be determined appropriately taking into account differences in the image storage functions of the viewer terminals 300, but can be set to a period corresponding to several tens of frames of image, for example.

[0230] In this way, the distribution server computer 100 of Example 2 begins generating and distributing virtual camera viewpoint video of the flying performance (viewpoint video from a virtual camera moving in the air) before the start of the flying performance period, and the viewer terminal 300 buffers the distributed virtual camera viewpoint video. Therefore, when the flying performance period begins, the video can be instantly changed from the viewer avatar viewpoint video that was displayed before the start of the flying performance period to the virtual camera viewpoint video, thereby preventing the occurrence of a problem in which the viewer is unable to view the initial video of the "flying performance".

[0231] Furthermore, when the period of the ``flying performance'' ends, because the viewer avatar viewpoint video was selected before the ``flying performance'' began, the viewpoint video switches from the flying performance's virtual camera viewpoint video (viewpoint video from the virtual camera moving in the air) to the viewer avatar viewpoint video upon the end of the ``flying performance'' period. However, even in this case, if the viewer avatar viewpoint video is generated and displayed from the end of the ``flying performance'' period, there is a risk that a malfunction will occur in which the viewer avatar viewpoint video is not displayed on viewer terminal 300 during the period until the viewer avatar viewpoint video is generated and displayed, just as at the start of the ``flying performance''.

[0232] For this reason, even when the period of these "flying effects" ends, the distribution server computer 100 of the second embodiment sends an end TM (timing) advance notification to the viewer terminal 300, as shown in FIG. 28, thereby notifying the viewer terminal 300 in advance of the end timing of the "flying effects" period when the distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will end.

[0233] In response to receiving the end TM (timing) advance notification, the viewer terminal 300 determines in advance the timing at which the distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the distribution server computer 100 will be stopped, and starts the viewer viewpoint video generation process at a specified timing before that timing, thereby pre-generating the viewer avatar viewpoint video.

[0234] Furthermore, the distribution server computer 100 starts distributing the virtual space update data from the time of sending the end TM (timing) advance notification, and the viewer terminal 300 executes virtual space processing to update the virtual space data using the received virtual space update data, so that at the specified timing mentioned above, a viewer avatar viewpoint image that reflects the situation of the virtual live venue at that time can be generated.

[0235] In this way, the distribution server computer 100 of Example 2 sends an end TM (timing) advance notification to the viewer terminal 300 before the end of the flying performance period, enabling the viewer terminal 300 to generate the viewer avatar viewpoint video in advance before the end of the flying performance period, thereby preventing a problem from occurring in which the viewer avatar viewpoint video is not displayed on the viewer terminal 300.

[0236] In addition, in Figure 28, the "flying performance" is used as an example, but similar control is also performed during the "pre-show speech" and "opening performance" periods, when the viewpoint image is forcibly changed from the viewer avatar viewpoint image to the virtual camera viewpoint image.

[0237] As described above, in the virtual space content distribution system of Example 2, there are multiple viewer users, and each viewer user is assigned a viewer terminal 300. When the viewer terminal 300 displays the viewer avatar viewpoint video, which is a viewpoint video of the virtual live venue (virtual space) corresponding to the viewer avatar, the viewer avatar viewpoint video is generated based on the virtual space update data (virtual space data) transmitted from the distribution server computer 100. Since the generation of these viewer avatar viewpoint videos, which have a large processing load, is performed by the viewer terminal 300, more viewer users can participate in the virtual live.

[0238] Furthermore, the virtual space content of Example 2 is content that allows a performer user to participate in a virtual live venue as a performer avatar 1, and is content in which the performer avatar 1 performs musical actions (performances) such as singing and dancing in the virtual live venue, thereby increasing the interest of the virtual space content.

[0239] Furthermore, since the virtual space update data (virtual space data) of Example 2 includes movement data for the performer avatar for generating the movements of the performer avatar 1, the movements of the performer avatar 1 can be accurately reflected in the viewer avatar viewpoint image generated in the viewer terminal 300.

[0240] In addition, the common viewpoint video displayed in common on each viewer terminal 300 in Example 2, such as the video of the "pre-show speech," "opening performance," and "early performance," is generated in the distribution server computer 100 and distributed to each viewer terminal 300, thereby preventing unnecessary increase in the processing load of the entire system due to duplicate processing being executed in each viewer terminal 300.

[0241] Furthermore, in the viewer terminal 300 of Example 2, during the performance period (specific period) other than the "MC period" and "flying performance" periods, during which either the viewer avatar view point video or the virtual camera view point video can be selected and displayed, the virtual camera view point video is generated and displayed in the viewer terminal 300, thereby preventing problems such as video interruptions when the view point video is switched.

[0242] In addition, during special periods such as "costume change performance" and "doppelganger performance" when the processing load on the distribution server computer 100 in Example 2 is high, each viewer terminal 300 also generates and displays the virtual camera viewpoint image (common viewpoint image), thereby preventing the processing load on the distribution server computer 100 from becoming temporarily extremely high.

[0243] Furthermore, the distribution server computer 100 of Example 2 transmits advance data including basic data of the virtual live venue (virtual space) and basic data of the performer avatar 1 and viewer avatars who will virtually participate in the virtual live venue to each viewer terminal 300 before the virtual live starts, thereby preventing the occurrence of a problem in which the viewer avatar's viewpoint image is not displayed properly at the start of the virtual live.

[0244] In addition, the distribution server computer 100 of Example 2 transmits virtual camera viewpoint video (common viewpoint video) to viewer terminals 300 for which it cannot confirm that they have completed receiving the advance data. In this way, it is possible to prevent the processing load on the distribution server computer 100 from increasing excessively due to the transmission of basic data, etc. after the start of the virtual live broadcast, while also preventing viewer users from being unable to watch the virtual live broadcast.

[0245] <Modification of Example 2> FIG. 29 is a flowchart showing the process of generating a viewer's viewpoint video in the modification 2-1 of the second embodiment.

[0246] In the above-mentioned Example 2, a form has been shown in which viewer avatar viewpoint images including the movements of the performer avatar 1 are always generated uniformly, but the present invention is not limited to this, and these viewer avatar viewpoint images may be generated in a number of ways with different processing loads in image generation, depending on the selection of the viewer user at the viewer terminal 300.

[0247] In other words, as the viewer terminal 300 used by the viewer user, as exemplified in Example 1, it is expected that terminals with different processing capabilities will be used, such as a desktop personal computer (PC) with high processing capabilities, or a smartphone terminal or tablet terminal with relatively lower processing capabilities than a personal computer (PC). However, if the viewer avatar viewpoint video is always generated uniformly on terminals with such different processing capabilities, the viewer avatar viewpoint video may not be generated properly on a terminal with low processing capabilities, which may result in a misalignment between the viewer avatar viewpoint video and audio, or the viewer avatar viewpoint video may become discontinuous and unnatural. Therefore, for example, when generating viewer avatar viewpoint video including performer movements, it may be possible to allow the viewer to select appropriately depending on the processing capabilities of the terminal he or she is using whether to generate viewer avatar viewpoint video including high-definition performer movements, which requires a high processing load for generating the video, or to generate viewer avatar viewpoint video including low-definition performer movements, which requires a low processing load for generating the video.

[0248] Specifically, the viewer viewpoint video generation process executed in the viewer terminal 300 may be as shown in the flow diagram of Fig. 29, and in the viewer viewpoint video generation process, first, the performer movement generation mode selected by the viewer user in the viewer terminal 300 is identified (step S101). Then, it is determined whether the identified performer movement generation mode is a high processing load mode (step S102), and if the high processing load mode is selected (Y in step S102), a viewer avatar viewpoint video including high-definition performer movements is generated (step S103). On the other hand, if the high processing load mode is not selected (N in step S102), a viewer avatar viewpoint video including low-definition performer movements is generated (step S104).

[0249] Although FIG. 29 shows an example of two modes, a high processing load mode and a low processing load mode, the present invention is not limited to this, and the processing load modes may be three or more modes.

[0250] In addition, in the above-mentioned Example 2 (which is also the same as Example 1), an example is given in which all actions, including the movement of the viewer avatar of the viewer user, are reflected in each viewer avatar by distributing virtual space update data, but the present invention is not limited to this. For example, in the case where the actions of these viewer avatars are actions that cannot normally be performed but become possible when a special fee is paid, for example, an operation to perform a special action that allows the viewer avatar to appeal to the performer avatar 1, the special action is directed to the performer user and not to other viewer users, so it is possible to prevent the virtual space update data including these special actions from being distributed to the viewer terminals 300 of other viewer users, thereby preventing an excessive increase in the processing load of the distribution server computer 100 by making these special actions executable.

[0251] In this way, if virtual space update data including special actions is not distributed to viewer terminals 300 of other viewer users, as shown in variant example 2-2 of Figure 30, when viewer user A performs an operation corresponding to the special action, causing viewer user A's viewer avatar to perform a special action, virtual space update data A corresponding to viewer user A's viewer avatar performing the special action is transmitted from the distribution server computer 100 to viewer terminal 300 of viewer user A, but virtual space update data B in which viewer user A's viewer avatar is not performing the special action is distributed to viewer terminal 300 of viewer user B, who is different from viewer user A.

[0252] Therefore, different virtual space update data is delivered from the distribution server computer 100 to the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B, thereby increasing the flexibility of the virtual space update data and making it possible to generate different viewer avatar viewpoint images on the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B.

[0253] Furthermore, viewer terminal 300 of viewer user A can generate and display an image of viewer avatar of viewer user A performing a special action different from the actions that viewer avatar of viewer user B is capable of performing based on virtual space update data A that is different from virtual space update data B delivered to viewer terminal 300 of viewer user B, thereby increasing viewer user A's interest in the virtual live performance, which is the virtual space content.

[0254] In addition, in the above-mentioned Example 2 (which is also the same as Example 1), the viewer avatars of viewer users virtually participating in the fourth area are illustrated as normal avatars in which the avatar appearance differs for each viewer user, but the present invention is not limited to this. For example, in order to reduce the processing load when generating various viewpoint images in a virtual live performance, the viewer avatars of viewer users virtually participating in the fourth area may be simplified avatars in which the avatar appearance is simplified.

[0255] Note that Figure 30 shows an example of executing a special action that becomes possible when a special fee is paid. However, for example, in the case of using a simplified avatar as described above, normal actions other than the movement of the simplified avatar, such as waving, clapping, and changing the direction of the viewer avatar, may be prevented from being included in the virtual space update data distributed to other viewer users, just like special actions, thereby further reducing the processing load on the distribution server computer 100. EXAMPLES

[0256] Next, a virtual space content delivery system according to a third embodiment will be described below with reference to Fig. 31 to Fig. 46. Note that the same components as those in the first and second embodiments will be denoted by the same reference numerals and description thereof will be omitted.

[0257] The configuration of the virtual space content distribution system of Example 3 is almost the same as the configuration of Example 1 shown in FIG. 1, as shown in FIG. 32. However, the performer terminals 200 include performer terminal 200A used by performer user A and performer terminal 200B used by performer user B. As shown in FIG. 31, performer avatar 1A corresponding to performer user A and performer avatar 1B corresponding to performer user B appear as a group (unit) in the virtual live performance, which is different from Example 1. In addition, as described below, during the performance period other than the MC period when at least one of performer user A or performer user B is talking, the pre-recorded actions of performer user A and / or performer user B are reflected in the actions of performer avatar 1A and / or performer avatar 1B and distributed as a performance for the virtual live performance, which is a characteristic feature of Example 3.

[0258] Furthermore, since performer user A in Example 3 is the same as the performer user in Example 1, performer avatar 1A is the same as performer avatar 1 in Example 1.

[0259] In addition, in Example 3, as the virtual placement objects 43 placed on the virtual stage, a star virtual placement object 43A related to performer avatar 1A (performer user A) is placed at a predetermined position on the virtual stage where performer avatar 1A is located (rear position on the right side as one faces the virtual stage), and a heart virtual placement object 43B related to performer avatar 1B (performer user B) is placed at a predetermined position on the virtual stage where performer avatar 1B is located (rear position on the left side as one faces the virtual stage).

[0260] In addition, in Example 1, star-shaped virtual aerial objects 50'-52' corresponding to performer avatar 1 were placed as virtual aerial objects 50'-52', but in Example 3, these are changed to a shape having both a star and a heart shape to correspond to performer avatar 1A and performer avatar 1B.

[0261] 31 does not show viewer avatars virtually participating in the virtual audience area for convenience, but viewer users can virtually participate in the virtual live venue as viewer avatars, as in Example 1. As for the virtual audience area in which viewer users can virtually participate as viewer avatars, a third area which is a special area and a fourth area which is a non-special area are formed, as in Example 1, but the present invention is not limited to this, and for example, the virtual audience areas may be provided with a third area which is a special area corresponding to performer avatar 1A and a third area which is a special area corresponding to performer avatar 1B.

[0262] In Example 3, as described above, there are two performer users, performer user A and performer user B, so two different studios are used. Specifically, since the performer user in Example 1 and performer user A in Example 3 are the same, performer user A in Example 3 uses the studio (studio A) shown in Fig. 11, just like the performer user in Example 1. On the other hand, performer user B uses studio B shown in Fig. 33.

[0263] In studio B used by performer user B, as shown in FIG. 33, base stations 220a and 220b are arranged in the same manner as in studio A, in cooperation with wearable sensors 220C1-220C5 worn by performer user B, just like performer user A, to enable detection of the movements of performer user B in almost real time, as well as a projector (display device B211) that projects onto a screen S and a display device C212. In addition, a performer terminal 200B is installed in a control room (not shown) separated by a glass window, and an assistant operator O assists in the operation of performer terminal 200B.

[0264] Furthermore, the display device C212 installed in studios A and B is capable of displaying information such as each pose (pre-recorded pose) that performer user A and performer user B should assume and the remaining time until each pose is assumed, as shown in Figures 34 and 35, so that performer user A and performer user B can easily understand the pre-recorded poses by visually checking them.

[0265] Next, the progress of the virtual live performance delivered by the virtual space content delivery system of the third embodiment will be described with reference to Figs. 36 and 37. In the virtual live performance of the third embodiment, an opening performance is first performed to indicate that a virtual live performance by performer avatars 1A and 1B is about to begin. In this opening performance, performer avatars 1A and 1B do not appear on the virtual stage, and these opening performances are delivered as images recorded on video at the virtual live venue due to the absence of performer avatars 1A and 1B. Although not shown in Fig. 36, a "live virtual space participation period" is provided before these opening performances, as in the second embodiment, and viewer users can virtually participate in the virtual live venue, which is the target virtual space of the virtual live performance, during the live virtual space participation period.

[0266] When the opening performance ends, the performer entrance performance begins, in which the performer avatars 1A and 1B appear on the virtual stage. In other words, during the performer entrance performance, the performer avatars 1A and 1B appear on the virtual stage G and give a greeting talk to the virtually participating viewer avatars, an introduction to the first performance, and other such MC duties.

[0267] When transitioning from the performer entrance performance to the pre-recorded first performance in which performer avatars 1A and 1B sing and dance, a transition period 1 involving start-posture assimilation is set immediately before the end of the performer entrance performance, in order to have performer user A and performer user B assume the initial pose (posture) at the start of the first performance.

[0268] Furthermore, when the first performance ends and transitions to the first MC by performer avatars 1A and 1B, as shown in FIG. 41, a transition period 2 is set immediately before the end of the first MC period, in which the display of the virtual live performance on the viewer terminal 300 temporarily goes dark, and a blackout effect (reduced brightness effect) is executed so that the viewer user is temporarily unable to see the posture (movement) of performer avatars 1A and 1B.

[0269] Furthermore, during transition period 2 of Example 3, if the display of the virtual live performance on the viewer terminal 300 were to suddenly change from the display state of Figure 41(c) to a black display as shown in Figure 41(f), there is a risk that the viewer user would mistakenly believe that this is a malfunction of the equipment. Therefore, before the display is blacked out as in Figure 41(f), a performance is implemented in which the contrast of the performer avatars 1A and 1B is gradually reduced, as shown in Figures 41(d) and (e), so that the viewer user can easily recognize that the black display is a performance and not a malfunction.

[0270] In this way, when the pre-recorded first performance ends and the first MC begins in which the movements of performer users A and B are reflected in performer avatars 1A and 1B almost in real time, a blackout effect is executed during transition period 2 to make performer avatars 1A and 1B invisible to viewer users. Even if the postures of performer avatars 1A and 1B, which reflect the postures of performer users A and B at the start of the first MC, are significantly different from the postures of performer avatars 1A and 1B at the end of the first performance, resulting in unnatural movements with discontinuous posture changes, these unnatural movements will not be distributed to viewer users and be visible, and this will prevent viewer users from realizing that the performances and MCs were pre-recorded, which would significantly reduce their interest in the virtual live performance (virtual space content).

[0271] Furthermore, in the case where both performer avatars 1A, 1B appear on the virtual stage G, such as in transition period 2, if an object effect is executed in which performer avatars 1A, 1B are hidden by objects corresponding to each performer avatar 1A, 1B, as described below, each of the multiple performer avatars 1A, 1B will be hidden by a different object, which may excessively increase the processing load on the distribution server computer 100 and cause problems in distribution. However, as described above, by executing a blackout effect that does not significantly increase the processing load by darkening the display, even if multiple performer avatars appear, the processing load on the distribution server computer 100 is not increased, and this is preferable, as it can prevent problems in distribution due to an increase in processing load.

[0272] When the first MC transitions to a pre-recorded second performance in which performer avatar 1A sings and dances, a transition period 3 is set immediately before the end of the first MC period, during which a blackout effect is implemented to make performer avatars 1A and 1B temporarily invisible to audience users.

[0273] In this way, by executing a blackout effect during the transition period when the number of performer avatars appearing on virtual stage G changes, it is possible to reduce not only the discomfort felt by viewers and users due to a discontinuity in movement caused by performer avatar 1A's posture being significantly different between the end of the first MC and the beginning of the second performance, but also the discomfort felt by viewers and users due to the change in the number of performer avatars appearing.

[0274] When transitioning from the second performance to the second MC by performer avatar 1A, a transition period 4 is set just before the end of the second performance period, in which an object 1 concealment performance is executed in which ``star'' objects corresponding to performer avatar 1A fly together and cover performer avatar 1A, as shown in Figure 42.

[0275] In addition, as shown in Figure 42, the costume of performer avatar 1A is changed before and after the object 1 hiding performance in Example 3. Changing the costume in this manner before and after the object 1 hiding performance (transition period 4) is preferable because it makes the viewer user recognize that performer avatar 1A has been hidden by an object due to the change in costume, making it difficult for the viewer user to recognize the original purpose of the hiding performance, which is to "avoid giving the viewer user a sense of discomfort due to discontinuity in movement." However, the present invention is not limited to this, and it is also possible not to change the costume before and after the object 1 hiding performance (transition period 4).

[0276] Furthermore, as described above, when the pre-recorded second performance ends and the second MC begins in which performer user A's movements are reflected in performer avatar 1A almost in real time, an object 1 concealment performance is executed that makes performer avatar 1A invisible to viewer users. Even if the posture of performer avatar 1A, which reflects performer user A's posture at the start of the second MC, differs significantly from the posture of performer avatar 1A at the end of the second performance, resulting in unnatural movements with discontinuous posture changes, these unnatural movements will not be visible to viewer users, and this prevents viewer users from realizing that the performances and MCs were pre-recorded, which would significantly reduce their interest in the virtual live performance (virtual space content).

[0277] When transitioning from the second MC to the pre-recorded third performance in which performer avatar 1A sings and dances, a transition period 5 is set immediately before the end of the second MC period, during which start posture assimilation is performed to have performer user A assume the initial pose (posture) at the start of the third performance.

[0278] When transitioning from the third performance to the third MC by performer avatar 1A (partway through, performer avatars 1A and 1B), a transition period 6 is set immediately before the end of the third performance period, during which end posture assimilation is performed to have performer user A assume the final pose (posture) at the end of the third performance.

[0279] When the performance transitions from the third MC to the pre-recorded fourth performance in which performer avatar 1B sings and dances, a transition period 7 is set immediately before the end of the third MC period, during which a blackout effect is implemented to make performer avatars 1A and 1B temporarily invisible to audience users.

[0280] In this way, by executing a blackout effect when the number of performer avatars appearing on the virtual stage G changes (changing from 2 to 1), as in the case of transition period 3 described above, this is preferable because it reduces the discomfort that viewer users feel when the number of performer avatars changes without excessively increasing the processing load on the distribution server computer 100, but the present invention is not limited to this. For example, the number of performer avatars may be changed in advance before transition period 7 begins, and during transition period 7, an effect may be implemented in which performer avatar 1B is concealed by an object corresponding to performer avatar 1B.

[0281] When transitioning from the fourth performance to the fourth MC by performer avatar 1B, as shown in FIG. 37, a transition period 8 is set immediately before the end of the fourth performance period, in which end posture assimilation is performed to have performer user B assume the final pose (posture) at the end of the fourth performance.

[0282] When the performance transitions from the fourth MC to the pre-recorded fifth performance in which performer avatar 1B sings and dances, a transition period 9 is set immediately before the end of the fourth MC period, during which start posture assimilation is performed to have performer user B assume the initial pose (posture) at the start of the fifth performance.

[0283] When transitioning from the fifth performance to the fifth MC by performer avatar 1B (and then performer avatars 1A and 1B halfway through), a transition period 10 is set just before the end of the fifth performance period, in which an object 2 hiding performance is executed in which "heart" objects corresponding to performer avatar 1B fly together to hide performer avatar 1B, as shown in Figure 43.

[0284] In addition, as shown in Figure 43, the costume of performer avatar 1B is changed before and after the object 2 hiding performance in Example 3. Changing the costume in this manner before and after the object 2 hiding performance (transition period 10) is preferable because it makes the viewer user recognize that performer avatar 1B has been hidden by an object due to the change in costume, making it difficult for the viewer user to recognize the original purpose of the hiding performance, which is to "avoid giving the viewer user a sense of discomfort due to discontinuity in movement." However, the present invention is not limited to this, and it is also possible not to change the costume before and after the object 2 hiding performance (transition period 10).

[0285] Furthermore, as described above, when the pre-recorded fifth performance ends and the fifth MC begins in which performer user B's movements are reflected in performer avatar 1B almost in real time, an object 2 concealment effect is executed that makes performer avatar 1B invisible to the viewer user. Even if the posture of performer avatar 1B, which reflects performer user B's posture at the start of the fifth MC, differs significantly from the posture of performer avatar 1B at the end of the fifth performance, resulting in unnatural movements with discontinuous posture changes, these unnatural movements will not be visible to the viewer user, and this prevents the viewer user from realizing that the performances and MCs were pre-recorded, which would significantly reduce their interest in the virtual live performance (virtual space content).

[0286] When the performance transitions from the 5th MC to the pre-recorded 6th performance in which performer avatars 1A and 1B sing and dance, a transition period 11 is set immediately before the end of the 5th MC period, during which a blackout effect is implemented to make performer avatars 1A and 1B temporarily invisible to audience users.

[0287] When transitioning from the sixth performance to the sixth MC by performer avatars 1A and 1B, a transition period 12 is set immediately before the end of the sixth performance period, during which end posture assimilation is performed to cause performer users A and B to assume the final pose (posture) at the end of the sixth performance.

[0288] When the performance transitions from the 6th MC to the pre-recorded 7th performance in which performer avatar 1B sings and dances, a transition period 13 is set immediately before the end of the 6th MC period, during which a blackout effect is implemented to make performer avatars 1A and 1B temporarily invisible to audience users.

[0289] When transitioning from the seventh performance to the seventh MC by performer avatars 1A and 1B, a transition period 14 is set immediately before the end of the seventh performance period, during which a blackout effect is implemented to make performer avatar 1B temporarily invisible to the audience users.

[0290] Although Figure 37 only shows up to the 7th MC, the virtual live performance in Example 3 will be carried out in the order of 8th performance, 8th MC, 9th performance, 9th MC, etc., and a transition period n is set between each of these performances and each MC.

[0291] The virtual live of the third embodiment, which proceeds as described above, is managed by the delivery server computer 100 based on the time schedule data (hereinafter, TS data) shown in Fig. 38. Note that the TS data (also called timeline data) is stored in the storage 103 of the delivery server computer 100 in a form included in the event management program 110, as described in the first embodiment.

[0292] The TS data of the third embodiment is data in a table format in which the scheduled time of each timing, the elapsed time from the start of the live, the content of the timing, and the file name of the associated data related to the timing are stored in correspondence with a timing ID assigned to each individual timing in the virtual live.

[0293] As shown in FIG. 38, these timings include the start and end timings of the opening performance and the performer entrance performance, and the timing when performer avatars 1A and 1B appear during the performer entrance performance, as well as the start and end timings of each transition period (which also corresponds to the start timing of the performance and MC).For example, the associated data corresponding to the end timing of the transition period (the start timing of the performance) stores the file name of the pre-recorded movement data of performer user A and / or performer user B for the performance that is being started (performer movement data for performance).

[0294] In addition, as for the associated data corresponding to the start timing of the transition period, if a blackout effect is executed during the transition period, the file name of data related to the blackout effect (contrast adjustment data and data for the blackout screen) is stored; if an object hiding effect is executed during the transition period, the file name of data related to the object hiding effect (object data regarding the object to be used and data for moving the object (trajectory data)) is stored; if a posture assimilation process is executed during the transition period, the file name of data related to the posture assimilation process (posture assimilation type data, posture data for the start posture and end posture, data for the posture determination timing, etc.) is stored.

[0295] In this way, when the posture assimilation process is executed during the transition period, as described below, the data of the start posture and end posture, which are the pre-recorded postures shown to performer user A and / or performer user B, are stored in advance as posture data, and it is possible to accurately identify which transition period this posture data will be used in using the TS data.

[0296] Next, the flow of execution of each program, each MC, each performance, and each process in the virtual space content delivery system of the third embodiment will be described with reference to Figs.

[0297] As shown in FIG. 39, the distribution server computer 100 of the third embodiment executes a period determination process to determine whether the current time falls within a program period, an MC period, or another period.

[0298] In addition, in Figure 39, due to space constraints, the description of other periods is omitted. However, during other periods, for example, during the opening performance period, processing corresponding to each period is carried out, such as processing to distribute pre-recorded video footage corresponding to the opening performance.

[0299] In addition, the period determination process can determine which period a given point in time falls into by comparing the elapsed time determined from the timer value of an elapsed time timer that began counting (timing) at the start of the virtual live with the elapsed time in the TS data.

[0300] If the period determination process determines that the MC period is occurring, performer avatar information is imported, which includes the facial expressions and movements of the performer users scanned by motion scan (body motion, face motion) transmitted from performer terminals 200A, B, as well as the voice information of the performer users input to performer terminal 200.Based on the imported performer avatar information and the viewer avatar information transmitted from each viewer terminal 300, the facial expressions, movements, and voice of performer user A are reflected in performer avatar 1A, the facial expressions, movements, and voice of performer user B are reflected in performer avatar 1B, and a virtual space update process is executed to reflect the movements of each viewer user in the viewer avatar.

[0301] In addition, in Example 3, unlike Example 1, the voice of the performer user input to the performer terminal 200 by the performer voice input process is also included in the performer avatar information, but as in Example 1, the voice of the performer user may be transmitted to the distribution server computer 100 separately from the performer avatar information.

[0302] On the other hand, if the period determination process determines that the performance is currently in progress, performer action data for the performance, in which the facial expressions, actions, and singing of performer user A and performer user B have been pre-recorded, is read.

[0303] The performer movement data for the performance in Example 3 is data of the facial expressions, dance movements, and singing of performer user A and / or performer user B in the performance that has been recorded in advance in chronological order by motion scanning or voice input, and the data is read in sequence and reflected in the facial expressions, movements, and voice of performer avatar 1A and / or performer avatar 1B, thereby enabling the singing and dancing of each performance to be accurately reproduced.

[0304] Then, based on the loaded performer movement data for the performance and the viewer avatar information transmitted from each viewer terminal 300, the performer movement data for the performance is reflected in the performer avatar 1A and / or the performer avatar 1B, and a virtual space update process is executed to reflect the movements of each viewer user in the viewer avatar.

[0305] Thus, in Example 3, during the MC period, performer avatar information transmitted from performer terminals 200A, B is reflected in performer avatar 1A and / or performer avatar 1B, so that the facial expressions, movements, and speech of performer users A, B are reflected and distributed in almost real time in performer avatar 1A and / or performer avatar 1B.

[0306] Meanwhile, during the performance period, the pre-recorded facial expressions, movements, and singing voices of performer users A and B are reflected in performer avatar 1A and / or performer avatar 1B as performer movement data for the performance recorded in chronological order, thereby distributing the singing and dancing of performer avatar 1A and / or performer avatar 1B in each performance.

[0307] Next, it is determined whether or not that point in time is within a transition period, and if it is not within the transition period set immediately before the end of a performance period or an MC period, the process returns to the period determination process. If it is within a transition period, it is further determined whether the type of transition period is a type of effect in which either a blackout effect or an object hiding effect is executed during the transition period.

[0308] If the transition period type is not a performance type, that is, if it is a posture assimilation type in which posture assimilation processing is performed during the transition period, proceed to B in Figure 40, and if the transition period type is a performance type, determine whether the performance to be performed during the transition period is a blackout performance.

[0309] Furthermore, information regarding the type of transition period, the performance to be performed during the transition period, and the type of posture assimilation processing (start posture or end posture) is described in transition period information files stored as related data of the TS data, and can be identified from these transition period information files.

[0310] If the effect to be executed during the transition period is a blackout effect, a blackout effect reflection process is executed to reflect the contents of the blackout effect (such as the contrast of the performer avatar and the brightness of the display) in the virtual space data updated above, and then a virtual space update data generation process and a virtual space update data distribution process are executed before returning to the period determination process.

[0311] In this way, the contents of the blackout effect are reflected in the virtual space data updated in the virtual space update process, so that the blackout effect is applied to each performance and each MC and distributed.

[0312] On the other hand, if the effect to be executed during the transition period is not a blackout effect, that is, if it is an object hiding effect (a general term for the object 1 hiding effect and the object 2 hiding effect), an object effect reflection process is executed to reflect the contents of the object hiding effect (additional placement of the corresponding object based on the trajectory data), and then a virtual space update data generation process and a virtual space update data distribution process are executed before returning to the period determination process.

[0313] In this way, the content of the object hiding performance is reflected in the virtual space data updated by the virtual space update process, and the object hiding performance is applied to the performer avatars 1A, 1B appearing in each performance and each MC and distributed.

[0314] In addition, when blackout and object hiding effects are executed, pre-recorded performer action data for the performance is read and reflected in the virtual space data rather than performer avatar information. Therefore, during the transition period when blackout and object hiding effects are executed, the voices of performer users A and B in each studio are not reflected in performer avatars 1A and 1B, and the voices of performer users A and B are not distributed. In addition, the performer action data for the performance corresponding to the transition period is also data that does not contain the voices of performer users A and B. Therefore, in Example 3, the voices of performer users A and B are not distributed during the period when blackout and object hiding effects are executed, but the present invention is not limited to this, and during the period when these blackout and object hiding effects are executed, specific message voices of performer users A and B may be distributed based on the performer action data for the performance.

[0315] Furthermore, in the determination of the transition type, if the transition type is the posture assimilation type, the process proceeds to B in FIG. 40, where it is determined whether or not it is time to start posture assimilation processing.

[0316] If it is time to start the posture assimilation process, the performer terminals 200A,B execute a posture assimilation information transmission process in which posture assimilation information including posture assimilation type data, posture data of the final posture and start posture, remaining period data, and information on the determination timing, which are necessary to execute the posture assimilation process shown in Fig. 40, is transmitted to the performer terminals 200A,B, and then the process returns to the period determination process. In response to receiving the posture assimilation information transmitted in the posture assimilation information transmission process, the performer terminals 200A,B start the posture assimilation process shown in Fig. 41.

[0317] The posture assimilation type data, posture data, remaining period data, and judgment timing information contained in the posture assimilation information can utilize file name data identified from information on related data stored in the TS data in association with the transition period corresponding to the elapsed time.

[0318] Then, the process goes from posture assimilation information transmission processing to posture assimilation correspondence processing, and then executes virtual space update data generation processing and virtual space update data distribution processing, before returning to the period determination processing.

[0319] Furthermore, since no correction information or posture assimilation information is transmitted from the performer terminals 200A, B at the time the posture assimilation information transmission process is executed, it is possible to proceed to the virtual space update data generation process without going through the posture assimilation support process.

[0320] If it is not the time to start the posture assimilation process, a posture assimilation response process is executed to perform processing corresponding to the correction information and posture assimilation information described below that are transmitted when the posture assimilation process is executed on the performer terminals 200A and B, and then a virtual space update data generation process and a virtual space update data distribution process are executed before returning to the period determination process.

[0321] In the posture assimilation processing of Example 3, when correction information is received from performer terminals 200A, B, a process is executed to update the virtual space data by reflecting the posture (motion data) contained in the correction information in the posture of performer avatars 1A, 1B, and when posture assimilation information is received from performer terminals 200A, B indicating that the posture of performer users A, B at the time of judgment is poor posture, a process is executed to reflect the effect of darkening the image being viewed by the viewer user in the virtual space data.

[0322] Next, the posture assimilation process executed in the performer terminals 200A and 200B of the third embodiment will be described with reference to FIG.

[0323] First, each data such as posture assimilation type, posture data, remaining period, and determination timing included in the posture assimilation information received from the distribution server computer 100 is stored (step S300). Note that, as for posture assimilation type data in the third embodiment, data "0" is assigned to a type of transition from a program (pre-recorded) to an MC (not pre-recorded) (type of assimilating to the posture at the end of the program), and data "1" is assigned to a type of transition from an MC (not pre-recorded) to a program (pre-recorded) (type of assimilating to the posture at the start of the program).

[0324] In addition, the remaining period data is data indicating the length of the transition period, and when the posture assimilation type is "0", the data indicates the length of the period until the performance period ends (the length of the period until the MC period starts), and when the posture assimilation type is "1", the data indicates the length of the period until the performance period starts (the length of the period until the MC period ends).

[0325] The data on the determination timing indicates the length of the remaining period at which the determination timing should be made, and is data indicating the timing of 1 within the remaining period.

[0326] Next, a flag initialization process (step S301) is executed to initialize the values ​​of a blackout effect execution flag and a correction operation execution flag used in the posture assimilation process.

[0327] Then, a posture assimilation screen display process (step S302) is executed to display the posture assimilation screen shown in Fig. 34 and Fig. 35 on the display device C212. Note that when the transition period ends (the remaining period corresponds to 0), the posture assimilation process ends, and the display of the posture assimilation screen ends.

[0328] As shown in Figures 34 and 35, the posture assimilation screen of Example 3 has a pre-recorded pose display area 250 in which the target posture to be taken by performer users A and B is displayed as an image of performer avatars 1A and 1B, a current pose display area 251 in which performer avatars 1A and 1B reflecting the posture of performer users A and B at that time are displayed, a remaining period display area 252 in which the remaining period until the performance start timing or MC start timing, which is the remaining period until performer avatars 1A and 1B assume the target posture, is displayed in the form of a time bar, a match display area 253 in which the match between the target posture and the current pose is displayed, a microphone muted display area 255 which indicates that the microphone is muted, and a blackout effect in progress display area 256 which indicates that a blackout effect is being executed.

[0329] In the case of posture assimilation type "1" for transition from MC to a program, the remaining period display area 252 displays "X seconds until program start timing" and the time bar displays the progress in red, as shown in Fig. 34. On the other hand, in the case of posture assimilation type "0" for transition from a program to MC, the remaining period display area 252 displays "X seconds until MC start timing" and the time bar displays the progress in green, as shown in Fig. 35.

[0330] In this way, by displaying the time bar in different colors depending on the posture assimilation type, performer users A and B can intuitively recognize whether the posture assimilation type is posture assimilation when transitioning to an MC or posture assimilation when transitioning to a program, and can take appropriate action according to the type.As a result, even if the same posture assimilation screen is used, as in Example 3, posture assimilation can be performed well.

[0331] The outer frame of the current pose display area 251 is displayed in a plurality of colors, namely, blue, yellow, and red, according to the degree of matching displayed in the degree of matching display area 253, and inside the current pose display area 251, four indicator units 260 each having a round shape and corresponding to each of the four limbs are provided. The indicator units 260 are displayed in a plurality of colors, namely, blue, yellow, and red, just like the outer frame, to indicate the matching status of the postures of each of the four limbs.

[0332] Normally nothing is displayed in the muted state display area 255, but when the microphone is muted, the message "Microphone muted" is displayed.

[0333] Furthermore, normally nothing is displayed in the blackout effect display area 256, but as described below, the message "Blackout effect in progress" is displayed during the period when the blackout effect execution flag is set.

[0334] As described above, when the posture assimilation screen is displayed by the posture assimilation screen display process, the target postures to be taken by the performer users A and B are images of the performer avatars 1A and 1B, and an image of the performer avatars 1A and 1B seen from behind is displayed in the pre-recorded pose display area 250. In this way, by displaying the pre-recorded poses as images seen from behind, the performer users A and B do not need to invert the left and right to recognize the postures, as compared to when the pre-recorded poses are displayed as images seen from the front, and can intuitively recognize the postures. In addition, in the posture assimilation screen, the current pose display area 251 also displays images of the performer avatars 1A and 1B reflecting the postures of the performer users A and B at that time, and the performer avatars 1A and 1B are displayed as images seen from behind, so that the performer users A and B can intuitively grasp the difference between the current postures of the performer users A and B and the pre-recorded poses, which is preferable, but the present invention is not limited to this, and the display modes of the pre-recorded poses and the current poses may be displayed in modes other than those described above.

[0335] After the posture assimilation screen display process is executed, the remaining period update process (step S303) is executed. In the remaining period update process, when the remaining period timer has not started counting (when the posture assimilation process is started and the remaining period update process is executed for the first time), the remaining period timer is set to the timer count value corresponding to the remaining period included in the posture assimilation information stored in step S300, and the timer count is started by subtracting the count value, whereas when the timer count has already started, the timer count value corresponding to a predetermined execution cycle (for example, a cycle corresponding to 30 frames / second, which is the screen update cycle) assigned to execution of the posture assimilation process in the performer terminals 200A,B is subtracted and updated.

[0336] Next, a remaining period display update process (step S304) is executed. The remaining period display update process updates the display of the time bar or the like, which is the display of the remaining period display area 252 on the posture assimilation screen, so as to correspond to the remaining period specified by the remaining period timer updated in the remaining period update process (step S303).

[0337] Next, a blackout performance display update process (step S305) is executed. In the blackout performance display update process, when the blackout performance execution flag is set in step S316 described below, the display "blackout performance in progress" is displayed in the blackout performance execution display area 256. By displaying the display "blackout performance in progress" in this manner, performer users A and B can recognize that a blackout performance in which their own position is not visible to the viewer user is being executed, and can proceed appropriately after the blackout performance in response to the execution of the blackout performance. In addition, in Example 3, a display of "Blackout effect in progress" is displayed, but the present invention is not limited to this, and a display of "Object hiding effect in progress" may be displayed when an object hiding effect is being executed, or a display of "Invisibility effect in progress" may be displayed during the period when either a blackout effect or an object hiding effect is being executed.

[0338] Next, a voice input-related process (step S306) is executed. In the voice input-related process, if the posture assimilation type stored in step S300 is "0" which controls the voice to be in a muted state, that is, if the transition period is a posture assimilation type in which the transition is from a program (pre-recorded) to an MC (not pre-recorded), the voice is controlled to be in a muted state (input disabled state) and a display of "microphone muted" is displayed in the muted display area 255. In this way, by displaying the display of "microphone muted", the performer users A and B can recognize that their voices are not distributed, and can perform a preparatory movement for the final posture, for example, by performing a dance while actually singing along with the singing of the pre-recorded program projected on the screen S. In addition, in order to make it easier for the performer users A and B to sing along with the singing of the program, the performer users A and B may be made to wear wireless earphones or the like, and the performer users A and B may be able to listen to the sound of the virtual live venue including the sound of the program through the wireless earphones or the like. Furthermore, since performer users A and B are able to assume the pose at the end of the performance by dancing along with their singing, wireless earphones or the like that allow performer users A and B to listen to the audio from the virtual live venue, including the audio of the performance, also constitute one means for performer users A and B to assume the pose at the end of the performance.

[0339] Next, a performer posture display update process (step S307) is executed. In the performer posture display update process, the postures of performer users A and B acquired by motion scanning are reflected in performer avatars 1A and 1B, and rear images of performer avatars 1A and 1B reflecting the postures are generated and displayed in current pose display area 251.

[0340] Next, it is determined whether or not a correction operation in progress flag, which is set in step S319 described later, is set (step S308). If the correction operation in progress flag is not set, the process proceeds to a posture comparison process in step S310.

[0341] On the other hand, if the correction action execution flag is set, the process proceeds to a correction information transmission process (step S309). In the correction information transmission process, correction information including posture data of a correction posture corresponding to the time point included in the correction action generated in time series in step S318 described later is transmitted to the delivery server computer 100. The corrected posture data included in the correction information transmitted in this manner is reflected in the postures of the performer avatars 1A, 1B in the delivery server computer 100 and delivered.

[0342] Next, a posture comparison process (step S310) is executed. In the posture comparison process, the postures (poses) of the performer avatars 1A and 1B updated in step S307 are compared with the postures (poses) of the performer avatars 1A and 1B of the target postures to be taken by the performer users A and B displayed in the pre-recorded pose display area 250 to calculate the degree of agreement. In the comparison with the postures (poses), not only the degree of agreement is calculated by comparing the overall postures of the avatars, but also the degree of agreement is calculated for each of the right arm, the left arm, the right leg, and the left leg in the third embodiment. In addition, for the determination of the degree of agreement between these postures, a known method can be used, such as a method of determining the degree of agreement by image processing using, for example, the outline extraction of the performer avatars 1A and 1B. However, it is preferable to use a method for these comparison processes that can be performed at a relatively high speed within the update period of the image layer, as described above.

[0343] Next, a posture evaluation information update process (step S311) is executed. In the posture evaluation information update process, the degree of agreement displayed in the degree of agreement display area 253 is updated to the overall degree of agreement calculated in the posture comparison process (step S310), the display color (blue, yellow, red) of the frame of the current pose display area 251 is updated to a color corresponding to the degree of agreement, and the display color (blue, yellow, red) of the indicator unit 260 is updated to a color corresponding to each degree of agreement calculated in the posture comparison process (step S310) for each of the right arm, left arm, right leg, and left leg.

[0344] In Example 3, when the degree of match is 95% or more, the display color is blue, when the degree of match is less than 60%, the display color is red, and when the degree of match is between 60% and less than 95%, the display color is yellow; however, the present invention is not limited to this, and these colors, the number of types of colors, and the range of degrees of match corresponding to each color may be changed appropriately depending on the type of avatar, performance, MC, etc.

[0345] In this way, not only the degree of matching numerical value but also the color of the frame in the current pose display area 251, which attracts a lot of attention from performer users A and B, allows them to intuitively grasp the matching status of the postures. This makes it easier for performer users A and B to assimilate their own postures with the target posture and also allows them to intuitively recognize which part of the body - the right arm, the left arm, the right leg, or the left leg - needs to be corrected.

[0346] Next, it is determined whether or not that time is posture determination timing for determining whether or not there is a possibility that performer users A and B can perform the target posture without any problems (step S312). Whether or not it is posture determination timing is determined based on whether the timer value of the remaining period timer, which is stored as the determination timing in step S300, has reached the corresponding timer value. Note that in the third embodiment, as shown in Figs. 34 and 35, the determination timing is set to the time when the remaining period is as short as 1 second, but these determination timings may be changed as appropriate depending on the avatar, the performance, the state of the MC, etc.

[0347] If it is not the posture determination timing, the process returns to step S303. On the other hand, if it is the posture determination timing, it is determined whether the degree of match displayed in the degree of match display area 253 in the posture evaluation information update process (step S311) is equal to or greater than a match determination value (e.g., 95% (blue)) (step S313).

[0348] If the degree of match is equal to or greater than the match determination value, the process returns to step S303. On the other hand, if the degree of match is not equal to or greater than the match determination value, the process further determines whether the degree of match is less than a limit value (for example, less than 60% (red)) (step S314).

[0349] If the degree of agreement is less than the limit value, the process proceeds to step S315, where it is determined that poor posture makes it difficult to achieve the target posture within the remaining period, and posture assimilation information indicating poor posture is sent to distribution server computer 100 (step S315), and a blackout effect execution flag is set (step S316), after which the process returns to step S303. By sending the posture assimilation information in this manner, as described above, a blackout effect is executed in distribution server computer 100, in which performer avatars 1A, 1B become invisible.

[0350] If the degree of agreement is not below the limit value, the process proceeds to step S317, where it is determined whether the posture assimilation type is "1", that is, whether the type is a transition from MC (not pre-recorded) to program (pre-recorded).

[0351] If the posture assimilation type is not "1", that is, if the program is moving from a program (pre-recorded) to an MC (not pre-recorded), the process returns to step S303. On the other hand, if the posture assimilation type is "1", a correction movement generation process (step S318) is executed to generate movements to correct the postures of performer users A and B.

[0352] In this correction action generation process, a movement that changes from the posture of the performer users A and B at the posture determination timing to a target posture during the remaining period is generated in a time series. Specifically, as shown in FIG. 46, a target posture (a) is set as the posture at the final time point of the remaining period, and the target posture (a) is compared with a current posture (b), and the intermediate posture is set as a primary intermediate posture at the intermediate time point (primary intermediate time point) of the remaining period. Then, a secondary A intermediate posture is further generated between the primary intermediate posture (c) and the current posture (b), and the secondary A intermediate posture is set as the posture at the secondary A intermediate time point between the posture determination timing and the intermediate time point. Similarly, a secondary B intermediate posture is further generated between the primary intermediate posture (c) and the target posture (a), and the secondary B intermediate posture is set as the posture at the secondary B intermediate time point between the final time point and the intermediate time point of the remaining period. In this way, the intermediate postures are generated in tertiary, quaternary, quinary, etc. in sequence, and the generated times are set as the postures in the corresponding remaining period, thereby generating a correction action.

[0353] In the third embodiment, as shown in FIG. 46, a form in which a corrective movement is generated by successively generating intermediate postures is exemplified; however, the present invention is not limited to this. For example, a corrective movement may be generated by inputting a target posture (a), a current posture (b), and a movement period into a known motion morphing program.

[0354] Then, the process proceeds from the correction movement generation process to step S319, where the correction movement execution flag is set, and then the process returns to step S303. Note that, as a result of the correction movement execution flag being set in step S319, the correction information transmission process of the above-mentioned step S309 is executed, and the postures at each point in time of the correction movement generated in the correction movement generation process (step S318) are transmitted as correction information to distribution server computer 100, so that the correction movements are forcibly reflected in performer avatars 1A and 1B.

[0355] As the above posture movement processing is executed by performer terminals 200A and 200B, the flow of transition period 5 in which the posture movement processing is executed is as shown in Fig. 45. Note that Fig. 45 shows transition period 5, and therefore the case of performer user A, but the same is true for transition period 9, which is the transition period from performer user B's MC to the performance, and also for transition period 12 when performer user A and performer user B both appear. However, in the case of transition period 12, if the posture of either performer user A or performer user B is a poor posture with a matching degree of less than 60, a blackout effect is executed even if the posture of the other is good.

[0356] If performer user A's posture is a good posture with a matching rate of 95% or more at the posture determination timing of transition period 5, then no posture correction or blackout effect is performed, and performer user A's posture is reflected in performer avatar 1A, and the performance period begins (Figure 45 (a1)).

[0357] On the other hand, if performer user A's posture at the posture determination timing of transition period 5 is a slightly bad posture with a matching degree in the range of 60 to 94% as shown in Fig. 45(b), a correction action is generated in performer terminal 200A to change the posture at the posture determination timing of Fig. 45(b1) to the posture at the start of the performance of Fig. 45(b3), and the generated correction action is reflected in performer avatar 1A as shown in Fig. 45(b2), so that the posture at the end of the MC period (end of the transition period) is made the same as the posture at the start of the performance, and the performance period (pre-recording) starts. In this way, if performer user A's posture is a slightly bad posture that can be corrected by correction, a correction action is generated and reflected in performer avatar 1A, so that the position of the raised hand of performer avatar 1A is low at the end of the MC period, and therefore it is possible to prevent a discontinuous movement in which the position of the raised hand of performer avatar 1A suddenly becomes high at the start of the performance period, which would be unnatural, and thus reduce the interest of the content.

[0358] Furthermore, if the posture of performer user A at the posture determination timing of transition period 5 is a poor posture with a matching degree of less than 60, as shown in Figure 45(c), specifically, for example, if the posture is with the left and right hands raised in opposite directions, a blackout effect is executed as shown in Figure 45(c2) from the posture determination timing to the end of the MC period (the end of the transition period), making the movements (posture) of performer avatar 1A invisible. This prevents the hand of performer avatar 1A from suddenly being raised in the wrong direction at the start of the performance period, which would look unnatural and would reduce the interest of the content.

[0359] As described above, in the virtual space content distribution system of Example 3, when a non-pre-recorded content MC (first action content) and a pre-recorded content performance (second action content) are distributed in succession, during the transition period, which is a predetermined period for switching between the MC (first action content) and the performance (second action content), virtual space content is distributed in which a specific effect (blackout effect or object hiding effect) is executed that makes it difficult for the viewer user to recognize the posture of performer avatars 1A and 1B.This makes it difficult for the viewer user to recognize the posture of performer avatars 1A and 1B, thereby preventing a decrease in interest in the virtual space content caused by the viewer feeling uncomfortable due to the large difference in posture of performer avatars 1A and 1B during the switch.

[0360] Furthermore, the performer terminals 200A, B of the virtual space content distribution system of the above-mentioned Example 3 are equipped with a display device C212 which serves as a posture presentation means capable of presenting a posture corresponding to the first movement of the performer avatars 1A, 1B in a performance (second movement content) as a target posture to be taken by the performer users A, B at the end of the MC (first movement content) when the performance (second movement content) is distributed following the MC (first movement content). This allows the performer users A, B to take the presented target posture at the end of the MC (first movement content) in which their own movement is reflected in the performer avatars 1A, 1B. This prevents a decrease in interest in the virtual space content caused by the viewer feeling uncomfortable because the posture of the performer avatars 1A, 1B at the start of the performance (second movement content) is significantly different from that at the end of the MC (first movement content).

[0361] In the virtual space content delivery system of the third embodiment described above, when a performance (second action content) is delivered following an MC (first action content), posture assimilation processing is executed at the end of the MC (first action content), including a posture assimilation screen display process, a remaining period update process, a remaining period display update process, a performer posture display update process, a posture comparison process, a posture evaluation information update process (first process) for making the performer users A and B take the initial start postures of the performer avatars 1A and 1B in the performance (second action content), and a correction action generation process (second process) for correcting the difference between the start postures and the postures of the performer users A and B. When the MC (first action content) is delivered following a performance (second action content), during a transition period (predetermined period) in switching between the MC (first action content) and the performance (second action content), the postures of the performer avatars 1A and 1B are assimilated by the viewer users. In order to execute specific effect processing (blackout effect reflection processing and object effect reflection processing) for executing specific effects (blackout effect and object hiding effect) that are difficult to recognize, when a performance (second action content) is distributed following an MC (first action content), the postures of performer users A and B reflected in performer avatars 1A and 1B are assimilated to the starting posture of performer avatars 1A and 1B in the performance (second action content), and when an MC (first action content) is distributed following a performance (second action content), the execution of specific effects makes it difficult for viewer users to recognize the postures of the performer avatars. This prevents a decrease in interest in the virtual space content caused by the viewer feeling uncomfortable because the postures of performer avatars 1A and 1B at the start of distribution of the MC (first action content) are significantly different from their postures at the end of the performance (second action content).

[0362] Furthermore, in the virtual space content distribution system of the above-mentioned Example 3, when an MC (first action content) is distributed following a performance (second action content), a posture assimilation process is executed at the start of the MC (first action content) to cause performer users A and B to assume the final ending posture of performer avatars 1A and 1B in the performance (second action content).By doing so, when an MC (first action content) is distributed following a performance (second action content), performer users A and B can be made to assume the final ending posture of performer avatars 1A and 1B in the performance (second action content) at the start of the MC (first action content).This makes it possible to prevent a decrease in interest in the virtual space content caused by the viewer feeling uncomfortable because the posture of performer avatars 1A and 1B at the start of distribution of the MC (first action content) is significantly different from the final posture at the end of the performance (second action content).

[0363] Furthermore, in the virtual space content distribution system of the above-mentioned third embodiment, when an MC (first action content) and a performance (second action content) are distributed consecutively, specific processes including a posture assimilation process which is the first specific process for preventing the viewer user from recognizing that the postures of the performer avatars 1A, 1B are not continuous when the action content is switched, and a blackout effect reflection process and an object effect reflection process which are the second specific processes are executed to prevent the viewer user from recognizing that the postures of the performer avatars 1A, 1B are in a significantly different and discontinuous state when the action content is switched, thereby preventing a decrease in interest in the virtual space content caused by the viewer feeling uncomfortable due to the significantly different postures of the performer avatars 1A, 1B when the action content is switched.

[0364] <Modification of the third embodiment> Fig. 47 is a diagram showing a modified example 3-1 of the embodiment 3. As described above, in the embodiment 3, the performer avatars 1A and 1B are made invisible by darkening the screen display of the viewer terminal 300, but the screen that makes the performer avatars 1A and 1B invisible is not limited to the darkened screen, and the performer avatars 1A and 1B may be made invisible by executing a loading screen display effect that displays a loading screen indicating that new data is being loaded, as shown in Fig. 47. In this way, the viewer user recognizes that it is a time point when new data needs to be loaded in order to proceed with the virtual live performance, and is less likely to feel uncomfortable about the performer avatars 1A and 1B becoming invisible, which further prevents a decrease in interest in the content.

[0365] In addition, although a loading screen is shown as an example in Figure 47, the present invention is not limited to this, and the screen that makes the performer avatars 1A, 1B invisible may be a black screen or a screen other than the loading screen.

[0366] Furthermore, these screens do not have to be screens that hide the entirety of the performer avatars 1A and 1B, but may be, for example, screens in which an object used in an object concealment performance hides parts of the performer avatars 1A and 1B other than their faces, that is, screens that hide the movements (postures) of the performer avatars 1A and 1B.

[0367] 48 is a diagram showing a modified example 3-2 of the embodiment 3. As described above, in the embodiment 3, a blackout effect or an object hiding effect is executed as an effect during the transition period to make the performer avatars 1A and 1B invisible, thereby preventing the viewer user from feeling uncomfortable due to discontinuous postures, but the present invention is not limited to this, and instead of hiding the performer avatars 1A and 1B so that the viewer user cannot see them, an effect may be executed to move the viewer user's viewpoint to a position other than the performer avatars 1A and 1B, thereby guiding the viewer user not to see the movements of the performer avatars 1A and 1B.

[0368] Specifically, as shown in Fig. 48, when the transition period starts, as shown in Fig. 48(d), a performance is executed in which the size of a virtual arrangement object 43A of a star arranged on a virtual stage G increases, so that the viewpoints of the viewer users are focused on the virtual arrangement object 43A and the viewpoints of the viewer users are diverted from the performer avatars 1A and 1B. At this time, the viewpoints of the viewer users are guided to focus on the virtual arrangement object 43A as a common viewpoint image of a virtual camera, for example, a virtual camera C1, which is a virtual camera whose viewpoint can be managed by the event manager M, instead of the viewpoint of the viewer avatar. Then, as shown in Figure 48(e), by creating a common viewpoint image of virtual camera C1 that approaches the enlarged virtual arrangement object 43A as the virtual arrangement object 43A enlarges, a viewpoint guiding effect is implemented in which the common viewpoint image does not include the body parts of performer avatars 1A, 1B, and as virtual arrangement object 43A gradually shrinks, the image becomes one in which the heads of the body parts of performer avatars 1A, 1B become visible, and then an MC period can be started in which the entire bodies of performer avatars 1A, 1B are visible.

[0369] Fig. 49 is a diagram showing a modified example 3-3 of the embodiment 3. As described above, in the embodiment 3, the performer users A and B are made to take target postures, which are the postures at the end and start of the pre-recorded performance, during the transition period. However, instead of making the performer users A and B take the target postures, similar to the case where a correction motion is generated in the above-mentioned correction motion generation process, the performer terminal 200 or the delivery server computer 100 may execute a transition period motion generation process that automatically generates a transition period motion that changes from the posture in Fig. 49(c) to the posture in Fig. 49(f) based on the target posture (the posture at the start of the performance in the example shown in Fig. 49) shown in Fig. 49(f) from the posture at the start of the transition period of the performer users A and B at the start of the transition period shown in Fig. 49(c) and the length of the transition period, and the transition period motion generated in the transition period motion generation process may be reflected in the performer avatars 1A and 1B. Furthermore, the transition period movement generation process can be performed by inputting the target posture shown in Figure 49(f) and the posture at the start of the transition period and the transition period shown in Figure 49(c) into the publicly known motion morphing program described above to generate transition period movements.

[0370] Furthermore, the form of generating a transition period movement can be used to great effect particularly when there is a high probability that one of performer users A and B will not be able to properly assume the starting pose, such as in transition period 1 of Example 3, where performer users A and B both need to assume the starting pose in an unfamiliar situation at the start of a virtual live performance.

[0371] Furthermore, when performer users generate transition period actions in a virtual live performance for a group of three or more performers, if the transition period action generation process for generating these transition period actions is executed centrally on the distribution server computer 100, the processing load during the transition period on the distribution server computer 100 will become excessively large, which may disrupt distribution. Therefore, it is preferable to execute these transition period action generation processes on each performer terminal 200, and transmit each posture of the transition period action generated on each performer terminal 200 to the distribution server computer 100 as performer avatar information.

[0372] In addition, in the above-mentioned Example 3, when the posture assimilation process is executed during the transition period from MC to performance, the voices of performer users A and B are distributed even during the transition period, but the present invention is not limited to this. During the transition period from MC to performance, in order to enable performer users A and B to take the target posture without worrying about their voices, the microphones may be muted so that the voices of performer users A and B are not distributed, as in the case of going from performance to MC shown in Figure 35. In this way, when the microphones are muted, specific voices such as message voices of performer users A and B that have been recorded in advance may be distributed.

[0373] In addition, in the above-mentioned Example 3, in the posture assimilation process, if the posture matching rate is less than 60% when the posture determination timing is reached, only a blackout effect is executed, which allows for rapid processing. However, the present invention is not limited to this, and instead of or in addition to these blackout effects, other effects, such as object hiding effects, may be executed in which the movements of the performer avatars 1A, 1B are made invisible to the viewer user.

[0374] In addition, in the above-mentioned Example 3, four indicator sections 260 corresponding to the limbs are displayed on the posture assimilation screen so that performer users A and B can easily understand the parts that differ from the target posture, but the present invention is not limited to this, and a form in which these indicator sections 260 are not displayed may also be used.

[0375] In the above-mentioned Example 3, the posture assimilation screen does not display the posture assimilation status of performer avatars performing together. However, for example, in a case where performer avatars 1A and 1B both assume the initial posture at the start of the performance, such as in transition period 1, it may be possible to notify by displaying on display device C that the postures of performer users A and B both have a degree of similarity of 95% or more.

[0376] Furthermore, rather than simply notifying that the postures of both performer users A and B are within a specified range of 95% or more as described above, for example, information on the degree of similarity may be sent from performer user B's performer terminal 200B to performer terminal 200A, and the degree of similarity of performer user B's posture assimilation may be updated and displayed on display device C of performer user A's performer terminal 200A, and information on the degree of similarity may be sent from performer user A's performer terminal 200A to performer terminal 200B, and the degree of similarity of performer user A's posture assimilation may be updated and displayed on display device C of performer user B's performer terminal 200B, allowing performer users A and B to continuously grasp the posture assimilation state of the other performer avatar with whom they are performing together.

[0377] Furthermore, in the above-mentioned Example 3, the performances are exemplified as including both singing and dancing, but the present invention is not limited to this, and these performances may include only singing or only dancing, or may include acting, comedy sketches, etc. other than singing and dancing.

[0378] Furthermore, in the above-mentioned Example 3, when a blackout effect or object hiding effect is executed during a transition period, performer users A and B can assume any posture, so an example is shown in which the posture assimilation screen is not displayed on display device C of performer terminal 200; however, the present invention is not limited to this, and for example, even during a transition period in which a blackout effect or object hiding effect is executed, a posture assimilation screen in which the target posture is not displayed may be similarly displayed to allow the user to recognize the period until the start of the MC or performance, and to notify the user that a blackout effect or object hiding effect is being executed by displaying the posture assimilation screen.

[0379] In addition, in the above-mentioned Example 3, an example is given of a form in which, during the transition period when transitioning from a performance to an MC, an object hiding effect is mainly executed, and when transitioning from an MC to a performance, a blackout effect is executed based on the increase in the number of performer avatars; however, the present invention is not limited to this, and, such as executing a blackout effect regardless of the number of performer avatars during the transition period when transitioning from a performance to an MC, and executing an object hiding effect when transitioning from a performance to an MC, by executing different effects (specific effects) when transitioning from a performance to an MC and when transitioning from a performance to an MC, it is possible to make it easier for the viewer user to grasp the transition situation; however, the present invention is not limited to this, and the same effect (specific effect) may be executed when transitioning from a performance to an MC and when transitioning from a performance to an MC.

[0380] In addition, in the above-mentioned Example 3, an example is given of an object hiding performance in which different objects are used for performer avatars 1A and 1B, but the present invention is not limited to this, and an object hiding performance may be performed using a virtual object common to performer avatars 1A and 1B (for example, an object with a unit name) as these objects.

[0381] In addition, in the above-mentioned Example 3, an example was given of an object concealment performance in which objects gather around performer avatars 1A, 1B and cause performer avatars 1A, 1B to become hidden, and then scatter to make performer avatars 1A, 1B visible. However, the present invention is not limited to this, and for example, a performance form in which performer avatars 1A, 1B are absent when the gathered objects scatter, and then performer avatars 1A, 1B appear in changed costumes when the objects gather again and scatter.

[0382] In addition, in the above-mentioned Example 3, an example is given of a form in which the target postures are presented to the performer users A and B by displaying them on the display device C in the performer terminals 200A and B, but the present invention is not limited to this, and these target postures may also be presented by voice, for example, such as "close your feet and raise your right hand," or the positions of the hands and feet in the target postures may be indicated in space using a laser marker or the like.

[0383] In the above-mentioned third embodiment, the object hiding effect is executed only when moving from a performance to an MC, and is not executed when moving from an MC to a performance, but the present invention is not limited to this, and the object hiding effect may be executed when moving from an MC to a performance. Note that these object hiding effects may be executed when a posture action process is executed when moving from an MC to a performance, and the posture of performer users A and B in the posture assimilation process is poor.

[0384] In addition, in the above-mentioned Example 3, an example is given of the transition period being the period immediately before the end of a program or MC, and being included in the program period or MC period, but the present invention is not limited to this, and these transition periods may be set between the program period and the MC period, with the program period and the MC period being separate periods.

[0385] Furthermore, the transition period may be a period that includes the point at which the performance and the MC are switched over, and that spans the performance period and the MC. In this case, for example, when transitioning from a performance (pre-recorded) to an MC, a posture determination timing may be set just before the end of the performance (for example, 0.5 seconds before the end of the performance), and if the degree of similarity at the posture determination timing is a limit value (for example, less than 60% (red)), the transition period may be extended into the MC period, and a blackout effect may be executed for the first second of the MC period, so that the performance is not interrupted by a blackout effect in the middle of the performance; or, if the degree of similarity at the posture determination timing is between the limit value of 60% and less than 95%, a corrective action may be generated and executed in the first few seconds of the MC period, thereby preventing the postures of performer avatars 1A, 1B from suddenly changing significantly, causing discomfort to the viewer user.

[0386] The transition period may also be a specified period at the beginning of a transitioning performance or MC, and blackout effects, object hiding effects, posture assimilation processing, etc. may be executed during the specified period at the beginning of a transitioning performance or MC.

[0387] In addition, in the above-mentioned third embodiment, in the posture assimilation process, a blackout effect or a correction action generation process is executed depending on the result of the posture determination timing, but the present invention is not limited to this, and a form in which these blackout effects and correction action generation processes are not executed may also be used.

[0388] Also, although not specifically implemented in the above-mentioned Example 3, when performer terminals 200A, B transmit posture assimilation information to execute a blackout performance, it is possible not only to notify that the blackout performance is being executed, but also to indicate the correct positions where performer users A, B should be located, for example, using racer markers or lighting, so that they can begin to move in the correct positions when the blackout performance ends.

[0389] Furthermore, when the correction actions are forcibly reflected in the movements of the performer avatars 1A and 1B in the above-mentioned Example 3, no particular restrictions are placed on the viewpoint of the viewer user. However, as is implemented in the viewpoint guidance performance of the above-mentioned variant example 3-2, during the period when these correction actions are forcibly reflected in the movements of the performer avatars 1A and 1B, it may be possible to restrict the viewpoint to only the common viewpoint image of a virtual camera whose viewpoint the event manager M can manage, for example, virtual camera C1.

[0390] In addition, in the above-mentioned Example 3, an example is given of a form in which a posture assimilation screen is displayed on the display device C212, but the present invention is not limited to this, and these posture assimilation screens may be displayed on a part of the screen S or on a head-mounted display (HMD) worn by the performer user.

[0391] In the above-mentioned third embodiment, a form of pre-recording is exemplified in which performer motion data for a performance, which describes the motions of performer users A and B in chronological order, is stored in distribution server computer 100, but the present invention is not limited to this, and instead of reflecting this performer motion data for a performance in the motions of performer avatars 1A and 1B, video of performer avatars 1A and 1B reflecting the motions of performer users A and B for the performance may be recorded and stored, and the recorded and stored video may be distributed. Note that distribution of these recorded and stored videos may be applied to all performances, or may be applied only to some performances, and the performer motion data for the performance may be used for the other performances.

[0392] In addition, in the above-mentioned Example 3, an example is given of a form in which performer movement data for a performance is stored in the distribution server computer 100, thereby enabling the performance to be distributed smoothly even if there is a communication failure with the performer terminals 200A, B, etc. However, the present invention is not limited to this, and this performer movement data for a performance may be stored in the performer terminals 200A, B and transmitted to the distribution server computer 100 as performer avatar information for the performer user during the performance period.

[0393] In addition, in the above-mentioned Example 3, an example was given of a form in which the blackout effect reflection process and the object effect reflection process are executed in the distribution server computer 100, and the posture assimilation process is executed in the performer terminals 200A, B, but the present invention is not limited to this, and each of these processes may be executed in cooperation between the distribution server computer 100 and the performer terminals 200A, B.

[0394] In addition, in the above-mentioned Example 3, an example was given in which the MC is action content (not pre-recorded) in which the actions of performer users A and B are reflected in the actions of performer avatars 1A and 1B in almost real time, and the performance is action content (pre-recorded) in which the actions of performer users A and B are not reflected in the actions of performer avatars 1A and 1B in almost real time, but the present invention is not limited to this, and the MC may be action content (pre-recorded) and the performance may be action content (not pre-recorded) in which the actions of performer users A and B are reflected in the actions of performer avatars 1A and 1B in almost real time.

[0395] In addition, in the above-mentioned Example 3, an example was given of a form in which an MC and a program are executed alternately, but the present invention is not limited to this, and other content other than the MC and the program (e.g., a game) may be distributed, and this other content may be either pre-recorded or not pre-recorded.

[0396] In addition, in the above-mentioned Example 3, an example is given in which the entire repertoire is motion content (pre-recorded) and the entire MC is motion content (not pre-recorded), but the present invention is not limited to this, and part of the repertoire may be motion content (not pre-recorded) and part of the MC may be motion content (pre-recorded).

[0397] In addition, in the above-mentioned Example 3, an example is given of a form in which MC is distributed during different performance periods, but the present invention is not limited to this, and for example, an MC period may be set during the same performance period, or a short performance period may be set during the same MC period.

[0398] Furthermore, although not described in detail in the third embodiment, a flying performance period, such as that of the first embodiment, may be provided during each performance, and the configurations of the first and second embodiments can be applied to the third embodiment.

[0399] The present invention has been described above based on each embodiment and modified examples, but the specific configuration is not limited to the above-mentioned embodiments and modified examples, and the present invention also includes modifications and additions that do not deviate from the gist of the present invention.

[0400] For example, in the above embodiments and modifications, the viewer avatar viewpoint video is generated and displayed on the viewer terminal 300 side, but the present invention is not limited to this, and for example, in the case of a live event with few virtual participants, the viewer avatar viewpoint video may also be generated and distributed on the distribution server computer 100 side. Also, the viewer user may be allowed to select whether the viewer avatar viewpoint video is generated on the viewer terminal 300 side or on the distribution server computer 100 side depending on the processing capacity of the terminal he / she owns and the available data communication environment, or the distribution server computer 100 may specify the processing capacity of the viewer terminal 300 with which it is connected for communication, and determine whether the viewer terminal 300 side or the distribution server computer 100 side is generated based on the specified processing capacity.

[0401] In addition, in each of the above embodiments and modified examples, an example is given of the distribution server computer 100 being installed in a studio operated by the event management organization that hosts the virtual live performance, but the present invention is not limited to this, and as the distribution server computer 100, a server computer owned in a data center or the like by the company that rents out the server computer may be used, or as described above, a server computer provided by a cloud service may be used, and the installation form of these server computers may be any form that provides the functions of the distribution server computer 100.

[0402] Furthermore, in each of the above embodiments and modified examples, a performer terminal 200 is provided for each performer user in a one-to-one correspondence, but the present invention is not limited to this, and multiple people may share one performer terminal 200. If the performer user is a unit or group consisting of multiple members, for example, a viewpoint tracking target designation process may be executed in at least one of the distribution server computer 100 and the viewer terminal 300, allowing the viewer user to select a member to follow the viewpoint.

[0403] In addition, while the above embodiments and modified examples illustrate a form in which a performer user sings, dances, and other performance actions are performed by one person, the present invention is not limited to this, and the singer who sings and the actor who performs the dancing actions may be separate persons, in which case the singer's performer terminal and the actor's performer terminal may be separate, and the singer may sing in a location other than the studio where the actor performs the performance actions.

[0404] In addition, in each of the above embodiments and variant examples, the Internet network is used as an example of the computer communication network connecting the distribution server computer 100, the administrator terminal 150, the performer terminal 200, and the viewer terminal 300, but the present invention is not limited to this, and these communication networks may be composed of local area networks capable of data communication only within a specific area, or may include a local area network and local data communication as part of them.

[0405] In addition, in each of the above embodiments and variant examples, the viewer terminal 300 is exemplified as a smartphone P or a computer (PC), but the present invention is not limited to this, and these computers may be portable laptops, tablet terminals, e-book readers, wearable computers, game consoles, and various other information processing devices capable of outputting at least 2D video and audio.

[0406] In addition, in each of the above embodiments and variant examples, a performer user performs in a studio, including singing, but the present invention is not limited to this. For example, a performer user may use a performer terminal 200 installed in a home or the like to appear in a virtual live performance from a home or the like.

[0407] Furthermore, in each of the above embodiments and variant examples, a virtual live performance has been given as an example of virtual space content, but the present invention is not limited to this, and such virtual space content may be any content that is beneficial to viewer users, such as a fan meeting where performer users gather with their fans, or a talk session, and that is related to a virtual space in which performer avatar 1 and viewer avatars virtually participate.

[0408] In addition, in each of the above embodiments and modified examples, an example is given in which viewpoint switching of the four virtual cameras C1 to C4 as fixedly positioned (set) virtual cameras or the airborne moving virtual camera can be performed in advance by settings and operations on the administrator terminal 150, but the present invention is not limited to this, and for example, these viewpoint switching may be performed in real time by the event manager M on the administrator terminal 150 using a viewpoint switching controller or the like constituting the operation input device 161.

[0409] Furthermore, in each of the above embodiments and modified examples, an example is given of the use of virtual cameras C1 to C4 fixedly positioned (set) in the virtual space as the virtual cameras, but the present invention is not limited to this. For example, in addition to or instead of the virtual cameras C1 to C4, a movable virtual moving camera may be placed in the virtual live venue, and the position and imaging direction of the virtual moving camera in the virtual space may be controlled by the event manager M, a cameraman, or the like through operation of the manager terminal 150 to produce the viewpoint image of the virtual moving camera.

[0410] In addition, in the above embodiments and modified examples, a form is exemplified in which the viewpoint image of the virtual camera C2 is displayed on the screen S so that the facial expression of the performer user can be detected to be reflected in the facial expression of the performer avatar, but the present invention is not limited to this, and for example, if the performer user has a high-performance head-mounted display with a sensor function to detect the facial expression of the performer user, the performer user may wear such a high-performance head-mounted display to perform the performance, which may improve the performer user's sense of immersion in the virtual live performance. Also, in cases where the facial expression of the performer user does not need to be reflected in detail in the facial expression of the performer avatar, the performer user may wear a normal head-mounted display to perform the performance.

[0411] In addition, instead of the performer user's singing or singing actions (performance), the singing and singing actions (performance actions) of a virtual performer created using computer graphics, etc., can be recorded as performer avatar information on a distribution server computer, and a virtual live performance can be held by playing back the recorded singing voice and singing actions (performance actions) as the live performance progresses.

[0412] In this way, when singing or singing actions (performance actions) are performed in advance to record and play back performer avatar information on the distribution server computer 100 to distribute a virtual live performance, the configuration of the virtual space content distribution system at the time of distribution will be made up of the distribution server computer 100, an administrator terminal 150, and a viewer terminal 300, and will not include a performer terminal 200, which is a performer user terminal that can be used by a performer user to perform operations to move the performer avatar 1 in the virtual space.

[0413] In addition, the "start time of the first action content", "end time of the first action content", "start time of the second action content", and "end time of the second action content" in the present invention correspond to the "start time of the MC period", "start time of the MC period", "start time of the program period", and "end time of the program period" in the third embodiment and each modification, but these "start times of the MC period" may be before the talk in the MC actually starts, and similarly, the "end time of the MC period" may be a time when a certain period of time has passed since the talk in the MC ended. Similarly, the "start time of the program period" may be a timing before the dance or music playback of the program actually starts, or may be a timing just before the dance or singing starts after the music playback (intro) starts, and the "end time of the program period" may be a timing when a certain period of time has passed since the dance or singing of the program ended and there is silence, or conversely, it may be a timing when the dance or singing of the program has ended but the ending music is still being played, and these times (timings) may be appropriately determined according to the aspect of the program and MC.

[0414] In the present invention, "switching of content" refers to the end or start of reflecting the movements of the performer user in the movements of the performer avatars 1A and 1B in almost real time.

[0415] In addition, movements in this invention generally refer to a change in posture over a unit period of time, and include multiple postures that are consecutive in a time series; however, in this invention, these movements also include stationary movements in which the posture does not change over a certain period of time, and in this case, the stationary movement and the posture are the same thing.

[0416] In addition, in the present invention, "use by the performer" does not mean operation by the performer user, but also includes use by the performer user simply to view virtual space content. [Explanation of symbols]

[0417] 1. Performer Avatar 100 Distribution server computer 150 Administrator terminal 200 Performer terminals 300 Viewer terminals

Claims

1. A virtual space content delivery system capable of delivering virtual space content including at least an action content in which a performer avatar acts in a virtual space, comprising: a performer terminal that can be used by a performer corresponding to the performer avatar; A viewer user terminal on which a viewer user can view the virtual space content; a server computer connected to the performer terminal and the viewer user terminal via a communication network and capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; Including, the action content includes a first action content in which the action of the performer inputted to the performer terminal is reflected in the performer avatar when the action content is distributed, and a second action content recorded in advance before the action content is distributed; When the first action content and the second action content are delivered in succession, a specific process is executed to prevent a viewer user from recognizing that the posture of the performer avatar is not continuous when switching between action contents, the process including a blackout process for blacking out the action content displayed on the viewer user terminal. A virtual space content delivery system comprising:

2. When the second operation content is to be distributed following the first operation content, a first identification process is executed as the identification process; When the first operation content is distributed following the second operation content, a second specific process different from the first specific process is executed as the specific process.

2. The virtual space content delivery system according to claim 1.

3. the first identification process includes a posture assimilation process for making the performer assume an initial starting posture of the performer avatar in the second action content at the end of the first action content; The second specific processing includes a specific performance processing for executing a specific performance that makes it difficult for a viewer user to recognize the movement of the performer avatar, 3. The virtual space content delivery system according to claim 2.

4. The starting posture is stored in advance, The performer terminal includes a posture display means for displaying the starting posture so that the performer can visually recognize the starting posture.

4. The virtual space content delivery system according to claim 3.

5. the first specific processing includes a specific performance processing for executing a specific performance in which a posture of the performer avatar is difficult for a viewer user to recognize, The second identification process includes a posture assimilation process for making the performer assume a final end posture of the performer avatar in the second action content at the start of distribution of the first action content.

3. The virtual space content delivery system according to claim 2.

6. The end posture is stored in advance, The performer terminal includes a posture display means for displaying the end posture so that the performer can visually confirm the end posture.

6. The virtual space content delivery system according to claim 5.

7. The posture display means is capable of displaying the posture of the performer avatar as a viewpoint image from behind the performer avatar.

7. The virtual space content delivery system according to claim 4 or 6.

8. The posture display means is capable of displaying period information capable of specifying a period until the start of distribution of the second action content or a period until the start of distribution of the first action content.

7. The virtual space content delivery system according to claim 4 or 6.

9. The posture assimilation process includes a process of identifying a difference between the posture displayed by the posture display means and the posture of the performer, and reflecting a correction action corresponding to the identified difference in the action of the performer avatar.

7. The virtual space content delivery system according to claim 4 or 6.

10. When a difference between the posture displayed by the posture display means and the posture of the performer is greater than a specified value at a determination timing when the remaining period until the distribution start timing of the second action content or the remaining period until the distribution start timing of the first action content becomes a predetermined specific period, the fact that the difference is greater than a specified value is notified on the performer terminal.

7. The virtual space content delivery system according to claim 4 or 6.

11. At a determination timing when the remaining period until the start timing of distribution of the second action content or the remaining period until the start timing of distribution of the first action content becomes a predetermined specific period, when a difference between the posture displayed by the posture display means and the posture of the performer is larger than a limit value, a specific performance is executed in which the posture of the performer avatar is difficult for a viewer user to recognize.

7. The virtual space content delivery system according to claim 4 or 6.

12. A virtual space content distribution program for distributing virtual space content including at least an action content in which a performer avatar acts in a virtual space, comprising: The action content includes a first action content in which the action of the performer inputted into the performer terminal when the action content is distributed is reflected in the performer avatar, and a second action content recorded in advance before the action content is distributed, the first action content being provided from a server computer that is connected via a communication network to a performer terminal that can be used by the performer corresponding to the performer avatar and to a viewer user terminal on which a viewer user can view the virtual space content, and that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; a process to be executed when the first action content and the second action content are delivered successively, the process including a specific process for preventing a viewer user from recognizing that the posture of the performer avatar is not continuous when the action content is switched; The specific process includes a darkening process for darkening the motion content displayed on the viewer user terminal. A virtual space content delivery program comprising:

13. A virtual space content distribution method for distributing virtual space content including at least an action content in which a performer avatar acts in a virtual space, comprising: The action content includes a first action content in which the action of the performer inputted into the performer terminal when the action content is distributed is reflected in the performer avatar, and a second action content recorded in advance before the action content is distributed, the first action content being provided from a server computer that is connected via a communication network to a performer terminal that can be used by the performer corresponding to the performer avatar and to a viewer user terminal on which a viewer user can view the virtual space content, and that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; a procedure to be executed when the first action content and the second action content are delivered successively, the procedure including a specific procedure for preventing a viewer user from recognizing that the posture of the performer avatar is not continuous when the action content is switched; The specific procedure includes a dimming procedure for dimming the motion content displayed at the viewer user terminal. A virtual space content distribution method comprising:

Images