Virtual space content distribution system, virtual space content distribution program, and virtual space content distribution method

Abstract

To suppress a decrease in interest in a virtual space content due to simplification of an avatar.SOLUTION: In a virtual space content distribution system capable of distributing a virtual space content in a virtual space, viewer avatars included in the virtual space content include a first viewer avatar and a second viewer avatar having a smaller processing load for display output than the first viewer avatar, the viewer avatar in the virtual space content is the second viewer avatar in a specific period during a distribution period of the virtual space content, and a viewer avatar of a viewer satisfying a special condition among the viewer avatars set as the second viewer avatar can be changed from the second viewer avatar to the first viewer avatar.SELECTED DRAWING: Figure 87

Description

[Technical Field]

[0001] The present invention relates to a virtual space content distribution system capable of distributing virtual space content in a virtual space, a virtual space content distribution program, and a virtual space content distribution method. [Background technology]

[0002] Conventionally, a system has been known in which multiple viewer terminals owned by viewers (viewer users) are connected to a server computer capable of executing processes related to virtual space content and processes related to the distribution of virtual space content via a communication network, and a virtual character (viewer avatar, first viewer avatar) representing the viewer can participate in a virtual space for distributing the virtual space content (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-186797 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, since a viewer avatar corresponding to each viewer user can participate in the virtual space, these viewer avatars that participate in the virtual space also need to be displayed and output as avatars participating in the virtual space of the virtual space content. However, as the number of viewer avatars participating in the virtual space increases, the processing load related to the display and output of the virtual space content increases, which may cause problems in the display and output of the virtual space content. Therefore, in order to prevent such problems from occurring, it is possible to simplify the aspects of the viewer avatars and use simplified avatars that reduce the processing load for display and output. However, if all viewer avatars were uniformly simplified avatars, there is a risk that the interest in the virtual space content would be reduced.

[0005] The present invention has been made in response to such problems, and aims to provide a virtual space content distribution system, a virtual space content distribution program, and a virtual space content distribution method that can reduce the processing load for displaying and outputting viewer avatars in order to prevent display output problems and thereby prevent a decline in the interest of virtual space content. [Means for solving the problem]

[0006] The virtual space content distribution system of claim 1 comprises: A virtual space content distribution system capable of distributing virtual space content in a virtual space, a viewer user terminal capable of outputting an image of the virtual space content, the viewer user terminal being capable of viewing the image; a server computer that is connected to the viewer user terminal via a communication network so as to be capable of data communication, and that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; At least There are a plurality of viewer users, and each viewer user is assigned a viewer user terminal; The virtual space content may include a viewer avatar by allowing the viewer user to participate in the virtual space that is the subject of the virtual space content using the viewer user terminal; the viewer avatars include a first viewer avatar (e.g., a full avatar) and a second viewer avatar (e.g., a simplified avatar in Example 6) that requires a smaller processing load for displaying than the first viewer avatar; During a specific period during the distribution period of the virtual space content (for example, a special MC period in Example 6), the viewer avatar in the virtual space content is set to the second viewer avatar; Among the viewer avatars that are set as the second viewer avatars, the viewer avatar of a viewer that satisfies special conditions (satisfies specific conditions, satisfies being selected by a performer, and is located on the virtual stage G) (for example, the special viewer avatar in Example 6) can change from the second viewer avatar to the first viewer avatar, Among the viewer avatars that are set as the second viewer avatars, the viewer avatars of the viewer who do not satisfy the special condition cannot change from the second viewer avatar to the first viewer avatar. It is characterized by the following. According to this feature, the processing load related to display output is reduced by making the viewer avatar the second viewer avatar, while the viewer avatar of a viewer who meets special conditions can be made the first viewer avatar, thereby increasing the interest in the virtual space content.

[0007] The virtual space content distribution system of claim 2 is the virtual space content distribution system according to claim 1, the virtual space content includes content by a performer avatar; a performer terminal that is usable by a performer corresponding to the performer avatar and is connected to the server computer via the communication network so as to be capable of data communication; The special condition includes that a predetermined operation is performed by the performer on a specific viewer avatar on the performer terminal. It is characterized by the following. According to this feature, the performer can arbitrarily determine the viewer avatar that can be transformed into the first viewer avatar.

[0008] The virtual space content distribution system of claim 3 is the virtual space content distribution system of claim 1, a terminal usable by an administrator who manages the virtual space, the administrator terminal being connected to the server computer via the communication network so as to be capable of data communication; The special condition includes a predetermined operation being performed by the administrator on the administrator terminal for a specific viewer avatar (for example, as shown in Modification Example 6-1, a selection operation being performed on the administrator terminal 150). It is characterized by the following. According to this feature, the administrator can arbitrarily determine the viewer avatar that can be changed into the first viewer avatar.

[0009] The virtual space content distribution system of claim 4 is the virtual space content distribution system according to claim 2 or 3, a means for identifying the viewer avatar that satisfies a prerequisite corresponding to the special condition; A presentation means for presenting the viewer avatar identified by the identification means (for example, in the performer terminal 200 of the sixth embodiment, a part in which the color of the simplified viewer avatar that satisfies the specific condition is changed to the characteristic color of the candidate to notify the user); Equipped with It is characterized by the following. According to this feature, a viewer avatar that can be transformed into a first viewer avatar can be easily determined from viewer avatars that satisfy the prerequisites.

[0010] The virtual space content distribution system of claim 5 is the virtual space content distribution system according to claim 4, The prerequisites include conditions regarding payment for the virtual space content. It is characterized by the following. According to this feature, it is possible to determine a viewer avatar that can be changed into the first viewer avatar from among viewer avatars that satisfy the payment conditions.

[0011] The virtual space content distribution system of claim 6 is the virtual space content distribution system according to claim 4, The preconditions include a condition regarding a virtual item possessed by the viewer avatar. It is characterized by the following. According to this feature, a viewer avatar that can be transformed into the first viewer avatar can be determined from among viewer avatars that satisfy the conditions related to the virtual item.

[0012] The virtual space content distribution system of claim 7 is the virtual space content distribution system according to claim 1, the virtual space includes a special area (e.g., a virtual stage G) assigned to a performer avatar participating in the virtual space, and a specific area (e.g., a virtual audience area) assigned to a viewer avatar participating in the virtual space; The special condition includes that the viewer avatar is located in the special area (e.g., virtual stage G). It is characterized by the following. According to this feature, the viewer avatar located in the specific area can be a viewer avatar that can be changed into the first viewer avatar.

[0013] The virtual space content distribution system of claim 8 is the virtual space content distribution system according to claim 7, the virtual space has a special object (e.g., a transparent object B1) at the boundary between the special area and the specific area, For the viewer avatar that does not satisfy the special condition, a collision between the viewer avatar and the special object is determined, and for the viewer avatar that satisfies the special condition, a collision between the viewer avatar and the special object is not determined. It is characterized by the following. This feature makes it possible to prevent viewer avatars that do not satisfy the special conditions from entering the special area.

[0014] The virtual space content distribution program of claim 9 comprises: A virtual space content distribution program for distributing virtual space content in a virtual space, The virtual space content includes: A content including a video that can be viewed by a plurality of viewer users at a viewer user terminal assigned to each of the viewer users, The viewer user terminal may be connected via a communication network to a server computer capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content, and the viewer user terminal may be connected to the server computer so as to be capable of data communication with the server computer, and the viewer user terminal may include a viewer avatar by allowing the viewer avatar to participate in the virtual space that is the target of the virtual space content; the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for displaying than the first viewer avatar; and a viewer avatar process for setting the viewer avatar in the virtual space content as the second viewer avatar during a specific period during the distribution period of the virtual space content, and for enabling the viewer avatar that satisfies a special condition among the second viewer avatars to be changed to the first viewer avatar, and for disabling the viewer avatar of a viewer that does not satisfy the special condition to be changed to the first viewer avatar. It is characterized by the following. According to this feature, the processing load associated with the output process can be reduced by making the viewer avatar the second viewer avatar, while the viewer avatar of a viewer who meets special conditions can be made the first viewer avatar, thereby increasing the interest in the virtual space content.

[0015] The virtual space content distribution method of claim 10 comprises: A virtual space content distribution method for distributing virtual space content in a virtual space, comprising: The virtual space content includes: A content including a video that can be viewed by a plurality of viewer users at a viewer user terminal assigned to each of the viewer users, The viewer user terminal may be connected via a communication network to a server computer capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content, and the viewer user terminal may be connected to the server computer so as to be capable of data communication with the server computer, and the viewer user terminal may include a viewer avatar by allowing the viewer avatar to participate in the virtual space that is the target of the virtual space content; the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for displaying than the first viewer avatar; A viewer avatar procedure is included in which the viewer avatar in the virtual space content is set as the second viewer avatar during a specific period during the distribution period of the virtual space content, and among the viewer avatars set as the second viewer avatars, the viewer avatars that satisfy a special condition are made changeable to the first viewer avatar, and the viewer avatars of viewers who do not satisfy the special condition are made unchangeable to the first viewer avatar. It is characterized by the following. According to this feature, the processing load associated with the output process can be reduced by making the viewer avatar the second viewer avatar, while the viewer avatar of a viewer who meets special conditions can be made the first viewer avatar, thereby increasing the interest in the virtual space content.

[0016] Furthermore, the present invention may have only the invention-specific matters set forth in the claims of the present invention, or may have the invention-specific matters set forth in the claims of the present invention as well as configurations other than the invention-specific matters. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram showing an example of the system configuration of a virtual space content delivery system according to a first embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of the configuration of a delivery server computer in a virtual space content delivery system according to a first embodiment of the present invention. [Figure 3] FIG. 2 is a diagram showing various data stored in a storage device of a delivery server computer according to the first embodiment of the present invention. [Figure 4] 1 is a diagram illustrating an example of the configuration of a performer terminal in a virtual space content distribution system according to a first embodiment of the present invention. [Figure 5] 1 is a diagram illustrating an example of the configuration of a viewer terminal in a virtual space content delivery system according to a first embodiment of the present invention. [Figure 6] FIG. 2 is a diagram illustrating an example of the configuration of an administrator terminal in the virtual space content delivery system according to the first embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing an example of the configuration of event data in the virtual space content delivery system according to the first embodiment of the present invention. [Figure 8] FIG. 2 is a diagram showing an example of the configuration of performer user data in the virtual space content distribution system according to the first embodiment of the present invention. [Figure 9] FIG. 2 is a diagram showing an example of the configuration of viewer user data in the virtual space content delivery system according to the first embodiment of the present invention. [Figure 10] 1 is an explanatory diagram showing data exchanged between each terminal and a server computer that constitutes the virtual space content distribution system according to the first embodiment of the present invention. FIG. [Figure 11] 1 is a schematic diagram showing the facilities of a studio used by a performer user in a virtual space content delivery system according to a first embodiment of the present invention. [Figure 12] 1 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content distribution system according to a first embodiment of the present invention. [Figure 13] 3 is a diagram showing an example of a display on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 14] 3 is a diagram showing an example of a display on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 15] 3 is a diagram showing an example of a display on a performer terminal constituting the virtual space content distribution system according to the first embodiment of the present invention. FIG. [Figure 16] 1A to 1C are diagrams showing examples of display by zooming operations on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 17] 1A to 1C are diagrams showing examples of display by zooming operations on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 18] 3 is a diagram showing an example of a display on a viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. FIG. [Figure 19] FIG. 2 is a diagram showing an example of a display during a flying performance on a viewer terminal constituting the virtual space content distribution system according to the first embodiment of the present invention. [Figure 20] FIG. 3 is a flowchart showing an example of a viewer viewpoint video control process executed in the viewer terminal constituting the virtual space content delivery system according to the first embodiment of the present invention. [Figure 21] FIG. 10 is an explanatory diagram showing the contents of a virtual live performance distributed by a virtual space content distribution system according to a second embodiment of the present invention. [Figure 22] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. [Figure 23] FIG. 10 is a diagram showing a participant user table used in a delivery server computer that constitutes a virtual space content delivery system according to a second embodiment of the present invention. [Figure 24] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. [Figure 25] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. [Figure 26] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. [Figure 27] FIG. 10 is an explanatory diagram showing the relationship between the period type of a virtual live show and the entity that generates the virtual camera viewpoint video in the virtual space content delivery system according to the second embodiment of the present invention. [Figure 28] FIG. 10 is an explanatory diagram showing how a virtual camera viewpoint video and a viewer viewpoint video are generated in a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a second embodiment of the present invention. [Figure 29] FIG. 10 is a flowchart showing a viewer viewpoint video generation process according to a modified example of the second embodiment of the present invention. [Figure 30] FIG. 10 is a diagram showing the contents of virtual space update data transmitted from a delivery server computer in a modified example of the second embodiment of the present invention. [Figure 31] FIG. 10 is an explanatory diagram showing the contents of a virtual live performance distributed by a virtual space content distribution system according to a third embodiment of the present invention. [Figure 32] FIG. 10 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content distribution system according to a third embodiment of the present invention. [Figure 33] FIG. 10 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content distribution system according to a third embodiment of the present invention. [Figure 34] FIG. 10 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content distribution system according to a third embodiment of the present invention. [Figure 35]FIG. 10 is a diagram showing a virtual live concert venue, which is a virtual space used in a virtual space content distribution system according to a third embodiment of the present invention. [Figure 36] FIG. 10 is a diagram showing an example of the structure of time schedule data (TS data) stored in a delivery server computer that constitutes a virtual space content delivery system according to a third embodiment of the present invention. [Figure 37] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. [Figure 38] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. [Figure 39] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system according to a third embodiment of the present invention. [Figure 40] FIG. 11 is an explanatory diagram showing a friend user data table stored in a viewer terminal that constitutes a virtual space content distribution system according to a third embodiment of the present invention. [Figure 41] FIG. 11 is an explanatory diagram showing an example of an avatar simplification determination process in the virtual space content delivery system according to the third embodiment of the present invention. [Figure 42] FIG. 11 is an explanatory diagram showing an example of a viewer viewpoint video generation process in the virtual space content delivery system according to the third embodiment of the present invention. [Figure 43] FIG. 11 is an explanatory diagram showing an example of a virtual camera viewpoint video generation process (viewer terminal) in the virtual space content delivery system according to the third embodiment of the present invention. [Figure 44] (A) is an explanatory diagram showing the appearance of each avatar generated in the viewer viewpoint video generation process in the virtual space content distribution system of Example 3 of the present invention, and (B) is an explanatory diagram showing the appearance of each avatar generated in the virtual camera viewpoint video generation process (viewer terminal) in the virtual space content distribution system of Example 3 of the present invention. [Figure 45](A) is an explanatory diagram showing the appearance of a performer avatar in a virtual space content distribution system of Example 3 of the present invention, and (B) is an explanatory diagram showing the appearance of a viewer avatar in a virtual space content distribution system of Example 3 of the present invention. [Figure 46] FIG. 10 is an explanatory diagram showing changes in the appearance of an avatar in a virtual space content delivery system according to a third embodiment of the present invention. [Figure 47] FIG. 10 is an explanatory diagram showing changes in the appearance of an avatar in a virtual space content delivery system according to a third embodiment of the present invention. [Figure 48] FIG. 10 is an explanatory diagram showing changes in the appearance of an avatar in a virtual space content delivery system according to a third embodiment of the present invention. [Figure 49] FIG. 10 is an explanatory diagram showing changes in the appearance of a viewer avatar in a virtual space content delivery system according to a third embodiment of the present invention. [Figure 50] FIG. 10 is an explanatory diagram showing changes in the appearance of a performer avatar in a modified example of the third embodiment of the present invention. [Figure 51] FIG. 10 is an explanatory diagram of a simplified avatar as a viewer avatar in a modified example of the third embodiment of the present invention. [Figure 52] FIG. 10 is an explanatory diagram showing data exchanged between a delivery server computer and a viewer terminal that constitute a virtual space content delivery system in a modified example of the third embodiment of the present invention. [Figure 53] FIG. 13 is an explanatory diagram illustrating an example of a pre-data transmission process according to a modified example of the third embodiment of the present invention. [Figure 54] FIG. 10 is an explanatory diagram of a simplified avatar as a viewer avatar in a modified example of the third embodiment of the present invention. [Figure 55] FIG. 10 is an explanatory diagram showing the transition of the appearance of a viewer avatar in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 56] FIG. 10 is an explanatory diagram of a viewer avatar in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 57]FIG. 10 is an explanatory diagram showing the interaction between a delivery server computer and a viewer terminal during a live virtual space participation period in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 58] FIG. 10 is an explanatory diagram of a participant user table in the virtual space content delivery system according to the fourth embodiment of the present invention. [Figure 59] FIG. 10 is an explanatory diagram showing the exchange between a delivery server computer and a viewer terminal relating to a friend registration function in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 60] FIG. 10 is an explanatory diagram of a friend user data table in the virtual space content delivery system according to the fourth embodiment of the present invention. [Figure 61] FIG. 10 is an explanatory diagram of a period during which a simplified target area can be designated in a virtual space content distribution system according to a fourth embodiment of the present invention. [Figure 62] 10 is a flowchart showing a simplification target area designation operation process in the virtual space content distribution system according to the fourth embodiment of the present invention. [Figure 63] FIG. 10 is an explanatory diagram showing a display mode of a viewer terminal when a simplified target area designation operation process is executed in a virtual space content distribution system according to a fourth embodiment of the present invention. [Figure 64] 10 is a flowchart showing a viewer's viewpoint video generation process in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 65] FIG. 10 is an explanatory diagram of the appearance of viewer avatars in a simplification target area and a non-simplification target area in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 66] FIG. 4 is an explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system according to a modified example 4-1 of the present invention. [Figure 67] FIG. 4 is an explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system according to a modified example 4-2 of the present invention. [Figure 68]FIG. 10 is an explanatory diagram of a case where the setting of a simplification target area is not completed during the period during which the simplification target area can be designated in the virtual space content distribution system according to the modified example 4-3 of the present invention. [Figure 69] FIG. 10 is an explanatory diagram showing the interaction between a distribution server computer and a viewer terminal during a live virtual space participation period in a virtual space content distribution system in variant example 4-4 of the present invention. [Figure 70] FIG. 10 is an explanatory diagram of the number of areas subject to simplification for each viewer terminal specification in a virtual space content distribution system according to variant example 4-4 of the present invention. [Figure 71] An explanatory diagram of the appearance of viewer avatars in areas subject to simplification and areas not subject to simplification in a virtual space content distribution system in variant example 4-5 of the present invention. [Figure 72] FIG. 10 is an explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system according to variant example 4-6 of the present invention. [Figure 73] FIG. 10 is an explanatory diagram of a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 74] FIG. 10 is an explanatory diagram of a storage of a delivery server computer in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 75] FIG. 10 is an explanatory diagram showing the exchange between a delivery server computer and a viewer terminal in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 76] FIG. 10 is an explanatory diagram of a participant user table in the virtual space content delivery system according to the fourth embodiment of the present invention. [Figure 77] 10 is a flowchart showing a process of determining a participating virtual live venue in the virtual space content distribution system according to the fourth embodiment of the present invention. [Figure 78] An explanatory diagram showing the interactions between the viewer terminal of a viewer avatar participating in a sub-virtual live venue, the distribution server computer, and the performer terminal in a virtual space content distribution system according to a fourth embodiment of the present invention. [Figure 79]FIG. 10 is an explanatory diagram showing interactions between a viewer terminal, a performer terminal, and a chat information server computer in a virtual space content distribution system according to a fourth embodiment of the present invention. [Figure 80] FIG. 10 is an explanatory diagram showing the display modes of a performer terminal and a viewer terminal in a virtual space content distribution system according to a fourth embodiment of the present invention. [Figure 81] FIG. 10 is an explanatory diagram comparing information outputted from each terminal in a virtual space content delivery system according to a fourth embodiment of the present invention. [Figure 82] FIG. 5 is an explanatory diagram comparing information output by each terminal in a virtual space content distribution system according to a modified example 5-1 of the present invention. [Figure 83] FIG. 5 is an explanatory diagram of an image displayed on a virtual display in a virtual space content distribution system according to a modified example 5-2 of the present invention. [Figure 84] FIG. 10 is an explanatory diagram showing the interaction between a distribution server computer and a viewer terminal during a live virtual space participation period in a virtual space content distribution system in variant example 5-3 of the present invention. [Figure 85] FIG. 10 is an explanatory diagram showing the display mode of a viewer terminal in a virtual space content distribution system in a modified example 5-3 of the present invention. [Figure 86] FIG. 10 is an explanatory diagram regarding whether a viewer avatar can move to another virtual live venue in a virtual space content distribution system according to variant example 5-3 of the present invention. [Figure 87] FIG. 13 is an explanatory diagram showing the transition of the appearance of a viewer avatar in a virtual space content delivery system according to a sixth embodiment of the present invention. [Figure 88] FIG. 13 is an explanatory diagram of a virtual live venue in a virtual space content distribution system according to a sixth embodiment of the present invention. [Figure 89] (a) is an explanatory diagram of viewer avatars in a virtual space content distribution system according to a sixth embodiment of the present invention, and (b) is an explanatory diagram of whether or not collision detection is performed for each avatar in a virtual space content distribution system according to a sixth embodiment of the present invention. [Figure 90]FIG. 13 is a diagram showing a display screen when selecting one special viewer avatar from viewer avatars that satisfy the prerequisites on the display device 211 of the performer terminal 200 in Example 6 of the present invention. [Figure 91] (a) is a diagram showing how a special viewer avatar that has been converted to a full avatar by meeting certain conditions interacts with a performer avatar during the special MC period, and (b) is a diagram showing the situation in which a special viewer avatar changes from a simple avatar to a full avatar. [Figure 92] FIG. 13 is an explanatory diagram showing the flow of setting a special viewer avatar in a virtual space content delivery system according to a sixth embodiment of the present invention. [Figure 93] FIG. 20 is a flow diagram showing the contents of a virtual space update process in a virtual space content delivery system according to a sixth embodiment of the present invention. [Figure 94] FIG. 20 is an explanatory diagram showing whether or not collision detection is performed between a viewer avatar and a transparent object in a virtual space content delivery system according to a sixth embodiment of the present invention. [Figure 95] FIG. 13 is an explanatory diagram showing the display mode of a viewer terminal in a virtual space content delivery system according to a sixth embodiment of the present invention. [Figure 96] FIG. 10 is a diagram showing the display contents when one exceptional viewer avatar is selected from viewer avatars that satisfy the prerequisites on the display device 160 of the administrator terminal in Modification Example 6-1 of the present invention. [Figure 97] FIG. 6 is an explanatory diagram showing whether or not collision detection is performed between each avatar and a transparent object in the virtual space content distribution system according to the modified example 6-2 of the present invention. [Figure 98] FIG. 13 is an explanatory diagram showing the progress of a virtual live performance in Example 7 of the present invention, the transition of the appearance of a viewer avatar, the availability of an operation to change a simple avatar, and changes in the display appearance of an overlapping avatar. [Figure 99] FIG. 13 is an explanatory diagram of performer avatars and viewer avatars in a virtual space content distribution system according to a seventh embodiment of the present invention. [Figure 100]FIG. 13 is a diagram showing the display screen of a viewer terminal when an item selection operation is performed in a virtual space content delivery system according to a seventh embodiment of the present invention. [Figure 101] FIG. 13 is a diagram showing a change menu screen displayed on the viewer terminal 300 corresponding to the viewer avatar who has participated in the third and fourth areas in Example 7 of the present invention. [Figure 102] FIG. 13 is a diagram showing changes in the viewing screen displayed on the viewer terminal 300 corresponding to the viewer avatar who participated in Example 7 of the present invention. [Figure 103] FIG. 10 is an explanatory diagram showing the interaction between the distribution server computer and the viewer terminal during the live virtual space participation period and the pre-show period in the virtual space content distribution system according to the seventh embodiment of the present invention. [Figure 104] 13 is a flowchart of a viewer's viewpoint video generation process in a virtual space content delivery system according to a seventh embodiment of the present invention. [Figure 105] FIG. 13 is an explanatory diagram comparing a full avatar and a simple avatar in a virtual space content delivery system according to a seventh embodiment of the present invention. [Figure 106] FIG. 7 is an explanatory diagram of the colors of items corresponding to performer avatars in a virtual space content distribution system in variant example 7-1 of the present invention. [Figure 107] FIG. 7 is an explanatory diagram of the operation of Chemilight in the virtual space content distribution system according to the modified example 7-2 of the present invention. [Figure 108] FIG. 7 is an explanatory diagram showing the interaction between a delivery server computer and a viewer terminal in a virtual space content delivery system according to a modified example 7-3 of the present invention. [Figure 109] FIG. 10 is an explanatory diagram of a friend user data table in the virtual space content distribution system according to the modified example 7-3 of the present invention. [Figure 110] FIG. 10 is an explanatory diagram of a friend user data table in the virtual space content distribution system according to the modified example 7-3 of the present invention. [Figure 111] FIG. 7 is an explanatory diagram of a full avatar and a simple avatar in a virtual space content distribution system according to variant example 7-3 of the present invention. [Figure 112] FIG. 10 is an explanatory diagram of the case where the appearance of a simple avatar is determined at the start of a joint program in a virtual space content distribution system according to variant example 7-4 of the present invention. [Figure 113] FIG. 7 is an explanatory diagram of a performer avatar and a simple avatar in a virtual space content distribution system according to variant example 7-5 of the present invention. [Figure 114] FIG. 7 is an explanatory diagram of a performer avatar and a simple avatar in a virtual space content distribution system according to variant example 7-6 of the present invention. [Figure 115] FIG. 13 is a diagram showing the configuration of a virtual space according to an eighth embodiment of the present invention. [Figure 116] FIG. 13 is a diagram showing the configuration of a city area in a virtual space according to an eighth embodiment of the present invention. [Figure 117] FIG. 13 is a diagram showing the configuration of a park area in a virtual space according to an eighth embodiment of the present invention. [Figure 118] FIG. 13 is a diagram showing a viewpoint image from a viewer avatar participating in an event held in a park area in a virtual space content delivery system according to an eighth embodiment of the present invention. [Figure 119] FIG. 13 is a diagram showing a viewpoint image from a viewer avatar located outside an event area held in a park area in a virtual space content delivery system according to an eighth embodiment of the present invention. [Figure 120] FIG. 13 is an explanatory diagram showing the flow of event setting in a virtual space content delivery system according to an eighth embodiment of the present invention. [Figure 121] FIG. 13 is a diagram showing the state of a viewer avatar when participating in an event in a virtual space content delivery system according to an eighth embodiment of the present invention. [Figure 122] FIG. 13 is a flowchart showing the details of an event space process executed in a virtual space content delivery system according to an eighth embodiment of the present invention. [Figure 123] FIG. 8 is a diagram showing a viewpoint image from a viewer avatar participating in an event in a virtual space content distribution system according to variant example 8-1 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] The following describes an embodiment of a virtual space content distribution system according to the present invention, with reference to the accompanying drawings. The same or similar components in the drawings are designated by the same reference numerals, and redundant description will be omitted. The virtual space content distribution program according to the present invention may be used by the entire system, which includes a server computer and terminals (described later) that constitute the virtual space content distribution system, or it may operate only on a portion of the devices that constitute the virtual space content distribution system, such as the server computer or terminals. Each process described below is executed by a program stored in the device on which the process is performed. [Example]

[0019] Fig. 1 is a block diagram showing an example of the system configuration of a virtual space content distribution system in Example 1, which is an embodiment of the present invention, and in the system of Example 1 shown in Fig. 1, a virtual live performance in which a performer avatar 1 corresponding to a performer user performs a live event in a virtual live venue shown in Fig. 12 is distributed as virtual space content. Note that in the following, distribution means transmission to multiple people and is synonymous with transmission.

[0020] As shown in FIG. 1, the virtual space content distribution system of this embodiment 1 is mainly composed of a distribution server computer 100 that can execute various processes related to the virtual live venue shown in FIG. 11, which is a virtual space, and various processes related to the distribution of the virtual live event, an administrator terminal 150 that is used by an event manager M of an event management organization that operates and manages the virtual live event to manage the distribution of the virtual live event, a performer terminal 200 that is used by performer users, and a viewer terminal 300 that can be used by viewer users who virtually participate in and watch the virtual live event, and the distribution server computer 100, administrator terminal 150, performer terminal 200, and viewer terminal 300 are connected to each other so that they can communicate data with each other via the Internet N, which is an open computer network.

[0021] In this first embodiment, the administrator terminal 150 and the performer terminal 200 are also shown as being connected to the distribution server computer 100 via the Internet network N, but the present invention is not limited to this. For example, if the studio shown in Figure 11 is a studio operated by an event management organization that provides studios to performer users, and the distribution server computer 100 is installed in these studios, the administrator terminal 150 and the performer terminal 200 may be locally connected to the distribution server computer 100 via data communication, or may be connected via a local area network LAN within the studio facility.

[0022] 1, there are multiple viewer users, such as viewer user A, viewer user B, viewer user C, etc., and viewer terminals 300 include viewer terminals 300a, 300b, 300c, etc., which can be used by each of viewer users A, B, C, etc. Although there are typically multiple viewer users, there may be one or four or more, and the number can be determined appropriately depending on the virtual space, such as a virtual live venue. Hereinafter, viewer user A, viewer user B, viewer user C, etc. may be collectively referred to as viewer users, and viewer terminals 300a, 300b, 300c, etc. may be collectively referred to as viewer terminals 300.

[0023] Furthermore, in this Example 1, an example is given in which there is one performer user, but the present invention is not limited to this, and the virtual live performance may be a joint event in which multiple performer users who are working individually virtually participate, or a group event in which a group of multiple people who are working together virtually participate.In this way, when multiple performers are appearing, performer terminal 200a, performer terminal 200b, performer terminal 200c, etc. may be provided for each performer user, or if a group of multiple people are performing, one performer terminal 200 may be used by multiple people.

[0024] Also, in Figure 1, the distribution server computer 100 is shown as a single unit, but the present invention is not limited to this. These distribution server computers 100 may be configured with multiple server computers, such as a first server computer that mainly performs virtual space processing and a second server computer that mainly performs distribution processing, or may be configured with a cloud server, etc., so that the number of server computers can be changed appropriately depending on the scale of the event, the number of participating listener users, etc.

[0025] Similarly, although the administrator terminal 150 is illustrated as a single unit in Figure 1, the present invention is not limited to this, and it goes without saying that there may be multiple administrator terminals 150 depending on the number of performer users and viewer users virtually participating in the live performance.

[0026] <Distribution server computer> Fig. 2 is a diagram showing the configuration of the delivery server computer 100 used in this Example 1. As described above, the delivery server computer 100 used in this Example 1 is a normal server computer with relatively excellent processing power that is installed in a studio operated by an event management organization, and as shown in Fig. 2, has a processor (CPU) 101 connected to a data bus 108, a memory (RAM) 102, a storage 103 such as a hard disk, a communication interface (I / F) 104 that is capable of two-way communication using the TCP / IP protocol via the Internet, which is an open computer network, and an input / output interface (I / F) 105 to which an input device such as a keyboard can be connected.

[0027] In this embodiment 1, an input / output interface (I / F) 105 is provided to enable local operation, but if remote operation is possible via a communication interface (I / F) 104, the input / output interface (I / F) 105 may not be provided.

[0028] In addition to an operating system (server OS) for providing server functions (not shown), storage 103 of distribution server computer 100 stores various data and programs as shown in Fig. 3. Specifically, the following are mainly stored: event management program 110 for providing an event management function for managing live events, virtual space generation program 120 for providing a virtual space for a virtual live venue, a performer user management program 130 for providing a performer user management function for managing performer users, and a viewer user management program 132 for providing a viewer user management function for managing viewer users.

[0029] The event management program 110 is a program for managing a virtual live event, which is an event held at a virtual live venue (see Figure 12), in cooperation with the administrator terminal 150, and is capable of managing the progress of the virtual live event using event data 111 (see Figure 7), time schedule data (TS data) 112, and music data 113 stored in the storage 103 together with the event management program 110.

[0030] An example of the event data 111 used in the present embodiment 1 is shown in Fig. 7. The event data 111 stores information such as an event name, a file name of time schedule (TS) data, a performer ID uniquely assigned to the event, a file name of a song list, a file name of a participating viewer list in which information of special viewer users who are permitted to participate virtually is registered in a third area of ​​the virtual audience area described later, and a file name of a participating viewer list in which information of viewer users who have reserved virtual participation is registered in a fourth area of ​​the virtual audience area described later, all of which are associated with an event ID uniquely assigned to the event.

[0031] The time schedule data (TS data) 112 is data in which the order of progression of the virtual live event to be held and the programs such as songs are described in chronological order, and the virtual live event is managed so that it progresses based on the time schedule described in the time schedule data (TS data) 112. The time schedule data (TS data) 112, event data 111, etc. can be changed, updated, or added by the event manager M operating the manager terminal 150.

[0032] In addition, the song data 113 is data for the performance parts (so-called karaoke) of each song sung by the performer user, and is played by the audio control program 124 described later based on the time schedule data (TS data) 112 by the event management program 110, and is transmitted to the performer terminal 200 and the viewer terminal 300 (see Figure 11).

[0033] The virtual space generation program 120 is a program for providing a virtual live venue, which is the virtual space shown in Figure 12, and has the function of providing a virtual live venue in which performer avatars and viewer avatars are virtually participating, using virtual space data 121 that describes the structure of the virtual live venue and is stored in storage 103 together with the virtual space generation program 120, object data 122 for placing various objects to be placed in the virtual live venue (including virtual placement object 43, which is a star decoration on the stage, and virtual aerial objects 50-52, which are star objects in the sky), and performer avatar data included in performer user data 131, which will be described later, and viewer avatar data included in viewer user data 133.

[0034] 12, viewpoint image control program 123 provides the function of generating viewpoint images from virtual cameras C1-C4 virtually set up in the virtual live venue and a virtual camera moving in the air (not shown), and also provides the function of displaying images from the viewpoints of viewer avatars 11-14 virtually participating in the third area of ​​the virtual live venue and viewer avatars 21-32 virtually participating in the fourth area on viewer terminal 300. Virtual camera C2 is a virtual camera set up to display images from the viewpoint of performer avatar 1 on the stage of the virtual live venue, and the viewpoint image from virtual camera C2 is projected onto screen S set up in the studio by projector 211 (described later) connected to performer terminal 200 in cooperation with performer terminal 200.

[0035] The audio control program 124 is a program for controlling the audio (including music) distributed during the virtual live event, and specifically provides the function of playing back the music data 113 and synthesizing the audio produced by the performer user with the played music and distributing it to the administrator terminal 150 and the viewer terminal 300.

[0036] The comment control program 125 cooperates with a viewer comment control program stored in the viewer terminal 300 described below to provide a function for displaying various comments entered by each viewer user at the viewer terminal 300 in chronological order on the performer terminal 200 and the viewer terminal 300 during a virtual live event.

[0037] The gift control program 126 is a program that cooperates with a viewer program stored in the viewer terminal 300 described below to control gifts given from viewer users to performer users, and specifically provides functions such as a function that allows the viewer user to purchase gifts based on information on the purchase price set for various gifts stored in the gift data 134, a processing function that allows the viewer user to give a gift purchased using the gift object data included in the gift data 134 to the performer user (including processing for presentation when giving the gift), and a processing function that changes the owner of a gift that has been given from the viewer user to the performer user.

[0038] The performer user management program 130 provides a management function for information relating to performer users, an authentication function, and the like, based on performer user data 131 shown in FIG. 8, which is stored in the storage 103 together with the performer user management program 130 .

[0039] An example of the performer user data 131 used in Example 1 is shown in Figure 8. The performer user data 131 stores various information, such as the performer user's account (email address), name, authentication information, file name of the avatar data of the avatar used in the virtual space, and file name of the item list in which the performer user's owned items are registered, in association with a performer user ID uniquely assigned to the performer user. Although not shown in Figure 8, information on virtual value such as points owned by each performer user may also be stored as the performer user data 131.

[0040] Performer users can be authenticated by matching their performer user ID, account, and authentication information, and items (gifts) that each performer user has received from viewers can be identified from an item list.

[0041] The viewer user management program 132 provides functions such as management of information relating to viewer users and authentication functions based on viewer user data 133 shown in FIG. 9, which is stored in the storage 103 together with the program.

[0042] An example of the viewer user data 133 used in Example 1 is shown in Figure 9. The viewer user data 133 stores various information associated with a viewer user ID uniquely assigned to the viewer user, such as the viewer user's account (email address), authentication information, the file name of the avatar data of the avatar used in the virtual space, owned points which are the number of points owned that can be used in the virtual space, the file name of an item list in which items (gifts) purchased using points are registered, and personal information such as name, date of birth, and telephone number. Although not shown in Figure 9, nicknames (comment names) displayed together with comments are also registered in the viewer user data 133, and the nicknames (comment names) are displayed together with comments. Furthermore, points can be increased, for example, by purchasing points from a predetermined operating company.

[0043] <Presenter terminal> Fig. 4 is a diagram showing an example of the configuration of a performer terminal 200 in the virtual space content distribution system of this embodiment 1. In this embodiment 1, as shown in Fig. 11, performer terminal 200 is installed in a control room adjacent to a studio where performers perform their actions, and uses a normal computer with relatively excellent processing power, and as shown in Fig. 4, it has a processor (CPU) 201 connected to a data bus 208, memory (RAM) 202, storage 203 such as a hard disk, a communication interface (I / F) 204 capable of two-way communication using the TCP / IP protocol via the Internet, which is an open computer network, an image processing unit 206 including a graphics processing unit (GPU) to which display devices A210 to C212 are connected, and an input / output interface (I / F) 205 to which various input / output devices are connected.

[0044] Connected to the input / output interface (I / F) 205 are input devices such as a motion sensor 220 including multiple wearable sensors 220C1-220C5 (see FIG. 11) worn by the performer user on the left and right limbs and waist, an expression input device 221 consisting of an imaging camera for inputting the performer's facial expressions, a voice input device 222 consisting of a sound-collecting microphone worn on the performer user's head, and an operation input device 224 such as a keyboard or touch panel operable by an assistant operator O assisting the performer user. Note that these input devices may be provided with controllers that the performer user can hold to perform various operations, allowing the performer to perform various operations himself or herself without the assistance of the assistant operator O.

[0045] In this embodiment 1, the performer user performs performance actions in accordance with the music (including musical actions such as singing and dancing), so an assistant operator O assists with operations during the performance. However, it is also possible to have multiple assistant operators O, or to set the operation content in advance in the performer terminal 200, for example, in a sequence program, so that an assistant operator O is not provided.

[0046] An audio output device 223 including high-performance earphones (ear monitors) worn by the performer user and speakers placed in the control room is connected to the input / output interface (I / F) 205 as an output device, so that the sound of the music being played sent from the distribution server computer 100 is output to the performer user via the high-performance earphones (ear monitors), and the music audio is output from the speakers along with the performer's voice, allowing an assistant operator O or the like to check the status of the music, including the voice produced by the performer user, in the control room.

[0047] In this Example 1, high-performance earphones (in-ear monitors) are used to avoid the inconvenience of the reproduced music audio being input from the sound collection microphone worn by the performer user, but the present invention is not limited to this, and the music audio may be output from speakers in the studio, etc., as long as the inconvenience of the music audio being input can be avoided.

[0048] In the first embodiment, the sound collecting microphones are worn by the performers, but these sound collecting microphones may be installed on the floor, wall, or ceiling of the studio.

[0049] Here, we will explain the motion sensor 220 used in the performer terminal 200 of this Example 1. Any motion sensor can be used as long as it can properly detect (measure) the body movements (movements) of the performer user. In this Example 1, in order to enable more accurate and shorter-period detection of the performer's movements, multiple wearable sensors 220C1 to 220C5 are used that are worn by the performer user on the body. Note that the motion sensor 220 may also be a type that does not require the performer to wear any equipment, such as a LiDAR that uses laser light or the like.

[0050] In this Example 1, in order to reduce the burden of movement, particularly the head movement, caused by wearing sensors etc. when the performer user performs a performance movement, the performer user's head movement is detected by image recognition using images captured by the imaging camera that constitutes the facial expression input device 221, as described below, and five wearable sensors 220C1 to 220C5 are used to detect movements other than the head. However, for example, wearable sensors may also be provided on the performer user's head, or more wearable sensors (7 or more) may be worn to detect more detailed movements.

[0051] As shown in FIG. 11, the wearing sensors 220C1 to 220C5 detect their own positions and orientations in cooperation with base station 220a and base station 220b installed in a studio room separated from the adjacent control room by a glass window.

[0052] Base station 220a and base station 220b may use, for example, a multi-axis laser emitter. Base station 220a emits a synchronous flashing light and then scans the laser light around, for example, a vertical axis. Base station 220b scans the laser light around, for example, a horizontal axis.

[0053] Each of the mounting sensors 220C1 to 220C5 may include a plurality of optical sensors that detect the incidence of blinking light and laser light from the base station 220a and the base station 220b.

[0054] Each of the wearing sensors 220C1 to 220C5 is capable of detecting its own position and orientation based on the time difference between the timing of the flashing light and the timing of the laser light, the light reception time at each optical sensor, the angle of incidence of the laser light detected by each optical sensor, and other information as necessary. For example, the ViveTracker provided by HTC CORPORATION or a base station can be suitably used.

[0055] Base station 220a and base station 220b emit flashing light and perform laser light scanning at regular intervals, so the detection information of each of wearable sensors 220C1-220C5 is updated at each interval. Detection information indicating the position and orientation of each motion sensor calculated by each of these wearable sensors 220C1-220C5 is input to performer terminal 200 via short-range wireless communication and transmitted to distribution server computer 100 as performer avatar information together with facial motion information (see FIG. 10).

[0056] In this embodiment 1, an example is shown in which two base stations, base station 220a and base station 220b, are used, but the present invention is not limited to this, and the number of base stations may be three or more.

[0057] Alternatively, the motion sensor 220 may be configured such that each of the multiple motion sensors worn by the performer user is equipped with a number of infrared LEDs or visible light LEDs, and the position and orientation of each of the motion sensors can be detected by detecting the light from these LEDs with an infrared camera installed on the floor or wall of the studio.

[0058] Furthermore, in order to reduce the burden on the performer user, motion sensor 220 may be replaced with a motion sensor using lighter reflective markers instead of wearable sensors 220C1-220C5. That is, reflective markers attached to various parts of the performer user's body with adhesive tape or the like may be photographed to generate photographic data, and the photographic data may be image-processed to detect the positions and orientations of the reflective markers, thereby detecting the performer user's movements. In this case, a cameraman may be positioned in the studio to photograph the performer user, and the cameraman may photograph the performer user from a direction suitable for detection, thereby enabling the performer user's movements to be detected well.

[0059] Furthermore, the motion sensor 220 may be a suit with a built-in inertial sensor, such as the inertial sensor suit for the MVN motion capture system marketed by Xsens.com, and the movements of the performer user may be detected by analyzing the sensor signal output from the inertial sensor.

[0060] Any facial expression input device 221 can be used as long as it can appropriately detect the head movements including the face of the performer user. In this embodiment 1, specifically, it is configured with an imaging camera (digital camera) arranged so as to be able to continuously capture head images including the face of the performer user, and the head movements and facial movements (facial expressions) of the performer user captured by the imaging camera are detected by image recognition, and facial motion information, which is the detected head movements and facial movements (facial expressions), is transmitted to the distribution server computer 100 as performer avatar information together with body motion information, as shown in Fig. 10.

[0061] The facial expression input device 221 is not limited to the imaging camera (digital camera) used in the first embodiment, and may be, for example, a 3D camera capable of detecting the depth of a person's face, or a mobile terminal such as a smartphone equipped with a LiDAR device. In this case, the mobile terminal may be worn by the performer user.

[0062] As described above, the performer terminal 200 of this Example 1 has three display devices A210 to C212, where display device A210 is an LCD monitor or the like installed in the control room as shown in FIG. 11, display device B211 is a projector that projects images onto a screen S installed in the studio, and display device C212 is a large vertical display installed adjacent to the screen S in the studio.

[0063] Display device A210, located in the studio's adjustment room, displays images from each viewpoint of virtual cameras C1 to C4, viewpoint images from a virtual camera moving in the air during the flying performance period described below, the content of comments, and a message input window into which assistant operator O can input messages he or she wishes to convey to the performer users.

[0064] Meanwhile, a screen S projected from a projector, which is a display device B211 installed in the studio, displays an image from the viewpoint of a virtual camera C2 (see Figure 12), which is the performer's viewpoint, including, for example, an image including avatars of viewers virtually participating in the live performance in the third and fourth areas, as shown in Figure 15, thereby allowing the performer user to check the status of the viewer users virtually participating in the virtual live performance through the image.

[0065] In addition, the display device C212 installed in the studio displays comments from viewer users, as well as comments and messages entered by the assistant operator O, so that the performer user can check the comments from viewer users and messages from the assistant operator O even during the performance.

[0066] In addition to an operating system (OS) for operating the computer that is the performer terminal 200, the storage 203 stores a performer program that, when executed by the processor (CPU) 201, etc., works in cooperation with the distribution server computer 100 to provide various functions, including the ability for the performer user to control the movement of the performer avatar 1 in a virtual live performance, which is virtual space content.

[0067] As shown in Figure 4, the performer program includes a performer avatar control program, a performer viewpoint image control program, a performer viewpoint image control program, a performer voice control program, a performer comment control program, performer authentication data, etc.

[0068] The performer avatar control program is a program that mainly provides the function of scanning the performer user's body motion using the motion sensor 220 and scanning the performer user's facial motion (expression) using the facial expression input device 221, generating performer avatar information for operating the performer avatar, and transmitting it to the distribution server computer 100.

[0069] The performer's viewpoint video control program is a program that provides the function of generating and outputting a viewpoint video of virtual camera C2, which is the performer's viewpoint, based on the virtual space update data distributed from distribution server computer 100.

[0070] The performer audio control program is a program that outputs the playback sound of a song based on the playback music data distributed from the distribution server computer 100 via the audio output device 223, converts the audio input from the audio input device 222 into data and transmits it to the distribution server computer 100 as performer audio data, and provides the function of generating and outputting monitor audio consisting of the playback sound of the song and the performer audio.

[0071] The performer's comment control program is a program that provides a function for displaying comments and the like by viewer users that are distributed from the distribution server computer 100 .

[0072] The performer authentication data is data for authenticating the identity of the performer user when communicating with the distribution server computer 100 .

[0073] In addition, the performer avatar control program includes a motion parameter generation program that can generate parameter information for the position and rotation angle of the bones that make up the rig data (sometimes called skeleton data) that indicates the skeleton of the performer avatar included in the performer avatar data, as well as an image recognition processing program that performs image recognition of the performer user's head position and facial expression from images captured by an imaging camera.

[0074] In addition, in this Example 1, the performer terminal 200 is exemplified as a form in which a computer serving as the performer terminal 200 installed in a studio operated by an event management organization is jointly used by multiple performer users to hold a live event, and each performer user can use the computer as their own performer terminal 200 by inputting performer authentication data.However, the present invention is not limited to this, and for example, when a performer user appears in a virtual live performance from their home, etc., the performer terminal 200 may be a computer installed at the performer user's home and used exclusively by the performer user, and in such cases, storing performer authentication data is not necessarily required.

[0075] In this way, by using the performer terminal 200 constituting the virtual space content distribution system of the present embodiment 1, the performer user or the like can move the performer avatar 1 placed on the virtual stage G in the virtual live venue in conjunction with the performer user's own movements, and can reflect the performer user's own facial expressions in the facial expressions of the performer avatar 1. In addition, the performer user can distribute his or her own voice as the voice of the performer avatar 1 to the audience via the distribution server computer 100.

[0076] <Viewer terminal> Fig. 5 is a diagram showing an example of the configuration of a viewer terminal 300 in the virtual space content delivery system of Example 1. In Example 1, the viewer terminal 300 is a smartphone P carried by a viewer user or a regular desktop computer (PC) installed at home or the like by the viewer user, and as shown in Fig. 5, the viewer terminal 300 has a processor (CPU) 301 connected to a data bus 308, a memory (RAM) 302, a storage 303 such as a hard disk or non-volatile memory, a communication interface (I / F) 304 capable of two-way communication via the Internet, which is an open computer network, using the TCP / IP protocol, an image processing unit 306 including a graphics processing unit (GPU) connected to a display device 310, and an input / output interface (I / F) 305 connected to various input / output devices.

[0077] As described below, display device 310 is only required to be capable of displaying various viewpoint images generated based on virtual space data of a virtual live venue, which is a virtual space updated by virtual space update data (see FIG. 10) distributed from distribution server computer 100, using a viewer viewpoint image control program included in the viewer program stored in storage 303, as well as flying viewpoint images distributed from distribution server computer 100 during a flying performance period, as described below. If viewer terminal 300 is a smartphone P, the display device 310 corresponds to the display of the smartphone P. If viewer terminal 300 is an ordinary computer (PC), the display device 310 corresponds to a stationary display constituting the computer (PC). Note that these displays are not limited to those that display monoscopic images (2D images), and may be, for example, those that can display stereoscopic images (3D images) by displaying right-eye and left-eye images, such as a head-mounted display (HMD) that can be worn by the viewer.

[0078] The input / output interface (I / F) 305 is connected to an operation input device 321 consisting of one or more devices that allow the viewer to perform various operations as input devices, and an audio output device 322 such as earphones or speakers that can output live audio, etc., distributed from the distribution server computer 100.

[0079] It goes without saying that the audio output device 322 may not be a standalone device, but may be integrated with the display device 310 like the head-mounted display (HMD) described above.

[0080] The operation input device 321 can be any device that allows the viewer to perform various operations such as movement operations including moving the viewer avatar, zoom operations, operations related to comments, operations related to gifts, etc., and can be composed of at least one of, for example, a transparent touch panel formed on the surface of a smartphone display, a keyboard that constitutes a computer, a game controller, etc.

[0081] In this Example 1, as will be described later, the virtual audience area in the virtual live venue is divided into a third area near the virtual stage G in which only viewer users who have been specially permitted by the performer user can virtually participate, and a fourth area around the third area in which general viewer users can virtually participate, as shown in Figure 12. As described above, viewer users who virtually participate in the fourth area can use any operation input device 321 such as a touch panel, keyboard, or game controller, but viewer users who virtually participate in the third area can use an operation input device 321 such as an MVN motion capture suit with the above-mentioned inertial sensor built in, and can therefore, like performer users, control the viewer avatar in detail with their own movements.

[0082] However, even viewer users who are permitted to virtually participate in the third area cannot change the facial expressions of the viewer avatars, but the present invention is not limited to this, and the facial expressions of these viewer avatars may be changed, for example, as described above, by the viewer user using the camera function of a smartphone or a camera connected to a computer to recognize the viewer user's facial expression, even if it is at a level different from the change level of the performer user.

[0083] The various input / output devices connected to these input / output interfaces (I / F) 305 may be built-in or external, and in the case of external devices, the connection may be either wired or wireless.

[0084] In addition to an operating system (OS) for operating the viewer terminal 300, which is a smartphone or computer, the storage 303 also stores a viewer program as a viewing application that, when executed by the processor (CPU) 301 or the like, works in cooperation with the distribution server computer 100 to provide various functions, including a live viewing function for viewer users in a virtual live, which is virtual space content.

[0085] As shown in FIG. 5, the viewer program includes a viewer viewpoint image control program that provides the function of generating each viewpoint image of the virtual live to be displayed on the display device 310, a viewer audio control program that provides the function of outputting live audio from the audio output device 322 based on the live audio (data) of the virtual live distributed from the distribution server computer 100, a viewer comment control program that displays comments based on comment data distributed from the distribution server computer 100 and provides comment-related functions such as comments entered by the viewer and sending them to the distribution server computer 100, and viewer authentication data for authenticating the identity of the viewer user in a communication connection with the distribution server computer 100.

[0086] Although not shown in Figure 5, in addition to the above-mentioned programs, the program may include a gift control program that controls gifts, and an area designation program for reserving an area (position) in the virtual live venue where a viewer avatar will virtually participate in the live event in advance, as shown in Variation Example 3-1 (see Figures 42 and 43) described below, but other programs may also be included.

[0087] In the first embodiment, the comments input by the viewer are short messages in text format, but are not limited to these short messages in text format and may be, for example, messages in still image format, messages in video format, or any other electronic message format. The comments are displayed on the display device 310 superimposed on each viewpoint video.

[0088] In this way, by using the viewer terminal 300 that constitutes the virtual space content distribution system of this Example 1, the viewer user can move and operate the viewer avatar that is virtually participating in the virtual live venue, and by changing the viewpoint of the viewer avatar, the viewpoint image displayed on the display device 310 can be changed, and the zoom state (viewpoint state) can also be changed, allowing the viewer avatar to operate and cheer on the performer avatar 1.

[0089] Note that a viewer avatar virtually participating in the third area is only allowed to move within the third area and cannot move to the fourth area. On the other hand, a viewer avatar virtually participating in the fourth area is only allowed to move within the fourth area and cannot move to the third area. However, the present invention is not limited to this, and a viewer avatar virtually participating in three areas may be allowed to move to the fourth area. Note that, naturally, a viewer avatar cannot move onto the virtual stage G, etc., so the viewer avatar's movement onto the virtual stage G will not obstruct the visibility of the performer avatar 1.

[0090] Furthermore, by displaying viewpoint images on these display devices 310 and outputting live audio, users can enjoy the virtual live performance with the feeling that they are actually present at the virtual live performance venue.

[0091] Although a detailed explanation is omitted, the comment function provided by the viewer comment control program allows viewers to enjoy the virtual live performance while checking comments entered by themselves and other viewer users, and the gift function provided by the gift control program allows viewers to liven up the virtual live performance by gifting items they own to performer avatar 1.

[0092] <Administrator terminal> Fig. 6 is a diagram showing an example of the configuration of an administrator terminal 150 in the virtual space content distribution system of this embodiment 1. In this embodiment 1, the administrator terminal 150, together with the performer terminal 200, uses a normal computer installed in a control room adjacent to the studio, and as shown in Fig. 6, has a processor (CPU) 151 connected to a data bus 158, a memory (RAM) 152, a storage 153 such as a hard disk, a communication interface (I / F) 154 capable of two-way communication using the TCP / IP protocol via the Internet, which is an open computer network, an image processing unit 156 including a graphics processing unit (GPU) connected to a display device 160, and an input / output interface (I / F) 155 connected to various input / output devices.

[0093] As will be described later, the display device 160 is configured by one or more stationary displays, and is capable of individually displaying the viewpoint images of each of the virtual cameras C1 to C4 described below and the virtual camera moving in the air during the flying performance period, which are generated based on the virtual space data of the virtual live venue, which is a virtual space updated by the virtual space update data (see Figure 10) distributed from the distribution server computer 100, using an administrator viewpoint image control program included in the administrator program stored in storage 153, as well as being capable of displaying comments based on the comment data distributed from the distribution server computer 100.

[0094] The input / output interface (I / F) 155 is connected to an operation input device 161 consisting of one or more devices that allow the event manager M to perform various operations as input devices, and an audio output device 162 such as earphones, headphones, or speakers that can output live audio, etc., distributed from the distribution server computer 100.

[0095] The operation input device 161 can be any device that allows the event manager M to perform various operations such as input and settings related to the virtual live event, and may be composed of at least one of, for example, a keyboard that constitutes a computer, a transparent touch panel formed on the surface of a display, a viewpoint switching controller, etc.

[0096] The various input / output devices connected to the input / output interface (I / F) 155 may be connected either by wire or wirelessly.

[0097] In addition to the operating system (OS) for operating the computer that is the administrator terminal 150, the storage 153 also stores an administrator program that is executed by the processor (CPU) 151, etc. to provide various functions in cooperation with the distribution server computer 100, including distribution management functions related to the distribution of virtual live shows, which are virtual space content.

[0098] As shown in FIG. 6, the administrator programs include an administrator viewpoint video control program, an administrator audio control program, an administrator comment control program, an audience management program, a performer management program, and the like.

[0099] The viewpoint video control program for administrator is a program that provides a function for generating viewpoint videos from each virtual camera to be displayed on the display device 160, as well as a function for changing and switching between each viewpoint.

[0100] The administrator audio control program is a program that provides a function for outputting live audio from the audio output device 162 based on audio data of a virtual live event distributed from the distribution server computer 100 .

[0101] The administrator's comment control program is a program that displays comments based on comment data distributed from the distribution server computer 100, and provides comment-related functions such as prohibiting comments from being distributed and selecting viewers to prevent them from being distributed.

[0102] The viewer management program is a program for managing viewer users who virtually participate in the virtual live performance, and the performer management program is a program for managing performer users who virtually participate in the virtual live performance.

[0103] Although not shown in Figure 6, the system includes an authentication program for authenticating whether the administrator who is the operator is the real person, and a schedule program for editing the time schedule and song order of the virtual live performance, but other programs may also be included.

[0104] In this way, by operating the administrator terminal 150 that constitutes the virtual space content distribution system of this embodiment 1, the event administrator M can implement various settings for executing the event, such as settings related to the program, order of songs, performance, time schedule, and the trajectory of the aerial moving virtual camera during the flying performance period, and the virtual live performance is managed based on the settings made in this way.

[0105] <Virtual live venue> The virtual live venue, which is the virtual space used in Example 1, is shown in Fig. 12. As shown in Fig. 12, the virtual live venue, like a live venue in real space, is a virtual space that has a virtual stage G where performer users appear as performer avatars 1 and a virtual audience area where viewer users who will become audience members virtually participate as viewer avatars.

[0106] As shown in Fig. 12, virtual stage G has a trapezoidal floor with a relatively large area, and performer avatar 1 can move on virtual stage G in the same way as in a live venue in real space. On the side of virtual stage G opposite the virtual audience seats, as shown in Fig. 12, a virtual stage wall is formed that is divided into three parts: a central part, a right part, and a left part. In front of these virtual stage walls, virtual displays 40, 41, and 42 are provided, and images and videos for performance purposes are virtually displayed on these virtual displays 40, 41, and 42.

[0107] A virtual star-shaped object 43 related to the performer user is placed between the virtual displays 40, 41, and 42, and large virtual star-shaped aerial objects 50 to 52 are placed in a second area on the virtual stage G, which is an area above the first area surrounded by the virtual stage walls, as shown in Fig. 12. The virtual aerial objects 50 to 52 are movable within the second area.

[0108] A virtual audience area where viewer users can virtually participate in the virtual live performance venue as viewer avatars is provided in front of the virtual stage G. As shown in Fig. 12, this virtual audience area is divided into a third area, which is a special area near the center of the virtual stage G where performer avatar 1 appears, and a fourth area, which is formed to surround the third area and is located at a greater distance from the center of the virtual stage G than the third area.

[0109] These third areas are available to special viewer users who are permitted to participate virtually, up to a predetermined upper limit. In this embodiment 1, as shown in FIG. 12, for example, viewer avatars 11 to 14 of four viewer users who have met a predetermined virtual participation record and have been permitted to participate virtually by the performer user are placed.

[0110] In addition, the fourth area is arranged with viewer avatars 21 to 32 of general viewer users who have booked virtual participation before the start of the virtual live event. Note that in Fig. 12, the viewer avatars 21 to 32 are shown in a simplified manner for convenience, but are the same avatars as the viewer avatars 11 to 14.

[0111] Furthermore, each viewer avatar 11 to 14 placed in the third area can be moved within the third area by the viewer user operating the viewer terminal, and each viewer avatar 21 to 32 placed in the fourth area can also be moved within the fourth area by the viewer user operating the viewer terminal.

[0112] Four virtual cameras C1 to C4 are virtually placed (set) in the virtual live performance venue, as shown in Fig. 12. Virtual camera C1 is virtually placed in front of performer avatar 1 so as to face the performer avatar 1, and the viewpoint image captured by virtual camera C1 is a viewpoint image viewed from a position close to the front of performer avatar 1, as shown in Fig. 13(a).

[0113] Virtual camera C2 is a camera virtually placed (set) above the head of performer avatar 1, and the viewpoint image from virtual camera C2 is a performer viewpoint image looking at the virtual audience area from performer avatar 1, as shown in Figure 15.

[0114] Virtual camera C3 is a virtual camera virtually placed on the virtual display 41 diagonally behind the performer avatar 1, and the viewpoint image from virtual camera C3 is a viewpoint image looking at the virtual audience area from diagonally behind the performer avatar 1, as shown in Figure 13(c).

[0115] Virtual camera C4 is a virtual camera virtually placed on the virtual display 42 diagonally behind the performer avatar 1, and the viewpoint image from virtual camera C4 is a viewpoint image looking at the virtual audience area from diagonally behind the performer avatar 1, as shown in Figure 13(b).

[0116] Furthermore, in this Example 1, in addition to the fixedly placed (set) virtual cameras C1 to C4, aerial moving virtual cameras (not shown) are virtually placed so as to be movable in the air within the virtual space of the virtual live venue. The aerial trajectories along which these aerial moving virtual cameras move and the viewpoint directions (angles) at individual positions on the aerial trajectories are set in advance by the administrator terminal 150. During the flying performance period described below, the performer avatar 1 virtually flies so as to follow the aerial trajectory of the aerial moving virtual camera, thereby generating a viewpoint video of the virtually flying performer avatar 1 captured by the aerial moving virtual camera.

[0117] In this embodiment 1, as shown in Figure 12, an example is given in which four virtual cameras C1 to C4 are virtually placed as fixedly placed (set) virtual cameras, but the present invention is not limited to this, and the number of these fixedly placed (set) virtual cameras may be five or more, or conversely, without setting up fixedly placed (set) virtual cameras, only the viewpoint images seen from the viewpoints corresponding to each avatar virtually participating in the virtual live venue may be displayed on the performer terminal 200 and the viewer terminal 300.

[0118] Furthermore, in this Example 1, an example is given of a configuration in which there is one virtual camera moving in the air, but the present invention is not limited to this, and a configuration in which there are multiple virtual cameras moving in the air, or conversely, a configuration in which there is no virtual placement of a virtual camera moving in the air, may also be used.

[0119] <Studio> 11 is a diagram showing the studio used in Example 1, which has an adjacent control room separated by a glass window as described above, in which performer users perform various movements corresponding to singing in the studio, and an assistant operator O assists with the operation of performer terminal 200 installed in the control room. Around performer users wearing wearable sensors 220C1 to 220C5, base stations 220a and 220b are mounted on stands, and an imaging camera constituting facial expression input device 221 is mounted on a stand and positioned substantially in front of the performer users.

[0120] A screen S is installed on the wall facing the performer user in the studio, and a projector (display device B211) installed on the ceiling of the studio projects the viewpoint image of virtual camera C2 onto screen S, so that the viewpoint image seen from performer avatar 1 as seen in the virtual audience area is displayed on screen S, as shown in Figure 15.This allows the performer user to proceed with the live performance while constantly keeping track of the behavior of the viewer avatars virtually participating in the virtual audience area (the audience's reactions to the live performance).

[0121] A large vertical display serving as the display device C212 is placed to the side of the screen S, and is configured to display comments from viewers and messages that the assistant operator O wants to convey to the performer users. The display device C212 may also be configured to display information (performer support information) such as the lyrics and lines of the songs sung by the performers during the live performance.

[0122] <Operation of virtual space content distribution system> FIG. 10 is an explanatory diagram illustrating the operation of the virtual space content distribution system of the first embodiment, specifically, the flow of distribution of video and audio of a virtual live performance.

[0123] First, the video system will be described. As mentioned above, distribution server computer 100 is capable of providing a virtual space of a virtual live venue based on virtual space generation program 120 and avatar data of performer users and viewer users, and the movements of each avatar of these performer users and viewer users are reflected in the movements of performer avatar 1, and the movements (operations) of viewer users are reflected in the movements of the viewer avatar, as virtual space data describing the state of the virtual space is updated by performer avatar information transmitted from performer terminal 200 and viewer avatar information transmitted from viewer terminal 300.

[0124] Although not shown in Figure 10, when the performer terminal 200 and the viewer terminal 300 communicate with the distribution server computer 100 to virtually participate in the virtual live performance, virtual space data of the virtual live performance venue is delivered to and stored in advance from the distribution server computer 100.As will be described later, this pre-stored virtual space data is sequentially updated by virtual space update data that is continuously delivered from the distribution server computer 100, allowing the performer terminal 200 and the viewer terminal 300 to identify the latest state of the virtual live performance venue.

[0125] Specifically, the movements and facial expressions of the performer user in the studio are scanned at predetermined time intervals using the body motion scan and face motion scan functions of the performer avatar control program, and performer avatar information is generated and sent to the distribution server computer 100.

[0126] On the other hand, viewer users who are permitted to virtually participate in the third area use motion sensors, just like performers, and are scanned at predetermined time intervals using the body motion scan and face motion scan functions, and viewer avatar information is generated and sent to the distribution server computer 100.

[0127] Furthermore, a viewer user permitted to virtually participate in the fourth area can use the touch panel, keyboard, or controller of the viewer terminal 300 to perform various operations such as movement, changing the viewpoint direction, changing the viewpoint (zoom), raising a hand, clapping a hand, jumping, etc., to make the viewer avatar perform the corresponding action. Then, viewer avatar information is generated based on the operation related to the viewer avatar's action among these operations, and is transmitted to the distribution server computer 100.

[0128] Furthermore, viewer users virtually participating in the third area can also use a controller or the like to perform various operations such as moving, changing the viewpoint (zoom), raising their hands, clapping their hands, jumping, etc. In this case, a menu item corresponding to each action is displayed, and the menu item corresponding to each action can be selected using a controller or the like.

[0129] In this way, an avatar action operation process is executed in the viewer terminal 300 to receive operations corresponding to each action to be performed by the viewer avatar. The avatar action operation process also includes processes such as displaying and erasing menu items corresponding to each of the above-mentioned actions.

[0130] In this way, based on the performer avatar information transmitted from the performer terminal 200 and the viewer avatar information transmitted from the viewer terminal 300, the distribution server computer 100 executes a virtual space update process to update the virtual space data, thereby reflecting the actions (operations) of the performer users and the viewer users in each avatar virtually participating in the virtual live venue.

[0131] Then, after executing a virtual space update data generation process for generating virtual space update data based on the virtual space data before the update and the virtual space data after the update, the generated virtual space update data is distributed to the performer terminal 200 and the viewer terminal 300.

[0132] As shown in Figure 10, at the performer terminal 200 and viewer terminal 300 to which the virtual space update data has been distributed, the pre-stored virtual space data is updated based on the distributed virtual space update data, and a performer perspective video generation process and a viewer perspective video generation process are executed using the updated virtual space data, thereby generating performer perspective video and viewer perspective video based on the updated virtual space data, and the generated performer perspective video and viewer perspective video are displayed on the performer terminal 200 and the viewer terminal 300 (viewer perspective video output process).

[0133] In this way, in this embodiment 1, the viewpoint images of each viewer user who is virtually participating in the virtual live venue using a viewer avatar are generated and displayed on the viewer terminal 300 of each viewer user.By generating the viewer avatar viewpoint images of each viewer user on the distribution server computer 100, it is possible to avoid a situation in which the processing load on the distribution server computer 100 becomes significantly large, which would prevent many viewer users from being able to participate in the virtual live performance, and it is also possible to avoid a situation in which distribution becomes difficult due to an increased processing load.

[0134] In this embodiment 1, the display device 310 of the viewer terminal 300 of the viewer user of viewer avatar 13 who is virtually participating in the third area close to the performer avatar 1 on the virtual stage G generates and displays a viewpoint image in which the almost front of the performer avatar 1 is displayed large, which is the viewpoint image seen from the viewer avatar 13 who is close to the performer avatar 1, as shown in Figure 13(d).On the other hand, the display device 310 of the viewer terminal 300 of the viewer user of viewer avatar 28 who is virtually participating in the fourth area far from the performer avatar 1 on the virtual stage G, for example, at a position behind the viewer avatar 13, displays a viewpoint image in which the almost front of the performer avatar 1 appears relatively small through the viewer avatar 13 and viewer avatar 14 who are virtually participating in the third area, as shown in Figure 13(e).

[0135] In addition, the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 32 who is virtually participating at the end position of the virtual stage G in the fourth area, which is set up in a U-shape when viewed from above, displays a viewpoint image as seen from the viewer avatar 32 positioned diagonally forward of the performer avatar 1, as shown in Figure 13(f), in which the performer avatar 1 appears relatively small over the viewer avatar 14 who is virtually participating in the third area.

[0136] Note that all of the display examples shown in Figure 13 illustrate the case where the viewer terminal 300 is a stationary computer (PC) and the display device 310 is a stationary display. However, if the viewer terminal 300 is a smartphone P, tablet, etc., the viewpoint image of the virtual camera C1 will be displayed as shown in Figure 14(a), and if the viewer terminal 300 is the viewer terminal 300 of the viewer avatar 13, it will be displayed as shown in Figure 14(b).

[0137] Next, regarding the audio system, in the distribution server computer 100, music data 113 is played back by the audio control program 124 based on the time schedule data (TS data) 112, and is transmitted to the performer terminal 200 as a played music piece (data) (music playback process).

[0138] In the performer terminal 200, the music audio from the played music (data) sent from the distribution server computer 100 is output to the performer user through high-performance earphones (ear monitors) worn by the performer user (played music output process), and when the performer user sings along with the output played music, the performer voice input from the sound collection microphone (voice input device 222) is converted into data and sent to the distribution server computer 100 as the performer voice (data).

[0139] The distribution server computer 100 generates distribution audio (data) from the performer audio (data) received from the performer terminal 200 and the music played back in the music playback process described above (distribution audio generation process), and distributes the generated distribution audio (data) to the viewer terminal 300 (generated audio distribution process).

[0140] At the viewer terminal 300, the distributed audio (data) distributed from the distribution server computer 100 is output from an audio output device 322 such as earphones or speakers, allowing the viewer to hear the live audio as if the performer avatar 1 were singing in a virtual live venue.

[0141] Furthermore, as described above, in the performer terminal 200, the performer's voice (data) is transmitted to the distribution server computer 100, and as shown in Figure 10, a monitor voice generation process and a monitor voice output process are executed, so that the music voice of the played song and the performer's voice are output from speakers located in the control room, allowing an assistant operator O or the like to check the performer user's singing status, etc. in the control room.

[0142] <Viewer's viewpoint video control processing> Next, a viewer's viewpoint video control process executed in the viewer terminal 300 based on a viewer's viewpoint video control program included in the viewer's program will be described with reference to FIG.

[0143] In the viewer viewpoint video control process, first, it is determined whether or not the performer avatar 1 is currently in a flying effect period, in which the performer avatar 1 is virtually flying, as shown in Fig. 19 (step S1). Whether or not the performer avatar 1 is currently in a flying effect period can be determined from effect status data indicating that the performer avatar 1 is currently in a flying effect period, which is sent from the delivery server computer 100. When the delivery server computer 100 determines that the performer avatar 1 is currently in a flying effect period based on the time schedule data, the delivery server computer 100 sends effect status data indicating that the performer avatar 1 is currently in a flying effect period to the viewer terminal 300.

[0144] If the flying effect is in progress (Y in step S1), the process proceeds to step S21. If the flying effect is not in progress (N in step S1), it is determined whether viewpoint-related operations, including viewpoint movement operations (including viewpoint changes due to avatar movement operations and motion operations) and zoom operations, are disabled (step S2).

[0145] If the viewpoint-related operation is invalid (Y in step S2), the process proceeds to step S7. If the viewpoint-related operation is not invalid (N in step S2), it is determined whether or not there is an operation that changes the viewpoint, such as an operation to move or move the avatar (step S3), and if there is an operation (Y in step S3), the viewer viewpoint is changed according to the operation, and if there is no operation (N in step S3), the process proceeds to step S5, where it is determined whether or not there is a zoom operation (step S5).

[0146] If a zoom operation is performed (Y in step S5), the image area displayed on the display device 310 is changed to an image area corresponding to the operation, and if a zoom operation is not performed (N in step S5), it is determined whether or not a motion tracking setting is present (step S7).

[0147] This motion tracking setting can be set by the viewer user at will on the viewer terminal 300, for example, before the start of a live performance or during a live performance, depending on the type of device they use for operation and their level of proficiency in operation. Specifically, this setting is set when, for example, they are not familiar with operating the operation input device 321, such as a touch panel, keyboard, or game controller, or when they are not familiar with changing the viewpoint by moving or moving the avatar, or when, as will be described later, they are unable to properly change the viewpoint in response to the movement or movement of the performer avatar 1, for example, when the virtual participation position of their own viewer avatar is far from the performer avatar 1 and the display is zoomed in, causing the performer avatar 1 to be distorted even by the slightest movement of the performer avatar 1.

[0148] If there is no motion follow-up setting (N in step S7), the process proceeds to step S11, whereas if there is a motion follow-up setting (Y in step S7), it is further determined whether the motion follow-up condition is met (step S8).

[0149] In this embodiment 1, the motion tracking condition is a condition under which it is almost certain that the performer avatar 1 will no longer be displayed properly on the display device 310. Specifically, if the head (or face) of the performer avatar 1 is within a specified range around the display area of ​​the display device 310, it is highly likely that the movement or motion of the performer avatar 1 will cause the head (face) to move outside the display area, causing the performer avatar 1 to no longer be displayed properly, and therefore it is determined that the motion tracking condition is met.

[0150] In this way, in the first embodiment, by determining that the movement tracking condition is met before the head (face) of the performer avatar 1 disappears from the display area, it is possible to significantly reduce the occurrence of the head (face) of the performer avatar 1 disappearing from the display area, but the present invention is not limited to this, and these movement tracking conditions may be the movement of the head (face) outside the display area, or other conditions may be used as the movement tracking conditions. In other words, any conditions can be used as these movement tracking conditions as long as they can prevent the performer avatar 1 from disappearing from the display area properly.

[0151] If the action tracking condition is met (Y in step S8), after disabling the viewpoint-related operation (step S12), the viewpoint image displayed in the display area of ​​the display device 310 is automatically changed to an action tracking viewpoint in which the head (face) of the performer avatar 1 is within a non-establishment range surrounded by a predetermined range around the display area, and the viewpoint image seen from the action tracking viewpoint is displayed in the display area of ​​the display device 310 (step S13).

[0152] On the other hand, if the action tracking condition is not met (N in step S8), it is further determined whether or not the viewpoint-related operation is disabled (step S9), and if the viewpoint-related operation is disabled (Y in step S9), the disabled viewpoint-related operation is enabled (step S10) and then the process proceeds to step S11, and if the viewpoint-related operation is not disabled (N in step S9), the process proceeds to step S11 without going through step S10.

[0153] In step S11, since the action tracking setting is not set or the action tracking condition is not met, the viewpoint video of the viewer avatar continues to be displayed in the display area of ​​display device 310 (step S11).

[0154] Also, as mentioned above, if the flying performance period in which the performer avatar 1 virtually flies during a live performance has arrived, the result of step S1 is judged as Y, and the process proceeds to step S21, where it is determined whether it is the start timing of the flying performance period (step S21).

[0155] If it is the start timing of the flying performance period (Y in step S21), the flying viewpoint image delivered from the delivery server computer 100 is displayed (step S22) in place of the viewer avatar viewpoint image or action-tracking viewpoint image that was displayed in the display area of ​​the display device 310 before the start of the flying performance period, and then viewpoint-related operations are disabled (step S23) and the process returns to step S1.

[0156] In this embodiment 1, during the flying performance period, the same flying viewpoint video is displayed on the viewer terminals 300 of all viewer users. Therefore, instead of generating the flying viewpoint video on each viewer terminal 300 side, like viewer avatar viewpoint video that differs for each viewer user, the flying viewpoint video is generated on the distribution server computer 100 side and distributed to the viewer terminals 300. This allows the flying viewpoint video, which involves movement of the viewpoint through the air and therefore places a heavy load on the viewer terminals 300 to be displayed well even on viewer terminals 300 with low processing power. However, the present invention is not limited to this, and these flying viewpoint videos may be generated on each viewer terminal 300 side, just like viewer avatar viewpoint videos.

[0157] On the other hand, if it is not the start timing of the flying effect period (N in step S21), it is further determined whether it is the end timing of the flying effect period (step S24).

[0158] Whether or not it is the end timing of the flying performance period may be determined based on data indicating the end contained in the flying viewpoint video data transmitted from the distribution server computer 100, or may be determined based on the aforementioned performance status data that is different from the flying viewpoint video.

[0159] If it is not the end of the flying performance period (N in step S24), that is, if it is during the flying performance period, proceed to step S26, update the flying viewpoint video, and then return to step 1; on the other hand, if it is the end of the flying performance period (Y in step S24), enable viewpoint-related operations (step S25), proceed to step S11, display the viewer avatar's viewpoint video in the display area of ​​display device 310 instead of the flying viewpoint video that was displayed in the display area of ​​display device 310 during the flying performance period, and then return to step S1.

[0160] The viewpoint videos displayed on the viewer terminal 300 as a result of the viewer viewpoint video control process shown in FIG. 20 being executed in the viewer terminal 300 of the first embodiment will be described with reference to FIGS.

[0161] Fig. 16 is a diagram showing the display contents of the display device 310 when a zoom operation is performed on the viewer terminal 300 of Example 1. In the default state where no zoom operation is performed, as shown in Fig. 16(a), a viewpoint video of the video area centered on the upper body of the performer avatar 1 is displayed. In this default state, when the viewer user performs a zoom-up operation, for example, by touching the "+" display (not shown) if the viewer terminal 300 is a smartphone P, or by operating the "+" key on the keyboard if the viewer terminal 300 is a computer (PC), a viewpoint video of the video area centered on the head (face) of the performer avatar 1 is displayed, for example, as shown in Fig. 16(b).

[0162] On the other hand, if the viewer user performs a zoom-down operation, for example, by touching the "-" display (not shown) if the viewer terminal 300 is a smartphone P, or by operating the "-" key on the keyboard if the viewer terminal 300 is a computer (PC), a viewpoint image of the image area including the entire body of the performer avatar 1 will be displayed, as shown in Figure 16(c), for example.

[0163] In Figure 16, in order to make the zoom function easier to understand, the explanation was given using the viewpoint image of virtual camera C1 rather than the viewpoint of the viewer avatar, but as shown in Figure 17, zooming up and down can also be performed in the same way from the viewpoint of the viewer avatar.

[0164] Also, although not described in detail in the viewer viewpoint image control process of Figure 20, for example, when performer avatar 1 appears on virtual stage G for the first time when a live performance begins, the viewpoint of the viewer avatar will not be directed toward performer avatar 1, and there is a possibility that the appearing performer avatar 1 will not be displayed on the viewer terminal 300 for a long period of time.Therefore, the period when performer avatar 1 first appears may be set as an appearance performance period, and during this performance period, the viewpoint image displayed on the viewer terminal 300 may be automatically switched to the viewpoint image of virtual camera C1, etc., rather than the viewpoint image of the listener avatar, and individual viewer users may be allowed to zoom in on these viewpoint images of virtual camera C1, etc.

[0165] 17 shows an example of a viewer avatar's viewpoint image, that of viewer avatar 28, who is virtually participating in area 4. As shown in FIG. 17, the viewpoint image of viewer avatar 28 includes the heads of viewer avatar 13 and viewer avatar 14, who are virtually participating in area 3, and the image shows performer avatar 1 seen through the heads of viewer avatar 13 and viewer avatar 14.

[0166] Therefore, the viewer user of viewer avatar 28 can, for example, perform a zoom-in operation to change the image area so that the upper body of performer avatar 1 is displayed in the center, as shown in Figure 17(b), thereby preventing viewer avatar 13 and viewer avatar 14 from being displayed and allowing the viewer user to enjoy the live performance.

[0167] However, in the zoomed-in state shown in Figure 17(b), for example, as the live performance progresses and the performer avatar 1 begins to move and make large movements, if the viewer user is unable to properly change the viewpoint of the viewer avatar 28 to match the movements of the performer avatar 1, the performer avatar 1 may not be displayed properly, as shown in Figures 18(d) to (f).In this case, if the viewer pays attention to adjusting the viewpoint, it may become difficult to concentrate on the live performance and they may not be able to enjoy it properly.

[0168] However, in this Example 1, when the viewer user sets the action tracking setting as described above, as shown in Figures 18(a) to (c), the action tracking condition is met when the head of the performer avatar 1 enters a specified area around the display area, the viewpoint is automatically changed to the action tracking viewpoint, and the viewpoint image of the action tracking viewpoint is displayed, so the viewer user does not need to pay attention to operating the viewpoint, and can concentrate on the live performance and enjoy it to the fullest.

[0169] As the live performance progresses and the flying performance period begins, as shown in Figure 19, the viewpoint images of individual viewer avatars are automatically changed to flying viewpoint images seen from a virtual camera moving in the air, without any operation by the viewer user.

[0170] Specifically, before performer avatar 1 takes to the air, an image of a wing object growing larger is displayed on the back of performer avatar 1. Then, performer avatar 1 looks up at virtual star object 50 in the air, which is virtually placed in the second area of ​​the virtual live venue, and the viewpoint of the viewer avatar moves in the same way. At this time, light effects 61 appear around performer avatar 1.

[0171] After that, when the performer avatar 1 begins virtual flight, the viewpoint image displayed on the display device 310 automatically switches from the viewpoint image of the viewer avatar up to Figure 19(b) to a flying viewpoint image seen from a virtual camera moving in the air, as shown in Figure 19(c).

[0172] During the flying performance period, the aerial moving virtual camera moves through the air within the second area based on a pre-set aerial trajectory, as described above, and the flying viewpoint image seen from this aerial moving virtual camera is displayed on the display device 310 as shown in Figure 19(d).

[0173] <Modification of Example 1> In the above-described first embodiment, the general action (first action) is an action in which the performer avatar 1 stands without moving to the virtual stage G, the special action (second action) is an action in which the performer avatar 1 performs a virtual flight, and the specific action (third action) is an action in which the performer avatar 1 moves suddenly sideways. However, the present invention is not limited to this and may be an action corresponding to the content to be distributed. For example, the general action (first action) may be an action (normal action) that the performer avatar performs for the longest period of time in the distributed content, the special action (second action) may be an action that is performed in the shortest special period in the distributed content or the action that is performed least frequently, and the specific action (third action) may be an action that is performed for a shorter period of time than the general action (first action) but a longer period of time than the special action, or an operation that is performed less frequently than the general action (first action) but more frequently than the special action. Note that the special action (second action) may include the specific action (third action).

[0174] Furthermore, in the above-mentioned Example 1, an example was given of a form having both a function of switching to a viewpoint corresponding to a special action (second action) and a function of following a specific action (third action), but the present invention is not limited to this, and may be a virtual space content distribution system having only one of these functions.

[0175] In the above-mentioned Example 1, an example was given in which the viewpoint of the viewer user is changed and followed in accordance with the movements of the performer avatar, but the present invention is not limited to this, and for example, these changes and following of the viewpoint may be performed in accordance with specific effects such as smoke or fireworks (pillars of fire) devices, special lighting, deformation of the virtual stage G, etc., which are effects on the virtual stage G. [Example]

[0176] Next, a virtual space content distribution system according to a second embodiment will be described below with reference to Fig. 21 to Fig. 28. Note that the same components as those in the first embodiment will be denoted by the same reference numerals and the description thereof will be omitted.

[0177] The configuration of the virtual space content distribution system of Example 2 is the same as that of Example 1 shown in Figure 1, but the progress of the virtual live performance is different, and therefore the period during which viewpoint images from virtual cameras C1 to C4 and the airborne moving virtual camera (hereinafter collectively referred to as virtual camera viewpoint images) are distributed and the entity that generates these virtual camera viewpoint images are different from those of Example 1, which is a characteristic feature of Example 2.

[0178] 21 is a diagram showing the progress of a virtual live concert distributed in the virtual space content distribution system of Example 2. The virtual live concert of Example 2 first has a "live virtual space participation period" and a "pre-show" period as periods before the start of the virtual live concert, and during the live virtual space participation period, viewer users can virtually participate in the virtual live concert venue, which is the virtual space that is the target of the virtual live concert.

[0179] The period up to the timing for checking the completion of the pre-data download, which is set just before the end of the "pre-show speech" period, is the pre-data download (DL) period, and virtual space data (basic data) of the virtual live venue, avatar data of performer avatar 1, and avatar data of other viewer users who participated virtually during the period during which participation in the live virtual space is possible are downloaded (DL) as pre-data from the distribution server computer 100 to the viewer terminal 300 of the viewer user who participated virtually.

[0180] At the timing of checking whether the advance data DL has been completed, which is the timing at which the advance data DL period ends, it is checked whether the advance data DL has been completed at each viewer terminal 300.

[0181] Note that if the download of the advance data (DL) is not completed by the time of the advance data DL completion check timing due to differences in the model, processing power, or communication environment of the viewer terminal 300, the download of the advance data (DL) will continue. However, in this case, data that is highly important for the distribution of the virtual live show, such as the virtual space data (basic data) of the virtual live venue required for the opening performance and the first MC, and the avatar data of performer avatar 1, will be downloaded (DL) with priority.

[0182] Furthermore, in Example 2, as described above, an example is given in which the data required to generate viewpoint images at the viewer terminal 300 during the distribution of a virtual live performance is downloaded (DL) as advance data, but the present invention is not limited to this, and this data, particularly special data required individually for each performance and each MC described below, may be downloaded (DL) sequentially each time at a timing corresponding to the timing at which each performance and each MC is distributed.

[0183] During the "pre-show introduction" period, explanations about the virtual live performance are provided through video footage, etc. Therefore, by making the "pre-show introduction" period a period during which advance data can be downloaded (DL), each viewer terminal 300 can also download (DL) from the distribution server computer 100 the avatar data of viewer avatars who virtually participate in the live virtual space just before the end of the participation period.

[0184] As shown in FIG. 21, the virtual live performance in Example 2 progresses as follows: "Opening performance" → "First MC" → "First performance" → "Second MC" → "Second performance" ... "Opening performance" is a performance in which performer avatar 1 appears on virtual stage G. "First MC" is a performance in which performer avatar 1 appears on virtual stage G, greets the audience through a talk, and introduces the first performance to be performed. "First performance" is a performance that includes singing and dancing by performer avatar 1 (performer), and also includes a flying performance in which video of performer avatar 1 virtually flying around the virtual live venue is captured by a virtual camera moving in the air. "Second MC" is a performance in which performer avatar 1 (performer) mainly introduces the second performance through a talk. The "second performance" includes singing and dancing by performer avatar 1 (performer), and also includes a costume change performance using footage virtually captured by virtual camera C1 of performer avatar 1 changing clothes, and an avatar split performance using footage virtually captured by virtual camera C1 of performer avatar 1 splitting into multiple people.

[0185] The processing load on the distribution server computer 100 during each period in which these effects and programs are executed is the lowest ("low") during the period in which participation in the live virtual space is possible, while it is the highest ("high") during the period in which the special effects "flying effect," "costume change effect," and "double-self effect" are executed. The processing load on the distribution server computer 100 during the "first performance" from the "introduction," "opening effect," "first MC," and "flying effect" is "medium." The processing load is also "medium" during the "first performance" after the "flying effect" and the "second performance" from the "second MC" and "costume change effect." The processing load is also "medium" during the "second performance" from the "costume change effect" to the "double-self effect." The processing load is also "medium" during the "second performance" from the "costume change effect" to the "double-self effect."

[0186] Next, we will explain the viewpoint images that can be viewed on the viewer terminal 300 for each period in the virtual live performance in Example 2. During the period in which participation in the live virtual space is possible, from the time of virtual participation until the start of the "pre-performance speech," only the viewer avatar viewpoint images can be displayed.

[0187] Then, when the "pre-show speech" begins, only the virtual camera viewpoint video, specifically the viewpoint video of virtual camera C1, can be displayed. In this way, the viewpoint video of the virtual camera is limited to the virtual camera viewpoint video during the "pre-show speech" even though it is before the start of the virtual live performance. This is because the pre-show speech also includes explanations of important points to note about the virtual live performance, and the purpose is to prevent viewer users from not viewing the pre-show speech being performed on virtual stage G due to the viewer avatar viewpoint video being displayed on viewer terminal 300.

[0188] As will be described later, these virtual camera viewpoint images of the "pre-show speech" are generated by the distribution server computer 100 and distributed to each viewer terminal 300, so they can also be displayed on viewer terminals 300 that have not yet completed downloading the advance data.

[0189] For viewer terminals 300 that have completed downloading the advance data at the above-mentioned advance data download completion check timing, the viewer avatar perspective image can be displayed in addition to the virtual camera perspective image during the period of the "first performance" from the "first MC" to the "flying performance", as well as the period after the "flying performance".

[0190] On the other hand, for viewer terminals 300 that have not completed downloading the advance data at the above-mentioned advance data download completion check timing, the viewer avatar viewpoint image cannot be accurately generated, and therefore, by distributing the virtual camera viewpoint image generated by the distribution server computer 100, only the virtual camera viewpoint image can be displayed during the entire virtual live broadcast, and the viewer avatar viewpoint image cannot be displayed.

[0191] Furthermore, among the images of the "Flying Performance," "Costume Change Performance," and "Double Ego Performance," which have a "high" processing load on the distribution server computer 100, the virtual camera viewpoint images of virtual camera C1 for the "Costume Change Performance" and "Double Ego Performance," as well as the virtual camera viewpoint images during the second performance period other than these "Costume Change Performance" and "Double Ego Performance," are generated on each viewer terminal 300 rather than on the distribution server computer 100, thereby reducing the processing load on the distribution server computer 100 for generating the virtual camera viewpoint images of virtual camera C1 for these "Costume Change Performance" and "Double Ego Performance."

[0192] Furthermore, since the processing load on the distribution server computer 100 during the "flying effect" period is high, it is conceivable that the "flying effect" could be generated at the viewer terminal 300, just like the "costume change effect" and "doppelganger effect." In this case, however, detailed trajectory data of the movement trajectory of the virtual camera moving through the air within the virtual live venue must be distributed in advance to each viewer terminal 300 to generate the video. Therefore, in Example 2, the "flying effect" video is generated and distributed at the distribution server computer 100, but the present invention is not limited to this, and the "flying effect" video may also be generated at each viewer terminal 300 in order to reduce the processing load on the distribution server computer 100.

[0193] Here, the participant user table used in the delivery server computer 100 of the second embodiment will be briefly described with reference to FIG.

[0194] The participant user table is a data table used to identify the participation status of viewer users in virtual live events and the download status of advance data for each virtual live event registered in the event data shown in Figure 7.As shown in Figure 23, the table is capable of storing data such as participation identification data (participation identification flag) that can identify the participation status of the participant (viewer user) in association with the participant ID (viewer user ID), the participation area of ​​the participant (viewer user), and advance data DL identification data (advance data DL identification flag) that can identify the download status of advance data.

[0195] In addition, the participant ID item is used to register all viewer user IDs of prospective participants described in the file of each participant group stored in the event data corresponding to the virtual live.

[0196] When a participant ID (viewer user ID) is registered, the participation identification data (participation identification flag) is registered as "0," which corresponds to a non-participating state, and is updated to "1," which indicates participation, when the participation of each participant (viewer user) is permitted through authentication. Therefore, the participation identification data (participation identification flag) makes it possible to identify, among the viewer users who are scheduled to participate, those who are participating and those who could not participate for some reason.

[0197] The participation area data is data that can identify whether the participation area of ​​each participant (viewer user) is the third area or the fourth area, and is data that is registered when the participation of each participant (viewer user) is permitted through authentication.

[0198] The advance data DL identification data (advance data DL identification flag) is updated from "0", indicating incomplete, to "1", indicating complete, if the advance data download (DL) is completed based on a check of whether the advance data download is complete at the advance data DL completion check timing described above. Therefore, this advance data DL identification data (advance data DL identification flag) makes it easy to identify all participants (viewer users) who have not completed the advance data download (DL) among the participants (viewer users) virtually participating.

[0199] Next, a flow of downloading advance data when a viewer user virtually participates in a virtual live concert venue in the second embodiment will be described with reference to FIG.

[0200] First, the viewer user performs an operation to participate in the virtual live event that the viewer plans to virtually participate in, for example, by selecting a participation menu or operating a specific key on the keyboard to which the participation operation is assigned, in the viewer program (viewing application) running on the viewer terminal 300.

[0201] By executing this participation operation, a participation request including the viewer user ID and authentication data for authenticating the viewer user is transmitted from the viewer terminal 300 to the distribution server computer 100.

[0202] Upon receiving the participation request, the delivery server computer 100 executes participant authentication processing, provided that the participation period is within the live virtual space participation period. In the participant authentication processing, it is determined whether the viewer user is a prospective participant of the corresponding virtual live performance based on whether the viewer user ID included in the received participation request is registered as a participant ID in the participant user database corresponding to the virtual live performance to be attended, and the viewer user is authenticated by comparing the authentication data included in the participation request with the authentication data described in the authentication information file identified from the authentication information file data stored in association with the viewer user ID in the viewer user data shown in Figure 9.

[0203] If the viewer user is a prospective participant and has been authenticated using the authentication data, the participant identification data (participation identification flag) in the participant user table is updated to "1" corresponding to "participation."

[0204] The distribution server computer 100 then transmits (replies with) advance data download instruction information to the viewer terminal 300 that sent the participation request, in order to have the viewer terminal 300 download advance data corresponding to the virtual live event to be participated in. By receiving the advance data download instruction information, the viewer terminal 300 can confirm that participation has been permitted.

[0205] In response to receiving the advance data DL instruction information, the viewer terminal 300 identifies advance data to be downloaded and transmits to the delivery server computer 100 an advance data DL request requesting the download of the identified advance data.

[0206] In response to receiving this advance data DL request, the delivery server computer 100 executes advance data transmission processing.

[0207] In this advance data transmission process, the requested advance data is transmitted to the viewer terminal 300 that sent the advance data DL request. As mentioned above, the advance data includes the virtual space data (basic data) of the virtual live venue, the avatar data of the performer avatar 1, and the avatar data of other viewer users who are already virtually participating.

[0208] In the advance data transmission process, the avatar data of the viewer user who sent the advance data download request is additionally distributed as advance data to viewer terminals 300 other than the viewer terminal 300 that sent the advance data download request but whose participation has already been confirmed in the participant data table. As a result, the avatar data of all viewer users virtually participating in the virtual live show, along with the virtual space data (basic data) and avatar data of the performer avatars, are distributed in advance to and stored in the viewer terminals 300 of each viewer user virtually participating in the virtual live show.

[0209] Then, at the aforementioned advance data download completion check timing (see Figure 21), the advance data stored in the viewer terminal 300 is checked to see if all advance data has been downloaded, and if the advance data download has been completed, the advance data download identification data (advance data download identification flag) in the participant user table is updated from "0", indicating incomplete, to "1", indicating complete.

[0210] Furthermore, for participants (viewer users) whose advance data download identification data (advance data download identification flag) remains at "0" indicating incompleteness due to the advance data download not being completed, only the virtual camera viewpoint image can be displayed, as mentioned above.

[0211] Next, a situation in which the subject from which various viewpoint videos displayed on the viewer terminal 300 are generated changes during each period of the virtual live performance will be described with reference to FIGS.

[0212] First, during the live virtual space participation period when only the viewer avatar's viewpoint video can be displayed on the viewer terminal 300, as shown in Fig. 24, the virtual space data already stored in the viewer terminal 300 is sequentially updated by receiving advance data based on virtual space update data transmitted from the distribution server computer 100, including performer avatar motion data for generating the movements of the performer avatar 1 that perform movements linked to the movements of the performer user, and the viewer terminal 300 executes a viewer viewpoint video generation process based on the updated virtual space data to generate a viewer avatar's viewpoint video, which is then output (displayed) by the video output process. Note that although the performer terminal 200 is omitted in Fig. 24, the virtual space update data is also transmitted to the performer terminal 200, as shown in Fig. 10 in Example 1.

[0213] Furthermore, during the periods of "introduction," "opening performance," and "flying performance," when only virtual camera viewpoint video can be displayed on viewer terminals 300, as shown in Fig. 25, in the virtual space update data distribution process, distribution server computer 100 transmits virtual space update data only to performer terminals 200. Then, following the virtual space update data distribution process, a virtual camera viewpoint video generation process is executed in which virtual camera viewpoint video is generated, and a virtual camera viewpoint video data distribution process is executed in which the video data of the virtual camera viewpoint video generated in the virtual camera viewpoint video generation process is distributed to all viewer terminals 300 participating in the virtual live performance.

[0214] The virtual camera viewpoint video data distributed from the distribution server computer 100 in the virtual camera viewpoint video data distribution process is displayed by executing a video output process in the viewer terminal 300 that receives the virtual camera viewpoint video data.

[0215] In other words, as shown in Figure 27, for the periods of ``introduction,'' ``opening performance,'' and ``flying performance,'' when only virtual camera viewpoint images can be displayed, the virtual camera viewpoint images are generated in distribution server computer 100.

[0216] Furthermore, during the performance period excluding the MC period and the flying performance period, during which the viewer terminal 300 can select and display either the viewer avatar viewpoint image or the virtual camera viewpoint image, as shown in FIG. 26, the virtual space update data distribution process is executed in the distribution server computer 100, and the virtual space update data is distributed to the viewer terminal 300 of each viewer user participating in the virtual live performance, thereby executing the virtual space update process in the viewer terminal 300.

[0217] Then, the viewer terminal 300 determines whether or not the viewer avatar viewpoint video has been selected as the video to be displayed. If the viewer avatar viewpoint video has been selected, the viewer avatar viewpoint video generation process is executed to generate the viewer avatar viewpoint video, similar to the flow shown in FIG. 24, and the generated viewer avatar viewpoint video is displayed by the video output process.

[0218] On the other hand, if the viewer avatar viewpoint video is not selected as the video to be displayed, i.e., if the virtual camera viewpoint video is selected as the video to be displayed, the virtual camera viewpoint video generation process is executed in the viewer terminal 300 to generate the virtual camera viewpoint video, and the generated virtual camera viewpoint video is displayed by the video output process.

[0219] Furthermore, the viewer user may be able to select which of the virtual cameras C1, C3, or C4 the virtual camera viewpoint image generated in this virtual camera viewpoint image should be, or the distribution server computer 100 may instruct all viewer terminals 300.

[0220] 27, during the program period excluding the MC period and the flying effect period when either the virtual camera viewpoint video or the viewer avatar viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated on the viewer terminal 300. In this way, during the period when either the virtual camera viewpoint video or the viewer avatar viewpoint video can be selected and displayed on the viewer terminal 300, switching between the virtual camera viewpoint video and the viewer avatar viewpoint video is performed at the viewer user's choice. Therefore, compared to when virtual camera viewpoint video data is displayed by distribution from the distribution server computer 100, switching between these videos can be performed without problems such as video interruptions, and while preventing an increase in the processing load on the distribution server computer 100 that accompanies switching between these videos.

[0221] On the other hand, during the periods of "introduction," "opening performance," and "flying performance," in which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only in the distribution server computer 100 and distributed to each viewer terminal 300. In this way, during the periods in which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only in the distribution server computer 100, so that it is possible to prevent unnecessary increases in the processing load of the entire system due to the process of generating the virtual camera viewpoint video being executed redundantly on the viewer terminal 300.

[0222] Here, the situation when the viewpoint video is switched on the viewer terminal 300 will be described with reference to FIG. 28, taking as an example a period of "flying effect" in which only the virtual camera viewpoint video can be displayed.

[0223] For example, as shown in Figure 28, if a viewer terminal 300 displays viewer avatar viewpoint video during the "first MC" period and the first performance period up to the "flying performance," and the distribution server computer 100 starts distributing virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the start of the "flying performance" period in which only virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) can be displayed, the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will not be displayed on the viewer terminal 300 during the period from the start of this distribution to the display of the virtual camera viewpoint video, which could result in a problem in which the viewer will not be able to view the first video of the "flying performance."

[0224] Therefore, at the start of a "flying performance" in which the viewpoint image is forcibly changed from the viewer avatar viewpoint image to the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air), as shown in FIG. 28, the distribution server computer 100 of Example 2 sends a BF advance notification to the viewer terminal 300 at a specific timing before the start of the "flying performance", thereby notifying the viewer terminal 300 of the distribution start (buffering start) timing at which advance distribution of the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air) will begin before the start of the flying performance period.

[0225] In response to receiving the BF advance notification, the viewer terminal 300 determines the timing at which distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) will begin from the distribution server computer 100, and begins buffering the received virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the time that this timing occurs.

[0226] The timing at which the distribution of these virtual camera viewpoint images (viewpoint images from a virtual camera moving in the air) begins, i.e., the timing at which buffering begins, can be determined appropriately taking into account differences in the image storage function of the viewer terminal 300, but can be set to a period corresponding to several tens of frames of image, for example.

[0227] In this way, the distribution server computer 100 of Example 2 starts generating and distributing the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air) of the flying effect before the start of the flying effect period, and the viewer terminal 300 buffers the distributed virtual camera viewpoint image, so that when the flying effect period starts, the image can be instantly changed from the viewer avatar viewpoint image that was displayed before the start of the flying effect period to the virtual camera viewpoint image, thereby preventing the viewer from being unable to view the initial image of the ``flying effect''.

[0228] Furthermore, when the "flying effect" period ends, because the viewer avatar viewpoint video was selected before the "flying effect" began, the viewpoint video switches from the flying effect's virtual camera viewpoint video (viewpoint video from the virtual camera moving in the air) to the viewer avatar viewpoint video in response to the end of the "flying effect" period. However, even in this case, if the viewer avatar viewpoint video is generated and displayed from the end of the "flying effect" period, there is a risk that a malfunction will occur in which the viewer avatar viewpoint video will not be displayed on viewer terminal 300 during the period until the viewer avatar viewpoint video is generated and displayed, just as at the start of the "flying effect".

[0229] Therefore, even when the period of these "flying effects" ends, the distribution server computer 100 of Example 2 sends an end TM (timing) advance notification to the viewer terminal 300 as shown in Figure 28, thereby notifying the viewer terminal 300 in advance of the end timing of the "flying effects" period when the distribution of the virtual camera viewpoint image (viewpoint image from a virtual camera moving in the air) will end.

[0230] In response to receiving the end TM (timing) advance notification, the viewer terminal 300 identifies in advance the timing at which the distribution of the virtual camera viewpoint video (viewpoint video from a virtual camera moving in the air) from the distribution server computer 100 will be stopped, and starts the viewer viewpoint video generation process at a predetermined timing before that timing to pre-generate the viewer avatar viewpoint video.

[0231] Furthermore, the distribution server computer 100 starts distributing the virtual space update data from the time of sending the end TM (timing) advance notification, and the viewer terminal 300 executes virtual space processing to update the virtual space data using the received virtual space update data, so that at the specified timing mentioned above, a viewer avatar viewpoint image that reflects the situation of the virtual live venue at that time can be generated.

[0232] In this way, the distribution server computer 100 of Example 2 sends an end TM (timing) advance notification to the viewer terminal 300 before the end of the flying performance period, allowing the viewer terminal 300 to generate the viewer avatar viewpoint video in advance before the end of the flying performance period, thereby preventing the viewer terminal 300 from failing to display the viewer avatar viewpoint video.

[0233] In Figure 28, the "flying performance" is used as an example, but similar control is also performed during the "pre-show speech" and "opening performance" periods, when the viewpoint image is forcibly changed from the viewer avatar viewpoint image to the virtual camera viewpoint image.

[0234] As described above, in the virtual space content distribution system of Example 2, there are multiple viewer users, and each viewer user is assigned a viewer terminal 300. When the viewer terminal 300 displays the viewer avatar viewpoint video, which is a viewpoint video of the virtual live venue (virtual space) corresponding to the viewer avatar, the viewer avatar viewpoint video is generated based on the virtual space update data (virtual space data) transmitted from the distribution server computer 100. Since the generation of these viewer avatar viewpoint videos, which have a large processing load, is performed by the viewer terminal 300, more viewer users can participate in the virtual live performance.

[0235] Furthermore, the virtual space content of Example 2 is content that allows a performer user to participate in a virtual live venue as performer avatar 1, and is content in which performer avatar 1 performs musical actions (performances) such as singing and dancing in the virtual live venue, thereby increasing the interest in the virtual space content.

[0236] Furthermore, since the virtual space update data (virtual space data) of Example 2 includes movement data for the performer avatar for generating the movements of the performer avatar 1, the movements of the performer avatar 1 can be accurately reflected in the viewer avatar viewpoint image generated in the viewer terminal 300.

[0237] Furthermore, the common viewpoint images that are commonly displayed on each viewer terminal 300 in Example 2, such as the images of the "pre-show speech," "opening performance," and "flying performance," are generated in the distribution server computer 100 and distributed to each viewer terminal 300, thereby preventing redundant processing from being executed on each viewer terminal 300, thereby preventing unnecessary increases in the processing load of the entire system.

[0238] Furthermore, in the viewer terminal 300 of Example 2, during the performance period (specific period) other than the "MC period" and "flying performance" periods, during which either the viewer avatar viewer ...

[0239] In addition, during the "costume change performance" and "duplicate ego performance" periods, which are special periods in which the processing load on the distribution server computer 100 in Example 2 is high, each viewer terminal 300 also generates and displays virtual camera viewpoint images (common viewpoint images), thereby preventing the processing load on the distribution server computer 100 from temporarily becoming extremely large.

[0240] Furthermore, the distribution server computer 100 of Example 2 transmits advance data including basic data of the virtual live venue (virtual space) and basic data of the performer avatar 1 and viewer avatars virtually participating in the virtual live venue to each viewer terminal 300 before the virtual live starts, thereby preventing the occurrence of a problem where the viewer avatar's viewpoint image is not displayed properly at the start of the virtual live.

[0241] In addition, the distribution server computer 100 of Example 2 transmits virtual camera viewpoint video (common viewpoint video) to viewer terminals 300 for which it cannot confirm that they have completed receiving the preliminary data. By doing this, it is possible to prevent the processing load on the distribution server computer 100 from increasing excessively due to the transmission of basic data, etc. after the start of the virtual live broadcast, while also preventing viewer users from being unable to watch the virtual live broadcast.

[0242] <Modification of Example 2> FIG. 29 is a flowchart showing the process of generating a viewer's viewpoint video in Modification 2-1 of the second embodiment.

[0243] In the above-mentioned Example 2, a form was shown in which the viewer avatar viewpoint images including the movements of the performer avatar 1 are always generated uniformly, but the present invention is not limited to this, and these viewer avatar viewpoint images may be generated in multiple forms with different processing loads in image generation, depending on the selection of the viewer user on the viewer terminal 300.

[0244] In other words, the viewer terminal 300 used by the viewer user may be a desktop personal computer (PC) with high processing power, as exemplified in Example 1, or a smartphone terminal or tablet terminal with relatively lower processing power than a personal computer (PC). It is expected that devices with different processing powers will be used. However, if viewer avatar perspective video is always generated uniformly on devices with such different processing powers, devices with low processing power will not be able to generate the viewer avatar perspective video properly, which may result in a mismatch between the viewer avatar perspective video and audio, or the viewer avatar perspective video may appear discontinuous and unnatural. Therefore, for example, when generating viewer avatar perspective video that includes actor movements, the viewer may be allowed to select, as appropriate, depending on the processing power of the device they are using, whether to generate viewer avatar perspective video that includes high-definition actor movements, which requires a high processing load for generating the video, or whether to generate viewer avatar perspective video that includes low-definition actor movements, which requires a low processing load for generating the video.

[0245] Specifically, the viewer-perspective video generation process executed in viewer terminal 300 may be the one shown in the flowchart of Fig. 29. In this viewer-perspective video generation process, first, the actor movement generation mode selected by the viewer user on viewer terminal 300 is identified (step S101). Then, it is determined whether the identified actor movement generation mode is a high processing load mode (step S102). If the high processing load mode is selected (Y in step S102), viewer-avatar-perspective video including high-resolution actor movements is generated (step S103). On the other hand, if the high processing load mode is not selected (N in step S102), viewer-avatar-perspective video including low-resolution actor movements is generated (step S104).

[0246] Although Figure 29 illustrates two modes, a high processing load mode and a low processing load mode, the present invention is not limited to this, and these processing load modes may be three or more modes.

[0247] Furthermore, in the above-mentioned Example 2 (which is also the same as Example 1), an example is given in which all actions, including the movement of the viewer user's viewer avatar, are reflected in each viewer avatar by distributing virtual space update data, but the present invention is not limited to this. For example, if the viewer avatar's actions are actions that are normally impossible to perform but become possible if a special fee is paid, such as an operation to perform a special action that allows the viewer avatar to appeal to the performer avatar 1, the special action is directed at the performer user and not at other viewer users, so it is possible to prevent the virtual space update data including these special actions from being distributed to the viewer terminals 300 of other viewer users, thereby preventing the processing load on the distribution server computer 100 from increasing excessively by making these special actions executable.

[0248] In this way, if virtual space update data including special actions is not distributed to the viewer terminals 300 of other viewer users, as shown in variant example 2-2 of Figure 30, when viewer user A performs an operation corresponding to the special action, causing viewer user A's viewer avatar to perform a special action, virtual space update data A corresponding to viewer user A's viewer avatar performing the special action is sent from the distribution server computer 100 to viewer user A's viewer terminal 300, but virtual space update data B in which viewer user A's viewer avatar is not performing the special action is distributed to the viewer terminal 300 of viewer user B, who is different from viewer user A.

[0249] Therefore, different virtual space update data is delivered from the delivery server computer 100 to the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B, which increases the flexibility of the virtual space update data and makes it possible to generate different viewer avatar viewpoint images on the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B.

[0250] Furthermore, viewer terminal 300 of viewer user A can generate and display an image of viewer avatar of viewer user A performing a special action different from the actions that viewer avatar of viewer user B is capable of, based on virtual space update data A that is different from virtual space update data B delivered to viewer terminal 300 of viewer user B, thereby increasing viewer user A's interest in the virtual live performance, which is virtual space content.

[0251] Furthermore, in the above-mentioned Example 2 (similar to Example 1), the viewer avatars of viewer users virtually participating in the fourth area are illustrated as normal avatars in which the avatar appearance differs for each viewer user, but the present invention is not limited to this. For example, in order to reduce the processing load when generating various viewpoint images in a virtual live performance, the viewer avatars of viewer users virtually participating in the fourth area may be simplified avatars in which the avatar appearance is simplified.

[0252] Note that Figure 30 illustrates an example of a case where a special action is performed when a special fee is paid. However, for example, in the case of using a simplified avatar as described above, normal actions other than the movement of the simplified avatar, such as waving, clapping, and changing the direction of the viewer avatar, may be excluded from the virtual space update data distributed to other viewer users, just like special actions, thereby further reducing the processing load on the distribution server computer 100.

[0253] In the first and second embodiments, the video displayed on the viewer terminal 300 is automatically switched between viewer-perspective video and virtual camera-perspective video in accordance with time schedule data preset by the administrator, such as when a flying effect is performed as shown in Fig. 19, but the present invention is not limited to this, and the video displayed on the viewer terminal 300 may be switched between virtual camera-perspective video and viewer-perspective video as appropriate in response to an operation by the viewer user. This allows the viewer user to view the virtual live event from various viewpoints through their own operation, thereby increasing the enjoyment of the virtual live event.

[0254] Furthermore, as described above, even if the image displayed on the viewer terminal 300 can be switched between virtual camera perspective image and viewer perspective image by the viewer user's operation, the viewer user's operation of the viewer terminal 300 (operation to switch the image to be displayed) may be disabled during the period when a performance that is intended to make a strong impression on the viewer user, such as a flying performance (see Figure 19), is being performed, thereby preventing the viewer user from missing the performance. [Example]

[0255] Next, a virtual space content distribution system according to a third embodiment will be described below with reference to Fig. 31 to Fig. 56. Note that the same components as those in the first and second embodiments will be given the same reference numerals and descriptions thereof will be omitted.

[0256] As shown in Fig. 31, the virtual space content distribution system of Example 3 is provided with the same live performance content as Example 2. In particular, in Example 3, a program is provided for the live performance by the avatars of two performers, Performer A and Performer B.

[0257] Immediately after the start of the first MC period, performer avatar 1A of performer A, the first performer, appears, and then a solo performance by performer avatar 1A of performer A is performed as the first act.During the second MC period, performer avatar 1B of performer B, the second performer, appears (the number of performer avatars increases from one to two), and then a joint performance by performer avatar 1A of performer A and performer avatar 1B of performer B is performed as the second act.

[0258] Furthermore, in the third embodiment, it is possible to execute a light-emitting effect in which the virtual aerial objects 50 to 52 emit light as part of the virtual live performance. In particular, a light-emitting effect is not executed during the virtual live performance venue participation period and the introductory speech period (light-emitting intensity of light-emitting effect: none), but the virtual aerial objects 50 to 52 emit light with a relatively weak light-emitting intensity during the opening performance period, the first MC period, and the second MC period (light-emitting intensity of light-emitting effect: weak), and the virtual aerial objects 50 to 52 emit light with a stronger light-emitting intensity during the first performance period and the second performance period than during the opening performance period, the first MC period, and the second MC period (light-emitting intensity of light-emitting effect: strong).

[0259] Also, as shown in Figure 31, the period immediately up until the start of the first MC period is a period in which performer A's performer avatar 1A does not appear, and therefore there is no movement by performer A's performer avatar 1A (performer A's performer avatar movement: none); the first MC period and second MC period after performer A's performer avatar 1A appears are periods in which performer A's performer avatar 1A's movements are relatively small (performer A's performer avatar movement: small); and the first performance period and second performance period are periods in which performer A's performer avatar 1A dances and sings, and therefore the movements of performer A's performer avatar are larger (more intense) than in other periods (performer A's performer avatar movement: large).

[0260] Similarly, the period until performer B's performer avatar 1B appears during the second MC period is a period in which performer B's performer avatar 1B does not move (performer B's performer avatar movement: none), the second MC period after performer B's performer avatar 1B appears is a period in which performer B's performer avatar 1B moves relatively little (performer B's performer avatar movement: small), and the second performance period is a period in which performer B's performer avatar 1B moves larger (more intense) than in other periods because performer B's performer avatar dances and sings together with performer A's performer avatar 1B (performer B's performer avatar movement: large).

[0261] Furthermore, since the period during which participation in the virtual live venue is possible is a period during which multiple viewer terminals 300 sequentially connect to the distribution server computer 100, as time passes, viewer avatars who virtually participate in the virtual live venue through this connection are gradually added and placed within the virtual space.

[0262] 31 and 32(a), immediately after the start of the virtual live venue participation period, no light-emitting effects are being performed, performer avatars 1A and 1B of the performers are not being displayed, and the number of viewer avatars virtually participating in the virtual live venue is small, so there are few viewer avatars that require updates of location information, etc. in distribution server computer 100, resulting in a relatively light update processing load for updating the virtual space. Also, for each viewer terminal 300, as mentioned above, the number of viewer avatars virtually participating in the virtual live venue is small, so the drawing processing load for drawing viewer avatars is relatively low.

[0263] 32(b), the amount of viewer avatar information that is the target of updating the virtual space data in the distribution server computer 100 increases, increasing the processing load for updating the virtual space in the distribution server computer 100. Furthermore, as the number of viewer avatars virtually participating in the virtual live venue increases, the number of viewer avatars that are the target of drawing in each viewer terminal 300 also increases, increasing the processing load for drawing the viewer avatars.

[0264] Furthermore, as the number of viewer avatars virtually participating in the virtual live venue increases, the number of viewers performing movement operations from the viewer terminal 300 also increases, as shown in Figures 31 and 33, and the number of viewer avatars moving within the virtual live venue may also increase.

[0265] Since moving viewer avatars change information such as their current location in the virtual live venue more frequently than viewer avatars that remain stationary in a specific location, an increase in the number of viewer avatars moving within the virtual live venue, as shown in Figures 31 and 33, increases the processing load for updating the virtual space on the distribution server computer 100. At this time, if the number of moving viewer avatars increases in each viewer terminal 300, the amount of drawing required for each viewer avatar to move also increases, and so does the processing load for drawing on the viewer terminal 300.

[0266] 31 and 34(a), when a light-emitting effect is performed during each performance, the delivery server computer 100 must calculate how light hits each viewer avatar and performer avatar and the shadows they generate based on the current position of each avatar and the distance and angle from the light source (virtual aerial objects 50-52 in this Example 3), increasing the update load of updating the virtual space on the delivery server computer 100. At this time, the viewer terminal 300 must also perform drawing to make the virtual aerial objects 50-52 glow in accordance with the light-emitting effect, drawing to change the way the light hits each viewer avatar, and drawing a shadow for each viewer avatar, increasing the drawing processing load.

[0267] In addition, in this embodiment 3, the light intensity of the virtual aerial objects 50 to 52 may be increased as a light-emitting effect (see Figure 31).When the light intensity is increased in this way, the processing load associated with the calculation of how the light hits each viewer avatar and performer avatar and the shadows that are generated in the distribution server computer 100 described above increases further, which results in a further increase in the update load of the virtual space updated by the distribution server computer 100, and similarly, a further increase in the drawing processing load in each viewer terminal 300.

[0268] Furthermore, when performer avatar 1A appears, as shown in FIGS. 31 and 34(b), performer avatar information is added to the information to be updated in the virtual space data in the delivery server computer 100, thereby increasing the update processing load of the virtual space update by the delivery server computer 100. Furthermore, when performer avatar 1B appears, as shown in FIGS. 31 and 34(b), performer avatar information of performer avatar 1B is added to the information to be updated in the virtual space data in the delivery server computer 100 in addition to the performer avatar information of performer avatar 1A, thereby further increasing the update processing load of the virtual space update by the delivery server computer 100. In the viewer terminal 300, the process of drawing performer avatar 1A of performer A starts when performer avatar 1A appears, thereby increasing the drawing processing load. Furthermore, when performer avatar 1B appears, the process of drawing performer avatar 1B of performer B starts in addition to the process of drawing performer avatar 1A, thereby further increasing the drawing processing load. In this embodiment 3, after performer B's performer avatar 1B appears (after the number of performer avatars becomes two), the number of performer avatars does not decrease, so the process of drawing performer avatar 1A and the process of drawing performer avatar 1B continue until the end of the virtual live performance.

[0269] During each performance, as shown in Figures 31 and 35, in addition to increasing the light emission intensity of virtual aerial objects 50-52 as a light-emitting effect, each performer avatar 1A, 1B performs a performance such as singing or dancing. Therefore, as described above, the processing load associated with the calculation processing of the lighting conditions and generated shadows for each viewer avatar and performer avatar in distribution server computer 100 further increases, and the update processing load for updating the virtual space by distribution server computer 100 further increases. Also, at each viewer terminal 300, the drawing processing load increases for drawing the lighting conditions and shadows for each viewer avatar that are generated by illuminating virtual aerial objects 50-52, and for drawing performer avatars 1A, 1B.

[0270] In this embodiment 3, when the processing load on the distribution server computer 100 or the viewer terminal 300 increases due to an increase in the number of viewer avatars and performer avatars placed in the virtual live venue, an increase in the light intensity of the light-emitting effects, an increase in the number of moving viewer avatars and performer avatars, an increase in the amount of movement of the performer avatars, etc., as described above, the display mode of the viewer avatar is changed from a full avatar, which requires a relatively large processing load to display, to a simplified avatar, which requires a relatively small processing load to display, thereby making it possible to prevent an excessive increase in the processing load on the entire equipment that makes up the virtual space content distribution system, such as the distribution server computer 100 and the viewer terminal 300.

[0271] <Time schedule data> The virtual live performance of Example 3, which proceeds as described above, is managed by the delivery server computer 100 based on time schedule data (TS data), as shown in Fig. 36. The time schedule data is stored in the storage 103 of the delivery server computer 100 in a form included in the event management program 110.

[0272] The time schedule data is data in a table format that stores the scheduled time of each timing, the elapsed time from the start of the live performance, the content of the timing, and the file name of the associated data related to the timing, in association with a timing ID assigned to each timing in the virtual live performance.

[0273] 36, these timings include the start and end timings of the opening performance and the performer entrance performance, the timings when performer avatars 1A and 1B appear during the performer entrance performance, as well as the start and end timings of each transition period and the timings when each viewer avatar is simplified (first viewer avatar simplified timing T1, second viewer avatar simplified timing T2, and third viewer avatar simplified timing T3). Hereinafter, in this embodiment, these first viewer avatar simplified timing T1, second viewer avatar simplified timing T2, and third viewer avatar simplified timing T3 may be simply referred to as timings T1, T2, and T3.

[0274] Timing T1 is the timing immediately before the start of the first performance period, when the light intensity of the light-emitting effects increases and the movements of performer avatar 1A become more intense, thereby increasing the rendering processing load on viewer terminal 300. Timing T2 is the timing immediately before the start of the second MC period, when the rendering processing load on viewer terminal 300 increases due to the addition of performer avatar 1B. Timing T3 is the timing at the start of the second performance period, when the light intensity of the light-emitting effects increases and the movements of performer avatars 1A and 1B become more intense, thereby increasing the rendering processing load on viewer terminal 300. These timings T1 to T3 are set in advance to change the viewer avatar from a full avatar to a simplified avatar, just before the rendering processing load on viewer terminal 300 increases, as described above, at a stage before the processing load has increased.

[0275] <Video generation during the period when live virtual space participation is possible> Furthermore, during the live virtual space participation period in which only the viewer avatar viewpoint video can be displayed on the viewer terminal 300 in this embodiment 3, as shown in Figure 37, as in embodiment 2, after the viewer terminal 300 executes the avatar operation processing, and the distribution server computer 100 executes the virtual space update processing and virtual space update data generation processing, the distribution server computer 100 further executes the avatar simplification determination processing to determine whether the viewer avatar should be changed from a full avatar to a simplified avatar.

[0276] Still, after executing the avatar simplification determination process, the distribution server computer 100 executes the virtual space update data distribution process in the same manner as in the second embodiment, and transmits the simplification determination value data, which is the result of the avatar simplification determination process, together with the virtual space update data to the viewer terminal 300.

[0277] The viewer terminal 300 executes a virtual space update process, a viewer viewpoint video generation process, and a video output process based on receiving the virtual space update data and the simplification determination value data. In particular, in the third embodiment, although details will be described later, by using the received simplification determination value data, a process of drawing (generating) the viewer avatar as a full avatar or a simple avatar is performed in the viewer viewpoint video generation process.

[0278] <Video Generation during the Performance Period Excluding the MC Period and the Flying Effect Period> Also, during the performance period excluding the MC period and the flying effect period in which the viewer avatar viewpoint video and the virtual camera viewpoint video can be displayed on the viewer terminal 300 in the third embodiment, as shown in FIG. 38, similar to the second embodiment, after executing the avatar operation operation process on the viewer terminal 300 and the virtual space update process and the virtual space update data generation process on the distribution server computer 100, the distribution server computer 100 further determines whether to change the display of the viewer avatar from a full avatar to a simple avatar. An avatar simplification determination process is executed. Note that this avatar simplification determination process is the same process as the avatar simplification determination process shown in FIG. 38.

[0279] After executing the avatar simplification determination process, the distribution server computer 100 executes the virtual space update data distribution process in the same manner as in the second embodiment, and transmits the simplification determination value data, which is the result of the avatar simplification determination process, together with the virtual space update data to the viewer terminal 300.

[0280] Based on the received virtual space update data and simplification determination value data, the viewer terminal 300 executes a virtual space update process, a viewer viewpoint video generation process if a viewer avatar viewpoint video is selected as the video to be displayed, and a virtual camera designated video generation process if a virtual camera viewpoint video is selected as the video to be displayed. In particular, in the third embodiment, the received simplification determination value data is used to perform a process of drawing (generating) a viewer avatar as a full avatar or a simplified avatar in the viewer viewpoint video generation process and the virtual camera designated video generation process, as will be described in detail later.

[0281] <About friend registration> Furthermore, in the virtual space content distribution system of the present embodiment 3, a viewer can register other viewers as his / her friends by operating the viewer terminal 300. Note that the storage 303 of the viewer terminal 300 of the present embodiment 3 stores a friend user data table (see FIG. 40) that stores the user IDs and nicknames of viewer users who have been registered as friends in advance.

[0282] As shown in Figure 39, the viewer terminal 300 executes a viewer avatar selection process to select a specific viewer avatar from multiple viewer avatars by accepting operation of the viewer's operation input device 321 during the period during which participation in the live virtual space is permitted.Then, the viewer terminal 300 identifies the file name of the selected viewer avatar (viewer avatar file name) and executes a viewer avatar inquiry process to send viewer avatar inquiry information including the identified viewer avatar file name to the distribution server computer 100.

[0283] When the delivery server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID stored in association with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in Fig. 9. Then, the delivery server computer 100 transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

[0284] When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process on the display device 310 to prompt the viewer to register the viewer (viewer user ID) of the selected viewer avatar as a friend by associating it with the viewer avatar file name.

[0285] At this time, if the viewer performs an operation to register a friend, a friend addition process is executed to store the viewer user ID and the viewer avatar file name in association with each other in the friend user data table, and if the operation to register a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process to display on the display device 310 an indication that the friend registration will not be performed.

[0286] <Avatar simplification determination process> Fig. 41 is a flowchart showing an example of the avatar simplification determination process shown in Fig. 38. In the avatar simplification determination process, the distribution server computer 100 first determines whether or not it is currently within the virtual live venue participation period (see Fig. 31) by, for example, referring to time schedule data (step S401). If it is not currently within the virtual live venue participation period (step S401; No), the process proceeds to step S407. If it is currently within the virtual live venue participation period (step S401; Yes), the process further determines whether or not the participant number threshold reached flag has been set (step S402). The participant number threshold reached flag indicates that the number of viewer avatars placed in the virtual live venue (the number of viewers watching the virtual live performance) has already reached a predetermined threshold.

[0287] If the participant number threshold reached flag is not set (step S402; Yes), the distribution server computer 100 proceeds to step S407; if the participant number threshold reached flag is not set (step S402; No), the distribution server computer 100 further refers to the participant user table shown in FIG. 23 to identify the number of participants in the live performance (the number of viewer avatars virtually participating in the virtual live performance venue) (step S403), and determines whether the identified number of participants in the live performance has reached the threshold (step S404).

[0288] The threshold value for the number of participants is a value determined based on the number of full viewer avatars that can be rendered at one time on each viewer terminal 300 and the number of full viewer avatars that can be placed in the virtual live venue. For example, if the limit for the number of full viewer avatars that can be rendered at one time on each viewer terminal 300 or the limit for the number of full viewer avatars that can provisionally participate in the virtual live venue is 200, the administrator of the distribution server computer 100 can set the threshold value for the number of participants to 180, which is about 10% lower than the limit. The threshold value for the number of participants is not simply the number of viewer avatars to be rendered, but is a value determined solely by the processing load corresponding to the number of viewer avatars that can be rendered on the viewer terminal 300.

[0289] The threshold value for the number of participants may be set to a value different from the above by the administrator of the delivery server computer 100 on the administrator terminal 150, taking into consideration the performance (drawing processing capability) of each viewer terminal 300, etc.

[0290] If the number of participants has not reached the threshold value (step S404; No), the process proceeds to step S407, but if the number of participants for the live event has reached the threshold value (step S404; Yes), the distribution server computer 100 adds 1 to the simplification determination value based on the fact that the rendering processing load of the viewer avatars in full avatar form on the viewer terminal 300 is approaching its limit (step S405), and sets a number-of-participants threshold reached flag (step S406), and then proceeds to step S407. The simplification determination value, which will be described in detail later, is a value for specifying the range in which each viewer avatar is displayed on each viewer terminal 300 as a simplified avatar with a small number of polygons, and is set to "0" as its initial value.

[0291] In step S407, the distribution server computer 100 determines whether it is within the virtual live venue participation period or the MC period by, for example, referring to the time schedule data (step S407). If it is neither within the virtual live venue participation period nor the MC period (step S407; No), the process proceeds to step S413. If it is within the virtual live venue participation period or the MC period (step S407; Yes), the distribution server computer 100 determines whether the moving number threshold reached flag has been set (step S408). The moving number threshold reached flag indicates that, among the viewer avatars placed in the virtual live venue, the number of viewer avatars moving within the virtual live venue as full avatars has already reached a predetermined threshold.

[0292] If the moving number threshold reached flag is set (step S408; Yes), proceed to step S413; if the moving number threshold reached flag is not set (step S408; No), the distribution server computer 100 determines the number of viewer avatars currently moving (the number of viewer terminals 300 being moved by each viewer) from the viewer avatar information received from each viewer terminal 300 (step S409).

[0293] In step S409, the distribution server computer 100 stores the location information of each viewer avatar within the virtual live venue for each unit time (e.g., 10 ms), compares the stored location information of each viewer avatar within the virtual live venue with the location information of each viewer avatar within the virtual live venue previously stored, and tallys up the number of viewer avatars whose stored location information differs from that previously stored, thereby identifying the number of viewer avatars currently moving.

[0294] The delivery server computer 100 then determines whether the identified number of moving viewer avatars (the number of viewer avatars currently moving) has reached a threshold value (step S410). The threshold value for the number of moving viewer avatars in step S410 is a value determined from the number of moving viewer avatars in full avatar form that can be rendered at one time on each viewer terminal 300. For example, if the limit for the number of moving viewer avatars in full avatar form that can be rendered at one time on each viewer terminal 300 is 200, the administrator of the delivery server computer 100 can set the threshold value for the number of moving viewer avatars to 150, which is 25% lower than the limit. The threshold value for the number of moving viewer avatars is not simply the number of viewer avatars to be rendered, but is a value determined solely by the processing load corresponding to the number of viewer avatars that can be rendered on the viewer terminal 300.

[0295] The threshold value for the number of moving viewer avatars may be set to a value different from the above by the administrator of the delivery server computer 100, taking into consideration the performance (drawing processing capability) of each viewer terminal 300, etc.

[0296] If the number of moving viewer avatars has not reached the threshold value (step S410; No), proceed to S413; if the number of moving viewer avatars has reached the threshold value (step S410; Yes), the distribution server computer 100 adds 1 to the simplification judgment value based on the fact that the processing load for drawing full avatars on the viewer terminal 300 is approaching its limit (step S411), sets the moving number threshold reached flag, and proceeds to step S413 (step S412).

[0297] In step S413, the delivery server computer 100 determines whether or not the current time is timing T1 immediately before the start of the first performance (step S413). If it determines that the current time is timing T1 immediately before the start of the first performance (step S413; Yes), the process proceeds to step S416.

[0298] If it is determined that the current time is not timing T1 immediately before the start of the first performance (step S413; No), the distribution server computer 100 determines whether it is timing T2 when performer avatar 1B of performer B appears (step S414). If it is determined that the current time is timing T2 immediately before performer avatar 1B of performer B appears (step S414; Yes), the process proceeds to step S416.

[0299] Furthermore, if it is determined that the time is not timing T2 immediately before performer avatar 1B of performer B appears (step S414; No), delivery server computer 100 determines whether the time is timing T3 immediately before the start of the second performance (step S415). If it is determined that the time is timing T3 immediately before the start of the second performance (step S415; Yes), it proceeds to step S416, and if it is determined that the time is not timing T3 immediately before the start of the second performance (step S415; No), it proceeds to step S417.

[0300] In step S416, the distribution server computer 100 adds 1 to the simplification judgment value based on the fact that the drawing processing load of the viewer terminal 300 will increase immediately after the above-mentioned times T1, T2, and T3 are reached, and proceeds to step S417 (step S416).

[0301] Furthermore, in step S417, the delivery server computer 100 executes a combined simplification determination process. This combined simplification determination process determines whether the total processing load of the viewer terminal 300, including the rendering processing load of the viewer avatar as a full avatar, the calculation of the shadow generation of each viewer avatar, and the rendering processing load of the performer avatar, is approaching its limit, even if, for example, the threshold value for the number of participants has not been reached in step S404, the threshold value for the number of moving viewer avatars has not been reached in step S410, or it is not timing T1 to T3 in steps S413 to S415. If this total processing load is approaching its limit, the delivery server computer 100 increments the simplification determination value by 1, as in steps S405, S411, and S416.

[0302] In this third embodiment, the simplification determination value is incremented by 1 at timing T2 because the number of performer avatars will be two immediately after (performer avatar 1B appears in addition to performer avatar 1A). In other words, the threshold value for the number of performer avatars is set to "2," but this threshold value is not simply the number of performer avatars to be drawn, but is a value determined by the processing load corresponding to the number of performer avatars that can be drawn on the viewer terminal 300.

[0303] The threshold value for the number of performer avatars may be set differently from the above by the administrator of the distribution server computer 100, taking into consideration the performance (drawing processing capability) of each viewer terminal 300. In particular, in this Example 3, if the threshold value for the number of performer avatars is set to "1", the simplification judgment value will be incremented by 1 immediately before performer avatar 1A of performer A appears during the first MC period.

[0304] In the composite simplification judgment process of this embodiment 3, an example is given of a form in which a judgment is made as to whether the processing load obtained by adding up the processing load for drawing the viewer avatar as a full avatar in the viewer terminal 300, the calculation for generating the shadow of each viewer avatar, the processing load for drawing the performer avatar, etc. is approaching the limit of the processing load of the viewer terminal 300. However, the present invention is not limited to this, and the composite simplification judgment process may add 1 to the simplification judgment value on the condition that the processing load of the viewer terminal 300 has reached a processing load different from the processing load corresponding to the threshold value for the number of participants or the threshold value for the number of moving viewer avatars.

[0305] After executing the process of step S417, the delivery server computer 100 performs only the setting for sending the simplification determination value, and ends the avatar simplification determination process (step S418).

[0306] <Viewer's viewpoint video generation processing> FIG. 42 is a flowchart showing an example of the viewer viewpoint video generation process shown in FIG. 38. In the viewer viewpoint video generation process, the viewer terminal 300 first generates a video of a performer avatar as a full avatar as shown in FIG. 45(a) (step S501). As shown in FIG. 45(a), performer avatars 1A and 1B include full avatars created based on the preferences of each performer (performers A and B in this Example 3), and simplified avatars that are simplified versions of the full avatars. A simplified avatar is an avatar model created with fewer polygons than a full avatar (number of polygons: performer's full avatar > performer's simplified avatar), and therefore imposes a lighter rendering processing load on each viewer terminal 300 than a full avatar.

[0307] Next, it is determined whether the simplification determination value is 0 (step S502). If the simplification determination value is 0 (step S502; Yes), full avatars are generated for both the viewer avatars of viewers who are registered as friends and the viewer avatars of viewers who are not registered as friends, and the process proceeds to step S509 (step S503). As shown in FIG. 45(b), full avatars are provided as viewer avatars, created based on the preferences of each viewer. These full avatars as viewer avatars are avatar models created with approximately the same number of polygons as the performer's simplified avatar (number of polygons: performer's simplified avatar ≒ viewer's full avatar), and the shape, color scheme, etc. differ for each viewer.

[0308] In this third embodiment, an example is given in which the performer's full avatar and the viewer's full avatar are created with approximately the same number of polygons, but the present invention is not limited to this. As long as the number of polygons in the performer's full avatar and the viewer's full avatar is less than the number of polygons in the performer's full avatar, the number of polygons in the viewer's full avatar may be more than the number of polygons in the performer's simplified avatar, or the number of polygons in the performer's simplified avatar may be more than the number of polygons in the viewer's full avatar.

[0309] 45(b), simplified avatars created with fewer polygons than each viewer's full avatar are also provided as viewer avatars. These simplified avatars have a common form regardless of the form of each viewer's full avatar, and are created with fewer polygons than each viewer's full avatar (number of polygons: viewer's full avatar > viewer's simplified avatar), which reduces the rendering processing load on each viewer terminal 300 compared to full avatars.

[0310] Returning to FIG. 42, if the simplification determination value is not 0 (step S502; No), the viewer terminal 300 further determines whether the simplification determination value is 1 (step S504). If the simplification determination value is 1 (step S504; Yes), the viewer terminal 300 determines whether the friend viewer avatar identified flag is set (step S505a). The friend viewer avatar identified flag is a flag indicating that a viewer avatar file name existing in the friend user data table has been identified. If the friend viewer avatar identified flag is set (step S505a; Yes), the process proceeds to step S506. If the friend viewer avatar identified flag is not set (step S505a; No), the viewer terminal 300 refers to the friend user data table shown in FIG. 40, identifies a viewer avatar data file name existing in the friend user data table, and stores the identified viewer avatar data file name (step S505b). Furthermore, the viewer terminal 300 sets the friend viewer avatar identified flag and proceeds to step S506 (step S505c).

[0311] Then, in step S506, the viewer terminal 300 generates a full avatar image of the viewer avatar (friend's viewer avatar) corresponding to the viewer avatar data file name identified in step S505 (step S506), while generating simplified avatar images of the viewer avatars (other people's viewer avatars) other than the viewer avatar corresponding to the viewer avatar data file name identified in step S505b (step S507), and proceeds to step S509.

[0312] Also, if the simplification determination value is not 1 (step S504; No), the viewer terminal 300 generates images of all viewer avatars as simplified avatars, regardless of whether the viewer is registered as a friend or not, since the simplification determination value is 2 or more (step S508), and proceeds to step S509.

[0313] In step S509, the viewer terminal 300 generates viewer viewpoint videos of other viewer avatars other than the performer avatars 1A and 1B and the viewer avatar (step S509), and ends the viewer viewpoint video generation process.

[0314] As described above, in this Example 3, as shown in Fig. 44(a), the performer avatars 1A and 1B generated in the viewer viewpoint video generation process are always generated as full avatars regardless of the value of the simplification determination process. In other words, the performer avatars 1A and 1B are always displayed as full avatars from the viewer avatar viewpoint of the viewer operating the viewer terminal 300.

[0315] Furthermore, among the viewer avatars, for viewer avatars of viewers who are registered as friends, if the simplification determination value is 1 or less, a video is generated using a full avatar, whereas if the simplification determination value is 2 or more, a video is generated using a simplified avatar. In other words, from the viewpoint of the viewer avatar of the viewer operating the viewer terminal 300, a viewer avatar who is registered as a friend is displayed as a full avatar if the simplification determination value is 1 or less, whereas if the simplification determination value is 2 or more, a video is generated using a simplified avatar.

[0316] Furthermore, among the viewer avatars, for viewer avatars of viewers who are not registered as friends, if the simplification determination value is 0, a video is generated using a full avatar, whereas if the simplification determination value is 1 or greater, a video is generated using a simplified avatar. In other words, from the viewpoint of the viewer avatar of the viewer operating the viewer terminal 300, a viewer avatar who is not registered as a friend is displayed using a full avatar only if the simplification determination value is 0, whereas if the simplification determination value is 1 or greater, a video is generated using a simplified avatar.

[0317] <Virtual camera viewpoint video generation process (viewer terminal)> 43 is an example of a flowchart showing the virtual camera viewpoint video generation process (virtual camera viewpoint video generation process executed by viewer terminal 300) shown in FIG. 38. In the virtual camera viewpoint video generation process, viewer terminal 300 first generates full avatar videos of performer avatars 1A and 1B (step S511), and determines whether the simplification determination value is 0 (step S512). If the simplification determination value is 0 (step S512; Yes), full avatar videos of the viewer avatar of the viewer operating the viewer terminal 300 and all other viewer avatars, regardless of whether they are registered as friends, are generated (step S513), and the process proceeds to step S523.

[0318] If the simplification determination value is not 0 (step S512; No), the viewer terminal 300 further determines whether the simplification determination value is 1 (step S514). If the simplification determination value is 1 (step S514; Yes), the viewer terminal 300 determines whether the friend viewer avatar identified flag is set (step S515a). If the friend viewer avatar identified flag is set (step S515a; Yes), the process proceeds to step S516. If the friend viewer avatar identified flag is not set (step S515a; No), the viewer terminal 300 refers to the friend user data shown in FIG. 40 and identifies the viewer avatar data file name present in the friend user data (step S515b). The viewer terminal 300 also sets the friend viewer avatar identified flag and proceeds to step S516 (step S515c).

[0319] Then, the viewer terminal 300 generates images of the viewer's own viewer avatar operating the viewer terminal 300 and the viewer avatar corresponding to the viewer avatar data file name identified in step S515b (friend's viewer avatar) using full avatars (step S516), while generating images of viewer avatars (other people's viewer avatars) other than the viewer's own viewer avatar operating the viewer terminal 300 and the viewer avatar corresponding to the viewer avatar data file name identified in step S515 using simplified avatars (step S517), and proceeds to step S523.

[0320] If the simplification determination value is not 1 (step S514; No), it is further determined whether the simplification determination value is 2 (step S518). If the simplification determination value is 2 (step S518; No), the viewer terminal 300 generates a full avatar image of the viewer avatar of the viewer operating the viewer terminal 300 (step S520), while generating simplified avatar images of viewer avatars other than the viewer user of the viewer (step S521), and proceeds to step S523.

[0321] If the simplification determination value is not 2 (step S518; No), the viewer terminal 300 determines that the simplification determination value is 3 or greater, and generates images of the viewer avatar of the viewer operating the viewer terminal 300 and all other viewer avatars, regardless of whether they are registered as friends, using simplified avatars (step S522), and then proceeds to step S523. In step S523, the viewer terminal 300 generates other virtual camera viewpoint images and ends the virtual camera viewpoint image generation process (step S523).

[0322] As described above, in this Example 3, as shown in Fig. 44(b), performer avatars 1A and 1B generated by the viewer viewpoint video generation process of the viewer terminal 300 are always generated as full avatars regardless of the value of the simplification determination process. In other words, from the viewer avatar viewpoint of the viewer operating the viewer terminal 300, the performer avatars are always displayed as full avatars.

[0323] Furthermore, among the viewer avatars, for the viewer avatar of the viewer operating the viewer terminal 300, if the simplification judgment value is 2 or less, a full avatar image is generated, whereas if the simplification judgment value is 3 or more, a simplified avatar image is generated. In other words, from the virtual camera viewpoint, the viewer avatar of the viewer operating the viewer terminal 300 is displayed as a full avatar if the simplification judgment value is 2 or less, whereas it is displayed as a simplified avatar if the simplification judgment value is 3 or more.

[0324] Furthermore, among the viewer avatars, for viewer avatars of viewers who are registered as friends, if the simplification determination value is 1 or less, a video is generated using a full avatar, whereas if the simplification determination value is 2 or more, a video is generated using a simplified avatar. In other words, from the virtual camera viewpoint, if the simplification determination value is 1 or less, a viewer avatar of a viewer who is registered as a friend is displayed using a full avatar, whereas if the simplification determination value is 2 or more, a video is generated using a simplified avatar.

[0325] Furthermore, for viewer avatars of viewers who are not registered as friends, if the simplification determination value is 0, a video is generated using a full avatar, whereas if the simplification determination value is 1 or greater, a video is generated using a simplified avatar. In other words, from the virtual camera viewpoint, viewer avatars of viewers who are not registered as friends are displayed using a full avatar only if the simplification determination value is 0, whereas if the simplification determination value is 1 or greater, a video is generated using a simplified avatar.

[0326] In this third embodiment, as shown in Figures 38, 42, and 43, an example has been given in which each viewer avatar in the viewer's viewpoint video and virtual camera's viewpoint video generated in the viewer terminal 300 is changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load on the viewer terminal 300. However, the present invention is not limited to this, and each viewer avatar in the virtual camera's viewpoint video generated in the delivery server computer 100 may also be changed from a full avatar to a simplified avatar, as shown in Figure 25. In this case, each viewer avatar may be changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load on the delivery server computer 100.

[0327] Furthermore, since it is only necessary to prevent an excessive increase in the processing load of the entire virtual space content distribution system of the present invention, the viewer avatar in the viewer perspective image and virtual camera perspective image generated by the viewer terminal 300 may be changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load on the distribution server computer 100, or the viewer avatar in the virtual camera perspective image generated by the distribution server computer 100 may be changed from a full avatar to a simplified avatar in response to an increase in the drawing processing load on the viewer terminal 300.

[0328] In this third embodiment, as described above, an example has been given in which the viewer avatar is changed from a full avatar to a simplified avatar in the viewer terminal 300 in accordance with the processing load corresponding to the number of avatars that can be drawn in the viewer terminal 300. However, the present invention is not limited to this, and the viewer avatar may be changed from a full avatar to a simplified avatar simply based on exceeding a threshold value for the number of participants, a threshold value for the number of moving viewer avatars, or a threshold value for the number of performer avatars.

[0329] Next, changes in the display of performer avatars 1A and 1B and viewer avatars within the virtual live venue will be described using as an example the display from the viewpoint of virtual camera C2 on viewer terminal 300 of the viewer operating viewer avatar 11. In the following description, viewer avatars 12, 13, and 14 will be described as avatars of viewers who have been registered as friends by the viewer operating viewer avatar 11, and the other viewer avatars will be described as avatars of viewers who have not been registered as friends by the viewer operating viewer avatar 11.

[0330] In this Example 3, we will explain the changes in the display of performer avatars 1A, 1B and viewer avatars from the viewpoint of virtual camera C2, but we will not explain the display of performer avatars 1A, 1B and viewer avatars from the viewer viewpoint (the viewpoint from viewer avatar 11) because it is the same as from the viewpoint of virtual camera C2 except that viewer avatar 11 is not displayed.

[0331] In addition, in this embodiment 3, since a combined simplification determination process (step S417) is performed in the avatar simplification determination process shown in Figure 41, the viewer avatar does not necessarily change from a full avatar to a simplified avatar at the timing when the number of participants reaches a threshold value, when the number of moving viewer avatars reaches a threshold value, or at any of times T1 to T3. However, to avoid complicating the explanation, the following will use an example in which the viewer avatar changes from a full avatar to a simplified avatar at the timing when the number of participants reaches a threshold value, when the number of moving viewer avatars reaches a threshold value, or at times T1 to T3.

[0332] First, as shown in Figure 46, in the case where neither the threshold value for the number of participants nor the threshold value for the number of moving viewer avatars is determined to have been reached within the virtual live venue, performer avatar 1A of performer A is displayed as a full avatar from the moment he appears, which is immediately after the start of the first MC period, and performer avatar 2B of performer B is displayed as a full avatar from the moment he appears during the second MC period. Note that both performer avatar 1A of performer A and performer avatar 1B of performer B remain displayed as full avatars without changing to simple avatars.

[0333] On the other hand, from the start of display during the virtual live concert venue participation period until timing T1, immediately before the start of the performance (first performance) by performer avatar 1A of performer A, all viewer avatars are displayed as full avatars (see FIG. 49(a)). From timing T1, immediately after the start of the first performance, the light intensity of the light-emitting effects increases and the movements of performer avatar 1A become more intense, increasing the load on the rendering process on viewer terminal 300. In response to this, the simplification determination value is updated from 0 to 1, and viewer avatars of viewers who are not registered as friends by the viewer operating viewer avatar 11 begin to be displayed as simplified avatars (see FIG. 49(b)).

[0334] Also, immediately after timing T2, performer avatar 1B appears, that is, as the number of performer avatars increases from 1 to 2, the drawing processing load on the viewer terminal 300 increases, and the simplification judgment value is updated from 1 to 2, and the viewer avatar 11 begins to be displayed as a simplified avatar as the viewer avatar of the viewer who has been registered as a friend by the viewer operating the viewer avatar 11 (see Figure 49 (c)).

[0335] Then, from timing T3, the luminous intensity of the lighting effects increases and the second performance begins, in which the movements of performer avatars 1A and 2B of performers A and B become more intense, and the simplification judgment value is updated from 2 to 3, and the viewer avatar 11 itself begins to be displayed as a simplified avatar (see Figure 49 (D)).

[0336] 47, when the determination of whether the participant number threshold has been reached is performed in the virtual live venue, performer avatar 1A of performer A is displayed as a full avatar from the moment he appears, which is immediately after the start of the first MC period, and performer avatar 1B of performer B is displayed as a full avatar from the moment he appears during the second MC period. Note that both performer avatar 1A of performer A and performer avatar 1B of performer B remain displayed as full avatars without changing to simple avatars.

[0337] On the other hand, from the time display begins during the virtual live venue participation period until it is determined that the number of participants has reached the threshold during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see Figure 49(a)). However, if it is determined that the number of participants has reached the threshold during the virtual live venue participation period, the simplified judgment value is updated from 0 to 1 at the time of reaching the threshold, and viewer avatars of viewers who are not registered as friends by the viewer operating viewer avatar 11 begin to be displayed as simplified avatars (see Figure 49(b)).

[0338] From timing T1, the first performance begins, in which the light intensity of the lighting effects increases and the movements of the performer avatar 1A become more intense, causing the drawing processing load on the viewer terminal 300 to increase.In response to this, the simplification judgment value is updated from 1 to 2, and the viewer avatar 11 begins to be displayed as a simplified avatar as the viewer avatar of a viewer who has been registered as a friend by the viewer operating the viewer avatar 11 (see Figure 49(c)).

[0339] Then, immediately after timing T2, performer avatar 1B appears, that is, the number of performer avatars increases from 1 to 2, which increases the drawing processing load on the viewer terminal 300, and the simplification judgment value is updated from 2 to 3, and the viewer avatar 11 itself begins to be displayed as a simplified avatar (see Figure 49 (D)).

[0340] 48, when the threshold value for the number of participants and the threshold value for the number of moving viewer avatars are determined to have been reached within the virtual live venue, performer avatar 1A of performer A is displayed as a full avatar from the moment when he appears, which is immediately after the start of the first MC period, and performer avatar 1B of performer B is displayed as a full avatar from the moment when he appears during the second MC period. Note that both performer avatar 1A of performer A and performer avatar 1B of performer B remain displayed as full avatars without changing to simple avatars.

[0341] On the other hand, from the time display begins during the virtual live venue participation period until it is determined that the number of participants has reached the threshold during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see Figure 49(a)). However, if it is determined that the number of participants has reached the threshold during the virtual live venue participation period, the simplified judgment value is updated from 0 to 1 at the time of reaching the threshold, and viewer avatars of viewers who are not registered as friends by the viewer operating viewer avatar 11 begin to be displayed as simplified avatars (see Figure 49(b)).

[0342] In addition, if it is determined that the number of moving viewer avatars during the period in which participation in the virtual live venue is possible has reached the threshold value, the simplification determination value is updated from 1 to 2 at the time of reaching the threshold, and the viewer avatars of viewers who have been registered as friends by the viewer operating the viewer avatar 11 will begin to be displayed as simplified avatars (see Figure 49(c)).

[0343] Then, from timing T1, the first performance begins, in which the light intensity of the lighting effects increases and the movements of the performer avatar 1A become more intense, causing the drawing processing load on the viewer terminal 300 to increase, and the simplification judgment value is updated from 2 to 3, and the viewer avatar 11 itself begins to be displayed as a simplified avatar (see Figure 49 (D)).

[0344] In this Example 3, even after the simplification judgment value is updated to 3 and all viewer avatars begin to be displayed as simplified avatars, the simplification judgment value may continue to be updated to 4 or 5, but even if the simplification judgment value is updated to 4 or more, all viewer avatars will not change further from the simplified avatars.

[0345] As described above, in this Example 3, as shown in FIGS. 41 to 44 and 46 to 49, the number of displayed performer avatars increases to two, which increases the processing load for drawing the performer avatars; the movement of the performer avatars becomes larger (more intense) with the start of the first performance period or the second performance period, which increases the processing load for updating the virtual space and the processing load for drawing the performer avatars; the light intensity of the light-emitting effects increases with the start of the first performance period or the second performance period, which increases the processing load for drawing the virtual space and each avatar; and the number of viewer avatars virtually participating in the virtual live venue (the number of participants in the live performance) exceeds a threshold number, which increases the processing load for updating the virtual space and the processing load for viewing. By changing the viewer avatars in the virtual live venue from full avatars to simplified avatars based on either the occurrence of an increase in the drawing processing load of the viewer avatars, or the number of moving viewer avatars (number of people moving) among the viewer avatars virtually participating in the virtual live venue exceeding a threshold value, which increases the processing load for updating the virtual space and the processing load for drawing the viewer avatars (specific conditions of the present invention being met), the viewer avatars in the virtual live venue are changed from full avatars to simplified avatars, thereby reducing the processing load on the viewer terminal 300 for displaying (drawing) these viewer avatars, thereby preventing an excessive increase in the processing load on the entire virtual space content distribution system due to the above-mentioned specific conditions being met.

[0346] Furthermore, in this embodiment 3, time schedule data for managing the progress of the virtual live is stored in the storage 103 of the distribution server computer 100, and the distribution server computer 100 is able to identify the timing just before the processing load increases from the time schedule data using the avatar simplification determination process shown in Figure 41, so that the timing when the processing load increases can be accurately identified as timings T1 to T3 before the processing load actually increases.

[0347] In particular, in this embodiment 3, the viewer avatar is changed from a full avatar to a simplified avatar at times T1 to T3 before the timing when the drawing processing load of the performer avatars increases as the number of displayed performer avatars increases to two, the timing when the update processing load of the virtual space and the drawing processing load of the performer avatars increase as the movements of the performer avatars become larger (more intense) with the start of the first performance period or the second performance period, and the timing when the drawing processing load of the virtual space and each avatar increases as the light intensity of the light-emitting effects increases based on the start of the first performance period or the second performance period.This prevents the processing load required for these processes from overlapping and causing an excessive increase in the processing load of the entire virtual space content distribution system.

[0348] In this embodiment 3, an example is given in which time schedule data is stored in storage 103 of the distribution server computer 100, and the distribution server computer 100 changes the viewer avatar on the viewer terminal 300 from a full avatar to a simplified avatar based on the timings T1 to T3 identified by the distribution server computer 100. However, the present invention is not limited to this, and the time schedule data may be stored in storage 303 of the viewer terminal 300, and the viewer terminal 300 itself may identify the timings T1 to T3 from the time schedule data and change the viewer avatar from a full avatar to a simplified avatar based on the identification.

[0349] Furthermore, in this embodiment 3, an example is given in which the viewer avatar is changed from a full avatar to a simplified avatar from timings T1 to T3 just before the drawing processing load increases, but the present invention is not limited to this, and if the viewer terminal 300 has sufficient drawing processing capacity, the viewer avatar may be changed from a full avatar to a simplified avatar at the timing when the drawing processing load increases.

[0350] Furthermore, as shown in Figure 49, the viewer avatar of the viewer operating the viewer terminal 300 (viewer avatar 11 in the example of Figure 49) is eventually changed from a full avatar to a simplified avatar and displayed as a virtual camera-specified image, thereby further preventing excessive increases in processing load.

[0351] Furthermore, in this embodiment 3, the viewer avatar of a viewer operating another viewer terminal 300 takes priority over the viewer's own viewer avatar (viewer avatar 11 in the example of Figure 49) operating the viewer terminal 300, and changes from a full avatar to a simplified avatar, thereby effectively preventing excessive increases in processing load.

[0352] In this Example 3, the viewer avatars are divided into the viewer avatar of the viewer who operates the viewer terminal 300 (viewer avatar 11 in the example of Figure 49), viewer avatars of viewers who have been registered as friends by the viewer (viewer avatars 12, 13, and 14 in the example of Figure 49), and viewer avatars of viewers who have not been registered as friends by the viewer, and the priority for changing from a full avatar to a simplified avatar is set highest for the viewer avatar of a viewer who has not been registered as a friend, and lowest for the viewer avatar of the viewer who operates the viewer terminal 300 (viewer avatar 11 in the example of Figure 49) (priority for changing from a full avatar to a simplified avatar: viewer avatar of a viewer who has not been registered as a friend > viewer avatar of a viewer who has been registered as a friend > viewer avatar of the viewer who operates the viewer terminal 300), but the present invention is not limited to this, and the priority for changing from a full avatar to a simplified avatar may be other than the above, and the priority for changing from a full avatar to a simplified avatar may not be set among viewer avatars.

[0353] 45 to 48, full avatars and simplified avatars are provided for both the performer avatars 1A, 1B and the viewer avatars, but while the viewer avatars change from full avatars to simplified avatars as the drawing processing load on the viewer terminal 300 increases, only full avatars are displayed as performer avatars 1A, 1B regardless of the increase in drawing processing load on the viewer terminal 300. In other words, during the virtual live performance, simplified avatars, which require a small drawing processing load to display as performer avatars 1A, 1B, are never displayed, preventing a decrease in interest in the live performance due to the display of simplified avatars as performer avatars 1A, 1B.

[0354] In this embodiment 3, an example is given in which a full avatar and a simple avatar are provided as performer avatars 1A and 1B, but the present invention is not limited to this. In the first place, since there is no opportunity for simple avatars to be displayed as performer avatars 1A and 1B, it is also possible to provide only full avatars as performer avatars 1A and 1B and not provide simple avatars.

[0355] Furthermore, in this third embodiment, a performer terminal 200 operable by a performer is provided as a device constituting the virtual space content distribution system, and the movements of performer avatars 1A and 1B that immediately reflect the performer's operations can be distributed as virtual space content. However, the present invention is not limited to this, and the movements of performer avatars 1A and 1B that have been filmed or created in advance may also be distributed as virtual space content.

[0356] Furthermore, in this embodiment 3, as shown in FIG. 41, based on the number of viewer avatars installed in the virtual live venue or the number of viewer avatars moving around reaching a predetermined threshold, the distribution server computer 100 univocally determines that the drawing processing load on the viewer terminal 300 is approaching its limit, and causes the viewer terminal 300 to change the viewer avatar from a full avatar to a simplified avatar. However, the present invention is not limited to this, and in reality, it is extremely unlikely that each viewer will be using the same viewer terminal 300. Therefore, even if the distribution server computer 100 determines that the drawing processing load on the viewer terminal 300 is approaching its limit, if the viewer terminal 300 still has sufficient drawing processing capacity remaining to draw viewer avatars, performer avatars, etc., the viewer terminal 300 may continue to display the viewer avatar as a full avatar.

[0357] In addition, in this embodiment 3, an example was given in which the distribution server computer 100 determines that the drawing load on the viewer terminal 300 will increase and changes the viewer avatar from a full avatar to a simplified avatar on the viewer terminal 300 when the number of viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the number of moving viewer avatars among the viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the light intensity of the light-emitting effects increases, when the movements of the performer avatars become larger (more intense), or when the number of performer avatars increases.However, the present invention is not limited to this, and the viewer terminal 300 itself may determine that the drawing processing load on the viewer terminal 300 will increase and change the viewer avatar from a full avatar to a simplified avatar when the number of viewer avatars virtually participating in the virtual live venue exceeds a threshold number, when the light intensity of the light-emitting effects increases, when the movements of the performer avatars among the viewer avatars virtually participating in the virtual live venue exceed a threshold number, when the light intensity of the light-emitting effects increases, when the movements of the performer avatars become larger (more intense), or when the number of performer avatars increases. In this case, the viewer watching the virtual live using the viewer terminal 300 may operate the viewer terminal 300 to set a threshold number for the number of viewer avatars virtually participating in the virtual live venue, or a threshold number for the number of viewer avatars that are moving.

[0358] Furthermore, in the third embodiment, an example has been given in which the viewer avatars of viewers who are not friends of the viewer user using the viewer terminal 300 are preferentially displayed as simplified avatars over the viewer avatars of viewers who are friends and have a strong relationship with the viewer user using the viewer terminal 300, but the present invention is not limited to this. Blood relatives such as parents and siblings of the viewer user using the viewer terminal 300, and related parties who belong to the same organization (e.g., school, company, club, etc.) as the viewer user using the viewer terminal 300, may be registered in advance using a procedure similar to that shown in Fig. 39, and the viewer avatars of viewers who are not blood relatives or related parties may be preferentially displayed as simplified avatars. In other words, the concept of "other viewer users who are closely related to each viewer user" in the third embodiment may include not only friends but also blood relatives of the viewer user himself or herself, related parties who belong to the same organization, etc.

[0359] Furthermore, in this embodiment 3, an example was given of changing the viewer avatar from a full avatar to a simplified avatar based on an increase in the processing load of the viewer terminal 300, but the present invention is not limited to this. It is also possible to provide multiple simplified avatars with different drawing loads as viewer avatars, and when the processing load of the viewer terminal 300 increases, to display a simplified avatar according to the drawing capability of each viewer terminal 300 as the viewer avatar.

[0360] <Modification of Example 3> The present invention has been described above based on various embodiments and modifications, but the specific configuration is not limited to the above-mentioned embodiments and modifications, and modifications and additions that do not deviate from the gist of the present invention are also included in the present invention.

[0361] For example, in the above-mentioned Example 3, an example was given in which both full avatars and simple avatars are provided as performer avatars 1A and 1B, but the simple avatars are not displayed as performer avatars 1A and 1B. However, the present invention is not limited to this, and as a variant example 3-1, a case may be provided in which performer avatars 1A and 1B are displayed as simple avatars, or a period may be provided in which each performer can select whether to display performer avatars 1A and 1B as full avatars or simple avatars by operating the performer terminal 200.

[0362] 50, during periods other than the performance period, it is possible to select whether performer A's performer avatar 1A and performer B's performer avatar 1B are displayed as full avatars or simplified avatars by operating each performer terminal 200. Furthermore, during the performance period, if only performer A's performer avatar 1A sings or dances, performer A's performer avatar 1A may be displayed as a full avatar while performer B's performer avatar 1B may be displayed as a simplified avatar, thereby reducing the processing load on viewer terminal 300 for displaying performer B's performer avatar 1B. Also, during the performance period, if only performer B's performer avatar 1B sings or dances, performer B's performer avatar 1B may be displayed as a full avatar while performer A's performer avatar 1A may be displayed as a simplified avatar, thereby reducing the processing load on viewer terminal 300 for displaying performer A's performer avatar 1A.

[0363] Furthermore, if the performance period is a period in which both performer A's performer avatar 1A and performer B's performer avatar 1B are singing or dancing, then both performer A's performer avatar 1A and performer B's performer avatar 1B may be displayed.

[0364] Furthermore, in the third embodiment, a simplified avatar of only one type is provided as the viewer avatar, regardless of whether the performer is singing or dancing. However, the present invention is not limited to this. As shown in FIG. 51 as a modified example 3-2, the simplified avatar of the viewer may have different types depending on the performer avatar performing the performance. In particular, during non-performance periods such as the virtual live venue participation period, opening performance period, and MC period, the viewer avatar may be displayed in the type shown in the third embodiment, while during each performance period, the viewer avatar may be displayed as a simplified avatar with a type that is preset for each performer, depending on the performer avatar performing the singing or dancing. In this way, the type of the simplified avatar as the viewer avatar changes depending on the performer avatar, thereby enhancing the interest of the virtual live performance.

[0365] Furthermore, when the appearance of the viewer avatar is to be varied depending on the performer, as shown in Figures 52 and 53, in the advance data transmission process, the distribution server computer 100 identifies the performers for whom a simple avatar has already been set for the viewer (performers for whom data of a simple avatar for the viewer has already been stored in the storage 103 of the distribution server computer 100) from the advance data download request received from the viewer terminal 300 (step S601), and then sets up the transmission of the data of the simple avatar for the viewer for each identified performer (step S602) and sets up the transmission of other advance data from the received advance data download request (step S603), and then transmits this transmission-set advance data to the viewer terminal 300 (step S604).

[0366] By doing this, the viewer terminal 300 can receive data of a simple viewer avatar corresponding to each performer (performer avatar 1A, 1B) as part of the advance data from the distribution server computer 100, and can display a simple avatar of a form corresponding to each performer avatar 1A, 1B as a viewer avatar during the performance period.

[0367] In addition, in this variant example 3-2, we have exemplified a form in which a simple avatar of a form corresponding to each performer avatar 1A, 1B is displayed as the viewer avatar, but the present invention is not limited to this, and during the performance period of a performer who does not have a simple avatar for the viewer set as pre-data (a performer who does not have data of a simple avatar for the viewer stored in advance in storage 103 in the distribution server computer 100), a simple avatar of the same form as during the non-performance period may be displayed as the viewer avatar, as shown in Figure 51.

[0368] Furthermore, in this modification 3-2, an example was given in which simplified avatars corresponding to the performer avatars 1A and 1B are displayed as viewer avatars, but the present invention is not limited to this, and the simplified avatars displayed as viewer avatars may be changed over time. For example, as shown in Fig. 54 as modification 3-3, a heart-shaped character may be displayed as the viewer's simplified avatar from the start of the performance until a predetermined timing S, while a star-shaped character may be displayed as the viewer's simplified avatar after timing S.

[0369] In addition, in the above-mentioned Example 3, an example was given in which the viewer avatar and the performer avatars 1A and 1B each have two types of avatars: a full avatar and a simplified avatar with a smaller drawing processing load than the full avatar. However, the present invention is not limited to this, and as a variant example 3-4, the viewer avatar and the performer avatars 1A and 1B may each have a plurality of simplified avatars with different drawing processing loads (for example, a first simplified avatar with a smaller drawing processing load than the full avatar and a second simplified avatar with a smaller drawing processing load than the first simplified avatar).

[0370] In this way, when multiple simple avatars with different drawing processing loads are provided, the state of the viewer avatar may be changed in stages from full avatar, to first simple avatar, to second simple avatar, etc. so that the drawing processing load of the viewer terminal 300 decreases gradually, under conditions such as the number of viewer avatars virtually participating in the virtual live venue exceeding a threshold number, the number of moving viewer avatars among the viewer avatars virtually participating in the virtual live venue exceeding a threshold number, or one of times T1 to T3 being reached.

[0371] Furthermore, in the third embodiment, a viewer avatar is changed from a full avatar to a simplified avatar when the number of moving viewer avatars among those virtually participating in the virtual live venue exceeds a threshold number. However, the present invention is not limited to this. As a modified example 3-5, a viewer avatar may be changed from a full avatar to a simplified avatar when the number of viewer avatars virtually participating in the virtual live venue that are dancing, jumping, or the like but not moving exceeds a threshold number. In other words, the "performer avatar moving in virtual space" in the third embodiment may include not only performer avatars moving within the virtual live venue, but also performer avatars that are performing actions other than moving within the virtual live venue.

[0372] In addition, in the above-mentioned Example 3, an example was given in which each viewer avatar is first displayed as a full avatar and then changed to a simplified avatar based on the fulfillment of certain conditions, but the present invention is not limited to this.As a variant example 3-6, if it is certain that the number of viewer avatars virtually participating in the virtual live venue will exceed a predetermined threshold number, each viewer avatar may be displayed as a simplified avatar from the beginning.

[0373] In the third embodiment, the number of displayed performer avatars increases to two, which increases the processing load for drawing the performer avatars; the movement of the performer avatars becomes larger (more intense) with the start of the first performance period or the second performance period, which increases the processing load for updating the virtual space and the processing load for drawing the performer avatars; the light intensity of the light-emitting effects increases based on the start of the first performance period or the second performance period, which increases the processing load for drawing the virtual space and each avatar; the number of viewer avatars virtually participating in the virtual live venue (the number of participants in the live performance) exceeds a threshold number, which increases the processing load for updating the virtual space and the processing load for drawing the viewer avatars; The specific condition in this invention is that the number of moving viewer avatars (number of people moving) among the avatars exceeds a threshold, resulting in an increase in the update processing load of the virtual space and the drawing processing load of the viewer avatar, and an example has been given in which the viewer avatar is changed from a full avatar to a simplified avatar based on the fulfillment of any of these specific conditions, but the present invention is not limited to this, and as variant example 3-7, the specific conditions in this invention may be only some of the above, or may include conditions other than those listed above as long as they increase the processing load on the distribution server computer 100 and viewer terminal 300 that constitute the virtual space content distribution system of the present invention.

[0374] 41 to 43, the third embodiment illustrates a case in which the viewer avatar is changed from a full avatar to a simplified avatar using the same thresholds (threshold for the number of participants and the threshold for the number of moving viewer avatars) when generating a viewer-perspective video and when generating a virtual camera-perspective video. However, the present invention is not limited to this. As a modification example 3-8, the viewer avatar may be changed from a full avatar to a simplified avatar using different thresholds when generating a viewer-perspective video and when generating a virtual camera-perspective video. For example, as shown in FIG. 13, when generating a virtual camera C1 video, unlike when generating a virtual camera C2 video, a virtual camera C3 video, a virtual camera C4 video, and each viewer-perspective video, almost only performer avatars are rendered, so the threshold for the number of participants and the threshold for the number of moving viewer avatars may be set high. On the other hand, in the virtual camera C2 video, the virtual camera C3 video, the virtual camera C4 video, and each viewer-perspective video, each viewer avatar is actively displayed in addition to the performer avatars, so the threshold for the number of participants and the threshold for the number of moving viewer avatars may be set high.

[0375] Furthermore, even in viewer-perspective images, if the viewer avatar's line of sight is not directed toward virtual stage G, it is more difficult to draw a performer avatar as viewer-perspective image than when the viewer avatar's line of sight is directed toward virtual stage G, and therefore the drawing processing load caused by drawing the performer avatar is less likely to be high. Therefore, the threshold for the number of participants and the threshold for the number of moving viewer avatars may be set higher than when the viewer avatar's line of sight is directed toward virtual stage G.

[0376] In addition, in the above-mentioned Example 3, an example was given of a form in which, as the drawing processing load of the viewer terminal 300 increases, the viewer avatars of viewers who are not registered as friends, then the viewer avatars of viewers who are registered as friends, and finally the viewer's own viewer avatar are changed to simplified avatars in that order. However, the present invention is not limited to this, and as a variant example 3-9, as the drawing processing load of the viewer terminal 300 increases, the viewer avatars may be changed to simplified avatars in order from the viewer avatar that is farthest from the viewer's own viewer avatar. [Example]

[0377] Next, a virtual space content distribution system according to a fourth embodiment will be described below with reference to Fig. 55 to Fig. 65. Note that the same components as those in the other embodiments including the first and second embodiments will be denoted by the same reference numerals and description thereof will be omitted.

[0378] As shown in Figures 55 and 56, the virtual space content distribution system of Example 4 uses, as viewer avatars, full avatars that require a relatively large processing load (drawing processing load) for display and have different appearances for each corresponding viewer user (viewer terminal 300) so that each viewer avatar can be identified, and simplified avatars that require a smaller drawing processing load than these full avatars.

[0379] In the virtual live concert held in the virtual space content distribution system of this Example 4, the event will proceed in the order of "live virtual space participation period," "pre-show speech," and "opening performance," as in Examples 1 and 2. After the "opening performance," the event will proceed as follows: "First MC" → "First performance" → "Second MC" → "Second performance" → "Third MC" → "Third performance."

[0380] The "1st MC," "2nd MC," and "3rd MC" are performed by performer avatar 1 appearing on virtual stage G, who greets the audience through a talk and introduces the next performance. The "1st Performance," "2nd Performance," and "3rd Performance" are performances that include singing and dancing by the performer avatar (performer).

[0381] In this virtual live performance, in the fourth embodiment, first, during the period when participation in the live virtual space is possible, each viewer user operates the viewer terminal 300, thereby allowing the viewer avatar corresponding to the viewer terminal 300 to participate in the virtual live performance as a full avatar. Then, from the start of the "first MC" period, each viewer terminal 300 changes the viewer avatar, which had been a full avatar until then, to a simplified avatar, thereby reducing the processing load for displaying the viewer avatar.

[0382] In particular, in this Example 4, as shown in Fig. 61, the live virtual space participation period is a simplification target area designation period in which the viewer user can select an area (simplification target area) in which the viewer avatar will be changed from a full avatar to a simplified avatar from the third and fourth areas of the virtual audience area by operating the viewer terminal 300. Note that if the viewer user does not designate a simplification target area during the live virtual space participation period in this Example 4, both the third and fourth areas become non-simplification target areas, and the viewer avatar will not be displayed as a simplified avatar.

[0383] Below, a series of processes for selecting an area (simplification target area) where a viewer avatar is changed from a full avatar to a simplified avatar from the third and fourth virtual audience areas will be described with reference to FIGS.

[0384] As shown in Figure 57, during the period when participation in the live virtual space is possible, the viewer terminal 300 that has allowed a viewer avatar to participate in the virtual live venue first executes a simplification target area designation operation process to accept an operation from the viewer user regarding which area to change the viewer avatar from a full avatar to a simplified avatar.

[0385] At this time, when the viewer terminal 300 receives an operation from the viewer user, it transmits to the distribution server computer 100 simplification target area information including the participant ID (viewer user ID) of the viewer user.

[0386] When the distribution server computer 100 receives simplification target area information from the viewer terminal 300, it identifies the participant ID (viewer user ID) and which area is to be designated as the simplification target area from the simplification target area information, and performs a participant user table update process to associate the data of the identified simplification target area with the participant ID and store it in the participant user table, as shown in Figure 58.

[0387] Furthermore, based on the execution of the participant user table update process, the delivery server computer 100 executes a participant user table update completion information sending process to send participant user table update completion information to the viewer terminal 300. Then, based on receiving the participant user table update completion information, the viewer terminal 300 executes a participant user table update completion notification process to notify the display device 310 that the update of the participant user table has been completed, that is, that the viewer terminal 300 has completed registration of whether the third area or the fourth area is to be the simplification target area.

[0388] In this way, by the viewer terminal 300 and the distribution server computer 100 executing the series of processes shown in Figure 57, the viewer terminal 300 can set the area to be simplified, and the distribution server computer 100 can determine whether the viewer terminal 300 has set the third or fourth area as the area to be simplified.

[0389] Furthermore, the virtual space content delivery system in the fourth embodiment has a friend registration function that allows a viewer user to register other viewer users as friends by operating the viewer terminal 300.

[0390] As such a friend registration function, as shown in FIG. 59, the viewer terminal 300 executes a viewer avatar selection process to select a specific viewer avatar from among multiple viewer avatars by accepting operation of the viewer's operation input device 321 during the period during which participation in the live virtual space is possible, identifies the file name of the selected viewer avatar (viewer avatar file name), and executes a viewer avatar inquiry process to send viewer avatar inquiry information including the identified viewer avatar file name to the distribution server computer 100.

[0391] When the delivery server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID stored in association with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in Fig. 9. Then, the delivery server computer 100 transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

[0392] When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process on the display device 310 to prompt the viewer to register the viewer user (viewer user ID) of the selected viewer avatar as a friend by associating it with the viewer avatar file name.

[0393] At this time, if the viewer performs an operation to register the viewer as a friend, the viewer terminal 300 executes a friend addition process to store the viewer user ID and the viewer avatar file name in association with each other in the friend user data table. If the operation to register the viewer as a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process to display on the display device 310 a message indicating that the viewer will not be registered as a friend.

[0394] In this fourth embodiment, a friend user data table (see Figure 60) that stores the user ID and avatar data of the viewer user is created in advance in the storage 303 of the viewer terminal 300, and when the friend addition process is executed, the friend user data table is updated to associate the viewer user ID with the viewer avatar file name and store the associated data.

[0395] Fig. 62 is a flowchart showing the simplification target area designation operation processing shown in Fig. 57. In the simplification target area designation operation processing, the viewer terminal 300 first executes the simplification target area designation operation processing to accept, from the viewer user, an operation to designate either the third area or the fourth area of ​​the virtual audience area as the simplification target area (step S801).

[0396] 63(a) and 63(b), while the simplification target area designation operation process is being executed, a window menu 310WA for the viewer user to designate the simplification target area is displayed on the display device 310. The window menu 310WA includes items for "Third Area" and "Fourth Area" that can be designated as the simplification target area, and below these items are check boxes for selecting whether or not to preferentially display friends as full avatars in the simplification target area.

[0397] 62, it is determined whether or not a third area designation operation has been performed to designate the third area as the simplification target area in the simplification target area designation operation processing (step S802). If a third area designation operation has been performed (step S802; Yes), the third area is stored as the simplification target area (step S803).

[0398] If there is no third area designation operation (step S802; No) or after executing the process of step S803, the viewer terminal 300 determines whether there has been a fourth area designation operation that sets the fourth area as the simplification target area in the simplification target area designation operation process (step S804).If there has been a fourth area designation operation (step S804; Yes), the viewer terminal 300 stores the fourth area as the simplification target area (step S805).

[0399] Then, if there is no fourth area designation operation (step S804; No) or after executing the processing of step S805, the viewer terminal 300 transmits simplification target area information corresponding to the area stored as the simplification target area to the distribution server computer 100 (step S806).

[0400] The viewer terminal 300 also determines whether a full avatar priority display operation has been performed for a friend in the simplification target area designation operation processing, that is, whether a check box located at the bottom of the window menu 310WA has been checked by the viewer user, as shown in Fig. 63(b) (step S807). If a full avatar priority display operation has been performed (step S807; Yes), the viewer terminal 300 sets a friend full avatar priority display flag indicating that viewer avatars of friends located in the simplification target area should be displayed as full avatars, and ends the simplification target area designation operation processing (step S808).

[0401] Furthermore, if there has been no full avatar priority display operation (step S807; No), the viewer terminal 300 determines whether or not the friendly avatar priority display flag has already been set (step S809). If the friendly avatar priority display flag has not been set (step S809; No), the viewer terminal 300 terminates the simplification target area designation operation process, and if the friendly avatar priority display flag has been set (step S809; Yes), the viewer terminal 300 further determines whether or not an operation to cancel the friendly avatar priority display has been performed, that is, whether or not the check box located at the bottom of the window menu 310WA has been unchecked by the viewer user's operation (step S810).

[0402] If the operation to cancel the friendly avatar priority display has not been performed (step S810; No), the simplified target area designation operation processing is terminated, and if the operation to cancel the friendly avatar priority display has been performed (step S810; Yes), the friendly avatar priority display flag is cleared and the simplified target area designation operation processing is terminated (step S811).

[0403] 64 is a flowchart showing the viewer-perspective video generation process executed by the viewer terminal 300 in Example 4. In the viewer-perspective video generation process, the viewer terminal 300 first generates full avatar videos of performer avatars (step S501), and determines whether the current time is after the start of the first MC period in the virtual live show (step S502). If the current time is before the start of the first MC period in the virtual live show (step S502; No), the viewer terminal 300 generates full avatar videos of both viewer avatars of viewer users who have been registered as friends and other viewer users who have not been registered as friends, and ends the viewer-perspective video generation process (step S503).

[0404] Furthermore, if the current time is after the start of the first MC period in the virtual live (step S502; Yes), the viewer terminal 300 further determines whether a friend viewer avatar identified flag has been set, indicating that the viewer avatar of the viewer user who has been registered as a friend has already been identified in the viewer perspective video generation process (step S504).

[0405] If the friend viewer avatar identified flag is not set (step S504; No), the friend user data table shown in FIG. 60 is referenced, the friend viewer avatar data file name is identified and stored (step S505), and the friend viewer avatar identified flag is set (step S506).

[0406] Furthermore, if the friend viewer avatar identified flag is set (step S504; Yes) or after executing the processing of step S506, the viewer terminal 300 identifies the simplification target area stored in the simplification target area designation operation processing of Figure 62 (step S507), and generates images of all viewer avatars outside the identified simplification target area using full avatars (step S508).

[0407] Next, the viewer terminal 300 generates simplified avatars for the viewer avatars of other viewers in the simplified area (viewer avatars of viewers who are not registered as friends) (step S509), and generates full avatars for the viewer avatars of friends outside the simplified area (step S510).The viewer terminal 300 also determines whether a friend full avatar priority display flag is set (step S511).

[0408] If the friend full avatar priority display flag is set (step S511; Yes), the viewer avatar of the friend in the simplified area is generated as a full avatar and the processing of step S514 is executed (step S512). If the friend full avatar priority display flag is not set (step S511; No), the viewer avatar of the friend in the simplified area is generated as a simplified avatar and the processing of step S514 is executed (step S513).

[0409] By executing the viewer perspective video generation process as described above, in this Example 4, when comparing the appearance of each viewer avatar generated by the viewer perspective video generation process when the friendly avatar priority display flag is not set, as shown in Figure 65(a), viewer avatars located in either the simplified area or the non-simplified area prior to the start of the first MC period are all displayed as full avatars, regardless of whether they are viewer avatars of viewer users who are registered as friends.

[0410] On the other hand, after the start of the second MC period, all viewer avatars placed in the simplified area will be displayed as simplified avatars regardless of whether they are viewer avatars of viewer users who are registered as friends, while all viewer avatars placed in the non-simplified area will be displayed as full avatars regardless of whether they are viewer avatars of viewer users who are registered as friends.

[0411] Furthermore, as shown in Figure 65(b), when the friendly avatar priority display flag is set and the appearance of each viewer avatar generated by the viewer perspective video generation process is compared, all viewer avatars located in either the simplified area or the non-simplified area prior to the start of the first MC period are displayed as full avatars, regardless of whether they are viewer avatars of viewer users who are registered as friends.

[0412] On the other hand, after the start of the second MC period, among the viewer avatars placed in the simplified area, the viewer avatars of viewer users who are registered as friends will be displayed as full avatars, and the viewer avatars of viewer users who are not registered as friends will be displayed as simplified avatars. Also, all viewer avatars placed in the non-simplified area will be displayed as full avatars, regardless of whether they are viewer avatars of viewer users who are registered as friends.

[0413] As described above, in the virtual space content distribution system in the fourth embodiment, the viewer user can set the viewer avatar to be a simplified avatar by executing the simplification target area designation operation process shown in Fig. 62 and the viewer viewpoint video generation process shown in Fig. 64. This allows the viewer user's intention to be reflected in the viewer avatar displayed as a simplified avatar, and therefore reduces the decrease in interest in the virtual space live broadcast caused by using a simplified avatar that has a smaller processing load for displaying and outputting the viewer avatar in order to reduce the occurrence of display output problems.

[0414] Furthermore, in the virtual space content distribution system of this Example 4, by executing the simplification target area designation operation process shown in Figure 62 and the viewer viewpoint video generation process shown in Figure 64, the viewer user can operate the viewer terminal 300 to designate the simplification target area for the viewer avatar and simplified avatar from the third and fourth areas, thereby further reducing the loss of interest in the virtual live performance due to the viewer avatar in an area not intended by the viewer user becoming a simplified avatar.

[0415] Furthermore, in the virtual space content distribution system in this Example 4, as shown in Figures 59 and 60, other viewer users can be registered as their own friends, and as shown in Figure 65, the viewer avatar of a viewer user who has become a friend can be set to be displayed as a full avatar even when the viewer user is participating in a simplified area. This makes it easier for the viewer avatar of a viewer user who is a friend to be displayed as a full avatar, and when multiple viewer avatars are participating in a virtual live broadcast, it becomes easier to distinguish between the viewer avatar of a viewer user who is a friend and the viewer avatar of a viewer user who is not a friend.

[0416] Furthermore, in this Example 4, the viewer user can operate the viewer terminal 300 to set only one of the third and fourth areas of the virtual audience area as the simplification target area. In addition, by setting both the third and fourth areas as simplification target areas, the processing load related to the display output can be reduced compared to when only one area is set as the simplification target area, so the level of processing load related to the display output can be changed depending on the situation.

[0417] <Modification of Example 4> 62 and 63, the fourth embodiment described above allows the viewer user to select whether to designate the third and fourth areas of the virtual audience area as simplification target areas by executing the simplification target area designation operation process, but the present invention is not limited to this. As a modified example 4-1, the viewer user may select how many of the viewer avatars participating in the virtual live performance are to be simplified by their operation, as shown in Fig. 66. When the number of viewer avatars to be simplified is selected in this way, the selected number of viewer avatars may be set as the upper limit, and the viewer avatars closest to the viewer avatar corresponding to the viewer terminal 300 may be displayed as simplified avatars, or any viewer avatar from all the viewer avatars participating in the virtual live performance may be displayed as a simplified avatar.

[0418] In this way, the viewer user can set the number of viewer avatars as simple avatars, and it is possible to further prevent the viewer user from losing interest in the virtual live performance due to the number of viewer avatars not intended by the viewer user becoming simple avatars.

[0419] 62 and 63, the above-described fourth embodiment is configured such that it is possible to select whether or not to designate the third and fourth areas of the virtual audience area as simplification target areas by executing the simplification target area designation operation process, but the present invention is not limited to this, and as a modified example 4-2, it is also possible to allow the viewer user to select the proportion of viewer avatars to be simplified avatars relative to the number of viewer avatars participating in the virtual live performance by a selection operation, as shown in Fig. 67. Note that when the proportion of viewer avatars to be simplified is selected in this way, the viewer avatars closest to the viewer avatar corresponding to the viewer terminal 300 may be displayed as simplified avatars in order to achieve the selected proportion, or any viewer avatar from all viewer avatars participating in the virtual live performance may be displayed as a simplified avatar.

[0420] In this way, the viewer user can set the ratio of the number of viewer avatars to be simplified avatars to the number of viewer avatars participating in the virtual live performance, and therefore, it is possible to further prevent a decrease in the interest of the virtual live performance caused by a ratio of viewer avatars that is not intended by the viewer user becoming simplified avatars.

[0421] In addition, in the above-mentioned Example 4, as shown in Figure 61, the period during which live virtual space participation is permitted is the period during which a simplified area can be designated, and if the viewer user does not designate a simplified area during the live virtual space participation period, both the third and fourth areas of the virtual audience area become non-simplification areas and the viewer avatars are not displayed as simplified avatars in either case. However, the present invention is not limited to this, and if the viewer user does not designate a simplified area during the live virtual space participation period, the viewer terminal 300 may automatically set both the third and fourth areas of the virtual audience area as simplified area at the end of the live virtual space participation period. In this way, both the third and fourth areas of the virtual audience area are automatically set as simplified area, thereby reducing the processing load related to the display output of the viewer avatar compared to when only one of the third and fourth areas is designated as the simplified area.

[0422] Furthermore, in variant example 4-3, as shown in Figure 70, only one of the third and fourth areas of the virtual audience area can be set as the simplification target area, and by setting both the third and fourth areas as simplification target areas, it is possible to reduce the processing load related to display output compared to when only one area is set as the simplification target area.

[0423] Furthermore, in the above-mentioned Example 4, an example was given of a form in which the area to be simplified is determined by the viewer user's operation regardless of the processing power (specifications) of the viewer terminal 300, but the present invention is not limited to this. As a variant example 4-4, as shown in Figures 69 and 70, the viewer terminal 300 may identify the processing power of the viewer terminal 300 and determine the area to be simplified in accordance with the identified processing power without accepting the viewer user's operation.

[0424] 69, during the period in which participation in the live virtual section is possible, the viewer terminal 300 executes a terminal specification identification process to identify the processing capabilities (specs) of the viewer terminal 300, such as the clock speed and number of cores of the CPU 101 of the viewer terminal 300 and the memory capacity of the RAM 102. The terminal specification identification process also classifies the processing capabilities of the identified viewer terminal 300 into three categories: "medium" which indicates average processing capabilities, "low" which indicates below-average processing capabilities, and "high" which indicates above-average processing capabilities.

[0425] Next, the viewer terminal 300 executes a simplification target area determination process to determine the number of areas to be designated as simplification target areas from the third and fourth virtual audience areas. In the simplification target area determination process, as shown in FIG. 70, if the processing capacity of the identified viewer terminal 300 is "low," the number of simplification target areas is determined to be two (the third and fourth areas). If the processing capacity of the identified viewer terminal 300 is "medium," the number of simplification target areas is determined to be one (either the third or fourth area). Note that if the processing capacity of the identified viewer terminal 300 is "medium," the simplification target areas may be determined randomly from the third and fourth areas, or the simplification target areas may be determined from the third and fourth areas based on whether the viewer avatar corresponding to the viewer terminal 300 is participating in the third or fourth area, or whether the number of viewer avatars participating in the third area or the fourth area is greater.

[0426] After determining the simplification target area in the simplification target area determination process, the viewer terminal 300 transmits simplification target area information including the participant ID (viewer user ID) of the viewer user to the distribution server computer 100.

[0427] When the distribution server computer 100 receives simplification target area information from the viewer terminal 300, it identifies the participant ID (viewer user ID) and which area is to be designated as the simplification target area from the simplification target area information, and performs a participant user table update process to associate the data of the identified simplification target area with the participant ID and store it in the participant user table, as shown in Figure 58.

[0428] Furthermore, based on the execution of the participant user table update process, the delivery server computer 100 executes a participant user table update completion information sending process to send participant user table update completion information to the viewer terminal 300. Then, based on receiving the participant user table update completion information, the viewer terminal 300 executes a participant user table update completion notification process to notify the display device 310 that the update of the participant user table has been completed, that is, that the viewer terminal 300 has completed registration of whether the third area or the fourth area is to be the simplification target area.

[0429] In this way, the viewer terminal 300 itself identifies the processing capacity of the viewer terminal 300 and sets the simplification target area based on the identified processing capacity without any operation by the viewer user, thereby increasing the viewer user's freedom in setting the simplification target area and enabling the viewer user to set an appropriate simplification target area that corresponds to the processing capacity of the viewer terminal 300 they are using without having to perform any operation themselves.

[0430] Furthermore, in the above-described fourth embodiment, as shown in FIG. 65, an example was given in which each viewer avatar generated in the viewer viewpoint video generation process (each viewer avatar generated in the viewer terminal 300) can be an image of a simplified avatar according to the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process shown in FIG. 62. However, the present invention is not limited to this, and as a modified example 4-5, as shown in FIGS. 71(a) to 71(c), each viewer avatar generated in the viewer viewpoint video generation process can be an image of a simplified avatar according to the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process shown in FIG. On the other hand, each viewer avatar generated in the virtual camera viewpoint video generation process (each viewer avatar generated in the distribution server computer 100) may all be an image of a simplified avatar regardless of the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process. By doing this, the processing load related to the display output of the viewer avatar viewpoint video on the viewer terminal 300 can be reduced, and by generating virtual camera viewpoint video with all viewer avatars as simplified avatars on the distribution server computer 100, the processing load related to the display output of the virtual camera viewpoint video can also be reduced.

[0431] Furthermore, the above-described Example 4 exemplifies a form in which a viewer user can set a simplification target area in which viewer avatars participating from the third and fourth areas of the virtual audience area are set as simplified avatars by operating the viewer terminal 300, but the present invention is not limited to this. As shown in Figures 72(a) and 72(b) as Modification 4-6, when the viewer user operates the viewer terminal 300, a message explaining what settings are possible on the viewer terminal 300 may be displayed on the display device 310 as an auxiliary function for setting the simplification target area. In this way, the viewer user can accurately perform operations for setting the viewer avatars as simplified avatars by visually recognizing the message.

[0432] In the example shown in Figure 72, a message is displayed by the viewer user operating the "HELP" area located in the window menu 310WA, but the present invention is not limited to this, and the message may be displayed without the viewer user operating the "HELP" area.

[0433] Furthermore, in variant example 4-6, an auxiliary function for setting the simplification target area is exemplified in which a message is displayed on the display device 310 explaining what settings are possible on the viewer terminal 300, but the present invention is not limited to this, and an auxiliary function for setting the simplification target area may also be to provide operating instructions to the viewer user by displaying an image on the display device 310 or outputting sound from a speaker, etc. [Example]

[0434] Next, a virtual space content distribution system according to a fifth embodiment will be described below with reference to Fig. 73 to Fig. 81. Note that the same components as those in the other embodiments including the first and second embodiments will be denoted by the same reference numerals and description thereof will be omitted.

[0435] As shown in Figure 73, the virtual space content distribution system of Example 5 has, in addition to the distribution server computer 100, multiple sub-venue server computers 400 (sub-venue server computers 400a, 400b, etc.) as server computers that perform virtual space processing, and these sub-venue server computers 400 are connected to the distribution server computer 100, administrator terminal 150, performer terminal 200, and each viewer terminal 300 via the Internet network N.

[0436] These sub-venue server computers 400, like the distribution server computer 100, are capable of executing various processes related to the virtual live venue and the distribution of virtual live performances, and provide a new virtual live venue (hereinafter referred to as a sub-virtual live venue) on the condition that the number of viewer avatars participating in the virtual live venue provided by the distribution server computer 100 reaches a predetermined number, and are used to present performances using performer avatars in the same way as the virtual live venue of the distribution server computer 100 (hereinafter referred to as the main virtual live venue).

[0437] 74, a sub-virtual live venue control program 140 is stored in the storage 103 of the delivery server computer 100 in this fifth embodiment. The delivery server computer 100 uses the sub-virtual live venue control program 140 to determine whether the number of viewer avatars participating in the main virtual live venue or sub-virtual live venue has reached a predetermined constant, and if the predetermined number has been reached, restricts the participation of new viewer avatars in the virtual live venue where the number of viewer avatars has reached the predetermined number, and controls the allocation of new viewer avatars to participate in other sub-virtual live venues.

[0438] In addition, the virtual space content distribution system of this embodiment 5 has a chat information server computer 170, as shown in Figure 73, and this chat information server computer 170 is connected to the distribution server computer 100, administrator terminal 150, performer terminal 200, each viewer terminal 300, and each sub-venue server computer 400 via the Internet network N.

[0439] The chat information server computer 170 is a computer for transmitting integrated chat information, which is chat information input by viewer users operating input devices such as keyboards and touch panels provided on each viewer terminal 300, rearranged in chronological order to the performer terminal 200 and each viewer terminal 300.

[0440] In the fifth embodiment, the performer terminal 200 and each viewer terminal 300 receive integrated chat information from the chat information server computer 170, and display the same chat information on the display screen.

[0441] Next, the flow of a viewer user virtually participating in a virtual live concert venue in this fifth embodiment will be described with reference to FIG.

[0442] First, the delivery server computer 100 executes participant authentication processing when it receives a participation request from the viewer terminal 300, as in Fig. 22. Then, it executes participation virtual live venue determination processing using the sub virtual live venue control program 140. At this time, the delivery server computer 100 determines the live venue in which the viewer avatar will participate.

[0443] Thereafter, as in Figure 22, the distribution server computer 100 updates the participant user table and transmits advance data, the viewer terminal 300 identifies advance data, transmits an advance data download request, and receives and stores the advance data.

[0444] Furthermore, when updating the participant user table after the participating virtual live venue determination process, the distribution server computer 100 stores data indicating the participating virtual live venue determined in the participating virtual live venue determination process, linked to the participant ID corresponding to the viewer terminal 300 that sent the participation request.

[0445] Specific processing of the participating virtual live venue determination processing will be explained using Figure 77(a). Note that in the following explanation, for simplicity, the virtual space content distribution system will be described as having two sub-venue server computers (sub-venue server computers 400a, 400b), but the present invention is not limited to this, and the virtual space content distribution system may have only one sub-venue server computer, or three or more sub-venue server computers.

[0446] In the virtual live performance participation determination process, the distribution server computer 100 first determines whether a second sub-virtual live venue participation permission flag, indicating that the viewer avatar is permitted to participate in the second sub-virtual live venue provided by the sub venue server computer 400b, is set (step S811). If the second sub-virtual live venue participation permission flag is not set (step S811; No), the distribution server computer 100 determines whether a first sub-virtual live venue participation permission flag, indicating that the viewer avatar is permitted to participate in the first sub-virtual live venue provided by the sub venue server computer 400a, is set (step S812). The first and second sub-virtual live venues are virtual live venues that mimic the main virtual live venue and have a virtual audience area and a virtual stage G.

[0447] If the first sub-virtual live venue participation permission flag is not set (step S812; No), the number of viewer avatars participating in the main virtual live venue provided by the distribution server computer 100 is identified by referring to the participant user table shown in Figure 76 (step S813), and it is determined whether the number of viewer avatars participating in the main virtual live venue is greater than or equal to the predetermined participating virtual live venue change provision number (step S814).

[0448] 77(b), in this fifth embodiment, the upper limit number of viewer avatars that can participate in the main virtual live venue is set to 200. On the other hand, the specified number of participating virtual live venue changes in the main virtual live venue is set to 150, which is 75% of the upper limit number of viewer avatars that can participate in the main virtual live venue.

[0449] That is, in step S814, distribution server computer 100 determines whether the number of viewer avatars participating in the main virtual live venue is 150 or more.

[0450] If the number of viewer avatars participating in the main virtual live venue is less than the predetermined participating virtual live venue change specified number (step S814; No), the distribution server computer 100 determines the participating virtual live venue to be the main virtual live venue and ends the participating virtual live venue determination process (step S815).Alternatively, if the number of viewer avatars participating in the main virtual live venue is equal to or greater than the predetermined participating virtual live venue change specified number (step S814; Yes), the distribution server computer 100 sets the first sub-virtual live venue participation permission flag (step S816), determines the participating virtual live venue to be the first sub-virtual live venue, and ends the participating virtual live venue determination process (step S817).

[0451] Also, if the first sub-virtual live venue participation permission flag is set (step S812; Yes), the distribution server computer 100 identifies the number of viewer avatars participating in the first sub-virtual live venue provided by the sub-venue server computer 400a by referring to the participant user table shown in FIG. 76 (step S818), and determines whether the number of viewer avatars participating in the first sub-virtual live venue is greater than or equal to the predetermined participating virtual live venue change provision number (step S819).

[0452] 77(b), in this embodiment 5, the upper limit number of viewer avatars that can participate in the first sub-virtual live venue is set to 100. On the other hand, the specified number of participating virtual live venue changes in the first sub-virtual live venue is set to 75, which is 75% of the upper limit number of viewer avatars that can participate in the first sub-virtual live venue.

[0453] That is, in step S819, distribution server computer 100 determines whether the number of viewer avatars participating in the first sub virtual live venue is 75 or more.

[0454] As shown in FIG. 77(b), the upper limit on the number of viewer avatars that can participate in the second sub-virtual live venue is set to 100, the same as the upper limit on the number of viewer avatars that can participate in the first sub-virtual live venue.

[0455] If the number of viewer avatars participating in the first sub-virtual live venue is less than the predetermined participating virtual venue change specified number (step S819; No), the distribution server computer 100 determines the participating virtual live venue to be the first virtual live venue and ends the participating virtual live venue determination process (step S820).Alternatively, if the number of viewer avatars participating in the first sub-virtual live venue is equal to or greater than the predetermined participating virtual live venue change specified number (step S819; Yes), the distribution server computer 100 sets the second sub-virtual live venue participation permission flag (step S821), determines the second sub-virtual live venue to be the participating virtual live venue, and ends the participating virtual live venue determination process (step S822).

[0456] After the participating virtual live venue determination process, the distribution server computer 100 updates the participant table by storing data indicating the virtual live venue determined in steps S815, S817, S820, and S822 in association with the participant ID corresponding to the viewer terminal 300 that sent the participation request.

[0457] In this fifth embodiment, it is possible for up to a maximum number of viewer avatars to participate in each virtual live venue, but in this case, if the viewer avatars all move at once, if the performer avatars perform vigorously, or if a light-emitting effect is executed in which a specific object is illuminated as part of the virtual live performance, the processing load related to the display of these viewer avatars and performer avatars may become enormous, which may cause problems in the distribution of the virtual live performance. For this reason, in this fifth embodiment, control is performed to allocate new viewer avatars participating in the virtual live performance to another virtual live venue before the number of viewer avatars participating in each virtual live venue reaches the maximum number. This reduces the processing load related to the display of viewer avatars and performer avatars on each server computer in preparation for such cases as the above-mentioned cases where the viewer avatars all move at once, where the performer avatars perform vigorously, or if a light-emitting effect is executed in which a specific object is illuminated as part of the virtual live performance, thereby enabling the virtual live performance to be distributed without any problems.

[0458] Furthermore, in this fifth embodiment, the upper limit number (200) of viewer avatars that can participate in the main virtual live venue provided by distribution server computer 100 is greater than the upper limit number (100) of viewer avatars that can participate in the first sub virtual live venue provided by first sub venue server computer 400a and the second sub virtual live venue provided by second sub venue server computer 400b. In other words, in the virtual space content distribution system of the present invention, when adding new sub venue server computers in accordance with the number of viewer avatars participating in the virtual live, it is sufficient to connect a less expensive server computer with lower processing power for virtual live distribution than distribution server computer 100 each time. This eliminates the need to prepare a server computer with processing power for virtual live distribution equivalent to that of distribution server computer 100, and allows the virtual space content distribution system to be constructed inexpensively.

[0459] In this embodiment 5, the maximum number of viewer avatars that can participate in the main virtual live venue is set to be greater than the maximum number of viewer avatars that can participate in the first sub-virtual live venue and the second sub-virtual live venue, but the present invention is not limited to this, and the maximum number of viewer avatars that can participate in the main virtual live venue may be the same as the maximum number of viewer avatars that can participate in the first sub-virtual live venue and the second sub-virtual live venue.

[0460] Furthermore, in this embodiment 5, as shown in Figure 77(b), the number of participant-available virtual live venue changes for the main virtual live venue and the first sub-virtual live venue are both set to 75% of the maximum number of participant-avatars for each virtual live venue, but the present invention is not limited to this, and the ratio of the number of participant-available virtual live venue changes to the maximum number of participant-avatars for the main virtual live venue and the ratio of the number of participant-available virtual live venue changes to the maximum number of participant-avatars for the first sub-virtual live venue may be different.

[0461] By executing the participating virtual live venue determination process as described above, after the live virtual space participation period has ended, the distribution server computer 100 executes the virtual space update process, virtual space update data generation process, and virtual space update data distribution process, as in Fig. 10, and transmits the virtual space update data to the viewer terminals 300 and performer terminals 200 of the viewer avatars participating in the main virtual live venue. The distribution server computer 100 also executes the music playback process, distribution audio generation process, and generated audio distribution process, as in Fig. 10.

[0462] 78, after the live virtual space participation period has ended, when the distribution server computer 100 receives performer avatar information from the performer terminal 200, it executes a virtual stage area space update process to update only the data for the area of ​​virtual stage G among the virtual space data. This reflects the movements (operations) of the performer avatar on virtual stage G. Then, it generates virtual stage area update data based on the data for the area of ​​virtual stage G before the update and the area data for virtual stage G after the update, and transfers the generated virtual stage area update data to the sub-venue server computers 400a, 400b (virtual stage area space update data generation and transmission process).

[0463] Furthermore, when the distribution server computer 100 receives viewer avatar information from a viewer terminal 300 whose viewer avatar is participating in the first sub-virtual live venue or the second sub-virtual live venue, it refers to the participant user table to identify the sub-virtual live venue in which the viewer avatar of the viewer terminal 300 is participating from the participant ID included in the received viewer avatar information, and executes a viewer avatar information transfer process to transfer the viewer avatar information to the sub-venue server computer 400a, 400b corresponding to the identified sub-virtual live venue.

[0464] In other words, regardless of the virtual live venue in which the viewer avatar is participating, the distribution server computer 100 always receives viewer avatar information from each viewer terminal 300. Then, if the received viewer avatar information is from a viewer terminal 300 whose viewer avatar is participating in the first sub-virtual live venue or the second sub-virtual live venue, the distribution server computer 100 transfers the viewer avatar information to the sub-venue server computer 400a, 400b corresponding to the sub-virtual live venue in which the viewer avatar is participating. Therefore, in the fifth embodiment, the distribution server computer 100 can grasp the total number of viewer avatars (total number of viewer users) participating in the virtual live performance and the actions of each viewer avatar.

[0465] When the sub-venue server computers 400a, 400b receive virtual stage area space update data from the distribution server computer 100, they execute a virtual stage area space reflection process to reflect the contents of the virtual stage area space update data in the movements (operations) of the performer avatars on the virtual stage G of the first sub-virtual live venue and the second sub-virtual live venue.

[0466] In addition, when the sub-venue server computers 400a, 400b receive viewer avatar information from the distribution server computer 100, they perform a viewer area space update process that reflects the behavior of each viewer avatar in the virtual audience area (viewer area space) based on the viewer avatar information.

[0467] After executing the above-mentioned virtual stage area space reflection process and viewer area space update process, the sub-venue server computers 400a, 400b execute a virtual space update data generation process to generate virtual space update data based on the data of the area of ​​the virtual stage G before the update and the data of the area of ​​the virtual stage G after the update, and the data of the virtual audience area (viewer area space) before the update and the data of the virtual audience area (viewer area space) after the update, and then execute a virtual space update data distribution process to distribute the generated virtual space update data to the viewer terminal 300 and the performer terminal 200.

[0468] In the viewer terminal 300 to which the virtual space update data is delivered from the sub-venue server computers 400a, 400b, as in Figure 10, the pre-stored virtual space data is updated based on the delivered virtual space update data, and a performer perspective video generation process and a viewer perspective video generation process are executed using the updated virtual space data, thereby generating performer perspective video and viewer perspective video based on the updated virtual space data, and the generated performer perspective video and viewer perspective video of the first sub-virtual live venue and the second sub-virtual live venue are displayed on the performer terminal 200 and the viewer terminal 300 (viewer perspective video output process).

[0469] In this embodiment, virtual space update data is not distributed from the sub-venue server computers 400a, 400b to the performer terminal 200. Therefore, while the performer terminal 200 displays an image of the main virtual live venue, it does not display an image of the sub-virtual live venue.

[0470] In the example shown in Figure 78, as in Figure 10, the distribution server computer 100 performs music playback processing, distribution audio generation processing, and generated audio distribution processing, and the performer terminal 200 performs playback music output processing, performer audio input processing, monitor audio generation processing, and monitor audio output processing, thereby causing the viewer terminal 300 to perform audio output processing, so these are not shown in the figure.

[0471] Next, the exchange of chat information and integrated chat information between the performer terminal 200, each viewer terminal 300, and the chat information server computer 170 will be described.

[0472] 79, a viewer terminal 300 whose viewer avatar is participating in the main virtual live venue and a viewer terminal 300 whose viewer avatar is participating in the sub virtual live venue execute a chat information input acceptance process to accept input of chat information from viewer users using input devices such as a keyboard or touch panel provided on each viewer terminal 300. At this time, when the input of chat information is accepted, the viewer terminal 300 executes a chat information sending process to send the entered chat information to the chat information server computer 170.

[0473] When the chat information server computer 170 receives chat information from each viewer terminal 300, it executes an integrated chat information generation process to generate integrated chat information by arranging the received chat information in chronological order, and transmits the generated integrated chat information to the performer terminal 200 and each viewer terminal 300.

[0474] Then, as shown in FIG. 80, the performer terminal 200 and each viewer terminal 300 execute an integrated chat information display process to display the same integrated chat information in the chat display area CW provided at a predetermined position on the screen of the display device of each terminal (in this embodiment, the lower right corner of the screen) based on the reception of the integrated chat information.

[0475] In summary, in this Example 5, as shown in Fig. 81, the viewer terminal 300 and the performer terminal 200 whose viewer avatar is participating in the main virtual live venue display integrated chat information and images of the main virtual live venue as performer-perspective video and viewer-perspective video, and output music and performer audio. Also, the viewer terminal 300 whose viewer avatar is participating in the first sub-virtual live venue displays integrated chat information and images of the first sub-virtual live venue as performer-perspective video and viewer-perspective video, and output music and performer audio, and the viewer terminal 300 whose viewer avatar is participating in the second sub-virtual live venue displays integrated chat information and images of the second sub-virtual live venue as performer-perspective video and viewer-perspective video, and output music and performer audio.

[0476] As described above, in this Example 5, when the number of viewer avatars participating in the main virtual live venue reaches 75% of the maximum number of viewer avatars that can participate due to viewer avatars participating in the main virtual live venue, participation of new viewer avatars in the main virtual live venue is restricted, while participation of new viewer avatars in the first sub-virtual live venue is permitted, thereby preventing the virtual live from becoming impossible to stream due to the participation of viewer avatars in the main virtual live venue exceeding the maximum number of viewer avatars that can participate, which is 200.

[0477] Furthermore, common integrated chat information is displayed on both the viewer terminal 300 corresponding to the viewer avatar participating in the main virtual live venue and the viewer terminal 300 corresponding to the viewer avatar participating in the sub virtual live venue, so that the viewer user can see the common integrated chat information regardless of which virtual live venue the viewer avatar is participating in. This makes it difficult for a viewer user whose viewer avatar is participating in a sub virtual live venue to recognize that the viewer avatar is participating in the sub virtual live venue, preventing a decrease in interest in the virtual live due to the viewer avatar's participation in the sub virtual live venue.

[0478] In addition to the main virtual live venue and first sub-virtual live venue, each with a maximum of 200 viewer avatars that can participate, a second virtual live venue is provided, also with a maximum of 200 viewer avatars that can participate. When the number of viewer avatars participating in the first virtual live venue reaches 150, the distribution server computer 100 restricts the participation of new viewer avatars in the first sub-virtual live venue, while allowing new viewer avatars to participate in the second sub-virtual live venue. This eliminates the risk of the virtual live performance becoming impossible to distribute due to more than 150 viewer avatars participating in the first virtual live venue, while preventing a decline in interest in the virtual live performance.

[0479] <Modification of Example 5> In the above-mentioned Example 5, the viewer terminal 300 and the performer terminal 200 where the viewer avatar is participating in the main virtual live venue display images of the main virtual live venue as performer-perspective images and viewer-perspective images, but do not display images of the fir...

Claims

1. A virtual space content distribution system capable of distributing virtual space content in a virtual space, a viewer user terminal capable of outputting an image of the virtual space content, the viewer user terminal being capable of viewing the image; a server computer that is connected to the viewer user terminal via a communication network so as to be capable of data communication, and that is capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content; At least There are a plurality of viewer users, and each viewer user is assigned a viewer user terminal; The virtual space content may include a viewer avatar by allowing the viewer user to participate in the virtual space that is the subject of the virtual space content using the viewer user terminal; the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for displaying than the first viewer avatar; During a specific period during a distribution period of the virtual space content, the viewer avatar in the virtual space content is set to the second viewer avatar; Among the viewer avatars that are set as the second viewer avatars, the viewer avatar of a viewer that satisfies a special condition can be changed from the second viewer avatar to the first viewer avatar, Among the viewer avatars that are set as the second viewer avatars, the viewer avatars of the viewer who do not satisfy the special condition cannot change from the second viewer avatar to the first viewer avatar. A virtual space content distribution system.

2. the virtual space content includes content by a performer avatar; a performer terminal that is usable by a performer corresponding to the performer avatar and is connected to the server computer via the communication network so as to be capable of data communication; The special condition includes that a predetermined operation is performed by the performer on a specific viewer avatar on the performer terminal.

2. The virtual space content delivery system according to claim 1.

3. a terminal usable by an administrator who manages the virtual space, the administrator terminal being connected to the server computer via the communication network so as to be capable of data communication; The special condition includes that a predetermined operation is performed by the administrator on a specific viewer avatar on the administrator terminal.

2. The virtual space content delivery system according to claim 1.

4. a means for identifying the viewer avatar that satisfies a prerequisite corresponding to the special condition; a presentation means for presenting the viewer avatar identified by the identification means; Equipped with 4. The virtual space content delivery system according to claim 2 or 3.

5. The prerequisites include conditions regarding payment for the virtual space content.

5. The virtual space content delivery system according to claim 4.

6. The preconditions include a condition regarding a virtual item possessed by the viewer avatar.

5. The virtual space content delivery system according to claim 4.

7. the virtual space includes a special area assigned to a performer avatar participating in the virtual space and a specific area assigned to the viewer avatar participating in the virtual space; The special condition includes that the viewer avatar is located in the special area.

2. The virtual space content delivery system according to claim 1.

8. the virtual space has a special object at a boundary between the special area and the specific area; For the viewer avatar that does not satisfy the special condition, a collision between the viewer avatar and the special object is determined, and for the viewer avatar that satisfies the special condition, a collision between the viewer avatar and the special object is not determined.

8. The virtual space content delivery system according to claim 7.

9. A virtual space content distribution program for distributing virtual space content in a virtual space, The virtual space content includes: A content including a video that can be viewed by a plurality of viewer users at a viewer user terminal assigned to each of the viewer users, The viewer user terminal may be connected via a communication network to a server computer capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content, and the viewer user terminal may be connected to the server computer so as to be capable of data communication with the server computer, and the viewer user terminal may include a viewer avatar by allowing the viewer avatar to participate in the virtual space that is the target of the virtual space content; the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for displaying than the first viewer avatar; and a viewer avatar process for setting the viewer avatar in the virtual space content as the second viewer avatar during a specific period during the distribution period of the virtual space content, and for enabling the viewer avatar set as the second viewer avatar that satisfies a special condition to be changed to the first viewer avatar, and for disabling the viewer avatar of a viewer who does not satisfy the special condition to be changed to the first viewer avatar. A virtual space content distribution program.

10. A virtual space content distribution method for distributing virtual space content in a virtual space, comprising: The virtual space content includes: A content including a video that can be viewed by a plurality of viewer users at a viewer user terminal assigned to each of the viewer users, The viewer user terminal may be connected via a communication network to a server computer capable of executing at least processes related to the virtual space and processes related to the distribution of the virtual space content, and the viewer user terminal may be connected to the server computer so as to be capable of data communication with the server computer, and the viewer user terminal may include a viewer avatar by allowing the viewer avatar to participate in the virtual space that is the target of the virtual space content; the viewer avatars include a first viewer avatar and a second viewer avatar that requires a smaller processing load for displaying than the first viewer avatar; A viewer avatar procedure is included in which the viewer avatar in the virtual space content is set as the second viewer avatar during a specific period during the distribution period of the virtual space content, and among the viewer avatars set as the second viewer avatars, the viewer avatars that satisfy a special condition are made changeable to the first viewer avatar, and the viewer avatars of viewers who do not satisfy the special condition are made unchangeable to the first viewer avatar. A virtual space content distribution method comprising:

Images