Virtual Space Provision System, Virtual Space Provision Program, and Virtual Space Provision Method

Abstract

Preventing problems from occurring in the display output of the virtual space video. 【Solution means】A virtual space providing system that provides a virtual space in which a user avatar corresponding to a user can participate, including a user terminal and a server computer capable of executing processing related to video distribution in the virtual space. The virtual space can set a target space for an event, and the user avatar includes a first mode avatar and a second mode avatar that is a mode with a smaller processing load for display output than the first mode avatar. Within the area of the target space, the user avatar is set as the second mode avatar.

Description

Technical Field

[0001] The present invention relates to a virtual space providing system, a virtual space providing program, and a virtual space providing method for providing a virtual space in which a user avatar corresponding to a user can participate.

Background Art

[0002] Conventionally, a user uses a user terminal connected to a server computer for constructing a virtual space to participate a virtual character (viewer avatar, first viewer avatar) indicating the user in the virtual space, and it is known that a perspective video from the virtual character is displayed and output on each user terminal (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, since avatars corresponding to respective users can participate in the virtual space, it is necessary to display and output each of these avatars participating in the virtual space as a video of the virtual space. For example, when an event such as a live event on the road is held in the virtual space, a large number of avatars gather in the area of these events, and a large number of other avatars are included in the perspective video of the user participating in the event. Thus, it is necessary to draw and display these large number of avatars, resulting in a problem that a defect occurs in the display and output of the video of the virtual space.

[0005] The present invention has been made paying attention to such problems, and an object thereof is to provide a virtual space providing system, a virtual space providing program, and a virtual space providing method capable of preventing a problem from occurring in the display output of an image in a virtual space.

Means for Solving the Problems

[0006] The virtual space providing system according to claim 1 is a virtual space providing system that provides a virtual space in which a user avatar (for example, a viewer avatar) corresponding to a user (for example, a viewer) can participate, at least a user terminal (for example, the viewer terminal 300 in Example 8) capable of outputting an avatar viewpoint video of the virtual space viewed from at least the user avatar, a server computer (for example, the virtual space server computer 100' in Example 8) that is connected to the user terminal via a communication network so as to be capable of data communication, and that can execute at least processing related to the virtual space and processing related to video distribution of the virtual space, and includes at least the virtual space is configured to be able to set a target space (for example, an event area) of an event, the user avatar includes a first mode avatar (for example, the full avatar in Example 8) and a second mode avatar (for example, the first simple avatar in Example 8) that has a smaller processing load for display output than the first mode avatar, within the area of the target space, the user avatar is set as the second mode avatar (for example, as shown in FIG. 118 of Example 8, the part where the viewer avatar located within the event area is set as the first simple avatar) and is characterized by this. According to this feature, by setting the user avatar as the second mode avatar with a small processing load for display output, it is possible to prevent the processing load for video output of the target space from increasing significantly due to an increase in the number of user avatars located within the area of the target space.

[0008] Claim2 The virtual space providing system of 1 is the virtual space providing system described in claim wherein the mode of the second mode avatar is different between the user avatar located within the area of the target space and the user avatar located outside the area of the target space (for example, the viewer avatar within the event area is the first simple avatar, and the viewer avatar outside the event area is the second simple avatar with a lower processing load for display output than the first simple avatar). It is characterized by this. According to this feature, it is possible to easily distinguish whether the user avatar is the user avatar in either the area within or outside the area of the target space.

[0009] Claim 3 The virtual space providing system of 2 is the virtual space providing system described in claim wherein the degree of reduction in the processing load for display output of the mode of the user avatar located outside the area of the target space is greater than that of the mode of the user avatar located within the area of the target space (for example, the second simple avatar in Example 8 has a lower processing load for display output than the first simple avatar). It is characterized by this. According to this feature, it is possible to prevent an adverse effect on display output caused by, for example, user avatars gathering outside the area around the target space.

[0010] Claim 4 The virtual space providing system of is the virtual space providing system described in claim 1, and when the avatar viewpoint video seen from the user avatar located within the area of the target space includes the user avatar located outside the area of the target space, the user avatar is made non - display (for example, in Modification Example 8 - 1 shown in FIG. 123, the viewer avatar outside the event area is made non - display). It is characterized by this. According to this feature, it is possible to reliably prevent an increase in processing load caused by including a user avatar located outside the area of the target space in the avatar viewpoint video.

[0011] Claim 5 The virtual space providing system of is the virtual space providing system according to claim 1, wherein in the user terminal used by the user corresponding to the user avatar located outside the area of the target space, the virtual space video within the area of the target space is made unable to be output (for example, as shown in FIG. 119 of Example 8, in the viewpoint video of the viewer avatar outside the event area, the video within the event area is replaced with the image of the event information and is made unable to be output). It is characterized by this. According to this feature, it is possible to promote the entry of the user into the target space of the event.

[0012] Claim 6 The virtual space providing system of is the virtual space providing system according to claim 1, wherein the user avatar can be moved inside the virtual space by the user operating the user terminal, and can participate in the event set in the target space by moving from outside the area of the target space to inside the area, and includes participation confirmation means for performing participation confirmation for the user to participate in the event set in the target space when moving from outside the area of the target space to inside the area (for example, as shown in FIG. 121(a) in Example 8, when the viewer avatar enters the event area, a participation confirmation screen is displayed to perform participation confirmation). It is characterized by this. According to this feature, it is possible to prevent the occurrence of troubles caused by the user participating in the event by moving regardless of their own will.

[0013] Claim 7 The virtual space providing system of is the virtual space providing system according to claim 1, wherein Near-user avatar identification means for identifying a user avatar located in a vicinity area outside the area of the target space and close to the target space; Event notification means for notifying the user terminal of the user corresponding to the user avatar identified by the near-user avatar identification means of information regarding the holding of an event corresponding to the target space (for example, as shown in FIG. 120 in Example 8, transmitting a general event holding notification to a viewer avatar located in a vicinity area of an event area to inform of the scheduled holding of an event); Comprising Characterized by this. According to this feature, it is possible to notify the user corresponding to the user avatar located in the vicinity area of the target space of information regarding the holding of an event, and it is possible to effectively encourage participation in the event.

[0014] Claim 8 The virtual space providing system according to claim 1, wherein A terminal that is data communicably connected to the server computer via a communication network, and is a performer terminal (for example, the performer terminal 200 in Example 8) that can be used by a performer who participates the performer avatar in the virtual space in an event; A terminal that is data communicably connected to the server computer via a communication network, and is an administrator terminal (for example, the administrator terminal 150' in Example 8) that can be used by an administrator who manages the virtual space; Including In the performer terminal, a performer can execute an event holding request operation for requesting the holding of an event at the location where the performer avatar is located in the virtual space; On the administrator terminal, an area including the position where the performer avatar of the performer who has performed the event holding request operation in response to the event holding request by the event holding request operation is located can be set as the target space of the event (for example, as shown in FIG. 117 of Example 8, the area including the position of the performer avatar of the performer who is the event holder is set as the event area by the administrator on the administrator terminal 150’). It is characterized by this. According to this feature, the performer can hold an event at a place in the virtual space that the performer desires.

[0015] Claim 9 The virtual space providing system of 8 is the virtual space providing system described in On the performer terminal, the performer can specify the participation conditions for the event targeted by the event holding request operation (for example, as shown in FIG. 120 of Example 8, the part where the performer specifies the holding conditions such as the participation fee and the upper limit of the number of participants and sends an event holding request). It is characterized by this. According to this feature, the performer can specify the desired participation conditions.

[0016] Claim 1 0 The virtual space providing program of is a virtual space providing program for providing a virtual space in which a user avatar corresponding to a user can participate. The virtual space video output on a user terminal that is data communicably connected via a communication network to a server computer capable of executing at least processing related to the virtual space and processing related to video distribution of the virtual space includes an avatar viewpoint video of the virtual space viewed from the user avatar of the user. The virtual space can be set as the target space of the event, and the user avatar includes a first mode avatar and a second mode avatar that is a mode with a smaller processing load for display output than the first mode avatar. Without providing an object that encloses the target space It is set to be configurable, and the user avatar includes a first mode avatar and a second mode avatar that is a mode with a smaller processing load for display output than the first mode avatar. Within the area of the target space, the user avatar is made to be the second-mode avatar and, when the user avatar located outside the region of the target space is included in the avatar viewpoint video seen from the user avatar located within the region of the target space, setting the form of the user avatar to the second form avatar including avatar mode control processing. It is characterized by this. According to this feature, by using the second-mode avatar with a small processing load for displaying and outputting the user avatar, it is possible to prevent the processing load for video output of the target space from increasing significantly due to an increase in the number of user avatars located within the area of the target space.

[0017] Claim 1 1 The virtual space providing method of is A virtual space providing method for providing a virtual space in which a user avatar corresponding to a user can participate, A terminal that is data communicably connected via a communication network to a server computer capable of executing at least processing related to the virtual space and processing related to video distribution of the virtual space, and the video of the virtual space output on the user terminal available to the user includes an avatar perspective video of viewing the virtual space from the user avatar of the user, The virtual space can set an event target space Without providing an object that encloses the target space and is settable, The user avatar includes a first-mode avatar and a second-mode avatar that is a mode with a smaller processing load for display output than the first-mode avatar, Within the area of the target space, the user avatar is made to be the second-mode avatar and, when the user avatar located outside the region of the target space is included in the avatar viewpoint video seen from the user avatar located within the region of the target space, setting the form of the user avatar to the second form avatar including avatar mode control procedures. It is characterized by this. According to this feature, by using the second-mode avatar with a small processing load for displaying and outputting the user avatar, it is possible to prevent the processing load for video output of the target space from increasing significantly due to an increase in the number of user avatars located within the area of the target space.

[0018] Furthermore, the present invention may have only the invention-specific matters described in the claims of the present invention, or may have configurations other than the invention-specific matters together with the invention-specific matters described in the claims of the present invention.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Figure 56

Figure 57

Figure 58

Figure 59

Figure 60

Figure 61

Figure 62

Figure 63

Figure 64

Figure 65

Figure 66

Figure 67

Figure 68

Figure 69

Figure 70

Figure 71

Figure 72

Figure 73

Figure 74

Figure 75

Figure 76

Figure 77

Figure 78

Figure 79

Figure 80

Figure 81

Figure 82

Figure 83

Figure 84

Figure 85

Figure 86

Figure 87

Figure 88

Figure 89

Figure 90

Figure 91

Figure 92

Figure 93

Figure 94

Figure 95

Figure 96

Figure 97

Figure 98

Figure 99

Figure 100

Figure 101

Figure 102

Figure 103

Figure 104

Figure 105

Figure 106

Figure 107

Figure 108

Figure 109

【Figure 11 It is an explanatory diagram of the friend user data table in the virtual space providing system in Modification Example 7-3 of the present invention. ​ It is an explanatory diagram of the full avatar and the simple avatar in the virtual space providing system in Modification Example 7-3 of the present invention. ​ It is an explanatory diagram when determining the mode of the simple avatar at the start of a joint performance in the virtual space providing system in Modification Example 7-4 of the present invention. ​ It is an explanatory diagram of the performer avatar and the simple avatar in the virtual space providing system in Modification Example 7-5 of the present invention. ​ It is an explanatory diagram of the performer avatar and the simple avatar in the virtual space providing system in Modification Example 7-6 of the present invention. ​ It is a diagram showing the configuration of the virtual space in Example 8 of the present invention. ​ It is a diagram showing the configuration of the city area in the virtual space of Example 8 of the present invention. ​ It is a diagram showing the configuration of the park area in the virtual space of Example 8 of the present invention. ​ It is a diagram showing the perspective video from the viewer avatar who participated in the event held in the park area in the virtual space providing system of Example 8 of the present invention. ​This is a diagram showing a perspective video from a viewer avatar located outside the event area held in the park area in the virtual space providing system according to Example 8 of the present invention. ​ This is an explanatory diagram showing the flow of event settings in the virtual space providing system according to Example 8 of the present invention. ​ This is a diagram showing the state of a viewer avatar in participating in an event in the virtual space providing system according to Example 8 of the present invention. ​ This is a flowchart showing the processing contents of event space processing executed in the virtual space providing system according to Example 8 of the present invention. ​ This is a diagram showing a perspective video from a viewer avatar who has participated in an event in the virtual space providing system according to Modification Example 8-1 of the present invention.

Embodiments for Carrying Out the Invention

[0020] The embodiments for carrying out the virtual space providing system of the present invention will be described below with reference to the drawings based on examples. In a plurality of drawings, the same or similar components are denoted by the same reference numerals, and redundant explanations are omitted. Also, the virtual space content distribution program according to the present invention may be a program used in the entire system composed of a server computer and each terminal described later that constitutes the virtual space providing system, or it may be a program that operates only on a part of the devices that constitute the virtual space providing system, for example, a server computer and each terminal. Each of the processes described below is executed by a program stored in the device where the process is performed.

Examples

[0021] FIG. 1 is a block diagram showing a system configuration example of a virtual space providing system in Example 1 which is one embodiment of the present invention. In the system of Example 1 shown in FIG. 1, a virtual live in which a performer avatar 1 corresponding to a performer user conducts a live event in the virtual live venue shown in FIG. 12 is distributed as virtual space content. Note that hereinafter, "distribution" means transmission to a plurality of parties and is synonymous with "transmission".

[0022] As shown in FIG. 1, the virtual space providing system of Example 1 mainly includes a distribution server computer 100 capable of executing various processes related to the virtual live venue shown in FIG. 11 which is a virtual space and various processes related to the distribution of virtual live, an event manager M of an event operation organization that operates and manages virtual live events, a manager terminal 150 used by the event manager M to manage the distribution of virtual live events, a performer terminal 200 used by a performer user, and a viewer terminal 300 that can be used by a viewer user who virtually participates in a virtual live and views the virtual live. The distribution server computer 100, the manager terminal 150, the performer terminal 200, and the viewer terminal 300 are connected to each other via the Internet network N which is an open computer network so as to be capable of data communication.

[0023] Note that in Example 1, a form in which the manager terminal 150 and the performer terminal 200 are also connected to the distribution server computer 100 via the Internet network N is exemplified, but the present invention is not limited thereto. For example, in the case where the studio shown in FIG. 11 is a studio operated by an event operation organization and the event operation organization provides the studio to a performer user, and the distribution server computer 100 is installed in these studios, the manager terminal 150 and the performer terminal 200 may be locally data communication-connected to the distribution server computer 100, or may be communication-connected via a local area network LAN within the studio facility.

[0024] Also, as shown in FIG. 1, there are a plurality of viewer users, such as viewer user A, viewer user B, viewer user C, etc. As the viewer terminals 300, there are viewer terminals 300a, 300b, 300c, etc. that can be used by individual viewer users A, B, C, etc. Note that the number of viewer users is usually plural, but it may be one person or four or more people, and the number may be appropriately determined according to the virtual live venue or the like that is a virtual space. Hereinafter, viewer users A, B, C, etc. may be collectively referred to as viewer users, and viewer terminals 300a, 300b, 300c, etc. may be collectively referred to as viewer terminals 300.

[0025] In addition, in this Example 1, a form in which there is one performer user is illustrated, but the present invention is not limited to this. The virtual live may be a co-starring event in which a plurality of performer users who are acting alone virtually participate, or a group event in which a plurality of people who are acting together virtually participate. In this way, when a plurality of performers appear, performer terminals 200a, 200b, 200c, etc. may be provided for each performer user, or if a plurality of people appear as a group, one performer terminal 200 may be used by a plurality of people.

[0026] In FIG. 1, the distribution server computer 100 is shown as a single unit, but the present invention is not limited to this. These distribution server computers 100 may be composed of a plurality of server computers such as a first server computer mainly performing virtual space processing and a second server computer mainly performing distribution processing, or may be composed of a cloud server or the like, and the number of server computers may be appropriately changed according to the scale of the event, the number of listener users participating, etc.

[0027] Similarly, although the administrator terminal 150 is shown as a single unit in FIG. 1, the present invention is not limited thereto, and it goes without saying that these administrator terminals 150 may be plural according to the number of performer users and viewer users who virtually participate in the live event, etc.

[0028] <Distribution server computer> FIG. 2 is a diagram showing the configuration of the distribution server computer 100 used in the first embodiment. The distribution server computer 100 used in the first embodiment is a normal server computer with relatively excellent processing capabilities installed in a studio operated by an event management organization as described above. As shown in FIG. 2, it includes a processor (CPU) 101 connected to a data bus 108, a memory (RAM) 102, a storage 103 such as a hard disk, a communication interface (I / F) 104 capable of two-way communication by the TCP / IP protocol via the Internet which is an open computer network, and an input / output interface (I / F) 105 to which an input device such as a keyboard can be connected.

[0029] In addition, in the first embodiment, an input / output interface (I / F) 105 is provided to enable local operations. However, when remote operations via the communication interface (I / F) 104 are possible, the input / output interface (I / F) 105 may not be provided.

[0030] In the storage 103 of the distribution server computer 100, in addition to an operation system (server OS) for providing a server function (not shown), various data and programs are stored as shown in FIG. 3. Specifically, mainly, an event management program 110 for providing an event management function for managing live events, a virtual space generation program 120 for providing a virtual space of a virtual live venue which is a virtual space, a performer user management program 130 for providing a performer user management function for managing performer users, and a viewer user management program 132 for providing a viewer user management function for managing viewer users are stored.

[0031] The event management program 110 is a program for managing a virtual live event, which is an event held in a virtual live venue (see Fig. 12) in cooperation with the administrator terminal 150. The progress of the virtual live event can be managed using the event data 111 (see Fig. 7), time schedule data (TS data) 112, and music data 113 stored in the storage 103 together with the event management program 110.

[0032] The event data 111 used in the first embodiment is illustrated in Fig. 7. The event data 111 is associated with an event ID uniquely assigned to the event, and stores information such as the event name, the file name of the time schedule (TS) data, the performer ID uniquely assigned to the performer user who is the performer, the file name of the music list, the file name of the participation viewer list in which information of special viewer users who are permitted virtual participation is registered in the third area of the virtual audience area described later, and the file name of the participation viewer list in which information of viewer users who have reserved virtual participation is registered in the fourth area of the virtual audience area described later.

[0033] The time schedule data (TS data) 112 is data in which the order of progress of the virtual live event to be held and the programs such as music are described in time series, and the virtual live event is managed to proceed based on the time schedule described in the time schedule data (TS data) 112. Further, the time schedule data (TS data) 112, the event data 111, etc. can be changed, updated, and added by the event administrator M operating the administrator terminal 150.

[0034] The music data 113 is data of the performance part (so-called karaoke) of each music sung by the performer user, and is reproduced by the voice control program 124 described later based on the time schedule data (TS data) 112 by the event management program 110 and transmitted to the performer terminal 200 and the viewer terminal 300 (see Fig. 11).

[0035] The virtual space generation program 120 is a program for providing a virtual live venue which is the virtual space shown in FIG. 12. Using virtual space data 121 which describes the structure of the virtual live venue and the like stored in the storage 103 together with the virtual space generation program 120, object data 122 for arranging various objects (including virtual arrangement object 43 which is a star decoration on the stage and virtual air objects 50 to 52 which are star objects in the sky) and the like in the virtual space, and the performer avatar data included in the performer user data 131 described later and the viewer avatar data included in the viewer user data 133, it has a function of providing a virtual live venue in which performer avatars and viewer avatars virtually participate.

[0036] As shown in FIG. 12, the viewpoint video control program 123 provides a function of generating viewpoint videos from virtual cameras C1 to C4 and an aerial movement virtual camera (not shown) virtually set in the virtual live venue, and cooperates with the viewer terminal 300 to provide a function of displaying videos from the viewpoints of viewer avatars 11 to 14 virtually participating in the third area of the virtual live venue and viewer avatars 21 to 32 virtually participating in the fourth area on the viewer terminal 300. Note that the virtual camera C2 is a virtual camera set to be the video from the viewpoint of the performer avatar 1 on the stage of the virtual live venue, and the viewpoint video from the virtual camera C2 is projected onto the screen S provided in the studio by a projector 211 described later connected to the performer terminal 200 in cooperation with the performer terminal 200.

[0037] The audio control program 124 is a program for controlling the audio (including music) distributed in the virtual live event. Specifically, it provides a function of playing the music data 113 and a function of synthesizing the music played with the voice uttered by the performer user and distributing it to the administrator terminal 150 and the viewer terminal 300.

[0038] The comment control program 125 provides a function to display various comments input by each viewer user on the viewer terminal 300 in chronological order on the performer terminal 200 and the viewer terminal 300 in cooperation with the viewer comment control program stored in the viewer terminal 300 described later.

[0039] The gift control program 126 is a program for controlling gifts presented from viewer users to performer users in cooperation with the viewer program stored in the viewer terminal 300 described later. Specifically, it provides functions such as the function for viewer users to purchase gifts based on the information of the purchase unit price set for various gifts stored in the gift data 134, the processing function (including processing such as effects when presenting) for viewer users to present the gifts purchased using the object data of the gifts included in the gift data 134 to performer users, and the processing function for changing the owner of the presented gift from the viewer user to the performer user.

[0040] The performer user management program 130 provides a management function and an authentication function for information related to performer users based on the performer user data 131 shown in FIG. 8 stored in the storage 103 together with the performer user management program 130.

[0041] The performer user data 131 used in the first embodiment is illustrated in FIG. 8. The performer user data 131 stores various information such as the account (email address), name, authentication information, file name of the avatar data of the avatar used in the virtual space, and file name of the item list in which the owned items are registered, in association with the performer user ID uniquely assigned to the performer user. Although not shown in FIG. 8, information on virtual values such as points owned by each performer user may be stored as the performer user data 131.

[0042] The performer user can be authenticated by verifying the performer user ID, account, authentication information, etc., and can identify the items (gifts) owned by each performer user by being given by viewers from the item list.

[0043] Based on the viewer user data 133 shown in FIG. 9 stored in the storage 103 together with the program, the viewer user management program 132 provides a management function and an authentication function for information related to the viewer user.

[0044] The viewer user data 133 used in the first embodiment is illustrated in FIG. 9. The viewer user data 133 is associated with the viewer user ID uniquely assigned to the viewer user, and includes the account (e-mail address) of the viewer user, authentication information, the file name of the avatar data of the avatar used in the virtual space, the number of owned points which is the number of points available for use in the virtual space, the file name of the item list in which the items (gifts) purchased using the points are registered, personal information such as name, date of birth, telephone number, and other various information. Although not shown in FIG. 9, the nickname (comment name) displayed together with the comment is also registered in the viewer user data 133, and the nickname (comment name) is displayed together with the comment. Also, the points can be increased, for example, by purchasing from a predetermined operating company.

[0045] <Performer terminal> FIG. 4 is a diagram showing a configuration example of the performer terminal 200 in the virtual space providing system of the first embodiment. In the first embodiment, as shown in FIG. 11, the performer terminal 200 is provided in an adjustment room adjacent to the studio where the performer user performs acting operations, and uses a normal computer with relatively excellent processing capabilities. As shown in FIG. 4, it has a processor (CPU) 201 connected to a data bus 208, a memory (RAM) 202, a storage 203 such as a hard disk, a communication interface (I / F) 204 capable of two-way communication via the TCP / IP protocol through the Internet which is an open computer network, an image processing unit 206 including a graphics processing unit (GPU) to which display devices A210 to display devices C212 are connected, and an input / output interface (I / F) 205 to which various input / output devices are connected.

[0046] Connected to the input / output interface (I / F) 205 are, as input devices, a motion sensor 220 including a plurality of mounting sensors 220C1 to 220C5 (see FIG. 11) worn by the performer user on the left and right hands, feet, and waist, an expression input device 221 composed of an imaging camera for inputting the expression of the performer's face, a voice input device 222 composed of a sound collecting microphone worn on the head of the performer user, an operation input device 224 such as a keyboard or a touch panel on which an auxiliary operator O for assisting the performer user can be operably provided, and the like. Incidentally, as these input devices, a controller that can perform various operations by being held by the performer user may be provided so that the performer himself / herself can perform various operations without the auxiliary operator O.

[0047] In the first embodiment, in order for the performer user to perform a performance operation (including music operations such as singing and dancing) in accordance with the music, the operation during the performance is assisted by the auxiliary operator O. However, the auxiliary operator O may be plural, or the form without providing the auxiliary operator O may be adopted by setting the operation content in advance in the performer terminal 200, for example, in a sequence program or the like.

[0048] The input / output interface (I / F) 205 is connected to an audio output device 223 including high-function earphones (ear monitors) worn by the performer user as an output device and speakers arranged in the adjustment room. As a result, the sound of the reproduced music transmitted from the distribution server computer 100 is output to the performer user via the high-function earphones (ear monitors), and the music sound is output from the speakers together with the voice of the performer. Thus, the auxiliary operator O and the like can confirm the situation of the music including the voice uttered by the performer user in the adjustment room.

[0049] In the first embodiment, high-function earphones (ear monitors) are used to avoid the inconvenience caused by the input of the reproduced music sound from the sound collection microphone worn by the performer user. However, the present invention is not limited to this. As long as the inconvenience caused by the input of the music sound can be avoided, the music sound may be output from speakers in the studio or the like.

[0050] In the first embodiment, the form in which the sound collection microphone is worn by the performer user is exemplified. However, these sound collection microphones may be of an installation type installed on the floor, wall, or ceiling of the studio.

[0051] Here, the motion sensor 220 used in the performer terminal 200 of the first embodiment will be described. Any motion sensor 220 can be used as long as it can appropriately detect (measure) the movement (operation) of the body of the performer user. In the first embodiment, a plurality of wearable sensors 220C1 to 220C5 worn by the performer user on the body are used to enable more accurate and short-cycle detection of the performer's movement. Note that a motion sensor 220 of a type in which the performer does not wear a device, such as LiDER using laser light or the like, may be used.

[0052] In the present Example 1 as well, in order to reduce the operational burden due to wearing sensors and the like when the performer user performs a performance operation, particularly the operational burden on the head, the movement of the head of the performer user is detected by image recognition using an image captured by the imaging camera that constitutes the expression input device 221 as described later, and a form in which five wearable sensors 220C1 to 220C5 are used for movements other than the head is exemplified. However, for example, wearable sensors may also be provided on the head or the like of the performer user, or further, in order to detect more detailed movements, more (7 or more) wearable sensors may be worn.

[0053] As shown in FIG. 11, the wearable sensors 220C1 to 220C5 cooperate with the base stations 220a and 220b installed in the studio room that separates the adjacent adjustment rooms and the glass windows to detect their own positions and orientations.

[0054] As an example, these base stations 220a and 220b can use multi-axis laser emitters. After emitting the blinking light for synchronization, the base station 220a scans the laser light, for example, around the vertical axis. The base station 220b scans the laser light, for example, around the horizontal axis.

[0055] Each of the wearable sensors 220C1 to 220C5 may include a plurality of optical sensors that detect the incidence of the blinking light and the laser light from the base stations 220a and 220b.

[0056] Each of the wearable sensors 220C1 to 220C5 is capable of detecting its own position and orientation based on the time difference between the incidence timing of the blinking light and the incidence timing of the laser light, the light reception time at each optical sensor, the incidence angle of the laser light detected by each optical sensor, and other information as necessary. For example, the ViveTracker provided by HTC CORPORATION or a base station can be preferably used.

[0057] Since base stations 220a and 220b emit blinking light and scan laser light at regular intervals, the detection information of each of the wearable sensors 220C1 to 220C5 is updated for each such interval. Then, the detection information indicating the position and orientation of each motion sensor calculated by each of these wearable sensors 220C1 to 220C5 is input to the performer terminal 200 via short-range wireless communication, and is transmitted as performer avatar information, together with the face motion information, to the distribution server computer 100 (see FIG. 10).

[0058] In addition, in the present First Embodiment, a form using two base stations, namely base station 220a and base station 220b, is illustrated, but the present invention is not limited thereto, and three or more of these base stations may be used.

[0059] Alternatively, a plurality of motion sensors worn by the performer user may each be equipped with a number of infrared LEDs or visible light LEDs, and the light from these LEDs may be detected by an infrared camera provided on the floor or wall of the studio, thereby detecting the position and orientation of each of the motion sensors.

[0060] In addition, in order to reduce the burden on the performer user, instead of the wearable sensors 220C1 to 220C5, a motion sensor using a lighter reflective marker may be used. That is, a reflective marker attached to each part of the performer user's body with adhesive tape or the like is photographed to generate photographed data, and the position and orientation of the reflective marker are detected by performing image processing on this photographed data, whereby the motion of the performer user may be detected. In this case, a cameraman for photographing the performer user may be arranged within the studio, and the cameraman may photograph the performer user so that the motion of the performer user can be photographed from a direction suitable for detection, thereby enabling good detection.

[0061] Also, as the motion sensor 220, a suit with a built-in inertial sensor, for example, a suit with a built-in inertial sensor for the MVN motion capture system marketed by Xsens.com, may be used, and the motion of the performer user may be detected by analyzing the sensor signal output from the inertial sensor.

[0062] As long as the facial expression input device 221 can appropriately detect the motion of the head including the face of the performer user, any device can be used. In the first embodiment, specifically, it is composed of an imaging camera (digital camera) arranged so as to be able to continuously capture an image of the head including the face of the performer user, and the motion of the head and the motion (facial expression) of the face of the performer user captured by the imaging camera are detected by image recognition, and the information of the face motion, which is the detected motion of the head and the motion (facial expression) of the face, is transmitted as performer avatar information together with the body motion information to the distribution server computer 100 as shown in FIG. 10.

[0063] Note that the facial expression input device 221 is not limited to the imaging camera (digital camera) used in the first embodiment. For example, the facial expression input device 221 may be a 3D camera capable of detecting the depth of a person's face, or for example, a mobile terminal such as a smartphone equipped with a LiDER device may be used. In this case, these mobile terminals may be worn by the performer user.

[0064] In the performer terminal 200 of the first embodiment, as described above, it has three display devices A210 to display device C212. The display device A210 is, as shown in FIG. 11, a liquid crystal monitor or the like provided in the adjustment room, the display device B211 is a projector that projects an image onto the screen S provided in the studio, and the display device C212 is a vertical large display provided adjacent to the screen S in the studio.

[0065] On the display device A210 placed in the adjustment room of the studio, images from the viewpoints of the virtual cameras C1 to C4, the viewpoint images from the aerial movement virtual camera during the flying effect period described later, the content of the commentary, and a message input window for inputting messages that the assistant operator O wants to convey to the performer user are displayed.

[0066] On the other hand, on the screen S projected from the projector which is the display device B211 provided in the studio, there is displayed an image including the avatars of the viewers who are virtually participating in the live event in the third region and the fourth region, for example, as shown in FIG. 15, which is the viewpoint image of the virtual camera C2 (see FIG. 12) from the viewpoint of the performer. Thus, the performer user can confirm the situation of the viewer users who are virtually participating in the virtual live event through the image.

[0067] In addition, on the display device C212 provided in the studio, comments from the viewer users, comments and messages input by the assistant operator O, etc. are displayed, so that the performer user can confirm the comments of the viewer users even during the performance and can also confirm the messages from the assistant operator O.

[0068] In the storage 203, in addition to the operation system (OS) for operating the computer which is the performer terminal 200, there is stored a performer program for providing various functions including the operation control function of the performer avatar 1 by the performer user in the virtual live event which is virtual space content, in cooperation with the distribution server computer 100, by being executed by the processor (CPU) 201 etc.

[0069] As shown in FIG. 4, the performer program includes a performer avatar control program, a performer viewpoint image control program, a performer viewpoint image control program, a performer voice control program, a performer comment control program, performer authentication data, etc.

[0070] The performer avatar control program mainly scans the body motion of the performer user using the motion sensor 220 and scans the face motion (expression) of the performer user using the expression input device 221, and provides a function of generating performer avatar information for operating the performer avatar and transmitting it to the distribution server computer 100.

[0071] The performer perspective video control program is a program that provides a function of generating and outputting a perspective video of a virtual camera C2, which is the performer's perspective, based on the virtual space update data distributed from the distribution server computer 100.

[0072] The performer voice control program outputs the playback sound of the music based on the music playback data distributed from the distribution server computer 100 via the voice output device 223, digitizes the voice input from the voice input device 222, transmits it to the distribution server computer 100 as performer voice data, and provides a function of generating and outputting a monitor voice composed of the playback sound of the music and the performer voice.

[0073] The performer comment control program is a program that provides a function of displaying comments and the like by the viewer user distributed from the distribution server computer 100.

[0074] The performer authentication data is data for authenticating that the performer user is the actual person in the communication connection with the distribution server computer 100.

[0075] In addition, the performer avatar control program includes a motion parameter generation program that can generate parameter information on the positions and rotation angles of bones (bones) that make up rig data (sometimes called skeleton data) indicating the skeleton of the performer avatar included in the performer avatar data, and also includes an image recognition processing program for image-recognizing the position and expression of the head of the performer user from the image captured by the imaging camera.

[0076] In addition, in the first embodiment, the performer terminal 200 is a computer serving as the performer terminal 200 installed in a studio operated by an event management organization, and is jointly used by a plurality of performer users. When holding a live event, each performer user can use the performer terminal 200 by inputting performer authentication data. However, the present invention is not limited to this. For example, when a performer user participates in a virtual live from home or the like, the performer terminal 200 may be a dedicated computer installed at the performer user's home, and only the owning performer user may use it. In such a case, it is not always necessary to store the performer authentication data.

[0077] As described above, by using the performer terminal 200 that constitutes the virtual space providing system of the first embodiment by a performer user or the like, the performer user can operate the performer avatar 1 arranged on the virtual stage G in the virtual live venue in conjunction with his / her own actions, and can reflect his / her own expression on the expression of the performer avatar 1. In addition, the performer user's own voice can be distributed to viewers through the distribution server computer 100 as the voice of the performer avatar 1.

[0078] <Viewer terminal> FIG. 5 is a diagram showing a configuration example of the viewer terminal 300 in the virtual space providing system of the first embodiment. In the first embodiment, the viewer terminal 300 is a smartphone P carried by a viewer user or a stationary ordinary computer (PC) installed at the viewer user's home or the like. As shown in FIG. 5, it includes a processor (CPU) 301 connected to a data bus 308, a memory (RAM) 302, a storage 303 such as a hard disk or a non-volatile memory, a communication interface (I / F) 304 capable of two-way communication using the TCP / IP protocol via the Internet which is an open computer network, an image processing unit 306 including a graphics processing unit (GPU) to which a display device 310 is connected, and an input / output interface (I / F) 305 to which various input / output devices are connected.

[0079] As will be described later, the display device 310 can display each viewpoint video generated based on the virtual space data of the virtual live venue, which is a virtual space updated by virtual space update data (see FIG. 10) distributed from the distribution server computer 100 by the viewer viewpoint video control program included in the viewer program stored in the storage 303, and the flying viewpoint video distributed from the distribution server computer 100 during the flying effect period to be described later, as long as it can display them. When the viewer terminal 300 is the smartphone P, the display of the smartphone P is applicable. When the viewer terminal 300 is a normal computer (PC), the stationary display that constitutes the computer (PC) is applicable. Note that these displays are not limited to those that display a planar view image (2D image), and may be able to display a stereoscopic image (3D image) by displaying a right-eye image and a left-eye image, such as a head-mounted display (HMD) that a viewer can wear.

[0080] Connected to the input / output interface (I / F) 305 are, as input devices, an operation input device 321 composed of one or a plurality of devices that enable a viewer to perform various operations, and an audio output device 322 such as earphones or a speaker that can output live audio or the like distributed from the distribution server computer 100.

[0081] Needless to say, the audio output device 322 may not be a single device and may be integrated with the display device 310 like the above-described head-mounted display (HMD).

[0082] The operation input device 321 can use any device as long as it can execute various operations such as operation operations including movement of the viewer avatar, zoom operations, operations related to comments, operations related to gifts, etc. For example, it is composed of at least one or more of a transparent touch panel formed on the display surface of a smartphone, a keyboard that constitutes a computer, a game controller, and the like.

[0083] Furthermore, in the first embodiment, as a viewer, as will be described later, the virtual audience area in the virtual live venue is divided into a third area near the virtual stage G where only viewer users specially permitted by the performer user can virtually participate, as shown in FIG. 12, and a fourth area around the third area where general viewer users can virtually participate. Viewer users who virtually participate in the fourth area may use an operation input device 321 such as a touch panel, a keyboard, or a game controller as described above. However, viewer users who virtually participate in the third area are viewer users who can finely operate the viewer avatar according to their own actions, like the performer user, by owning, for example, an MVN motion capture suit with the above-described inertial sensor as the operation input device 321.

[0084] However, even viewer users permitted to virtually participate in the third area cannot change the expression of the viewer avatar. However, the present invention is not limited to this, and the expressions of these viewer avatars may be changed, for example, as described above, by having the viewer user use the camera function of a smartphone or a camera connected to a computer to recognize the facial expression of the viewer user, even at a level different from the change level of the performer user.

[0085] In addition, various input / output devices connected to these input / output interfaces (I / F) 305 may be built-in or external. In the case of an external device, it may be connected in either a wired or wireless connection form.

[0086] In the storage 303, in addition to an operation system (OS) for operating a smartphone or a computer which is the viewer terminal 300, a viewer program for providing various functions including a live viewing function related to viewer users in a virtual live which is virtual space content by being executed by a processor (CPU) 301 or the like is stored as a viewing app in cooperation with the distribution server computer 100.

[0087] As shown in FIG. 5, the viewer program includes a viewer perspective video control program that provides a function of generating each perspective video of the virtual live to be displayed on the display device 310, a viewer audio control program that provides a function of outputting live audio (data) of the virtual live distributed from the distribution server computer 100 from the audio output device 322, a function of displaying comments based on the comment data distributed from the distribution server computer 100, and a function related to comments such as transmitting the comments input by the viewer to the distribution server computer 100. It also includes viewer authentication data for authenticating that the viewer user is the person himself / herself in the communication connection with the distribution server computer 100.

[0088] Although not shown in FIG. 5, in addition to the above-described programs, there are included a gift control program for performing control related to gifts, and an area designation program for reserving an area (position) where the viewer avatar virtually participates in the virtual live venue in advance before the start of the live event, as shown in Modification Example 3-1 (see FIGS. 42 and 43) described later. However, other programs other than these may also be included.

[0089] In the first embodiment, the comments input by the viewer are short text messages in text format. However, they are not limited to these short text messages in text format. For example, they may be messages in still image format, messages in video format, or any other arbitrary format of electronic messages. The comments are superimposed and displayed on each perspective video on the display device 310.

[0090] In this way, by using the viewer terminal 300 that constitutes the virtual space providing system of the first embodiment by the viewer user, the viewer avatar virtually participating in the virtual live venue can be moved and operated, and the viewpoint video displayed on the display device 310 can be changed by changing the viewpoint by the viewer avatar. At the same time, the zoom state (state of the viewpoint) can also be changed, and the viewer avatar can be operated to cheer for the performer avatar 1.

[0091] In addition, in the case of a viewer avatar that has virtually participated in the third area, it is only possible to move within the third area and it is not possible to move to the fourth area. On the other hand, in the case of a viewer avatar that has virtually participated in the fourth area, it is only possible to move within the fourth area and it is not possible to move to the third area. However, the present invention is not limited to this, and in the case of a viewer avatar that has virtually participated in the third area, it may be possible to move to the fourth area. In addition, of course, the viewer avatar cannot move to the virtual stage G or the like, so the visibility of the performer avatar 1 will not be hindered by the movement of the viewer avatar on the virtual stage G. <00006>

[0092] In addition, while the viewpoint video is displayed on these display devices 310 and the live audio is output, the virtual live can be enjoyed as if one were present at the virtual live venue.

[0093] In addition, although detailed description is omitted, by the comment function provided by the viewer comment control program, the virtual live can be enjoyed while checking the comments input by oneself and other viewer users, and by the gift function by the gift control program, the virtual live can be livened up by giving the items one owns to the performer avatar 1.

[0094] <Administrator terminal> FIG. 6 is a diagram showing a configuration example of the administrator terminal 150 in the virtual space providing system of the first embodiment. In the first embodiment, the administrator terminal 150, together with the performer terminal 200, uses a normal computer provided in an adjustment room adjacent to the studio. As shown in FIG. 6, it includes a processor (CPU) 151 connected to a data bus 158, a memory (RAM) 152, a storage 153 such as a hard disk, a communication interface (I / F) 154 capable of two-way communication by the TCP / IP protocol via the Internet which is an open computer network, an image processing unit 156 including a graphics processing unit (GPU) to which a display device 160 is connected, and an input / output interface (I / F) 155 to which various input / output devices are connected.

[0095] As will be described later, the display device 160 can individually display the perspective videos of each of the virtual cameras C1 to C4 and the aerial movement virtual camera during the flying effect period, which are generated based on the virtual space data of the virtual live venue, which is a virtual space updated by the virtual space update data (see FIG. 10) distributed from the distribution server computer 100 by the administrator perspective video control program included in the administrator program stored in the storage 153, and can display comments based on the comment data distributed from the distribution server computer 100. It may be configured by one or more stationary displays.

[0096] Connected to the input / output interface (I / F) 155 as input devices are an operation input device 161 composed of one or more devices that enable various operations by the event administrator M, and a voice output device 162 such as earphones, headphones, or speakers capable of outputting live audio and the like distributed from the distribution server computer 100.

[0097] The operation input device 161 can be any device as long as the event manager M can execute various operations such as input and setting related to the virtual live event. For example, it can be composed of at least one of the keyboard that makes up a computer, the transparent touch panel formed on the display surface, the viewpoint switching controller, and the like.

[0098] In addition, various input / output devices connected to these input / output interfaces (I / F) 155 may be in any connection form, either wired or wireless.

[0099] In the storage 153, in addition to the operation system (OS) for operating the computer which is the administrator terminal 150, various functions including the distribution management function related to the distribution of the virtual live which is virtual space content are provided in cooperation with the distribution server computer 100 by the execution of the processor (CPU) 151 and the like. A management program for this purpose is stored.

[0100] As shown in FIG. 6, the management program for administrators includes a management viewpoint video control program, a management voice control program, a management comment control program, a viewer management program, a performer management program, and the like.

[0101] The management viewpoint video control program is a program that provides a function of generating a viewpoint video from each virtual camera to be displayed on the display device 160 and a function of changing and switching each viewpoint.

[0102] The management voice control program is a program that provides a function of outputting the live voice based on the voice data of the virtual live distributed from the distribution server computer 100 from the voice output device 162.

[0103] The management comment control program displays comments based on the comment data distributed from the distribution server computer 100, and provides functions related to comments such as prohibiting the distribution of comments and selecting viewers so that they are not distributed.

[0104] The viewer management program is a program for managing viewer users who virtually participate in a virtual live event, and the performer management program is a program for managing performer users who virtually participate in a virtual live event.

[0105] Although not shown in FIG. 6, there are included an authentication program for the administrator (the operator) to authenticate whether the administrator is the actual person, a schedule program for editing the time schedule, music order, etc. of the virtual live event, and other programs may be included.

[0106] In this way, by the event administrator M operating the administrator terminal 150 that constitutes the virtual space providing system of the first embodiment, various settings for executing an event can be carried out, for example, settings regarding the program, music order, production, time schedule, the trajectory along which the virtual camera moves in the air during the flying production period, etc. And the virtual live event is managed based on the content set in this way.

[0107] <Virtual live venue> The virtual live venue, which is the virtual space used in the first embodiment, is shown in FIG. 12. As shown in FIG. 12, the virtual live venue is a virtual space that, similar to the live venue in the real space, has a virtual stage G on which the performer user appears as the performer avatar 1 and a virtual audience area in which the viewer users who become the audience virtually participate as viewer avatars.

[0108] As shown in FIG. 12, the virtual stage G has a trapezoidal floor surface with a relatively large area, and the performer avatar 1 can move on the virtual stage G, similar to the live venue in the real space. On the side opposite to the virtual audience direction of the virtual stage G, as shown in FIG. 12, a virtual stage wall divided into three parts: a central part, a right side part, and a left side part is formed, and virtual displays 40, 41, and 42 are provided on the front surfaces of these virtual stage walls, and images and videos for production are virtually displayed on the virtual displays 40, 41, and 42.

[0109] Between the virtual displays 40, 41, and 42, there is a virtual arrangement object 43 of stars related to the performer user. In the second area on the virtual stage G, which is an area above the first area surrounded by the virtual stage wall, as shown in FIG. 12, large virtual sky objects 50 to 52 of stars are arranged. The virtual sky objects 50 to 52 can move within the second area.

[0110] In front of the virtual stage G, there is a virtual audience area where viewer users can virtually participate in the virtual live venue as viewer avatars. As shown in FIG. 12, this virtual audience area includes a third area, which is a special area close to the center of the virtual stage G where the performer avatar 1 appears on stage, and a fourth area formed to surround the third area and having a greater distance from the center of the virtual stage G than the third area.

[0111] In these third areas, within a predetermined upper limit number range, special viewer users who are permitted virtual participation can virtually participate. In the first embodiment, as shown in FIG. 12, for example, viewer avatars 11 to 14 of four viewer users who meet a predetermined virtual participation record and are permitted virtual participation by the performer user are arranged.

[0112] Also, in the fourth area, viewer avatars 21 to 32 of general viewer users who reserved virtual participation before the start of the virtual live are arranged. In FIG. 12, the viewer avatars 21 to 32 are shown in a simplified form for convenience, but they are the same avatars as the viewer avatars 11 to 14.

[0113] Each of the viewer avatars 11 to 14 arranged in the third area can move within the third area by the viewer user operating the viewer terminal. Each of the viewer avatars 21 to 32 arranged in the fourth area can also move within the fourth area by the viewer user operating the viewer terminal.

[0114] In the virtual live venue, as shown in FIG. 12, four virtual cameras C1 to C4 are virtually arranged (set). The virtual camera C1 is a virtual camera that is virtually arranged in front of the performer avatar 1 so as to face the performer avatar 1. The perspective video by the virtual camera C1 is, as shown in FIG. 13(a), a perspective video of the performer avatar 1 viewed from a position near the front.

[0115] The virtual camera C2 is a camera that is virtually arranged (set) above the performer avatar 1. The perspective video by the virtual camera C2 is, as shown in FIG. 15, a performer's perspective video of viewing the virtual audience area from the performer avatar 1.

[0116] The virtual camera C3 is a virtual camera that is virtually arranged on the virtual display 41 diagonally behind the performer avatar 1. The perspective video by the virtual camera C3 is, as shown in FIG. 13(c), a perspective video of viewing the virtual audience area from diagonally behind the performer avatar 1.

[0117] The virtual camera C4 is a virtual camera that is virtually arranged on the virtual display 42 diagonally behind the performer avatar 1. The perspective video by the virtual camera C4 is, as shown in FIG. 13(b), a perspective video of viewing the virtual audience area from diagonally behind the performer avatar 1.

[0118] Also, in the first embodiment, in addition to the fixed virtual cameras C1 to C4, an airborne moving virtual camera (not shown) that can move in the air within the space of the virtual live venue, which is a virtual space, is virtually arranged. The airborne trajectory along which these airborne moving virtual cameras move and the perspective direction (angle) at each position on the airborne trajectory, etc. are preset by the administrator terminal 150. During the flying performance period described later, the performer avatar 1 virtually flies so as to follow the airborne trajectory of the airborne moving virtual camera, and a perspective video of the virtually flying performer avatar 1 captured by the airborne moving virtual camera is generated.

[0119] Furthermore, in the first embodiment, as shown in FIG. 12, a form is exemplified in which four virtual cameras C1 to C4 are virtually arranged as fixedly arranged (set) virtual cameras. However, the present invention is not limited to this, and the number of these fixedly arranged (set) virtual cameras may be five or more. Conversely, without providing fixedly arranged (set) virtual cameras, only the perspective video seen from the perspective corresponding to each avatar virtually participating in the virtual live venue may be displayed on the performer terminal 200 and the viewer terminal 300.

[0120] Also, in the first embodiment, a form in which there is one air-moving virtual camera is exemplified. However, the present invention is not limited to this, and a form in which there are a plurality of these air-moving virtual cameras may be used. Conversely, a form in which air-moving virtual cameras are not virtually arranged may also be used.

[0121] <Studio> FIG. 11 is a diagram showing the studio used in the first embodiment. As described above, an adjustment room separated by a glass window is provided adjacent to the studio. The performer user performs each operation corresponding to singing in the studio, and the auxiliary operator O assists in operating the performer terminal 200 installed in the adjustment room. Around the performer user wearing the mounting sensors 220C1 to 220C5, the base station 220a and the base station 220b are mounted on the stand, and in almost the front position of the performer user, an imaging camera constituting the expression input device 221 is mounted on the stand.

[0122] A screen S is provided on the wall surface where the performer user confronts in the studio. The perspective video of the virtual camera C2 is projected onto the screen S from the projector (display device B211) arranged on the ceiling of the studio. As shown in FIG. 15, the perspective video seen from the performer avatar 1 looking at the virtual audience area is displayed on the screen S, and the performer user can proceed with the live while sequentially grasping the state of the viewer avatars virtually participating in the virtual audience area (the reaction of the viewers to the live).

[0123] Also, a vertical large display as a display device C212 is arranged at a side position of the screen S, and comments of the viewers and messages that the auxiliary operator O wants to convey to the performer user are displayed on the large display. Note that lyrics of the songs sung by the performer during the live performance, information such as lines (performer support information), etc. may be displayed on the display device C212.

[0124] <Operation of the virtual space providing system> FIG. 10 is an explanatory diagram for explaining the operation of the virtual space providing system according to the first embodiment, specifically, the flow in which the video and audio of the virtual live performance are distributed.

[0125] First, the video system will be described. As described above, the distribution server computer 100 is capable of providing a virtual space of a virtual live venue based on the virtual space generation program 120 and avatar data of performer users and viewer users. The operations of the avatars of these performer users and viewer users are updated by virtual space data in which the state of the virtual space is described by the performer avatar information transmitted from the performer terminal 200 and the viewer avatar information transmitted from the viewer terminal 300. Thus, the operation of the performer user is reflected in the operation of the performer avatar 1, and the operation (operation) of the viewer user is reflected in the operation of the viewer avatar.

[0126] Note that although not shown in FIG. 10, in the performer terminal 200 and the viewer terminal 300, virtual space data of the virtual live venue is pre-distributed and stored from the distribution server computer 100 in connection with communication with the distribution server computer 100 for virtual participation in the virtual live performance. The virtual space data pre-stored in this way is sequentially updated by virtual space update data sequentially distributed from the distribution server computer 100 as described later, so that the latest state of the virtual live venue can be specified by the performer terminal 200 and the viewer terminal 300.

[0127] Specifically, the actions and expressions of the performer user in the studio described above are scanned at predetermined time intervals by the functions of body motion scanning and face motion scanning by the performer avatar control program described above, and performer avatar information is generated and transmitted to the distribution server computer 100.

[0128] On the other hand, since the viewer user permitted virtual participation in the third area uses a motion sensor in the same way as the performer, the viewer user is scanned at predetermined time intervals by the functions of body motion scanning and face motion scanning, and viewer avatar information is generated and transmitted to the distribution server computer 100.

[0129] In addition, the viewer user permitted virtual participation in the fourth area can perform various operations such as movement operations, viewpoint direction change operations, viewing situation (zoom) change operations, raising hand operations, clapping hand operations, and jumping operations by using the touch panel, keyboard, or controller of the viewer terminal 300, thereby causing the viewer avatar to perform corresponding actions. Among these operations, viewer avatar information is generated based on the operations related to the actions of the viewer avatar and transmitted to the distribution server computer 100.

[0130] Note that the viewer user virtually participating in the third area can also execute various operations such as movement, viewing situation (zoom) change operations, raising hand operations, clapping hand operations, and jumping operations by using a controller or the like. In this case, menu items corresponding to the respective actions are displayed, and the menu items corresponding to the respective actions can be selected by using a controller or the like.

[0131] In this way, in the viewer terminal 300, an avatar action operation process for receiving operations corresponding to the respective actions to be performed by the viewer avatar is executed. Note that the avatar action operation process also includes processes such as display and deletion of menu items corresponding to the above-described respective actions.

[0132] Based on the presenter avatar information transmitted from the presenter terminal 200 and the viewer avatar information transmitted from the viewer terminal 300 in this way, the distribution server computer 100 executes a virtual space update process for updating the virtual space data, thereby reflecting the actions (operations) of the presenter user and the viewer user on each avatar virtually participating in the virtual live venue.

[0133] Then, after executing a virtual space update data generation process for generating virtual space update data based on the virtual space data before the update and the virtual space data after the update, the generated virtual space update data is distributed to the presenter terminal 200 and the viewer terminal 300.

[0134] In the presenter terminal 200 and the viewer terminal 300 to which the virtual space update data has been distributed, as shown in FIG. 10, based on the distributed virtual space update data, the pre-stored virtual space data is updated, and a presenter viewpoint video generation process and a viewer viewpoint video generation process using the updated virtual space data are executed, so that a presenter viewpoint video and a viewer viewpoint video based on the updated virtual space data are generated, and the generated presenter viewpoint video and viewer viewpoint video are displayed on the presenter terminal 200 and the viewer terminal 300 (viewer viewpoint video output process).

[0135] In this way, in the first embodiment, the viewpoint videos of each viewer user who is virtually participating in the virtual live venue by the viewer avatar are generated and displayed on the viewer terminal 300 side of each viewer user, so that the processing load of the distribution server computer 100 is significantly increased by generating the viewer avatar viewpoint videos of these viewer users by the distribution server computer 100, and it is possible to avoid the situation where many viewer users cannot participate in the virtual live, and it is also possible to avoid the situation where distribution becomes difficult due to an increase in the processing load.

[0136] In the present Example 1, on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 13 virtually participating in the third region near the performer avatar 1 on the virtual stage G, as shown in FIG. 13(d), a viewpoint video is generated and displayed in which the substantially front face of the performer avatar 1, which is the viewpoint video seen from the viewer avatar 13 near the performer avatar 1, is largely displayed. On the other hand, in the fourth region far from the performer avatar 1 on the virtual stage G, for example, at the position behind the viewer avatar 13 where the viewer avatar 28 is virtually participating, on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 28, as shown in FIG. 13(e), a viewpoint video is displayed in which the substantially front face of the performer avatar 1 appears relatively small through the viewer avatar 13 and the viewer avatar 14 virtually participating in the third region, which is the viewpoint video seen from the viewer avatar 28 far from the performer avatar 1.

[0137] Further, on the display device 310 of the viewer terminal 300 of the viewer user of the viewer avatar 32 virtually participating at the end side position of the virtual stage G in the fourth region set in a U-shaped top view, as shown in FIG. 13(f), a viewpoint video is displayed in which the performer avatar 1 appears relatively small through the viewer avatar 14 virtually participating in the third region, which is the viewpoint video seen from the viewer avatar 32 at the diagonally forward position of the performer avatar 1.

[0138] Note that the display examples shown in FIG. 13 all illustrate the case where the viewer terminal 300 is a stationary computer (PC) and the display device 310 is a stationary display. However, for example, when the viewer terminal 300 is a smartphone P or a tablet, in the case of the viewpoint video of the virtual camera C1, it is displayed as shown in FIG. 14(a), and when the viewer terminal 300 is the viewer terminal 300 of the viewer avatar 13, it is displayed as shown in FIG. 14(b).

[0139] Next, the audio system will be described. In the distribution server computer 100, music data 113 is reproduced by the audio control program 124 based on the time schedule data (TS data) 112, and is transmitted to the performer terminal 200 as the reproduced music (data) (music reproduction process).

[0140] At the performer terminal 200, the music audio from the reproduced music (data) transmitted from the distribution server computer 100 is output to the performer user through the high - performance earphone (ear monitor) worn by the performer user (reproduced music output process). At the same time, as the performer user sings in accordance with the output reproduced music, the performer voice input from the sound - collecting microphone (voice input device 222) is digitized and transmitted to the distribution server computer 100 as the performer voice (data).

[0141] The distribution server computer 100 generates distribution audio (data) from the performer voice based on the performer voice (data) received from the performer terminal 200 and the reproduced music reproduced in the above - mentioned music reproduction process (distribution audio generation process), and distributes the generated distribution audio (data) to the viewer terminal 300 (generated audio distribution process).

[0142] At the viewer terminal 300, the distribution audio is output from the audio output device 322 such as an earphone or a speaker by the distribution audio (data) distributed from the distribution server computer 100, so that the viewer can listen to the live audio as if the performer avatar 1 is singing in a virtual live venue.

[0143] In addition, at the performer terminal 200, as described above, the performer voice (data) is transmitted to the distribution server computer 100. As shown in FIG. 10, by executing the monitor voice generation process and the monitor voice output process, the music audio of the reproduced music and the performer voice are output from the speaker arranged in the adjustment room, so that the auxiliary operator O and the like can confirm the singing situation of the performer user in the adjustment room.

[0144] <Viewer Perspective Video Control Process> Next, the viewer perspective video control process executed based on the viewer perspective video control program included in the viewer program in the viewer terminal 300 will be described with reference to FIG. 20.

[0145] In the viewer perspective video control process, first, it is determined whether or not the performer avatar 1 is in the flying production period of virtual flight as shown in FIG. 19 (step S1). Whether or not it is in the flying production period can be specified by the production state data indicating that it is in the flying production period from the distribution server computer 100. Incidentally, the distribution server computer 100 transmits the production state data indicating that it is in the flying production period to the viewer terminal 300 when it becomes the flying production period based on the time schedule data.

[0146] If it is in the flying production period (Y in step S1), the process proceeds to step S21. On the other hand, if it is not in the flying production period (N in step S1), it is determined whether or not the perspective-related operations including the movement operation of the perspective (including the change of the perspective by the movement operation or action operation of the avatar) and the zoom operation are disabled (step S2).

[0147] If the perspective-related operations are disabled (Y in step S2), the process proceeds to step S7. If the perspective-related operations are not disabled (N in step S2), it is determined whether or not there is an operation that changes the perspective such as the movement operation or action operation of the avatar (step S3). If there is an operation (Y in step S3), the viewer perspective is changed according to the operation. If there is no operation (N in step S3), the process proceeds to step S5 and it is determined whether or not there is a zoom operation (step S5).

[0148] If there is a zoom operation (Y in step S5), the video area displayed on the display device 310 is changed to the video area according to the operation. If there is no zoom operation (N in step S5), it is determined whether or not there is an action following setting (step S7).

[0149] This operation tracking setting can be arbitrarily set on the viewer terminal 300, for example, before or during the live broadcast, according to the type of device used by the viewer user and their proficiency in operations. Specifically, for example, when not accustomed to operating an operation input device 321 such as a touch panel, keyboard, game controller, etc., or not accustomed to changing the viewpoint by moving or operating the avatar, as will be described later, since the virtual participation position of the viewer's own avatar is far from the performer avatar 1 and is enlarged and displayed in a zoomed-in state, even a slight movement of the performer avatar 1 may cause the performer avatar 1 not to be properly displayed, etc. This setting is made when the viewpoint cannot be changed well in response to the movement or operation of the performer avatar 1.

[0150] If there is no operation tracking setting (N in step S7), the process proceeds to step S11. On the other hand, if there is an operation tracking setting (Y in step S7), it is further determined whether the operation tracking condition is satisfied (step S8).

[0151] In the first embodiment, the operation tracking condition is defined as a condition under which it is almost certain that the performer avatar 1 is not properly displayed on the display device 310. Specifically, when the head (or face) of the performer avatar 1 enters a predetermined range near the periphery of the display area of the display device 310, since there is a significantly high possibility that the head (or face) will move outside the display area and not be properly displayed due to the movement or operation of the performer avatar 1, it is determined that the operation tracking condition is satisfied.

[0152] As described above, in the first embodiment, by determining that the motion tracking condition is satisfied before the head (face) of the performer avatar 1 disappears from the display area, it is possible to significantly reduce the situation where the head (face) of the performer avatar 1 disappears from the display area. However, the present invention is not limited to this, and these motion tracking conditions may be based on the condition that the head (face) has moved outside the display area, or other conditions may be used as the motion tracking conditions. That is, as long as these motion tracking conditions can prevent the performer avatar 1 from being poorly displayed in the display area, any conditions can be used.

[0153] When the motion tracking condition is satisfied (Y in step S8), after invalidating the viewpoint-related operation (step S12), as the viewpoint video displayed in the display area of the display device 310, the viewpoint is automatically changed to the motion tracking viewpoint where the head (face) of the performer avatar 1 is within the non-satisfaction range surrounded by a predetermined range around the display area, and the viewpoint video seen from the motion tracking viewpoint is displayed in the display area of the display device 310 (step S13).

[0154] On the other hand, when the motion tracking condition is not satisfied (N in step S8), it is further determined whether the viewpoint-related operation is invalidated (step S9). If the viewpoint-related operation is invalidated (Y in step S9), the invalidated viewpoint-related operation is enabled (step S10) and then the process proceeds to step S11. If the viewpoint-related operation is not invalidated (N in step S9), the process proceeds to step S11 without passing through step S10.

[0155] In step 11, due to the motion tracking setting not being set or the motion tracking condition not being satisfied, the viewpoint video of the viewer avatar continues to be displayed in the display area of the display device 310 (step S11).

[0156] Also, as described above, when it becomes the flying production period in which the performer avatar 1 virtually flies during the live, it is determined as Y in step S1 and proceeds to step S21, and it is determined whether it is the start timing of the flying production period (step S21).

[0157] If it is the start timing of the flying production period (Y in step S21), instead of the viewer avatar's viewpoint video or the motion-following viewpoint video displayed in the display area of the display device 310 before the start of the flying production period, a flying viewpoint video distributed from the distribution server computer 100 is displayed (step S22), and then, the viewpoint-related operation is disabled (step S23), and the process returns to step S1.

[0158] In addition, in the present Example 1, during the flying production period, since the same flying viewpoint video is displayed on the viewer terminals 300 of all viewer users, rather than generating the flying viewpoint video on each viewer terminal 300 side like different viewer avatar viewpoint videos for individual viewer users, it is generated on the distribution server computer 100 side and distributed to the viewer terminals 300, so that even a viewer terminal 300 with low processing power can display the flying viewpoint video with a large drawing load of the viewpoint video well by involving the aerial movement of the viewpoint. However, the present invention is not limited to this, and these flying viewpoint videos may be generated on each viewer terminal 300 side in the same manner as the viewer avatar viewpoint videos.

[0159] On the other hand, if it is not the start timing of the flying production period (N in step S21), it is further determined whether it is the end timing of the flying production period (step S24).

[0160] Whether it is the end timing of the flying production period may be specified based on the data indicating the end included in the data of the flying viewpoint video transmitted from the distribution server computer 100, or may be specified by the above-described production state data different from the flying viewpoint video.

[0161] If it is not the end timing of the flying effect period (N in step S24), that is, if it is during the flying effect period, the process proceeds to step S26. After updating the flying perspective video, the process returns to step 1. On the other hand, if it is the end timing of the flying effect period (Y in step S24), after enabling the viewpoint-related operation (step S25), the process proceeds to step S11. Instead of the flying perspective video that was displayed in the display area of the display device 310 during the flying effect period, the viewpoint video of the viewer avatar is displayed in the display area of the display device 310, and then the process returns to step 1.

[0162] As described above, by executing the viewer viewpoint video control process shown in FIG. 20 in the viewer terminal 300 of the first embodiment, each viewpoint video displayed on the viewer terminal 300 will be described with reference to FIGS. 16 to 19.

[0163] FIG. 16 is a diagram showing the display content of the display device 310 when a zoom operation is performed in the viewer terminal 300 of the first embodiment. In the default state where no zoom operation is performed, as shown in FIG. 16(a), the viewpoint video of the video area centered on the upper body of the performer avatar 1 is displayed. In this default state, when the viewer user performs a zoom-in operation, for example, if the viewer terminal 300 is a smartphone P, an operation of touching the display of “+” (not shown), or if the viewer terminal 300 is a computer (PC), when the “+” key on the keyboard is operated, for example, as shown in FIG. 16(b), the viewpoint video of the video area centered on the head (face) of the performer avatar 1 is displayed.

[0164] On the other hand, when the viewer user performs a zoom-out operation, for example, if the viewer terminal 300 is a smartphone P, an operation of touching the display of “-” (not shown), or if the viewer terminal 300 is a computer (PC), when the “-” key on the keyboard is operated, for example, as shown in FIG. 16(c), the viewpoint video of the video area including the whole body of the performer avatar 1 is displayed.

[0165] Also, in FIG. 16, for the sake of easy understanding of the zoom function, the description was made using the perspective video of the virtual camera C1 instead of the perspective of the viewer avatar. However, as shown in FIG. 17, zooming in and zooming out can also be performed in the same way for the perspective of the viewer avatar.

[0166] Further, in the viewer perspective video control process of FIG. 20, although not described in detail, for example, when the live broadcast starts and the performer avatar 1 first appears on the virtual stage G, since the perspective of the viewer avatar is not directed at the performer avatar 1, there is a possibility that the period during which the appearing performer avatar 1 is not displayed on the viewer terminal 300 becomes long. Thus, the period when the performer avatar 1 first appears, etc. is set as the appearance performance period, etc. During these performance periods, the perspective video displayed on the viewer terminal 300 is automatically switched to the perspective video of the virtual camera C1, etc. instead of the perspective video of the listener avatar, and zooming in, etc. may also be made possible for each viewer user to execute in these perspective videos of the virtual camera C1.

[0167] FIG. 17 shows, as an example of the perspective video of the viewer avatar, the perspective video of the viewer avatar 28 who is virtually participating in the fourth area. As shown in FIG. 17, the perspective video of the viewer avatar 28 includes the heads of the viewer avatars 13 and 14 who are virtually participating in the third area, and it is a video of seeing the performer avatar 1 over the heads of these viewer avatars 13 and 14.

[0168] Therefore, the viewer user of the viewer avatar 28 can, for example, perform a zoom-in operation and change to a video area where the upper body of the performer avatar 1 is centrally displayed as shown in FIG. 17(b), so that the viewer avatars 13 and 14 are not displayed and enjoy the live broadcast.

[0169] However, as shown in Fig. 17(b), in the zoomed-up state, for example, when the live event progresses and the performer avatar 1 moves or makes large movements, if the viewer user cannot successfully change the viewpoint of the viewer avatar 28 according to the movements of these performer avatars 1, as shown in Figs. 18(d) to (f), the performer avatar 1 may not be displayed properly. Therefore, by paying attention to the operation of the viewpoint, it becomes difficult to concentrate on the live event, and there may be a situation where the live event cannot be enjoyed well.

[0170] However, in the first embodiment, as described above, when the viewer user sets the motion tracking, as shown in Figs. 18(a) to (c), when the head of the performer avatar 1 enters a predetermined area around the display area, the motion tracking condition is satisfied and the viewpoint is automatically changed to the motion tracking viewpoint, and the viewpoint video of the motion tracking viewpoint is displayed. Therefore, the viewer user does not need to pay attention to the operation of the viewpoint, and can concentrate on the live event and enjoy the live event well.

[0171] Then, when the live event progresses to the flying performance period, as shown in Fig. 19, the viewpoint video of each viewer avatar is automatically changed to the flying viewpoint video seen from the aerial movement virtual camera without the viewer user operating.

[0172] Specifically, before the performer avatar 1 makes a virtual flight, a video in which the object of the wings becomes larger on the back of the performer avatar 1 is displayed. Then, when the performer avatar 1 looks up at the virtual aerial object 50 of the star virtually arranged in the second area of the virtual live venue, the viewpoint of the viewer avatar also moves similarly. At this time, a light effect 61 appears around the performer avatar 1.

[0173] After that, when the performer avatar 1 moves to virtual flight, the viewpoint video displayed on the display device 310 automatically switches from the viewpoint video of the viewer avatar until Fig. 19(b) to the flying viewpoint video seen from the aerial movement virtual camera as shown in Fig. 19(c).

[0174] During the flying performance period, as described above, the virtual camera in the air moves within the second region based on a preset air trajectory, and the flying perspective video seen from these virtual cameras moving in the air is displayed on the display device 310 as shown in FIG. 19(d).

[0175] <Modification Example of Embodiment 1> In the above-described Embodiment 1, the general operation (the first operation) is an operation in which the performer avatar 1 stands without moving to the virtual stage G, the special operation (the second operation) is an operation in which the performer avatar 1 performs virtual flight, and the specific operation (the third operation) is an operation in which the performer avatar 1 makes a sudden lateral movement. However, the present invention is not limited to this, and may be an operation according to the content to be distributed. For example, the general operation (the first operation) may be an operation (normal operation) that the performer avatar executes for the longest period in the content to be distributed. The special operation (the second operation) may be an operation executed during a special period that is the shortest period or an operation with the lowest frequency in the content to be distributed. The specific operation (the third operation) may be an operation with a shorter execution period than the general operation (the first operation) but a longer execution period than the special operation, or an operation with a lower execution frequency than the general operation (the first operation) but a higher execution frequency than the special operation. Note that the special operation (the second operation) may include the specific operation (the third operation).

[0176] Also, in the above-described Embodiment 1, an example is shown in which the virtual space providing system has both a switching function to a viewpoint corresponding to the special operation (the second operation) and a following function that follows the specific operation (the third operation). However, the present invention is not limited to this, and a virtual space providing system having only one of these functions may be sufficient.

[0177] In the above-described Example 1, a form in which the viewpoint of the viewer user is changed and followed according to the actions of the performer avatar was exemplified. However, the present invention is not limited to this. For example, the change and follow-up of these viewpoints may be executed according to specific effects such as the effects of smoke or fireworks (fire pillars), special lighting, and the deformation of the virtual stage G, which are effects in the virtual stage G.

Example

[0178] Next, the virtual space providing system of Example 2 will be described below with reference to FIGS. 21 to 28. Note that the same components as those in Example 1 are denoted by the same reference numerals and the description thereof is omitted.

[0179] The configuration of the virtual space providing system of Example 2 is the same as the configuration of Example 1 shown in FIG. 1, but the content of the progress of the virtual live is different, so the period during which the viewpoint videos from the virtual cameras C1 to C4 and the airborne virtual camera (hereinafter collectively referred to as the virtual camera viewpoint videos) are distributed, and the entity that generates these virtual camera viewpoint videos are different from those in Example 1, which are the characteristic parts of this Example 2.

[0180] FIG. 21 is a diagram showing the progress content of the virtual live distributed in the virtual space providing system of Example 2. In the virtual live of Example 2, first, as the period before the start of the virtual live, a "period during which participation in the live virtual space is possible" and a "pre-performance explanation" period are provided, and the viewer user can virtually participate in the virtual live venue, which is the virtual space targeted by the virtual live, during the period during which participation in the live virtual space is possible.

[0181] Note that the period up to the pre-data DL completion check timing set at the end of the "pre-performance explanation" period is regarded as the pre-data download (DL) period, and virtual space data (basic data) of the virtual live venue, avatar data of the performer avatar 1, and avatar data of other viewer users who have virtually participated are downloaded (DL) from the distribution server computer 100 to the viewer terminals 300 of the viewer users who have virtually participated during the period during which participation in the live virtual space is possible as pre-data.

[0182] Also, at the pre-data DL completion check timing, which is the end timing of the pre-data download (DL) period, it is checked whether the download (DL) of the pre-data has been completed at each viewer terminal 300.

[0183] In addition, if the download (DL) of the pre-data has not been completed by the pre-data DL completion check timing due to differences in the models, processing capabilities, communication environments, etc. of the viewer terminals 300, the download (DL) of the pre-data is continued. However, in this case, data that is highly important in the distribution of the virtual live, for example, virtual space data (basic data) of the virtual live venue required for the opening performance or the first MC, avatar data of the performer avatar 1, etc. are preferentially downloaded (DL).

[0184] Also, in the second embodiment, as described above, an example is shown in which data required to generate a viewpoint video at the viewer terminal 300 in the distribution of the virtual live is downloaded (DL) as pre-data. However, the present invention is not limited to this, and for these data, particularly special data that is individually required in each act and each MC described later, etc., at the timing corresponding to the timing when each of these acts and each MC are distributed, they may be sequentially downloaded (DL) each time.

[0185] In addition, during the period of the "pre-opening explanation", explanations about the virtual live are introduced by video images or the like. Therefore, by making the period during which the pre-data can be downloaded (DL) also during these "pre-opening explanation" periods, the avatar data of the viewer avatar who virtually participated near the end of the virtual participation period in the live virtual space can also be downloaded (DL) by each viewer terminal 300 from the distribution server computer 100.

[0186] Also, as shown in FIG. 21, the virtual live of Example 2 proceeds as follows: "Opening performance" -> "First MC" -> "First act" -> "Second MC" -> "Second act".... The "Opening performance" is a performance in which the performer avatar 1 appears on the virtual stage G. The "First MC" is a performance in which the performer avatar 1 who has appeared on the virtual stage G greets the viewers by talking and introduces the first act to be performed first. The "First act" includes the singing and dancing of the performer avatar 1 (performer), and includes a flying performance using a video obtained by virtually imaging the performer avatar 1 flying in the virtual live venue from an aerial movement virtual camera in the middle. The "Second MC" is a performance in which the performer avatar 1 (performer) mainly introduces the second act by talking. The "Second act" includes the singing and dancing of the performer avatar 1 (performer), and includes a costume change performance using a video obtained by virtually imaging the situation where the performer avatar 1 changes costumes by the virtual camera C1 and a splitting performance using a video obtained by virtually imaging the situation where the performer avatar 1 splits into multiple people by the virtual camera C1 in the middle.

[0187] Regarding the processing load of the distribution server computer 100 in each period when these various performances and acts are executed, the "lowest" is during the period when the virtual live space can be participated in, while the "highest" is during the periods when the special performances "flying performance", "costume change performance", and "splitting performance" are executed. The processing load of the distribution server computer 100 is "medium" during the period from the "pre-opening explanation" to the "First act" including the "Opening performance", "First MC", and "flying performance". The processing load is also "medium" during the period of the "First act" after the "flying performance", the period of the "Second MC", and the period of the "Second act" until the "costume change performance". The processing load is also "medium" during the period of the "Second act" from the "costume change performance" to the "splitting performance", and the processing load is also "medium" during the period of the "Second act" after the "splitting performance".

[0188] Next, regarding each period in the virtual live of Example 2, the viewpoint videos viewable on the viewer terminal 300 will be described. During the period when the virtual live space can be participated in from virtual participation until the "pre-opening explanation" starts, only the viewer avatar viewpoint video is displayable.

[0189] And at the timing when the "pre-show announcement" starts, only the virtual camera viewpoint video, specifically, the viewpoint video of the virtual camera C1, can be displayed. In this way, even though it is before the start of the virtual live, setting the viewpoint video in the "pre-show announcement" to only the virtual camera viewpoint video is because these pre-show announcements also include explanations of precautions regarding the virtual live. By displaying the viewer avatar viewpoint video on the viewer terminal 300, it aims to prevent the occurrence of viewer users who do not watch the pre-show announcement being conducted on the virtual stage G.

[0190] In addition, as will be described later, the virtual camera viewpoint videos of these "pre-show announcements" are generated by the distribution server computer 100 and distributed to each viewer terminal 300, so they can be displayed even on viewer terminals 300 where the download of the pre-data has not been completed.

[0191] For viewer terminals 300 where the download of the pre-data has been completed at the above-mentioned pre-data DL completion check timing, during the period of the "first act" from "the first MC" to the "flying performance", and during the period after the "flying performance", in addition to the virtual camera viewpoint video, the viewer avatar viewpoint video can be displayed.

[0192] On the other hand, for viewer terminals 300 where the download of the pre-data has not been completed at the above-mentioned pre-data DL completion check timing, since the viewer avatar viewpoint video cannot be accurately generated, when the virtual camera viewpoint video generated by the distribution server computer 100 is distributed, only the virtual camera viewpoint video can be displayed during all periods of the virtual live, and the viewer avatar viewpoint video cannot be displayed.

[0193] Also, among the videos of "Flying Performance", "Costume Change Performance", and "Doppelganger Performance" where the processing load of the distribution server computer 100 is "high", the virtual camera viewpoint videos of the virtual camera C1 for "Costume Change Performance" and "Doppelganger Performance" are generated not on the distribution server computer 100 but on each viewer terminal 300, similar to the virtual camera viewpoint videos during the second act period other than these "Costume Change Performance" and "Doppelganger Performance", so as to reduce the processing load of the distribution server computer 100 for generating the virtual camera viewpoint videos of the virtual camera C1 for these "Costume Change Performance" and "Doppelganger Performance".

[0194] In addition, regarding the period of the "Flying Performance", since it is a period with a high processing load on the distribution server computer 100, it is conceivable to generate it on the viewer terminal 300 in the same way as the "Costume Change Performance" and "Doppelganger Performance". However, in this case, since it is necessary to pre-distribute detailed trajectory data such as the movement trajectory of the airborne moving virtual camera moving within the virtual live venue to each viewer terminal 300 to generate the video, in the second embodiment, the distribution server computer 100 generates and distributes the video of the "Flying Performance". However, the present invention is not limited to this, and for the video of the "Flying Performance", it may also be generated on each viewer terminal 300 for the purpose of reducing the processing load of the distribution server computer 100.

[0195] Here, the participant user table used in the distribution server computer 100 of the second embodiment will be briefly described with reference to FIG. 23.

[0196] The participant user table is a data table used to identify the participation status of viewer users in virtual live events and the download status of pre-data, etc., in individual virtual live events registered in the event data shown in Fig. 7. As shown in Fig. 23, it is a table capable of storing data such as participation identification data (participation identification flag) that can identify the participation status of the participant (viewer user) in association with the participant ID (viewer user ID), the participation area of the participant (viewer user), and pre-data DL identification data (pre-data DL identification flag) that can identify the download status of pre-data.

[0197] In addition, in the item of the participant ID, all the viewer user IDs of the participants scheduled to watch described in the files of each participant group stored corresponding to the virtual live in the event data are registered and used.

[0198] The participation identification data (participation identification flag) has "0" corresponding to the non-participation status registered at the time of registration of the participant ID (viewer user ID), and is updated to "1" indicating participation when the participation of each participant (viewer user) is permitted by authentication. Therefore, based on this participation identification data (participation identification flag), it is possible to identify the viewer users who have participated and the viewer users who could not participate for some reason among the viewer users scheduled to participate.

[0199] The data of the participation area is data that can identify whether the participation area of each participant (viewer user) is the third area or the fourth area, and is the data registered when the participation of each participant (viewer user) is permitted by authentication.

[0200] The pre-data DL identification data (pre-data DL identification flag) is updated from "0" indicating incompletion to "1" indicating completion based on the check of whether the download (DL) of the pre-data has been completed at the pre-data DL completion check timing described above. Therefore, by these pre-data DL identification data (pre-data DL identification flags), it is possible to easily identify all the participants (viewer users) among the virtual participants for whom the download (DL) of the pre-data has not been completed.

[0201] Next, the process of downloading pre-data when viewer users virtually participate in the virtual live venue in Example 2 will be described with reference to FIG. 22.

[0202] First, the viewer user executes an operation to participate in a virtual live that the user plans to virtually participate in, for example, selects a participation menu or operates a specific key to which the participation operation is assigned on the keyboard, in the viewer program (viewing app) running on the viewer terminal 300.

[0203] By executing this participation operation, a participation request including the viewer user ID and authentication data for authenticating the viewer user is sent from the viewer terminal 300 to the distribution server computer 100.

[0204] In the distribution server computer 100 that has received the participation request, the participant authentication process is executed on the condition that it is within the live virtual space participation period. In the participant authentication process, based on whether the viewer user ID included in the received participation request is registered as a participant ID in the participant user database corresponding to the virtual live event in question, it is determined whether the viewer user is a scheduled participant for the corresponding virtual live event. Also, in the viewer user data shown in FIG. 9, the authentication data described in the authentication information file specified from the authentication information file data stored in association with the viewer user ID is compared with the authentication data included in the participation request to authenticate the viewer user.

[0205] If the viewer user is a scheduled participant and the authentication using the authentication data is completed, the participation identification data (participation identification flag) in the participant user table is updated to "1" corresponding to "participation".

[0206] Then, the distribution server computer 100 transmits (returns) pre-data DL instruction information to the viewer terminal 300 that is the source of the participation request in order to cause it to download the pre-data corresponding to the virtual live event in question. The viewer terminal 300 can confirm that participation is permitted by receiving the pre-data DL instruction information.

[0207] In response to receiving the pre-data DL instruction information, the viewer terminal 300 identifies the pre-data to be downloaded and transmits a pre-data DL request to the distribution server computer 100 requesting the download of the identified pre-data.

[0208] In response to receiving this pre-data DL request, the distribution server computer 100 executes the pre-data transmission process.

[0209] In this pre-data transmission process, the pre-data requested for the viewer terminal 300 that is the source of the pre-data DL request is transmitted. Note that, as described above, the pre-data includes virtual space data (basic data) of the virtual live venue, avatar data of the performer avatar 1, and avatar data of other viewer users who have already virtually participated, etc.

[0210] Note that, in the pre-data transmission process, for the viewer terminal 300 of a viewer user other than the source of the pre-data DL request and whose participation has already been confirmed by the participant data table, the avatar data of the viewer user who is the source of the pre-data DL request is additionally distributed as pre-data. As a result, on the viewer terminal 300 of each viewer user who has virtually participated in the virtual live, in addition to the virtual space data (basic data) and the avatar data of the performer avatar, the avatar data of all viewer users who have virtually participated in the virtual live is pre-distributed and stored in advance.

[0211] Then, the pre-data stored in the viewer terminal 300 in this way is checked to see whether the download of all the pre-data is completed at the above-mentioned pre-data DL completion check timing (see Fig. 21). When the download of the pre-data is completed, the pre-data DL identification data (pre-data DL identification flag) in the participant user table is updated from "0" indicating incomplete to "1" indicating completion.

[0212] Note that, for a participant (viewer user) whose pre-data DL identification data (pre-data DL identification flag) remains "0" indicating incomplete because the download of the pre-data is not completed, only the virtual camera viewpoint video is made displayable as described above.

[0213] Next, the situation where the entity that generates the various viewpoint videos displayed on the viewer terminal 300 changes in each period of the virtual live will be described with reference to Figs. 24 to 27.

[0214] First, during the period when only the viewer avatar viewpoint video can be displayed on the viewer terminal 300 and the live virtual space can be participated in, as shown in FIG. 24, based on the virtual space update data including the actor avatar operation data for generating the operations of the actor avatar 1 that performs operations linked to the operations of the actor user and is transmitted from the distribution server computer 100, the virtual space data already stored in the viewer terminal 300 by receiving the pre-data is sequentially updated. Based on the updated virtual space data, the viewer viewpoint video generation process is executed on the viewer terminal 300 to generate the viewer avatar viewpoint video, which is output (displayed) by the video output process. In FIG. 24, the actor terminal 200 is omitted, but the virtual space update data is also transmitted to the actor terminal 200 as shown in FIG. 10 in the first embodiment.

[0215] Also, for each of the periods of "pre-opening explanation", "opening performance", and "flying performance" during which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, as shown in FIG. 25, in the distribution server computer 100, in the virtual space update data distribution process, the virtual space update data is transmitted only to the actor terminal 200. Then, following the virtual space update data distribution process, a virtual camera viewpoint video generation process for generating the virtual camera viewpoint video and a virtual camera viewpoint video data distribution process for distributing the video data of the virtual camera viewpoint video generated in the virtual camera viewpoint video generation process to all viewer terminals 300 participating in the virtual live are executed.

[0216] The virtual camera viewpoint video data distributed from the distribution server computer 100 in these virtual camera viewpoint video data distribution processes is displayed by executing the video output process on the viewer terminal 300 that has received the virtual camera viewpoint video data.

[0217] That is, as shown in FIG. 27, for each of the periods of "pre-opening explanation", "opening performance", and "flying performance" during which only the virtual camera viewpoint video can be displayed, the virtual camera viewpoint video is generated in the distribution server computer 100.

[0218] Also, during the program period except for the MC period and the flying effect period, which is a period when either the viewer avatar viewpoint video or the virtual camera viewpoint video can be selected and displayed on the viewer terminal 300, as shown in FIG. 26, by executing the virtual space update data distribution process in the distribution server computer 100, virtual space update data is distributed to the viewer terminals 300 of each viewer user participating in the virtual live, and thus the virtual space update process is executed in the viewer terminal 300.

[0219] Then, in the viewer terminal 300, it is determined whether the viewer avatar viewpoint video is selected as the video to be displayed. If the viewer avatar viewpoint video is selected, similar to the flow shown in FIG. 24, the viewer viewpoint video generation process is executed to generate the viewer avatar viewpoint video, and the generated viewer avatar viewpoint video is displayed by the video output process.

[0220] On the other hand, when the viewer avatar viewpoint video is not selected as the video to be displayed, that is, when the virtual camera viewpoint video is selected as the video to be displayed, the virtual camera viewpoint video generation process is executed in the viewer terminal 300 to generate the virtual camera viewpoint video, and the generated virtual camera viewpoint video is displayed by the video output process.

[0221] Note that for the virtual camera viewpoint video generated in this virtual camera viewpoint video, the viewer user may be able to select whether it is the viewpoint video of any of the virtual cameras C1, C3, and C4, or the distribution server computer 100 may instruct all the viewer terminals 300.

[0222] That is, as shown in FIG. 27, during the performance period excluding the MC period and the flying effect period in which either the virtual camera viewpoint video or the viewer avatar viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated on the viewer terminal 300. In this way, for the period in which either the virtual camera viewpoint video or the viewer avatar viewpoint video can be selected and displayed on the viewer terminal 300, since the mutual switching between the virtual camera viewpoint video and the viewer avatar viewpoint video is executed according to the selection of the viewer user, compared with the case of displaying the virtual camera viewpoint video data by distribution from the distribution server computer 100, it is possible to execute the switching of these videos without causing problems such as the video being interrupted, and while preventing an increase in the processing load of the distribution server computer 100 associated with the switching of these videos.

[0223] On the other hand, for each of the periods of "pre-performance explanation", "opening performance", and "flying effect", which are periods in which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, the virtual camera viewpoint video is generated only on the distribution server computer 100 and distributed to each viewer terminal 300. In this way, during the period in which only the virtual camera viewpoint video can be displayed on the viewer terminal 300, since the virtual camera viewpoint video is generated only on the distribution server computer 100, it is possible to prevent the processing load of the entire system from increasing uselessly due to the processing of generating the virtual camera viewpoint video being duplicated on the viewer terminal 300.

[0224] Here, the situation when the viewpoint video is switched on the viewer terminal 300 will be described with reference to FIG. 28, taking the "flying effect" period, which is a period in which only the virtual camera viewpoint video can be displayed, as an example.

[0225] For example, as shown in FIG. 28, in the viewer terminal 300 where the viewer avatar viewpoint video is being displayed during the period of "First MC" and the period of the first act up to the "Flying Effect", when the period of the "Flying Effect" during which only the virtual camera viewpoint video (viewpoint video from the airborne virtual camera) can be displayed starts, if the distribution server computer 100 starts distributing the virtual camera viewpoint video (viewpoint video from the airborne virtual camera), there is a risk that the virtual camera viewpoint video (viewpoint video from the airborne virtual camera) will not be displayed on the viewer terminal 300 during the period from the start of this distribution until the virtual camera viewpoint video is displayed, resulting in the problem that the viewer cannot view the first video of the "Flying Effect".

[0226] Therefore, as shown in FIG. 28, the distribution server computer 100 of the second embodiment notifies the viewer terminal 300 of the distribution start (buffering start) timing at which the pre-distribution of the virtual camera viewpoint video (viewpoint video from the airborne virtual camera) starts before the start of the "Flying Effect" by transmitting a BF prior notice to the viewer terminal 300 at a specific timing before the start timing of the "Flying Effect" when the viewpoint video is forcibly changed from the viewer avatar viewpoint video to the virtual camera viewpoint video (viewpoint video from the airborne virtual camera).

[0227] In response to receiving the BF prior notice, the viewer terminal 300 specifies the timing at which the distribution of the virtual camera viewpoint video (viewpoint video from the airborne virtual camera) starts from the distribution server computer 100, and starts buffering the received virtual camera viewpoint video (viewpoint video from the airborne virtual camera) when that timing is reached.

[0228] The timing at which the distribution of these virtual camera viewpoint videos (viewpoint videos from the airborne virtual camera) starts, that is, the timing at which buffering starts, may be appropriately determined in consideration of the differences in the video storage functions of the viewer terminal 300. For example, it may be a period corresponding to several tens of frames as the number of video frames.

[0229] Thus, the distribution server computer 100 of Example 2 starts generating and distributing the virtual camera viewpoint video (viewpoint video from the virtual camera moving in the air) of the flying effect from before the start of the flying effect period. Since the viewer terminal 300 buffers the distributed virtual camera viewpoint video, when the flying effect period starts, the video can be instantaneously changed from the viewer avatar viewpoint video displayed before the start of the flying effect period to the virtual camera viewpoint video, thereby preventing the problem that the viewer cannot view the first video of the "flying effect".

[0230] Also, when the "flying effect" period ends, since the viewer avatar viewpoint video was selected before the start of the "flying effect", the viewpoint video switches from the virtual camera viewpoint video of the flying effect (viewpoint video from the virtual camera moving in the air) to the viewer avatar viewpoint video in response to the end of the "flying effect" period. However, even in this case, if the viewer avatar viewpoint video is generated and displayed starting from when the "flying effect" period ends, there is a risk of a problem that the viewer avatar viewpoint video is not displayed on the viewer terminal 300 during the period until the viewer avatar viewpoint video is generated and displayed, similar to when the "flying effect" starts.

[0231] Therefore, even when the "flying effect" period ends, the distribution server computer 100 of Example 2 notifies the viewer terminal 300 of the end timing of the "flying effect" period when the distribution of the virtual camera viewpoint video (viewpoint video from the virtual camera moving in the air) ends by sending a pre-notification of the end TM (timing) to the viewer terminal 3 as shown in FIG. 28.

[0232] In response to receiving a pre-notification of the end TM (timing), the viewer terminal 300 identifies in advance the timing at which the distribution of the virtual camera viewpoint video (viewpoint video from the airborne virtual camera) stops from the distribution server computer 100, and starts the viewer viewpoint video generation process from a predetermined timing before that timing to pre-generate the viewer avatar viewpoint video.

[0233] In addition, from the time of sending the pre-notification of the end TM (timing), the distribution server computer 100 executes the distribution of the virtual space update data, so the viewer terminal 300 executes the virtual space process of updating the virtual space data with the received virtual space update data. Therefore, at the above-mentioned predetermined timing, a viewer avatar viewpoint video reflecting the situation of the virtual live venue at that time can be generated.

[0234] In this way, the distribution server computer 100 of the second embodiment sends a pre-notification of the end TM (timing) to the viewer terminal 300 before the end of the flying effect period, so that the viewer terminal 300 can pre-generate the viewer avatar viewpoint video before the end of the flying effect period. Therefore, it is possible to prevent the problem that the viewer avatar viewpoint video is not displayed on the viewer terminal 300.

[0235] In addition, in FIG. 28, the "flying effect" is taken as an example for explanation, but the same control is also performed for the periods of the "pre-opening explanation" and the "opening effect" in which the viewpoint video is forcibly changed from the viewer avatar viewpoint video to the virtual camera viewpoint video.

[0236] In the virtual space providing system of the second embodiment described above, there are a plurality of viewer users, and a viewer terminal 300 is assigned to each viewer user. When the viewer terminal 300 displays a viewer avatar viewpoint video, which is a viewpoint video of a virtual live venue (virtual space) corresponding to the viewer avatar, the viewer avatar viewpoint video is generated based on virtual space update data (virtual space data) transmitted from the distribution server computer 100. Since the generation of the viewer avatar viewpoint video, which involves a large processing load, is performed by the viewer terminal 300, more viewer users can participate in the virtual live event.

[0237] In addition, the virtual space content of the second embodiment is content that allows a performer user to participate in the virtual live venue as the performer avatar 1, and the performer avatar 1 performs musical operations (performances) such as singing and dancing in the virtual live venue. Therefore, the interest of the virtual space content can be improved.

[0238] In addition, since the virtual space update data (virtual space data) of the second embodiment includes action data for the performer avatar for generating the actions of the performer avatar 1, the actions of the performer avatar 1 can also be accurately reflected in the viewer avatar viewpoint video generated on the viewer terminal 300.

[0239] Regarding the common viewpoint videos that are commonly displayed on each viewer terminal 300 in the second embodiment, such as the videos of "pre-opening explanation", "opening performance", and "flying performance", they are generated on the distribution server computer 100 and distributed to each viewer terminal 300. Therefore, it is possible to prevent the processing load of the entire system from increasing unnecessarily due to duplicate processing being executed on each viewer terminal 300.

[0240] Also, in the viewer terminal 300 of Example 2, during the performance period (specific period) other than the "MC period" and the period of the "flying effect", which is the period during which either the viewer avatar viewpoint video or the virtual camera viewpoint video can be selected and displayed, since the virtual camera viewpoint video is generated and displayed on the viewer terminal 300, it is possible to prevent problems such as the video being interrupted when the viewpoint video is switched.

[0241] Also, during the periods of the "costume change effect" and the "duplication effect", which are special periods with a large processing load on the distribution server computer 100 in Example 2, since each viewer terminal 300 also generates and displays the virtual camera viewpoint video (common viewpoint video), it is possible to prevent the processing load on the distribution server computer 100 from temporarily becoming extremely large.

[0242] Also, since the distribution server computer 100 in Example 2 pre-transmits the preliminary data including the basic data of the virtual live venue (virtual space) and the basic data of the performer avatar 1 and the viewer avatar who virtually participate in the virtual live venue to each viewer terminal 300 before the virtual live starts, it is possible to prevent the occurrence of a problem that the viewer avatar viewpoint video is not displayed properly at the start of the virtual live.

[0243] Also, the distribution server computer 100 in Example 2 transmits the virtual camera viewpoint video (common viewpoint video) to the viewer terminal 300 for which the completion of reception of the preliminary data cannot be confirmed. By doing so, while preventing the processing load on the distribution server computer 100 from excessively increasing due to the transmission of the basic data and the like after the start of the virtual live, it is also possible to prevent the viewer user from being unable to view the virtual live.

[0244] <Modification Example of Example 2> FIG. 29 is a flowchart showing the processing content of the viewer viewpoint video generation process in Modification Example 2-1 for Example 2.

[0245] In the above-described Example 2, the form of always uniformly generating the viewer avatar perspective video including the operation of the performer avatar 1 was shown. However, the present invention is not limited to this, and these viewer avatar perspective videos may be generated in a plurality of modes with different processing loads in video generation according to the selection of the viewer user on the viewer terminal 300.

[0246] That is, as an example shown in Example 1, the viewer terminal 300 used by the viewer user may be a stationary personal computer (PC) with high processing power, or may be a smartphone terminal or a tablet terminal with relatively lower processing power than a personal computer (PC). It is assumed that terminals with different processing powers are used. However, if the viewer avatar perspective video is always uniformly generated on such terminals with different processing powers, on a terminal with low processing power, the viewer avatar perspective video cannot be appropriately generated, and there is a risk that the viewer avatar perspective video and the audio will be out of sync, or the viewer avatar perspective video will become a discontinuous and unnatural video. Therefore, for example, when generating a viewer avatar perspective video including the performer's movement, whether to generate a high-definition viewer avatar perspective video including a performer's movement with a high processing load related to video generation, or to generate a low-definition viewer avatar perspective video including a performer's movement with a low processing load related to video generation, may be appropriately selected by the viewer according to the processing power of the terminal being used.

[0247] Specifically, the viewer perspective video generation process executed on the viewer terminal 300 may be as shown in the flowchart of FIG. 29. In this viewer perspective video generation process, first, the viewer terminal 300 identifies the actor motion generation mode selected by the viewer user (step S101). Then, it determines whether the identified actor motion generation mode is a high processing load mode (step S102). If the high processing load mode is selected (Y in step S102), a viewer avatar perspective video including high-definition actor motions is generated (step S103). On the other hand, if the high processing load mode is not selected (N in step S102), a viewer avatar perspective video including low-definition actor motions is generated (step S104).

[0248] Note that in FIG. 29, a form with two modes, namely a high processing load mode and a low processing load mode, is exemplified. However, the present invention is not limited to this, and these processing load modes may be three or more modes.

[0249] Also, in the above-described Example 2 (the same applies to Example 1), a form is exemplified in which all operations including the movement of the viewer avatar of the viewer user are reflected in each viewer avatar by the distribution of virtual space update data. However, the present invention is not limited to this. For example, as operations of these viewer avatars, operations that are normally not executable but become possible when a special fee is paid, for example, when the viewer terminal 300 enables an operation of performing a special operation that can appeal the viewer's own avatar to the actor avatar 1. Since this special operation is an operation directed to the actor user and not to other viewer users, the virtual space update data including these special operations is not distributed to the viewer terminals 300 of other viewer users, and by enabling these special operations, it is possible to prevent the processing load of the distribution server computer 100 from increasing excessively.

[0250] Thus, when virtual space update data including special operations is not distributed to the viewer terminals 300 of other viewer users, as shown in Modification Example 2-2 of FIG. 30, when the viewer avatar of viewer user A performs a special operation by executing an operation corresponding to the special operation, virtual space update data A corresponding to the fact that the viewer avatar of viewer user A is performing a special operation is transmitted from the distribution server computer 100 to the viewer terminal 300 of viewer user A. However, virtual space update data B indicating that the viewer avatar of viewer user A is not performing a special operation is distributed to the viewer terminal 300 of viewer user B, who is different from viewer user A.

[0251] Therefore, different virtual space update data will be distributed from the distribution server computer 100 to the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B. As a result, the degree of freedom of the virtual space update data can be increased, and different viewer avatar viewpoint videos can be generated on the viewer terminal 300 of viewer user A and the viewer terminal 300 of viewer user B.

[0252] In addition, based on virtual space update data A, which is different from virtual space update data B distributed to the viewer terminal 300 of viewer user B, the viewer terminal 300 of viewer user A can generate and display a video of the viewer avatar of viewer user A performing a special operation different from the possible operations of the viewer avatar of viewer user B. Therefore, the interest of viewer user A in the virtual live, which is virtual space content, can be improved.

[0253] Also, in the above-described Example 2 (the same applies to Example 1), regarding the viewer avatars of the viewer users virtually participating in the fourth region, although an example is shown in which the form of the avatar is a normal avatar that varies among individual viewer users, the present invention is not limited thereto. For example, for the purpose of reducing the processing load when generating various perspective videos in a virtual live, the viewer avatars of the viewer users virtually participating in the fourth region may be made into simple avatars with simplified avatar forms.

[0254] In addition, in FIG. 30, an example is shown of the case of executing a special operation that becomes possible when a special fee is paid. However, for example, as described above, in the case of using a simple avatar, for normal operations other than the movement of the simple avatar, such as waving the hand, tapping the hand, and changing the orientation of the viewer avatar, similar to the special operation, by not including them in the virtual space update data distributed to other viewer users, the processing load on the distribution server computer 100 can be further reduced.

[0255] In addition, in the above-described Example 1 and Example 2, an example is shown in which the video displayed on the viewer terminal 300 is automatically switched between the viewer perspective video and the virtual camera perspective video according to the time schedule data preset on the administrator side, such as when executing the flying effect shown in FIG. 19. However, the present invention is not limited thereto, and the video displayed on the viewer terminal 300 may be appropriately switched between the virtual camera perspective video and the viewer perspective video according to the operation by the viewer user. By doing so, the viewer user can view the virtual live from various perspectives by their own operation, and the interest of the virtual live can be improved.

[0256] Furthermore, as described above, even when the video displayed on the viewer terminal 300 can be switched between a virtual camera viewpoint video and a viewer viewpoint video by an operation by the viewer user, during the execution period of an effect that the viewer user wants to strongly impress, such as a flying effect (see FIG. 19), the operation of the viewer terminal 300 by the viewer user (the operation of switching the displayed video) may be disabled to prevent the viewer user from missing the effect.

Example

[0257] Next, the virtual space providing system of Example 3 will be described below with reference to FIGS. 31 to 56. For the same configurations as in Examples 1 and 2, the same reference numerals will be given and the description will be omitted.

[0258] As shown in FIG. 31, the virtual space providing system of Example 3 is provided with the same live progress content as in Example 2. In particular, in this Example 3, during the live performance, a performance by the avatars of two performers, Performer A and Performer B, is provided.

[0259] After the performer avatar 1A of the first performer, Performer A, appears from the timing immediately after the start of the first MC period, a solo performance by the performer avatar 1A of the performer A is executed as the first performance. After the performer avatar 1B of the second performer, Performer B, appears (the number of performer avatars increases from one to two) from the timing during the second MC period, a joint performance by the performer avatar 1A of the performer A and the performer avatar 1B of the performer B is executed as the second performance.

[0260] Also, in the third embodiment, it is possible to execute a light emission effect in which the virtual airborne objects 50 to 52 emit light as part of the virtual live performance. In particular, the light emission effect is not executed during the period when participation in the virtual live venue is possible and during the period of the pre-opening explanation (light emission intensity of the light emission effect: none). During the opening performance period, the first MC period, and the second MC period, the virtual airborne objects 50 to 52 emit light with a relatively weak light emission intensity (light emission intensity of the light emission effect: weak). During the first act period and the second act period, the virtual airborne objects 50 to 52 emit light with a stronger light emission intensity than during the opening performance period, the first MC period, and the second MC period (light emission intensity of the light emission effect: strong).

[0261] Also, as shown in FIG. 31, until immediately after the start of the first MC period, since the performer avatar 1A of performer A has not appeared, it is a period without the operation of the performer avatar 1A of performer A (operation of the performer avatar of performer A: none). During the first MC period and the second MC period after the performer avatar 1A of performer A appears, the operation of the performer avatar 1A of performer A is relatively small (operation of the performer avatar of performer A: small). During the first act period and the second act period, since the performer avatar 1A of performer A performs dance and singing, the operation of the performer avatar of performer A becomes larger (more intense) than in other periods (operation of the performer avatar of performer A: large).

[0262] Similarly, until the performer avatar 1B of performer B appears during the second MC period, it is a period without the operation of the performer avatar 1B of performer B (operation of the performer avatar of performer B: none). During the second MC period after the performer avatar 1B of performer B appears, the operation of the performer avatar 1B of performer B is relatively small (operation of the performer avatar of performer B: small). During the second act period, since the performer avatar of performer B performs dance and singing together with the performer avatar 1B of performer A, the operation of the performer avatar 1B of performer B becomes larger (more intense) than in other periods (operation of the performer avatar of performer B: large).

[0263] In addition, since the period during which virtual live venue participation is possible is the period in which multiple viewer terminals 300 sequentially connect to the distribution server computer 100, as time passes, viewer avatars that have virtually participated in the virtual live venue due to such connections are gradually added and arranged in the virtual space.

[0264] For this reason, as shown in FIGS. 31 and 32(a), immediately after the start of the period during which virtual live venue participation is possible, the light emission effect is not executed, the display of the performer avatars 1A and 1B of each performer is not present, and since the number of viewer avatars that have virtually participated in the virtual live venue is small, there are few viewer avatars that require updates of position information and the like in the distribution server computer 100, and the update processing load for virtual space updates is in a relatively small state. Also, for each viewer terminal 300, since the number of viewer avatars that have virtually participated in the virtual live venue is small as described above, the drawing processing load for drawing the viewer avatars is in a relatively low state.

[0265] When the number of viewer avatars that have virtually participated in the virtual live venue increases from this state, as shown in FIG. 32(b), since the viewer avatar information to be updated in the virtual space data in the distribution server computer 100 increases, the update processing load for virtual space updates by the distribution server computer 100 increases. Also, since the number of viewer avatars to be drawn in each viewer terminal 300 increases as the number of viewer avatars that have virtually participated in the virtual live venue increases, the drawing processing load for drawing the viewer avatars also increases.

[0266] In addition, when the number of viewer avatars that have virtually participated in the virtual live venue increases, as shown in FIGS. 31 and 33, the number of viewers who perform movement operations from the viewer terminals 300 also increases, so the number of viewer avatars moving within the virtual live venue may also increase.

[0267] Since the information such as the current position in the virtual live venue of a moving viewer avatar changes more frequently than that of a viewer avatar standing still at a specific location, when the number of viewer avatars moving within the virtual live venue increases as shown in FIGS. 31 and 33, the update processing load of virtual space update by the distribution server computer 100 increases. At this time, also in each viewer terminal 300, when the number of moving viewer avatars increases, the drawing amount due to the movement of each viewer avatar also increases, so the drawing processing load in the viewer terminal 300 also increases.

[0268] During each program period, when performing a light emission effect, as shown in FIGS. 31 and 34(a), in the distribution server computer 100, based on the current position of each avatar, the distance and angle from the light sources (virtual airborne objects 50 to 52 in this Example 3), etc., calculation processing of the way light hits and the generated shadows for each viewer avatar and performer avatar occurs, so the update load of virtual space update by the distribution server computer 100 increases. At this time, also in each viewer terminal 300, due to the light emission effect, drawing to make the virtual airborne objects 50 to 52 emit light, drawing to make the way light hits different for each viewer avatar, and drawing of the shadow for each viewer avatar are performed, so the drawing processing load also increases.

[0269] In addition, in this Example 3, there may be a case where the light emission intensity of the virtual airborne objects 50 to 52 is increased as the light emission effect (see FIG. 31). When the light emission intensity increases in this way, the processing load associated with the calculation processing of the way light hits and the generated shadows for each viewer avatar and performer avatar in the distribution server computer 100 described above further increases. As a result, the update load of virtual space update by the distribution server computer 100 further increases, and similarly, the drawing processing load in each viewer terminal 300 also further increases.

[0270] In addition, when the performer avatar 1A appears, as shown in FIGS. 31 and 34(b), since the performer avatar information is added to the information to be updated of the virtual space data in the distribution server computer 100, the update processing load of the virtual space update by the distribution server computer 100 increases. Further, when the performer avatar 1B appears, as shown in FIGS. 31 and 34(b), in addition to the performer avatar information of the performer avatar 1A, the performer avatar information of the performer avatar 1B is also added as the information to be updated of the virtual space data in the distribution server computer 100, so the update processing load of the virtual space update by the distribution server computer 100 further increases. In the viewer terminal 300, since the process of drawing the performer avatar 1A of the performer A starts from the appearance timing of the performer avatar 1A, the drawing processing load increases. Further, from the appearance timing of the performer avatar 1B, in addition to the process of drawing the performer avatar 1A, the process of drawing the performer avatar 1B of the performer B also starts, so the drawing processing load further increases. Note that in the third embodiment, after the performer avatar 1B of the performer B appears (after the number of performer avatars becomes two), since the number of performer avatars does not decrease, the process of drawing the performer avatar 1A and the process of drawing the performer avatar 1B continue until the end of the virtual live show.

[0271] During each act period, as shown in FIGS. 31 and 35, in addition to the increase in the emission intensity of the virtual airborne objects 50 to 52 as the light emission effects, each of the performer avatars 1A and 1B performs a performance such as singing or dancing. For this reason, as described above, the processing load associated with the calculation process of the way the light hits each viewer avatar and the shadow generated by the distribution server computer 100 for each viewer avatar and performer avatar further increases, so the update processing load of the virtual space update by the distribution server computer 100 further increases. Also, in each viewer terminal 300, the drawing processing load for drawing the way the light hits each viewer avatar and the shadow generated by emitting the virtual airborne objects 50 to 52 and for drawing the performer avatars 1A and 1B increases.

[0272] In the third embodiment, when the processing load on the distribution server computer 100 and the viewer terminal 300 increases due to an increase in the number of viewer avatars and performer avatars arranged in the virtual live venue, an increase in the emission intensity of the light emission effect, an increase in the number of operating viewer avatars and performer avatars, an increase in the amount of movement of the performer avatar, etc., as described above, the display mode of the viewer avatar is changed from a full avatar with a relatively large processing load for display to a simple avatar with a relatively small processing load for display, thereby suppressing an excessive increase in the processing load on the entire device constituting the virtual space providing system, such as the distribution server computer 100 and the viewer terminal 300.

[0273] <Time schedule data> As described above, the virtual live of the third embodiment proceeds as follows. As shown in FIG. 36, it is managed by the distribution server computer 100 based on time schedule data (TS data). The time schedule data is stored in a form included in the event management program 110 in the storage 103 of the distribution server computer 100.

[0274] The time schedule data is data in a table format in which the scheduled time of the timing, the elapsed time since the start of the live, the content of the timing, and the file name of the related data related to the timing are stored in association with the timing ID assigned in the order of individual timings in the virtual live.

[0275] As these timings, as shown in FIG. 36, in addition to the start and end timings of the opening production and the performer appearance production, and the timings at which the performer avatars 1A and 1B appear during the performer appearance production, the start and end timings of each transition period, and the timings for simplifying each viewer avatar (viewer avatar first simplification timing T1, viewer avatar second simplification timing T2, viewer avatar third simplification timing T3) are included. Hereinafter, in this embodiment, the viewer avatar first simplification timing T1, the viewer avatar second simplification timing T2, and the viewer avatar third simplification timing T3 may be simply referred to as timings T1, T2, and T3.

[0276] Timing T1 is the timing immediately before the start of the first act period when the light emission intensity of the light emission production increases and the operation of the performer avatar 1A becomes intense, increasing the drawing processing load on the viewer terminal 300. Timing T2 is the timing immediately before the start of the second MC period when the addition of the performer avatar 1B increases the drawing processing load on the viewer terminal 300. Timing T3 is the start timing of the second act period when the light emission intensity of the light emission production increases and the operations of the performer avatars 1A and 1B become intense, increasing the drawing processing load on the viewer terminal 300. These timings T1 to T3 are provided to change the viewer avatar from the full avatar to the simple avatar in advance at a stage where the processing load has not yet increased immediately before the drawing processing load on the viewer terminal 300 increases as described above.

[0277] <Video Generation During the Live Virtual Space Participable Period> Also, during the live virtual space participable period in which only the viewer avatar viewpoint video can be displayed on the viewer terminal 300 in the third embodiment, as shown in FIG. 37, similar to the second embodiment, after performing the avatar movement operation process on the viewer terminal 300, the virtual space update process and the virtual space update data generation process on the distribution server computer 100, the distribution server computer 100 further executes an avatar simplification determination process for determining whether to change the display of the viewer avatar from the full avatar to the simple avatar.

[0278] After the avatar simplification determination process is executed, the distribution server computer 100 executes virtual space update data distribution processing in the same manner as in the second embodiment, and thereby transmits the simplification determination value data, which is the result of the avatar simplification determination process, together with the virtual space update data to the viewer terminal 300.

[0279] The viewer terminal 300 executes virtual space update processing, viewer viewpoint video generation processing, and video output processing based on the reception of the virtual space update data and the simplification determination value data. In particular, in the third embodiment, although details will be described later, by using the received simplification determination value data, a process of drawing (generating) the viewer avatar as a full avatar or a simple avatar is performed in the viewer viewpoint video generation processing.

[0280] <Video Generation during the Performance Period Excluding the MC Period and the Flying Effect Period> Also, during the performance period excluding the MC period and the flying effect period in which the viewer avatar viewpoint video and the virtual camera viewpoint video can be displayed on the viewer terminal 300 in the third embodiment, as shown in FIG. 38, similar to the second embodiment, after the avatar movement operation processing is performed on the viewer terminal 300 and the virtual space update processing and the virtual space update data generation processing are performed on the distribution server computer 100, the distribution server computer 100 further performs an avatar simplification determination process for determining whether to change the display of the viewer avatar from a full avatar to a simple avatar. Note that this avatar simplification determination process is the same process as the avatar simplification determination process shown in FIG. 38.

[0281] After the avatar simplification determination process is executed, the distribution server computer 100 executes virtual space update data distribution processing in the same manner as in the second embodiment, and thereby transmits the simplification determination value data, which is the result of the avatar simplification determination process, together with the virtual space update data to the viewer terminal 300.

[0282] On the viewer terminal 300, based on the reception of the virtual space update data and the simplified determination value data, when the viewer avatar viewpoint video is selected as the video to be displayed in the virtual space update process, the viewer viewpoint video generation process is executed; when the virtual camera viewpoint video is selected as the video to be displayed, the virtual camera designated video generation process is executed. In particular, in the third embodiment described later, by using the received simplified determination value data, in the viewer viewpoint video generation process and the virtual camera designated video generation process, a process of drawing (generating) the viewer avatar as a full avatar or a simplified avatar is performed.

[0283] <Regarding friend registration> Also, in the virtual space providing system of the third embodiment, a viewer can register another viewer as their friend by operating the viewer terminal 300. Note that in the storage 303 of the viewer terminal 300 of the third embodiment, a friend user data table (see FIG. 40) that stores the user IDs and nicknames of the viewer users who have been pre-registered as friends is stored.

[0284] As shown in FIG. 39, on the viewer terminal 300, when the viewer avatar selection process for selecting a specific viewer avatar from a plurality of viewer avatars is executed by receiving an operation of the operation input device 321 of the viewer during the live virtual space participation period, the file name of the selected viewer avatar (viewer avatar file name) is specified, and the viewer avatar inquiry process for transmitting viewer avatar inquiry information including the specified viewer avatar file name to the distribution server computer 100 is executed.

[0285] When the distribution server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID associated with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in FIG. 9. Then, the distribution server computer 100 transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

[0286] When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process that prompts an operation to register the viewer (viewer user ID) of the selected viewer avatar as a friend in association with the viewer avatar file name for the viewer on the display device 310.

[0287] At this time, when the viewer performs an operation to register a friend, a friend addition process is executed to store the viewer user ID and the viewer avatar file name in association in the friend user data table. When the operation to register a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process that displays a message indicating that no friend registration is performed on the display device 310.

[0288] <Avatar Simplification Determination Process> FIG. 41 is a flowchart showing an example of the avatar simplification determination process shown in FIG. 38. In the avatar simplification determination process, the distribution server computer 100 first determines whether it is within the virtual live venue participation period (see FIG. 31) by referring to the time schedule data or the like (step S401). If it is not within the virtual live venue participation period (step S401; No), the process proceeds to step S407. If it is within the virtual live venue participation period (step S401; Yes), it further determines whether the participation number threshold reached flag is set (step S402). Note that the participation number threshold reached flag is a flag indicating that the number of viewer avatars (the number of viewers watching the virtual live) arranged in the virtual live venue has reached a predetermined threshold.

[0289] If the distribution server computer 100 determines that the participation number threshold reached flag is not set (step S402; Yes), the process proceeds to step S407. If the participation number threshold reached flag is set (step S402; No), it further refers to the participant user table shown in FIG. 23 to identify the number of participants in the live (the number of viewer avatars virtually participating in the virtual live venue) (step S403), and determines whether the identified number of participants in the live has reached the threshold (step S404).

[0290] Note that the participation number threshold is a value determined from the number of viewer avatars in full avatars that can be drawn at one time on each viewer terminal 300 and the number of viewer avatars in full avatars that can be arranged in the virtual live venue. For example, when the limit number of viewer avatars in full avatars that can be drawn at one time on each viewer terminal 300 and the limit number of viewer avatars in full avatars that can virtually participate in the virtual live venue is 200, the administrator of the distribution server computer 100 or the like may set it to 180, which is about 10% lower than the limit number. The participation number threshold is a value determined by the processing load corresponding to the number of viewer avatars that can be drawn on the viewer terminal 300, not simply the number of viewer avatars to be drawn.

[0291] Still, considering the performance (rendering processing ability) of each viewer terminal 300 and the like, an administrator or the like of the distribution server computer 100 may appropriately set a value different from the above in the administrator terminal 150 as the threshold value of the number of participants.

[0292] When the number of participants has not reached the threshold value (step S404; No), the process proceeds to step S407. When the number of participants in the live event that has reached the threshold value (step S404; Yes), based on the fact that the rendering processing load of the viewer avatar in full avatar is almost at the limit in the viewer terminal 300, the distribution server computer 100 adds 1 to the simplification determination value (step S405), sets the participation number threshold reached flag (step S406), and proceeds to step S407. Still, the simplification determination value is a value for specifying the range to be displayed with a simple avatar having a small number of polygons constituting each viewer avatar to be displayed in each viewer terminal 300. As an initial value, "0" is set.

[0293] In step S407, the distribution server computer 100 determines whether it is during the virtual live venue participation period or the MC period by referring to the time schedule data or the like (step S407). When it is neither during the virtual live venue participation period nor the MC period (step S407; No), the process proceeds to step S413. When it is during the virtual live venue participation period or the MC period (step S407; Yes), it is determined whether the moving number threshold reached flag is set (step S408). Still, the moving number threshold reached flag is a flag indicating that the number of viewer avatars moving in the virtual live venue in full avatar among the viewer avatars arranged in the virtual live venue has already reached a predetermined threshold value.

[0294] If the moving person count threshold reached flag is set (step S408; Yes), the process proceeds to step S413. If the moving person count threshold reached flag is not set (step S408; No), the distribution server computer 100 identifies the number of viewer avatars currently moving (the number of viewer terminals 300 on which a moving operation is being performed by each viewer) from the viewer avatar information received from each viewer terminal 300 (step S409).

[0295] In step S409, the distribution server computer 100 may, for example, store the position information of each viewer avatar in the virtual live venue every unit time (e.g., 10 ms), and compare the stored position information of each viewer avatar in the virtual live venue with the position information of each viewer avatar in the virtual live venue stored last time, and identify the number of viewer avatars moving by aggregating the number of viewer avatars for which position information different from the previous storage is stored.

[0296] Then, the distribution server computer 100 determines whether the identified number of viewer avatars moving (the number of viewer avatars currently moving) has reached the threshold (step S410). Note that the threshold for the number of viewer avatars moving in step S410 is a value determined from the number of viewer avatars moving with full avatars that can be drawn at one time on each viewer terminal 300. For example, when the limit number of viewer avatars moving with full avatars that can be drawn at one time on each viewer terminal 300 is 200, the administrator of the distribution server computer 100 or the like may set it to 150, which is 25% lower than the limit number. The threshold for the number of viewer avatars moving is a value determined by the processing load corresponding to the number of viewer avatars that can be drawn on the viewer terminal 300, not simply the number of viewer avatars to be drawn.

[0297] Note that the administrator of the distribution server computer 100 or the like may appropriately set a value different from the above in consideration of the performance (drawing processing ability) of each viewer terminal 300 or the like for the threshold of the number of viewer avatars moving.

[0298] If the number of moving viewer avatars has not reached the threshold (step S410; No), the process proceeds to S413. If the number of moving viewer avatars has reached the threshold (step S410; Yes), the distribution server computer 100 adds 1 to the simplification determination value based on the fact that the rendering processing load of the viewer avatar in full avatar is almost at the limit on the viewer terminal 300 (step S411), sets the moving person threshold reached flag, and proceeds to step S413 (step S412).

[0299] In step S413, the distribution server computer 100 determines whether it is the timing T1 immediately before the start of the first act (step S413). If it is determined that the time is the timing T1 immediately before the start of the first act (step S413; Yes), the process proceeds to step S416.

[0300] If it is determined that the time is not the timing T1 immediately before the start of the first act (step S413; No), the distribution server computer 100 determines whether it is the timing T2 when the performer avatar 1B of performer B appears (step S414). If it is determined that the current time is the timing T2 immediately before the appearance of the performer avatar 1B of performer B (step S414; Yes), the process proceeds to step S416.

[0301] Also, if it is determined that the time is not the timing T2 immediately before the appearance of the performer avatar 1B of performer B (step S414; No), the distribution server computer 100 determines whether the time is the timing T3 immediately before the start of the second act (step S415). If it is determined that the time is the timing T3 immediately before the start of the second act (step S415; Yes), the process proceeds to step S416. If it is determined that the time is not the timing T3 immediately before the start of the second act (step S415; No), the process proceeds to step S417.

[0302] In step S416, upon reaching the above-described timings T1, T2, and T3, the distribution server computer 100 adds 1 to the simplification determination value based on the fact that the rendering processing load of the viewer terminal 300 immediately increases, and proceeds to step S417 (step S416).

[0303] Also, in step S417, the distribution server computer 100 executes a composite simplification determination process. The composite simplification determination process determines, for example, whether the total processing load, which is the sum of the rendering processing load of the viewer avatar as a full avatar, the generation calculation of the shadow of each viewer avatar, and the rendering processing load of the performer avatar in the viewer terminal 300, is approaching the limit of the processing load of the viewer terminal 300 even when the threshold number of participants has not been reached in step S404, the threshold number of moving viewer avatars has not been reached in step S410, or none of the timings T1 to T3 are met in steps S413 to S415. If the total processing load is approaching the limit of the processing load of the viewer terminal 300, the process of adding 1 to the simplification determination value may be performed in the same manner as in steps S405, S411, and S416.

[0304] Note that in the third embodiment, an example is shown in which, at timing T2, the simplification determination value is incremented by 1 based on the fact that the number of performer avatars will immediately become 2 (the appearance of performer avatar 1B in addition to performer avatar 1A). That is, the threshold number of performer avatars is set to "2", but the threshold number of performer avatars is a value determined by the processing load corresponding to the number of performer avatars that can be rendered in the viewer terminal 300, not simply the number of performer avatars to be rendered.

[0305] Incidentally, the threshold value of the number of performer avatars may be set to a value different from the above as appropriate by the administrator of the distribution server computer 100 or the like in consideration of the performance (drawing processing ability) of each viewer terminal 300. In particular, in the third embodiment, when "1" is set as the threshold value of the number of performer avatars, the simplified determination value is incremented by 1 at the timing immediately before the appearance of the performer avatar 1A of the performer A during the first MC period.

[0306] Incidentally, in the composite simplification determination process of the third embodiment, an example is shown in which it is determined whether the total processing load including the drawing processing load of the viewer avatar as a full avatar in the viewer terminal 300, the generation calculation of the shadow of each viewer avatar, the drawing processing load of the performer avatar, etc. has reached near the limit of the processing load of the viewer terminal 300. However, the present invention is not limited to this. As the composite simplification determination process, the simplified determination value may be incremented by 1 on the condition that the processing load of the viewer terminal 300 has reached a processing load different from the processing load corresponding to the threshold value of the number of participants or the threshold value of the number of viewer avatars in motion.

[0307] After executing the process of step S417, the distribution server computer 100 only performs the transmission setting of the simplified determination value and ends the avatar simplification determination process (step S418).

[0308] <Viewer viewpoint video generation process> FIG. 42 is a flowchart showing an example of the viewer viewpoint video generation process shown in FIG. 38. In the viewer viewpoint video generation process, the viewer terminal 300 first generates the video of the performer avatar as a full avatar shown in FIG. 45(a) (step S501). Incidentally, as shown in FIG. 45(a), as the performer avatars 1A and 1B, there are a full avatar created based on the taste of each performer (performer A and performer B in the third embodiment) and a simplified avatar obtained by simplifying the full avatar. The simplified avatar is an avatar model created with a smaller number of polygons than the full avatar (number of polygons: full avatar of the performer > simplified avatar of the performer), and the drawing processing load in each viewer terminal 300 is lighter than that of the full avatar.

[0309] Next, it is determined whether the simplification determination value is 0 (step S502). If the simplification determination value is 0 (step S502; Yes), the videos of both the viewer avatars of the registered friends and the viewer avatars of the unregistered friends are generated as full avatars, and the process proceeds to step S509 (step S503). Incidentally, as shown in FIG. 45(b), as the viewer avatars, full avatars created based on the preferences of each viewer are provided. These full avatars as viewer avatars are avatar models created with approximately the same number of polygons as the simple avatars of the performers (number of polygons: simple avatar of the performer ≒ full avatar of the viewer), and the shape, color scheme, etc. are different for each viewer.

[0310] Incidentally, in the third embodiment, a form in which the full avatars of the performers and the full avatars of the viewers are created with approximately the same number of polygons is exemplified, but the present invention is not limited to this. Regarding the number of polygons of these full avatars of the performers and the full avatars of the viewers, if the number of polygons of the full avatar of the performer is less, the number of polygons of the full avatar of the viewer may be more than the number of polygons of the simple avatar of the performer, or the number of polygons of the simple avatar of the performer may be more than the number of polygons of the full avatar of the viewer.

[0311] Also, as shown in FIG. 45(b), as the viewer avatars, simple avatars created with fewer polygons than the full avatars of each viewer are also provided. The simple avatars as the viewer avatars are in a common form regardless of the form of the full avatars of each viewer and are created with fewer polygons than the full avatars of each viewer (number of polygons: full avatar of the viewer > simple avatar of the viewer), and the drawing processing load is lighter on each viewer terminal 300 than for the full avatars.

[0312] Returning to FIG. 42, when the simplified determination value is not 0 (step S502; No), the viewer terminal 300 further determines whether the simplified determination value is 1 (step S504). When the simplified determination value is 1 (step S504; Yes), it is determined whether the friend viewer avatar identified flag is set (step S505a). Note that the friend viewer avatar identified flag is a flag indicating that the viewer avatar file name existing in the friend user data table has been identified. When the friend viewer avatar identified flag is set (step S505a; Yes), the process proceeds to step S506. When the friend viewer avatar identified flag is not set (step S505a; No), the friend user data table shown in FIG. 40 is referred to, the viewer avatar data file name existing in the friend user data table is identified, and the identified viewer avatar data file name is stored (step S505b). Further, the viewer terminal 300 sets the friend viewer avatar identified flag and proceeds to step S506 (step S505c).

[0313] Then, in step S506, while the viewer terminal 300 generates the video of the viewer avatar (friend's viewer avatar) corresponding to the viewer avatar data file name identified in step S505 in full avatar (step S506), it generates the video of the viewer avatar excluding the viewer avatar corresponding to the viewer avatar data file name identified in step S505b (other people's viewer avatar) in simplified avatar (step S507), and proceeds to step S509.

[0314] Also, when the simplified determination value is not 1 (step S504; No), the viewer terminal 300 generates the video of all viewer avatars in simplified avatar regardless of whether they are viewers registered as friends assuming that the simplified determination value is 2 or more (step S508), and proceeds to step S509.

[0315] In step S509, the viewer terminal 300 generates other performer avatars 1A and 1B and viewer viewpoint videos other than the viewer avatar (step S509), and ends the viewer viewpoint video generation process.

[0316] As described above, in the third embodiment, as shown in FIG. 44(a), the performer avatars 1A and 1B generated in the viewer viewpoint video generation process are always generated as full avatars regardless of the value of the simplification determination process. That is, from the viewer avatar viewpoint of the viewer operating the viewer terminal 300, the performer avatars 1A and 1B are always displayed as full avatars.

[0317] Also, for the viewer avatars of the viewers registered as friends, when the simplification determination value is 1 or less, the video is generated as a full avatar, while when the simplification determination value is 2 or more, the video is generated as a simple avatar. That is, from the viewer avatar viewpoint of the viewer operating the viewer terminal 300, the viewer avatars registered as friends are displayed as full avatars when the simplification determination value is 1 or less, while they are displayed as simple avatars when the simplification determination value is 2 or more.

[0318] Furthermore, for the viewer avatars of the viewers not registered as friends, when the simplification determination value is 0, the video is generated as a full avatar, while when the simplification determination value is 1 or more, the video is generated as a simple avatar. That is, from the viewer avatar viewpoint of the viewer operating the viewer terminal 300, the viewer avatars not registered as friends are displayed as full avatars only when the simplification determination value is 0, while they are displayed as simple avatars when the simplification determination value is 1 or more.

[0319] <Virtual camera viewpoint video generation process (viewer terminal)> FIG. 43 is an example of a flowchart showing the virtual camera viewpoint video generation process (the virtual camera viewpoint video generation process executed by the viewer terminal 300) shown in FIG. 38. In the virtual camera viewpoint video generation process, the viewer terminal 300 first generates the videos of the performer avatars 1A and 1B in full avatar (step S511), and determines whether the simplification determination value is 0 (step S512). If the simplification determination value is 0 (step S512; Yes), the viewer terminal 300 generates the videos of the viewer's own viewer avatar who is operating the viewer terminal 300 and all other viewer avatars regardless of whether they are friend-registered in full avatar (step S513), and proceeds to step S523.

[0320] If the simplification determination value is not 0 (step S512; No), it is further determined whether the simplification determination value is 1 (step S514). If the simplification determination value is 1 (step S514; Yes), it is determined whether the friend viewer avatar identified flag is set (step S515a). If the friend viewer avatar identified flag is set (step S515a; yes), the process proceeds to step S516. If the friend viewer avatar identified flag is not set (step S515a; No), the friend user data shown in FIG. 40 is referred to, and the viewer avatar data file name existing in the friend user data is specified (step S515b). Further, the viewer terminal 300 sets the friend viewer avatar identified flag and proceeds to step S516 (step S515c)

[0321] Then, the viewer terminal 300 generates, in full avatar, the video of the viewer avatar of the viewer operating the viewer terminal 300 itself and the viewer avatar (the viewer avatar of a friend) corresponding to the viewer avatar data file name specified in step S515b (step S516). On the other hand, the viewer terminal 300 generates, in simple avatar, the video of the viewer avatars other than the viewer avatar of the viewer operating the viewer terminal 300 itself and the viewer avatar corresponding to the viewer avatar data file name specified in step S515 (the viewer avatars of others) (step S517), and proceeds to step S523.

[0322] If the simplification determination value is not 1 (step S514; No), it is further determined whether the simplification determination value is 2 (step S518). If the simplification determination value is 2 (step S518; No), the viewer terminal 300 generates, in full avatar, the video of the viewer avatar of the viewer operating the viewer terminal 300 itself (step S520). On the other hand, the viewer terminal 300 generates, in simple avatar, the video of the viewer avatars other than the viewer of the viewer operating the viewer terminal 300 itself (step S521), and proceeds to step S523.

[0323] If the simplification determination value is not 2 (step S518; No), assuming that the simplification determination value is 3 or more, the viewer terminal 300 generates, in simple avatar, the video of the viewer avatar of the viewer operating the viewer terminal 300 itself and all other viewer avatars regardless of whether they are registered as friends (step S522), and proceeds to step S523. In step S523, the viewer terminal 300 generates other virtual camera viewpoint videos and ends the virtual camera viewpoint video generation process (step S523).

[0324] As described above, in the third embodiment, as shown in FIG. 44(b), the performer avatars 1A and 1B generated in the viewer viewpoint video generation process of the viewer terminal 300 are always generated as full avatars regardless of the value of the simplification determination process. That is, from the viewer avatar viewpoint of the viewer operating the viewer terminal 300, the performer avatars are always displayed as full avatars.

[0325] Also, among the viewer avatars, for the viewer avatar of the viewer himself / herself who operates the viewer terminal 300, when the simplification determination value is 2 or less, the video is generated as a full avatar, while when the simplification determination value is 3 or more, the video is generated as a simple avatar. That is, from the virtual camera viewpoint, the viewer avatar of the viewer himself / herself who operates the viewer terminal 300 is displayed as a full avatar when the simplification determination value is 2 or less, while it is displayed as a simple avatar when the simplification determination value is 3 or more.

[0326] Also, among the viewer avatars, for the viewer avatars of the viewers registered as friends, when the simplification determination value is 1 or less, the video is generated as a full avatar, while when the simplification determination value is 2 or more, the video is generated as a simple avatar. That is, from the virtual camera viewpoint, the viewer avatars of the viewers registered as friends are displayed as full avatars when the simplification determination value is 1 or less, while they are displayed as simple avatars when the simplification determination value is 2 or more.

[0327] Furthermore, among the viewer avatars, for the viewer avatars of the viewers not registered as friends, when the simplification determination value is 0, the video is generated as a full avatar, while when the simplification determination value is 1 or more, the video is generated as a simple avatar. That is, from the virtual camera viewpoint, the viewer avatars of the viewers not registered as friends are displayed as full avatars only when the simplification determination value is 0, while they are displayed as simple avatars when the simplification determination value is 1 or more.

[0328] Furthermore, in the third embodiment, as shown in FIGS. 38, 42, and 43, regarding the viewer viewpoint video and the virtual camera viewpoint video generated on the viewer terminal 300, a form in which each viewer avatar is changed from a full avatar to a simple avatar according to an increase in the rendering processing load on the viewer terminal 300 was exemplified. However, the present invention is not limited to this. As shown in FIG. 25, regarding the virtual camera viewpoint video generated on the distribution server computer 100, each viewer avatar may also be changed from a full avatar to a simple avatar. In this case, each viewer avatar may be changed from a full avatar to a simple avatar according to an increase in the rendering processing load on the distribution server computer 100.

[0329] Furthermore, since it is only necessary to prevent an excessive increase in the processing load of the entire virtual space providing system of the present invention, according to an increase in the rendering processing load on the distribution server computer 100, the viewer avatar in the viewer viewpoint video or the virtual camera viewpoint video generated on the viewer terminal 300 may be changed from a full avatar to a simple avatar, or according to an increase in the rendering processing load on the viewer terminal 300, the viewer avatar in the virtual camera viewpoint video generated on the distribution server computer 100 may be changed from a full avatar to a simple avatar.

[0330] Furthermore, in the third embodiment, as described above, a form in which the viewer avatar is changed from a full avatar to a simple avatar on the viewer terminal 300 according to the processing load corresponding to the number of avatars that can be rendered on the viewer terminal 300 was exemplified. However, the present invention is not limited to this. The viewer avatar may also be changed from a full avatar to a simple avatar based on simply exceeding a threshold value of the number of participants, a threshold value of the number of viewer avatars during movement, or a threshold value of the number of performer avatars.

[0331] Next, regarding the display changes of the performer avatars 1A and 1B and the viewer avatars in the virtual live venue, an example of the display from the viewpoint of the virtual camera C2 on the viewer terminal 300 of the viewer operating the viewer avatar 11 will be described. Also, hereinafter, the viewer avatars 12, 13, and 14 will be described as the avatars of viewers who have been friend-registered by the viewer operating the viewer avatar 11, and other viewer avatars will be described as the avatars of viewers who have not been friend-registered by the viewer operating the viewer avatar 11.

[0332] Note that in the present Example 3, the display changes of the performer avatars 1A and 1B and the viewer avatars from the viewpoint of the virtual camera C2 will be described. However, regarding the display of the performer avatars 1A and 1B and the viewer avatars from the viewer's viewpoint (the viewpoint from the viewer avatar 11), since it is the same as the virtual camera C2 viewpoint except that the viewer avatar 11 is not displayed, the description will be omitted.

[0333] Also, in the present Example 3, since the composite simplification determination process (step S417) is executed in the avatar simplification determination process shown in FIG. 41, it is not necessarily the case that the viewer avatar changes from the full avatar to the simple avatar at the timing when the number of participants reaches the threshold, or at the timing when the number of viewer avatars in motion reaches the threshold, or at any of the timings T1 to T3. However, in order to prevent the description from becoming complicated, hereinafter, an example in which the viewer avatar changes from the full avatar to the simple avatar at the timing when the number of participants reaches the threshold, or at the timing when the number of viewer avatars in motion reaches the threshold, or at the timings T1 to T3 will be used for the description.

[0334] First, as shown in FIG. 46, when neither the determination of reaching the threshold number of participants nor the determination of reaching the threshold number of viewer avatars in motion is executed in the virtual live venue, the performer avatar 1A of performer A is displayed as a full avatar from the appearance timing, which is immediately after the start of the first MC period, and the performer avatar 2B of performer B is displayed as a full avatar from the appearance timing during the second MC period. Note that both the performer avatar 1A of performer A and the performer avatar 1B of performer B are maintained in full avatar display without changing to a simple avatar.

[0335] On the other hand, until timing T1, which is immediately before the start of the performance (the first performance) by the performer avatar 1A of performer A after the display starts during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see FIG. 49(a)). From timing T1, as the light emission intensity of the light emission effect increases from the immediately subsequent first performance start timing and the actions of the performer avatar 1A become more intense, resulting in an increase in the rendering processing load on the viewer terminal 300, the simplification determination value is updated from 0 to 1, and the display as a simple avatar is started for the viewer avatars of viewers who are not friend-registered by the viewer operating the viewer avatar 11 (see FIG. 49(b)).

[0336] Also, from timing T2, as the performer avatar 1B appears immediately afterwards, that is, as the number of performer avatars increases from 1 to 2, resulting in an increase in the rendering processing load on the viewer terminal 300, the simplification determination value is updated from 1 to 2, and the display as a simple avatar is started for the viewer avatars of viewers who are friend-registered by the viewer operating the viewer avatar 11 (see FIG. 49(c)).

[0337] Then, from timing T3, as the second performance, in which the light emission intensity of the light emission effect increases immediately afterwards and the actions of the performer avatars 1A and 2B of performer A and performer B become more intense, starts, the simplification determination value is updated from 2 to 3, and the display of the viewer avatar 11 itself as a simple avatar is started (see FIG. 49(D)).

[0338] As shown in FIG. 47, when the determination of reaching the participation number threshold is executed in the virtual live venue, the performer avatar 1A of performer A is displayed as a full avatar from the appearance timing which is the timing immediately after the start of the first MC period, and the performer avatar 1B of performer B is displayed as a full avatar from the appearance timing during the second MC period. Incidentally, both the performer avatar 1A of performer A and the performer avatar 1B of performer B are maintained in the full avatar display without changing to a simple avatar.

[0339] On the other hand, from the start of the display during the virtual live venue participation period until it is determined that the participation number threshold is reached during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see FIG. 49(a)). However, when it is determined that the participation number threshold is reached during the virtual live venue participation period, the simplified determination value is updated from 0 to 1 at the reaching determination timing, and the display as a simple avatar is started for the viewer avatars of viewers not registered as friends by the viewer operating the viewer avatar 11 (see FIG. 49(b)).

[0340] From timing T1, the first act starts in which the light emission intensity of the light emission effect increases immediately and the movement of the performer avatar 1A becomes intense, and as the drawing processing load on the viewer terminal 300 increases, the simplified determination value is updated from 1 to 2, and the display as a simple avatar is started for the viewer avatars of viewers registered as friends by the viewer operating the viewer avatar 11 (see FIG. 49(c)).

[0341] Then, from timing T2, immediately afterwards, the performer avatar 1B appears, that is, as the number of performer avatars increases from 1 to 2, the drawing processing load on the viewer terminal 300 increases, and the simplified determination value is updated from 2 to 3, and the display of the viewer avatar 11 itself as a simple avatar is started (see FIG. 49(D)).

[0342] As shown in FIG. 48, when the determination of reaching the threshold value of the number of participants and the determination of reaching the threshold value of the number of viewer avatars in motion are executed in the virtual live venue, the presenter avatar 1A of Presenter A is displayed as a full avatar from the appearance timing, which is immediately after the start of the first MC period. The presenter avatar 1B of Presenter B is displayed as a full avatar from the appearance timing during the second MC period. Note that both the presenter avatar 1A of Presenter A and the presenter avatar 1B of Presenter B are maintained in the display as full avatars without changing to simple avatars.

[0343] On the other hand, from the start of the display during the virtual live venue participation period until it is determined that the threshold value of the number of participants is reached during the virtual live venue participation period, all viewer avatars are displayed as full avatars (see FIG. 49(a)). However, if it is determined that the threshold value of the number of participants is reached during the virtual live venue participation period, the simplified determination value is updated from 0 to 1 at the timing of the determination, and the display as a simple avatar is started for the viewer avatars of the viewers who are not friend-registered by the viewer operating the viewer avatar 11 (see FIG. 49(b)).

[0344] Also, if it is determined that the number of viewer avatars in motion reaches the threshold value during the virtual live venue participation period, the simplified determination value is updated from 1 to 2 at the timing of the determination, and the display as a simple avatar is started for the viewer avatars of the viewers who are friend-registered by the viewer operating the viewer avatar 11 (see FIG. 49(c)).

[0345] Then, from timing T1, in response to the increase in the drawing processing load in the viewer terminal 300 due to the start of the first act in which the light emission intensity of the light emission effect immediately increases and the operation of the presenter avatar 1A becomes intense, the simplified determination value is updated from 2 to 3, and the display of the viewer avatar 11 itself as a simple avatar is started (see FIG. 49(D)).

[0346] Furthermore, in the third embodiment, even after the simplified determination value is updated to 3 and the display of all viewer avatars in the simple avatar is started, the simplified determination value may be updated to 4 or 5. However, even if the simplified determination value is updated to 4 or more, the all viewer avatars will not change further from the simple avatar.

[0347] As described above, in the third embodiment, as shown in FIGS. 41 to 44 and FIGS. 46 to 49, when the number of performer avatars to be displayed increases to 2, the rendering processing load of the performer avatars increases. When the operation of the performer avatars becomes large (intense) at the start of the first act period or the second act period, the update processing load of the virtual space and the rendering processing load of the performer avatars increase. Based on the start of the first act period or the second act period, when the emission intensity of the emission effect increases, the rendering processing load of the virtual space and each avatar increases. When the number of viewer avatars (the number of live participants) virtually participating in the virtual live venue exceeds the threshold number, the update processing load of the virtual space and the rendering processing load of the viewer avatars increase. When the number of moving viewer avatars (the number of moving people) among the viewer avatars virtually participating in the virtual live venue exceeds the threshold, the update processing load of the virtual space and the rendering processing load of the viewer avatars increase. Based on the occurrence of any of these (the establishment of the specific conditions of the present invention), by changing the viewer avatars in the virtual live venue from full avatars to simple avatars, the processing load for the viewer terminal 300 to display (render) these viewer avatars is reduced. Therefore, it is possible to prevent an excessive increase in the overall processing load of the virtual space providing system due to the establishment of the above specific conditions.

[0348] Also, in the third embodiment, time schedule data for managing the progress of the virtual live is stored in the storage 103 of the distribution server computer 100. Since the distribution server computer 100 can specify the timing immediately before the processing load increases from the time schedule data in the avatar simplification determination process shown in FIG. 41, the timing at which the processing load increases can be accurately specified at timings T1 to T3 before the actual increase in the processing load.

[0349] In particular, in the third embodiment, the timing at which the number of displayed performer avatars increases to two, thereby increasing the rendering processing load of the performer avatars, the timing at which the operations of the performer avatars become large (intense) due to the start of the first act period or the second act period, thereby increasing the update processing load of the virtual space and the rendering processing load of the performer avatars, and the timing at which the light emission intensity of the light emission effect increases based on the start of the first act period or the second act period, thereby increasing the rendering processing load of the virtual space and each avatar, by changing the viewer avatar from a full avatar to a simple avatar from timings T1 to T3 before these timings, the timings of the increase in the number of these performer avatars, the timings at which the operations of the performer avatars 1A and 1B intensify, and the timings at which the light emission intensity is enhanced as the light emission effect do not overlap with the timing of changing the viewer avatar from a full avatar to a simple avatar, thereby preventing the processing loads required for these processes from overlapping and the overall processing load of the virtual space providing system from excessively increasing.

[0350] In the third embodiment, the form in which the time schedule data is stored in the storage 103 of the distribution server computer 100 and the distribution server computer 100 changes the viewer avatar from a full avatar to a simple avatar based on identifying timings T1 to T3 has been exemplified. However, the present invention is not limited to this, and the time schedule data may be stored in the storage 303 of the viewer terminal 300, and the viewer terminal 300 itself may identify timings T1 to T3 from the time schedule data and execute the change of the viewer avatar from a full avatar to a simple avatar based on the identification.

[0351] Also, in the third embodiment, the form of changing the viewer avatar from a full avatar to a simple avatar from timings T1 to T3 immediately before the rendering processing load increases has been exemplified. However, the present invention is not limited to this, and when there is sufficient margin in the rendering processing ability of the viewer terminal 300, the viewer avatar may be changed from a full avatar to a simple avatar at the timing when the rendering processing load increases.

[0352] Also, as shown in FIG. 49, as the virtual camera designated video, even for the viewer avatar of the viewer himself / herself who operates the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49), it will ultimately change from the full avatar to the simple avatar and be displayed, so that an excessive increase in the processing load can be further prevented.

[0353] Also, in the third embodiment, since the viewer avatars of the viewers who operate other viewer terminals 300 change from the full avatar to the simple avatar with priority over the viewer avatar of the viewer himself / herself who operates the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49), an excessive increase in the processing load can be effectively prevented.

[0354] In addition, in the third embodiment, the viewer avatars are divided into the viewer avatar of the viewer himself / herself who operates the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49), the viewer avatars of the viewers who are friend-registered by the viewer (viewer avatars 12, 13, 14 in the example of FIG. 49), and the viewer avatars of the viewers who are not friend-registered by the viewer. The priority of changing from the full avatar to the simple avatar is set such that the viewer avatars of the viewers who are not friend-registered are the highest, and the viewer avatar of the viewer himself / herself who operates the viewer terminal 300 (viewer avatar 11 in the example of FIG. 49) is the lowest (priority of changing from the full avatar to the simple avatar: viewer avatars of viewers who are not friend-registered > viewer avatars of viewers who are friend-registered > viewer avatar of the viewer himself / herself who operates the viewer terminal 300). However, the present invention is not limited to this, and the priority of changing from the full avatar to the simple avatar may be other than the above, or the priority of changing from the full avatar to the simple avatar may not be set among the viewer avatars.

[0355] Also, in the third embodiment, as shown in FIGS. 45 to 48, both the performer avatars 1A and 1B and the viewer avatar are provided with full avatars and simple avatars. However, as for the viewer avatar, while the full avatar changes to the simple avatar as the rendering processing load of the viewer terminal 300 increases, as for the performer avatars 1A and 1B, only the full avatar is always displayed regardless of the increase in the rendering processing load of the viewer terminal 300. That is, during the virtual live show, since a simple avatar with a small rendering processing load for displaying as the performer avatars 1A and 1B is not displayed, it is possible to prevent a decrease in the interest of the live show due to the display of the simple avatar as the performer avatars 1A and 1B.

[0356] In addition, in the third embodiment, a form in which the performer avatars 1A and 1B are provided with full avatars and simple avatars is exemplified, but the present invention is not limited to this. Since there is no opportunity to display the simple avatar as the performer avatars 1A and 1B in the first place, only the full avatar may be provided as the performer avatars 1A and 1B without providing the simple avatar.

[0357] Also, in the third embodiment, a performer terminal 200 operable by a performer is provided as a device constituting the virtual space providing system, and as virtual space content, the operations of the performer avatars 1A and 1B in which the operations of the performer are immediately reflected can be distributed. However, the present invention is not limited to this, and as the virtual space content, the operations of the performer avatars 1A and 1B photographed or produced in advance may be distributed.

[0358] In addition, in the third embodiment, as shown in FIG. 41, based on the fact that the number of viewer avatars installed in the virtual live venue and the number of viewer avatars in motion have reached a predetermined threshold value, the distribution server computer 100 univocally determines that the rendering processing load in the viewer terminal 300 is almost at its limit, and an example is given in which the viewer avatar in the viewer terminal 300 is changed from a full avatar to a simple avatar. However, the present invention is not limited to this. In reality, since it is extremely unlikely that each viewer is using the same viewer terminal 300, even when the distribution server computer 100 determines that the rendering processing load in the viewer terminal 300 is almost at its limit, if there is still sufficient rendering processing capacity on the viewer terminal 300 side for rendering viewer avatars, performer avatars, etc., the viewer terminal 300 side may continue to display the viewer avatar as a full avatar.

[0359] In addition, in the third embodiment, when the number of viewer avatars virtually participating in the virtual live venue exceeds the threshold number, when the number of viewer avatars in motion among the viewer avatars virtually participating in the virtual live venue exceeds the threshold number, when the emission intensity of the light emission effect increases, when the movement of the performer avatar becomes large (intense), when the number of performer avatars increases, the distribution server computer 100 determines that the rendering load of the viewer terminal 300 increases and changes the viewer avatar from a full avatar to a simple avatar for the viewer terminal 300. However, the present invention is not limited to this. When the number of viewer avatars virtually participating in the virtual live venue exceeds the threshold number, when the number of viewer avatars in motion among the viewer avatars virtually participating in the virtual live venue exceeds the threshold number, when the emission intensity of the light emission effect increases, when the movement of the performer avatar becomes large (intense), when the number of performer avatars increases, the viewer terminal 300 itself may determine that the rendering processing load of the viewer terminal 300 increases and change the viewer avatar from a full avatar to a simple avatar. In this case, the viewer himself / herself who is viewing the virtual live using the viewer terminal 300 may set the threshold number of the number of viewer avatars virtually participating in the virtual live venue and the threshold number of the number of viewer avatars in motion by operating the viewer terminal 300.

[0360] Also, in the third embodiment, an example is given in which the viewer avatar of a viewer who is not a friend is preferentially displayed as a simple avatar over the viewer avatar of a viewer who is a friend and has a high relationship with the viewer user using the viewer terminal 300. However, the present invention is not limited to this. Relatives such as the parents and siblings of the viewer user using the viewer terminal 300, and persons belonging to the same organization (for example, school, company, club, etc.) as the viewer user using the viewer terminal 300 are registered in advance in the same procedure as shown in FIG. 39 in advance, and the viewer avatars of viewers who are not these relatives or related persons may be preferentially displayed as simple avatars. That is, the "other viewer users with a high relationship with oneself" in the third embodiment may be a concept including not only friends but also relatives of the viewer user himself and related persons belonging to the same organization.

[0361] Also, in the third embodiment, an example is given in which the viewer avatar is changed from a full avatar to a simple avatar based on an increase in the processing load of the viewer terminal 300. However, the present invention is not limited to this. A plurality of simple avatars with different drawing loads are provided as viewer avatars, and when the processing load of the viewer terminal 300 increases, a simple avatar corresponding to the drawing ability may be displayed as the viewer avatar for each viewer terminal 300.

[0362] <Modification Example of the Third Embodiment> As described above, the present invention has been described based on each of the embodiments and modification examples. However, the specific configuration is not limited to the above-described embodiments and modification examples, and changes and additions within the scope not departing from the gist of the present invention are also included in the present invention.

[0363] For example, in the third embodiment described above, although both a full avatar and a simplified avatar are prepared as the performer avatars 1A and 1B, a form was exemplified in which the simplified avatar is not displayed as the performer avatars 1A and 1B. However, the present invention is not limited to this. As a modification example 3-1, a case where the performer avatars 1A and 1B are displayed as simplified avatars may be provided, or a period may be provided during which it is possible to select whether to display the performer avatars 1A and 1B as full avatars or simplified avatars by operating the performer terminals 200 by each performer.

[0364] For example, as shown in FIG. 50, during a period other than the performance period, both the performer avatar 1A of performer A and the performer avatar 1B of performer B may be made selectable as to whether to be displayed as a full avatar or a simplified avatar by an operation from each performer terminal 200. Also, during the performance period, when the performance period is a performance period in which only the performer avatar 1A of performer A sings or dances, the performer avatar 1A of performer A is displayed as a full avatar while the performer avatar 1B of performer B is displayed as a simplified avatar, so as to reduce the processing load for displaying the performer avatar 1B of performer B on the viewer terminal 300. When the performance period is a performance period in which only the performer avatar 1B of performer B sings or dances, the performer avatar 1B of performer B may be displayed as a full avatar while the performer avatar 1A of performer A is displayed as a simplified avatar, so as to reduce the processing load for displaying the performer avatar 1A of performer A on the viewer terminal 300.

[0365] Also, when the performance period is a performance period in which both the performer avatar 1A of performer A and the performer avatar 1B of performer B sing or dance, both the performer avatar 1A of performer A and the performer avatar 1B of performer B may be displayed.

[0366] In addition, in the third embodiment, as an example of the simple avatar of the viewer avatar, a form in which only one type of simple avatar is provided regardless of the performer who sings or dances was illustrated. However, the present invention is not limited to this. As shown in FIG. 51 as Modification 3-2, the form of the simple avatar of the viewer may be made different according to the performer avatar who performs the program. In particular, during non-program periods such as the period during which virtual live venue participation is possible, the opening performance period, and the MC period, the viewer avatar is displayed in the manner shown in the third embodiment, while during each program period, the simple avatar of the manner set in advance according to each performer according to the performer avatar who sings or dances may be used to display the viewer avatar. By doing so, the form of the simple avatar as the viewer avatar changes according to the performer avatar, so the interest of the virtual live can be improved.

[0367] In addition, when the form of the viewer avatar is thus made different according to the performer, as shown in FIGS. 52 and 53, from the pre-data download request received by the distribution server computer 100 from the viewer terminal 300 in the pre-data transmission process, identify the performer for whom the simple avatar has been set in advance for the viewer (the performer for whom the simple avatar data for the viewer is stored in the storage 103 in the distribution server computer 100 in advance) (step S601), perform the transmission setting of the simple avatar data for the viewer for each identified performer (step S602), and perform the transmission setting of other pre-data from the received pre-data download request (step S603), and the pre-data for which these transmission settings have been performed may be transmitted to the viewer terminal 300 (step S604).

[0368] By doing so, the viewer terminal 300 can receive the simple avatar data for the viewer corresponding to each performer (performer avatars 1A, 1B) as part of the pre-data from the distribution server computer 100, and display the simple avatar in the manner corresponding to each performer avatar 1A, 1B as the viewer avatar during the program period.

[0369] In the present Modification Example 3-2, an example has been given of a form in which a simple avatar in a mode corresponding to each performer avatar 1A, 1B is displayed as a viewer avatar. However, the present invention is not limited to this. For a performer who has no setting of a simple avatar for the viewer as pre-data (a performer in whom data of a simple avatar for the viewer is not stored in the storage 103 in the distribution server computer 100 in advance), during the performance period, as shown in FIG. 51, a simple avatar in the same mode as during the non-performance period may be displayed as the viewer avatar.

[0370] Also, in the present Modification Example 3-2, an example has been given of a form in which a simple avatar in a mode corresponding to each performer avatar 1A, 1B is displayed as a viewer avatar. However, the present invention is not limited to this, and the simple avatar displayed as the viewer avatar may be made different as time elapses. For example, as shown in FIG. 54 as Modification Example 3-3, a heart-shaped character may be displayed as the simple avatar of the viewer until a predetermined timing S from the start of the performance, while a star-shaped character may be displayed as the simple avatar of the viewer after the timing S.

[0371] Also, in the above Example 3, an example has been given of a form in which two types of avatars, i.e., a full avatar and a simple avatar with a smaller drawing processing load than the full avatar, are provided for the viewer avatar and the performer avatars 1A, 1B, respectively. However, the present invention is not limited to this. As Modification Example 3-4, for the viewer avatar and the performer avatars 1A, 1B, a plurality of simple avatars with different drawing processing loads (for example, a first simple avatar with a smaller drawing processing load than the full avatar and a second simple avatar with a smaller drawing processing load than the first simple avatar) may be provided.

[0372] Thus, when providing a plurality of simple avatars with different rendering processing loads, the number of viewer avatars virtually participating in the virtual live venue exceeds the threshold number, the number of viewer avatars in motion among the viewer avatars virtually participating in the virtual live venue exceeds the threshold number, reaching any of the timings T1 to T3, etc. Under such conditions, the mode of the viewer avatar may be changed step by step to a full avatar, a first simple avatar, a second simple avatar, etc., so that the rendering processing load of the viewer terminal 300 decreases.

[0373] Also, in the third embodiment, an example is given of a form in which the viewer avatar is changed from a full avatar to a simple avatar on the condition that the number of viewer avatars in motion among the viewer avatars virtually participating in the virtual live venue exceeds the threshold number. However, the present invention is not limited to this. As a modification example 3-5, even if the viewer avatar is not in motion, if the number of viewer avatars performing actions such as dancing or jumping among the viewer avatars virtually participating in the virtual live venue exceeds the threshold number, the viewer avatar may be changed from a full avatar to a simple avatar. That is, the “performer avatar operating in the virtual space” in the third embodiment may include, in addition to the performer avatar moving within the virtual live venue, a performer avatar performing an action other than movement within the virtual live venue.

[0374] Also, in the third embodiment, an example is given of a form in which each viewer avatar is first displayed as a full avatar and then changed to a simple avatar based on the fulfillment of specific conditions. However, the present invention is not limited to this. As a modification example 3-6, when it is surely seen that the number of viewer avatars virtually participating in the virtual live venue exceeds a predetermined threshold number, each viewer avatar may be displayed as a simple avatar from the beginning.

[0375] In addition, in the third embodiment, as the number of displayed performer avatars increases to two, the rendering processing load of the performer avatars increases. When the operation of the performer avatars becomes large (intense) at the start of the first act period or the second act period, the update processing load of the virtual space and the rendering processing load of the performer avatars increase. Based on the start of the first act period or the second act period, as the emission intensity of the light emission effect increases, the rendering processing load of the virtual space and each avatar increases. When the number of viewer avatars (number of live participants) virtually participating in the virtual live venue exceeds the threshold number, the update processing load of the virtual space and the rendering processing load of the viewer avatars increase. When the number of moving viewer avatars (number of moving people) among the viewer avatars virtually participating in the virtual live venue exceeds the threshold, the update processing load of the virtual space and the rendering processing load of the viewer avatars increase. These are specified conditions in the present invention. Based on the establishment of any of these specified conditions, an example is given of a form in which the viewer avatar is changed from a full avatar to a simple avatar. However, the present invention is not limited to this. As Modification Example 3-7, the specified conditions in the present invention may be only a part of the above, or may include events other than the above conditions as long as they increase the processing load in the distribution server computer 100 or the viewer terminal 300 constituting the virtual space providing system of the present invention.

[0376] In addition, in the third embodiment, as shown in FIGS. 41 to 43, when generating the viewer perspective video and when generating the virtual camera perspective video, the same threshold values (the threshold value of the number of participants and the threshold value of the number of viewer avatars in motion) are used to change the viewer avatar from a full avatar to a simple avatar. However, the present invention is not limited to this. As a modification 3-8, different threshold values may be used when generating the viewer perspective video and when generating the virtual camera perspective video to change the viewer avatar from a full avatar to a simple avatar. For example, as shown in FIG. 13, when generating the virtual camera C1 video, since almost only the performer avatar is drawn, which is different from the cases of generating the virtual camera C2 video, the virtual camera C3 video, the virtual camera C4 video, and each viewer perspective video, while setting a high threshold value for the number of participants and the threshold value of the number of viewer avatars in motion, in the virtual camera C2 video, the virtual camera C3 video, the virtual camera C4 video, and each viewer perspective video, since each viewer avatar is also actively displayed in addition to the performer avatar, the threshold value for the number of participants and the threshold value of the number of viewer avatars in motion may be set high.

[0377] Also, even in the case of the viewer perspective video, when the line of sight of the viewer avatar is not directed at the virtual stage G, since it is more difficult to draw the performer avatar as the viewer perspective video than when the line of sight of the viewer avatar is directed at the virtual stage G, the rendering processing load due to rendering the performer avatar is less likely to increase. Therefore, the threshold value for the number of participants and the threshold value of the number of viewer avatars in motion may be set higher than when the line of sight of the viewer avatar is directed at the virtual stage G.

[0378] In addition, in the third embodiment, as an example, the viewer avatar of a viewer who is not friend-registered, the viewer avatar of a viewer who is friend-registered, and the viewer avatar of oneself are changed to simple avatars in this order in response to an increase in the rendering processing load of the viewer terminal 300. However, the present invention is not limited to this. As a modification 3-9, each time the rendering processing load of the viewer terminal 300 increases, the viewer avatar may be changed to a simple avatar in order from the viewer avatar that is farthest from one's own viewer avatar.

Example

[0379] Next, the virtual space providing system of Example 4 will be described below with reference to FIGS. 55 to 65. For the same configurations as those in other examples including Examples 1 and 2, the same reference numerals will be given and the description will be omitted.

[0380] As shown in FIGS. 55 and 56, the virtual space providing system of Example 4 uses, as viewer avatars, full avatars having different appearances for each viewer user (viewer terminal 300) corresponding to a relatively large processing load (drawing processing load) for display and being able to identify each viewer avatar, and simple avatars having a smaller drawing processing load than these full avatars.

[0381] In the virtual live implemented in the virtual space providing system of this Example 4, as in Examples 1 and 2, it proceeds in the order of "virtual live space participation period", "pre-opening explanation", and "opening performance". After the "opening performance", it proceeds as "First MC" → "First performance" → "Second MC" → "Second performance" → "Third MC" → "Third performance".

[0382] In the "First MC", "Second MC", and "Third MC", the performer avatar 1 who appears on the virtual stage G greets the viewer users by talking and introduces the next performance to be staged. The "First performance", "Second performance", and "Third performance" are performances including singing and dancing by the performer avatar (performer).

[0383] In such a virtual live progression, in this Example 4, first, during the virtual live space participation period, each viewer user can participate in the virtual live with a full avatar as the viewer avatar corresponding to the viewer terminal 300 by operating the viewer terminal 300. Then, from the start timing of the period of the "First MC", each viewer terminal 300 can reduce the processing load for displaying the viewer avatar by changing the viewer avatar, which has been a full avatar until then, to a simple avatar.

[0384] In particular, in the fourth embodiment, as shown in FIG. 61, the period during which the live virtual space can be participated in is a period during which the area (area to be simplified) in which the viewer avatar changes from a full avatar to a simple avatar by the viewer user operating the viewer terminal 300 can be selected from the third area and the fourth area of the virtual audience area. That is, it is a period during which the area to be simplified can be specified. If the viewer user does not specify the area to be simplified during the period during which the live virtual space can be participated in according to the fourth embodiment, both the third area and the fourth area are areas not to be simplified, and the viewer avatar will not be displayed as a simple avatar.

[0385] Hereinafter, a series of processes for selecting the area (area to be simplified) in which the viewer avatar changes from a full avatar to a simple avatar from the third area and the fourth area of the virtual audience area will be described based on FIGS. 57 to 65.

[0386] As shown in FIG. 57, during the period during which the live virtual space can be participated in, the viewer terminal 300 that has allowed the viewer avatar to participate in the virtual live venue first executes a simplified target area designation operation process for receiving an operation from the viewer user as to which area the viewer avatar should be changed from a full avatar to a simple avatar.

[0387] At this time, when the viewer terminal 300 receives an operation from the viewer user, it transmits simplified target area information including the participant ID (viewer user ID) of the viewer user to the distribution server computer 100.

[0388] When the distribution server computer 100 receives the simplified target area information from the viewer terminal 300, it identifies the participant ID (viewer user ID) and which area is to be the simplified target area from the simplified target area information, and as shown in FIG. 58, executes a participant user table update process for storing the data of the identified simplified target area in association with the participant ID in the participant user table.

[0389] In addition, the distribution server computer 100 executes a participant user table update completion information transmission process of transmitting participant user table update completion information to the viewer terminal 300 based on having executed the participant user table update process. Then, based on having received the participant user table update completion information, the viewer terminal 300 executes a participant user table update completion notification process of notifying, on the display device 310, that the update of the participant user table has been completed, that is, that the registration of which of the third area and the fourth area as the simplification target area has been completed in the viewer terminal 300.

[0390] In this way, by the viewer terminal 300 and the distribution server computer 100 executing the series of processes shown in FIG. 57, the viewer terminal 300 can set the simplification target area, and the distribution server computer 100 side can grasp which of the third area and the fourth area the viewer terminal 300 has set as the simplification target area.

[0391] In addition, the virtual space providing system in the fourth embodiment includes a friend registration function that allows a viewer user to register another viewer user as a friend by operating the viewer terminal 300.

[0392] As such a friend registration function, as shown in FIG. 59, the viewer terminal 300 executes a viewer avatar selection process of selecting a specific viewer avatar from among a plurality of viewer avatars by receiving an operation of the viewer's operation input device 321 during the live virtual space participation period, specifies the file name (viewer avatar file name) of the selected viewer avatar, and executes a viewer avatar inquiry process for transmitting viewer avatar inquiry information including the specified viewer avatar file name to the distribution server computer 100.

[0393] When the distribution server computer 100 receives viewer avatar inquiry information from the viewer terminal 300, it executes a viewer avatar identification process to identify the viewer user ID associated with the viewer avatar file name included in the viewer avatar inquiry information from the viewer user data shown in FIG. 9. Then, the distribution server computer 100 transmits viewer user ID information including the viewer user ID to the viewer terminal 300 that has transmitted the viewer avatar inquiry information.

[0394] When the viewer terminal 300 receives the viewer user ID information, it executes a friend addition operation process that prompts an operation to register, as a friend, the viewer user (viewer user ID) of the viewer avatar selected by the viewer in the display device 310 in association with the viewer avatar file name.

[0395] At this time, when the viewer performs an operation to register a friend, the viewer terminal 300 executes a friend addition process of storing the viewer user ID and the viewer avatar file name in association with each other in the friend user data table. Also, when the operation to register a friend is not performed, the viewer terminal 300 executes a friend addition cancellation process of displaying a message indicating that no friend registration is performed on the display device 310.

[0396] In addition, in the present Example 4, a friend user data table (see FIG. 60) for storing the user ID and avatar data of the viewer user is created in advance in the storage 303 of the viewer terminal 300. When executing the friend addition process, the viewer user ID and the viewer avatar file name may be additionally stored in association with each other by updating the friend user data table.

[0397] FIG. 62 is a flowchart showing the simplified area designation operation process shown in FIG. 57. In the simplified area designation operation process, the viewer terminal 300 first executes a simplified area designation operation process for receiving from the viewer user an operation as to whether the third area or the fourth area of the virtual audience area is to be the simplified area (step S801).

[0398] In addition, as shown in FIGS. 63(a) and 63(b), on the display device 310 during the execution of the simplified area designation operation process, a window menu 310WA for the viewer user to designate the simplified area is displayed. As the window menu 310WA, items of "third area" and "fourth area" that can be designated as the simplified area are arranged, and below these items, check boxes for selecting whether to preferentially display friends in the simplified area in full avatar are arranged.

[0399] Returning to FIG. 62, it is determined whether there is a third area designation operation to set the third area as the simplified area in the simplified area designation operation process (step S802). If there is a third area designation operation (step S802; Yes), the third area is stored as the simplified area (step S803).

[0400] Also, when there is no third area designation operation (step S802; No), or after the viewer terminal 300 executes the process of step S803, it is determined whether there is a fourth area designation operation to set the fourth area as the simplified area in the simplified area designation operation process (step S804). If there is a fourth area designation operation (step S804; Yes), the fourth area is stored as the simplified area (step S805).

[0401] Then, when there is no fourth area designation operation (step S804; No), or after the viewer terminal 300 executes the process of step S805, the viewer terminal 300 transmits simplified area information corresponding to the area stored as the simplified area to the distribution server computer 100 (step S806).

[0402] Also, the viewer terminal 300 determines whether there is a full avatar priority display operation for a friend in the simplified area designation operation process, that is, as shown in FIG. 63(b), whether a check box arranged at the lower part of the window menu 310WA is checked by the operation of the viewer user (step S807). When there is a full avatar priority display operation (step S807; Yes), a friend full avatar priority display flag indicating that the viewer avatars of friends arranged in the simplified area are to be displayed as full avatars is set, and the simplified area designation operation process is terminated (step S808).

[0403] Also, when there is no full avatar priority display operation (step S807; No), the viewer terminal 300 determines whether the friend full avatar priority display flag has already been set (step S809). When the friend full avatar priority display flag is not set (step S809; No), the simplified area designation operation process is terminated. When the friend full avatar priority display flag is set (step S809; Yes), it further determines whether a cancellation operation of the full avatar priority display has been executed, that is, whether the check has been removed from the check box arranged at the lower part of the window menu 310WA by the operation of the viewer user (step S810).

[0404] When the cancellation operation of the full avatar priority display has not been executed (step S810; No), the simplified area designation operation process is terminated. When the cancellation operation of the full avatar priority display has been executed (step S810; Yes), the friend full avatar priority display flag is cleared, and the simplified area designation operation process is terminated (step S811).

[0405] FIG. 64 is a flowchart showing the viewer viewpoint video generation process executed by the viewer terminal 300 in the fourth embodiment. In the viewer viewpoint video generation process, the viewer terminal 300 first generates the video of the performer avatar as a full avatar (step S501), and determines whether it is after the start timing of the period of the first MC in the virtual live (step S502). If it is before the start timing of the period of the first MC in the virtual live (step S502; No), the viewer terminal 300 generates the videos of the viewer avatars of both the registered friend viewer users and the unregistered other viewer users as full avatars, and ends the viewer viewpoint video generation process (step S503).

[0406] Also, if it is after the start timing of the period of the first MC in the virtual live (step S502; Yes), the viewer terminal 300 further determines whether a friend viewer avatar identification completed flag indicating that the viewer avatar of the registered friend viewer user has already been identified in the viewer viewpoint video generation process is set (step S504).

[0407] If the friend viewer avatar identification completed flag is not set (step S504; No), the viewer terminal 300 refers to the friend user data table shown in FIG. 60, identifies and stores the friend viewer avatar data file name (step S505), and sets the friend viewer avatar identification completed flag (step S506).

[0408] Also, when the friend viewer avatar identification completed flag is set (step S504; Yes) or after the execution of the process in step S506, the viewer terminal 300 identifies the simplification target area stored in the simplification target area designation operation process in FIG. 62 (step S507), and generates the videos of all viewer avatars outside the identified simplification target area as full avatars (step S508).

[0409] Next, the viewer terminal 300 generates the video of the viewer avatar of others in the area to be simplified (the viewer avatar of viewers who are not registered as friends) with a simplified avatar (step S509), and generates the video of the viewer avatar of friends outside the area to be simplified with a full avatar (step S510). Further, the viewer terminal 300 determines whether or not the friend full avatar priority display flag is set (step S511).

[0410] When the friend full avatar priority display flag is set (step S511; Yes), the viewer avatar of a friend in the area to be simplified is generated with a full avatar and the process of step S514 is executed (step S512). When the friend full avatar priority display flag is not set (step S511; No), the video of the viewer avatar of a friend in the area to be simplified is generated with a simplified avatar and the process of step S514 is executed (step S513).

[0411] By executing the viewer perspective video generation process as described above, in the fourth embodiment, as shown in FIG. 65(a), when comparing the modes of each viewer avatar generated in the viewer perspective video generation process when the friend full avatar priority display flag is not set, at a timing earlier than the start of the first MC period, for the viewer avatars arranged in either the area to be simplified or the area outside the area to be simplified, all are displayed as full avatars regardless of whether they are the viewer avatars of viewer users who are registered as friends.

[0412] On the other hand, at a timing after the start of the second MC period, all the viewer avatars arranged in the area to be simplified are displayed as simplified avatars regardless of whether they are the viewer avatars of viewer users who are registered as friends, while all the viewer avatars arranged in the area outside the area to be simplified are displayed as full avatars regardless of whether they are the viewer avatars of viewer users who are registered as friends.

[0413] Also, as shown in FIG. 65(b), when comparing the states of each viewer avatar generated in the viewer perspective video generation process when the friendly avatar priority display flag is set, at a timing before the start of the first MC period, for the viewer avatars placed in either the simplification target area or the non-simplification target area, regardless of whether they are the viewer avatars of the viewer users registered as friends, all are displayed as full avatars.

[0414] On the other hand, at a timing after the start of the second MC period, among the viewer avatars placed in the simplification target area, the viewer avatars of the viewer users registered as friends are displayed as full avatars, and the viewer avatars of the viewer users not registered as friends are displayed as simplified avatars. Also, the viewer avatars placed in the non-simplification target area are all displayed as full avatars regardless of whether they are the viewer avatars of the viewer users registered as friends.

[0415] As described above, in the virtual space providing system according to the fourth embodiment, by executing the simplification target area designation operation process shown in FIG. 62 and the viewer perspective video generation process shown in FIG. 64, the viewer user himself / herself can execute the setting regarding making the viewer avatar a simplified avatar. As a result, the intention of the viewer user can be reflected regarding the viewer avatar to be displayed as a simplified avatar, so it is possible to suppress a decrease in the interest of the virtual space live due to reducing the processing load for displaying the viewer avatar to a simpler avatar with a smaller load to suppress the occurrence of display output defects.

[0416] Also, in the virtual space providing system according to the fourth embodiment, by executing the simplification target area designation operation process shown in FIG. 62 and the viewer perspective video generation process shown in FIG. 64, the viewer user can operate the viewer terminal 300 to designate the simplification target area for making the viewer avatar and the simplified avatar from the third area and the fourth area. Therefore, it is possible to further suppress a decrease in the interest of the virtual live due to the viewer avatar in the area not intended by the viewer user becoming a simplified avatar.

[0417] Also, in the virtual space providing system according to the fourth embodiment, as shown in FIGS. 59 and 60, other viewer users can be registered as one's own friends, and as shown in FIG. 65, the viewer avatar of the viewer user who has become a friend can be set to be displayed as a full avatar even when participating in the simplification target area. Therefore, it becomes easier for the viewer avatar of the viewer user who is a friend to be displayed as a full avatar, and when a plurality of viewer avatars are participating in the virtual live, it is possible to make it easier to recognize the viewer avatar of the viewer user who is a friend and the viewer avatar of the viewer user who is not a friend.

[0418] Also, in the fourth embodiment, in addition to being able to set only one of the third area and the fourth area of the virtual audience area as the simplification target area by the viewer user operating the viewer terminal 300, by setting both the third area and the fourth area as the simplification target area, the processing load related to display output can be reduced compared to the case of setting only one area as the simplification target area. Therefore, the level of the processing load related to display output can be changed according to the situation.

[0419] <Modification Example of Embodiment 4> Also, in the above-described fourth embodiment, as shown in FIGS. 62 and 63, by executing the simplification target area designation operation process, it is possible to select whether or not the third area and the fourth area of the virtual audience area are to be the simplification target areas. However, the present invention is not limited to this. As shown in FIG. 66 as Modification Example 4-1, among the viewer avatars participating in the virtual live, it may be possible to select, by the operation of the viewer user, which viewer avatars are to be the simplification targets. In addition, when the number of viewer avatars to be simplified is selected in this way, with the selected number of viewer avatars as the upper limit number, the viewer avatars close in position to the viewer avatars corresponding to the viewer terminal 300 may be displayed as simplified avatars in order from the closest, or any viewer avatar within all the viewer avatars participating in the virtual live may be displayed as a simplified avatar.

[0420] By doing so, as a simple avatar, the viewer user himself / herself can set the number of viewer avatars, so it is possible to further suppress the decrease in the interest in the virtual live due to the fact that an unintended number of viewer avatars become simple avatars.

[0421] Also, in the above-described Example 4, as shown in FIGS. 62 and 63, by executing the simplified target area designation operation process, it was made possible to select whether or not to set the third area and the fourth area of the virtual audience area as the simplified target areas. However, the present invention is not limited to this. As a modification example 4-2, as shown in FIG. 67, the ratio of the number of viewer avatars to be simple avatars with respect to the number of viewer avatars participating in the virtual live may be made selectable by the selection operation of the viewer user. In addition, when the ratio of the viewer avatars to be simplified is selected in this way, the viewer avatars closest in position to the viewer avatars corresponding to the viewer terminal 300 may be displayed as simple avatars in order from the ones with the selected ratio, or any viewer avatar within all the viewer avatars participating in the virtual live may be displayed as a simple avatar.

[0422] By doing so, the viewer user himself / herself can set the ratio of the number of viewer avatars to be simple avatars with respect to the number of viewer avatars participating in the virtual live, so it is possible to further suppress the decrease in the interest in the virtual live due to the fact that an unintended ratio of viewer avatars become simple avatars.

[0423] Also, in the above-described Example 4, as shown in FIG. 61, the live virtual space participation available period is set as the simplification target area designation available period. When the viewer user does not set the simplification target area during the live virtual space participation available period, both the third area and the fourth area of the virtual audience area become non-simplification target areas, and neither the viewer avatar is displayed as a simple avatar. However, the present invention is not limited to this. When the viewer user does not specify the simplification target area during the live virtual space participation available period, the viewer terminal 300 may automatically and uniquely set both the third area and the fourth area of the virtual audience area as the simplification target area at the end timing of the live virtual space participation available period. By doing so, it is possible to reduce the processing load related to the display output of the viewer avatar compared to the case where only one of the third area and the fourth area is set as the simplification target area.

[0424] Furthermore, in Modification Example 4-3, as shown in FIG. 70, it is possible to set only one of the third area and the fourth area of the virtual audience area as the simplification target area, and by setting both the third area and the fourth area as the simplification target area, it is possible to reduce the processing load related to the display output compared to the case where only one area is set as the simplification target area.

[0425] Also, in the above-described Example 4, a mode in which the simplification target area is determined by the operation of the viewer user regardless of the processing capacity (specifications) of the viewer terminal 300 was illustrated. However, the present invention is not limited to this. As shown in FIGS. 69 and 70 as Modification Example 4-4, the viewer terminal 300 may identify the processing capacity of the viewer terminal 300 and determine the simplification target area without receiving the operation of the viewer user according to the identified processing capacity.

[0426] For example, as shown in FIG. 69, during the live virtual interval participation period, the viewer terminal 300 executes a terminal specification determination process for specifying the processing capabilities (specifications) of the viewer terminal 300, such as the number of clock cycles and cores of the CPU 101 and the memory capacity of the RAM 102 that the viewer terminal 300 has. In the terminal specification determination process, a process of classifying the specified processing capabilities of the viewer terminal 300 into three categories: "medium" which is the average processing capability, "low" which is the processing capability below the average, and "high" which is the processing capability above the average, is also performed.

[0427] Next, the viewer terminal 300 executes a simplification target area determination process for determining the number of areas to be used as the simplification target area from the third area and the fourth area of the virtual audience area. In the simplification target area determination process, as shown in FIG. 70, if the processing capability of the specified viewer terminal 300 is "low", the number of simplification target areas is determined to be two (the third area and the fourth area), and if the processing capability of the specified viewer terminal 300 is "medium", the number of simplification target areas is determined to be one (either the third area or the fourth area). Incidentally, when the processing capability of the specified viewer terminal 300 is "medium", the simplification target area may be randomly determined from the third area and the fourth area, or the simplification target area may be determined from the third area and the fourth area depending on which of the third area and the fourth area the viewer avatar corresponding to the viewer terminal 300 is participating in, or which of the number of viewer avatars participating in the third area and the number of viewer avatars participating in the fourth area is larger.

[0428] After determining the simplification target area in the simplification target area determination process, the viewer terminal 300 transmits simplification target area information including the participant ID (viewer user ID) of the viewer user to the distribution server computer 100.

[0429] When the distribution server computer 100 receives the area information to be simplified from the viewer terminal 300, it specifies the participant ID (viewer user ID) from the area information to be simplified and which area is to be the area to be simplified, and as shown in FIG. 58, executes a participant user table update process of associating the data of the specified area to be simplified with the participant ID and storing it in the participant user table.

[0430] Also, based on having executed the participant user table update process, the distribution server computer 100 executes a participant user table update completion information transmission process of transmitting participant user table update completion information to the viewer terminal 300. Then, based on having received the participant user table update completion information, the viewer terminal 300 executes a participant user table update completion notification process of notifying that the update of the participant user table has been completed in the display device 310, that is, that the registration of which of the third area and the fourth area is to be the area to be simplified has been completed in the viewer terminal 300.

[0431] In this way, the viewer terminal 300 itself specifies the processing ability of the viewer terminal 300, and based on the specified processing ability, sets the area to be simplified without relying on the operation by the viewer user, thereby increasing the degree of freedom of the viewer user in setting the area to be simplified and being able to set an appropriate area to be simplified corresponding to the processing ability of the viewer terminal 300 used by the viewer user without the viewer user himself / herself performing an operation.

[0432] In the above-described Example 4, as shown in FIG. 65, each viewer avatar generated in the viewer perspective video generation process (each viewer avatar generated by the viewer terminal 300) is made into a video of a simplified avatar according to the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process shown in FIG. 62. However, the present invention is not limited to this. As a modification example 4-5, as shown in FIGS. 71(a) to 71(c), each viewer avatar generated in the viewer perspective video generation process is made into a video of a simplified avatar according to the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process shown in FIG. 62. On the other hand, each viewer avatar generated in the virtual camera perspective video generation process (each viewer avatar generated by the distribution server computer 100) may be made into a video of a simplified avatar regardless of the setting of the simplification target area selected by the viewer user in the simplification target area designation operation process. By doing so, the processing load related to the display output of the viewer avatar perspective video on the viewer terminal 300 can be reduced, and in the distribution server computer 100, by generating the virtual camera perspective video with all viewer avatars as simplified avatars, the processing load related to the display output of the virtual camera perspective video can also be reduced.

[0433] Also, the above-described Example 4 illustrates a form in which it is possible to set a simplification target area in which the viewer avatar participating from the third area and the fourth area of the virtual audience area is made into a simplified avatar by the viewer user operating the viewer terminal 300. However, the present invention is not limited to this. As a modification example 4-6, as shown in FIGS. 72(a) and 72(b), when the viewer user operates the viewer terminal 300, as an auxiliary function related to the setting of the simplification target area, a message explaining what settings are possible on the viewer terminal 300 may be displayed on the display device 310. By doing so, the viewer user can accurately perform an operation for setting to make the viewer avatar into a simplified avatar by visually recognizing the message.

[0434] Further, in the example shown in FIG. 72, an example is illustrated in which a message is displayed when a viewer user operates a "HELP" area arranged within the window menu 310WA. However, the present invention is not limited to this, and the message may be displayed without the viewer user operating the "HELP" area.

[0435] Also, in Modification Example 4-6, an example is illustrated in which, as an auxiliary function related to the setting of the area to be simplified, a message explaining what settings can be made on the viewer terminal 300 in the display device 310 is displayed. However, the present invention is not limited to this, and as an auxiliary function related to the setting of the area to be simplified, an operation instruction may be given to the viewer user by displaying an image on the display device 310, outputting sound from a speaker, or the like.

Example

[0436] Next, the virtual space providing system of Example 5 will be described below with reference to FIGS. 73 to 81. Regarding the same configurations as those in other examples including Example 1 and Example 2, the same reference numerals are given and the description thereof is omitted.

[0437] As shown in FIG. 73, the virtual space providing system of Example 5 includes, as a server computer that performs virtual space processing, in addition to the distribution server computer 100, a plurality of sub-venue server computers 400 (sub-venue server computers 400a, 400b,...). These sub-venue server computers 400 are connected to the distribution server computer 100, the administrator terminal 150, the presenter terminal 200, and each viewer terminal 300 via the Internet network N.

[0438] These sub-venue server computers 400 can execute various processes related to the virtual live venue and various processes related to the distribution of virtual live, just like the distribution server computer 100. When the number of viewer avatars participating in the virtual live venue provided by the distribution server computer 100 reaches a predetermined specified number, a new virtual live venue (hereinafter referred to as a sub-virtual live venue) is provided and used to stage a program by a performer avatar, similar to the virtual live venue of the distribution server computer 100 (hereinafter referred to as the main virtual live venue).

[0439] In addition, as shown in FIG. 74, a sub-virtual live venue control program 140 is stored in the storage 103 of the distribution server computer 100 in the fifth embodiment. The distribution server computer 100 uses the sub-virtual live venue control program 140 to determine whether the number of viewer avatars participating in the main virtual live venue or the sub-virtual live venue has reached a predetermined constant number. If it has reached the specified number, the participation of new viewer avatars in the virtual live venue where the number of viewer avatars has reached the specified number is restricted, and control is performed to allocate new viewer avatars to participate in other sub-virtual live venues.

[0440] Also, as shown in FIG. 73, the virtual space providing system of the fifth embodiment has a chat information server computer 170, and these chat information server computers 170 are connected to the distribution server computer 100, the administrator terminal 150, the performer terminal 200, each viewer terminal 300, and each sub-venue server computer 400 via the Internet network N.

[0441] The chat information server computer 170 is a computer for transmitting integrated chat information, in which the chat information input by the viewer user operating an input device such as a keyboard or a touch panel provided in each viewer terminal 300 is sorted in time series, to the performer terminal 200 and each viewer terminal 300.

[0442] Furthermore, the performer terminal 200 and each viewer terminal 300 in the fifth embodiment receive integrated chat information from the chat information server computer 170, and display the same chat information on the display screen.

[0443] Next, the process by which the viewer user virtually participates in the virtual live venue in the fifth embodiment will be described with reference to FIG. 75.

[0444] First, when the distribution server computer 100 receives a participation request from the viewer terminal 300 as in FIG. 22, it executes participant authentication processing. Then, it executes participation virtual live venue determination processing using the sub-virtual live venue control program 140. At this time, the distribution server computer 100 determines the live venue in which the viewer avatar participates.

[0445] Thereafter, as in FIG. 22, the distribution server computer 100 updates the participant user table, performs pre-data transmission processing, the viewer terminal 300 identifies pre-data, transmits a pre-data download request, and receives and stores the pre-data.

[0446] Furthermore, when updating the participant user table after the participation virtual live venue determination processing, the distribution server computer 100 associates and stores data indicating the participation virtual live venue determined in the participation virtual live venue determination processing with the participant ID corresponding to the viewer terminal 300 that transmitted the participation request.

[0447] The specific processing of the participation virtual live venue determination processing will be described with reference to FIG. 77(a). In the following description, for simplicity, the sub-venue server computers constituting the virtual space providing system are described as two (sub-venue server computers 400a and 400b), but the present invention is not limited to this, and the sub-venue server computers constituting the virtual space providing system may be only one or three or more.

[0448] In the participation virtual live determination process, the distribution server computer 100 first determines whether a second sub-virtual live venue participation permission flag indicating that participation of the viewer avatar in the second sub-virtual live venue provided by the sub-venue server computer 400b is permitted is set (step S811). If the second sub-virtual live venue participation permission flag is not set (step S811; No), it is determined whether a first sub-virtual live venue participation permission flag indicating that participation of the viewer avatar in the first sub-virtual live venue provided by the sub-venue server computer 400a is permitted is set (step S812). Note that the first sub-virtual live venue and the second sub-virtual live venue are virtual live venues having a virtual audience area and a virtual stage G, imitating the main virtual live venue.

[0449] If the first sub-virtual live venue participation permission flag is not set (step S812; No), the distribution server computer 100 identifies the number of viewer avatars participating in the main virtual live venue provided by referring to the participant user table shown in FIG. 76 (step S813), and determines whether the number of viewer avatars participating in the main virtual live venue is equal to or greater than a predetermined participation virtual live venue change regulation number (step S814).

[0450] Note that, as shown in FIG. 77(b), in the fifth embodiment, the upper limit number of viewer avatars that can participate in the main virtual live venue is set to 200. On the other hand, the participation virtual live venue change regulation number in the main virtual live venue is set to 150, which is 75% of the upper limit number of viewer avatars that can participate in the main virtual live venue.

[0451] That is, in step S814, the distribution server computer 100 determines whether the number of viewer avatars participating in the main virtual live venue is 150 or more.

[0452] If the number of viewer avatars participating in the main virtual live venue is less than the predetermined number of participants for virtual live venue change (step S814; No), the distribution server computer 100 determines the participating virtual live venue as the main virtual live venue and ends the participating virtual live venue determination process (step S815). Also, if the number of viewer avatars participating in the main virtual live venue is equal to or more than the predetermined number of participants for virtual live venue change (step S814; Yes), the distribution server computer 100 sets the first sub-virtual live venue participation permission flag (step S816), determines the participating virtual live venue as the first sub-virtual live venue, and ends the participating virtual live venue determination process (step S817).

[0453] Also, if the first sub-virtual live venue participation permission flag is set (step S812; Yes), the distribution server computer 100 identifies the number of viewer avatars participating in the first sub-virtual live venue provided by the sub-venue server computer 400a by referring to the participant user table shown in FIG. 76 (step S818), and determines whether the number of viewer avatars participating in the first sub-virtual live venue is equal to or more than the predetermined number of participants for virtual live venue change (step S819).

[0454] Note that, as shown in FIG. 77(b), in the present Example 5, the upper limit number of viewer avatars that can participate in the first sub-virtual live venue is set to 100. On the other hand, the number of participants for virtual live venue change that can be in the first sub-virtual live venue is set to 75, which corresponds to 75% of the upper limit number of viewer avatars that can participate in the first sub-virtual live venue.

[0455] That is, in step S819, the distribution server computer 100 determines whether the number of viewer avatars participating in the first sub-virtual live venue is 75 or more.

[0456] Further, as shown in FIG. 77(b), the upper limit number of viewer avatars that can participate in the second sub-virtual live venue is set to 100, which is the same as the upper limit number of viewer avatars that can participate in the first sub-virtual live venue.

[0457] When the number of viewer avatars participating in the first sub-virtual live venue is less than a predetermined participation virtual live venue change regulation number (step S819; No), the distribution server computer 100 determines the participation virtual live venue as the first virtual live venue and ends the participation virtual live venue determination process (step S820). Also, when the number of viewer avatars participating in the first sub-virtual live venue is equal to or more than the predetermined participation virtual live venue change regulation number (step S819; Yes), the distribution server computer 100 sets the second sub-virtual live venue participation permission flag (step S821), determines the participation virtual live venue as the second sub-virtual live venue, and ends the participation virtual live venue determination process (step S822).

[0458] After the participation virtual live venue determination process, in the participant table update, the distribution server computer 100 stores, in association with the participant ID corresponding to the viewer terminal 300 that has transmitted the participation request, the data indicating the virtual live venue determined in steps S815, S817, S820, and S822.

[0459] Furthermore, in the present Example 5, although it is possible to allow viewer avatars up to the upper limit number to participate in each virtual live venue, in this case, when the viewer avatars operate simultaneously, or when the performer avatar performs intense movements, or when a light emission effect that causes a specific object to emit light is executed as part of the virtual live production, etc., the processing load related to the display of these viewer avatars and performer avatars becomes extremely large, and there is a risk that the distribution of the virtual live will be hindered. For this reason, in the present Example 5, before the number of viewer avatars participating in each virtual live venue reaches the upper limit number, control is performed to allocate newly participating viewer avatars to another virtual live venue, so as to prepare for the above cases where viewer avatars operate simultaneously, performer avatars perform intense movements, a light emission effect that causes a specific object to emit light is executed as part of the virtual live production, etc., and suppress the processing load related to the display of viewer avatars and performer avatars in each server computer, so that the virtual live can be distributed without problems.

[0460] Also, in the present Example 5, the upper limit number (200) of viewer avatars that can participate in the main virtual live venue provided by the distribution server computer 100 is larger than the upper limit number (100) of viewer avatars that can participate in the first sub-virtual live venue provided by the first sub-venue server computer 400a or the second sub-virtual live venue provided by the second sub-venue server computer 400b. That is, in the virtual space providing system of the present invention, when adding a new sub-venue server computer according to the number of viewer avatars participating in the virtual live, since it is only necessary to connect a server computer with a lower processing capacity and lower cost related to the distribution of the virtual live than the distribution server computer 100 each time, there is no need to prepare a server computer with a processing capacity equivalent to that of the distribution server computer 100, and the virtual space providing system can be constructed at low cost.

[0461] Also, in the fifth embodiment, although the upper limit number of viewer avatars that can participate in the main virtual live venue is set to be larger than the upper limit numbers of viewer avatars that can participate in the first sub-virtual live venue and the second sub-virtual live venue, the present invention is not limited thereto, and the upper limit number of viewer avatars that can participate in the main virtual live venue may be the same as the upper limit numbers of viewer avatars that can participate in the first sub-virtual live venue and the second sub-virtual live venue.

[0462] Also, in the fifth embodiment, as shown in FIG. 77(b), the specified numbers of virtual live venue changes that can be participated in the main virtual live venue and the first sub-virtual live venue are both set to 75% of the maximum number of viewer avatars that can participate in each virtual live venue. However, the present invention is not limited thereto, and the ratio of the specified number of virtual live venue changes to the maximum number of viewer avatars that can participate in the main virtual live venue and the ratio of the specified number of virtual live venue changes to the maximum number of viewer avatars that can participate in the first sub-virtual live venue may be different.

[0463] By executing the participation virtual live venue determination process as described above, after the live virtual space participation period ends, the distribution server computer 100 executes virtual space update processing, virtual space update data generation processing, and virtual space update data distribution processing in the same manner as in FIG. 10, and transmits virtual space update data to the viewer terminals 300 and the performer terminal 200 of the viewer avatars participating in the main virtual live venue. In addition, the distribution server computer 100 executes music playback processing, distribution voice generation processing, and generated voice distribution processing in the same manner as in FIG. 10.

[0464] On the other hand, as shown in FIG. 78, after the live virtual space participation period ends, when the distribution server computer 100 receives the presenter avatar information from the presenter terminal 200, it executes a virtual stage area space update process that updates only the data in the area of the virtual stage G among the virtual space data. This reflects the actions (operations) of the presenter avatar on the virtual stage G. Then, based on the data in the area of the virtual stage G before the update and the data in the area of the virtual stage G after the update, virtual stage area update data is generated, and the generated virtual stage area update data is transferred to the sub-venue server computers 400a and 400b (virtual stage area space update data generation and transmission process).

[0465] Also, when the distribution server computer 100 receives the viewer avatar information from the viewer terminal 300 where the viewer avatar is participating in the first sub-virtual live venue or the second sub-virtual live venue, by referring to the participant user table, it identifies the sub-virtual live venue in which the viewer avatar of the viewer terminal 300 is participating from the participant ID included in the received viewer avatar information, and executes a viewer avatar information transfer process that transfers the viewer avatar information to the sub-venue server computers 400a and 400b corresponding to the identified sub-virtual live venue.

[0466] That is, regardless of the virtual live venue in which the viewer avatar is participating, the distribution server computer 100 always receives the viewer avatar information from each viewer terminal 300. And when the received viewer avatar information is from a viewer terminal 300 where the viewer avatar is participating in the first sub-virtual live venue or the second sub-virtual live venue, the distribution server computer 100 transfers the viewer avatar information to the sub-venue server computers 400a and 400b corresponding to the sub-virtual live venue in which the viewer avatar is participating. For this reason, in the fifth embodiment, it is possible to grasp the total number of viewer avatars (total number of viewer users) participating in the virtual live and the actions of each viewer avatar by the distribution server computer 100.

[0467] When the sub-venue server computers 400a and 400b receive virtual stage area space update data from the distribution server computer 100, they execute virtual stage area space reflection processing to reflect the content of the virtual stage area space update data in the actions (operations) of the performer avatars on the virtual stage G in the first sub-virtual live venue or the second sub-virtual live venue.

[0468] Also, when the sub-venue server computers 400a and 400b receive viewer avatar information from the distribution server computer 100, they execute viewer area space update processing to reflect the actions of each viewer avatar in the virtual audience area (viewer area space) based on the viewer avatar information.

[0469] Then, after the sub-venue server computers 400a and 400b execute the above-described virtual stage area space reflection processing and viewer area space update processing, they execute virtual space update data generation processing to generate virtual space update data based on the data of the area of the virtual stage G before update and the data of the area of the virtual stage G after update, and the data of the virtual audience area (viewer area space) before update and the data of the virtual audience area (viewer area space) after update. After that, they execute virtual space update data distribution processing to distribute the generated virtual space update data to the viewer terminals 300 and the performer terminals 200.

[0470] In the viewer terminal 300 to which the virtual space update data is distributed from the sub-venue server computers 400a and 400b, similar to FIG. 10, based on the distributed virtual space update data, the pre-stored virtual space data is updated, and the performer viewpoint video generation processing and viewer viewpoint video generation processing using the updated virtual space data are executed. As a result, the performer viewpoint video and viewer viewpoint video based on the updated virtual space data are generated, and the performer viewpoint video and viewer viewpoint video in the first sub-virtual live venue or the second sub-virtual live venue thus generated are displayed on the performer terminal 200 and the viewer terminal 300 (viewer viewpoint video output processing).

[0471] Furthermore, virtual space update data is not distributed to the presenter terminal 200 of the fifth embodiment from the sub-venue server computers 400a and 400b. Therefore, while the image of the main virtual live venue is displayed on the presenter terminal 200, the image of the sub-virtual live venue is not displayed.

[0472] Furthermore, in the example shown in FIG. 78, similar to FIG. 10, the distribution server computer 100 executes music playback processing, distribution voice generation processing, and generated voice distribution processing, and the presenter terminal 200 executes reproduced music output processing, presenter voice input processing, monitor voice generation processing, and monitor voice output processing, so that the viewer terminal 300 executes voice output processing, which is not shown in the figure.

[0473] Next, the exchange of chat information and integrated chat information among the presenter terminal 200, each viewer terminal 300, and the chat information server computer 170 will be described.

[0474] As shown in FIG. 79, in the viewer terminal 300 where the viewer avatar is participating in the main virtual live venue or the viewer terminal 300 where the viewer avatar is participating in the sub-virtual live venue, a chat information input reception process is executed to receive input of chat information from the viewer user using an input device such as a keyboard or touch panel provided in each viewer terminal 300. At this time, when the input of chat information is received, the viewer terminal 300 executes a chat information transmission process of transmitting the input chat information to the chat information server computer 170.

[0475] When the chat information server computer 170 receives chat information from each viewer terminal 300, it executes an integrated chat information generation process of generating integrated chat information in which the received chat information is arranged in time series, and transmits the generated integrated chat information to the presenter terminal 200 and each viewer terminal 300.

[0476] Then, in the presenter terminal 200 and each viewer terminal 300, as shown in FIG. 80, based on receiving the integrated chat information, an integrated chat information display process is executed to display the same integrated chat information in a chat displ...

Claims

1. A virtual space providing system that provides a virtual space in which a user avatar corresponding to a user can participate, at least a user terminal capable of outputting an avatar viewpoint video of the virtual space seen from the user avatar, a server computer that is connected to the user terminal via a communication network so as to be capable of data communication, and that can execute at least processing related to the virtual space and processing related to video distribution of the virtual space, including at least, the virtual space is settable without providing an object surrounding the target space as the target space of the event, the user avatar includes a first mode avatar and a second mode avatar that is a mode with a smaller processing load for display output than the first mode avatar, within the area of the target space, the user avatar is the second mode avatar, when the avatar viewpoint video seen from the user avatar located within the area of the target space includes the user avatar located outside the area of the target space, the mode of the user avatar is the second mode avatar A virtual space providing system characterized by this.

2. The mode of the second mode avatar is different between the user avatar located within the area of the target space and the user avatar located outside the area of the target space The virtual space providing system according to claim 1, characterized by this.

3. The degree of reduction in the processing load for display output of the mode of the user avatar located outside the area of the target space is greater than that of the mode of the user avatar located within the area of the target space The virtual space providing system according to claim 2, characterized by this.

4. When the avatar viewpoint video seen from the user avatar located within the area of the target space includes the user avatar located outside the area of the target space, the user avatar is made non-displayed The virtual space providing system according to claim 1, characterized by this.

5. In the user terminal used by the user corresponding to the user avatar located outside the area of the target space, the virtual space video within the area of the target space cannot be output The virtual space providing system according to claim 1, characterized by this.

6. The user avatar, is made movable inside the virtual space by the user operating the user terminal, By moving from outside the area of the target space into the area, it is possible to participate in an event set in the target space, and when moving from outside the area of the target space into the area, it is provided with participation confirmation means for performing participation confirmation for the user regarding the event set in the target space. The virtual space providing system according to claim 1, characterized in that.

7. Proximity user avatar specifying means for specifying the user avatar located in a proximity area outside the area of the target space and close to the target space, and event notification means for notifying the user terminal of the user corresponding to the user avatar specified by the proximity user avatar specifying means of information regarding the holding of an event corresponding to the target space. Comprising The virtual space providing system according to claim 1, characterized in that.

8. A terminal data - communicably connected to the server computer via a communication network, being an actor terminal available for an actor who participates in the virtual space with an actor avatar performing a performance in an event, and a terminal data - communicably connected to the server computer via a communication network, being an administrator terminal available for an administrator who manages the virtual space, Including In the actor terminal, an actor can execute an event holding request operation for requesting the holding of an event at the location where the actor avatar is located in the virtual space, In the administrator terminal, in response to the event holding request by the event holding request operation, an area including the location where the actor avatar of the actor who performed the event holding request operation is located can be set as the target space of the event. The virtual space providing system according to claim 1, characterized in that.

9. In the actor terminal, an actor can specify the participation conditions for the event targeted by the event holding request operation. The virtual space providing system according to claim 8, characterized in that.

10. A virtual space providing program for providing a virtual space in which a user avatar corresponding to a user can participate, wherein the video of the virtual space output on a user terminal, which is a terminal data - communicably connected to a server computer via a communication network and capable of executing at least processing related to the virtual space and processing related to video distribution of the virtual space and available for the user, includes an avatar - viewpoint video of the virtual space seen from the user's user avatar. The virtual space can be set without providing an object that encloses the target space of the event. The user avatar includes a first-mode avatar and a second-mode avatar, which has a smaller processing load for display output than the first-mode avatar. Within the region of the target space, the user avatar is set as the second-mode avatar, and when the avatar viewpoint video seen from the user avatar located within the region of the target space includes the user avatar located outside the region of the target space, an avatar mode control process is included to set the mode of the user avatar as the second-mode avatar. A virtual space providing program characterized by the above.

11. A virtual space providing method for providing a virtual space in which a user avatar corresponding to a user can participate, A terminal that is data communicably connected via a communication network to a server computer capable of executing at least processing related to the virtual space and processing related to video distribution of the virtual space, and the video of the virtual space output on the user terminal available to the user includes an avatar viewpoint video of the virtual space seen from the user avatar of the user. The virtual space can be set without providing an object that encloses the target space of the event. The user avatar includes a first-mode avatar and a second-mode avatar, which has a smaller processing load for display output than the first-mode avatar. Within the region of the target space, the user avatar is set as the second-mode avatar, and when the avatar viewpoint video seen from the user avatar located within the region of the target space includes the user avatar located outside the region of the target space, an avatar mode control procedure is included to set the mode of the user avatar as the second-mode avatar. A virtual space providing method characterized by the above.

Images