Video distribution system, video distribution method, and video distribution program for live distribution of video including animation of character objects generated based on the movement of a distribution user

Abstract

To provide a video distribution system for live distributing video containing a virtual space and an animation of a character object generated on the basis of a motion of a distributor user.SOLUTION: In a video distribution system 1, a computer processor of a server device generates video in which a first avatar is featured with a character object on receipt of a display request from a first user while the first avatar is arranged in a first region.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosure in this specification relates to a video distribution system, a video distribution method, and a video distribution program that live-distribute videos including animations of character objects generated based on the movements of a broadcasting user. [Background technology]

[0002] A video distribution system has been known in the past that generates animation of a character object based on the movements of a broadcasting user and live-streams a video including the animation of the character object. Such a video distribution system is disclosed, for example, in Japanese Patent Application Laid-Open No. 2015-184689 (Patent Document 1) and "Yusuke Hasegawa, The Dawn of a New Era of Virtual Idols, PROJECT Maria, CG WORLD, Japan, Born Digital Co., Ltd., November 10, 2017, Vol. 231, pp. 74-79" (Non-Patent Document 1).

[0003] As disclosed in Japanese Patent Laid-Open Publication No. 2012-120098 (Patent Document 2), a video distribution system is also known that, in response to a request from a viewing user who is watching a video, displays an image including the viewing user's avatar in the background area of ​​the video being distributed. The viewing user can post their own comments on the video via their avatar. In this way, the viewing user can participate in the video being distributed via their avatar.

[0004] Japanese Patent Application Laid-Open Publication No. 2017-187995 (Patent Document 3) discloses a social media system in which multiple users make voice calls using images of avatars that have entered a virtual room. In the social media system described in this publication, an upper limit is set on the number of avatars that can enter each room. When a request to enter a room is made, the request to enter is permitted if the number of avatars already in the room does not exceed the limit set for the room. When a request to enter a room is permitted, a new avatar is permitted to enter the room. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-184689 [Patent Document 2] Japanese Patent Application Laid-Open No. 2012-120098 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-187995 [Non-patent literature]

[0006] [Non-Patent Document 1] Yusuke Hasegawa, The Dawn of a New Era of Virtual Idols: PROJECT MariA, CG WORLD, Japan, Born Digital Co., Ltd., November 10, 2017, Vol. 231, pp. 74-79 Summary of the Invention [Problem to be solved by the invention]

[0007] In a video distribution system, if an unlimited number of avatars are allowed to participate in a distributed video, the processing load to display a large number of avatars increases. Furthermore, as the number of avatars increases, the processing load for posting comments and gifting via the avatars also increases. If the processing load related to the avatars' actions becomes too high, there is a risk that the playback of the distributed video and the communication functions using the avatars will not be able to be properly provided.

[0008] On the other hand, limiting the number of avatars that can participate in a video being distributed in a distribution system may limit the participation of viewers who are highly engaged with the distribution system or the distributor who distributes the video. In a video distribution system, acquiring and maintaining highly engaged viewers is considered important for revitalizing the community between distributors and viewers and for managing that community in a healthy manner.

[0009] An object of the present disclosure is to provide technical improvements that solve or alleviate at least some of the problems of the prior art described above. A more specific object of the present disclosure is to provide a video distribution system, a video distribution method, and a video distribution program that reduce the processing load caused by the participation of avatars and do not hinder the participation of highly engaged viewers. [Means for solving the problem]

[0010] One aspect of the present invention relates to a video distribution system that live-streams a video including a virtual space and animation of a character object generated based on a broadcaster's movements. The video distribution system includes one or more computer processors that execute computer-readable instructions to determine whether a viewing user belongs to a first group or a second group in response to receiving a participation request from a viewing user who is watching the video, and to place a specific avatar of the viewing user in a first area within the virtual space if the viewing user is determined to belong to the first group, and to place the specific avatar in a second area within the virtual space in response to fulfilling a participation condition if the viewing user is determined to belong to the second group. Place it in.

[0011] In one embodiment, the first region is located closer to the character object in the virtual space than the second region.

[0012] In one embodiment, the participation condition is that the number of avatars of users other than the viewing user displayed in the second area is less than an upper limit of the number of avatars set for the second area.

[0013] In one embodiment, the participation condition is that the viewing user is selected by the broadcasting user.

[0014] In one embodiment, the video distribution system further includes a storage that stores avatar data for displaying the specific avatar in the virtual space, and displays the specific avatar in the virtual space based on the avatar data read from the storage.

[0015] A video distribution system according to one embodiment receives an action instruction for the specific avatar placed in the virtual space, and controls the movement of the specific avatar displayed in the virtual space based on the action instruction.

[0016] In one embodiment, the video distribution system receives a display request for the specific avatar while the specific avatar is located in the virtual space, and, in response to the display request being granted, displays the specific avatar in a third area in which the character object is located.

[0017] In one embodiment, the specific avatar is placed in the third area when the viewing user is selected by the broadcaster user after a request to display the specific avatar is accepted.

[0018] One aspect of the present invention relates to a video distribution method for live-streaming a video including animation of a character object generated based on a broadcast user's movements by executing computer-readable instructions by one or more computer processors, the video distribution method comprising: a step of determining whether a viewing user who watches the video belongs to a first group or a second group in response to receiving a request to participate in the video from the viewing user, placing a specific avatar of the viewing user in a first area within the virtual space if the viewing user is determined to belong to the first group; and a step of placing the specific avatar in a second area within the virtual space in response to fulfilling a participation condition in response to determining that the viewing user belongs to the second group.

[0019] One aspect of the present invention relates to a video distribution program for live-streaming a video including animation of a character object generated based on a broadcaster's movements, the video distribution program causing one or more computer processors to execute the following steps: upon receiving a request from a viewing user who is watching the video to join the video, determine whether the viewing user belongs to a first group or a second group; if it is determined that the viewing user belongs to the first group, place a specific avatar of the viewing user in a first area within the virtual space; and if it is determined that the viewing user belongs to the second group, place the specific avatar in a second area within the virtual space in response to fulfilling a participation condition. [Effects of the Invention]

[0020] According to an embodiment of the present invention, it is possible to suppress the processing load caused by having an avatar participate, and not to hinder the participation of highly engaged viewing users. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a block diagram illustrating a video distribution system according to an embodiment. [Figure 2] FIG. 2 is a schematic diagram showing the equipment of a studio that produces videos to be distributed by the video distribution system of FIG. 1. [Figure 3] 2 is a diagram illustrating user data stored in the video distribution system of FIG. 1. FIG. [Figure 4] FIG. 2 is a diagram illustrating avatar data stored in the video distribution system of FIG. 1. [Figure 5] 2 is a diagram illustrating participation management data stored in the video distribution system of FIG. 1. FIG. [Figure 6] 2 is a diagram illustrating a virtual space displayed on a video distributed in the video distribution system of FIG. 1. FIG. [Figure 7a]7A and 7B are diagrams showing an example of a display screen displayed on the client device 10a in one embodiment, in which a participation request button for allowing an avatar to participate in a video is displayed on the display screen of Fig. 7A. [Figure 7b] 7B is a diagram showing an example of a display screen displayed on the client device 10a in one embodiment. The display screen in FIG. 7B displays a display request button for requesting that an avatar be displayed together with the broadcast user's character. [Figure 7c] FIG. 10 is a diagram showing an example of a display screen including an image of a virtual space captured by another virtual camera. [Figure 7d] FIG. 10 is a diagram showing an example of a display screen including an image of a virtual space captured by yet another virtual camera. [Figure 7e] FIG. 10 is a diagram showing an example of a display screen indicating that a feature display request has been made. [Figure 7f] FIG. 10 is a diagram showing an example of a display screen on which an avatar and a broadcast user's character are displayed together. [Figure 8] FIG. 10 is a flowchart showing the flow of a video distribution process in one embodiment. [Figure 9] FIG. 10 is a flow diagram illustrating processing an avatar's join request in one embodiment. [Figure 10] FIG. 10 is a flow diagram illustrating a process for processing a request to display an avatar in an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] Various embodiments of the present invention will be described below with appropriate reference to the drawings, in which the same reference numerals are used to designate the same or similar components throughout the drawings.

[0023] A video distribution system according to one embodiment will be described with reference to Figures 1 to 6. Figure 1 is a block diagram showing a video distribution system 1 according to one embodiment, Figure 2 is a schematic diagram showing studio equipment where videos distributed by the video distribution system 1 are produced, Figures 3 to 5 are diagrams for explaining information stored in the video distribution system 1, and Figure 6 is a diagram showing a virtual space displayed in the video.

[0024] The video distribution system 1 includes client devices 10a to 10c, a server device 20, a studio unit 30, and a storage 25. The client devices 10a to 10c, the server device 20, and the storage 25 are connected to each other via a network 15 so that they can communicate with each other. As will be described later, the server device 20 is configured to distribute video including character animation. The characters included in this video may be motion-controlled within a virtual space.

[0025] Videos may be distributed from server device 20 to each of client devices 10a to 10c. A first viewing user who is a user of client device 10a, a second viewing user who is a user of client device 10b, and a third viewing user who is a user of client device 10c can view the distributed videos on their respective client devices. Video distribution system 1 may include fewer than three client devices or more than three client devices. Client devices 10a to 10c and other users who view videos distributed from server device 20 using client devices are collectively referred to as viewing users. Videos distributed from server device 20 may be viewed by client devices 10a to 10c that are capable of executing application software for viewing the videos. To view videos distributed from server device 20, application software having an instruction set that realizes the function of viewing the videos may be downloaded and installed on client devices 10a to 10c. Such application software having the function of viewing videos distributed from server device 20 may be referred to as "viewing application software" or "viewing app." Viewing of videos distributed from server device 20 may be performed by executing a function of a viewing app. In one embodiment, viewing of videos distributed from server device 20 is performed by a client device among client devices 10a to 10c on which a viewing app is installed, and may not be performed by a client device on which a viewing app is not installed. Viewing users who watch videos distributed from server device 20 can display their own avatars in the videos. In other words, viewing users can participate in the distributed videos through their own avatars. Viewing users can display their own avatars in the distributed videos and interact with the distributor of the video (or characters in the distributed videos) and other viewing users through the avatars.

[0026] The client devices 10a to 10c are information processing devices such as smartphones. In addition to smartphones, the client devices 10a to 10c may also be various other information processing devices capable of playing videos, such as mobile phones, tablet devices, personal computers, e-book readers, wearable computers, game consoles, and other types of information processing devices. Each of the client devices 10a to 10c may include a computer processor, memory, a communication I / F, a display, a sensor unit including various sensors such as a gyro sensor, a sound collection device such as a microphone, and storage for storing various information.

[0027] The display of each of the client devices 10a to 10c includes a display panel and a touch panel. The touch panel is configured to be able to detect touch operations (contact operations) by the player. The touch panel can detect various touch operations by the player, such as tapping, double tapping, and dragging. The touch panel may also be configured to include a capacitance-type proximity sensor and be able to detect non-contact operations by the player.

[0028] The viewing user can input a message related to the distributed video or other messages via the touch panel of the client device 10a to 10c. This message may be, for example, a text message, a still image message, a video message, or any other electronic message format. Each of the client devices 10a to 10c may be configured to post (transmit) the message input by the viewing user to the server device 20.

[0029] In the illustrated embodiment, the server device 20 includes a computer processor 21, a communication I / F 22, and a storage 23.

[0030] The computer processor 21 is an arithmetic device that loads an operating system and various programs that realize various functions from the storage 23 or other storage into memory and executes instructions contained in the loaded programs. The computer processor 21 is, for example, a CPU, an MPU, a DSP, a GPU, or various other arithmetic devices, or a combination thereof. The computer processor 21 may be implemented by an integrated circuit such as an ASIC, a PLD, an FPGA, or an MCU. Although FIG. 1 illustrates the computer processor 21 as a single component, the computer processor 21 may be a collection of multiple physically separate computer processors. In this specification, a program or instructions contained in the program described as being executed by the computer processor 21 may be executed by a single computer processor or may be distributed and executed by multiple computer processors. Furthermore, the program or instructions contained in the program executed by the computer processor 21 may be executed by multiple virtual computer processors. The functions implemented by the computer processor 21 are described below.

[0031] The communication I / F 22 is implemented as hardware, firmware, or communication software such as a TCP / IP driver or a PPP driver, or a combination of these. The server device 20 can send and receive data to and from other devices via the communication I / F 22.

[0032] The storage 23 is a storage device accessed by the computer processor 21. The storage 23 is, for example, a magnetic disk, an optical disk, a semiconductor memory, or any other storage device capable of storing data. Various programs can be stored in the storage 23. At least some of the programs and various data that can be stored in the storage 23 may be stored in a storage (for example, storage 25) that is physically separate from the server device 20. Details of the data stored in the storage 23 will be described later.

[0033] Many of the components of the studio unit 30 are arranged, for example, in a studio room R shown in FIG. 2. As shown in the figure, an actor A1 is performing in the studio room R. The studio unit 30 is configured to detect the movements and facial expressions of the actor A1 and output the detected information to the server device 20.

[0034] Actor A1 is a subject whose movements and facial expressions are captured by a group of sensors (described below) provided in studio unit 30. In the illustrated embodiment, actor A1 is a human who talks, performs, and takes other actions to progress a program that is distributed as a video. Actor A1 may be a human, an animal, or any other moving object (inanimate object). Actor A1 may be, for example, a robot capable of autonomous movement. The number of actors in studio room R may be one, or three or more.

[0035] The studio unit 30 includes six motion sensors 31a to 31f attached to actor A1, a controller 33a held in actor A1's left hand, a controller 33b held in actor A1's right hand, a camera 37a attached to actor A1's head via a wearing device 37b, and a wristwatch-type actor device 91 attached to actor A1's left arm. A microphone for acquiring audio data may be attached to the wearing device 37b. This microphone can acquire the speech of actor A1 as audio data. The microphone may be a wearable microphone attached to actor A1 via the wearing device 37b, or may be a stationary microphone installed on the floor, wall, or ceiling of studio room R. In addition to the above components, the studio unit 30 also includes a base station 35a, a base station 35b, a digital camera 36a, and a display 39. A supporter computer 40 is installed in an adjacent room separated from studio room R by a glass window. The server device 20 may be installed in the same room as the supporter computer 40.

[0036] The motion sensors 31a to 31f detect their own position and orientation in cooperation with the base stations 35a and 35b. In one embodiment, the base stations 35a and 35b are multi-axis laser emitters. The base station 35a emits a synchronous flashing light and then scans the laser light, for example, around a vertical axis. The base station 35b scans the laser light, for example, around a horizontal axis. Each of the motion sensors 31a to 31f may include multiple optical sensors that detect the incidence of the flashing light and laser light from the base stations 35a and 35b. Each of the motion sensors 31a to 31f may detect its own position and orientation based on the time difference between the incidence of the flashing light and the incidence of the laser light, the light reception time at each optical sensor, the incidence angle of the laser light detected by each optical sensor, and other information as needed. The motion sensors 31a to 31f may be, for example, a Vive Tracker provided by HTC CORPORATION. Base station 35a and base station 35b may be, for example, base stations provided by HTC CORPORATION.

[0037] Detection information indicating the position and orientation of each motion sensor calculated by each of the motion sensors 31a to 31f is transmitted to the server device 20. This detection information may be wirelessly transmitted from each of the motion sensors 31a to 31f to the server device 20. Since the base station 35a and the base station 35b emit flashing light and perform laser light scanning at regular intervals, the detection information of each motion sensor is updated at each interval. The number of base stations may be three or more. The positions of the base stations may be changed as appropriate. For example, in addition to or instead of the base stations arranged in the upper corners of the space to be detected by the tracking sensors shown in FIG. 2, paired base stations may be provided at an upper position and a lower position close to the floor. For example, the number of base stations may be four.

[0038] In the illustrated embodiment, actor A1 wears six motion sensors 31a to 31f. The motion sensors 31a, 31b, 31c, 31d, 31e, and 31f are attached to actor A1's left back of the hand, right back of the hand, left instep, right instep, waist, and top of the head, respectively. The motion sensors 31a to 31f may be attached to actor A1 via a wearing device. The motion sensors 31a to 31f shown in FIG. 2 are exemplary. The motion sensors 31a to 31f may be attached to various parts of actor A1. The number of motion sensors attached to actor A1 may be five or less or seven or more. In this way, the body movement of actor A1 can be detected by detecting the positions and orientations of the motion sensors 31a to 31f attached to various parts of actor A1's body. The body movement of actor A1 may be detected by an optical motion capture system.

[0039] In one embodiment, multiple motion sensors attached to actor A1 may each be equipped with multiple infrared LEDs, and the position and orientation of each motion sensor may be detected by detecting light from these infrared LEDs with an infrared camera installed on the floor or wall of studio room R. Visible light LEDs may be used instead of infrared LEDs, and the position and orientation of each motion sensor may be detected by detecting light from these visible light LEDs with a visible light camera. In this way, each of multiple motion sensors attached to the actor may be provided with a light-emitting unit (e.g., infrared LEDs or visible light LEDs), and the position and orientation of each motion sensor may be detected by detecting light from these light-emitting units with a light-receiving unit (e.g., infrared camera or visible light camera) installed in studio room R.

[0040] In one embodiment, a plurality of reflective markers can be used instead of the motion sensors 31a to 31f. The reflective markers are attached to each actor A1 using adhesive tape or the like. The actor A1 to which the reflective markers are attached is photographed to generate photographic data, and the photographic data is then image-processed to detect the position and orientation of the reflective markers. In one embodiment, the body movement of the actor A1 can be detected using a suit with a built-in inertial sensor. The movement of the actor A1 can be detected by analyzing the sensor signal output from the inertial sensor of the inertial sensor-embedded suit worn by the actor A1. The suit with a built-in inertial sensor can be a suit with a built-in inertial sensor compatible with a motion capture system called MVN developed by Xsens.com.

[0041] The controller 33a and the controller 33b output to the server device 20 a control signal according to the operation of the actor A1.

[0042] The digital camera 36a captures an image of the actor A1 and generates image data. The position, image capturing direction, and angle of view of the digital camera 36a are set in response to an operation by the operator. The digital camera 36a transmits image capturing setting information indicating its position, image capturing direction, and angle of view to the server device 20. Similarly, the position, image capturing direction, and angle of view of the digital camera 36b are set in response to an operation by the operator. In the illustrated embodiment, the digital camera 36a is provided to the left front of the actor A1.

[0043] As described above, the camera 37a is attached to the head of the actor A1. For example, the camera 37a is positioned so as to capture an image of the face of the actor A1. The camera 37a continuously captures images of the face of the actor A1 and acquires image data of the face of the actor A1. The camera 37a transmits the image data of the face of the actor A1 to the server device 20. The camera 37a may be a 3D camera capable of detecting the depth of a person's face. The camera 37a may be, for example, a LIDAR device provided in an iPhone X provided by Apple Inc. The iPhone X is attached, for example, to headgear worn on the head of the actor A1. The camera 37a may be a fixed type camera mounted on a camera stand installed in the studio room R, rather than being attached to the actors A1 and A2.

[0044] The display 39 is configured to display information received from the support computer 40. Information sent from the support computer 40 to the display 39 may include, for example, text information, image information, and various other types of information. The display 39 is placed in a position visible to the actor A1.

[0045] The actor device 91 is configured to generate a stimulus other than sound that can be perceived by the actor A1 in response to a drive signal from the server device 20. If the actor device 91 generates a sound, there is a risk that the sound will be mixed into the distributed content. Therefore, the actor device 91 is configured to generate a stimulus other than sound. The actor device 91 can generate a stimulus other than an auditory stimulus (typically, sound), for example, a tactile stimulus, a visual stimulus, or a stimulus that can be perceived by the actor A1 that is different from other auditory stimuli. The actor device 91 may include an actuator 91a. The actuator 91a is configured to be activated by a drive signal from the server device 20. The actuator 91a may be configured to vibrate when a drive signal is input. For example, an Apple Watch provided by Apple Inc. may be used as the actor device 91. The actor device 91 may have various shapes other than a wristwatch shape. The actor A1 can wear multiple actor devices 91 of the same type or different types. The actor device 91 may be configured to be activated in response to a drive signal from a source other than the server device 20. The multiple actor devices 91 may be associated with an actor ID that identifies actor A1. The actor devices 91 can transmit tactile stimuli to actor A1 via actuators 91a. The actor devices 91 may be LEDs (Light Emitting Diodes) or sensory display devices. The LEDs or sensory display devices can generate visual stimuli. The sensory display device may be an electrotactile display that drives nerve axons under the skin with current flowing from electrodes.

[0046] In the illustrated embodiment, the supporter computer 40 is installed in a room next to the studio room R. The room in which the supporter computer 40 is installed is separated from the studio room R by a glass window, so the operator of the supporter computer 40 (sometimes referred to as a "supporter" in this specification) can see the actor A1. In the illustrated embodiment, supporters B1 and B2 are present in the room as operators of the supporter computer 40.

[0047] The supporter computer 40 may be configured to change the settings of the components of the studio unit 30 in response to operations by the supporters B1 and B2. The supporter computer 40 can, for example, set the scanning intervals of the base stations 35a and 35b, set the positions or orientations of the digital cameras 36a and tracking sensors 36b, and change various settings of various other devices. The switching of active sensors, which will be described later, can be performed based on operations of the supporter computer 40 by the supporters B1 and B2. At least one of the supporters B1 and B2 can input a message to the supporter computer 40 and display the input message on the display 39.

[0048] The components and functions of the studio unit 30 shown in Figure 2 are exemplary. A studio unit 30 applicable to the present invention may include various components not shown. The studio unit 30 may include a projector. The projector can project video onto a screen S to be distributed to the client device 10a or other client devices.

[0049] On the other hand, filming for generating videos to be distributed may be performed using simpler equipment. For example, facial motion data, which is a digital representation of an actor's facial movements, may be generated using an information processing terminal such as a smartphone. Based on the facial motion data, an animation of a character object may be generated in which the facial expressions of the actor move in synchronization with the actor's facial movements but the body remains stationary. Such facial motion data may be generated by client device 10a-client device 10c. To generate the facial motion data, application software having an instruction set realizing the function of generating facial motion data may be downloaded and installed on client device 10a-client device 10c. Such application software capable of generating facial motion data may be referred to as "distribution application software" or "distribution app." The distribution app and the aforementioned viewing app may be separate application software, or a single application software may have both the functions of the distribution app and the aforementioned viewing app. This facial motion data may include tracking data of the actor's head posture, feature points, which are digital representations of the actor's facial features, and blended face data representing their deformation rates. An information processing terminal such as a smartphone may create an image showing an exaggerated facial expression of the actor based on the facial motion data and superimpose the image on the face of the character object. When generating animation of a character object based on facial motion data, motion sensors 31a, 31b, 31c, 31d, 31e, and 31f for detecting the body movement of the actor, base stations 35a, 35b, and digital cameras 36a, 36b are not required, so animation of a character object can be generated with simpler equipment.If motion sensors 31a, 31b, 31c, 31d, 31e, and 31f, base stations 35a, 35b, and digital camera 36a are not required, character objects can be generated in places other than studio room R, such as the actor's home, thereby increasing convenience.

[0050] Next, a description will be given of one aspect of information stored in the storage 23. In the illustrated embodiment, the storage 23 stores model data 23a, object data 23b, user data 23c, avatar data 23d, participation management data 23e, and various other information required for generating and distributing videos to be distributed.

[0051] The model data 23a is model data for generating character animation. The model data 23a may be three-dimensional model data for generating three-dimensional animation, or two-dimensional model data for generating two-dimensional model data. The model data 23a includes, for example, rig data (sometimes called "skeleton data") that indicates the skeleton of the character, and surface data that indicates the shape and texture of the character's surface. The model data 23a may include multiple different model data. The multiple model data may have different rig data or the same rig data. The multiple model data may have different surface data or the same surface data.

[0052] The object data 23b includes asset data for constructing a virtual space that constitutes a video. The object data 23b includes data for drawing the background of the virtual space that constitutes the video, data for drawing various objects displayed in the video, and data for drawing various other objects displayed in the video. The object data 23b may also include object position information that indicates the position of the object in the virtual space.

[0053] In addition to the above, the object data 23b may include gift objects that are displayed in the video based on display requests from viewing users of the client devices 10a to 10c. The gift objects may include effect objects, normal objects, and decorative objects. Viewing users can purchase desired gift objects.

[0054] An effect object is an object that affects the overall impression of the viewing screen of the distributed video, such as an object resembling confetti. The confetti object may be displayed across the entire viewing screen, thereby changing the overall impression of the viewing screen before and after its display. Effect objects may be displayed so as to overlap with character objects, but are different from decorative objects in that they are not displayed in association with any specific part of the character object.

[0055] A normal object is an object that serves as a digital gift from a viewing user to an actor (e.g., actor A1) or a character in a video, and is, for example, an object resembling a stuffed animal or a bouquet of flowers. In one aspect, a normal object is displayed on the video display screen so as not to contact a character object. In one aspect, a normal object is displayed on the video display screen so as not to overlap with a character object. A normal object may be displayed so as to overlap with an object other than a character object in the virtual space. A normal object may be displayed so as to overlap with a character object, but differs from a decorative object in that it is not displayed in association with a specific part of the character object. In one aspect, when a normal object is displayed so as to overlap with a character object, the normal object is displayed so as to overlap with a part of the character object other than the head, including the face, and not to overlap with the head of the character object.

[0056] A decorative object is an object that is displayed on the display screen in association with a specific part of a character object. In one aspect, a decorative object that is displayed on the display screen in association with a specific part of a character object is displayed on the display screen so as to be in contact with the specific part of the character object. In another aspect, a decorative object that is displayed on the display screen in association with a specific part of a character object is displayed on the display screen so as to cover part or all of the specific part of the character object.

[0057] Decorative objects include, for example, accessories (headbands, necklaces, earrings, etc.) worn by character objects, clothing (T-shirts, etc.), costumes, and other objects that can be worn by character objects. Object data 23b corresponding to decorative objects may include wearing position information indicating which part of the character object the decorative object is associated with. The wearing position information of a certain decorative object may indicate which part of the character object the decorative object is worn on. For example, if the decorative object is a headband, the wearing position information of the decorative object may indicate that the decorative object is worn on the "head" of the character object. If the decorative object is a T-shirt, the wearing position information of the decorative object may indicate that the decorative object is worn on the "torso" of the character object.

[0058] Each gift object may have a display time set according to its type. In one embodiment, the display time of a decorative object may be set longer than the display time of an effect object and the display time of a normal object. For example, the display time of a decorative object may be set to 60 seconds, the display time of an effect object may be set to 5 seconds, and the display time of a normal object may be set to 10 seconds.

[0059] The object data 23b may include a list indicating gift objects owned by users of the video distribution system 1. In the ownership list, object IDs that identify gift objects owned by the viewing users may be stored in association with the user's account information. The user's account information may be, for example, a user ID that identifies the user. The users may include, for example, the first to third viewing users of the client devices 10a to 10c.

[0060] As shown in FIG. 3, the user data 23c can include account information of each user (e.g., a viewing user) of the video distribution system 1, avatar information related to an avatar used by each user, and group information related to a group to which each user belongs. In other words, the storage 23 can store, in association with the account information of each user of the video distribution system 1, avatar information related to an avatar used by each user, group information related to a group to which each user belongs, and information related to other users. The account information of a certain user is, for example, a user ID that identifies the user. The user ID of the user may be issued when the user registers for use of a video service provided by the server device 20 using one of the client devices 10a to 10c. A common user ID may be used when using the service of the server device 20 using a viewing app and when using the service of the server device 20 using a distribution app, or different user IDs may be used when using the service of the server device 20 using a viewing app and when using the service of the server device 20 using a distribution app. If different user IDs are used when using the service of server device 20 using a viewing app and when using the service of server device 20 using a distribution app, the user ID for using the viewing app (viewing user ID) and the user ID for using the distribution app (distribution user ID) of the same user may be stored in association with each other. A service of server device 20 that can be received by using the viewing app is, for example, a service for viewing videos distributed from server device 20. A service of server device 20 that can be received by using the distribution app is, for example, a service for distributing videos including the user's own character object.

[0061] The avatar information of a user is, for example, an avatar ID that identifies an avatar used by the user in the video system 1. The avatar ID is assigned to the user when the user registers an avatar. The avatar is registered by the user selecting the head, hairstyle, clothes, accessories, and other body parts that constitute the avatar. The avatar may be registered by the client devices 10a to 10c on which a distribution application is installed. That is, the avatar may be registered by executing a function of the distribution application. In one embodiment, the avatar is registered by one of the client devices 10a to 10c on which the distribution application is installed, and may not be executable by a client device on which the distribution application is not installed. The avatar may be registered by application software (hereinafter, sometimes referred to as an "avatar creation application") that has a function of creating, editing, and / or registering an avatar, separate from the distribution application. When an avatar is registered using a distribution app and / or an avatar creation app, an avatar ID for identifying the avatar may be assigned and the avatar ID may be associated with the user's user ID and managed as user information, as described with reference to FIG. 3. After registering an avatar, the user may change the avatar. For example, after registering an avatar, the user may change some or all of the features of the avatar. For example, even after registration, the user may change the avatar's hairstyle, possessed items, and other features that make up the avatar.

[0062] The group information of a user is a group ID that identifies the group to which the user belongs. Groups to which a user of the video distribution system 1 can belong include, for example, a first group and a second group. A user of the distribution system 1 belongs to either the first group or the second group. The second group is a group different from the first group. Whether a user belongs to the first group or the second group can be determined in various ways. A group ID does not necessarily have to be used to determine whether a user belongs to the first group or the second group. For example, in user data, a flag can be set in the record of a user belonging to the first group, and a flag cannot be set in the record of a user belonging to the second group, thereby determining whether a user belongs to the first group or the second group. Which group a user belongs to can be determined based on various criteria. For example, a user with relatively high engagement with the video system 1 can belong to the first group, and a user with relatively low engagement with the video system 1 can belong to the second group. A user's engagement with the video system 1 can be expressed, for example, in points. When a user uses the video system 1, the user's points may increase or decrease depending on the manner of use. A user's points may increase, for example, by watching a video, streaming a video, posting a comment on a video, gifting a video streamer, purchasing a gift (gift object), logging in to the video system 1, launching application software for watching a video, sending a message to another user on a social networking service provided by the video system 1, receiving a positive rating from another user on the social networking service, and other uses of the video system 1. A user's points may decrease, for example, by being prohibited from viewing (blocked) by a video streamer, having a posted comment deleted, receiving a negative rating from another user on the social networking service provided by the video system 1, and other uses of the video system 1.User actions associated with point increases and decreases are not limited to those described above. A user's points may increase due to actions by the user that strengthen the user's connection with the broadcasting user or other viewing users, and conversely, may decrease due to actions by the user that weaken the user's connection with the broadcasting user or other viewing users.

[0063] In one embodiment, the first group is composed of fewer users than the second group. A user may be classified into the second group when registering for use with the video distribution system 1, and may leave the second group and belong to the first group if the user meets the criteria for being classified into the first group after starting to use the video distribution system 1. Grouping may not be performed when registering for use with the video distribution system 1, and only users who meet certain conditions after starting to use the system may be assigned to the first group. Whether a user belongs to the first group can be determined, for example, by setting a flag in the user data to identify users who belong to the first group and referencing the flag. By assigning users who have just started using the system to the second group, the number of users belonging to the first group can be made smaller than the number of users belonging to the second group. A user may change the group to which they belong depending on how they use the video distribution system 1. If a user no longer meets the criteria for being classified into the first group, they may be transferred from the first group to the second group.

[0064] In one aspect, the classification criteria for the first group may be adjusted so that the number of users belonging to the first group remains smaller than the number of users belonging to the second group. For example, the server device 20 may monitor the number of users belonging to the first group and the number of users belonging to the second group, and if the difference between the numbers becomes smaller than a predetermined number, the criteria for classification into the first group may be tightened. For example, if initially, users with 100 points or more are classified into the first group, when the difference between the number of users belonging to the first group and the number of users belonging to the second group becomes smaller than a predetermined value, the criteria may be changed so that users with 200 points or more are classified into the first group.

[0065] In one embodiment, an upper limit on the number of users who can belong to the first group may be set. In this case, a ranking of points may be created for users of the video distribution system 1, and the users at the top of the ranking up to the upper limit may belong to the first group. Since not all users at the top of the ranking are necessarily logged in to the video distribution system 1, a ranking of users who are logged in to the video system 1 may be created, and the users at the top of the ranking up to the upper limit may belong to the first group.

[0066] The points of each user referenced for grouping may be common points common to the entire video system 1, or individual points managed for each broadcast user. For example, for a given user, individual points for broadcast user A and individual points for broadcast user B may be managed separately. For videos broadcast by broadcaster A (i.e., videos including broadcaster A's character object), a ranking of the individual points managed for broadcaster A may be created, and the users at the top of the ranking up to the upper limit may be placed in a first group. In this way, a user who belongs to a second group in the ranking of common points common to the entire video system 1 may be placed in the first group for a specific broadcast user. By managing individual points in this manner, broadcast users can be grouped according to each user's individual level of engagement with the broadcast user.

[0067] As described above, if a user's points are higher than a predetermined threshold, the user is considered to belong to the first group, and conversely, if the user's points are lower than the predetermined threshold, the user is considered to belong to the second group. This allows users with points higher than the threshold to belong to the first group, and users with points lower than the threshold to belong to the second group. If points represent engagement with the video system 1, users with relatively high engagement with the video system 1 can belong to the first group, and users with relatively high engagement with the video system 1 can belong to the second group.

[0068] The grouping of users is not limited to those specifically described in this specification. Users may be classified into three or more groups. Furthermore, the classification criteria for determining which group a user belongs to may be changed as appropriate.

[0069] As shown in FIG. 4, the avatar data 23d can include avatar identification information of an avatar used by a user in the video distribution system 1 and avatar display information for displaying the avatar in a video. In other words, the storage 23 can store avatar display information for displaying an avatar in association with the avatar identification information of the avatar. The avatar identification information is, for example, an avatar ID for identifying an avatar. The avatar is displayed in a video as, for example, an image resembling a human or an animal. The avatar display information is information used to display the avatar in the video. The avatar information includes, for example, part information indicating images of the head, hairstyle, facial features (eyes, nose, mouth, etc.), torso, clothing, accessories, items, and other parts that make up the avatar. A user can register their own avatar by selecting a preferred part image. The avatar display information may include 2D display information for displaying an avatar in 2D in a video and 3D display information for displaying an avatar in 3D in a video. The 3D display information includes parts information indicating images of parts for displaying an avatar three-dimensionally in a video, rig data for expressing the avatar's three-dimensional movements, and other known information used to display an avatar three-dimensionally.

[0070] Viewing users can have their own avatars participate in videos distributed by the video distribution system 1. The participation management data 23e stored in the storage 23 is data for managing avatars participating in a video. As shown in FIG. 5, the participation management data 23e may include video identification information for identifying a video distributed by the video distribution system 1, participation restriction information for determining the upper limit of the number of avatars that can participate in the video, and participating avatar information for indicating the avatars participating in the video. The video identification information is, for example, a video ID for identifying the video. The participation restriction information for a certain video is, for example, the upper limit of the number of avatars that can participate in the video. The participation restriction information may include a second upper limit number that is the upper limit of the number of avatars that can be placed in a second area of ​​the virtual space, which will be described later. The participation restriction information may include, in addition to the second upper limit number, a first upper limit number that is the upper limit of the number of avatars that can be placed in a first area of ​​the virtual space, which will be described later. The participating avatar information for a certain video may be avatar IDs that identify the avatars participating in the video. When multiple avatars participate in a video, the participating avatar information includes the avatar IDs of each of the multiple avatars.

[0071] Next, the functions realized by the computer processor 21 will be described in more detail. By executing computer-readable instructions included in the distribution program, the computer processor 21 functions as a body motion data generation unit 21a, a facial motion data generation unit 21b, an animation generation unit 21c, a video generation unit 21d, a video distribution unit 21e, a gift processing unit 21f, a participation request processing unit 21g, an avatar display unit 21h, an avatar control unit 21i, and a display request processing unit 21j. At least some of the functions realized by the computer processor 21 may be realized by a computer processor other than the computer processor 21 of the video distribution system 1. At least some of the functions realized by the computer processor 21 may be realized by, for example, a computer processor installed in the supporter computer 40 or another computer processor (not shown).

[0072] The body motion data generator 21a generates body motion data for each part of the actor A1's body based on the detection information from each of the motion sensors 31a to 31f. The body motion data is generated as time passes. For example, the body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data can represent the body movements of the actor A1 as digital data in a time series. In the illustrated embodiment, the motion sensors 31a to 31f are attached to each of the actor A1's left and right limbs, waist, and head. Therefore, based on the detection information from the motion sensors 31a to 31f, it is possible to digitally represent the approximate position and orientation of the entire body of the actor A1 over time. The body motion data can, for example, specify the positions and rotation angles of the bones that make up the rig data included in the model data 23a.

[0073] The facial motion data generator 21b generates facial motion data, which is a digital representation of the facial movements of actor A1, based on the image data captured by the camera 37a. The facial motion data is generated as time passes. For example, the facial motion data may be generated at predetermined sampling time intervals. In this way, the facial motion data can digitally represent the facial movements (changes in facial expressions) of actor A1 in time series.

[0074] The animation generation unit 21c is configured to generate animation of a character object that moves in a virtual space and whose facial expression changes by applying the body motion data generated by the body motion data generation unit 21a and the facial motion data generated by the facial motion data generation unit 21b to predetermined model data included in the model data 23a. Specifically, the animation generation unit 21c can generate animation of a character object that moves in synchronization with the body and facial movement of actor A1, based on the body motion data and facial motion data related to actor A1.

[0075] The video generator 21d constructs a virtual space using the object data 23b, and generates a video including the virtual space and animation of a character object corresponding to the actor A1. The character object of the actor A1 may be positioned in the virtual space so as to match the position of the actor A1 relative to the digital camera 36a.

[0076] In one aspect, the video generation unit 21d determines setting information for the virtual camera (position in the virtual space, gaze position, gaze direction, and angle of view), determines a drawing area within the entire virtual space based on this setting information for the virtual camera, and generates video information for displaying the drawing area of ​​this virtual space.

[0077] The video generation unit 21d can combine the generated video with the voice of actor A1 acquired from the microphone of the studio unit 30. In this way, the video generation unit 21d can generate animation of a character object that moves in synchronization with the body and facial movements of actor A1, and generate a video for distribution in which the voice of actor A1 is combined with this animation.

[0078] Before distributing a video, the video generation unit 21d can load avatar display information required to display an avatar that may participate in the video. The video generation unit 21d can load 3D display information that takes time to load and imposes a large processing load, among the avatar display information. For example, the video generation unit 21d can read and load from the storage 23 avatar display information required to display an avatar of a user belonging to the first group. In one embodiment, the video generation unit 21d can read and load from the storage 23 3D display information required to 3D display an avatar of a user belonging to the first group.

[0079] In one embodiment, the animation generation unit 21d can load 3D display information for avatars whose number is equal to or less than the first upper limit number. In one embodiment, the animation generation unit 21d can load 3D display information for avatars whose number is equal to or less than the sum of the first upper limit number and the second upper limit number.

[0080] The loaded avatar display information can be referenced when an avatar is displayed in a video in response to a participation request from a user. The use of the loaded avatar display information when an avatar is displayed in a video will be described later.

[0081] The video distribution unit 21e distributes the video generated by the video generation unit 21d. This video is distributed to the client devices 10a to 10c and other client devices via the network 15. The received video is played on the client devices 10a to 10c. This video may be distributed to a client device (not shown) installed in the studio room R, and projected onto a screen S from this client device via a short-focus projector. This video may also be distributed to the supporter computer 40. This allows supporter B1 and supporter B2 to check the viewing screen of the video being distributed.

[0082] A user involved in generating and / or distributing a video distributed from the video distribution unit 21e may be referred to as a "distribution user." In the above example, the video distributed from the server device 20 includes a character object generated based on the body motion data and facial motion data of actor A1, and is distributed with the cooperation of supporter B1 and supporter B2 as necessary. Therefore, actor A1, supporter B1, and supporter B2 may each be referred to as a "distribution user," or actor A1, supporter B1, and supporter B2 may collectively be referred to as a "distribution user."

[0083] An example of a virtual space constructed by the video generating unit 21d is shown in FIG. 6. The virtual space is defined as a spherical space that surrounds its center position. A position in the virtual space is specified by a coordinate value in a three-dimensional global coordinate system that is set in the virtual space. However, for ease of explanation, only a part of the virtual space is shown in FIG. 6, and the coordinate system is not shown in FIG. 6. In FIG. 6, it is assumed that 10 users are participating in the video via avatars.

[0084] A virtual space 50 shown in FIG. 6 includes a character object 51 of actor A1, five virtual cameras 53a-53e, a stage object 54 representing a stage on which the character object 51 is placed, and avatar objects 56a-56c and 57a-57g representing avatars. The stage object 54 includes a floor object 54a corresponding to the studio floor on which the character object 51 stands and a screen object 54b extending vertically from the floor object 54a at one horizontal end of the floor object 54a. For example, the floor object 54a extends horizontally in the virtual space, and the screen object 54b extends vertically in the virtual space. A first region 52a and a second region 52b are defined within the virtual space 50. In the illustrated embodiment, the avatar objects 56a-56c are located within the first region 52a, and the avatar objects 57a-57g are located within the second region 52b. In the virtual space 50, the first region 52a is disposed closer to the character object 51 than the second region 52b. In one embodiment, the character object 51 is disposed within a floor object 54a. Therefore, the first region 52a is disposed closer to the floor object 54a than the second region 52b. The positions of the first region 52a and the second region 52b in the virtual space 50 are not limited to the positions shown in FIG. 6 . The first region 52a and the second region 52b may be disposed at any positions within the virtual space 50. For example, the first region 52a and the second region 52b may be disposed within the floor object 54a. For example, the first region 52a and the second region 52b may be disposed behind the character object 51 on the floor object 54a as viewed from the viewpoint of the virtual camera 53a. In one embodiment, the first region 52a is a region for displaying an avatar of a user who belongs to the first group, and the second region 52b is a region for displaying an avatar of a user who belongs to the second group (or does not belong to the first group). In this specification, the area where the character object 51 is placed may be referred to as the “third area.” In the illustrated example, the floor object 54a corresponds to the third area.The virtual space 50 includes comment display areas 55a to 55c in which comments posted by users watching the video are displayed. The comment display areas 55a to 55c may be objects in the virtual space 50. The virtual space 50 is not limited to the aspects explicitly described in this specification. The virtual space 50 may include various objects other than those explicitly described in this specification.

[0085] The virtual cameras 53a to 53e are arranged at different positions in the virtual space 50. All of the virtual cameras 53a to 53e are arranged so as to face the character object 51. The images constituting the moving image generated by the moving image generating unit 21d are images that display a rendering area of ​​the virtual space 50 viewed from the viewpoint of one of the virtual cameras 53a to 53e selected from the virtual cameras 53a to 53e. The setting information of the virtual cameras 53a to 53e (position in the virtual space, gaze position, gaze direction, and angle of view) may be changed during the video distribution by the operation of the actor A1, the supporter B1, the supporter B2, or an operator other than these.

[0086] As described above, three avatar objects 56a to 56c are arranged in the first area 52a, and seven avatar objects 57a to 57g are arranged in the second area 52b. In one embodiment, a first upper limit number that is an upper limit number of avatars that can be arranged in the first area 52a and a second upper limit number that is an upper limit number of avatars that can be arranged in the second area 52b are defined. The first upper limit number and the second upper limit number may be stored in the storage 23 as participation management data 23e.

[0087] 7a to 7f show display examples of a video generated by the video generation unit 21d and distributed by the video distribution unit 21e. FIGS. 7a to 7f show display images of a video distributed by the video distribution unit 21e and played on the client devices 10a to 10c. The following description will be given assuming that the display images in FIGS. 7a to 7f are displayed on the client device 10a used by the first viewing user. The display screen in FIG. 7a is displayed on the display of the client device 10a by executing a viewing application on the client device 10a and playing the video distributed from the server device 20. FIG. 7a shows a display example of an image 60a drawn in a drawing area defined with the virtual camera 53a as the viewpoint. However, it is assumed that the avatar 56a of the first viewing user is not positioned in the virtual space 50 at the time the image 60a is drawn. In other words, the avatar 56a is not positioned in the virtual space 50 corresponding to the image 60a.

[0088] As shown, the image 60a includes a character object 51, a floor object 54a on which the character object 51 stands, and a screen object 54b that defines the rear edge of the stage. The image 60a also includes a display field 65 in which comments from viewing users and system notifications are displayed.

[0089] The character object 51 is generated by applying the body motion data and facial motion data of the actor A1 to the model data for the actor A1 included in the model data 23a. The motion of the character object 51 is controlled based on the body motion data and facial motion data. Therefore, the character object 51 is controlled to move on the screen in synchronization with the body and facial movements of the actor A1.

[0090] In addition to the objects arranged in the virtual space 50, the display image 60a may also display operation buttons for receiving user operations. In the example shown in Fig. 6, the display image 60a includes a gift button 61 for gifting, a rating button 62 for providing a rating, and an avatar participation button 63 for allowing an avatar to participate. Operation buttons other than these may also be displayed on the display image 60a.

[0091] The gift button 61 is displayed on the display image 60a so as to be selectable by an operation on the client device 10a on which the display image 60a is displayed. The gift button 61 can be selected, for example, by tapping on an area of ​​the touch panel of the client device 10a where the gift button 61 is displayed. In one embodiment, when the gift button 61 is selected, a window is displayed on the display screen 60a for selecting a gift to give (gift) to the broadcaster broadcasting the video being viewed. The user of the client device 10a can purchase or select a gift to gift from the gifts displayed in the window. In another embodiment, in response to the selection of the gift button 61, a window including a list of purchased gifts and / or a list of selectable gifts is displayed on the display image 60a. In this case, the user watching the video can select a gift to gift from the gifts displayed in the window. In one embodiment, in response to the selection of the gift button 61 on the client device 10, a gifting request may be sent to the server device 60. This gifting request may include gift identification information (gift ID) that identifies a gift object corresponding to the gift to be gifted. In this way, a viewing user who is viewing a video can select gift button 61 to give a desired gift to the distributor user who is distributing the video.

[0092] The gift processing unit 21f receives a gifting request from the user's client device 10a and performs processing in accordance with the request. As described above, a user who is watching a video can send a gifting request to the server device 20 by operating their own client device.

[0093] In one aspect, upon receiving a gifting request, the gift processing unit 21f identifies a gift object to be gifted based on the gifting request and performs display processing according to the identified gift object. As described above, gift objects may include effect objects, normal objects, and decorative objects. When the gift object identified by the gifting request is an effect object, the gift processing unit 21f performs processing to display the effect object in the display image 60a of the video. For example, when a gifting request is made for an effect object corresponding to confetti, the gift processing unit 21f displays an effect object representing confetti in the display image 60a based on the gifting request. In one aspect, upon receiving a gifting request for a specific normal object, the gift processing unit 21f performs processing to display the normal object identified by the gifting request in the video. For example, when a gifting request is made for a normal object representing a stuffed teddy bear, the gift processing unit 21f displays the normal object representing the stuffed teddy bear in the display image 60a.

[0094] In one aspect, when the gift processing unit 21f receives a gifting request for a specific decorative object from a viewing user, the gift processing unit 21f adds the decorative object specified by the gifting request to a candidate list. This candidate list is displayed, for example, on the display of the supporter computer 40. Supporter B1 and supporter B2 can select decorative objects to display in the video from the candidate list displayed on the display of the supporter computer 40. The gift processing unit 21f displays the selected decorative object on the video display screen 60a. The selected decorative object is displayed on the display image 60a in association with a specific body part of the character object 51. For example, if the decorative object is an object representing a headband, the decorative object may be worn on the head of the character object 51. The decorative object may be displayed on the display screen 60a so as to move in conjunction with the movement of the specific body part of the character object. For example, when the head of a character object 51 wearing a headband moves, the selected decorative object representing the headband also moves in conjunction with the head of the character object 51, as if the headband were worn on the character object 51's head.

[0095] The rating button 62 is displayed on the display screen 60a so as to be selectable by a user of the client device 10a. The rating button 62 can be selected, for example, by tapping on an area of ​​the touch panel of the client device 10a where the rating button 62 is displayed. When the rating button 62 is selected by a user watching a video, rating update information indicating that a positive rating has been given to the distributor user (e.g., actor A1) who distributes the video or a positive rating has been given to the video being distributed may be transmitted to the server device 60. Selection of the rating button 62 may also transmit rating information indicating that a negative rating has been given to the video.

[0096] The avatar participation button 63 is displayed on the display screen 60a so as to be selectable by a first viewing user using the client device 10a. The avatar participation button 63 can be selected, for example, by tapping on an area of ​​the touch panel of the client device 10a where the avatar participation button 63 is displayed. When the avatar participation button 63 is selected by a first viewing user viewing a video, a participation request requesting that the avatar of the first viewing user participate in the video is transmitted to the server device 20. A user who transmits a participation request by selecting the avatar participation button 63 may be referred to as a "participation requesting user." The participation request from the participation requesting user may include the user ID of the participation requesting user and the avatar ID of the avatar of the participation requesting user. When the avatar participation button 63 is selected by a first viewing user who has not registered his or her own avatar, a screen prompting the first viewing user to register an avatar may be displayed on the client device 10a. In this case, the first viewing user can register an avatar by following the guidance on the screen.

[0097] The participation request processing unit 21g receives a participation request from a participation requesting user and determines whether or not to display the avatar of the participation requesting user based on the participation request. In one embodiment, the participation request processing unit 21g can determine whether the participation requesting user belongs to the first group or the second group. The participation request processing unit 21g can determine whether the participation requesting user belongs to the first group or the second group, for example, by referring to group information in the user data 23c. In a case where a group to which a user belongs is distinguished by setting a flag in the user ID of a user belonging to the first group rather than the group ID in the user information, if the flag is set for the participation requesting user, the participation request processing unit 21g can determine that the participation requesting user belongs to the first group, and if the flag is not set, the participation request processing unit 21g can determine that the participation requesting user belongs to the second group (or does not belong to the first group).

[0098] If it is determined that the participation requesting user belongs to the first group, the participation request processing unit 21g places the avatar of the participation requesting user in the first area 52a of the virtual space 50. For example, if a first viewing user transmits a participation request via the client device 10a and it is determined that the first viewing user belongs to the first group, an avatar 56a corresponding to the first viewing user is placed in the first area 52a, as shown in FIG. 6. If it is determined that the first viewing user belongs to the first group, the avatar 56a of the first viewing user may be displayed in the first area 52a without requiring any additional conditions. After the participation of the first viewing user's avatar 56a is approved, a visual effect related to the participation of the avatar 56a may be displayed before or after the avatar 56a is placed in the first area 52a. For example, a visual effect such as shining a spotlight on the avatar 56a may be used to highlight the avatar 56a. The visual effect related to the avatar's participation is not limited to the aspects explicitly described in this specification.

[0099] If it is determined that the participation requesting user belongs to the second group, the participation request processing unit 21g determines whether the participation requesting user satisfies the participation conditions, and if it is determined that the participation conditions are satisfied, places the avatar of the participation requesting user in the second area 52b. For example, if a first viewing user transmits a participation request via the client device 10a and it is determined that the first viewing user belongs to the second group, the avatar 56a is placed in the second area 52b, unlike the mode shown in Fig. 6. When the avatar 56a is placed in the second area 52b, the above-mentioned dramatic effects can also be displayed in the video.

[0100] When an avatar's participation in a video is permitted based on a participation request, the avatar may be prohibited from participating in other videos. For example, when an avatar's participation in a video is permitted based on a participation request, a flag indicating that the avatar is participating in the video may be set in avatar data 23d, and whether to permit a request to participate in another video from a user using the avatar may be determined based on the flag. Specifically, when a user using an avatar participating in a certain video makes a request to participate in another video, the avatar data 23d may be referenced, and if a flag indicating that the avatar is participating in the video is set for the avatar, the participation request may be rejected. The process of prohibiting a single avatar from participating in two or more videos may be performed by the participation request processing unit 21g.

[0101] As described above, if the participation requesting user belongs to the second group, it is determined whether the participation condition is satisfied. Next, this participation condition will be described. In one embodiment, the participation condition is that the number of avatars placed in the second area 52b is less than the second upper limit number set for the second area 52b. As an example, assume that the second upper limit number set for the second area 52b in FIG. 6 is "7." In this example, when a participation request is received from a participation requesting user belonging to the second group, seven avatars 57a to 57g, which corresponds to the upper limit number, are already placed in the second area 52b, and it is determined that the participation condition is not satisfied. In this case, the participation request from the participation requesting user is not permitted, and the avatar of the participation requesting user is not placed in the virtual space 50. As another example, assume that the second upper limit number set for the second area 52b in FIG. 6 is "12." In this example, when a participation request is received from a participation requesting user who belongs to the second group, it is determined that the participation conditions are met because seven avatars, which is fewer than the upper limit, are arranged in the second area 52b. In this case, the avatar of the participation requesting user is arranged in the second display area 52b.

[0102] An avatar placed in the first area 52a or the second area 52b can exit the video in which it is participating, in response to an operation by the user using the avatar. For example, when a first viewing user's avatar 56a is placed in the first area 52a or the second area 52b of a video, the first viewing user can operate the client device 10a to send an exit request to the server device 20 to cause the avatar 56a to exit. When the server device 20 accepts the exit request, the server device 20 (e.g., the participation request processing unit 21g) can perform processing to cause the first viewing user's avatar 56a to exit the video in response to the exit request. If an upper limit is set on the number of avatars that can participate in the first area 52a or the second area 52b, the exit of the avatar of the user who made the exit request in response to the exit request allows avatars of other users to participate in the first area 52a or the second area 52b. When an avatar leaves a video, a special effect related to the avatar's exit may be displayed in the video. For example, a spotlight may be shone on the exiting avatar. The special effect related to the avatar's participation is not limited to the manner explicitly described in this specification.

[0103] In one embodiment, the process of causing an avatar to exit a video may be performed in response to a user using the avatar terminating a viewing application that is being used to view the video.

[0104] In another embodiment, even if a user who has included an avatar in a video closes the viewing app that they are using to watch the video, the avatar may continue to be displayed in the video.

[0105] In another embodiment, the participation condition is that the participation requesting user is selected by the broadcast user. The broadcast users are, for example, actor A1, supporter B1, and supporter B2. When the participation request processing unit 21g receives a participation request from a participation requesting user, it may generate a participation requesting user list including an indication identifying the participation requesting user and present the participation requesting user list to the broadcast user. The participation request processing unit 21g may send the participation requesting user list to the supporter computer 40. Supporter B1 and supporter B2 can select users to allow to participate in the video from the participation requesting user list displayed on the display of the supporter computer 40. The participation requesting user list may be displayed on the display 39 of studio room R. Actor A1 may select users to allow to participate in the video from the participation requesting user list by operating controller 33a or controller 33b. The participation request processor 21g can place the avatar of the participation requesting user selected by the distribution user of the video (that is, permitted to participate in the video) in the second display area 52b.

[0106] The avatar display unit 21h performs processing for displaying, within the image, the avatars determined by the participation request processing unit 21g to be placed in the first area 52a and the second area 52b. For example, when displaying the avatar of a first viewing user in the first area 52a, the avatar display unit 21h identifies an avatar ID associated with the user ID of the first viewing user by referring to the user data 23c, and then identifies parts information associated with the avatar ID by referring to the avatar data 23d. The avatar display unit 21h constructs an avatar object representing the first viewing user's avatar based on the identified parts information and places the avatar object in the first area 52a. As a result, when the first area 52a is included in the rendering area of ​​the virtual camera, the avatar object representing the first viewing user's avatar is displayed in the video. When the avatar is to be displayed in 3D, the avatar object is constructed based on parts information for 3D display, and when the avatar is to be displayed in 2D, the avatar object is constructed based on parts information for 2D display.

[0107] Next, with reference to FIGS. 7b to 7d, a description will be given of a display image when the first viewing user selects the avatar participation button 63 and the avatar 56a of the first viewing user is placed in the first area 52a of the virtual space 50. FIG. 7b shows an example of a display image 60b that is rendered with the virtual camera 53a as the viewpoint, similar to the display image 60a shown in FIG. 7a. Therefore, the arrangement of the character object 51 and other objects in the display image 60b is substantially the same as the arrangement of the corresponding objects displayed in the display image 60a. Because the virtual camera 53a is positioned between the first area 52a and the character object 51, the avatar of the first viewing user placed in the first area 52a or the second area 52b in the display image 60b is not included in the rendering area with the virtual camera 53a as the viewpoint.

[0108] FIG. 7c shows another example of a video display in which the avatar 56a of the first viewing user is placed in the first area 52a within the virtual space 50. Unlike FIG. 7b, the display image 60c shown in FIG. 7c is rendered from the viewpoint of a virtual camera 53c. The virtual camera 53c is positioned from the right rear of the character object 51, facing the character object 51. Therefore, the display image 60c rendered from the viewpoint of the virtual camera 53c includes the character object 51 viewed from the right rear and the avatars 56a to 56c placed in the first area 52a. Due to the relative positions of the virtual camera 53c, the first area 52a, and the second area 52b, the avatars placed in the second area 52b are also displayed in the display image 60c so as to be partially hidden behind the avatars 56a to 56c. The avatars 56a to 56c and the avatars 57a to 57g may be displayed in 2D (two-dimensional) format. In this case, avatars 56a-56c and 57a-57g, which are displayed in 2D, are displayed in the virtual space 50 along with other objects, which are displayed in 3D. By displaying avatars 56a-56c and 57a-57g in 2D, the load of the rendering process can be reduced. Only the avatars 57a-57g in the back row may be displayed in 2D (two-dimensional). As a result of the reduced load of the rendering process, more avatars can be displayed on the screen. In one embodiment, when 3D display information of an avatar to be placed in the virtual space 50 is loaded by the video generation unit 21d before the start of distribution of a video, the avatar can be displayed in 3D during the video. The avatar display unit 21h can check whether 3D display information of the avatar to be placed in the virtual space 50 based on the participation request has been loaded. The avatar display unit 21h may display the avatar in 3D display information in a moving image when the 3D display information of the avatar has already been loaded, and may display the avatar in 2D display information in a moving image when the 3D display information of the avatar has not yet been loaded. In one embodiment, the avatar display unit 21h may perform runtime loading of the 3D display information of the avatar to be placed in the virtual space 50 based on the participation request, and may display the avatar in 3D display information in a moving image using the runtime-loaded 3D display information.That is, the server device 20 may load 3D display information of an avatar to be displayed in a video while the video is being distributed (in parallel with the distribution process of the video).

[0109] While an avatar is displayed in a video, the user of the avatar may change the features of the avatar. When the features of the avatar displayed in the video are changed, the avatar display unit 21h can change the appearance of the avatar in the video to reflect the change. For example, if a user changes the hairstyle of their avatar from black to blonde, the avatar display unit 21h can change the appearance of the avatar so that the change is reflected in the display image of the avatar displayed in the video. In addition to changing the features, the avatar display unit 21h can also reflect changes to the avatar during video distribution in the avatar in the video in real time or near real time.

[0110] FIG. 7d shows another example of a video display in which the avatar 56a of the first viewing user is placed in the first region 52a within the virtual space 50. The display image 60d shown in FIG. 7d is rendered with the virtual camera 53b as the viewpoint. The virtual camera 53b is positioned in front of the character object 51, farther from the character object 51 than the virtual camera 53a, and facing the character object 51. Therefore, the display image 60d rendered with the virtual camera 53b as the viewpoint includes a wider range of the virtual space 50 than the display images 60a and 60b. Specifically, the display image 60d includes the first region 52a and the second region 52b. Therefore, the display image 60d also displays the avatars 56a to 56c placed in the first region 52a and the avatars 57a to 57g placed in the second region 52b.

[0111] The display image may include emotion buttons 68. The emotion buttons 68 are buttons that allow the viewing user to express emotions felt while watching a video. By selecting the emotion buttons 68, the viewing user can select emotions such as being moved by the video, not empathizing with the video, or other emotions. The emotion buttons 68 may include a clap button for selecting an emotion of being moved by the video, a question mark button for indicating not empathizing with the video, a surprise button for indicating being surprised by the video, and other buttons. Each emotion button 68 may include a symbol or graphic display representing the emotion selected by the emotion button 68. For example, the clap button may include a graphic display of clapping hands, the question mark button may include a question mark, and the surprise button may include an exclamation mark. The emotion button 68 may be selected, for example, by tapping an area of ​​the touch panel of the client device 10a where the emotion button 68 is displayed. When the emotion button 68 is selected by a first viewing user who is watching a video, an emotion display request is transmitted to the server device 20. The emotion display request may include emotion identification information indicating the type of emotion button 68 selected by the first viewing user (such as an applause button, a question mark button, or a surprise button). By selecting the emotion button 68, multiple buttons associated with the emotion button 68 (such as an applause button or a surprise button) may be displayed in a pop-up. When multiple buttons associated with the emotion button 68 are displayed in a pop-up, the user can select a button that matches his or her emotion from the multiple buttons displayed in the pop-up. The emotion button 68 is displayed when the user's avatar is participating in a video, and may not be displayed when the user's avatar is not participating in a video.For example, in the illustrated embodiment, when display image 60a is displayed, the avatar of the first viewing user is not participating in the video, so the emotion button 68 is not displayed on display image 60a, but the emotion button 68 is displayed on display image 60b in which the avatar of the first viewing user is participating in the video and on other display images.

[0112] The avatar control unit 21i may receive an emotion display request from a first viewing user and control the behavior of the first viewing user's avatar based on the emotion display request. For example, if the first viewing user selects a clap button, which is a type of emotion button, and transmits an emotion display request, the avatar control unit 21i may cause the first viewing user's avatar 56a to perform a clapping action based on the emotion display request. When the avatar 56a is controlled to clap based on the emotion display request, if the avatar 56a is included in the drawing area as shown in FIG. 7c or 7d, an animation of the avatar 56a clapping may be displayed in the video. Conversely, if the avatar 56a is not included in the drawing area as shown in FIG. 7b, the sound of the avatar 56a clapping may be displayed in the video, but the clapping animation may not be displayed. In one embodiment, the emotion button may be displayed on the display screen of a video being viewed by a user only when the user's avatar is displayed in the video. For example, when the image 60b shown in Fig. 7b is displayed on the client device 10a of the first viewing user, the avatar 56a of the first viewing user is not displayed in the image 60b, and therefore the image 60b may not include the emotion button 68. Conversely, when the image 60c shown in Fig. 7c is displayed on the client device 10a of the first viewing user, the avatar 56a of the first viewing user is displayed in the image 60c, and therefore the emotion button 68 may be included in the image 60c.

[0113] In this way, based on an emotion display request from a user, the user's avatar performs a movement such as clapping in the video. The emotion display request is a type of action instruction that instructs the avatar to move. In another embodiment, in addition to the emotion button, an action instruction button that instructs the avatar to move may be included in the displayed image. The action instruction button is a button for instructing the avatar to move. The action instruction button can be used to instruct, for example, an action such as jumping, waving a hand, waving a glow stick, or other actions. When an action instruction button is selected, a request for action instruction is transmitted to the server device 20. The server device 20 can cause the avatar to move based on the action instruction request.

[0114] As shown in FIGS. 7b to 7d, when the display images 60b to 60d are displayed, the avatar 56a of the first viewing user has already been placed in the virtual space 50. Therefore, the display image 60b does not include the avatar participation button 63, but instead includes a display request button 64. The display request button 64 is a button for requesting that the viewing user's avatar be displayed together with the character object 51. The display request button 64 is displayed on the display screen 60b so as to be selectable by the first viewing user using the client device 10a. The display request button 64 can be selected, for example, by tapping an area on the touch panel of the client device 10a where the display request button 64 is displayed. When the display request button 64 is selected by the first viewing user who is watching a video, a display request requesting that the first viewing user's avatar 56a be displayed together with the character object 51 is transmitted to the server device 20.

[0115] The display request processing unit 21j receives a display request from a first viewing user and, based on the display request, displays the first viewing user's avatar 56a together with the character object 51 in the video. A request to display an avatar together with a character object (e.g., character object 51) included in the video being distributed is referred to as a "feature display request," and displaying an avatar together with the character object 51 in response to a feature display request is sometimes referred to as "feature display." As shown in FIG. 7e, when a user makes a feature display request, a notification 66 indicating that a feature display request has been made may be displayed in a display image 60e. This notification 66 may be displayed only in the image displayed on the client device of the user who made the feature display request, or may be displayed in images displayed on the client device of the user as well as on images displayed on client devices of other users. FIG. 7f shows an example of a display image in which the first viewing user's avatar 56a and the character object 51 are displayed together. In the display image 60f shown in FIG. 7f, the first viewing user's avatar 56a is displayed next to the character object 51. The display request processing unit 21j may display the avatar 56a on the floor object 54a based on a display request from the first viewing user. Since the character object 51 is placed on the floor object 54a, by placing the avatar 56a on the floor object 54a, the avatar 56a can be displayed together with the character object 51. A notification 67 for notifying that feature display has been performed may be displayed on the display image 60f.

[0116] When the avatar 56a is displayed together with the character object 51 based on the feature display request, the avatar 56a may be displayed in 3D. As described above, the avatar 56a may be displayed in 2D in the display image (e.g., display image 60c or 60d) before the feature display request is made. The display mode of the avatar 56a, which was displayed in 2D before the feature display request is made, may be changed to 3D in the feature display. Since the avatar 56a is displayed in 3D in the feature display, the display request button 64 may be displayed in the display image 60b displayed on the client device 10a of the first viewing user only if 3D display information for the avatar 56a has been loaded before the feature display request is received.

[0117] In one embodiment, the display request button 64 is displayed in a display image for a user using an avatar for which 3D display information is loaded, and the display request button 64 is not displayed in a display image for a user using an avatar for which 3D display information is not loaded, thereby preventing runtime loading of the 3D display information and reducing the processing load on the server device 20. In one embodiment, to avoid the processing load required for runtime loading, when a feature display request for an avatar is received from a user, it may be determined whether the 3D display information for the avatar has been loaded, and the avatar may be displayed in 3D only if it has been loaded. In this case, by displaying the display request button 64 in a display image for a user using an avatar for which 3D display information is loaded, and not displaying the display request button 64 in a display image for a user using an avatar for which 3D display information is not loaded, it is possible to prevent unnecessary transmission of a feature display request and unnecessary processing based on the feature display request.

[0118] When an avatar is featured in a video, a special effect related to the featured avatar may be displayed. For example, a special effect may be displayed as if the avatar is being highlighted in a spotlight. The same or a different special effect may be displayed when the featured avatar is terminated.

[0119] The display request processing unit 21j may measure the elapsed time since the avatar 56a of the first viewing user was displayed together with the character object 51 in the video. The display request processing unit 21j may display the elapsed time in the display image 60f. In one embodiment, an upper limit may be set for the time during which the character object 51 and the user's avatar are displayed together (the time during which a feature display is performed for a certain avatar) based on the feature display request. The display request processing unit 21j may display the upper limit time in the display image 60f. The display request processing unit 21j may calculate the remaining time during which the avatar 56a can be displayed based on the upper limit time and the time elapsed since the avatar 56a of the first viewing user was displayed together with the character object 51 in the video, and display the remaining time in the display image 60f. The elapsed time since the first viewing user's avatar 56a was displayed in the video together with the character object 51, the upper limit of the display time set for the avatar 56a, and the remaining time for which the avatar 56a can be displayed may be included only in the display image 60f displayed on the client device 10a of the first viewing user, or may be included in the display image 60f displayed on the client devices of the first viewing user and other users. By setting an upper limit of the time for which the feature display is performed, it is possible to provide more viewing users with an opportunity to display their own avatar together with the character object 51 in the feature display. Furthermore, by displaying the remaining time of the currently running feature display, it is possible to inform the viewing user of the time until the next avatar is featured.

[0120] The display request processing unit 21j can remove an avatar participating in a video from the video based on an instruction from the video distributor or another operator. The removed avatar will no longer be displayed on the display screen of the video. Similarly, the display request processing unit 21j can remove an avatar that is displayed as a featured display from the featured display based on an instruction from the video distributor or another operator. The removed avatar will no longer be displayed on the screen of the featured display. The broadcasting user can remove, for example, an avatar that has made an undesirable comment in a video, an avatar that has behaved undesirably in a video, or any other avatar that is undesirable to have participating in the video from the video and / or from the display screen of the featured display.

[0121] In one embodiment, when a user's avatar is featured, the avatar control unit 21i can control the movement of the avatar based on instructions from the user. For example, the avatar control unit 21i can receive an emotion display request from the user of the featured avatar and control the behavior of the avatar based on the emotion display request. The behavior of the avatar may also be controlled using face motion or body motion transmitted from the user's terminal.

[0122] In one embodiment, when a feature display request from a first viewing user is permitted by the broadcasting user, the display request processing unit 21j may display the avatar 56a of the first viewing user together with the character object 51. In response to receiving a feature display request from the first viewing user, the display request processing unit 21j may inquire of the broadcasting user whether or not to permit the display of the avatar 56a of the first viewing user together with the character object 51. When receiving a feature display request from the first viewing user, the display request processing unit 21j may generate a display request user list including the first viewing user and present the display request user list to the broadcasting user. The display request user list for a certain video indicates, in list form, information (e.g., user ID, user name, nickname, avatar ID, or other information) that identifies the user (or the user's avatar) who made the feature display request. The display request user list does not need to include all users who sent feature display requests. For example, the display request user list may include only users who have transmitted feature display requests and for whom 3D display information for their avatars has already been loaded. Such a display request user list does not include information identifying users who have transmitted feature display requests and for whom 3D display information for their avatars has not been loaded. Displaying an avatar for which 3D display information has not been loaded requires runtime loading of the 3D display information, which consumes a lot of processing resources in the server device 20. By including only users who use avatars for which 3D display information has already been loaded in the feature display request, runtime loading can be avoided.

[0123] The display request processing unit 21j can transmit the display request user list to the supporter computer 40. The supporter B1 and the supporter B2 can select users whose avatars they wish to allow to be displayed together with the character object from the display request user list displayed on the display of the supporter computer 40. The display request user list may be displayed on the display 39 of the studio room R. The actor A1 may select users whose avatars they wish to allow to be displayed together with the character object 51 from the display request user list by operating the controller 33a or the controller 33b. A user selected from the display request user list is given permission to display the avatar of that user together with the character object 51, and the avatar is displayed as a featured in the video.

[0124] In one embodiment, when a user is selected from the display request user list, the user may be deleted from the display request user list. In another embodiment, when the featured display of a user's avatar ends, the user may be deleted from the display request user list. This makes it possible to prevent an avatar that has been featured once from being featured again.

[0125] In one embodiment, even if a supporter or broadcaster user instructs a user to select a user to be featured from the display request list while a user is being featured, the avatar of the selected user may not be immediately featured, and the selected user may not be deleted from the display request list until the user is featured.

[0126] Next, a video distribution process according to one aspect will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing the flow of the video distribution process according to one embodiment. In the video distribution process, it is assumed that an actor A1 is present in a studio room R.

[0127] First, in step S11, body motion data, which is a digital representation of the body movements of actor A1, and face motion data, which is a digital representation of the facial movements (expressions) of actor A1, are generated. The body motion data is generated, for example, by the body motion data generation unit 21a, and the face motion data is generated, for example, by the face motion data generation unit 21b.

[0128] Next, in step S12, the body motion data and face motion data of actor A1 are applied to the model data for actor A1 to generate animation of a character object that moves in synchronization with the body and facial movements of actor A1. This animation is generated, for example, by the animation generation unit 21c.

[0129] Next, in step S13, a moving image including an image of the virtual space and an animation of a character object corresponding to actor A1 is generated. The voice of actor A1 may be synthesized into this moving image. The animation of the character object of actor A1 may be placed in the virtual space. The generation of this moving image is performed, for example, by the moving image generation unit 21d described above.

[0130] Next, the process proceeds to step S14, where the video generated in step S13 is distributed. This video is distributed to client devices 10a to 10c and other client devices via network 15. This video is distributed to supporter computer 40, and may also be projected onto a screen S in studio room R. The video is distributed continuously for a predetermined distribution period. The video distribution period can be set to, for example, 30 seconds, 1 minute, 5 minutes, 10 minutes, 30 minutes, 60 minutes, 120 minutes, or any other period of time.

[0131] Next, the process proceeds to step S15, where it is determined whether or not a termination condition for terminating the distribution of the video has been met. The termination condition may be, for example, that the end time of the distribution has arrived, that an instruction to terminate the distribution has been given by the supporter computer 40, or any other condition. If the termination condition has not been met, the processes of steps S11 to S14 are repeatedly executed, and the distribution of the video including animation synchronized with the movements of actor A1 continues. If it is determined that the termination condition has been met for the video, the distribution process of that video is terminated.

[0132] Next, a participation request process according to one aspect will be described with reference to Fig. 9. Fig. 9 is a flow diagram showing the flow of the participation request process in one embodiment. In the participation request process of Fig. 9, it is assumed that a first viewing user who is watching a video on the client device 10a has requested that the avatar 56a of the first viewing user participate in the video.

[0133] First, in step S21, a participation request is received from a first viewing user. The participation request is transmitted from the client device 10a to the server device 20 in response to the first viewing user selecting the avatar participation button 63 displayed on the client device 10a.

[0134] Next, in step S22, the group to which the user who sent the join request belongs is identified. If the user belongs to the first group or the second group, it is determined whether the first viewing user who sent the join request belongs to the first group or the second group. This determination is made, for example, by the join request processing unit 21g.

[0135] If it is determined that the first viewing user belongs to the first group, the participation request process proceeds to step S23. In step S23, the avatar 56a of the first viewing user is placed in the first area 52a of the virtual space 50. As a result, if the first area 52a is included in the rendering area of ​​the virtual camera, the avatar 56a of the first viewing user is displayed at a position in the video that corresponds to the first area 52a. The placement of the avatar 56a in the first area 52a is performed, for example, by the participation request processing unit 21g.

[0136] If it is determined that the first viewing user belongs to the second group, the participation request processing proceeds to step S24. In step S24, it is determined whether or not the participation condition is met. As described above, the participation condition is that the number of avatars arranged in second area 52b is less than the second upper limit number set for second area 52b, that the first viewing user is selected by the broadcasting user, or other conditions. The determination of whether or not the participation condition is met is made, for example, by the above-mentioned participation request processing unit 21g.

[0137] If it is determined that the participation conditions are met, the participation request processing proceeds to step S25. In step S25, the avatar 56a of the first viewing user is placed in the second area 52b of the virtual space 50. As a result, if the second area 52b is included in the rendering area of ​​the virtual camera, the avatar 56a of the first viewing user is displayed at a position in the video that corresponds to the second area 52b. The placement of the avatar 56a in the second area 52b is performed, for example, by the participation request processing unit 21g.

[0138] If it is determined that the participation conditions are not met, the participation request process is terminated without displaying the avatar 56a of the first viewing user. In another embodiment, if it is determined that the participation conditions are not met, it may be repeatedly determined whether the participation conditions are met.

[0139] Next, a display request process according to one embodiment will be described with reference to Fig. 10. The display request process is a process based on a display request from a user of an avatar placed in a virtual space, requesting that the avatar be displayed together with a character object 51. Fig. 10 is a flow diagram showing the flow of the display request process in one embodiment. In the display request process of Fig. 10, it is assumed that a display request has been made from a first viewing user who is watching a video on the client device 10a, requesting that the first viewing user's avatar 56a be displayed together with a character object 51.

[0140] First, in step S31, a display request is received from a first viewing user. The display request is transmitted from the client device 10a to the server device 20 in response to the first viewing user selecting a display request button 64 displayed on the client device 10a.

[0141] Next, in step S32, it is determined whether the display request from the first viewing user has been permitted by the distribution user. If it is determined that the display request has been permitted, the display request processing proceeds to step S33. This determination is made, for example, by the display request processing unit 21j described above.

[0142] In step S33, a process is performed to display the avatar 56a of the first viewing user together with the character object 51. For example, the avatar 56a is placed on the floor object 54a on which the character object 51 is placed. The placement of the avatar 56a on the floor object 54a is performed, for example, by the display request processing unit 21j.

[0143] If it is determined that permission has not been obtained from the broadcasting user, the display request process ends without displaying both the first viewing user's avatar 56a and the character object 51. In other embodiments, the determination in step S32 may be repeated until permission is obtained from the broadcasting user.

[0144] The effects achieved by the above embodiment will be described. In the above embodiment, in response to receiving a request from a first viewing user who is watching a video distributed from the server device 20 to participate in the video, it is determined whether the first viewing user belongs to the first group or the second group. In one embodiment, if the first viewing user has high engagement with the video distribution system 1 or the distribution user who is distributing the video they are viewing, the first viewing user is determined to belong to the first group. If it is determined that the first viewing user belongs to the first group, the avatar 56a of the first viewing user is placed in the first area 52a in the virtual space 50. In one embodiment, if it is determined that the first viewing user belongs to the first group, the avatar 56a is placed in the first area 52a without requiring any additional conditions. This does not prevent the avatars of users who belong to the first group with high engagement from participating in the video. In one embodiment, by setting criteria for belonging to the first group, the number of users belonging to the first group can be restricted to an appropriate number depending on the video being distributed. This makes it possible to prevent a large number of avatars from being displayed in the distributed video, thereby reducing the processing load on the server device 20.

[0145] In one embodiment, when it is determined that the first viewing user belongs to the second group, it is further determined whether or not the participation condition is satisfied, and in response to the determination that the participation condition is satisfied, the avatar 56a of the first viewing user is placed in the second area 52b within the virtual space 50. Thus, by setting the participation condition, the number of avatars displayed in the second area 52b can be kept to an appropriate number, thereby reducing the processing load on the server device 20.

[0146] In one embodiment, the first region 52a is placed closer to the character object 51 (or closer to the third region where the character object is placed) than the second region 52b in the virtual space 50. This allows a viewing user with high engagement to be placed on the character object 51. In one embodiment, the virtual cameras 53a to 53e are placed in the virtual space 50 so as to face the position where the character object 51 is placed. Therefore, an avatar placed in the first region 52a is more likely to be included in the rendering area of ​​the virtual camera than an avatar placed in the second region 52b. In this way, in one embodiment, the avatars of a first group of users with high engagement can be placed in the first region 52a, which is more likely to be included in the rendering area.

[0147] In one embodiment, the participation condition is that the number of avatars displayed in the second area 52b is less than the upper limit of the number of avatars set for the second area 52b. This makes it possible to keep the number of avatars displayed in the second area 52b at or below the upper limit, thereby reducing the processing load on the server device 20.

[0148] In one embodiment, a condition for participation is that the viewing user is selected by the broadcasting user. This allows the broadcasting user to select avatars of viewing users who are favorable to the broadcasting user to participate in the video. This makes it easier to maintain the health of the community formed through the broadcasted video.

[0149] In one embodiment, an instruction to operate an avatar placed in virtual space 50 is received, and the movement of the avatar is controlled based on the instruction to operate. This allows a viewing user to interact with characters included in a video or other viewing users participating in the video through the instruction to operate the avatar.

[0150] In one embodiment, a display request to display the viewing user's avatar together with the character object 51 is accepted while the avatar is placed in the virtual space 50, and in response to permission of this display request, the avatar is displayed in the third area where the character object 51 is placed. This provides the viewing user with a means for more closely communicating with the broadcasting user's character.

[0151] The embodiments of the present invention are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist thereof. For example, the video to be distributed may be filmed and generated outside the studio room R. For example, filming for generating the video to be distributed may be performed at the home of an actor or supporter. The client device 10a may have a function to transmit an image (e.g., display image 60f) in which the avatar 56a and the character object 51 are displayed together to another communication device. For example, the first viewing user can use the function of the client device 10a to distribute the image in which the avatar 56a and the character object 51 are displayed together to a social networking service within the video service provided by the server device 20 or to a social networking service different from the video service provided by the server device 20.

[0152] In the processing procedures described in this specification, particularly in processing procedures described using flow charts, it is possible to omit some of the steps that make up the processing procedures, to add steps that are not explicitly stated as steps that make up the processing procedures, and / or to change the order of the steps, and processing procedures in which such omissions, additions, or changes in order have been made are also included within the scope of the present invention as long as they do not deviate from the spirit of the present invention. The claims as originally filed in the original application of this application are set forth below. [1] A system that allows a first video, which includes a virtual space and animation of a character object generated based on a movement of a distribution user, to be viewed live on a client device of each of a plurality of viewing users, comprising: one or more computer processors; The one or more computer processors execute the computer-readable instructions to: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, a first avatar of the first user is placed in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; system. [2] In the second moving image, the first avatar is placed in a character display area in which the character object is placed. [1] The system described in [1]. [3] the display request is generated by the first client device in response to selection of a display request button displayed on the first client device of the first user; [1] or [2]. [4] the display request button is displayed on the first client device in response to the first avatar joining the first video. [3] The system described in. [5] In the second moving image, the first avatar is placed in a character display area in which the character object is placed; the display request button is displayed on the first client device only when 3D display information for displaying the first avatar in 3D is stored in a storage; the first avatar is displayed in 3D in the character display area in the second video; [4] The system described in [4]. [6] the first avatar is displayed in 2D in the first region; The system described in [5]. [7] measuring an elapsed time from when the first avatar is displayed together with the character object in the second video; calculating a remaining time for which the first avatar will be displayed together with the character object based on the elapsed time; displaying the remaining time together with the second video on the client device; [1] to [6]. [8] excluding the first avatar from the second video based on an instruction from the broadcasting user; [1] to [7]. [9] controlling the movement of the first avatar based on an action instruction generated in response to a selection of an action instruction button displayed on the first client device; [3] to [5].

[10] the action instruction button is displayed on the first client device only when the first avatar is participating in the first video. The system described in [9].

[11] The first avatar is prohibited from participating in any other video other than the first video while participating in the first video. [1] The system according to any one of [1] to

[10] .

[12] points are set for each of the plurality of viewing users, the points increasing or decreasing depending on the use of the system by each viewing user; assigning high-point users having points higher than a threshold among the plurality of users to the first group; [1] The system according to any one of [1] to

[11] .

[13] The first user's points increase as the first user watches videos on the system. The system described in

[12] .

[14] The first user's points increase when the first user distributes a video on the system. The system according to

[12] or

[13] .

[15] An upper limit is set on the number of viewing users who can belong to the first group. [1] The system according to any one of [1] to

[14] .

[16] The viewing users among the plurality of viewing users are made to belong to the first group in descending order of points until the upper limit number is reached. The system described in

[15] .

[17] the system is configured to enable live viewing of a second video, which includes animation of a character object generated based on movements of other distribution users, by each client device of the plurality of viewing users; The points of each viewing user are managed individually for the broadcast user and the other broadcast users.

[12] The system according to any one of

[16] to

[17] .

[18] A method for enabling live viewing of a first video, including a virtual space and animation of a character object generated based on a distributor user's movements, on each of a client device of a plurality of viewing users, by executing computer-readable instructions by one or more computer processors, comprising: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A method for providing the above.

[19] A program that allows a first video, including a virtual space and animation of a character object generated based on a distribution user's movement, to be viewed live on each of a plurality of viewing users' client devices, one or more computer processors, In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A program that executes the following. [Explanation of symbols]

[0153] 1. Video distribution system

Claims

1. A system that allows a first video including an animation of a character object placed in a virtual space to be viewed live on each of client devices of a plurality of viewing users, the system comprising: one or more computer processors; The one or more computer processors execute the computer-readable instructions to: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; system.

2. In the second video, the first avatar is placed in a character display area in which the character object is placed. The system of claim 1 .

3. the display request is generated by the first client device in response to a display request button displayed on the first client device of the first user being selected; 3. The system according to claim 1 or 2.

4. the display request button is displayed on the first client device in response to the first avatar participating in the first video. The system of claim 3 .

5. In the second moving image, the first avatar is placed in a character display area in which the character object is placed; the display request button is displayed on the first client device only when 3D display information for displaying the first avatar in 3D is stored in a storage; the first avatar is displayed in 3D in the character display area in the second video; The system of claim 4.

6. the first avatar is displayed in 2D in the first area; The system of claim 5.

7. measuring an elapsed time from when the first avatar is displayed together with the character object in the second video; calculating a remaining time for which the first avatar will be displayed together with the character object based on the elapsed time; displaying the remaining time together with the second video on the client device; A system according to any one of claims 1 to 6.

8. Removing the first avatar from the second video based on an instruction from a distribution user. A system according to any one of claims 1 to 7.

9. Generated in response to selection of an operation instruction button displayed on the first client device. controlling a movement of the first avatar based on an action instruction received from the user; 6. A system according to any one of claims 3 to 5.

10. the action instruction button is displayed on the first client device only when the first avatar is participating in the first video; The system of claim 9.

11. the first avatar is prohibited from participating in any other video other than the first video while participating in the first video; A system according to any one of claims 1 to 10.

12. points are set for each of the plurality of viewing users, the points increasing or decreasing depending on the use of the system by each viewing user; A high-point user having a point higher than a threshold among the plurality of viewing users is made to belong to the first group.

12. A system according to any one of claims 1 to 11.

13. The first user's points increase as the first user watches videos on the system. The system of claim 12.

14. The first user's points are increased by the first user distributing a video on the system.

14. A system according to claim 12 or 13.

15. An upper limit is set on the number of viewing users who can belong to the first group.

15. A system according to any one of claims 1 to 14.

16. The viewing users among the plurality of viewing users are made to belong to the first group in descending order of points until the upper limit number is reached. The system of claim 15.

17. A method for enabling live viewing of a first video including animation of a character object disposed in a virtual space by each of a plurality of viewing users on a client device, by executing computer-readable instructions by one or more computer processors, the method comprising: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A method for providing the above.

18. A program that allows a first video including an animation of a character object placed in a virtual space to be viewed live on each of a plurality of client devices of a plurality of viewing users, one or more computer processors, In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A program that executes the following.

Images