Video distribution system for live distributing video containing animation of character object generated on the basis of motion of distributor user, video distribution method, and video distribution program

Abstract

To provide a video distribution system for live distributing video containing a virtual space and an animation of a character object generated on the basis of a motion of a distributor user.SOLUTION: In a video distribution system 1, a computer processor of a server device generates video in which a first avatar is featured with a character object on receipt of a display request from a first user while the first avatar is arranged in a first region.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The disclosure in this specification relates to a video distribution system, a video distribution method, and a video distribution program that live-distributes videos including animations of character objects generated based on the movements of a broadcasting user. [Background technology]

[0002] Conventionally, a video distribution system has been known that generates animation of a character object based on the movement of a distribution user and live-distributes a video including the animation of the character object. Such a video distribution system is disclosed, for example, in JP2015-184689A (Patent Document 1) and "Yusuke Hasegawa, The Dawn of a New Era of Virtual Idols PROJECT Maria, CG WORLD, Japan, Born Digital Co., Ltd., November 10, 2017, Vol. 231, pp. 74-79" (Non-Patent Document 1).

[0003] As disclosed in JP 2012-120098 A (Patent Document 2), a video distribution system is also known that displays an image including an avatar of a viewing user in the background area of ​​a video being distributed in response to a request from the viewing user. The viewing user can post his / her own comments on the video via the avatar. In this way, the viewing user can participate in the video being distributed via the avatar.

[0004] Japanese Patent Laid-Open Publication No. 2017-187995 (Patent Document 3) discloses a social media system in which a voice call is made between a plurality of users using an image of an avatar that has entered a virtual room. In the social media system described in the publication, an upper limit is set for the number of avatars that can enter each room. When a request for entry into a certain room is made, the request for entry is permitted if the number of avatars already in the room does not exceed the limit number set for the room. When a request for entry into a certain room is permitted, a new avatar is permitted to enter the room. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-184689 A [Patent Document 2] JP 2012-120098 A [Patent Document 3] JP 2017-187995 A [Non-patent literature]

[0006] [Non-Patent Document 1] Yusuke Hasegawa, The Dawn of a New Era of Virtual Idols PROJECT MariA, CG WORLD, Japan, Born Digital Co., Ltd., November 10, 2017, Vol. 231, pp. 74-79 Summary of the Invention [Problem to be solved by the invention]

[0007] In a video distribution system, if an unlimited number of avatars are allowed to participate in a distributed video, there is a problem that the processing load for displaying a large number of avatars becomes large. In addition, as the number of avatars increases, there is a problem that the processing load for posting comments and gifting via the avatars becomes large. If the processing load related to the actions of the avatars becomes too high, there is a risk that the playback of distributed videos and communication functions using avatars cannot be provided normally.

[0008] On the other hand, limiting the number of avatars that can participate in a video being distributed in a distribution system may limit the participation of viewer users who are highly engaged with the distribution system or the distributor users who distribute the video. In a video distribution system, acquiring and maintaining highly engaged viewer users is considered important for revitalizing the community between distributor users and viewer users and for operating the community in a healthy manner.

[0009] An object of the present disclosure is to provide a technical improvement that solves or alleviates at least a part of the problems of the conventional technology described above. A more specific object of the present disclosure is to provide a video distribution system, a video distribution method, and a video distribution program that reduce the processing load caused by the participation of avatars and do not hinder the participation of highly engaged viewers. [Means for solving the problem]

[0010] One aspect of the present invention relates to a video distribution system that live-distributes a video including a virtual space and an animation of a character object generated based on a motion of a broadcasting user. The video distribution system includes one or more computer processors. The one or more computer processors execute computer-readable instructions to determine whether a viewing user who watches the video belongs to a first group or a second group in response to receiving a request to participate in the video from the viewing user, and, if it is determined that the viewing user belongs to the first group, place a specific avatar of the viewing user in a first area in the virtual space, and, if it is determined that the viewing user belongs to the second group, place the specific avatar in a second area in the virtual space in response to satisfying a participation condition. Place in.

[0011] In one embodiment, the first region is disposed closer to the character object in the virtual space than the second region.

[0012] In one embodiment, the participation condition is that the number of avatars of users other than the viewing user displayed in the second area is less than an upper limit of the number of avatars displayed in the second area.

[0013] In one embodiment, the condition for participation is that the viewing user is selected by the broadcast user.

[0014] The video distribution system according to one embodiment further includes a storage that stores avatar data for displaying the specific avatar in the virtual space, and displays the specific avatar in the virtual space based on the avatar data read from the storage.

[0015] A video distribution system according to one embodiment receives an action instruction for the specific avatar placed in the virtual space, and controls a movement of the specific avatar displayed in the virtual space based on the action instruction.

[0016] In one embodiment, the video distribution system accepts a display request for the specific avatar while the specific avatar is located in the virtual space, and, in response to the display request being granted, displays the specific avatar in a third area in which the character object is located.

[0017] In one embodiment, the specific avatar is arranged in the third area when the viewing user is selected by the broadcast user after a display request for the specific avatar is accepted.

[0018] One aspect of the present invention relates to a video distribution method for live-streaming a video including animation of a character object generated based on a movement of a broadcasting user by executing computer-readable instructions by one or more computer processors, the video distribution method comprising: a step of determining whether a viewing user who watches the video belongs to a first group or a second group in response to receiving a participation request from the viewing user who watches the video, the step of placing a specific avatar of the viewing user in a first area in the virtual space when it is determined that the viewing user belongs to the first group, and a step of placing the specific avatar in a second area in the virtual space in response to satisfying a participation condition in response to determining that the viewing user belongs to the second group.

[0019] One aspect of the present invention relates to a video distribution program for live-distributing a video including an animation of a character object generated based on a motion of a broadcasting user, the video distribution program causing one or more computer processors to execute the steps of: determining whether a viewing user who watches the video belongs to a first group or a second group in response to receiving a request to join the video from the viewing user, placing a specific avatar of the viewing user in a first area in the virtual space when it is determined that the viewing user belongs to the first group; and placing the specific avatar in a second area in the virtual space in response to satisfying a participation condition in response to determining that the viewing user belongs to the second group. Effect of the Invention

[0020] According to an embodiment of the present invention, it is possible to suppress the processing load caused by the participation of an avatar and not to hinder the participation of highly engaged viewing users. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing a video distribution system according to an embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing the facilities of a studio for producing videos to be distributed by the video distribution system of FIG. 1. [Diagram 3] 2 is a diagram illustrating user data stored in the video distribution system of FIG. 1. [Figure 4] 2 is a diagram illustrating avatar data stored in the video distribution system of FIG. 1. FIG. [Diagram 5] 2 is a diagram illustrating participation management data stored in the moving image distribution system of FIG. 1. [Figure 6] 2 is a diagram illustrating a virtual space displayed on a video distributed in the video distribution system of FIG. 1. [Figure 7a]7A and 7B are diagrams illustrating an example of a display screen displayed on a client device 10a in an embodiment, in which a participation request button for allowing an avatar to participate in a video is displayed on the display screen of FIG. [Figure 7b] 7B is a diagram showing an example of a display screen displayed on the client device 10a in one embodiment. In the display screen of Fig. 7B, a display request button for requesting that an avatar be displayed together with the broadcast user's character is displayed. [Figure 7c] FIG. 13 is a diagram showing an example of a display screen including an image of a virtual space captured by another virtual camera. [Figure 7d] FIG. 13 is a diagram showing an example of a display screen including an image of the virtual space captured by yet another virtual camera. [Figure 7e] FIG. 13 is a diagram showing an example of a display screen indicating that a feature display request has been made. [Figure 7f] FIG. 13 is a diagram showing an example of a display screen on which an avatar and a broadcast user's character are displayed together. [Figure 8] FIG. 11 is a flow diagram showing a flow of a video distribution process in one embodiment. [Figure 9] FIG. 13 is a flow diagram illustrating processing an avatar's join request in one embodiment. [Figure 10] FIG. 11 is a flow diagram illustrating a process for processing a request to display an avatar in one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Various embodiments of the present invention will now be described with reference to the accompanying drawings, in which the same reference numerals are used to designate the same or similar components throughout the drawings.

[0023] A video distribution system according to an embodiment will be described with reference to Fig. 1 to Fig. 6. Fig. 1 is a block diagram showing a video distribution system 1 according to an embodiment, Fig. 2 is a schematic diagram showing the equipment of a studio where videos to be distributed by the video distribution system 1 are produced, Fig. 3 to Fig. 5 are diagrams for explaining information stored in the video distribution system 1, and Fig. 6 is a diagram showing a virtual space displayed in the video.

[0024] The video distribution system 1 includes client devices 10a to 10c, a server device 20, a studio unit 30, and a storage 25. The client devices 10a to 10c, the server device 20, and the storage 25 are connected to each other via a network 15 so as to be able to communicate with each other. The server device 20 is configured to distribute video including character animation, as described below. The characters included in the video may be motion-controlled in a virtual space.

[0025] The video may be distributed from the server device 20 to each of the client devices 10a to 10c. A first viewing user who is a user of the client device 10a, a second viewing user who is a user of the client device 10b, and a third viewing user who is a user of the client device 10c can view the distributed video by each client device. The video distribution system 1 may include fewer than three client devices, or may include more than three client devices. The client devices 10a to 10c and other users who use client devices to view the video distributed from the server device 20 are collectively referred to as viewing users. The video distributed from the server device 20 may be viewed by the client devices 10a to 10c that can execute application software for viewing the video. In order to view the video distributed from the server device 20, application software having an instruction set that realizes a function for viewing the video may be downloaded and installed on the client devices 10a to 10c. Such application software having a function for viewing the video distributed from the server device 20 may be called "viewing application software" or "viewing application". The video distributed from the server device 20 may be viewed by executing a function of a viewing application. In one embodiment, the video distributed from the server device 20 may be viewed by a client device among the client devices 10a to 10c in which a viewing application is installed, and may not be viewed by a client device in which the viewing application is not installed. A viewing user who views a video distributed from the server device 20 can display his / her own avatar in the video. In other words, the viewing user can participate in the distributed video through his / her own avatar. The viewing user can display his / her own avatar in the distributed video, and can interact with the distribution user of the video (or a character in the distributed video) and other viewing users through the avatar.

[0026] The client devices 10a to 10c are information processing devices such as smartphones. The client devices 10a to 10c may be, other than smartphones, mobile phones, tablet terminals, personal computers, e-book readers, wearable computers, game consoles, and various other information processing devices capable of playing videos. Each of the client devices 10a to 10c may include a computer processor, a memory, a communication I / F, a display, a sensor unit including various sensors such as a gyro sensor, a sound collecting device such as a microphone, and a storage device for storing various information.

[0027] The display of each of the client devices 10a to 10c includes a display panel and a touch panel. The touch panel is configured to detect a touch operation (contact operation) by a player. The touch panel can detect various touch operations by the player, such as tapping, double tapping, and dragging. The touch panel may include a capacitance-type proximity sensor and be configured to detect a non-contact operation by a player.

[0028] The viewing user can input a message related to the distributed video or other messages via the touch panel of the client device 10a to 10c. The message may be, for example, a text message, a still image message, a video message, or any other electronic message format. Each of the client devices 10a to 10c can be configured to post (transmit) the message input by the viewing user to the server device 20.

[0029] In the illustrated embodiment, the server device 20 includes a computer processor 21, a communication I / F 22, and a storage 23.

[0030] The computer processor 21 is an arithmetic device that loads an operating system and various programs that realize various functions from the storage 23 or other storages into a memory and executes instructions included in the loaded programs. The computer processor 21 is, for example, a CPU, an MPU, a DSP, a GPU, various other arithmetic devices, or a combination of these. The computer processor 21 may be realized by an integrated circuit such as an ASIC, a PLD, an FPGA, or an MCU. Although the computer processor 21 is illustrated as a single component in FIG. 1, the computer processor 21 may be a collection of multiple physically separate computer processors. In this specification, a program described as being executed by the computer processor 21 or instructions included in the program may be executed by a single computer processor, or may be executed in a distributed manner by multiple computer processors. In addition, a program executed by the computer processor 21 or instructions included in the program may be executed by multiple virtual computer processors. The functions realized by the computer processor 21 will be described later.

[0031] The communication I / F 22 is implemented as hardware, firmware, or communication software such as a TCP / IP driver or a PPP driver, or a combination of these. The server device 20 can transmit and receive data to and from other devices via the communication I / F 22.

[0032] The storage 23 is a storage device accessed by the computer processor 21. The storage 23 is, for example, a magnetic disk, an optical disk, a semiconductor memory, or any other storage device capable of storing data. Various programs can be stored in the storage 23. At least a portion of the programs and various data that can be stored in the storage 23 may be stored in a storage (for example, the storage 25) that is physically separate from the server device 20. Details of the data stored in the storage 23 will be described later.

[0033] 2. As shown in the figure, an actor A1 is performing in the studio room R. The studio unit 30 is configured to detect the movements and facial expressions of the actor A1 and output the detected information to the server device 20.

[0034] Actor A1 is a subject whose movements and expressions are captured by a group of sensors (described later) provided in the studio unit 30. In the illustrated embodiment, actor A1 is a human who talks, performs, and takes other actions to progress a program that is distributed as a video. Actor A1 may be an animal or any other moving object (non-living object) other than a human. Actor A1 may be, for example, a robot capable of autonomous movement. The number of actors in studio room R may be one, or three or more.

[0035] The studio unit 30 includes six motion sensors 31a to 31f attached to the actor A1, a controller 33a held in the left hand of the actor A1, a controller 33b held in the right hand of the actor A1, a camera 37a attached to the head of the actor A1 via a wearing device 37b, and a wristwatch-type actor device 91 attached to the left arm of the actor A1. A microphone for acquiring voice data may be attached to the wearing device 37b. This microphone can acquire the speech of the actor A1 as voice data. The microphone may be a wearable microphone attached to the actor A1 via the wearing device 37b, or may be a stationary microphone installed on the floor, wall, or ceiling of the studio room R. In addition to the above components, the studio unit 30 includes a base station 35a, a base station 35b, a digital camera 36a, and a display 39. A supporter computer 40 is installed in an adjacent room separated from the studio room R by a glass window. The server device 20 may be installed in the same room as the supporter computer 40.

[0036] The motion sensors 31a to 31f detect their own position and orientation in cooperation with the base station 35a and the base station 35b. In one embodiment, the base station 35a and the base station 35b are multi-axis laser emitters. The base station 35a emits a flashing light for synchronization, and then scans the laser light, for example, around a vertical axis. The base station 35b scans the laser light, for example, around a horizontal axis. Each of the motion sensors 31a to 31f may include a plurality of optical sensors that detect the incidence of the flashing light and the laser light from the base station 35a and the base station 35b. Each of the motion sensors 31a to 31f may detect its own position and orientation based on the time difference between the incidence timing of the flashing light and the incidence timing of the laser light, the light reception time at each optical sensor, the incidence angle of the laser light detected by each optical sensor, and other information as necessary. The motion sensors 31a to 31f may be, for example, a Vive Tracker provided by HTC CORPORATION. Base station 35a and base station 35b may be, for example, base stations provided by HTC CORPORATION.

[0037] Detection information indicating the position and orientation of each of the motion sensors 31a to 31f calculated in each of the motion sensors is transmitted to the server device 20. This detection information may be wirelessly transmitted from each of the motion sensors 31a to 31f to the server device 20. The base station 35a and the base station 35b emit blinking light and perform laser light scanning at regular intervals, so that the detection information of each motion sensor is updated at each interval. The number of base stations may be three or more. The position of the base station may be changed as appropriate. For example, in addition to or instead of the base station arranged in the upper corner of the space to be detected by the tracking sensor shown in FIG. 2, a base station arranged in a pair at an upper position and a lower position close to the floor surface may be provided. For example, the number of base stations may be four.

[0038] In the illustrated embodiment, the actor A1 wears six motion sensors 31a to 31f. The motion sensors 31a, 31b, 31c, 31d, 31e, and 31f are respectively worn on the left back of the hand, the right back of the hand, the left instep, the right instep, the waist, and the top of the head of the actor A1. The motion sensors 31a to 31f may be worn on the actor A1 via a wearing tool. The motion sensors 31a to 31f shown in FIG. 2 are exemplary. The motion sensors 31a to 31f may be worn on various parts of the actor A1. The number of motion sensors worn on the actor A1 may be five or less, or seven or more. In this way, the body movement of the actor A1 can be detected by detecting the positions and orientations of the motion sensors 31a to 31f worn on each part of the body of the actor A1. The body movement of the actor A1 may be detected by an optical motion capture system.

[0039] In one embodiment, a number of infrared LEDs may be mounted on each of the multiple motion sensors attached to the actor A1, and the position and orientation of each of the motion sensors may be detected by detecting light from the infrared LEDs with an infrared camera installed on the floor or wall of the studio room R. Visible light LEDs may be used instead of infrared LEDs, and the position and orientation of each of the motion sensors may be detected by detecting light from the visible light LEDs with a visible light camera. In this way, a light emitting unit (e.g., an infrared LED or a visible light LED) may be provided on each of the multiple motion sensors attached to the actor, and the position and orientation of each of the motion sensors may be detected by detecting light from the light emitting unit with a light receiving unit (e.g., an infrared camera or a visible light camera) installed in the studio room R.

[0040] In one embodiment, a plurality of reflective markers can be used instead of the motion sensors 31a to 31f. The reflective markers are attached to each of the actors A1 with adhesive tape or the like. The actor A1 to which the reflective markers are attached is photographed to generate photographic data, and the photographic data is subjected to image processing, thereby making it possible to detect the position and orientation of the reflective markers. In one embodiment, the detection of the body motion of the actor A1 may be performed using a suit with an inertial sensor built in. The motion of the actor A1 can be detected by analyzing a sensor signal output from an inertial sensor of the inertial sensor built in suit worn by the actor A1. As the suit with an inertial sensor built in, a suit with an inertial sensor built in that is compatible with a motion capture system called MVN developed by Xsens.com can be used.

[0041] The controller 33a and the controller 33b output to the server device 20 a control signal in response to the operation of the actor A1.

[0042] The digital camera 36a captures an image of the actor A1 to generate imaging data. The position, imaging direction, and angle of view of the digital camera 36a are set according to an operation by an operator. The digital camera 36a transmits imaging setting information indicating its position, imaging direction, and angle of view to the server device 20. Similarly, the position, imaging direction, and angle of view of the digital camera 36b are set according to an operation by an operator. In the illustrated embodiment, the digital camera 36a is provided on the left front side of the actor A1.

[0043] The camera 37a is attached to the head of the actor A1 as described above. For example, the camera 37a is arranged so as to be able to capture an image of the face of the actor A1. The camera 37a continuously captures an image of the face of the actor A1 and acquires image data of the face of the actor A1. The camera 37a transmits the image data of the face of the actor A1 to the server device 20. The camera 37a may be a 3D camera capable of detecting the depth of a person's face. The camera 37a may be, for example, a LIDAR device provided in an iPhone X provided by Apple Inc. The iPhone X is attached to, for example, a head gear worn on the head of the actor A1. The camera 37a may be a fixed type camera mounted on a camera stand installed in the studio room R, instead of being mounted on the actors A1 and A2.

[0044] The display 39 is configured to display information received from the support computer 40. Information transmitted from the support computer 40 to the display 39 may include, for example, text information, image information, and various other types of information. The display 39 is disposed in a position viewable by the actor A1.

[0045] The actor device 91 is configured to generate a stimulus other than sound that can be perceived by the actor A1 in response to a drive signal from the server device 20. If the actor device 91 generates a sound, there is a risk that the sound will be synthesized into the distribution content, so the actor device 91 is configured to generate a stimulus other than sound. The actor device 91 can generate a stimulus other than an auditory stimulus (typically, sound), for example, a tactile stimulus, a visual stimulus, or a stimulus that can be perceived by the actor A1 that is different from the auditory stimulus other than these. The actor device 91 may include an actuator 91a. The actuator 91a is configured to be activated by a drive signal from the server device 20. The actuator 91a may be configured to vibrate when a drive signal is input. For example, an Apple Watch provided by Apple Inc. may be used as the actor device 91. The actor device 91 may have various shapes other than a wristwatch type. The actor A1 can wear a plurality of actor devices 91 of the same type or different types. The actor device 91 may be configured to be activated in response to a drive signal from a source other than the server device 20. The multiple actor devices 91 may be associated with an actor ID that identifies the actor A1. The actor devices 91 can transmit tactile stimuli to the actor A1 via an actuator 91a. The actor devices 91 may be LEDs (Light Emitting Diodes) or sensory display devices. The LEDs or sensory display devices can generate visual stimuli. The sensory display device may be an electrotactile display that drives nerve axons under the skin with current flowing from electrodes.

[0046] In the illustrated embodiment, the supporter computer 40 is installed in a room next to the studio room R. The room in which the supporter computer 40 is installed is separated from the studio room R by a glass window, so that the operator of the supporter computer 40 (sometimes referred to as a "supporter" in this specification) can see the actor A1. In the illustrated embodiment, supporters B1 and B2 are present in the room as operators of the supporter computer 40.

[0047] The supporter computer 40 may be configured to change the settings of the components of the studio unit 30 in response to the operations of the supporter B1 and the supporter B2. The supporter computer 40 can, for example, set the scanning intervals of the base stations 35a and 35b, set the positions or orientations of the digital cameras 36a and the tracking sensors 36b, and change various settings of various other devices. The switching of the active sensors, which will be described later, can be performed based on the operations of the supporter computer 40 by the supporter B1 and the supporter B2. At least one of the supporter B1 and the supporter B2 can input a message to the supporter computer 40 and display the input message on the display 39.

[0048] The components and functions of the studio unit 30 shown in Fig. 2 are exemplary. The studio unit 30 applicable to the present invention may include various components not shown. The studio unit 30 may include a projector. The projector can project a video image on a screen S to be distributed to the client device 10a or other client devices.

[0049] On the other hand, the shooting for generating the video to be distributed may be performed by a simpler facility. For example, face motion data, which is a digital representation of the facial movement of the actor, may be generated by an information processing terminal such as a smartphone, and an animation of a character object whose facial expression moves in synchronization with the facial movement of the actor but whose body does not move may be generated based on the face motion data. Such face motion data may be generated by the client devices 10a to 10c. In order to generate the face motion data, application software having an instruction set that realizes a function for generating face motion data may be downloaded and installed in the client devices 10a to 10c. Such application software having a function for generating face motion data may be called "distribution application software" or "distribution app". The distribution app and the above-mentioned viewing app may be separate application software, or a single application software may have both the function of the distribution app and the function of the above-mentioned viewing app. This face motion data may include tracking data of the actor's head posture, feature points that are digital representations of the feature points of the actor's face, and blended face data that represents the deformation rate. An information processing terminal such as a smartphone may create an image showing an exaggerated facial expression of the actor based on the facial motion data, and superimpose the image on the face of the character object. When generating animation of a character object based on facial motion data, motion sensors 31a, 31b, 31c, 31d, 31e, and 31f for detecting the body movement of the actor, base stations 35a, 35b, and digital cameras 36a, 36b are not required, so animation of a character object can be generated with simpler equipment.If the motion sensors 31a, 31b, 31c, 31d, 31e, and 31f, the base stations 35a, 35b, and the digital camera 36a are not required, the generation of character objects can be performed in places other than the studio room R, such as the actor's home, thereby increasing convenience.

[0050] Next, in one aspect, a description will be given of information stored in the storage 23. In the illustrated embodiment, the storage 23 stores model data 23a, object data 23b, user data 23c, avatar data 23d, participation management data 23e, and various other information required for generating and distributing videos to be distributed.

[0051] The model data 23a is model data for generating animation of a character. The model data 23a may be three-dimensional model data for generating three-dimensional animation, or may be two-dimensional model data for generating two-dimensional model data. The model data 23a includes, for example, rig data (sometimes called "skeleton data") indicating the skeleton of a character, and surface data indicating the shape and texture of the surface of the character. The model data 23a may include a plurality of model data that are different from each other. The plurality of model data may have different rig data from each other, or may have the same rig data. The plurality of model data may have different surface data from each other, or may have the same surface data.

[0052] The object data 23b includes asset data for constructing a virtual space that constitutes a video. The object data 23b includes data for drawing the background of the virtual space that constitutes the video, data for drawing various objects displayed in the video, and data for drawing various objects other than these that are displayed in the video. The object data 23b may include object position information that indicates the position of an object in the virtual space.

[0053] In addition to the above, the object data 23b may include gift objects that are displayed in the video based on a display request from the viewing user of the client device 10a to 10c. The gift objects may include effect objects, normal objects, and decorative objects. The viewing user may purchase a desired gift object.

[0054] An effect object is an object that affects the overall impression of the viewing screen of a distributed video, for example, an object that resembles confetti. The object that resembles confetti may be displayed over the entire viewing screen, thereby changing the overall impression of the viewing screen before and after the display. An effect object may be displayed so as to overlap with a character object, but differs from a decorative object in that it is not displayed in association with a specific part of the character object.

[0055] A normal object is an object that is a digital gift from a viewing user to an actor (e.g., actor A1) or a character in a video, and is, for example, an object that resembles a stuffed animal or a bouquet of flowers. In one aspect, a normal object is displayed on a display screen of a video so as not to come into contact with a character object. In one aspect, a normal object is displayed on a display screen of a video so as not to overlap with a character object. A normal object may be displayed so as to overlap with an object other than a character object in a virtual space. A normal object may be displayed so as to overlap with a character object, but is different from a decorative object in that a display is not made in association with a specific part of a character object. In one aspect, when a normal object is displayed so as to overlap with a character object, the normal object is displayed so as to overlap with a part of the character object other than the head, including the face, and not to overlap with the head of the character object.

[0056] A decorative object is an object that is displayed on the display screen in association with a specific part of a character object. In one aspect, a decorative object that is displayed on the display screen in association with a specific part of a character object is displayed on the display screen so as to be in contact with the specific part of the character object. In one aspect, a decorative object that is displayed on the display screen in association with a specific part of a character object is displayed on the display screen so as to cover a part or all of the specific part of the character object.

[0057] The decorative object is, for example, an accessory (headband, necklace, earring, etc.), clothing (T-shirt, etc.) or costume worn by a character object, and other objects that can be worn by a character object. The object data 23b corresponding to the decorative object may include wearing position information indicating which part of the character object the decorative object is associated with. The wearing position information of a certain decorative object may indicate which part of the character object the decorative object is worn on. For example, if the decorative object is a headband, the wearing position information of the decorative object may indicate that the decorative object is worn on the "head" of the character object. If the decorative object is a T-shirt, the wearing position information of the decorative object may indicate that the decorative object is worn on the "torso" of the character object.

[0058] Each gift object may have a display time set according to its type. In one embodiment, the display time of the decorative object may be set longer than the display time of the effect object and the display time of the normal object. For example, the display time of the decorative object may be set to 60 seconds, the display time of the effect object may be set to 5 seconds, and the display time of the normal object may be set to 10 seconds.

[0059] The object data 23b may include a list indicating gift objects owned by users of the video distribution system 1. In the owned list, an object ID that identifies a gift object owned by the viewing user may be stored in association with the user's account information. The user's account information may be, for example, a user ID that identifies the user. The users may include, for example, the first to third viewing users of the client devices 10a to 10c.

[0060] As shown in FIG. 3, the user data 23c can include account information of each user (e.g., a viewing user) of the video distribution system 1, avatar information related to an avatar used by each user, and group information related to a group to which each user belongs. In other words, the storage 23 can store, in association with the account information of each user of the video distribution system 1, avatar information related to an avatar used by each user, group information related to a group to which each user belongs, and information on other users. The account information of a certain user is, for example, a user ID that identifies the user. The user ID of the user may be issued when the user uses the client devices 10a to 10c to register for use of a video service provided by the server device 20. A common user ID may be used when using a viewing application to use the service of the server device 20 and when using a distribution application to use the service of the server device 20, or a different user ID may be used when using a viewing application to use the service of the server device 20 and when using a distribution application to use the service of the server device 20. When separate user IDs are used for using the service of server device 20 using a viewing application and for using the service of server device 20 using a distribution application, the user ID for using the viewing application (viewing user ID) and the user ID for using the distribution application (distribution user ID) of the same user may be stored in association with each other. The service of server device 20 that can be received by using the viewing application is, for example, a viewing service of videos distributed from server device 20. The service of server device 20 that can be received by using the distribution application is, for example, a service of distributing videos including the user's own character object.

[0061] The avatar information of a certain user is, for example, an avatar ID that identifies an avatar that the user uses in the video system 1. The avatar ID is assigned to the user when the user registers an avatar. The user registers an avatar by selecting the head, hairstyle, clothes, accessories, and other parts that constitute the avatar. The avatar may be registered by the client devices 10a to 10c in which a distribution application is installed. That is, the avatar may be registered by executing a function of the distribution application. In one embodiment, the avatar may be registered by a client device in which the distribution application is installed among the client devices 10a to 10c, and may not be executable by a client device in which the distribution application is not installed. The avatar may be registered by application software having a function of creating, editing, and / or registering an avatar that is separate from the distribution application (hereinafter, may be referred to as an "avatar creation application"). When an avatar is registered using a distribution application and / or an avatar creation application, an avatar ID for identifying the avatar may be assigned and the avatar ID may be associated with the user ID of the user and managed as user information, as described with reference to FIG. 3. After registering an avatar, the user may change the avatar. For example, after registering an avatar, the user may change some or all of the features of the avatar. For example, even after registration, the user may change the avatar's hairstyle, possessed items, and other features that constitute the avatar.

[0062] The group information of a certain user is a group ID that identifies a group to which the user belongs. Groups to which a user of the video distribution system 1 can belong include, for example, a first group and a second group. A user of the distribution system 1 belongs to either the first group or the second group. The second group is a group different from the first group. Whether a user belongs to the first group or the second group can be determined in various ways. It is not necessary to use a group ID to determine whether a user belongs to the first group or the second group. For example, in user data, if a flag is set in the record of a user who belongs to the first group and a flag is not set in the record of a user who belongs to the second group, it can be determined whether a user belongs to the first group or the second group. Which group a user belongs to can be determined by various criteria. For example, a user with a relatively high engagement with the video system 1 can belong to the first group, and a user with a relatively low engagement with the video system 1 can belong to the second group. A certain user's engagement with the video system 1 is expressed, for example, by points. When a user uses the video system 1, the points of the user may increase or decrease depending on the usage. A user's points may increase depending on, for example, watching a video, distributing a video, posting a comment on a video, gifting a distributor who distributes a video, purchasing a gift (gift object), logging in to the video system 1, starting application software for watching a video, sending a message to other users in a social networking service provided by the video system 1, receiving a positive evaluation from other users in the social networking service, and other uses of the video system 1. A user's points may decrease depending on, for example, being prohibited from viewing (being blocked) by a distributor of a video, having a posted comment deleted, receiving a negative evaluation from other users in a social networking service provided by the video system 1, and other uses of the video system 1.User actions associated with an increase or decrease in points are not limited to those mentioned above. A user's points may be increased by a user's actions that strengthen the connection between the user and the distributor user or other viewing users, and may be decreased by a user's actions that weaken the connection between the user and the distributor user or other viewing users.

[0063] In one embodiment, the first group is composed of fewer users than the second group. When a user registers for use with the video distribution system 1, the user may be classified into the second group, and when the user meets the criteria for being classified into the first group after starting to use the video distribution system 1, the user may leave the second group and belong to the first group. When registering for use with the video distribution system 1, the user may not be classified into groups, and only users who meet certain conditions after starting to use the system may belong to the first group. Whether a user belongs to the first group or not may be determined, for example, by setting a flag for identifying a user who belongs to the first group in the user data and referring to the flag. By making a user who has started to use the system belong to the second group, the number of users who belong to the first group can be made smaller than the number of users who belong to the second group. The group to which a user belongs may be changed depending on the usage mode of the video distribution system 1. When a user no longer meets the criteria for being classified into the first group, the group to which the user belongs may be changed from the first group to the second group.

[0064] In one embodiment, the classification criteria for the first group may be adjusted so that the number of users belonging to the first group is maintained to be smaller than the number of users belonging to the second group. For example, the server device 20 may monitor the number of users belonging to the first group and the number of users belonging to the second group, and when the difference between them becomes smaller than a predetermined number, the criteria for classification into the first group may be tightened. For example, in a case where users having 100 points or more are initially classified into the first group, the criteria may be changed so that users having 200 points or more are classified into the first group when the difference between the number of users belonging to the first group and the number of users belonging to the second group becomes smaller than a predetermined value.

[0065] In one embodiment, an upper limit number of users belonging to the first group may be determined. In this case, a ranking of points may be created for users of the video distribution system 1, and the users from the top of the ranking until the upper limit number may belong to the first group. Since not all users at the top of the ranking are necessarily logged in to the video distribution system 1, a ranking of users who are logged in to the video system 1 may be created, and the users from the top of the ranking until the upper limit number may belong to the first group.

[0066] The points of each user referred to for grouping may be common points common to the entire video system 1, or individual points managed for each broadcast user. For example, for a certain user, individual points for broadcast user A and individual points for broadcast user B may be managed separately. For a video distributed by broadcaster A (i.e., a video including a character object of broadcaster A), a ranking of individual points managed for this broadcaster A may be created, and users from the top of the ranking up to the upper limit may be placed in a first group. In this way, a user who belongs to the second group in the ranking of common points common to the entire video system 1 may be placed in the first group for a specific broadcast user. By managing individual points in this way, it is possible to group the broadcast user according to the degree of each user's individual engagement with the broadcast user.

[0067] As described above, if a user's points are higher than a predetermined threshold, the user is made to belong to the first group, and conversely, if the user's points are lower than the predetermined threshold, the user is made to belong to the second group. This allows users with points higher than the threshold to belong to the first group, and users with points lower than the threshold to belong to the second group. If the points represent engagement with the video system 1, users with relatively high engagement with the video system 1 can belong to the first group, and users with relatively high engagement with the video system 1 can belong to the second group.

[0068] The grouping of users is not limited to those specifically described in this specification. Users may be classified into three or more groups. In addition, the classification criteria that determine which group a user belongs to may be changed as appropriate.

[0069] As shown in FIG. 4, the avatar data 23d may include avatar identification information of an avatar used by a user in the video distribution system 1, and avatar display information for displaying the avatar in a video. In other words, the storage 23 may store avatar display information for displaying the avatar in association with the avatar identification information of the avatar. The avatar identification information is, for example, an avatar ID for identifying an avatar. The avatar is displayed in a video as, for example, an image imitating a human or an animal. The avatar display information is information used for displaying the avatar in the video. The avatar information may include, for example, part information indicating images of the head, hairstyle, facial parts (eyes, nose, mouth, etc.), torso, clothes, accessories, items, and other parts constituting the avatar. A user may register his / her own avatar by selecting a preferred part image. The avatar display information may include 2D display information for displaying the avatar in 2D in the video, and 3D display information for displaying the avatar in 3D in the video. The 3D display information includes parts information indicating images of parts for stereoscopically displaying the avatar in the video, rig data for expressing the avatar's three-dimensional movements, and other known information used for stereoscopically displaying the avatar.

[0070] A viewing user can have his / her avatar participate in a video distributed from the video distribution system 1. The participation management data 23e stored in the storage 23 is data for managing avatars participating in a video. As shown in FIG. 5, the participation management data 23e can include video identification information for identifying a video distributed in the video distribution system 1, participation restriction information for determining an upper limit number of avatars that can participate in the video, and participating avatar information indicating avatars participating in the video. The video identification information is, for example, a video ID for identifying a video. The participation restriction information for a certain video is, for example, an upper limit number of avatars that can participate in the video. The participation restriction information may include a second upper limit number that is an upper limit number of avatars that can be arranged in a second area of ​​the virtual space described later. The participation restriction information may include, in addition to the second upper limit number, a first upper limit number that is an upper limit number of avatars that can be arranged in a first area of ​​the virtual space described later. The participating avatar information for a certain video may be an avatar ID for identifying an avatar participating in the video. When multiple avatars participate in a video, the participating avatar information includes the avatar IDs of each of the multiple avatars.

[0071] Next, the functions realized by the computer processor 21 will be described in more detail. The computer processor 21 executes computer-readable instructions included in the distribution program to function as a body motion data generating unit 21a, a face motion data generating unit 21b, an animation generating unit 21c, a video generating unit 21d, a video distribution unit 21e, a gift processing unit 21f, a participation request processing unit 21g, an avatar display unit 21h, an avatar control unit 21i, and a display request processing unit 21j. At least a part of the functions realized by the computer processor 21 may be realized by a computer processor other than the computer processor 21 of the video distribution system 1. At least a part of the functions realized by the computer processor 21 may be realized by, for example, a computer processor installed in the supporter computer 40 or another computer processor not shown.

[0072] The body motion data generating unit 21a generates body motion data of each part of the body of the actor A1 based on the detection information of each of the motion sensors 31a to 31f. The body motion data is generated as time passes. For example, the body motion data may be generated at a predetermined sampling time interval. In this way, the body motion data can express the body movement of the actor A1 in time series as digital data. In the illustrated embodiment, the motion sensors 31a to 31f are attached to the left and right limbs, waist, and head of the actor A1, so that the position and orientation of the actor A1's entire body can be digitally expressed over time based on the detection information of the motion sensors 31a to 31f. The body motion data can specify, for example, the positions and rotation angles of bones constituting the rig data included in the model data 23a.

[0073] The face motion data generating unit 21b generates face motion data, which is a digital representation of the facial movement of the actor A1, based on the imaging data of the camera 37a. The face motion data is generated as time passes. For example, the face motion data may be generated at predetermined sampling time intervals. In this way, the face motion data can digitally represent the facial movement (changes in facial expression) of the actor A1 in time series.

[0074] The animation generation unit 21c is configured to generate animation of a character object that moves in a virtual space and changes its facial expression by applying the body motion data generated by the body motion data generation unit 21a and the facial motion data generated by the facial motion data generation unit 21b to predetermined model data included in the model data 23a. Specifically, the animation generation unit 21c can generate animation of a character object that moves in synchronization with the body and facial expression movements of actor A1, based on the body motion data and facial motion data related to actor A1.

[0075] The video generator 21d uses the object data 23b to construct a virtual space, and generates a video including the virtual space and an animation of a character object corresponding to the actor A1. The character object of the actor A1 may be placed in the virtual space so as to match the position of the actor A1 relative to the digital camera 36a.

[0076] In one aspect, the video generation unit 21d determines setting information for a virtual camera (position in the virtual space, gaze position, gaze direction, and angle of view), determines a drawing area of ​​the entire virtual space based on the setting information for this virtual camera, and generates video information for displaying the drawing area of ​​this virtual space.

[0077] The video generating unit 21d can synthesize the generated video with the voice of the actor A1 acquired from the microphone of the studio unit 30. In this manner, the video generating unit 21d can generate animation of a character object that moves in synchronization with the body and facial movements of the actor A1, and generate a video for distribution in which the voice of the actor A1 is synthesized with this animation.

[0078] Before distributing a video, the video generating unit 21d can load avatar display information required to display an avatar that may participate in the video. The video generating unit 21d can load 3D display information that takes a long time to load and imposes a large processing load for loading, among the avatar display information. For example, the video generating unit 21d can read and load avatar display information required to display an avatar of a user belonging to the first group from the storage 23. In one embodiment, the video generating unit 21d can read and load 3D display information required to 3D display an avatar of a user belonging to the first group from the storage 23.

[0079] In one embodiment, the animation generating unit 21d can load 3D display information of avatars whose number is equal to or less than the first upper limit number. In one embodiment, the animation generating unit 21d can load 3D display information of avatars whose number is equal to or less than the sum of the first upper limit number and the second upper limit number.

[0080] The loaded avatar display information can be referenced when an avatar is displayed in a video in response to a participation request from a user. The use of the loaded avatar display information when an avatar is displayed in a video will be described later.

[0081] The video distribution unit 21e distributes the video generated by the video generation unit 21d. This video is distributed to the client devices 10a to 10c and other client devices via the network 15. The received video is played on the client devices 10a to 10c. This video may be distributed to a client device (not shown) installed in the studio room R, and projected onto a screen S from this client device via a short focus projector. This video may also be distributed to the supporter computer 40. This allows supporter B1 and supporter B2 to check the viewing screen of the distributed video.

[0082] A user involved in generating and / or distributing a video distributed from the video distribution unit 21e may be called a distribution user. In the above example, the video distributed from the server device 20 includes a character object generated based on the body motion data and face motion data of the actor A1, and is distributed with the cooperation of the supporter B1 and the supporter B2 as necessary. For this reason, each of the actor A1, the supporter B1, and the supporter B2 may be referred to as a "distribution user," or the actor A1, the supporter B1, and the supporter B2 may be referred to collectively as a "distribution user."

[0083] An example of a virtual space constructed by the video generating unit 21d is shown in FIG. 6. The virtual space is defined as a spherical space surrounding a central position. A position in the virtual space is specified by a coordinate value in a three-dimensional global coordinate system set in the virtual space. However, for convenience of explanation, only a part of the virtual space is shown in FIG. 6, and the coordinate system is also omitted in FIG. 6. In FIG. 6, it is assumed that 10 users participate in the video through avatars.

[0084] In the virtual space 50 shown in FIG. 6, a character object 51 of the actor A1, five virtual cameras 53a to 53e, a stage object 54 representing a stage on which the character object 51 is placed, and avatar objects 56a to 56c and 57a to 57g representing avatars are arranged. The stage object 54 includes a floor object 54a corresponding to the studio floor on which the character object 51 stands, and a screen object 54b extending vertically from the floor object 54a at one end of the floor object 54a in the horizontal direction. For example, the floor object 54a extends in the horizontal direction in the virtual space, and the screen object 54b extends in the vertical direction in the virtual space. In the virtual space 50, a first area 52a and a second area 52b are defined. In the illustrated embodiment, the avatar objects 56a to 56c are arranged in the first area 52a, and the avatar objects 57a to 57g are arranged in the second area 52b. In the virtual space 50, the first region 52a is disposed closer to the character object 51 than the second region 52b. In one embodiment, the character object 51 is disposed within a floor object 54a. Therefore, the first region 52a is disposed closer to the floor object 54a than the second region 52b. The positions of the first region 52a and the second region 52b in the virtual space 50 are not limited to the positions shown in FIG. 6. The first region 52a and the second region 52b may be disposed at any positions within the virtual space 50. For example, the first region 52a and the second region 52b may be disposed within the floor object 54a. The first region 52a and the second region 52b may be disposed, for example, behind the character object 51 in the floor object 54a as viewed from the viewpoint of the virtual camera 53a. In one embodiment, the first region 52a is a region for displaying an avatar of a user belonging to the first group, and the second region 52b is a region for displaying an avatar of a user belonging to the second group (or not belonging to the first group). In this specification, the area in which the character object 51 is placed may be referred to as the third area. In the illustrated example, the floor object 54a corresponds to the third area.The virtual space 50 includes comment display areas 55a to 55c in which comments posted by users watching the video are displayed. The comment display areas 55a to 55c may be objects in the virtual space 50. The virtual space 50 is not limited to the aspects explicitly described in this specification. The virtual space 50 may include various objects other than those explicitly described in this specification.

[0085] The virtual cameras 53a to 53e are arranged at different positions in the virtual space 50. All of the virtual cameras 53a to 53e are arranged to face the character object 51. An image constituting a moving image generated by the moving image generating unit 21d is an image in which one virtual camera is selected from the virtual cameras 53a to 53e and a rendering area of ​​the virtual space 50 viewed from the viewpoint of the selected virtual camera is displayed. The setting information of the virtual cameras 53a to 53e (positions, gaze position, gaze direction, and angle of view in the virtual space) may be changed during the moving image distribution by the operation of the actor A1, the supporter B1, the supporter B2, or an operator other than these.

[0086] As described above, three avatar objects 56a to 56c are arranged in the first area 52a, and seven avatar objects 57a to 57g are arranged in the second area 52b. In one embodiment, a first upper limit number that is an upper limit number of avatars that can be arranged in the first area 52a and a second upper limit number that is an upper limit number of avatars that can be arranged in the second area 52b are determined. The first upper limit number and the second upper limit number may be stored in the storage 23 as participation management data 23e.

[0087] 7a to 7f show display examples of the video generated by the video generating unit 21d and distributed by the video distributing unit 21e. FIG. 7a to FIG. 7f show display images of the video distributed by the video distributing unit 21e and played on the client devices 10a to 10c. Hereinafter, the display images of FIG. 7a to FIG. 7f will be described on the assumption that they are displayed on the client device 10a used by the first viewing user. The display screen of FIG. 7a is displayed on the display of the client device 10a by executing a viewing application on the client device 10a and playing the video distributed from the server device 20. FIG. 7a shows a display example of an image 60a drawn as a drawing area defined with the virtual camera 53a as the viewpoint. However, it is assumed that the avatar 56a of the first viewing user is not arranged in the virtual space 50 at the time of drawing the image 60a. In other words, the avatar 56a is not arranged in the virtual space 50 corresponding to the image 60a.

[0088] As shown in the figure, the image 60a includes a character object 51, a floor object 54a on which the character object 51 stands, and a screen object 54b that defines the rear end of the stage. The image 60a also includes a display field 65 in which comments from viewing users and system notifications are displayed.

[0089] The character object 51 is generated by applying the body motion data and face motion data of the actor A1 to the model data for the actor A1 included in the model data 23a. The motion of the character object 51 is controlled based on the body motion data and face motion data. Therefore, the character object 51 is controlled to move on the screen in synchronization with the body and facial movement of the actor A1.

[0090] In addition to the objects arranged in the virtual space 50, the display image 60a may display operation buttons for receiving user operations. In the example shown in Fig. 6, the display image 60a includes a gift button 61 for gifting, a rating button 62 for providing a rating, and an avatar participation button 63 for allowing an avatar to participate. Operation buttons other than these may also be displayed in the display image 60a.

[0091] The gift button 61 is displayed on the display image 60a so as to be selectable by an operation on the client device 10a on which the display image 60a is displayed. The gift button 61 can be selected, for example, by a tap operation on an area on the touch panel of the client device 10a on which the gift button 61 is displayed. In one embodiment, when the gift button 61 is selected, a window for selecting a gift to be given (gifted) to a broadcaster who is broadcasting the video being viewed is displayed on the display screen 60a. The user of the client device 10a can purchase / select a gift to be gifted from among the gifts displayed in the window. In another embodiment, in response to the selection of the gift button 61, a window including a list of purchased gifts and / or a list of selectable gifts is displayed on the display image 60a. In this case, the user watching the video can select a gift to be gifted from among the gifts displayed in the window. In one embodiment, in response to the selection of the gift button 61 on the client device 10, a gifting request may be transmitted to the server device 60. This gifting request may include gift identification information (gift ID) that identifies a gift object corresponding to the gift to be gifted. In this way, a viewing user who is viewing a video can select gift button 61 to give a desired gift to the distributor user who is distributing that video.

[0092] The gift processing unit 21f receives a gifting request from the user's client device 10a and performs processing according to the request. As described above, a user who is watching a video can transmit a gifting request to the server device 20 by operating his / her client device.

[0093] In one embodiment, when the gift processing unit 21f receives a gifting request, it identifies a gift object to be gifted based on the gifting request, and performs display processing according to the identified gift object. As described above, the gift object may include an effect object, a normal object, and a decorative object. When the gift object identified by the gifting request is an effect object, the gift processing unit 21f performs processing for displaying the effect object in the display image 60a of the video. For example, when a gifting request is made for an effect object corresponding to confetti, the gift processing unit 21f displays an effect object showing confetti in the display image 60a based on the gifting request. In one embodiment, when the gift processing unit 21f receives a gifting request for a specific normal object, it performs processing for displaying the normal object identified by the gifting request in the video. For example, when a gifting request is made for a normal object showing a stuffed bear, the gift processing unit 21f displays the normal object showing the stuffed bear in the display image 60a.

[0094] In one aspect, when the gift processing unit 21f receives a gifting request for a specific decorative object from a viewing user, the gift processing unit 21f adds the decorative object specified by the gifting request to a candidate list. This candidate list is displayed, for example, on the display of the supporter computer 40. The supporter B1 and the supporter B2 can select a decorative object to be displayed in the video from the candidate list displayed on the display of the supporter computer 40. The gift processing unit 21f displays the selected decorative object on the display screen 60a of the video. The selected decorative object is displayed on the display image 60a in association with a specific part of the character object 51. For example, if the decorative object is an object representing a headband, the decorative object may be attached to the head of the character object 51. The decorative object may be displayed on the display screen 60a so as to move in association with the movement of the specific part of the character object. For example, when the head of the character object 51 wearing a headband moves, the selected decorative object representing the headband also moves in association with the head of the character object 51 as if the headband was attached to the head of the character object 51.

[0095] The rating button 62 is displayed on the display screen 60a so as to be selectable by a user using the client device 10a. The rating button 62 can be selected, for example, by a tap operation on an area of ​​the touch panel of the client device 10a where the rating button 62 is displayed. When the rating button 62 is selected by a user watching a video, rating update information indicating that a positive rating has been given to a distributor user (e.g., actor A1) who distributes the video or a positive rating has been given to the distributed video may be transmitted to the server device 60. Selection of the rating button 62 may transmit rating information indicating that a negative rating has been given to the video to the server device 60.

[0096] The avatar participation button 63 is displayed on the display screen 60a so as to be selectable by a first viewing user using the client device 10a. The avatar participation button 63 may be selected, for example, by a tap operation on an area of ​​the touch panel of the client device 10a where the avatar participation button 63 is displayed. When the avatar participation button 63 is selected by the first viewing user who is viewing a video, a participation request is transmitted to the server device 20, requesting that the avatar of the first viewing user participate in the video. A user who transmits a participation request by selecting the avatar participation button 63 may be called a "participation application user". The participation request from the participation application user may include the user ID of the participation application user and the avatar ID of the avatar of the participation application user. When the avatar participation button 63 is selected by a first viewing user who has not registered his / her own avatar, a screen prompting the registration of an avatar may be displayed on the client device 10a. In this case, the first viewing user can register an avatar by following the guidance on the screen.

[0097] The participation request processing unit 21g accepts a participation request from a participation requesting user, and determines whether or not the avatar of the participation requesting user is to be displayed based on the participation request. In one embodiment, the participation request processing unit 21g can determine whether the participation requesting user belongs to the first group or the second group. The participation request processing unit 21g can determine whether the participation requesting user belongs to the first group or the second group, for example, by referring to group information in the user data 23c. In a case where a group to which a user belongs is distinguished by setting a flag to the user ID of a user belonging to the first group instead of the group ID in the user information, it is possible to determine that the participation requesting user belongs to the first group if the flag is set for the participation requesting user, and to determine that the participation requesting user belongs to the second group (or not belong to the first group) if the flag is not set for the participation requesting user.

[0098] When it is determined that the participation requesting user belongs to the first group, the participation request processing unit 21g places the avatar of the participation requesting user in the first area 52a of the virtual space 50. For example, when a first viewing user transmits a participation request via the client device 10a and it is determined that the first viewing user belongs to the first group, as shown in FIG. 6, the avatar 56a corresponding to the first viewing user is placed in the first area 52a. When it is determined that the first viewing user belongs to the first group, the avatar 56a of the first viewing user may be displayed in the first area 52a without requiring any additional conditions. After the participation of the avatar 56a of the first viewing user is approved, a production effect related to the participation of the avatar 56a may be displayed before or after it is placed in the first area 52a. For example, a production in which the avatar 56a is lit in a spotlight can be performed to highlight the avatar 56a. The production related to the participation of the avatar is not limited to the aspects explicitly described in this specification.

[0099] When it is determined that the participation applying user belongs to the second group, the participation request processing unit 21g determines whether the participation applying user satisfies the participation conditions, and when it is determined that the participation conditions are satisfied, places the avatar of the participation applying user in the second area 52b. For example, when a first viewing user transmits a participation request via the client device 10a and it is determined that the first viewing user belongs to the second group, the avatar 56a is placed in the second area 52b, unlike the mode shown in Fig. 6. When the avatar 56a is placed in the second area 52b, the above-mentioned dramatic effects can also be displayed in the video.

[0100] When an avatar is permitted to participate in a video based on a participation request, the avatar may be prohibited from participating in other videos. For example, when an avatar is permitted to participate in a video based on a participation request, a flag indicating that the avatar is participating in the video may be set as avatar data 23d, and whether or not to permit a request for participation in another video from a user using the avatar may be determined based on the flag. Specifically, when a request for participation in another video is made from a user using an avatar participating in a certain video, the avatar data 23d may be referenced, and if a flag indicating that the avatar is participating in the video is set for the avatar, the participation request may be rejected. The process of prohibiting a single avatar from participating in two or more videos may be performed by the participation request processing unit 21g.

[0101] As described above, when the participation applying user belongs to the second group, it is determined whether or not the participation condition is satisfied. Next, the participation condition will be described. In one embodiment, the participation condition is that the number of avatars arranged in the second area 52b is less than the second upper limit number set for the second area 52b. As an example, it is assumed that the second upper limit number set for the second area 52b in FIG. 6 is "7". In this example, when a participation request from a participation applying user belonging to the second group is accepted, since seven avatars 57a to 57g, which corresponds to the upper limit number, are already arranged in the second area 52b, it is determined that the participation condition is not satisfied. In this case, the participation request from the participation applying user is not permitted, and the avatar of the participation applying user is not arranged in the virtual space 50. As another example, it is assumed that the second upper limit number set for the second area 52b in FIG. 6 is "12". In this example, when a participation request is received from a participation applying user belonging to the second group, it is determined that the participation condition is satisfied because seven avatars, which is fewer than the upper limit, are arranged in the second area 52b. In this case, the avatar of the participation applying user is arranged in the second display area 52b.

[0102] An avatar placed in the first area 52a or the second area 52b can exit from the video in which the user is participating, in response to an operation of the user who uses the avatar. For example, when an avatar 56a of a first viewing user is placed in the first area 52a or the second area 52b of a video, the first viewing user can transmit an exit request to the server device 20 to cause the avatar 56a to exit by operating the client device 10a. When the exit request is accepted in the server device 20, the server device 20 (e.g., the participation request processing unit 21g) can perform a process to cause the avatar 56a of the first viewing user to exit from the video in response to the exit request. When an upper limit is set for the number of avatars that can participate in the first area 52a or the second area 52b, the exit of the avatar of the user who made the exit request in response to the exit request allows avatars of other users to participate in the first area 52a or the second area 52b. When an avatar leaves a video, a special effect related to the avatar's exit may be displayed in the video. For example, a special effect may be displayed such that a spotlight is shining on the exiting avatar. The special effect related to the avatar's participation is not limited to the aspects explicitly described in this specification.

[0103] In one embodiment, the process of causing an avatar to exit a video may be performed in response to a user using the avatar terminating a viewing application that is being used to view the video.

[0104] In another embodiment, even if a user who has included an avatar in a video closes the viewing app that is being used to view the video, the avatar may continue to be displayed in the video.

[0105] In another embodiment, the participation condition is that the participation request user is selected by the broadcast user. The broadcast users are, for example, actor A1, supporter B1, and supporter B2. When the participation request processing unit 21g accepts a participation request from the participation request user, it may generate a participation request user list including an indication for identifying the participation request user, and present the participation request user list to the broadcast user. The participation request user list for a certain video is a list of participation request users who have made a participation request for the video. The participation request processing unit 21g can transmit the participation request user list to the supporter computer 40. The supporter B1 and the supporter B2 can select a user to be allowed to participate from the participation request user list displayed on the display of the supporter computer 40. The participation request user list may be displayed on the display 39 of the studio room R. The actor A1 may select a user to be allowed to participate in the video from the participation request user list by operating the controller 33a or the controller 33b. The participation request processor 21g can place, in the second display area 52b, an avatar of a participation requesting user selected by a distribution user of the video (that is, permitted to participate in the video).

[0106] The avatar display unit 21h performs a process for displaying, in an image, the avatars that are determined by the participation request processing unit 21g to be placed in the first area 52a and the second area 52b. For example, when displaying the avatar of a first viewing user in the first area 52a, the avatar display unit 21h refers to the user data 23c to identify an avatar ID associated with the user ID of the first viewing user, and then refers to the avatar data 23d to identify parts information associated with the avatar ID. The avatar display unit 21h constructs an avatar object representing the avatar of the first viewing user based on the identified parts information, and places the avatar object in the first area 52a. As a result, when the first area 52a is included in the rendering area of ​​the virtual camera, the avatar object representing the avatar of the first viewing user is displayed in the video. When the avatar is displayed in 3D, the avatar object is constructed based on parts information for 3D display, and when the avatar is displayed in 2D, the avatar object is constructed based on parts information for 2D display.

[0107] Next, with reference to Figs. 7b to 7d, a display image will be described in a case where the avatar participation button 63 is selected by the first viewing user and the avatar 56a of the first viewing user is placed in the first area 52a of the virtual space 50. Fig. 7b shows an example of a display image 60b that is drawn with the virtual camera 53a as the viewpoint, similar to the display image 60a shown in Fig. 7a. Therefore, the arrangement of the character object 51 and other objects in the display image 60b is almost the same as the arrangement of the corresponding objects displayed in the display image 60a. Since the virtual camera 53a is placed between the first area 52a and the character object 51, in the display image 60b, the avatar of the first viewing user that is placed in the first area 52a or the second area 52b is not included in the drawing area with the virtual camera 53a as the viewpoint.

[0108] FIG. 7c shows another display example of a video in which the avatar 56a of the first viewing user is placed in the first area 52a in the virtual space 50. The display image 60c shown in FIG. 7c is different from FIG. 7b and is drawn with the virtual camera 53c as the viewpoint. The virtual camera 53c is placed from the right rear of the character object 51 toward the character object 51. Therefore, the display image 60c drawn with the virtual camera 53c as the viewpoint includes the character object 51 viewed from the right rear and the avatars 56a to 56c placed in the first area 52a. Due to the positional relationship between the virtual camera 53c, the first area 52a, and the second area 52b, the display image 60c displays the avatars placed in the second area 52b as if they are partly hidden behind the avatars 56a to 56c. The avatars 56a to 56c and the avatars 57a to 57g may be displayed in 2D (two-dimensional) form. In this case, the avatars 56a to 56c and 57a to 57g displayed in 2D are displayed in the virtual space 50 together with other objects displayed in 3D. By displaying the avatars 56a to 56c and 57a to 57g in 2D, the load of the drawing process can be reduced. Only the avatars 57a to 57g in the back row may be displayed in 2D (two-dimensional). As a result of the reduction in the load of the drawing process, more avatars can be displayed on the screen. In one embodiment, when 3D display information of an avatar to be placed in the virtual space 50 is loaded by the video generating unit 21d before the start of distribution of a video, the avatar can be displayed in 3D in the video. The avatar display unit 21h can check whether or not the 3D display information of an avatar to be placed in the virtual space 50 based on a participation request has been loaded. The avatar display unit 21h may display the avatar in 3D display information in a moving image when the 3D display information of the avatar has been loaded based on the participation request, and may display the avatar in 2D display information in a moving image when the 3D display information of the avatar has not been loaded. In one embodiment, the avatar display unit 21h may run-time load the 3D display information of the avatar to be placed in the virtual space 50 based on the participation request, and display the avatar in 3D in the moving image using the run-time loaded 3D display information.That is, the server device 20 may load 3D display information of an avatar to be displayed in a video during distribution of the video (in parallel with the distribution process of the video).

[0109] While an avatar is displayed in a video, the user of the avatar may change the features of the avatar to other features. When the features of the avatar displayed in the video are changed, the avatar display unit 21h can change the appearance of the avatar in the video to reflect the change. For example, when a user changes the hairstyle of his / her avatar from black to blonde, the avatar display unit 21h can change the appearance of the avatar so that the change is reflected in the display image of the avatar displayed in the video. In addition to changing the features, the avatar display unit 21h can reflect changes to the avatar during video distribution in the avatar in the video in real time or quasi-real time.

[0110] FIG. 7d shows another example of a video in which the avatar 56a of the first viewing user is placed in the first region 52a in the virtual space 50. The display image 60d shown in FIG. 7d is drawn with the virtual camera 53b as a viewpoint. The virtual camera 53b is placed in front of the character object 51, at a position farther from the character object 51 than the virtual camera 53a, so as to face the direction of the character object 51. Therefore, the display image 60d drawn with the virtual camera 53b as a viewpoint includes a wider range of the virtual space 50 than the display images 60a and 60b. Specifically, the display image 60d includes the first region 52a and the second region 52b. Therefore, the display image 60d also displays the avatars 56a to 56c placed in the first region 52a and the avatars 57a to 57g placed in the second region 52b.

[0111] An emotion button 68 may be displayed on the display image. The emotion button 68 is a button for the viewing user to express the emotion when viewing the video. The viewing user can select the emotion button 68 to indicate that the viewing user was moved by the video, that the viewing user could not empathize with the video, or other emotions of the viewing user. The emotion button 68 may include a clap button for selecting the emotion of being moved by the video, a question mark button for indicating that the viewing user could not empathize with the video, a surprise button for indicating that the viewing user was surprised by the video, and other buttons. Each of the emotion buttons 68 may include a symbol or graphic display expressing the emotion selected by the emotion button 68. For example, the clap button may include a graphic display showing clapping hands, the question mark button may include a question mark display, and the surprise button may include an exclamation mark display. The emotion button 68 may be selected, for example, by a tap operation on an area of ​​the touch panel of the client device 10a where the emotion button 68 is displayed. When the emotion button 68 is selected by a first viewing user who is viewing a video, an emotion display request is transmitted to the server device 20. The emotion display request may include emotion identification information indicating the type of emotion button 68 selected by the first viewing user (such as an applause button, a question mark button, or a surprise button). By selecting the emotion button 68, a plurality of buttons (such as an applause button, a surprise button, etc.) associated with the emotion button 68 may be displayed in a pop-up. When a plurality of buttons associated with the emotion button 68 are displayed in a pop-up, the user can select a button that matches his / her own emotion from the plurality of buttons displayed in the pop-up. The emotion button 68 is displayed when the user's avatar is participating in a video, and may not be displayed when the user's avatar is not participating in a video.For example, in the illustrated embodiment, when display image 60a is displayed, the avatar of the first viewing user is not participating in the video, so the emotion button 68 is not displayed on display image 60a, whereas the emotion button 68 is displayed on display image 60b in which the avatar of the first viewing user is participating in the video and on the other display images.

[0112] The avatar control unit 21i may receive an emotion display request from a first viewing user and control the behavior of the avatar of the first viewing user based on the emotion display request. For example, when an emotion display request is transmitted by the first viewing user selecting a clap button, which is a type of emotion button, the avatar control unit 21i may cause the avatar 56a of the first viewing user to perform an action of clapping based on the emotion display request. When the control of causing the avatar 56 to clap based on the emotion display request is performed, if the avatar 56a is included in the drawing area as shown in FIG. 7c or FIG. 7d, an animation of the avatar 56a clapping may be displayed in the video. Conversely, if the avatar 56a is not included in the drawing area as shown in FIG. 7b, the sound of the avatar 56a clapping may be displayed in the video, but the animation of clapping may not be displayed. In one embodiment, the emotion button may be displayed on the display screen of the video being viewed by the user only when the user's avatar is displayed in the video. For example, when the image 60b shown in Fig. 7b is displayed on the client device 10a of the first viewing user, the avatar 56a of the first viewing user is not displayed in the image 60b, and therefore the image 60b may not include the emotion button 68. Conversely, when the image 60c shown in Fig. 7c is displayed on the client device 10a of the first viewing user, the avatar 56a of the first viewing user is displayed in the image 60c, and therefore the emotion button 68 may be included in the image 60c.

[0113] In this way, based on an emotion display request from a user, the user's avatar performs a movement such as clapping in the video. The emotion display request is a type of action instruction that instructs the avatar to move. In another embodiment, in addition to the emotion buttons, an action instruction button that instructs the avatar to move may be included in the displayed image. The action instruction button is a button for instructing the avatar to move. The action instruction button can be used to instruct, for example, an action such as jumping, waving a hand, waving a glow stick, and other actions. When the action instruction button is selected, a request for action instruction is transmitted to the server device 20. The server device 20 can cause the avatar to move based on the action instruction request.

[0114] As shown in FIG. 7b to FIG. 7d, when the display images 60b to 60d are displayed, the avatar 56a of the first viewing user has already been placed in the virtual space 50, so the display image 60b does not include the avatar participation button 63, but includes a display request button 64 instead. The display request button 64 is a button for requesting that the avatar of the viewing user be displayed together with the character object 51. The display request button 64 is displayed on the display screen 60b so as to be selectable by the first viewing user using the client device 10a. The display request button 64 can be selected, for example, by a tap operation on an area on the touch panel of the client device 10a where the display request button 64 is displayed. When the display request button 64 is selected by the first viewing user who is viewing a video, a display request requesting that the avatar 56a of the first viewing user be displayed together with the character object 51 is transmitted to the server device 20.

[0115] The display request processing unit 21j receives a display request from the first viewing user, and displays the avatar 56a of the first viewing user together with the character object 51 in the video based on the display request. A request for displaying an avatar together with a character object (e.g., the character object 51) included in the video being distributed is called a "feature display request", and displaying an avatar together with the character object 51 in response to a feature display request may be called "feature display". As shown in FIG. 7e, when a user makes a feature display request, a notification 66 indicating that a feature display request has been made may be displayed in a display image 60e. This notification 66 may be displayed only in an image displayed on the client device of the user who made the feature display request, or may be displayed on an image displayed on the client device of the user as well as an image displayed on a client device of another user. An example of a display image in which the avatar 56a of the first viewing user and the character object 51 are displayed together is shown in FIG. 7f. In the display image 60f shown in FIG. 7f, the avatar 56a of the first viewing user is displayed next to the character object 51. The display request processing unit 21j may display the avatar 56a on the floor object 54a based on a display request from the first viewing user. Since the character object 51 is placed on the floor object 54a, by placing the avatar 56a on the floor object 54a, the avatar 56a can be displayed together with the character object 51. A notification 67 for notifying that the feature display has been implemented may be displayed on the display image 60f.

[0116] When the avatar 56a is displayed together with the character object 51 based on the feature display request, the avatar 56a may be displayed in 3D. As described above, the avatar 56a may be displayed in 2D in a display image (e.g., the display image 60c or the display image 60d) before the feature display request is made. The display mode of the avatar 56a that was displayed in 2D before the feature display request may be changed to 3D in the feature display. Since the avatar 56a is displayed in 3D in the feature display, the display request button 64 may be displayed in the display image 60b displayed on the client device 10a of the first viewing user only when 3D display information for the avatar 56a has been loaded before the feature display request is received.

[0117] In one embodiment, the display request button 64 is displayed in a display image for a user who uses an avatar for which 3D display information is loaded, and the display request button 64 is not displayed in a display image for a user who uses an avatar for which 3D display information is not loaded, thereby preventing runtime loading of the 3D display information and reducing the processing load of the server device 20. In one embodiment, in order to avoid the processing load required for runtime loading, when a feature display of an avatar is received from a user, it may be determined whether or not the 3D display information of the avatar has been loaded, and the avatar may be displayed in 3D only if the avatar has been loaded. In this case, the display request button 64 is displayed in a display image for a user who uses an avatar for which 3D display information is loaded, while the display request button 64 is not displayed in a display image for a user who uses an avatar for which 3D display information is not loaded, thereby preventing unnecessary transmission of a feature display request and unnecessary processing based on the feature display request.

[0118] When an avatar is featured in a video, a special effect related to the featured avatar may be displayed. For example, a special effect may be displayed as if the avatar is being lit by a spotlight so as to emphasize the featured avatar. The same or a different special effect may be displayed when the featured avatar is terminated.

[0119] The display request processing unit 21j may measure the elapsed time since the avatar 56a of the first viewing user is displayed together with the character object 51 in the video. The display request processing unit 21j may display the elapsed time in the display image 60f. In one embodiment, an upper limit may be set for the time during which the character object 51 and the user's avatar are displayed together (the time during which a feature display is performed for a certain avatar) based on the feature display request. The display request processing unit 21j may display the upper limit time in the display image 60f. The display request processing unit 21j may calculate the remaining time during which the avatar 56a can be displayed based on the upper limit time and the elapsed time since the avatar 56a of the first viewing user is displayed together with the character object 51 in the video, and display the remaining time in the display image 60f. The time that has elapsed since the avatar 56a of the first viewing user was displayed in the video together with the character object 51, the upper limit of the display time set for the avatar 56a, and the remaining time for which the avatar 56a can be displayed may be included only in the display image 60f displayed on the client device 10a of the first viewing user, or may be included in the display image 60f displayed on the client devices of the first viewing user and the other users. By setting an upper limit of the time for which the feature display is performed, it is possible to provide more viewing users with an opportunity to display their own avatar together with the character object 51 by the feature display. In addition, by displaying the remaining time of the feature display being performed, it is possible to inform the viewing user of the time until the next avatar is featured.

[0120] The display request processing unit 21j can exclude an avatar participating in a video from the video, based on an instruction from the distribution user of the video or another operator. The excluded avatar will no longer be displayed on the display screen of the video. Similarly, the display request processing unit 21j can exclude an avatar that is displayed as a featured display from the featured display, based on an instruction from the distribution user of the video or another operator. The excluded avatar will no longer be displayed on the screen of the featured display. The distribution user can exclude, for example, an avatar that has made an undesirable comment in a video, an avatar that has behaved undesirably in a video, or an avatar that is undesirable to be allowed to participate in other videos from the video and / or from the display screen of the featured display.

[0121] In one embodiment, when a user's avatar is featured, the avatar control unit 21i can control the movement of the avatar based on an instruction from the user. For example, the avatar control unit 21i can receive an emotion display request from the user of the featured avatar, and control the behavior of the avatar based on the emotion display request. The behavior of the avatar may be controlled using face motion or body motion transmitted from a user's terminal.

[0122] In an embodiment, when a feature display request from a first viewing user is permitted by the broadcasting user, the display request processing unit 21j may display the avatar 56a of the first viewing user together with the character object 51. In response to receiving a feature display request from the first viewing user, the display request processing unit 21j may inquire of the broadcasting user whether or not to permit the avatar 56a of the first viewing user to be displayed together with the character object 51. When receiving a feature display request from the first viewing user, the display request processing unit 21j may generate a display request user list including the first viewing user, and present the display request user list to the broadcasting user. The display request user list for a certain video indicates information (for example, a user ID, a user name, a nickname, an avatar ID, or other information) that identifies a user (or an avatar of the user) who has made a feature display request in the form of a list. The display request user list does not need to include all users who have sent feature display requests. For example, the display request user list may include only users who have transmitted feature display requests and for whom 3D display information of the avatars of those users has already been loaded. Such a display request user list does not include information identifying users who have transmitted feature display requests and for whom 3D display information of the avatars of those users has not been loaded. In order to perform 3D display of an avatar for which 3D display information has not been loaded, it is necessary to load the 3D display information at run time, which consumes a lot of processing resources of the server device 20. By including only users who use avatars for which 3D display information has already been loaded in the feature display request, run time loading can be avoided.

[0123] The display request processing unit 21j can transmit the display request user list to the supporter computer 40. The supporter B1 and the supporter B2 can select users of avatars to be permitted to be displayed together with the character object from the display request user list displayed on the display of the supporter computer 40. The display request user list may be displayed on the display 39 of the studio room R. The actor A1 may select users of avatars to be displayed together with the character object 51 from the display request user list by operating the controller 33a or the controller 33b. A user selected from the display request user list is permitted to display the avatar of that user together with the character object 51, and the avatar is featured in the video.

[0124] In one embodiment, when a certain user is selected from the display request user list, the certain user may be deleted from the display request user list. In another embodiment, when the featured display of a certain user's avatar ends, the certain user may be deleted from the display request user list. This makes it possible to prevent an avatar that has been featured once from being featured repeatedly.

[0125] In one embodiment, even if a supporter or broadcaster user instructs to select a user to be featured from the display request list while a certain user is being featured, the avatar of the selected user may not be featured immediately, and the selected user may not be deleted from the display request list until the user is featured.

[0126] Next, a video distribution process according to one aspect will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing the flow of the video distribution process in one embodiment. In the video distribution process, it is assumed that an actor A1 is present in a studio room R.

[0127] First, in step S11, body motion data, which is a digital representation of the body movement of actor A1, and face motion data, which is a digital representation of the facial movement (expression) of actor A1, are generated. The body motion data is generated, for example, by the body motion data generation unit 21a, and the face motion data is generated, for example, by the face motion data generation unit 21b.

[0128] Next, in step S12, the body motion data and face motion data of the actor A1 are applied to the model data for the actor A1 to generate animation of a character object that moves in synchronization with the body and facial movements of the actor A1. The generation of this animation is performed, for example, by the animation generation unit 21c described above.

[0129] Next, in step S13, a moving image including an image of the virtual space and an animation of a character object corresponding to the actor A1 is generated. The voice of the actor A1 may be synthesized into this moving image. The animation of the character object of the actor A1 may be placed in the virtual space. The generation of this moving image is performed, for example, by the above-mentioned moving image generating unit 21d.

[0130] Next, the process proceeds to step S14, where the video generated in step S13 is distributed. This video is distributed to client devices 10a to 10c and other client devices via network 15. This video is distributed to supporter computer 40, and may also be projected onto a screen S in studio room R. The video is distributed continuously for a predetermined distribution period. The video distribution period may be set to, for example, 30 seconds, 1 minute, 5 minutes, 10 minutes, 30 minutes, 60 minutes, 120 minutes, or any other time.

[0131] Next, the process proceeds to step S15, where it is determined whether or not a termination condition for terminating the distribution of the video has been met. The termination condition may be, for example, that the end time of the distribution has arrived, that an instruction to terminate the distribution has been given by the supporter computer 40, or other conditions. If the termination condition has not been met, the processes of steps S11 to S14 are repeatedly executed, and the distribution of the video including animation synchronized with the movement of the actor A1 is continued. If it is determined that the termination condition has been met for the video, the distribution process of the video is terminated.

[0132] Next, a participation request process according to one aspect will be described with reference to Fig. 9. Fig. 9 is a flow diagram showing the flow of the participation request process in one embodiment. In the participation request process of Fig. 9, it is assumed that a first viewing user who is viewing a video using the client device 10a has requested that the avatar 56a of the first viewing user participate in the video.

[0133] First, in step S21, a participation request from a first viewing user is accepted. The participation request is transmitted from the client device 10a to the server device 20 in response to the first viewing user selecting an avatar participation button 63 displayed on the client device 10a.

[0134] Next, in step S22, the group to which the user who transmitted the participation request belongs is identified. If the user belongs to the first group or the second group, it is determined whether the first viewing user who transmitted the participation request belongs to the first group or the second group. This determination is performed, for example, by the above-mentioned participation request processing unit 21g.

[0135] If it is determined that the first viewing user belongs to the first group, the participation request process proceeds to step S23. In step S23, the avatar 56a of the first viewing user is placed in the first area 52a of the virtual space 50. As a result, when the first area 52a is included in the rendering area of ​​the virtual camera, the avatar 56a of the first viewing user is displayed at a position in the video that corresponds to the first area 52a. The placement of the avatar 56a in the first area 52a is performed, for example, by the participation request processing unit 21g described above.

[0136] If it is determined that the first viewing user belongs to the second group, the participation request process proceeds to step S24. In step S24, it is determined whether or not the participation condition is satisfied. As described above, the participation condition is that the number of avatars arranged in the second area 52b is less than the second upper limit number set for the second area 52b, that the first viewing user is selected by the broadcasting user, or other conditions. The determination of whether or not the participation condition is satisfied is performed, for example, by the above-mentioned participation request processing unit 21g.

[0137] If it is determined that the participation conditions are met, the participation request process proceeds to step S25. In step S25, the avatar 56a of the first viewing user is placed in the second area 52b of the virtual space 50. As a result, when the second area 52b is included in the rendering area of ​​the virtual camera, the avatar 56a of the first viewing user is displayed at a position in the video that corresponds to the second area 52b. The placement of the avatar 56a in the second area 52b is performed, for example, by the participation request processing unit 21g described above.

[0138] If it is determined that the participation condition is not satisfied, the participation request process is terminated without displaying the avatar 56a of the first viewing user. In another embodiment, if it is determined that the participation condition is not satisfied, it may be repeatedly determined whether the participation condition is satisfied.

[0139] Next, a display request process according to one aspect will be described with reference to Fig. 10. The display request process is a process based on a display request from a user of an avatar placed in a virtual space, which requests that the avatar be displayed together with a character object 51. Fig. 10 is a flow diagram showing the flow of the display request process in one embodiment. In the display request process of Fig. 10, it is assumed that a display request has been made from a first viewing user who is viewing a video using the client device 10a, requesting that the avatar 56a of the first viewing user be displayed together with a character object 51.

[0140] First, in step S31, a display request is received from a first viewing user. The display request is transmitted from the client device 10a to the server device 20 in response to the first viewing user selecting a display request button 64 displayed on the client device 10a.

[0141] Next, in step S32, it is determined whether the display request from the first viewing user is permitted by the broadcast user. If it is determined that the display request is permitted, the display request process proceeds to step S33. This determination is made, for example, by the above-mentioned display request processing unit 21j.

[0142] In step S33, a process is performed to display the avatar 56a of the first viewing user together with the character object 51. For example, the avatar 56a is placed on the floor object 54a on which the character object 51 is placed. The placement of the avatar 56a on the floor object 54a is performed, for example, by the above-mentioned display request processing unit 21j.

[0143] If it is determined that permission has not been obtained from the broadcasting user, the display request process ends without displaying both the first viewing user's avatar 56a and the character object 51. In other embodiments, the determination in step S32 may be repeated until permission is obtained from the broadcasting user.

[0144] The effects achieved by the above embodiment will be described. In the above embodiment, in response to receiving a request from a first viewing user who watches a video distributed from the server device 20 to participate in the video, it is determined whether the first viewing user belongs to the first group or the second group. In one embodiment, when the engagement of the first viewing user with the video distribution system 1 or the distribution user who distributes the video being viewed is high, the first viewing user is considered to belong to the first group. When it is determined that the first viewing user belongs to the first group, the avatar 56a of the first viewing user is placed in the first area 52a in the virtual space 50. In one embodiment, when it is determined that the first viewing user belongs to the first group, the avatar 56a is placed in the first area 52a without requiring any additional conditions. This does not prevent the avatar of a user who belongs to the first group with high engagement from participating in the video. In one embodiment, the number of users who belong to the first group can be suppressed to an appropriate number according to the video to be distributed through setting a criterion for belonging to the first group. This makes it possible to prevent a large number of avatars from being displayed in the distributed video, thereby reducing the processing load on the server device 20.

[0145] In one embodiment, when it is determined that the first viewing user belongs to the second group, it is further determined whether or not the participation condition is satisfied, and in response to the determination that the participation condition is satisfied, the avatar 56a of the first viewing user is placed in the second area 52b in the virtual space 50. Thus, by setting the participation condition, the number of avatars displayed in the second area 52b can be kept to an appropriate number, thereby reducing the processing load on the server device 20.

[0146] In one embodiment, the first region 52a is disposed closer to the character object 51 (or closer to the third region in which the character object is disposed) than the second region 52b in the virtual space 50. This allows a viewing user with high engagement to be disposed on the character object 51. In one embodiment, the virtual cameras 53a to 53e are disposed in the virtual space 50 so as to face the position where the character object 51 is disposed. Therefore, an avatar disposed in the first region 52a is more likely to be included in the rendering region of the virtual camera than an avatar disposed in the second region 52b. In this way, in one embodiment, an avatar of a first group of users with high engagement can be disposed in the first region 52a which is more likely to be included in the rendering region.

[0147] In one embodiment, the condition for participation is that the number of avatars displayed in the second area 52b is less than the upper limit of the number of avatars to be displayed set for the second area 52b. This makes it possible to suppress the number of avatars displayed in the second area 52b to be equal to or less than the upper limit, thereby reducing the processing load on the server device 20.

[0148] In one embodiment, the condition for participation is that the viewing user is selected by the broadcasting user. This allows the broadcasting user to have an avatar of a viewing user that is favorable to the broadcasting user participate in the video. This makes it easier to maintain the health of the community formed through the broadcasted video.

[0149] In one embodiment, an instruction to operate an avatar placed in the virtual space 50 is received, and the movement of the avatar is controlled based on the instruction to operate. This allows the viewing user to interact with characters included in the video and other viewing users participating in the video through the instruction to operate the avatar.

[0150] In one embodiment, a display request is received to display the viewing user's avatar together with the character object 51 while the avatar is located in the virtual space 50, and in response to the display request being permitted, the avatar is displayed in the third area in which the character object 51 is located. This provides the viewing user with a means for more closely communicating with the broadcasting user's character.

[0151] The embodiment of the present invention is not limited to the above-mentioned embodiment, and various modifications are possible within the scope of the gist of the present invention. For example, the video to be distributed may be shot and generated outside the studio room R. For example, the shooting for generating the video to be distributed may be performed at the home of the actor or supporter. The client device 10a may have a function of transmitting an image in which the avatar 56a and the character object 51 are displayed together (for example, the display image 60f) to another communication device. For example, the first viewing user can use the function of the client device 10a to distribute an image in which the avatar 56a and the character object 51 are displayed together in a social networking service within the video service provided by the server device 20 or in a social networking service different from the video service provided by the server device 20.

[0152] In the processing procedures described in this specification, particularly in the processing procedures described using flow diagrams, it is possible to omit some of the steps that make up the processing procedures, to add steps that are not explicitly stated as steps that make up the processing procedures, and / or to change the order of the steps, and processing procedures in which such omissions, additions, or changes in order have been made are also included within the scope of the present invention as long as they do not deviate from the spirit of the present invention. The claims as originally filed in the original application of this application are set forth below. [1] A system for enabling a first video, which includes a virtual space and an animation of a character object generated based on a movement of a distribution user, to be live-viewed by each of a client device of a plurality of viewing users, comprising: one or more computer processors; The one or more computer processors execute the computer readable instructions to: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, a first avatar of the first user is placed in a first area within the virtual space, thereby allowing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; system. [2] In the second video, the first avatar is disposed in a character display area in which the character object is disposed. [1] The system described in. [3] the display request is generated by a first client device of the first user in response to a display request button displayed on the first client device being selected by the first client device. The system according to [1] or [2]. [4] the display request button is displayed on the first client device in response to the first avatar participating in the first video. [3] The system described in. [5] In the second moving image, the first avatar is disposed in a character display area in which the character object is disposed; the display request button is displayed on the first client device only when 3D display information for displaying the first avatar in 3D is stored in a storage; the first avatar is displayed in 3D in the character display area in the second video; [4] The system described in. [6] the first avatar is displayed in 2D in the first region; [5] The system described in. [7] measuring an elapsed time from when the first avatar is displayed together with the character object in the second video; calculating a remaining time for which the first avatar is to be displayed together with the character object based on the elapsed time; displaying the remaining time together with the second video on the client device; 1. A system according to any one of claims 1 to 6. [8] removing the first avatar from the second video based on an instruction from the broadcast user; 1. A system according to any one of claims 1 to 7. [9] controlling a movement of the first avatar based on an action instruction generated in response to a selection of an action instruction button displayed on the first client device; A system according to any one of claims 3 to 5.

[10] the action instruction button is displayed on the first client device only when the first avatar is participating in the first video. [9] The system described in.

[11] The first avatar is prohibited from participating in any other video other than the first video while participating in the first video. 10. The system according to claim 1, wherein

[12] For each of the plurality of viewing users, a point is set that increases or decreases according to the use of the system by each viewing user; A high-point user having a point higher than a threshold among the plurality of users is assigned to the first group. 1. A system according to any one of claims 1 to 11.

[13] The first user's points are increased by the first user viewing a video on the system.

[12] The system described in.

[14] The first user's points are increased by the first user distributing a video on the system. The system according to

[12] or

[13] .

[15] An upper limit is set for viewing users who can belong to the first group. 1. The system according to claim 1, wherein the system comprises:

[16] The viewing users among the plurality of viewing users are assigned to the first group in descending order of points until the upper limit is reached.

[15] The system described in.

[17] The system is configured to enable live viewing of a second video including animation of a character object generated based on a movement of another distribution user by each of the client devices of the plurality of viewing users; The points of each viewing user are managed individually for the broadcast user and the other broadcast users. The system according to any one of claims 12 to 16.

[18] A method for enabling a first video, including a virtual space and an animation of a character object generated based on a movement of a distributor user, to be live-viewed by each of a client device of a plurality of viewing users, by executing computer-readable instructions by one or more computer processors, the method comprising: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space to allow the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A method for providing the above.

[19] A program that enables a first video, including a virtual space and an animation of a character object generated based on a movement of a distribution user, to be live-viewed by each of a client device of a plurality of viewing users, one or more computer processors, In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space to allow the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A program that executes the following. [Explanation of symbols]

[0153] 1. Video distribution system

Claims

1. A system for enabling a first video including an animation of a character object arranged in a virtual space to be viewed live by each of a plurality of viewing users on a client device, the system comprising: one or more computer processors; The one or more computer processors execute the computer readable instructions to: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, a first avatar of the first user is placed in a first area within the virtual space, thereby causing the first user to participate in the first video; generating a second moving image in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; system.

2. In the second moving image, the first avatar is disposed in a character display area in which the character object is disposed. The system of claim 1 .

3. the display request is generated by a first client device of the first user in response to a display request button displayed on the first client device being selected by the first client device.

3. A system according to claim 1 or 2.

4. the display request button is displayed on the first client device in response to the first avatar participating in the first video. The system of claim 3.

5. In the second moving image, the first avatar is disposed in a character display area in which the character object is disposed; the display request button is displayed on the first client device only when 3D display information for displaying the first avatar in 3D is stored in a storage; the first avatar is displayed in 3D in the character display area in the second video; The system of claim 4.

6. the first avatar is displayed in 2D in the first region; The system of claim 5.

7. measuring an elapsed time from when the first avatar is displayed together with the character object in the second video; calculating a remaining time for which the first avatar is to be displayed together with the character object based on the elapsed time; displaying the remaining time together with the second video on the client device; A system according to any one of claims 1 to 6.

8. excluding the first avatar from the second video based on an instruction from the broadcast user; A system according to any one of claims 1 to 7.

9. controlling a movement of the first avatar based on an action instruction generated in response to a selection of an action instruction button displayed on the first client device; A system according to any one of claims 3 to 5.

10. the operation instruction button is displayed on the first client device only when the first avatar is participating in the first video; The system of claim 9.

11. The first avatar is prohibited from participating in another video different from the first video while participating in the first video. A system according to any one of claims 1 to 10.

12. For each of the plurality of viewing users, a point is set that increases or decreases according to the use of the system by each viewing user; A high-point user having a point higher than a threshold among the plurality of users is assigned to the first group. A system according to any one of claims 1 to 11.

13. The first user's points are increased by the first user viewing a video on the system. The system of claim 12.

14. The first user's points are increased by the first user distributing a video on the system.

14. A system according to claim 12 or 13.

15. An upper limit is set for viewing users who can belong to the first group. A system according to any one of claims 1 to 14.

16. The viewing users among the plurality of viewing users are made to belong to the first group in descending order of points until the upper limit number is reached. The system of claim 15.

17. the system is configured to enable live viewing of a second video, the second video including animation of a character object generated based on a movement of another distribution user, by a client device of each of the plurality of viewing users; The points of each viewing user are managed individually for the broadcast user and the other broadcast users.

17. A system according to any one of claims 12 to 16.

18. A method for enabling a first video, including an animation of a character object disposed in a virtual space, to be live-viewed by each of a client device of a plurality of viewing users by executing computer-readable instructions by one or more computer processors, the method comprising: In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space to allow the first user to participate in the first video; generating a second video in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A method for providing the above.

19. A program for enabling a first video including an animation of a character object arranged in a virtual space to be viewed live by each of a plurality of viewing users on a client device, the program comprising: one or more computer processors, In response to receiving a request to participate in the first video from a first user belonging to a first group among the plurality of viewing users, placing a first avatar of the first user in a first area within the virtual space to allow the first user to participate in the first video; generating a second video in which the first avatar is displayed together with the character object in response to a display request received from the first user while the first avatar is placed in the first area; A program that executes the following.

Images