Video distribution system, video distribution method, and video distribution program

Abstract

To provide a moving image distribution system, a moving image distribution method, and a moving image distribution program that can distribute a coacting moving image with an improved sense of unity.SOLUTION: A moving image distribution system including one or more computer processors distributes a coacting moving image including a first character generated on the basis of the movement of a first user and a second character generated on the basis of the movement of a second user, displays a first object in association with the first character, determines whether a matching object identical to the first object is displayed in association with the second character, and displays the same matching object as the first object in association with the second character in the coacting moving image when it is determined that the matching object is not displayed in association with the second character.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The disclosure herein relates to a video distribution system, a video distribution method, and a video distribution program. More specifically, the disclosure herein relates to a video distribution system, a video distribution method, and a video distribution program that live-distribute videos including characters generated based on user movements. [Background technology]

[0002] Various video distribution services have been known for some time. In recent years, a video distribution system that generates character animation based on the movement of a distribution user and live-distributes a video including the character animation has been attracting attention. Such a video distribution system is disclosed, for example, in JP2015-184689A (Patent Document 1) and "Yusuke Hasegawa, The Dawn of a New Era of Virtual Idols PROJECT MariA, CG WORLD, Japan, Born Digital Co., Ltd., November 10, 2017, Vol. 231, pp. 74-79" (Non-Patent Document 1).

[0003] "Virtual Cast" provided by Dwango Co., Ltd. is known as a video distribution service that distributes videos including characters generated based on the movements of a broadcasting user. In Virtual Cast, a character of a broadcasting user can participate in a video distributed by another broadcasting user, and the two broadcasting users can virtually appear together in the video through their respective characters.

[0004] There is also known a video distribution system that displays a gift object corresponding to a gift purchased by a viewing user who is viewing a video on a display screen in response to a request from the viewing user. For example, in the video distribution system disclosed in JP 2012-120098 A (Patent Document 2), the viewing user can purchase a gift item and provide the purchased gift item to the broadcasting user as a gift. The gift is displayed in the video being broadcast in response to a display request from the viewing user. The gift display request may be automatically generated in response to the purchase of the gift by the viewing user. The specification of Japanese Patent No. 6543403 (Patent Document 3) describes that a gifted object is displayed in association with a character that moves based on the movement of the broadcasting user. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-184689 A [Patent Document 2] JP 2012-120098 A [Patent Document 3] Patent No. 6543403 [Non-patent literature]

[0006] [Non-Patent Document 1] Yusuke Hasegawa, The Dawn of a New Era of Virtual Idols PROJECT MariA, CG WORLD, Japan, Born Digital Co., Ltd., November 10, 2017, Vol. 231, pp. 74-79 [Non-Patent Document 2] Dwango Co., Ltd., "[Virtual Cast] About Tokoro", [online], [Retrieved July 1, 2019], Internet〈URL:https: / / qa.nicovideo.jp / faq / show / 10740?category#id=718&site#domain=default〉 Summary of the Invention [Problem to be solved by the invention]

[0007] When a host user's character and a guest user's character appear together in a co-starring video, the appearance of the host user's character is determined by the host user's settings, and the appearance of the guest user's character is determined by the guest user's settings. In other words, the images of the characters included in the co-starring video are determined independently by each of the co-stars. For this reason, there is a problem that it is difficult to express a sense of unity in conventional co-starring videos.

[0008] The object of the present invention is to provide a technical improvement that solves or alleviates at least a part of the problems of the prior art described above. A more specific object of the present invention is to provide a video distribution system, a video distribution method, and a video distribution program that can distribute co-starring videos with improved uniformity. [Means for solving the problem]

[0009] One aspect of the present invention relates to a video distribution system including one or more computer processors, which execute computer-readable instructions to distribute a co-starring video including a first character generated based on a movement of a first user and a second character generated based on a movement of a second user, display a first object in association with the first character, determine whether a matching object identical to the first object is displayed in association with the second character, and, when it is determined that the matching object is not displayed in association with the second character, display a matching object identical to the first object in association with the second character in the co-starring video.

[0010] In one aspect of the present invention, the matching object displayed in association with the second character is displayed only in the co-starring video.

[0011] A video distribution system according to one embodiment of the present invention, when it is determined that the matching object is not displayed in association with the second character, prompts a display request to display the matching object in association with the second character, and displays the matching object in association with the second character in the co-starring video based on the display request.

[0012] A video distribution system according to one embodiment of the present invention, when it is determined that the matching object is not displayed in association with the second character, prompts one or more viewing users who are watching the co-starring video to make a display request to display the matching object in association with the second character.

[0013] A video distribution system according to one embodiment of the present invention, when it is determined that the matching object is not displayed in association with the second character, prompts the first user to make a display request to display the matching object in association with the second character.

[0014] A video distribution system according to one embodiment of the present invention, when it is determined that the matching object is not displayed in association with the second character, prompts the second user to make a display request to display the matching object in association with the second character.

[0015] A video distribution system according to an aspect of the present invention provides a benefit to the first user when the matching object is displayed in association with the second character in the co-starring video. A video distribution system according to an aspect of the present invention provides a benefit to the second user when the matching object is displayed in association with the second character in the co-starring video. A video distribution system according to an aspect of the present invention provides a benefit to the first viewing user when the matching object is displayed in response to a display request for displaying the matching object in association with the second character from a first viewing user among one or more viewing users watching the co-starring video. A video distribution system according to an aspect of the present invention provides a benefit to each of the first user, the second user, and one or more viewing users watching the co-starring video when the matching object is displayed in association with the second character in the co-starring video.

[0016] In one aspect of the present invention, the co-starring video is generated in response to receiving a co-starring application from the second user requesting to co-star with the first user.

[0017] In one aspect of the present invention, the first object is displayed in association with an attachment portion of the first character, and the matching object is displayed in association with the attachment portion of the second character.

[0018] One aspect of the present invention relates to a video distribution method executed by one or more computer processors executing computer-readable instructions, the video distribution method including the steps of: distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character, determining whether a matching object identical to the first object is displayed in association with the second character, and displaying a matching object identical to the first object in association with the second character in the co-starring video when it is determined that the matching object is not displayed in association with the second character.

[0019] One aspect of the present invention relates to a video distribution program, which causes one or more computer processors to execute the steps of: distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character, determining whether a matching object identical to the first object is displayed in association with the second character, and, when it is determined that the matching object is not displayed in association with the second character, displaying a matching object identical to the first object in association with the second character in the co-starring video. Effect of the Invention

[0020] According to an embodiment of the present invention, it is possible to provide a video distribution system, a video distribution method, and a video distribution program that are capable of distributing co-starring videos with improved uniformity. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing a video distribution system according to an embodiment. [Diagram 2]2 is a schematic diagram showing a broadcaster user who broadcasts a video distributed in the video distribution system of FIG. 1 and a broadcaster user device used by the broadcaster user. FIG. [Diagram 3] 2 is a diagram illustrating gift object information stored in the video distribution system of FIG. 1. FIG. [Figure 4] 2 is a diagram illustrating owned object information stored in the video distribution system of FIG. 1. FIG. [Diagram 5] 2 is a diagram illustrating object display information stored in the video distribution system of FIG. 1. [Figure 6] 1A-1C are diagrams illustrating example images of a video displayed on an audience user device 10 in one embodiment. [Figure 7a] FIG. 13 is a diagram showing an example of a video image displayed on a broadcast user device 20a in one embodiment. [Figure 7b] FIG. 13 shows an example of a video image displayed on a broadcast user device 20b in one embodiment. [Figure 8] 8 is a diagram showing an example of an image of a moving image displayed on an audience user device 10 in one embodiment. [Figure 9] 9 is a diagram showing an example of an image of a moving image displayed on a viewer user device 10 in one embodiment, and a wearable object is displayed in the image of FIG. [Figure 10a] FIG. 13 is a diagram showing an example of a video image displayed on a broadcast user device 20a in one embodiment. [Figure 10b] FIG. 13 shows an example of a video image displayed on a broadcast user device 20b in one embodiment. [Figure 11] FIG. 2 is a diagram showing an example of an image of a co-starring video displayed on a viewing user device 10 in one embodiment. [Figure 12a] FIG. 13 is a diagram showing an example of an image of a co-starring video displayed on a broadcast user device 20a in one embodiment. [Figure 12b] FIG. 13 is a diagram showing an example of an image of a co-starring video displayed on a broadcast user device 20b in one embodiment. [Figure 13] FIG. 2 is a diagram showing an example of an image of a co-starring video displayed on a viewing user device 10 in one embodiment. [Figure 14] FIG. 11 is a flow diagram showing a flow of a video distribution process in one embodiment. [Figure 15] 1 is a flow diagram showing a process flow for distributing a co-starring video in one embodiment. [Figure 16] FIG. 11 is a flow diagram showing a process for displaying matching objects in an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Various embodiments of the present invention will now be described with reference to the accompanying drawings, in which the same reference numerals are used to designate the same or similar components throughout the drawings.

[0023] A video distribution system according to an embodiment will be described with reference to Figures 1 and 2. Figure 1 is a block diagram showing a video distribution system 1 according to an embodiment, and Figure 2 is a schematic diagram showing users Ua and Ub who distribute videos in the video distribution system 1 and distributor user devices 20a and 20b used by the users Ua and Ub. The distributor user device 20a has a function for distributing a video including a character object generated based on the movement of the user Ua in the video distribution system 1, and the distributor user device 20b has a function for distributing a video including a character object generated based on the movement of the user Ub in the video distribution system 1.

[0024] As will be described in detail later, the video distribution system 1 can generate a video including character objects of two or more distribution users and distribute the video. In other words, in the video distribution system 1, two or more distribution users can virtually appear together in the video through their respective character objects. In this way, when two or more distribution users virtually appear together in a video through their respective character objects, the video is called a "co-starring video." In this specification, the character object of the distribution user included in the video may be simply called a "character."

[0025] Both user Ua and user Ub are broadcast users who broadcast videos including characters generated based on movements, and also viewing users who watch videos of other broadcast users. In this specification, a user who watches videos using the broadcast user device 10, the broadcast user devices 20a and 20b, or other devices is called a viewing user. For this reason, the broadcast user devices 20a and 20b may have a function of watching videos broadcast in the video broadcasting system 1.

[0026] User Ua can make a co-starring application to apply to co-star with user Ub. When this co-starring application is permitted, a co-starring video including the characters of user Ub and user Ua is generated, and the co-starring video is distributed. When user Ua participates in a video distributed by user Ub and user Ub and user Ua co-star, user Ua who made the co-starring application and co-starred with user Ub is called a "guest user", and user Ub who distributes a video in which the guest user participates is sometimes called a "host user". The concepts of host user and guest user are relative. For example, conversely to the above example, when user Ub sends a co-starring application to user Ua and user Ub co-stars in user Ua's video based on this co-starring application, user Ua becomes the host user and user Ub becomes the guest user. In this specification, for convenience of explanation, it is assumed that an invitation is sent from user Ub to user Ua. In other words, it is assumed that user Ub is the host user and user Ua is the guest user. However, the explanation of user Ua in this specification can be extended to the explanation of guest users in general as much as possible. Similarly, the description of user Ub in this specification may be expanded to include descriptions of host users in general as much as possible. For example, even if a description is given of broadcast user Ua in this specification, the description may be understood as a description of guest users in general, and a description of broadcast user Ub may be understood as a description of host users in general.

[0027] The video distribution system 1 includes a viewing user device 10, broadcasting user devices 20a and 20b, a server device 60, and a storage 70. The viewing user device 10, broadcasting user device 20, server device 60, and storage 70 are connected to each other so as to be able to communicate with each other via a network 50. The server device 60 is configured to distribute videos including animations of characters of broadcasting user Ua and broadcasting user Ub, as described below.

[0028] This video is distributed from the server device 60 to the viewing user device 10 and the broadcasting user devices 20a, 20b. The distributed video is displayed on the display of the viewing user device 10. A viewing user, who is a user of the viewing user device 10, can view the distributed video using the viewing user device 10. Although only one viewing user device 10 is shown in FIG. 1 for the sake of simplicity, the video distribution system 1 may include multiple viewing user devices. By viewing the distributed video, the broadcasting users Ua, Ub can perform while checking the video within the video.

[0029] In the illustrated embodiment, the broadcast user device 20a includes a computer processor 21a, a communication I / F 22a, a display 23a, a camera 24a, and a microphone 25a. Similarly, the broadcast user device 20b includes a computer processor 21b, a communication I / F 22b, a display 23b, a camera 24b, and a microphone 25b. Since the broadcast user device 20a and the broadcast user device 20b have similar configurations and functions, when there is no need to particularly distinguish between them, they may be collectively referred to as the broadcast user device 20, and their components may also be referred to as the computer processor 21, etc.

[0030] The computer processor 21 is an arithmetic device that loads an operating system and various programs that realize various functions from a storage into a memory and executes instructions included in the loaded programs. The computer processor 21 is, for example, a CPU, an MPU, a DSP, a GPU, various other arithmetic devices, or a combination of these. The computer processor 21 may be realized by an integrated circuit such as an ASIC, a PLD, an FPGA, or an MCU. Although the computer processor 21 is illustrated in FIG. 1 as a single component, the computer processor 21 may be a collection of multiple physically separate computer processors.

[0031] The communication I / F 22 is implemented as hardware, firmware, or communication software such as a TCP / IP driver or a PPP driver, or a combination of these. The broadcast user device 20 can transmit and receive data to and from other devices via the communication I / F 22.

[0032] The display 23 has a display panel and a touch panel. The touch panel is configured to be able to detect a touch operation (contact operation) by the player. The touch panel can detect various touch operations by the player, such as tapping, double tapping, dragging, etc. The touch panel may be configured to include a capacitance-type proximity sensor and to be able to detect a non-contact operation by the player.

[0033] The camera 24 continuously captures images of the faces of the broadcast users Ua and Ub, and acquires imaging data of the faces of the broadcast users Ua and Ub. The imaging data of the faces of the broadcast users Ua and Ub captured by the camera 24 is transmitted to the server device 60 via the communication I / F 22. The camera 24 may be a 3D camera capable of detecting the depth of a person's face. The camera 24 may capture images of parts of the broadcast users Ua and Ub other than their faces, such as hands, legs, fingers, or other parts, to acquire imaging data.

[0034] The microphone 25 is a sound collection device configured to convert input voice into voice data. The microphone 25 is configured to acquire voice input of the broadcast users Ua and Ub. The voice input of the broadcast users Ua and Ub acquired by the microphone 25 is converted into voice data, and this voice data is transmitted to the server device 60 via the communication I / F 22.

[0035] The viewing user equipment 10 may include components similar to those of the broadcasting user equipment 20. For example, the viewing user equipment 10 may include a computer processor, a communication I / F, a display, and a camera. The viewing user equipment 10 may perform functions similar to those of the broadcasting user equipment 20 by downloading and installing certain application software.

[0036] The viewing user device 10 and the broadcasting user device 20 are information processing devices such as smartphones. The viewing user device 10 and the broadcasting user device 20 may be, in addition to smartphones, mobile phones, tablet terminals, personal computers, e-book readers, wearable computers, game consoles, and various other information processing devices capable of playing videos. In addition to the above components, each of the viewing user device 10 and the broadcasting user device 20 may be equipped with a sensor unit equipped with various sensors such as a gyro sensor, and a storage device for storing various information.

[0037] Next, a description will be given of the server device 60. In the illustrated embodiment, the server device 60 includes a computer processor 61, a communication I / F 62, and a storage 63.

[0038] The computer processor 61 is an arithmetic device that loads an operating system and various programs that realize various functions from the storage 63 or other storages into memory and executes instructions included in the loaded programs. The computer processor 61 is, for example, a CPU, an MPU, a DSP, a GPU, various other arithmetic devices, or a combination of these. The computer processor 61 may be realized by an integrated circuit such as an ASIC, a PLD, an FPGA, or an MCU. Although the computer processor 21 is illustrated in FIG. 1 as a single component, the computer processor 61 may be a collection of multiple physically separate computer processors.

[0039] The communication I / F 62 is implemented as hardware, firmware, or communication software such as a TCP / IP driver or a PPP driver, or a combination of these. The server device 60 can transmit and receive data to and from other devices via the communication I / F 62.

[0040] The storage 63 is a storage device accessed by the computer processor 61. The storage 63 is, for example, a magnetic disk, an optical disk, a semiconductor memory, or any other storage device capable of storing data. Various programs can be stored in the storage 63. At least a portion of the programs and various data that can be stored in the storage 63 may be stored in a storage (for example, the storage 70) that is physically separate from the server device 60.

[0041] In this specification, the programs or instructions included in the programs described as being executed by the computer processor 21 or the computer processor 61 may each be executed by a single computer processor or may be distributed and executed by multiple computer processors. Furthermore, the programs or instructions included in the programs executed by the computer processor 21 or the computer processor 61 may each be executed by multiple virtual computer processors.

[0042] Next, a description will be given of data stored in the storage 63. In the illustrated embodiment, the storage 63 stores model data 63a, object data 63b, gift object information 63c, owned object information 63d, object display information 63e, and various other data required for generating and distributing a distribution video.

[0043] The model data 63a is model data for generating animation of a character. The model data 63a may be three-dimensional model data for generating three-dimensional animation, or may be two-dimensional model data for generating two-dimensional model data. The model data 63a includes, for example, rig data (sometimes called "skeleton data") indicating the skeleton of the character's face and parts other than the face, and surface data indicating the shape and texture of the character's surface. The model data 63a may include a plurality of model data that are different from each other. The plurality of model data may have different rig data from each other, or may have the same rig data. The plurality of model data may have different surface data from each other, or may have the same surface data.

[0044] The object data 63b includes asset data for constructing a virtual space that constitutes a video. The object data 63b includes data for drawing the background of the virtual space that constitutes the video, data for drawing various objects displayed in the video, and data for drawing various objects other than these that are displayed in the video. The object data 63b may include object position information that indicates the position of the object in the virtual space.

[0045] The gift object information 63c is information for managing a gift object. An example of the gift object information 63c is shown in FIG. 3. As shown in the figure, in the gift object 63c, object information indicating the characteristics of the object is stored in association with object identification information for identifying the object. The object information of a certain object may include image data indicating an image of the object, type data indicating the type of the object, attachment position data indicating the attachment position on the character when the object is an attachment object described later, the price of the object, and various other data characterizing the object.

[0046] In one embodiment, the gift object is displayed on the video based on a gift display request from a viewing user who is viewing the video. There are various types of gift objects. The gift object may include, for example, an effect object corresponding to an effect gift, a normal object corresponding to a normal gift, an attachment object corresponding to an attachment gift, and a message object corresponding to a message gift. The viewing user can purchase a desired gift. The distinction between viewing users and broadcasting users (for example, broadcasting users Ua, Ub) is based on the actions of the users. For example, in the video broadcasting system 1, when a user watches a video, the user is called a viewing user, but when the user broadcasts a video, the user is called a broadcasting user.

[0047] An effect object representing an effect gift is an object that affects the overall impression of the viewing screen of a distributed video, for example, an object that resembles confetti. The object that resembles confetti may be displayed over the entire viewing screen, thereby changing the overall impression of the viewing screen before and after the display. An effect object may be displayed so as to overlap with a character, but is different from an attachment object in that it is displayed without being associated with a specific part of the character.

[0048] A normal object indicating a normal gift is an object indicating a gift from a viewing user to a broadcasting user (e.g., broadcasting users Ua and Ub), and is, for example, an object imitating a stuffed animal, a bouquet, an accessory, or an object suitable for a gift or present other than the above. In one aspect, a normal object is displayed on a display screen of a moving image so as not to contact the character. In one aspect, a normal object is displayed on a display screen of a moving image so as not to overlap with the character. A normal object may be displayed so as to overlap with an object other than a character in a virtual space. A normal object may be displayed so as to overlap with a character, but is different from an attached object in that a display associated with a specific part of a character is not performed. In one aspect, when a normal object is displayed so as to overlap with a character, the normal object is displayed so as to overlap with a part of the character other than the head including the face, and not to overlap with the head of the character. In one aspect, when a normal object is displayed so as to overlap with a character, the normal object is displayed so as to overlap with a part of the character other than the upper body including the face, and not to overlap with the upper body of the character.

[0049] The wearing object indicating the wearing gift is an object displayed on the display screen in association with a specific part (wearing part) of the character. In one embodiment, the wearing object displayed on the display screen in association with a specific part of the character is displayed on the display screen so as to be in contact with the specific part of the character. In one embodiment, the wearing object displayed on the display screen in association with a specific part of the character is displayed on the display screen so as to cover a part or all of the specific part of the character. The specific part may be specified by three-dimensional position information indicating a position in a three-dimensional coordinate space, or may be associated with position information in a three-dimensional coordinate space. For example, in the case of a character's head, the specific part may be defined in units of the front left side, front right side, rear left side, rear right side, center front side, center rear side, left eye, right eye, left ear, right ear, and the entire hair of the head.

[0050] The wearing object is, for example, an accessory (headband, necklace, earrings, etc.) worn by a character, clothing (T-shirt, etc.), costume, and other objects that can be worn by a character. The gift object information 63c corresponding to the wearing object may include wearing part information indicating which part of the character the wearing object is associated with. The wearing part information of a certain wearing object can indicate which part of the character the wearing object is worn on. For example, if the wearing object is a headband, the wearing part information of the wearing object may indicate that the wearing object is worn on the "head" of the character. If the wearing part information on the part where the wearing object is worn is specified as a position in a three-dimensional coordinate space, the wearing part information may be associated with multiple positions in the three-dimensional coordinate space. For example, the wearing part information indicating the position where the wearing object indicating a "headband" is worn may be associated with two parts of the character, the "left rear of the head" and the "right rear of the head". In other words, the wearing object indicating a "headband" may be worn on both the "left rear of the head" and the "right rear of the head". In the case where the wearing object is a T-shirt, the wearing part information of the wearing object may indicate that the wearing object is worn on the "torso" of the character.

[0051] Two types of wearable objects that share the same wearing location are displayed in the video at different times. In other words, two types of wearable objects that share the same wearing location are worn by a character at different times. In other words, two types of wearable objects that share the same wearing location are not worn by a character at the same time. For example, if a headband and a hat are both set as the wearing location to be "head," the headband and hat wearable objects are not displayed at the same time.

[0052] Each gift object may have a display time set according to its type. In one embodiment, the display time of the attachment object may be set longer than the display time of the effect object and the display time of the normal object. For example, the display time of the attachment object may be set to 60 seconds, the display time of the effect object may be set to 5 seconds, and the display time of the normal object may be set to 10 seconds.

[0053] The owned object information 63d is a list showing objects owned by users (viewing users and broadcasting users) of the video distribution system. An example of the owned object information 63c is shown in FIG. 4. As shown in the figure, the owned object information 63c stores an object ID that identifies an object owned by the viewing user in association with the account information of the viewing user. The account information of the viewing user is, for example, a user ID that identifies the viewing user. An object owned by the viewing user is, for example, the gift object described above. The broadcasting user can equip a character included in a video with an object that he or she owns while the video is being distributed.

[0054] The object display information 63e is information for managing an attached object displayed in association with a broadcast user's character in a video. An example of the object display information 63d is shown in FIG. 5. As shown in the figure, the object display information 63d stores an object ID that identifies an object displayed in the video in association with account information of the broadcast user who is broadcasting the video. The object display information 63d may store an object ID that identifies an attached object displayed in the video in association with account information of the broadcast user who is broadcasting the video in association with the broadcast user's character. The account information of the broadcast user is, for example, a user ID that identifies the broadcast user.

[0055] Next, the functions realized by the computer processors 21a and 21b will be described in more detail. The computer processor 21a functions as a face motion data generating unit 21a1 by executing computer-readable instructions included in the distribution program. Similarly, the computer processor 21b functions as a face motion data generating unit 21b1 by executing computer-readable instructions included in the distribution program. At least a part of the functions realized by the computer processors 21a and 21b may be realized by a computer processor other than the computer processors 21a and 21b of the video distribution system 1. At least a part of the functions realized by the computer processors 21a and 21b may be realized by, for example, a computer processor 61 mounted on the server device 60.

[0056] The facial motion data generating unit 21a1 generates facial motion data that is a digital representation of the facial movement of the broadcast user Ua based on the imaging data of the camera 24a. Similarly, the facial motion data generating unit 21b1 generates facial motion data that is a digital representation of the facial movement of the broadcast user Ub based on the imaging data of the camera 24b. The facial motion data is generated as time passes. The facial motion data may be generated at predetermined sampling time intervals. In this way, the facial motion data generated by the facial motion data generating unit 21a1 can digitally represent the facial movement (changes in facial expression) of the broadcast user Ua in a time series, and the facial motion data generated by the facial motion data generating unit 21b1 can digitally represent the facial movement (changes in facial expression) of the broadcast user Ub in a time series. The facial motion data generated by the facial motion data generating units 21a1 and 21b1 is transmitted to the server device 60 via the communication I / Fs 22a and 22b.

[0057] In the broadcast user device 20, in addition to the facial motion data generated by the facial motion data generating unit 21a1, body motion data may be generated which is a digital representation of the position and orientation of each part of the body of the broadcast user Ua, Ub other than the face. The broadcast user device 20 may transmit the body motion data to the server device 60 in addition to the facial motion data. In order to generate the body motion data, the broadcast users Ua, Ub may wear a motion sensor. The broadcast user device 20 may be configured to be able to generate the body motion data based on the detection information of the motion sensor worn by the broadcast users Ua, Ub. The body motion data may be generated at predetermined sampling time intervals. In this way, the body motion data represents the body movements of the broadcast users Ua, Ub in a time series as digital data. The generation of the body motion data based on the detection information of the motion sensor worn by the broadcast users Ua, Ub may be performed, for example, in a photography studio. The photography studio may be equipped with a base station, a tracking sensor, and a display. The base station may be a multi-axis laser emitter. The motion sensor attached to the broadcasting users Ua and Ub may be, for example, a Vive Tracker provided by HTC CORPORATION. The base station provided in the photography studio may be, for example, a base station provided by HTC CORPORATION. A supporter computer may be installed in a room separate from the photography studio. The display of the photography studio may be configured to display information received from the support computer. The server device 60 may be installed in the same room as the room in which the supporter computer is installed. The room in which the supporter computer is installed and the photography studio may be separated by a glass window. In this case, the operator of the supporter computer (sometimes referred to as a "supporter" in this specification) can visually recognize the broadcasting users Ua and Ub. The supporter computer may be configured to be able to change the settings of various devices provided in the photography studio in response to the operation of the supporter.The supporter computer can, for example, set the scanning interval by the base station, set the tracking sensor, and change various settings of various other devices. The supporter can input a message to the supporter computer and display the input message on the display of the photography studio.

[0058] The broadcast user device 20 may execute the broadcast program and / or other programs on a computer processor to realize functions other than the facial motion data generator 21a1. For example, the broadcast user device 20 may execute a viewing program for viewing videos to realize a function of playing received videos. The broadcast program and the viewing program may be packaged in one application software, or may be provided as separate application software.

[0059] The viewing user device 10 may be configured to provide the same functions as the broadcasting user device 20. For example, the viewing user device 10 may realize a function of playing received videos by executing a viewing program for video viewing, or may realize a function similar to that of the face motion data generator 21a1 by executing a broadcasting program.

[0060] Next, a more detailed description will be given of the functions realized by the computer processor 61. The computer processor 61 executes computer-readable instructions included in the distribution program to function as an animation generating unit 61a, a video generating unit 61b, a video distribution unit 61c, a gift request processing unit 61d, a gift purchase processing unit 61e, a co-starring application processing unit 61f, a determination unit 61g, a matching processing unit 61h, and a privilege granting unit 61i.

[0061] The animation generation unit 61a is configured to generate animation of a character by applying face motion data generated by the face motion data generation unit 21a1 of the broadcast user device 20 to predetermined model data included in the model data 63a. The animation generation unit 61a can generate animation of the character so that the character's facial expression changes based on the face motion data. Specifically, the animation generation unit 61a can generate animation of a character that moves in synchronization with the facial movement of the broadcast user Ua based on the face motion data related to the broadcast user Ua. Similarly, the animation generation unit 61a can generate animation of a character that moves in synchronization with the facial movement of the broadcast user Ub based on the face motion data related to the broadcast user Ub.

[0062] When body motion data regarding broadcast users Ua, Ub is provided from the broadcast user device 20, the animation generation unit 61a can generate animation of a character that moves in synchronization with the body and facial movements of broadcast users Ua, Ub based on the body motion data and facial motion data regarding broadcast users Ua, Ub.

[0063] When distributing a video of the broadcast user Ua, the video generating unit 61b can generate a background image showing a background using the object data 63b, and generate a video including this background image and an animation of a character corresponding to the broadcast user Ua. The video generating unit 61b can synthesize the generated video with the voice of the broadcast user Ua, which is generated based on the voice data received from the broadcast user device 20a. Similarly, when distributing a video of the broadcast user Ub, the video generating unit 61b can generate a background image showing a background using the object data 63b, and generate a video including this background image and an animation of a character corresponding to the broadcast user Ub. The video generating unit 61b can make the broadcast user Ua or broadcast user Ub attach an object that it owns to its own character in response to a request from the broadcast user Ua or broadcast user Ub. The object that can be attached is, for example, the above-mentioned attachment object. The broadcast user Ua or broadcast user Ub can transmit a display request to the server device 60 to attach an attachment object to their own character before or after the start of distribution of the video, by operating the broadcast user device 20a or broadcast user device 20b. The video generating unit 61b can have the wearable object attached to the character of the broadcast user device 20a or broadcast user device 20b based on this display request. The video generating unit 61b can synthesize the generated video with the voice of broadcast user Ub, which is generated based on the voice data received from the broadcast user device 20b. In the video generated by the video generating unit 61b, the characters corresponding to broadcast users Ua and Ub are displayed superimposed on a background image. In this way, the video generating unit 61b can generate animation of a character that moves in synchronization with the facial movements of broadcast user Ua or broadcast user Ub, and generate a video for distribution in which the voice of broadcast user Ua or broadcast user Ub is synthesized with this animation. A co-starring video including both the character of broadcast user Ua and the character of broadcast user Ub will be described later.

[0064] The video distribution unit 61c distributes the video generated by the video generation unit 61b. This video is distributed to the viewer user device 10 and other viewer user devices via the network 50. The generated video is also distributed to the broadcaster user devices 20a and 20b. The received video is played back in the viewer user device 10 and the broadcaster user devices 20a and 20b.

[0065] Display examples of videos distributed from the video distribution unit 61c are shown in Fig. 6, Fig. 7a, and Fig. 7b. In these figures, it is assumed that the broadcast user Ub uses the broadcast user device 20b to distribute a video including his / her own character, and that the video is being played on the viewing user device 10 and the broadcast user devices 20a and 20b. In Fig. 7a, it is assumed that the broadcast user Ua is not performing video distribution, and is watching a video distributed by another broadcast user, broadcast user Ub. For this reason, in the explanation of Fig. 6 and Figs. 7a to 7b, the broadcast user Ua may also be referred to as a viewing user. Fig. 6 shows a display example of a video being played on the viewing user device 10, Fig. 7a shows a display example of a video being played on the broadcast user device 20a, and Fig. 7b shows a display example of a video being played on the broadcast user device 20b.

[0066] 6, an image 30 of a moving image distributed from a server device 60 is displayed on the display of the viewing user device 10. The image 30 displayed on the viewing user device 10 includes a character 31 of the broadcasting user Ub generated by an animation generation unit 61a, a gift button 32, a rating button 33, a comment button 34, and a comment display area 35.

[0067] As shown in Fig. 7a, an image 40a of a video distributed from a server device 60 is displayed on the display of the broadcast user device 20a. This image 40a, like the image 30, includes a character 31 of the broadcast user Ub, a gift button 32, a rating button 33, a comment button 34, and a comment display area 35. In addition, the image 40a includes a co-starring request button 36 for requesting to co-star with the broadcast user Ub in a video distributed by the broadcast user Ub. The broadcast user device 20a can display the image 40a of the distributed video by, for example, executing application software including a viewing program.

[0068] As shown in Fig. 6, in one embodiment, the image 30 of the viewing user device 10 does not include the request to perform together button 36. For example, when application software including a distribution program has not been downloaded or installed in the viewing user device 10, or when application software including a distribution program has not been started in the viewing user device 10, the image 30 of the viewing user device 10 is generated so as not to include the request to perform together button 36. In another embodiment, the request to perform together button 36 may also be displayed in the image 30 of the viewing user device 10. For example, when application software including a distribution program has been downloaded or installed in the viewing user device 10, or when application software including a distribution program has been started in the viewing user device 10, the image 30 of the viewing user device 10 is generated so as to include the request to perform together button 36.

[0069] As described above, the character 31 is generated by applying the facial motion data of the broadcast user Ub to the model data included in the model data 63a, and therefore the facial expression changes in synchronization with the facial movements of the broadcast user Ub. When body motion data is provided from the broadcast user device 20, the character 31 can be controlled so that parts of the character 31 other than its face also change in synchronization with the body movements of the broadcast user Ub.

[0070] The gift button 32 is displayed on the image 30 so as to be selectable by an operation on the viewing user device 10. The gift button 32 can be selected, for example, by a tap operation on an area of ​​the touch panel of the viewing user device 10 where the gift button 32 is displayed. In one embodiment, when the gift button 32 is selected, a window for selecting a gift to be gifted to the broadcasting user (here, broadcasting user Ub) who is broadcasting the video being viewed is displayed on the image 30. The viewing user can purchase a gift to be gifted from among the gifts displayed on the window. In another embodiment, in response to the selection of the gift button 32, a window including a list of purchased gifts is displayed on the image 30. In this case, the viewing user can select a gift to be gifted from among the gifts displayed on the window. Gifts that can be gifted or purchased may include effect gifts, normal gifts, wearable gifts, and other gifts.

[0071] The rating button 33 is displayed on the image 30 so as to be selectable by a viewing user using the viewing user device 10. The rating button 33 may be selected, for example, by a tap operation on an area of ​​the touch panel of the viewing user device 10 where the rating button 33 is displayed. When the rating button 33 is selected by a viewing user viewing a video, rating information indicating that the video has been positively rated may be transmitted to the server device 60. The server device 60 can aggregate rating information from the viewing user device 10 and other viewing user devices. When the rating button 33 is selected, rating information indicating that the video has been negatively rated may be transmitted to the server device 60. Rating information indicating that a video has been positively rated may be called positive rating information, and rating information indicating that a video has been negatively rated may be called negative rating information.

[0072] The comment button 34 is displayed on the image 30 so as to be selectable by the user. When the comment button 34 is selected, for example, by a tap operation, a comment input window for inputting a comment is displayed on the image 30. The viewing user can input a comment via the input mechanism of the viewing user device 10 or the broadcasting user device 20a. The input comment is transmitted from the viewing user device 10 and the broadcasting user device 20a to the server device 60. The server device 60 accepts comments from the viewing user device 10, the broadcasting user device 20a, and other user devices, and displays the comments in a comment display area 35 in the image 30. In the comment display area 35, comments posted by each user are displayed, for example, in chronological order. The comment display area 35 occupies a part of the area of ​​the image 30. There is an upper limit to the number of comments that can be displayed in the comment display area 35. In the illustrated example, up to three comments can be displayed in the comment display area 35. When comments are posted that exceed the upper limit set in the comment display area 35, the comments are deleted from the comment display area 35 in order of oldest posted comments. Therefore, the more frequently comments are received, the shorter the display time of each comment in the comment area 35. In one embodiment, a scroll bar is provided on the image 30, and comments that have been deleted from the comment display area 35 can be viewed by operating the scroll bar to scroll through the comments.

[0073] As shown in Fig. 7b, an image 40b of a video distributed from the server device 60 is displayed on the display of the broadcast user device 20b. The image 40b displayed on the broadcast user device 20b includes a character 31 corresponding to the broadcast user Ub, display command buttons 42a to 42c for displaying the worn gift requested to be displayed by the viewing user, and a comment display area 35. The image 40b displayed on the broadcast user device 20b includes the same background image, character image, and comment as the images 30 and 40a. On the other hand, the image 40b differs from the images 30 and 40a in that it does not include the gift button 32, the rating button 33, the comment button 34, and the request to appear together button 36, but includes the display command buttons 42a to 42c.

[0074] The display instruction buttons 42a to 42c are displayed on the display screen 40 in response to receiving a display request from a viewing user to display a wearing gift, which will be described later. In the illustrated embodiment, three display instruction buttons 42a to 42c are displayed on the image 40b. Each of the display instruction buttons 42a to 42c is displayed on the display screen 40 so as to be selectable by the broadcasting user. When any of the display instruction buttons 42a to 42c is selected, for example, by a tap operation, an operation is performed to display the wearing gift corresponding to the selected display instruction button. In this manner, the display instruction buttons 42a to 42c are display instruction objects for issuing an instruction to display the wearing gift in the video being broadcast. For this reason, in this specification, the display instruction buttons 42a to 42c may be referred to as display instruction objects 42a to 42c. In addition, when it is not necessary to distinguish the display instruction objects 42a to 42c from one another, they may be simply referred to as display instruction objects 42. A specific example of displaying the wearing gift will be described later. The image 40b may be displayed on the supporter computer described above. The display instruction objects 42a to 42c may be selected in response to a supporter's operation of the supporter computer.

[0075] Each time a display request for displaying an attached gift is received, a display instruction object 42 corresponding to the display request is added to the image 40b. There is an upper limit to the number of display instruction objects 42 that can be displayed on the image 40b. In the illustrated embodiment, the upper limit number of display instruction objects 42 that can be displayed on the image 40b is three. In this case, the image 40b has a display area that can display three display instruction objects. When four or more display requests for attached gifts are received, the display instruction objects 42 corresponding to the fourth and subsequent display requests are not displayed on the image 40b. The display instruction object 42 corresponding to the fourth-received display request for attached gifts is displayed on the image 40b when one of the three display instruction objects 42 already displayed is selected and a space is created in the display area.

[0076] The gift request processing unit 61d receives a gift display request from a viewing user and performs processing for displaying a gift object corresponding to the display request. Each viewing user can transmit a gift display request to the server device 60 by operating his / her viewing user device. The gift display request may include the user ID of the viewing user, gift identification information (gift ID) that identifies the gift requested to be displayed and / or object identification information (object ID) that identifies the gift object corresponding to the gift requested to be displayed.

[0077] As described above, the gift object representing the gift may include an effect object corresponding to the effect gift, a normal object corresponding to the normal gift, and an attachment object corresponding to the attachment gift.

[0078] In one aspect, when the gift request processing unit 61d receives a display request for a specific normal gift from a viewing user, the gift request processing unit 61d performs processing for displaying a normal object representing the normal gift requested to be displayed in the video based on the display request. For example, when a display request for a normal gift representing a bag is made, the gift request processing unit 61d displays a normal object 37a representing a bag in the image 30 based on the display request, as shown in FIG. 8. Similarly, when a display request for a normal gift representing a teddy bear is made, the gift request processing unit 61d displays a normal object 37b representing a teddy bear in the image 30 based on the display request, as shown in FIG. 8. Although not shown in the figure, the normal objects 37a and 37b are also displayed in the image 40a in the broadcast user device 20a and the image 40b in the broadcast user device 20b, similar to the image 30.

[0079] In one embodiment, when the gift request processing unit 61d receives a display request for a specific effect gift from a viewing user, the gift request processing unit 61d performs a process for displaying an effect object corresponding to the effect gift requested to be displayed on the display image of the moving image based on the display request. For example, when a display request is made for an effect gift showing confetti or fireworks, the gift request processing unit 61d displays an effect object (not shown) corresponding to the effect gift showing confetti or fireworks on the image 30, the image 40a, and the image 40b based on the display request.

[0080] The display request for the normal gift may include a display position designation parameter that designates the display position of the normal object representing the normal gift. In this case, the gift request processing unit 61d can display the normal object at the position designated by the display position designation parameter. When the display position and display range of the character 31 are determined, the display position designation parameter can designate a relative position with respect to the character 31 as the display position of the normal object.

[0081] In one embodiment, the normal object may be displayed to move within the image 30 of the video. For example, the normal objects 37a and 37b may be displayed to fall from the top to the bottom of the screen. In this case, the normal objects 37a and 37b may be displayed within the image 30 from the start of the fall until the bottom of the image 30, and may disappear from the image 30 after falling to the bottom of the image 30. The movement direction of the normal objects 37a and 37b within the screen may be set arbitrarily. For example, the normal objects 37a and 37b may move from left to right, right to left, upper left to lower left, or in other directions in the image 30. The normal objects 37a and 37b can move on various trajectories. For example, the normal objects 37a and 37b can move on a straight trajectory, a circular trajectory, an elliptical trajectory, a spiral trajectory, or other trajectories. In addition to or instead of the display position designation parameter, the viewing user may include at least one of a movement direction parameter designating the movement direction of the normal objects 37a and 37b and a trajectory parameter designating the trajectory along which the normal objects 37a and 37b move, in the display request for the normal object. In one embodiment, among the effect objects and normal objects, those whose size in the virtual space is smaller than the reference size may be displayed so as to overlap part or all of them with the character 31. In one embodiment, among the effect objects and normal objects, those whose size in the image 30 is larger than the reference size are displayed at a position that does not overlap with the character 31. In another embodiment, among the effect objects and normal objects, those whose size in the virtual space is larger than the reference size may be displayed behind the character 31. The normal objects 37a and 37b may also be displayed so as to move as described above in the images 40a and 40b.

[0082] In one aspect, when the gift request processor 61d receives a display request from a viewing user for a specific wearing object, the gift request processor 61d displays the display instruction objects 42a-42c on the image 40b on the broadcast user device 20 based on the display request, as shown in FIG. 7b. Each of the display instruction objects 42a-42c is associated with the wearing gift for which the display request has been made. When any of the display instruction objects 42a-42c is selected, the wearing gift associated with the selected display instruction object is displayed in the video being broadcast. For example, when a headband in the shape of cat ears is associated with the display instruction object 42b, when the display instruction object 42b is selected, the gift request processor 61d displays the wearing object 38 representing the headband corresponding to the selected display instruction object 42b in the video being broadcast. Display examples of a video including the wearing object 38 representing the headband are shown in FIG. 9 and FIG. 10a-FIG. 10b. Also, as shown in FIG. 10b, the selected display instruction object 42b is deleted from the image 40b.

[0083] The wearing object is displayed in the video in association with a specific part (wearing part) of the character. For example, the wearing object may be displayed in the video so as to be in contact with the wearing part of the character. The wearing object 38 may be displayed in the video so as to be worn on the wearing part of the character. In one embodiment, the wearing object 38 representing a headband is associated with the head of the character. Therefore, in the display examples shown in FIG. 9 and FIG. 10a to FIG. 10b, the wearing object 38 is worn on the head of the character 31. The wearing object may be displayed on the display screen of the video so as to move in association with the movement of the wearing part of the character. For example, when the head of the character 31 wearing the wearing object 38 representing a headband moves, the wearing object 38 representing the headband also moves in association with the head of the character 31 as if the headband was worn on the head of the character 31.

[0084] As described above, the gift object information 63c may include wearing part information indicating which part of the character the wearing object is associated with. In one embodiment, when a wearing object is worn by a character, the gift request processing unit 61d prohibits the display of other wearing objects worn on the same or overlapping parts indicated by the wearing part information of the wearing object until the display time of the wearing object has elapsed. For example, a headband associated with both the "left rear of the head" and the "right rear of the head" overlaps with a hair accessory associated with the "left rear of the head", so when a headband is displayed, the display of a hair accessory with "left rear of the head" set as the wearing part information is prohibited. On the other hand, a headband associated with the "left rear of the head" and the "right rear of the head" and an earring associated with the "left ear (of the head)" and the "right ear (of the head)" can be worn at the same time because the wearing parts on the character do not overlap.

[0085] In one embodiment, in order to prohibit the display of multiple wearing objects having the same or overlapping wearing parts, the display instruction object 42 for displaying the wearing object to be prohibited from being displayed may be deactivated. In the embodiment shown in FIG. 7b, it is assumed that the display instruction object 42a is an object (button) for instructing the display of a hair accessory with "left rear of head" set as the wearing part information. In this case, since the character 31 is wearing the wearing object 38 representing a headband, the wearing object representing this hair accessory is prohibited from being displayed in the video while the wearing object 38 is worn by the character 31. In one embodiment, in order to prohibit the wearing object representing this hair accessory from being displayed in the video, the display instruction object 42a associated with the wearing object representing the hair accessory is deactivated. For example, the display instruction object 42a is made unselectable even if operated while the wearing object 38 is worn by the character 31. In another embodiment, the display instruction object 42a is erased from the image 40b while the wearing object 38 is worn by the character 31. The deactivated display instruction object 42a is activated again when the display time of the headband has elapsed. To reactivate the display instruction object 42a, for example, the display instruction object 42a that was made unselectable may be made selectable again, or the display instruction object 42a that was made hidden may be displayed again in the image 40b.

[0086] The gift request processing unit 61d performs processing for displaying a gift object in response to a gift display request from a viewing user, as well as processing for updating the object display information 63e. For example, when an attachment object 38 is displayed in a video distributed by the distribution user Ub, an object ID for identifying the attachment object 38a is stored in the object display information 63e in association with a user ID for identifying the distribution user Ub. Similarly, when another attachment object is displayed, an object ID for identifying the displayed attachment object is stored in association with the user ID of the distribution user Ub. When an attachment object displayed in a video disappears from the screen (when the attachment object is no longer worn), the gift request processing unit 61d may delete the object ID for identifying the attachment object that disappeared from the screen from the object display information 63e.

[0087] In one aspect, the gift purchase processing unit 61e transmits, in response to a request from a viewing user of a video, purchase information for each of a plurality of gift objects that can be purchased in relation to the video to the viewing user device (e.g., the viewing user device 10 and the broadcasting user device 20a) of the viewing user. The purchase information for each gift object may include the type of the gift object (effect object, normal object, or wearable object), an image of the gift object, the price of the gift object, and other information required for purchasing the gift object. The viewing user can select a gift object to purchase based on the purchase information of the gift object displayed on the viewing user device 10 or the broadcasting user device 20a. The selection of the gift object to be purchased may be performed by operating the viewing user device 10 or the broadcasting user device 20a. When the gift object to be purchased is selected by the viewing user, a purchase request for the gift object is transmitted to the server device 60. The gift purchase processing unit 61e performs a payment process based on the purchase request. When this payment process is completed, the purchased gift object is held by the viewing user. The storage 23 may store the gift ID of the purchased gift (or the gift object ID of the gift object indicating the gift) in association with the user ID of the viewing user who purchased the gift. The purchase of the gift object may be performed using a lottery, a loot box, or a roulette. In other words, the viewing user may obtain a gift item through the purchase of a lottery, a loot box, or a roulette without specifying the gift object to be purchased. In this specification, obtaining a gift item without specifying the gift object to be purchased may also be included in the "purchase" of a gift item.

[0088] The purchasable gift objects may be different for each video. The purchasable gift objects may be purchasable across multiple videos. That is, the purchasable gift objects may include unique gift objects that are unique to each video and common gift objects that are purchasable across multiple videos. For example, an effect object showing confetti may be a common gift object that is purchasable across multiple videos.

[0089] In one aspect, when an effect object is purchased while watching a predetermined video, the effect object to be purchased may be automatically displayed in the video being watched in response to completion of a payment process for purchasing the effect object. When a normal object is purchased while watching a predetermined video, the normal gift object to be purchased may be automatically displayed in the video being watched in response to completion of a payment process for purchasing the normal object, similar to the above.

[0090] In another aspect, in response to the completion of the payment process in the gift purchase processing unit 61e for the effect object to be purchased, a payment completion notice may be transmitted to the viewing user device 10, and a confirmation screen may be displayed on the viewing user device 10 to confirm with the viewing user whether or not to make a display request for the purchased effect object. When the viewing user selects to make a display request for the purchased effect object, a display request requesting the display of the purchased effect object may be transmitted from the client device of the viewing user to the gift request processing unit 61d, and the gift request processing unit 61d may perform a process of displaying the purchased effect object in the video 70. When the object to be purchased is a normal object, a confirmation screen may be displayed on the viewing user device 10 to confirm with the viewing user whether or not to make a display request for the purchased normal object, similar to the above.

[0091] When a co-starring application is received from another broadcast user for a broadcast user, the co-starring application processing unit 61f generates a co-starring video in which the broadcast user and the other broadcast user virtually co-star based on the co-starring application. In the following, a case will be described in which a broadcast user Ua who is watching a video including the character 31 of the broadcast user Ub applies to co-star with the broadcast user Ub. The co-starring application processing unit 61f can generate co-starring videos based on not only a co-starring application from the broadcast user Ua but also co-starring applications from various broadcast users.

[0092] In one embodiment, the co-starring request processing unit 61f accepts a co-starring request from the broadcast user Ua and generates a co-starring video in response to the co-starring request. The broadcast user Ua can transmit a co-starring request to the server device 60 by operating the broadcast user device 20a. Specifically, when the co-starring request button 36 displayed on the broadcast user device 20a is selected, a co-starring request is transmitted from the broadcast user device 20a to the server device 60. The co-starring request button 36 can be selected, for example, by a tap operation on an area on the touch panel of the broadcast user device 20a where the co-starring request button 36 is displayed. In response to the co-starring request button 36 being selected, application software including a broadcast program may be started in the broadcast user device 20a. The co-starring request transmitted from the broadcast user device 20a may include a user ID that identifies the broadcast user Ua. The co-starring request button 36 may be displayed from the time of video distribution, or may be displayed in response to the broadcast user Ub inviting the broadcast user Ua. The broadcast user Ub may send an invitation to the broadcast user Ua to perform together from the broadcast user device 20b to the broadcast user device 20a, either via the server device 60 or without the server device 60.

[0093] The co-starring application processing unit 61f may determine whether or not to permit the co-starring application in response to receiving the co-starring application from the broadcast user device 20a. The co-starring application processing unit 61f may determine whether or not to permit the co-starring application based on various information related to the broadcast user Ua who made the co-starring application. For example, if the evaluation of the broadcast user Ua by other users is higher than a predetermined standard, the co-starring application from the broadcast user Ua may be permitted, and conversely, if the evaluation of the broadcast user Ua by other users is lower than a predetermined standard, the co-starring application from the broadcast user Ua may be rejected. In addition, the broadcast user Ub may be inquired of whether or not to permit the co-starring application from the broadcast user Ua, and the co-starring application may be determined in response to the response.

[0094] In one embodiment, when the co-starring request from the broadcast user Ua is permitted, the co-starring request processing unit 61f generates a co-starring video in which the broadcast user Ua and the broadcast user Ub appear together. To generate the co-starring video, the co-starring request processing unit 61f acquires face motion data and voice data of the broadcast user Ua from the broadcast user Ua. The face motion data of the broadcast user Ua may be transmitted from the broadcast user device 20a to the server device 60 together with the co-starring request, or may be transmitted from the broadcast user device 20a to the server device 60 in response to a request from the server device 60 after the co-starring request is permitted. When the body movements of the broadcast user Ua are also expressed as animation, the body motion data may be provided to the server device 60 together with the face motion data of the broadcast user Ua. When the co-starring request processing unit 61f cannot acquire face motion data from the broadcast user Ua, it may acquire only voice data from the broadcast user Ua. For example, if the broadcast user device 20a used by the broadcast user Ua does not have the function of the facial motion data generation unit 21a1 or other functions for video distribution, if the function is disabled, or in other cases, the co-starring request processing unit 61f may acquire audio data from the broadcast user Ua without acquiring facial motion data (and body motion data).

[0095] In one embodiment, the co-starring application processing unit 61f generates a co-starring image by adding animation of the character of the broadcast user Ua to a video including the character 31 of the broadcast user Ub based on the facial motion data of the broadcast user Ua. The animation of the character of the broadcast user Ua may be generated in the animation generation unit 61a based on the facial motion data of the broadcast user Ua. The animation of the character of the broadcast user Ua may be generated by applying the facial motion data of the broadcast user Ua to the model data for the broadcast user Ua included in the model data 63a. For example, the co-starring application processing unit 61f can generate a co-starring video including the character of the broadcast user Ua and the character 31 of the broadcast user Ub by generating a layer including animation of the character of the broadcast user Ua and superimposing this layer on a video including the character 31 of the broadcast user Ub. When the co-starring application processing unit 61f acquires voice data but not face motion data (and body motion data) from the broadcast user Ua, it can generate a co-starring video including the character 31 of the broadcast user Ub and the voice of the broadcast user Ua by superimposing a voice generated based on the voice data acquired from the broadcast user Ua onto a video including the character 31 of the broadcast user Ub. Even when model data for the broadcast user Ua is not set as the model data 63a, the co-starring application processing unit 61f can generate a co-starring video including the character 31 of the broadcast user Ub and the voice of the broadcast user Ua by superimposing a voice generated based on the voice data acquired from the broadcast user Ua onto a video including the character 31 of the broadcast user Ub. In this way, the broadcast user Ua can co-star with the broadcast user Ub using his or her own voice.

[0096] When face motion data cannot be obtained from the broadcast user Ua, the co-starring application processing unit 61f distributes a co-starring video including the voice of the broadcast user Ua and the character 31 of the broadcast user Ub, and then generates the character of the broadcast user Ua based on the face motion data of the broadcast user Ua at the timing when the face motion data is obtained from the broadcast user Ua, thereby generating a co-starring video including the character of the broadcast user Ua and the character 31 of the broadcast user Ub. In this way, the co-starring application processing unit 61f can switch a co-starring video including the voice of the broadcast user Ua and the character 31 of the broadcast user Ub to a co-starring video including the character of the broadcast user Ua and the character 31 of the broadcast user Ub. Conversely, the co-starring application processing unit 61f can switch a co-starring video including the character of the broadcast user Ua and the character 31 of the broadcast user Ub to a co-starring video including the voice of the broadcast user Ua and the character 31 of the broadcast user Ub. In this way, a co-starring video in which voice co-starring is performed (sometimes referred to as an "audio co-starring video") and a co-starring video in which characters co-star are performed (sometimes referred to as a "character co-starring video") may be alternately switched.

[0097] The generated co-starring video is distributed by the video distribution unit 61c. The co-starring video may be created by the broadcast user device 20a or the broadcast user device 20b. When the co-starring video is generated by the broadcast user device 20a, the co-starring video is transmitted from the broadcast user device 20a to the server device 60, and distributed by the video distribution unit 61c of the server device 60. When the co-starring video is generated by the broadcast user device 20b, the co-starring video is transmitted from the broadcast user device 20b to the server device 60, and distributed by the video distribution unit 61c of the server device 60.

[0098] Display examples of a co-starring video distributed from video distribution unit 61c are shown in Fig. 11 and Fig. 12a to Fig. 12b. In these figures, it is assumed that distribution user Ua has applied to co-star with a video distributed by distribution user Ub, and as a result, a co-starring video including the character of distribution user Ua and the characters of the distribution users is distributed, and distribution user Ua has selected the co-starring application button 36 while viewing the image shown in Fig. 10a.

[0099] As shown in Fig. 11, a co-starring video image 130 distributed from the server device 60 is displayed on the display of the viewing user device 10. The co-starring video image 130 displayed on the viewing user device 10 includes a character 131 of the broadcasting user Ua in addition to a character 31 of the broadcasting user Ub. The character 131 of the broadcasting user Ua is generated in the animation generation unit 61a or the co-starring application processing unit 61f, as described above. The co-starring video image 130 includes a gift button 32, a rating button 33, a comment button 34, and a comment display area 35, similar to the original distribution image shown in Fig. 6.

[0100] As shown in the figure, the image 130 of the co-starring video is displayed horizontally. When the co-starring application processing unit 61f or the video distribution unit 61c distributes the co-starring image, it distributes it as a horizontally long image. As a result, the co-starring video is displayed as a horizontally long image 130 on the viewing user device 10. In one embodiment, as shown in FIG. 6, even if the original video (image) is vertically long, the co-starring image is converted to a horizontally long image and distributed. Since the co-starring image includes characters of two distribution users, by making it a horizontally long image, it is easy to display the characters of two people without overlapping. For example, in the co-starring image, the left half from the viewpoint of the viewing user may be an area assigned to the character of the original distribution user (host user), and the right half may be an area assigned to the character of the distribution user (guest user) who made the co-starring application. The allocation of areas to each distribution user may be determined arbitrarily so that the characters do not overlap.

[0101] As shown in FIG. 12a and FIG. 12b, the broadcast user device 20a displays an image 140a of the co-starring video broadcast from the server device 60, and the broadcast user device 20b displays an image 140b of the co-starring video. The images 140a and 140b contain the same background image, character image, and comment as the image 130. On the other hand, the images 140a and 140b differ from the image 130 in that they do not contain the gift button 32, the rating button 33, and the comment button 34, but contain the display instruction buttons 42a, 42c, and 42d. Since the broadcast user Ua and the broadcast user Ub appear together in the same co-starring image, the image 140a displayed on the broadcast user device 20a of the broadcast user Ua and the image 140b displayed on the broadcast user device 20b of the broadcast user Ub may be the same. The images 140a and 140b are also displayed horizontally in the same way as the image 130, from the viewpoint of arranging multiple characters without overlapping. In the example shown in Fig. 11 and Fig. 12a-12b, it is assumed that a display request for a wearing gift showing earrings is made by a viewing user viewing the co-starring image shown in Fig. 11, and as a result, a display instruction object 42d is added to the display image of the broadcast user device 20a and the broadcast user device 20b in response to the display request. The display instruction object 42d is an object (button) for instructing the display of earrings with "ear" set as the wearing body part information.

[0102] The determination unit 61g determines whether an object is displayed in association with one of the characters of the host user and the guest user in the co-starring video when the object is displayed in association with the other character. Therefore, the determination unit 61g can determine whether the other character is wearing the same wearing object when one of the characters of the host user and the guest user is wearing an wearing object. The determination by the determination unit 61g may be performed with reference to the object display information 63e. As described above, the object display information 63e stores an object ID that identifies the wearing object displayed in association with the character of the broadcast user in the video in association with the account information of the broadcast user. Therefore, based on the object display information 63e, it is possible to identify the wearing object worn by the character 31 of the host user and the wearing object worn by the character 131 of the guest user. Based on the object display information 63e, the determination unit 61g can determine whether the host user and the guest user are wearing the same wearing object.

[0103] A specific example of the determination by the determination unit 61g will be described with reference to the co-starring video shown in Fig. 11. In the image 130 of the co-starring video shown in Fig. 11, the attachment object 38 is displayed in association with the host user's character 31, but there is no attachment object displayed in association with the guest user's character 131. Therefore, the determination unit 61g determines, based on the object display information 63e, that the attachment object 38 displayed in association with the host user's character 31 is not displayed in association with the guest user's character 131 (the attachment object 38 is not attached to the character 131). When at least one of the host user's character 31 and the guest user's character 131 is wearing multiple attachment objects, the determination unit 61g can perform the above determination for each of the multiple attachment objects.

[0104] The matching processing unit 61h displays a matching object in the co-starring video according to the determination result of the determination unit 61g. Specifically, when the determination unit 61g determines that an object identical to the attachment object is not displayed in association with the other character when an attachment object is displayed for one of the characters of the host user and the guest user, the matching processing unit 61h performs processing to display a matching object identical to the object displayed in association with the one character in association with the other character.

[0105] A specific example of the process in the matching processing unit 61h will be described with reference to FIG. 11 and FIG. 13. When the co-starring video illustrated in FIG. 11 is being distributed, the determination unit 61g determines that the wearing object 38 displayed in association with the character 31 of the distribution user Ub (host user) is not displayed in association with the character 131 of the distribution user Ua (guest user). Based on the determination in the determination unit 61g, the matching processing unit 61h displays the matching object 138 in association with the guest user's character 131 not wearing the wearing object 38. In other words, the character 131 is made to wear the matching object 138. The matching object 138 is the same object as the wearing object 38. In the illustrated example, both the wearing object 38 and the matching object 138 are objects indicating a headband imitating cat ears. Since the wearing object 38 and the matching object 138 are the same wearing object, they may be identified by the same object ID.

[0106] The matching processing unit 61h may perform a process of displaying the matching object based on a display request requesting the display of the matching object. In one embodiment, when it is determined that the attachment object 38 displayed in association with the host user's character 31 in the co-starring video is not displayed in association with the guest user's character 131, the matching processing unit 61h may perform a process of prompting the viewing user of the co-starring video to transmit a display request for the matching object. For example, the gift button 32 may be highlighted in the image 130 of the viewing user device 10 playing the co-starring video. The highlighting of the gift button 32 may be performed, for example, by blinking the gift button 32, changing the color of the gift button 32, changing the size of the gift button 32, or in other manners. Various manners may be considered for the process of prompting the viewing user to make a display request for the matching object. The process of prompting the viewing user to make a display request for the matching object may include, for example, a process of highlighting the attachment object 38 attached to the character 31 in the image 130, a process of displaying a message indicating that the attachment object 38 is attached to only one of the characters in the image 130, a process of displaying a message prompting the viewing user to make a display request for the other matching objects, and other processes.

[0107] When a process is performed to prompt the viewing user to make a display request for the matching object, the viewing user can operate the viewing user device 10 to transmit a display request for the matching object to the server device 60. For example, the viewing user can operate the gift button 32 to select an object identical to the attachment object 38 as an object to be gifted, and transmit a display request for the selected object (matching object) to the server device 60. The display request for the matching object may include an object ID that identifies the matching object to be displayed. When the server device 60 receives a display request for the matching object from the viewing user device 10, the gift request processing unit 61d displays the matching object 138 in association with the character 131 based on the display request. Since the matching object 138 is a type of attachment object, a display instruction object associated with the matching object 138 may be displayed in at least one of the image 140a displayed on the broadcasting user device 20a and the image 140b displayed on the broadcasting user device 20b when a display request for the matching object is made. In this case, the matching object 138 may be displayed in the co-starring video in response to the selection of the display instruction object associated with the matching object 138. The matching object 138 may be displayed in the co-starring video in response to a display request for the matching object 138 being accepted by the server device 60, regardless of the selection of the display instruction object.

[0108] A display example of the matching object 138 is shown in Fig. 13. Fig. 13 shows a display example of an image of a co-starring video in which the matching object 138 is displayed in association with the character 131. As shown in Fig. 13, in the co-starring video 130 displayed on the viewing user device 10, the matching object 138 is displayed in association with the character 131. Similarly, in an image 140a displayed on the broadcasting user device 20a and an image 140b displayed on the broadcasting user device 20b, the matching object 138 is displayed in association with the character 131. As shown in the figure, the attachment object 38 and the matching object 138 are attached to the same attachment parts of the character 31 and the character 131. Specifically, the attachment object 38 and the matching object 138 are displayed in association with the head of the character 31 and the head of the character 131, respectively.

[0109] In another embodiment, when it is determined that the attachment object 38 displayed in association with the host user's character 31 in the co-starring video is not displayed in association with the guest user's character 131, the matching processing unit 61h can perform a process of prompting the user of the character 131 not wearing the attachment object 38 (the broadcast user Ua) to send a display request for the matching object. Various modes are conceivable for the process of prompting the broadcast user Ua, who is the user of the character 131 not wearing the attachment object 38, to send a display request for the matching object. The process of prompting the broadcast user Ua to send a display request for the matching object may include, for example, a process of highlighting the attachment object 38 worn on the character 31 in the image 140a, a process of displaying a message indicating that the attachment object 38 is worn on only one of the characters in the image 140a, a process of displaying a message prompting the broadcast user Ua to send a display request for the other matching object, and other processes. When the process of prompting the broadcast user Ua to send a display request for the matching object is performed, the broadcast user Ua can operate the broadcast user device 20a to send a display request for the matching object to the server device 60. The animation generating unit 61b displays the matching object 138 in association with the character 131 based on the display request.

[0110] In another embodiment, when it is determined that the attachment object 38 displayed in association with the host user's character 31 in the co-starring video is not displayed in association with the guest user's character 131, the matching processing unit 61h can perform processing to prompt the user of the character 31 wearing the attachment object 38 (broadcast user Ub) to send a display request for the matching object. Various modes are conceivable for processing to prompt the broadcast user Ub, who is the user of the character 31 wearing the attachment object 38, to send a display request for the matching object. The processing to prompt the broadcast user Ub to request the display of the matching object may include, for example, a processing to highlight the attachment object 38 worn on the character 31 in the image 140b, a processing to display a message indicating that the attachment object 38 is worn on only one of the characters in the image 140b, a processing to display a message prompting the broadcast user Ub to make a display request for the other matching object, and other processing. When processing to prompt the broadcast user Ub to request the display of the matching object is performed, the broadcast user Ub can operate the broadcast user device 20b to send a display request for the matching object to the server device 60. The animation generating unit 61b displays the matching object 138 in association with the character 131 based on the display request.

[0111] By displaying the matching object 138 in the co-starring video, both the character 31 and the character 131 will wear the same attachment object. This improves the uniformity of the co-starring video. More specifically, the uniformity of the appearance of the host user's character and the guest user's character in the co-starring video is improved.

[0112] The matching object 138 is displayed in the co-starring video based on a display request of the matching object 138 by the viewing user, a display request by the broadcast user Ua, or a display request by the broadcast user Ub. The matching object 138 may be attachable to the character 131 only in the co-starring video. In other words, a matching object (e.g., matching object 138) displayed in a specific co-starring video based on a display request of the matching object 138 by the viewing user, a display request by the broadcast user Ua, or a display request by the broadcast user Ub cannot be used by other videos.

[0113] When a matching object is displayed in association with one of the characters of the users who appear together in the co-starring video, the bonus granting unit 61i grants a bonus to the other user. For example, in the example shown in FIG. 11 and FIG. 13, when the matching object 138 is displayed in association with the character 131, the bonus granting unit 61i grants a bonus to the distribution user Ub who is the user of the character 131. When the matching object 138 is displayed in association with the character 131, the bonus granting unit 61i may grant a bonus to the distribution user Ua who is the user of the character 131. When the matching object 138 is displayed in association with the character 131, the bonus granting unit 61i may grant a bonus to the viewing user who is viewing the co-starring video. The bonus granting unit 61i may grant a bonus to all viewing users who are viewing the co-starring video, or may grant a bonus only to some of the viewing users who are viewing the co-starring video. The reward granting unit 61i may grant rewards to each of the broadcast user Ua, the broadcast user Ub, and the viewing user. The reward granting unit 61i may grant rewards only to the viewing users who have gifted the matching object 138 among the viewing users who are watching the co-starring video. The rewards granted by the reward granting unit 61i may include points that can be used in the video distribution service provided by the video distribution system 1, points that can be used on a specific platform other than the video distribution system 1, crypto assets such as bitcoin, gifts for gifting to other broadcast users (e.g., broadcast user Ub), objects that can be displayed in the distributed video, and various other rewards.

[0114] Next, a video distribution process according to one aspect will be described with reference to Fig. 14 to Fig. 16. Fig. 14 is a flow diagram showing the flow of a video distribution process in one embodiment, Fig. 15 is a flow diagram showing the flow of a process for distributing a co-starring video in one embodiment, and Fig. 16 is a flow diagram showing the flow of a process for displaying a matching object in one embodiment. In the video distribution process described with reference to these figures, a video generated based on face motion data acquired by a broadcast user Ub using a broadcast user device 20b is distributed, and it is assumed that a broadcast user Ua who is watching this video makes a co-starring request.

[0115] First, in step S11, face motion data, which is a digital representation of the facial movement (expression) of the broadcast user Ub, is generated. The generation of the face motion data is performed, for example, by the face motion data generator 121b of the broadcast user device 20b. In the broadcast user device 20b, voice data may be generated based on voice input from the broadcast users Ua and Ub. The generated face motion data and voice data are transmitted to the server device 60.

[0116] Next, in step S12, the facial motion data from the broadcast user device 20b is applied to the model data for the broadcast user Ub, thereby generating an animation of the character object 31 that moves in synchronization with the facial movement of the broadcast user Ub. The generation of this animation is performed, for example, by the animation generation unit 61a described above.

[0117] Next, in step S13, a moving image including an animation of the character object 31 of the broadcast user Ub is generated. The voice of the broadcast user Ub may be synthesized into this moving image. The animation of the character object 31 is displayed superimposed on a background image. The generation of this moving image is performed, for example, by the above-mentioned moving image generating unit 61b.

[0118] Next, the process proceeds to step S14, where the video generated in step S13 is distributed. This video is distributed to the viewer user device 10, other viewer user devices, and distributor user devices via the network 50. When the distributed video is played on the viewer user device 10, for example, an image of the video as shown in FIG. 5 is displayed on the viewer user device 10. The video is continuously distributed for a predetermined distribution period. The distribution period of the video can be set to, for example, 30 seconds, 1 minute, 5 minutes, 10 minutes, 30 minutes, 60 minutes, 120 minutes, or any other time. The video is distributed, for example, by the above-mentioned video distribution unit 61c.

[0119] Next, the process proceeds to step S15, where it is determined whether or not a termination condition for ending the distribution of the video has been met. The termination condition may be, for example, that the end time of the distribution has been reached, that the broadcast user Ub has performed an operation to end the distribution on the broadcast user device 20b, or other conditions. If the termination condition has not been met, the processes of steps S11 to S14 are repeatedly executed, and the distribution of the video including animation synchronized with the movements of the broadcast user Ub, or the co-starring video if the co-starring with the broadcast user Ua is continuing, is continued. If it is determined that the termination condition has been met for the video, the distribution process of the video is terminated.

[0120] Next, a co-starring video distribution process in one embodiment will be described with reference to Fig. 15. Fig. 15 is a flow diagram showing the flow of a process for distributing a co-starring video in one embodiment. In the co-starring video distribution process described with reference to Fig. 15, it is assumed that a co-starring application is made to the broadcast user Ub from the broadcast user Ua who is watching a video including the character object of the broadcast user Ub. The co-starring video distribution process is performed in parallel with the video distribution process shown in Fig. 14.

[0121] First, during distribution of a video including a character object of the broadcast user Ub, in step S21, it is determined whether or not a co-starring request has been made from a user other than the broadcast user Ub. The co-starring request is made, for example, by the broadcast user Ua. If a co-starring request has been made by the broadcast user Ua, the process proceeds to step S22.

[0122] In step S22, one or more guest users are selected from among the broadcast user Ua who requested the co-starring and other users. In this embodiment, it is assumed that the broadcast user Ua is selected as the guest user. The determination of whether or not to permit this co-starring request is performed, for example, by the co-starring request processing unit 61f.

[0123] In step S22, if the co-starring application of the broadcast user Ua is permitted, the process proceeds to step S23. In step S23, animation of the broadcast user Ua's character object 131 is generated based on the face motion data of the broadcast user Ua. Then, a co-starring video is generated by adding animation of the broadcast user Ua's character object 131 to the video being distributed (video including the character object 31 of the broadcast user Ub). The process in step S22 is performed, for example, by the above-mentioned co-starring application processing unit 61f.

[0124] Next, the process proceeds to step S24. In step S24, the co-starring video generated in step S23 is distributed. This co-starring video is distributed to the viewing user device 10 and other viewing user devices and broadcasting user devices via the network 50. The co-starring video includes, for example, an animation of the character object 131 of the broadcasting user Ua and an animation of the character object 31 of the broadcasting user Ub, as shown in Figs. 11 and 12a to 12b.

[0125] Next, the process proceeds to step S25. In step S25, it is determined whether the co-starring end condition is satisfied. For example, when the character object 131 of the broadcast user Ua who is the guest user is in an inactive state, when the co-starring user evaluation point becomes equal to or less than a predetermined threshold, when a co-starring end instruction is received from the broadcast user Ub who is the host user, or when a predetermined time or more has passed since the start of distribution of the co-starring video, it is determined that the co-starring end condition is satisfied. When the co-starring end condition is satisfied, the process proceeds to step S26.

[0126] In step S26, a process for ending the distribution of the co-starring video is performed. For example, the character object 131 of the broadcast user Ua, who is a guest user, is removed from the co-starring video, thereby ending the co-starring between the broadcast user Ua and the broadcast user Ub. Even if the co-starring ends, the video may continue to be distributed. The video distributed after the co-starring ends does not include the character object 131 of the broadcast user Ua.

[0127] Next, the flow of the process of displaying a matching object in one embodiment will be described with further reference to Fig. 16. In the display process of the matching object described with reference to Fig. 16, as shown in Fig. 11, a co-starring video in which the character 31 of the broadcast user Ub and the character 131 of the broadcast user Ua appear together is distributed, and it is assumed that the character 131 is not wearing the wearing object 38 that the character 31 wears. The display process of the matching object may be performed in parallel with the distribution process of the co-starring video shown in Fig. 15.

[0128] First, in step S31, a co-starring video in which the character 31 of the broadcast user Ub and the character 131 of the broadcast user Ua appear together is distributed. The generation and distribution of the co-starring video is performed, for example, by the above-mentioned co-starring application processing unit 61f and video distribution unit 61c.

[0129] Next, in step S32, it is determined whether the characters appearing together in the co-starring video are wearing the same attachment object. The determination process in step S32 is performed, for example, by the above-mentioned determination unit 61g. In the co-starring video, character 31, which is one of the characters appearing together, is wearing the attachment object 38, so it is determined whether character 131, which is the other of the characters appearing together, is wearing the attachment object 38. If character 131 is wearing the same object as attachment object 38, the display process of the matching object is terminated. If character 131 is not wearing the same object as attachment object 38 as in the example shown in FIG. 11, the process proceeds to step S33.

[0130] In step S33, it is determined whether a display request for a matching object has been made. The display request for a matching object is made, for example, by a viewing user of the co-starring video, a broadcast user Ua, or a broadcast user Ub. In step S33, a process may be performed to prompt at least one of the viewing users of the co-starring video, the broadcast user Ua, and the broadcast user Ub to transmit a display request for a matching object. In response to the process of prompting the transmission of the display request for the matching object, the viewing user of the co-starring video, the broadcast user Ua, or the broadcast user Ub can operate the viewing user device 10, the broadcast user device 20a, or the broadcast user device 20b to transmit a display request for a matching object to the server device 60. The determination of whether a display request for a matching object has been made is performed, for example, in the above-mentioned matching processing unit 61h. If a display request for a matching object has been made, the process proceeds to step S34.

[0131] In step S34, a display process is performed to display the matching object. For example, matching object 138 is displayed in association with character 131 in the co-starring video. As a result, character 31 of broadcast user Ub and character 131 of broadcast user Ua can wear the same wearable object in the co-starring video.

[0132] The display process of the matching object shown in Fig. 16 may be performed repeatedly during the distribution of the co-starring video. In the display process of the matching object, the matching object may be automatically displayed when it is determined that the characters appearing together are not wearing the same attachment object. For this reason, step S33 may be omitted.

[0133] Next, an example of the effect of the above embodiment will be described. According to the above embodiment, it is possible to make each of the characters appearing together in a co-starring video wear the same wearable object. For example, when the character 31 wears the wearable object 38 in the co-starring video, it is possible to make the character 131 wear the matching object 138 that is the same as the wearable object. This can improve the uniformity of the co-starring videos.

[0134] According to the above embodiment, the matching object is available only in the co-starring video in which the matching object is displayed. In one embodiment, the broadcasting user purchases an object (e.g., an attachment object) to be displayed in association with a character in the video and then displays the object in the video. For this reason, if an object that the broadcasting user purchased for a fee is given to another user who is a broadcasting partner in a form that the other user can freely use free of charge, fairness between users will be lacking. In the above embodiment, the matching object displayed in the co-starring video is available only in the co-starring video, and cannot be used in other videos. In this way, by limiting the scope of use of the matching object, it is possible to improve the uniformity of the co-starring videos while maintaining fairness between users of the video broadcasting system.

[0135] According to the above embodiment, a matching object can be displayed in a co-starring video based on a display request from a viewing user. Therefore, according to the above embodiment, gifting of a matching object by a viewing user can be promoted. If a viewing user does not own a matching object, the viewing user may purchase the matching object and then gift it. In this case, the purchase of the matching object is also promoted.

[0136] According to the above embodiment, a matching object can be displayed in a co-starring video based on a display request from a user of a character wearing the wearing object (for example, a broadcast user Ub). Therefore, according to the above embodiment, a user who does not own a wearing object or owns a small number of wearing objects can be given an incentive to appear in a co-starring video.

[0137] According to the above embodiment, a matching object can be displayed in a co-starring video based on a display request from a user of a character not wearing an attachment object (e.g., a broadcasting user Ua). If a user of a character not wearing an attachment object does not own the matching object at the time when the user is prompted to make a display request for the matching object, the user purchases the matching object before making the display request. Therefore, according to the above embodiment, it is possible to encourage users of characters not wearing attachment objects to purchase matching objects.

[0138] According to the above embodiment, when the matching object 138 is displayed in association with the character of the broadcast user Ua (guest user) and the broadcast user Ub (host user) who appear together in a co-starring video, a benefit is given to the broadcast user Ub who is the host user. Therefore, when distributing a co-starring video, an opportunity to obtain a benefit is provided by wearing a wearable object. Therefore, an incentive can be given to participants in the co-starring video (host user and guest user) to wear many wearable objects. This promotes the acquisition and purchase of wearable objects.

[0139] According to the embodiment described above, when the matching object 138 is displayed in association with the character of the broadcast user Ua, among the broadcast user Ua and the broadcast user Ub who appear together in the co-starring video, a privilege is granted to the broadcast user Ua who is a guest user. Therefore, since a privilege can be obtained by becoming a guest user in the co-starring video, it is possible to encourage the viewing user to apply to appear together.

[0140] According to the above embodiment, among the viewing users who are watching the co-starring video, a benefit is given to the viewing user who has gifted the matching object 138. This makes it possible to give the viewing user an incentive to gift the matching object 138.

[0141] According to the above embodiment, when the matching object 138 is displayed in association with the character of the broadcast user Ua, among the broadcast user Ua and the broadcast user Ub who appear together in the co-starring video, a bonus may be given to each of the broadcast user Ua, the broadcast user Ub, and the viewing user. This allows the bonus to be given to all users involved in the co-starring video, which in turn increases the amount of gifting and message sending after the bonus is given, thereby creating excitement in the co-starring video.

[0142] According to the embodiment described above, the matching object is worn on the same wearing part as the matching object worn by the co-star. For example, the matching object 138 is worn on the same wearing part (i.e., the head) as the wearing object 38. This can further improve the sense of unity in the co-starring videos.

[0143] In the video distribution system 1 according to the above embodiment, the broadcast users Ua and Ub can distribute videos including a character that moves according to their facial expressions using a broadcast user device 20 equipped with a camera such as a smartphone. In this way, in the video distribution system 1, the equipment for the broadcast users Ua and Ub to distribute videos is simplified, and a platform that is easy for the broadcast users Ua and Ub to participate in is realized.

[0144] In the processing procedures described in this specification, particularly in the processing procedures described using flow diagrams, it is possible to omit some of the steps that make up the processing procedures, to add steps that are not explicitly stated as steps that make up the processing procedures, and / or to change the order of the steps, and processing procedures in which such omissions, additions, or changes in order have been made are also included within the scope of the present invention as long as they do not deviate from the spirit of the present invention.

[0145] At least a part of the functions realized by the computer processor 61 may be realized by a computer processor other than the computer processor 61 of the video distribution system 1. At least a part of the functions realized by the computer processor 61 may be realized, for example, by the computer processor 21 of the broadcasting user device 20, or by a computer processor of the viewing user device 10. Specifically, some or all of the functions of the animation generation unit 61a and the video generation unit 61b may be executed in the broadcasting user device 20. For example, a video generated in the broadcasting user device 20 may be transmitted to the server device 60 and distributed from the server device 60 to the viewing user device 10.

[0146] When a broadcast user Ua who is watching a video including the character 31 of the broadcast user Ub makes a request to perform together with the broadcast user Ub, the animation of the character 131 of the broadcast user Ua who made the request to perform together may be generated by the server device 60, the broadcast user device 20a of the broadcast user Ua, the broadcast user device 20b of the broadcast user Ub, the viewing user device 10, or other devices. In one example, when the animation of the character 131 of the broadcast user Ua is generated by the broadcast user device 20b of the broadcast user Ub, face motion data and voice data of the broadcast user Ua are transmitted from the broadcast user device 20a of the broadcast user Ua to the broadcast user device 20b of the broadcast user Ub. In the broadcast user device 20b, the animation of the character 131 is generated by applying the face motion data and voice data of the broadcast user Ua received from the broadcast user device 20a to the model data for the broadcast user Ua. This model data for the broadcast user Ua may be transmitted from the server device 60 to the broadcast user device 20a. In another example, when animation of the character 131 of the broadcast user Ua is generated in the viewing user device 10, the facial motion data and voice data of the broadcast user Ua are transmitted from the broadcast user device 20a of the broadcast user Ua to the viewing user device 10, and the facial motion data and voice data of the broadcast user Ub are transmitted from the broadcast user device 20b of the broadcast user Ub to the viewing user device 10. In the viewing user device 10, animation of the character 131 may be generated by applying the facial motion data and voice data of the broadcast user Ua received from the broadcast user device 20a to the model data for the broadcast user Ua, and animation of the character 31 may be generated by applying the facial motion data and voice data of the broadcast user Ub received from the broadcast user device 20b to the model data for the broadcast user Ub. This model data for the broadcast user Ua and the model data for the broadcast user Ub may be transmitted from the server device 60 to the viewing user device 10. In this way, a co-starring video including animation of the broadcast user's character and / or animation of the character object may be generated in any device included in the video distribution system 1.The animation of the character may be generated by a device not explicitly shown in Fig. 1. To the device generating the animation of the character, facial motion data and voice data of broadcast user Ua are transmitted from broadcast user device 20a of broadcast user Ua, and facial motion data and voice data of broadcast user Ub are transmitted from broadcast user device 20b of broadcast user Ub. Body motion may be transmitted in addition to facial motion.

[0147] Furthermore, the generation of the co-starring video including the generated character 131 of broadcast user Ua and the broadcast video and character 31 of broadcast user Ub may be performed by any of the server device 60, the broadcast user device 20a of broadcast user Ua, the broadcast user device 20b of broadcast user Ub, or the viewing user device 10. The co-starring video may be generated by the server device 60, the broadcast user device 20a, the broadcast user device 20b, the viewing user device 10, and other devices in a shared manner. When a gift object is included in the co-starring video, data for displaying the gift object may be transmitted from the server device 60 or other devices to the device generating the co-starring video.

[0148] The co-starring video may include characters of three or more broadcast users. In other words, three or more broadcast users may virtually appear together in the video using their respective characters.

[0149] The inventions described in the claims of the original application of this application are set forth below. [1] one or more computer processors; The one or more computer processors execute the computer readable instructions to: Distributing a co-starring video including a first character generated based on a movement of a first user and a second character generated based on a movement of a second user; displaying a matching object, which is the same as a first object displayed in association with the first character and is available only in the co-starring video, in association with the second character in the co-starring video; Video distribution system. [2] the matching object being an object purchased by the first user; [1] A video distribution system according to the present invention. [3] determining whether a matching object identical to the first object is displayed in association with the second character; when it is determined that the matching object is not displayed in association with the second character, prompting a display request to display the matching object in association with the second character; displaying the matching object in association with the second character in the co-starring video based on the display request; A video distribution system according to [1] or [2]. [4] When it is determined that the matching object is not displayed in association with the second character, prompting one or more viewing users who are viewing the co-starring video to make a display request to display the matching object in association with the second character. [3] A video distribution system according to the present invention. [5] prompting the first user to make a display request to display the matching object in association with the second character when it is determined that the matching object is not displayed in association with the second character; [3] or [4]. A video distribution system according to the present invention. [6] when it is determined that the matching object is not displayed in association with the second character, prompting the second user to make a display request to display the matching object in association with the second character; A video distribution system according to any one of claims [3] to [5]. [7] When the matching object is displayed in association with the second character in the co-starring video, a benefit is given to the first user. A video distribution system according to any one of claims [1] to [6]. [8] giving a benefit to the second user when the matching object is displayed in association with the second character in the co-starring video; A video distribution system according to any one of claims [1] to [6]. [9] When the matching object is displayed in the co-starring video in association with the second character, a benefit is given to one or more viewing users who are viewing the co-starring video. A video distribution system according to any one of claims [1] to [6].

[10] when the matching object is displayed in response to a display request from a first viewing user among the one or more viewing users viewing the co-starring video to display the matching object in association with the second character, a privilege is given to the first viewing user. A video distribution system according to any one of claims [1] to [6].

[11] When the matching object is displayed in association with the second character in the co-starring video, a benefit is given to each of the first user, the second user, and one or more viewing users who are viewing the co-starring video. A video distribution system according to any one of claims [1] to [6].

[12] The co-starring video is generated in response to receiving a co-starring application from the second user to co-star with the first user. A video distribution system according to any one of claims [1] to

[11] .

[13] the first object is displayed in association with an attachment portion of the first character; the matching object is displayed in correspondence with the attachment portion of the second character; A video distribution system according to any one of claims [1] to

[12] .

[14] 1. A method for streaming video that is executed by one or more computer processors executing computer-readable instructions, comprising: A step of distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character; a step of displaying a matching object that is identical to the first object and that is available only in the co-starring video in association with the second character in the co-starring video; A video distribution method comprising:

[15] one or more computer processors, A step of distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character; a step of displaying a matching object that is identical to the first object and that is available only in the co-starring video in association with the second character in the co-starring video; A video distribution program that executes the following. [Explanation of symbols]

[0150] 1. Video distribution system 10 Viewing User Device 20a, 20b Distribution user device 61a Animation Generation Section 61b Video Generation Unit 61c Video Distribution Department 61d Gift request processing unit 61e Gift purchase processing section 61f Joint Applicant Processing Unit 61g Judgment part 61h Matching processing section 61i Benefits Granting Department

Claims

1. one or more computer processors; The one or more computer processors execute the computer readable instructions to: distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character based on a first display request; displaying a matching object identical to the first object in the co-starring video in association with the second character based on a second display request; Video distribution system.

2. determining whether a matching object identical to the first object is displayed in association with the second character; when it is determined that the matching object is not displayed in association with the second character, prompting one or more viewing users who are viewing the co-starring video to make the second display request; The video distribution system according to claim 1 .

3. determining whether a matching object identical to the first object is displayed in association with the second character; prompting the first user to make the second display request when it is determined that the matching object is not displayed in association with the second character; The video distribution system according to claim 1 .

4. determining whether a matching object identical to the first object is displayed in association with the second character; prompting the second user to make the second display request when it is determined that the matching object is not displayed in association with the second character; The video distribution system according to claim 1 .

5. the matching object being an object purchased by the first user; The video distribution system according to any one of claims 1 to 4.

6. the matching object is an object purchased by the second user; The video distribution system according to any one of claims 1 to 4.

7. The matching object is an object purchased by the one or more viewing users. The video distribution system according to claim 2.

8. the second display request is different from the first display request; A video distribution system according to any one of claims 1 to 7.

9. When the matching object is displayed in association with the second character in the co-starring video, a benefit is given to the first user. The video distribution system according to any one of claims 1 to 8.

10. providing a benefit to the second user when the matching object is displayed in association with the second character in the co-starring video; The video distribution system according to any one of claims 1 to 8.

11. When the matching object is displayed in the co-starring video in association with the second character, a benefit is given to one or more viewing users who are viewing the co-starring video. The video distribution system according to any one of claims 1 to 7.

12. when the matching object is displayed in response to a display request from a first viewing user among one or more viewing users viewing the co-starring video to display the matching object in association with the second character, a privilege is given to the first viewing user; The video distribution system according to any one of claims 1 to 8.

13. When the matching object is displayed in association with the second character in the co-starring video, a benefit is given to each of the first user, the second user, and one or more viewing users who are viewing the co-starring video. The video distribution system according to any one of claims 1 to 8.

14. The co-starring video is generated in response to receiving a co-starring application from the second user to co-star with the first user. The video distribution system according to any one of claims 1 to 13.

15. the first object is displayed in association with an attachment portion of the first character; the matching object is displayed in correspondence with the attachment portion of the second character; The video distribution system according to any one of claims 1 to 14.

16. 1. A method for streaming video that is executed by one or more computer processors executing computer-readable instructions, comprising: a step of distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character based on a first display request; displaying a matching object identical to the first object in the co-starring video in association with the second character based on a second display request; A video distribution method comprising:

17. one or more computer processors, a step of distributing a co-starring video including a first character generated based on a movement of a first user, a second character generated based on a movement of a second user, and a first object displayed in association with the first character based on a first display request; displaying, in the co-starring video, a matching object identical to the first object in correspondence with the second character based on a second display request; A video distribution program that executes the following:

Images