Program, Information Processing Method, and Information Processing System

Abstract

To increase opportunities for viewing users viewing a video to participate in distribution of the video, and realize smooth progress of the video.SOLUTION: A user device 12 causes execution of: a first output control process for outputting to a display a video including a first avatar object corresponding to a distributing user; a receiving process for receiving a selection operation by a viewing user for selecting a gift object; a gift transmitting process for transmitting, to a server, gift identification information of the selected gift object and information on a second avatar object corresponding to a viewing user; and a second output control process for obtaining data for outputting a video including the selected gift object and the first and second avatar objects, and then outputting the video based on the data to the display. The second output control process draws the second avatar object using the viewing user's tracking data and / or selected motion data and automatic motion data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a program, an information processing method, and Information processing system related to

Background Art

[0002] Conventionally, an information processing system that distributes a video including animation of an avatar object based on the movement of a user has been known (see, for example, Patent Document 1). In this system, a video including two characters is generated in real time. Also, the voices of voice actors are synthesized in real time with the video and distributed through a network.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in an information processing system that distributes a video, for the purpose of increasing the number of viewing users, or increasing the cumulative viewing time or the number of viewing times obtained by accumulating the viewing time, it is desired to increase the opportunity for viewing users to participate in the video being distributed. Specifically, it has been considered to make an avatar object corresponding to a viewing user appear in the video.

[0005] However, when rendering a video that includes a plurality of avatar objects, the actions intended by the user may not be rendered. In particular, when the avatar object corresponding to the viewing user performs an action related to the avatar object corresponding to the distributing user, the progress of the video may be hindered due to the actions of the viewing user not being performed smoothly. For example, when avatar objects shake hands with each other, the timing or position at which each avatar object extends its hand may be different, and the handshake action may not be successfully performed. As a result, the progress of the video may be hindered.

Means for Solving the Problem

[0006] In one aspect, a program is provided. The program causes one or more computers to execute a first output control process for outputting to a display a video including a first avatar object corresponding to a distributing user, a reception process for receiving a selection operation for selecting a gift object by a viewing user who views the video, a gift transmission process for transmitting to a server the gift identification information of the selected gift object and the identification information of the viewing user together with a request for output of the gift object, and a second output control process for outputting to the display the video based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user. In the second output control process, the second avatar object is rendered using at least one of the tracking data of the viewing user and the selection motion data selected by the viewing user, and preset automatic motion data.

[0007] In another aspect, an information processing method is provided. The information processing method includes: a first output control process in which one or more computers output a video including a first avatar object corresponding to a distribution user to a display; a reception process for receiving a selection operation in which a viewing user who views the video selects a gift object; a gift transmission process for transmitting gift identification information of the selected gift object and identification information of the viewing user to a server together with an output request for the gift object; and a second output control process for outputting the video to the display based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user. In the second output control process, the second avatar object is drawn using at least one of tracking data of the viewing user and selection motion data selected by the viewing user and preset automatic motion data.

[0008] In another aspect, an information processing method is provided. The information processing method includes: a first transmission process in which one or more computers transmit data for outputting a video including a first avatar object corresponding to a distribution user to at least a viewing user device used by the viewing user who views the video; a gift reception process in which the gift identification information of the gift object and the identification information of the first viewing user are received from the first viewing user device used by the first viewing user, together with a gift output request; and a second transmission process in which the gift identification information, an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user are transmitted to at least a distribution user device used by the distribution user and a second viewing user device used by a second viewing user other than the first viewing user. The output request transmitted in the second transmission process is transmitted together with data used for rendering the second avatar object or identification information of the data. The data used for rendering the second avatar object includes at least one of the tracking data of the first viewing user and the selected motion data selected by the first viewing user, and preset automatic motion data.

[0009] In another aspect, an information processing apparatus is provided. The information processing apparatus includes a first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a distribution user to at least a viewing user apparatus used by a viewing user who views the video, a gift reception unit that receives, from a first viewing user apparatus used by a first viewing user who is the viewing user, gift identification information of a gift object and identification information of the first viewing user together with an output request for the gift object, and a second transmission unit that transmits the gift identification information, an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user to at least a distribution user apparatus used by the distribution user and a second viewing user apparatus used by a second viewing user other than the first viewing user. The output request transmitted by the second transmission unit is transmitted together with data used for rendering the second avatar object or identification information of the data. The data used for rendering the second avatar object includes at least one of tracking data of the first viewing user and selection motion data selected by the first viewing user, and preset automatic motion data.

Advantages of the Invention

[0010] According to the present invention, it is possible to increase the opportunity for a viewing user who views a video to participate in video distribution and to smoothly progress the video.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0012] (First Embodiment) Hereinafter, a first embodiment of a program, an information processing method, and an information processing apparatus will be described.

[0013] <Information Processing System> Referring to FIG. 1, the information processing system 11 will be described. The information processing system 11 includes a user device 12 and a server 13. The user device 12 and the server 13 transmit and receive data via a network 14. The user device 12 receives data necessary for video distribution and viewing from the server 13 based on a user's input operation.

[0014] A user who distributes a video using the user device 12 is called a distributing user, and a user who views the distributed video is called a viewing user. A user can be both a distributing user and a viewing user. That is, when a user views a video, the user is a viewing user, and when the user distributes a video, the user is a distributing user. Also, the user device 12 used by the distributing user is referred to as the distributing user device 12A, and the user device 12 used by the viewing user is referred to as the viewing user device 12B. When not distinguishing between the distributing user and the viewing user, and between the distributing user device 12A and the viewing user device 12B, they are simply referred to as the user and the user device 12. There is one or more distributing user devices 12A. There is one or more viewing user devices 12B.

[0015] The user device 12 is a smartphone (multifunctional phone terminal), a tablet terminal, a personal computer, a console game machine, a wearable computer, or an information processing device capable of playing videos other than these devices. The wearable computer only needs to be equipped with a screen on which the user can view videos. For example, it can be a head-mounted display worn on the user's head, or a glasses-type wearable terminal, etc. The user device 12 may be a device that operates independently, or may be composed of a plurality of devices connected so as to be able to transmit and receive various data to and from each other. As an example of the latter user device 12, there is, for example, a system that performs outside-in tracking.

[0016] The user device 12 includes a control unit 20, a storage 22 (storage medium), and a communication interface (I / F) 23. The control unit 20 includes one or more arithmetic circuits such as a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural network Processing Unit). Further, the control unit 20 includes a memory which is a main storage device (storage medium) that can be read from and written to by the arithmetic circuit. The memory is composed of a semiconductor memory or the like. The control unit 20 reads an operating system and other programs from the storage 22 or an external storage into the memory and executes the instructions fetched from the memory. The communication I / F 23 can transmit and receive data to and from the server 13 and other user devices 12 via the network 14. The network 14 includes various networks such as a local area network and the Internet. The control unit 20 executes the first output control process and the second output control process of the claims.

[0017] The storage 22 is an auxiliary storage device (storage medium), for example, a storage medium such as a magnetic disk, an optical disk, or a semiconductor memory. The storage 22 can also be used by combining a plurality of storages.

[0018] Based on an input operation on the user's operation unit 27, the control unit 20 executes various programs recorded in the storage 22 to execute each process related to video distribution and each process related to viewing. Hereinafter, for convenience of explanation, the state of distributing video is referred to as the distribution mode, and the state of viewing a video distributed by another distributing user is referred to as the viewing mode.

[0019] Further, the user device 12 includes a sensor unit 24, a speaker 25, a microphone 26, an operation unit 27, and a display 28. At least one of the sensor unit 24, the speaker 25, the microphone 26, the operation unit 27, and the display 28 may be provided in the same device as the control unit 20, or may be provided in a manner of being connected to the device in which the control unit 20 is provided.

[0020] The sensor unit 24 is one or more sensors that detect face motion indicating changes in the user's expression and body motion indicating changes in the relative position of the user's body with respect to the sensor unit 24. Face motion includes movements such as blinking and opening and closing of the mouth. A known device can be used as the sensor unit 24. An example of the sensor unit 24 is a ToF sensor that measures the time of flight until light irradiated toward the user is reflected back from the user's face or the like, a camera that captures the user's face, and an image processing unit that processes the data captured by the camera. Further, the sensor unit 24 may include an RGB camera that images visible light and a near-infrared camera that images near-infrared light. The RGB camera and the near-infrared camera can use, for example, "True Depth" of "iPhone X (registered trademark)", "LIDAR" (Light Detection and Ranging, or Laser Imaging Detection and Ranging) such as "iPad Pro (registered trademark)", or other ToF sensors mounted on a smartphone. Specifically, this camera projects tens of thousands of invisible dots (points) onto the user's face or the like by a dot projector. Then, the reflected light of the dot pattern is detected and analyzed to form a depth map of the face, and an infrared image of the face or the like is captured to capture accurate face data. The arithmetic processing unit of the sensor unit 24 generates various types of information based on the depth map and the infrared image, compares the information with registered reference data, and calculates the depth of each point on the face (the distance between each point and the near-infrared camera) and the displacement at positions other than the depth.

[0021] In addition, the sensor unit 24 may have a function of tracking (hand tracking) not only the user's face but also the hand. Further, the sensor unit 24 may include a sensor that detects the position or orientation of a human body part other than the hand. The sensor unit 24 may further include sensors such as a sensor that detects speed or acceleration, such as an acceleration sensor, and a sensor that detects direction or orientation, such as a gyro sensor. The sensor unit 24 may have a spatial mapping function of recognizing an object in the real space where the user exists based on the detection results of the above ToF sensor or other known sensors and mapping the recognized object to a spatial map. Hereinafter, when explaining without particularly distinguishing the detection data of face motion and the detection data of body motion, it is simply referred to as "tracking data". Note that the image processing unit of the sensor unit 24 may be provided in the control unit 20.

[0022] The speaker 25 converts voice data into voice and outputs it. The microphone 26 inputs the voice spoken by the user and converts it into voice data. The display 28 outputs various images in response to an output instruction from the control unit 20.

[0023] The operation unit 27 can be selected according to the type of the user device 12. An example of the operation unit 27 is a touch panel integrated with the display 28. Another example of the operation unit 27 is an operation button, a keyboard, a mouse, a controller that can be operated by hand by the user, etc., provided on the housing of the user device 12 or the like. The controller may incorporate various known sensors such as inertial measurement sensors (IMU: Inertial Measurement Unit) such as an acceleration sensor and a gyro. Another example of the operation unit 27 may be a tracking device that identifies the movement of the user's hand, eyes, head, the direction of the line of sight, etc. In this mode, for example, based on the movement of the user's hand, the user's instruction is determined, and various operations such as starting or ending the distribution of the video, evaluating the message or video, and outputting a predetermined object (for example, a gift described later) can be executed. When the sensor unit 24 also has an input interface function such as a hand tracking function, the operation unit 27 can be omitted.

[0024] <Server> Next, the server 13 will be described. The server 13 is used by a service provider or the like that provides a service for distributing videos. The server 13 includes a control unit 30, a communication I / F 34, and a storage 35. The control unit 30 has one or more arithmetic circuits. The control unit 30 has the same configuration as the control unit 20 of the user device 12. Also, the communication I / F 34 and the storage 35 have the same configuration as those of the user device 12. The server 13 may be composed of one device or may be composed of a plurality of devices. When the server 13 is composed of a plurality of devices, those devices distribute and provide the functions of the control unit 30 and various data stored in the storage 35. The control unit 30 corresponds to the first transmission unit, the gift reception unit, and the second transmission unit in the claims.

[0025] <Video distribution method> As the video distribution method, one of the client rendering method, the browser rendering method, the video distribution method, and the server distribution method can be used.

[0026] The client rendering method is one in which the distribution user device 12A and the viewing user device 12B receive video composition data necessary for video generation from the server 13 and draw the video. The video composition data includes tracking data detected by the sensor unit 24 of the distribution user device 12A. Also, each user device 12 acquires voice data based on the speech of the distribution user and outputs the voice in synchronization with the video. In this case, various data such as an application program and rendering data for drawing an avatar are stored in the storage 22 of each user device 12.

[0027] The browser rendering method is a method in which the distribution user device 12A and the viewing user device 12B draw a video using a web browser program stored in the storage 22. At this time, the distribution user device 12A transmits tracking data and the like to the server 13. The user device 12 downloads a web page described in a markup language such as HTML (Hyper Text Markup Language) and using CSS (Cascading Style Sheets), JavaScript (registered trademark), etc. from the server 13. The web browser program started on the user device 12 draws an avatar object and other objects by JavaScript operating on the browser. In the browser rendering method, the user device 12 stores the data received from the server 13 in the volatile memory and stores data such as the URL of the web page for outputting the video in the non-volatile memory. For this reason, the rendering data of the avatar object is not stored in the user device 12.

[0028] The video distribution method is a method in which the distribution user device 12A generates video data. The viewing user device 12B outputs a video to the display 28 based on the video data received from the distribution user device 12A. Specifically, the distribution user device 12A generates video data using tracking data and the like. Also, the distribution user device 12A transmits the encoded video data and audio data to the server 13. The server 13 transmits the video data and audio data to the viewing user device 12B. The viewing user device 12B outputs a video based on the received video data and audio data.

[0029] The server distribution method is a method in which the server 13 generates video data. The server 13 receives tracking data and audio data from the distribution user device 12A. Then, the server 13 generates video data based on those data. The server 13 transmits the generated video data to the distribution user device 12A and the viewing user device 12B. The distribution user device 12A and the viewing user device 12B output a video to the display 28 based on the received video data.

[0030] In this way, either the user device 12 or the server 13 is responsible for generating the video data. The device responsible for generating the video data can be changed according to the distribution method. The "data for outputting a video" described in the claims corresponds to each data transmitted and received between the user device 12 and the server 13 in each of the above distribution methods. When the user device 12 is responsible for generating the video data, the viewing user device 12B may generate the video data, or the distribution user device 12A may generate the video data. Also, if necessary, the user device 12 transmits the generated video data to another user device 12 as the transmission destination.

[0031] Note that the distribution user device 12A and the viewing user device 12B may output videos in different formats. For example, the distribution user device 12A may output a video in the client rendering format, and the viewing user device 12B may output a video in the browser rendering format. Also, each viewing user device 12B may output a video based on video data distributed in different distribution formats.

[0032] In this embodiment, a method for each user device 12 to distribute a video in the client rendering format will be described. In the client rendering format, the user device 12 stores a video application program in the storage 22. Also, the storage 22 stores three-dimensional model data of an avatar object and other objects used for executing the video application program.

[0033] The control unit 20 of the distribution user device 12A transmits video composition data including tracking data acquired from the sensor unit 24 and audio data based on the sound collected by the microphone 26 to the server 13. The distribution user can set the parts of the avatar object corresponding to himself / herself to the preferred parts. The video composition data includes identification information (part ID) indicating the parts of the avatar object. Also, the distribution user can adjust the position of the virtual camera in the virtual space. The video composition data may include the position of the virtual camera. The virtual camera is set at a predetermined position in the virtual space. The tracking data and the audio data are provided with time stamps. The time stamp is generated based on reference time information based on a circuit (internal clock) having a timekeeping function of the control unit 20. Or the time stamp is generated based on time information transmitted from the server 13 or reference time information acquired from a server 13 providing a time distribution service. The time stamp may represent the elapsed time from the reference time or the time.

[0034] The control unit 20 of the viewing user device 12B stores the drawing data and drawing program for objects including each avatar object in the storage 22. Further, the control unit 20 receives identification information such as part IDs that make up the avatar object, data for operating the avatar object such as tracking data, and audio data from the server 13. The control unit 20 acquires the drawing data for objects including the avatar object already stored according to the received identification information of the avatar object. Then, the control unit 20 performs rendering using the acquired drawing data and the data for operating the avatar object, and outputs a video including the avatar object to the display 28. Also, the control unit 20 outputs audio from the speaker 25 in synchronization with the video.

[0035] Also, the control unit 20 of the distribution user device 12A and the control unit 20 of the viewing user device 12B perform rendering including avatar objects and objects other than avatar objects. The rendering here refers to a drawing process including acquisition of the position of the virtual camera, perspective projection, hidden surface elimination based on depth information associated with each object, etc. The rendering may be at least one of these processes, or may include processes such as shading and texture mapping.

[0036] The distribution user device 12A and the viewing user device 12B receive data such as posted messages or notifications from the viewing user from the server 13. The control unit 20 outputs video data obtained by synthesizing an image of the avatar object or other objects with the posted message or notification to the display 28. Also, the control unit 20 outputs the audio data in synchronization with the video data based on the time stamp from the speaker 25.

[0037] Furthermore, the control unit 20 transmits an output request for a gift object (gift output request), posts a message to the video, and evaluates the video (selection of a "like!" button or a "favorite" button, etc.) based on the operation of the viewing user.

[0038] <Configuration of Data Used for Video> Next, with reference to FIGS. 2 to 5, each data used for video distribution or viewing will be described in detail.

[0039] FIG. 2 shows an example of user management data 50. The user management data 50 is stored in the server 13. The user management data 50 includes user ID which is identification information of the user, data of coins, and points. Coins are media that can be purchased by the user. Points are media that can be obtained by actions such as viewing videos and distributing videos. In addition to this, the user management data 50 may include scores accumulated by participating in events. Coins, points, and scores can be used within the video distribution service. Note that the user device 12 may store the user management data 50.

[0040] FIG. 3 is an example of main body part data 51 of an avatar object. The main body part data 51 is stored in the storage 22 of the user device 12. The body parts will be described. The body parts are the parts that make up the parts of the avatar object. The data for drawing the body parts includes polygon data, skeleton data (bones) for expressing the movement of the avatar object, texture data, and the like. The skeleton data and texture data are stored in advance in the user device 12 at a predetermined timing such as when the application program is installed or when the application program is started. In addition, avatar attributes are associated with the body parts. Avatar attributes are, for example, types such as male type like "M type" and female type like "F type", height, skin color, body type, and the like. Depending on the avatar attributes, the skeleton data, polygon data, etc. of the avatar object are different. By the user selecting the avatar attributes, the body parts such as the upper body and the lower body are selected. Also, some of these body parts can be arbitrarily set by the user. For example, the user can select parts such as eyes, mouth, hairstyle, etc. Each user can register one or more avatar objects as avatars that the user can use. In this embodiment, it is possible to register three avatar objects.

[0041] The body part data 51 includes information such as the slot number, type, height, skin color, body type, and other body part information corresponding to the user's avatar object. The body part data 51 has various information for each slot number of the avatar. The type is the type of the avatar attribute selected by the user. The height, skin color, and body type are the identification information of the height, skin color, and body type selected by the user. The part is the ID of an individual part such as eyes, mouth, and hairstyle. Part IDs are assigned to each body part such as the head, torso, arms, legs, etc. that make up the avatar object.

[0042] FIG. 4 shows an example of the attachment part data 52. The attachment part data 52 is stored in the user device 12. Also, the attachment part data 52 may be stored in the server 13. The wearable part data 52 is data regarding wearable parts available to the user. The wearable parts can be those prepared in advance in the initial state when the use of the application is started, those that the user can exchange with a medium such as a coin, or those that can be obtained by executing a lottery (gacha). The wearable parts will be described. The wearable parts are parts used for a predetermined part of the avatar object. The wearable parts include texture data. The wearable parts may include skeleton data for expressing the sway accompanying the movement of the avatar object. For example, the wearable parts of clothes may include skeleton data. The texture data and the skeleton data are stored in the user device 12. The wearable parts are, for example, "tops" used for the upper body, "shoes" used for the feet, "skirts" used for the lower body, etc. The user can select a preferred wearable part from a plurality of wearable parts according to the avatar attributes of the avatar object. That is, the viewing user can use wearable parts for each avatar object with slot numbers "1" to "3".

[0043] Attribute data indicating the part of the avatar object is associated with the wearable parts. Note that the wearable parts may be those associated with a predetermined part of the avatar object, and may be other than the clothes of the avatar object, such as "cups", "glasses", "umbrellas", "musical instruments", etc. For example, a "cup" is associated with the hand part of the avatar object. By associating the "cup" part with the hand, an avatar object holding the "cup" can be output. The wearable parts include standard parts stored when the application program is installed, and parts given by the server 13 when the user exchanges with coins or participates in events.

[0044] The worn part data 52 includes a part ID, an item name, a body part, an attribute, a color, and a slot number of the avatar object to which the worn part is applied. The part ID is the identification information (ID) of the selected worn part. The item name is the name of the worn part. For example, the name of an item such as "beginner T-shirt" is stored. The body part is the body part of the avatar object to which the worn part is applied. The attribute is the attribute of the avatar to which the worn part is applied, and all attributes (e.g., "All"), "M type", or "F type" are set. The color indicates the color of the worn part. The slot number indicates the slot number of the avatar object to which the worn part is applied among the avatar objects registered by the user. Note that one worn part may be usable for the avatar objects in each slot.

[0045] In addition to the worn part data 52, the user device 12 may store non-worn part data. The non-worn parts are parts other than the worn parts. For example, the non-worn parts are objects such as wallpapers that are output as a background. The non-worn part data includes a part ID, an item name, and a color for each part. Note that the non-worn parts may be stored in the user device 12 in association with the target avatar object.

[0046] FIG. 5 shows an example of the gift data 53. The gift data 53 is stored at least in the server 13. The gift data 53 is data regarding a gift object. The gift object is an object given from a viewing user to a distributing user. Note that the gift object includes not only an object arranged in the virtual space but also an object that performs an effect in the virtual space such as fireworks or snow. The gift object is exchanged for the coins or points of the viewing user and output to the video.

[0047] The gift data 53 includes a gift ID, the name of the gift object, the number of coins, and the number of points. The gift ID is the identification information of the gift object. The name is the name of the gift object such as "rabbit stuffed toy". The number of coins indicates the number of coins exchanged for the gift object. The number of points indicates the number of points exchanged for the gift object.

[0048] (Operation) Next, the operations of the user device 12 and the server 13 will be described. In the viewing mode, the control unit 20 of the user device 12 executes a first output control process, a reception process, a gift transmission process, and an output control process.

[0049] In the first output control process, a video including the first avatar object corresponding to the distribution user is output to the display. In the reception process, a selection operation for selecting a gift object by the viewing user who views the video is received.

[0050] In the gift transmission process, the gift ID (gift identification information) of the selected gift object and the user ID of the viewing user are transmitted to the server 13 together with an output request for the gift object. When the viewing user device 12B transmits an output request for the gift object or at a different timing, information regarding the second avatar object corresponding to the viewing user is transmitted. The information regarding the second avatar object includes the part ID of the main body of the second avatar object, the part ID of the attached parts, etc. Note that the viewing user corresponding to the second avatar object is referred to as the first viewing user, and a viewing user other than the first viewing user is referred to as the second viewing user. When explaining without particularly distinguishing between the first viewing user and the second viewing user, for convenience, they are simply referred to as viewing users.

[0051] In the second output control process, a video based on data for outputting a video including the selected gift object, the first avatar object, and the second avatar object is output to the display 28. The second output control process may be executed when the reception process receives a selection operation for selecting a gift object by the first viewing user.

[0052] Also, in the second output control process, the second avatar object is drawn using the selection motion data selected by the first viewing user from among the tracking data of the first viewing user or a plurality of preset motion data, and the automatic motion data automatically used for the second avatar object.

[0053] The viewing user device 12B of the first viewing user acquires the tracking data of the viewing user from the sensor unit 24. For this reason, while this viewing user device 12B transmits the tracking data to the server 13, the second avatar object is drawn using the tracking data. Note that the viewing user device 12B of the first viewing user may receive the tracking data transmitted by itself or data obtained by processing the tracking data from the server 13, and draw the second avatar object using the received tracking data.

[0054] The control unit 30 of the server 13 executes a first transmission process, a gift reception process, and a second transmission process. In the first transmission process, data for outputting a video including the first avatar object corresponding to the distribution user is transmitted to at least the viewing user device 12B. That is, the server 13 does not necessarily transmit the tracking data and audio data of the distribution user to the distribution user device 12A. The server 13 transmits messages, gift output requests, etc. sent by the viewing user device 12B to the distribution user device 12A.

[0055] In the gift reception process, the gift ID of the gift object and the user ID of the viewing user are received from the viewing user device 12B (first viewing user device) used by the first viewing user, together with the output request of the gift object.

[0056] In the second transmission process, at least the distribution user device 12A and the viewing user device 12B used by the second viewing user other than the first viewing user are transmitted with the gift ID and an output request for outputting a moving image including the first avatar object and the second avatar object corresponding to the first viewing user.

[0057] The output request transmitted in the second transmission process is transmitted to the viewing user device 12B used by the second viewing user other than at least the distribution user device 12A and the first viewing user, together with the data used for drawing the second avatar object or the identification information of the data.

[0058] Specifically, the output request transmitted in the second transmission process may specify the data representing the operation of the second avatar object 150 and include a request for drawing using the specified data. Further, the data used for drawing the second avatar object may include at least one of the tracking data of the first viewing user and the selected motion data selected by the first viewing user, and the preset automatic motion data. Or, the data used for drawing the second avatar object may include at least one of the identification information of the tracking data of the first viewing user and the selected motion data selected by the first viewing user, and the identification information of the preset automatic motion data. The identification information of the data used for drawing the second avatar object may be identification information such as the part ID constituting the avatar object. When the first viewing user selects the selected motion data, the viewing user device 12B used by the first viewing user transmits the selected selected motion data or its identification information to the server 13.

[0059] Server 13 transmits the data that each user device 12 lacks. For example, the distribution user device 12A acquires the tracking data of the distribution user by the sensor unit 24. Therefore, the server 13 may not need to transmit the tracking data for rendering the first avatar object to the distribution user device 12A. Also, the viewing user device 12B of the first viewing user acquires the tracking data of the first viewing user by the sensor unit 24. Therefore, the server 13 may not need to transmit the tracking data for rendering the second avatar object to the viewing user device 12B used by the first viewing user.

[0060] Server 13 may not need to transmit the selection motion data to the user device 12 that stores the selection motion data. At the timing of using the selection motion data, server 13 transmits the identification information (ID) of the selection motion data selected by the first viewing user together with the instruction to use the selection motion to all user devices 12 that store the selection motion data, or to the distribution user device 12A and the viewing user device 12B of the second viewing user among the user devices 12 that store the selection motion data. The user device 12 that receives the identification information of the selection motion data acquires the motion data corresponding to the ID of this selection motion data from the storage 22. Also, this user device 12 renders the second avatar object using the acquired motion data.

[0061] Similarly, the server 13 may not transmit the automatic motion data to the user device 12 that has stored the automatic motion. At the timing of using the automatic motion data, the server 13 transmits the identification information (ID) of the automatic motion data together with the instruction to use the automatic motion to all the user devices 12 that have stored the automatic motion data, or to the distribution user device 12A and the viewing user device 12B of the second viewing user among the user devices 12 that have stored the automatic motion data. The user device 12 that has received the identification information of the automatic motion data acquires the motion data corresponding to the ID of this automatic motion data from the storage 22. Further, this user device 12 draws the second avatar object using the acquired motion data.

[0062] In the present embodiment, in the scene where the user device 12 outputs the gift object, the user device 12 draws the second avatar object 150 using the following (1) and (2). (1) The tracking data of the first viewing user who has transmitted the output request of the gift object. (2) Preset automatic motion data. As described above, this automatic motion data may be stored in each user device 12. Alternatively, the automatic motion data may be stored in the server 13 and transmitted to each user device 12 at the timing of use. There may be motion data used for the second avatar object and motion data used for the first avatar object in the automatic motion data. Further, the automatic motion data may be linked to the gift ID. Further, the automatic motion data may be associated with an attribute that is the type of the gift object.

[0063] The automatic motion data has a pre-specified usage part (usage area). That is, the automatic motion data is associated with the main body part ID of the avatar object. For example, the automatic motion data is data that causes the second avatar object to perform an action of stretching the arm part forward. For example, the automatic motion data is data that causes an action of associating a gift object associated with the hand part of the second avatar object with the hand part of the first avatar object.

[0064] Also, in a scene where a gift object is output, automatic motion data is also used for the first avatar object. The automatic motion data used for the first avatar object is data that causes an action of stretching the arm. By using the automatic motion data for the first avatar object and the second avatar object, an action of the first avatar object receiving the gift object held by the second avatar object can be drawn.

[0065] When a scene where the first avatar object receives the gift object held by the second avatar object is drawn using the tracking data of the first viewing user and the tracking data of the distributing user, the first viewing user may be confused about the action. In this case, the action of the second avatar object stops. Also, the position where the second avatar object stretches its hand may not match the position where the first avatar object stretches its hand, and there is a possibility that the transfer of the gift object cannot be properly drawn. In such a case, the progress of the video is hindered. On the other hand, by drawing a scene that is difficult to draw only with the tracking data as described above using the automatic motion data, the progress of the video can be made smooth.

[0066] Next, referring to FIG. 6, the processing procedure in the scenario where the server 13 outputs a gift object will be described. Hereinafter, a request to output a gift object is referred to as a gift output request. Also, in the following procedure, the description will focus on the drawing of the second avatar object. It is assumed that the first avatar object uses the tracking data of the distribution user except in the scenario of passing the gift object. Note that in this embodiment, basically, the second avatar object is not drawn in the video except in the scenario of displaying the gift object.

[0067] The server 13 determines whether it has received a gift output request from the viewing user device 12B used by the first viewing user (step S11). At this time, the server 13 receives the user ID of the first viewing user and the gift ID of the gift object together with the gift output request. Also, the server 13 has received from the viewing user device 12B of the first viewing user the part ID of the main body part constituting the avatar object of the first viewing user, the part ID of the attached part, etc., simultaneously with or before the gift output request. Note that when the server 13 receives a request to output the gift object 135, media such as coins or points may be given to the distribution user. Thereby, a distribution user who has received many gift objects 135 from viewing users can obtain more media such as coins or points.

[0068] If the server 13 has not received a gift output request (step S11: NO), it proceeds to step S16. When the server 13 determines that it has received a gift output request (step S11: YES), it determines whether it is the timing to output the gift object (step S12).

[0069] Here, the server 13 determines the timing to output the gift object according to the attributes (types) of the gift object. When a gift object such as a stuffed toy, for which the output timing is not particularly limited, is given, the server 13 determines that the gift object can be output when it receives a gift output request. In the case of a wearable object, it notifies the distribution user device 12A that a gift object which is a wearable object has been given. Further, when the distribution user performs an operation to permit the output of the wearable object, it determines that the gift object can be output. Also, there may be restrictions on the output of the gift object, such as when the display position of the gift object is determined. For example, gift objects such as "hanging scroll" and "tower cake" are displayed at a predetermined display position for a certain period, but a new gift object cannot be output unless the gift object already displayed at the display position is erased. In such a case, when the display position associated with the gift object becomes available, it determines that the gift object can be output.

[0070] If the server 13 determines that it is not the timing to output the gift object (step S12: NO), it waits for the output timing of the gift object. If the server 13 determines that it is the timing to output the gift object (step S12: YES), it determines whether or not the first viewing user who gave the gift object is viewing the video (step S13). At this time, the server 13 determines whether or not the first viewing user is viewing the video based on information indicating that the connection to the video has been disconnected from the viewing user device 12B used by the first viewing user, or receiving an error code for the information sent by the server 13.

[0071] When the server 13 determines that the first viewer who has given the gift object is watching the video (step S13: YES), it transmits a request to output the second avatar object and the gift object, together with the gift ID, to the distribution user device 12A, the viewer device 12B of the first viewer, and the viewer device 12B of the second viewer (step S14). For example, the server 13 transmits at least the information necessary to draw the second avatar object to the distribution user device 12A and the viewer device 12B of the second viewer. The information necessary to draw the second avatar object may include the main part ID and the attachment part ID of the second avatar object.

[0072] In addition, the server 13 transmits at least to the distribution user device 12A and the viewer device 12B of the second viewer the tracking data received from the viewer device 12B used by the first viewer, together with the output request, and the automatic motion data or the ID of the automatic motion data. At this time, the server 13 may transmit the gift ID and the output request to the viewer device 12B of the first viewer.

[0073] When the distribution user device 12A and the viewer device 12B of the second viewer receive the gift ID, the part ID of the main part of the second avatar object, and the part ID of the attachment part together with the gift output request, they create the gift object corresponding to the gift ID and draw the image of the second avatar object. At this time, the second avatar object is drawn using the automatic motion data. Also, the gift object is output in association with the second avatar object.

[0074] For example, draw a second avatar object in which a gift object of "bouquet" is associated with a hand part. Also, use the tracking data for a part of the second avatar object. For example, use the face tracking data of the first viewing user to draw the face of the second avatar object and reflect the expression of the first viewing user in the second avatar object. The timing of using the automatic motion data and the timing of using the tracking data are not particularly limited. After drawing the second avatar object using the automatic motion data, the tracking data may be used. Or after drawing the second avatar object using the tracking data, the automatic motion data may be used. Or, as described above, while using the automatic motion data for a predetermined part of the second avatar object, the tracking data may be used for parts of the second avatar object other than the predetermined part.

[0075] When the viewing user device 12B of the first viewing user receives a gift output request from the server 13 or receives an operation to give a gift object, it draws a gift object corresponding to the gift ID selected by the first viewing user. Also, the viewing user device 12B of the first viewing user outputs an image of the second avatar object. At this time, the viewing user device 12B of the first viewing user outputs a gift object in association with the second avatar object based on the automatic motion data.

[0076] At this time, each user device 12 also uses the automatic motion data for the first avatar object. For example, a gift object of "bouquet" held by the second avatar object is drawn as the first avatar object associated with the hand part so that the first avatar object receives it. Further, the user device 12 may use the tracking data of the distribution user for a part of the first avatar object. For example, the face of the first avatar object is drawn using the face tracking data of the distribution user, and the expression of the distribution user is reflected in the first avatar object. In this case, the server 13 receives the tracking data from the distribution user device 12A and transmits it to each viewing user device 12B.

[0077] On the other hand, when the server 13 determines that the first viewing user who gave the gift object is not viewing the video (step S13: NO), the server 13 outputs the gift object without outputting the second avatar object (step S15). Specifically, the server 13 transmits the gift ID to the user device 12. When the user device 12 receives the gift ID, it draws the gift object corresponding to the gift ID.

[0078] That is, there may be a slight deviation between the timing when the viewing user device 12B used by the first viewing user transmits the gift output request and the timing when the gift object is output. Also, depending on the type of the gift object, it may take time until the gift object is output. And there is a possibility that the first viewing user ends the viewing before the gift object is output after the viewing user device 12B transmits the gift output request.

[0079] If the second avatar object is output even though the first viewing user has finished viewing the video before the output of the gift object, it is difficult to use the tracking data of the first viewing user for the second avatar object. Also, if there is a user who recognizes that the first viewing user has finished viewing, they will feel uncomfortable that the second avatar object corresponding to the first viewing user is output. However, just because the first viewing user is not watching the video, if the gift object selected by consuming points or coins is not output to the video, the satisfaction of the distribution user and the first viewing user will decrease. Therefore, in such a case, the second avatar object is not output, and only the gift object is output to the video.

[0080] In step S16, if the server 13 does not end the distribution (step S16: NO), it returns to step S11 and repeatedly determines whether a gift output request has been received. If the server 13 ends the distribution (step S16: YES), the process ends.

[0081] (Viewing screen) Next, with reference to FIGS. 7 to 11, the screen output by the user device 12 to the display 28 will be described.

[0082] Referring to FIG. 7, the home screen 101 will be described. When the user starts the video application, the user device 12 receives a list of videos being distributed from the server 13. The user device 12 outputs the home screen 101 to the display 28 based on the list. A video selection section 102 indicating viewable videos is output on the home screen 101. When the user selects the video selection section 102, the user device 12 transmits the ID of the selected video to the server 13 together with a viewing request. The server 13 updates the number of viewing users and transmits data for drawing the video and audio data to the user device 12.

[0083] The viewing user device 12B that has received data or the like for drawing a video generates video data using the received data. Then, a viewing screen is output to the display 28 based on the video data.

[0084] Although the viewing screen and the distribution screen output to the distribution user device 12A have different GUIs (Graphical User Interfaces), except for exceptions, the content of the video including the avatar object is substantially the same. Therefore, in FIGS. 8 to 11, the viewing screen 100 will be illustrated and described as an example.

[0085] FIG. 8 shows an example of the viewing screen 100. The viewing screen 100 includes an image of a first avatar object 110 corresponding to the distribution user. Tracking data based on the actions of the distribution user is used for the first avatar object 110. Drawing data of the wearing parts 111 is used for the first avatar object 110. In the virtual space, a non-wearing part 125 that is a wallpaper is used. Also, the viewing user device 12B outputs sound from the speaker 25 in synchronization with the actions of the first avatar object 110.

[0086] Also, the viewing screen 100 includes the number of viewers 112 and the total number of hearts 113. The number of viewers 112 is the cumulative number of viewers who have watched the video. Or the number of viewers 112 may be the number of viewers currently watching the video. Also, the total number of hearts 113 is the number of times viewers have given a favorable evaluation to the video since the start of the video.

[0087] Also, the viewing screen 100 includes a viewing ranking button 114, a distributor ranking button 115, a sharing button 116, and a follow button 117. When the viewing user operates the viewing ranking button 114, the viewing user device 12B outputs the ranking of the viewing users based on the number of gifts sent among the viewing users who have watched the videos distributed by the distributor. The distributor ranking button 115 outputs the ranking of the distribution users based on the number of gifts sent among all the distribution users who distribute videos.

[0088] When the viewing user selects the shared button 116, the viewing user device 12B transmits the link (URL) of the video on other social network services, copies the link, or reads out the follow list of the users who are following each other, etc. When the viewing user selects the follow button 117, the server 13 associates this viewing user with the distribution user who distributes this video.

[0089] In addition, the viewing screen 100 includes a message input section 118 and a message display section 119. The message input section 118 is an input form that enables the viewing user to send a message to the distribution user. The message display section 119 is a message sent by each viewing user device 12B or a fixed message sent by the server 13 based on the action of the viewing user. The message display section 119 is output so that the latest one is at the top (or bottom), moves downward (or upward) when a new message display section 119 is added, and is erased when it reaches the bottom (or top) of the displayable area of the message display section 119.

[0090] In addition, the viewing screen 100 includes a camera button 120, a positive evaluation button 121, and a gift button 122. When the camera button 120 is selected and operated, the viewing user device 12B captures the viewing screen 100 being output. In addition, the captured viewing screen 100 is stored in the storage 22. When the positive evaluation button 121 is selected and operated, the viewing user device 12B outputs an indicator indicating a positive evaluation, such as a heart mark, on the viewing screen 100. In addition, the viewing user device 12B sends a request to reflect the positive evaluation to the server 13. When the server 13 receives the request to reflect the positive evaluation, it updates the total heart count 113.

[0091] When the gift button 122 is selected and operated by the first viewing user, the viewing user device 12B outputs a list of gift objects overlaid on the viewing screen 100 based on the gift data 53. The first viewing user selects a gift from the list of gift objects. The viewing user device 12B transmits a gift output request to the server 13. Note that the viewing user device 12B of the first viewing user may transmit the user ID of the first viewing user and the gift ID of the gift object selected by the first viewing user together with the gift output request.

[0092] FIG. 9 is a viewing screen 100 output after the viewing user device 12B selects a gift object. The viewing screen 100 includes a slot change section 139. The first viewing user operates the slot change section 139 to select an arbitrary slot number. A second avatar object 150 corresponding to the slot number is output on the viewing screen 100. The wearing parts 111 pre-registered by the first viewing user are used for the second avatar object 150.

[0093] In addition, a wearing part selection section 130 is output on the viewing screen 100. The second avatar object 150 corresponding to the first viewing user who transmitted the gift output request is output in the video to pass the gift object. Therefore, the first viewing user selects a wearing part with the wearing part selection section 130 in order to be an avatar object suitable for appearing in the video. For example, in a scene where the distribution user is congratulated, the first viewing user selects clothing suitable for the scene. When a theme is set for the video, the first viewing user selects clothing that matches the theme. Also, the first viewing user may select clothing that matches the season in the real world.

[0094] Note that the viewing screen 100 as illustrated in FIG. 9 may be output before the first viewing user selects a gift object. In this case, the first viewing user can change the second avatar object 150 and the wearing parts before selecting the gift object.

[0095] Also, after the first viewing user selects a gift object, a viewing screen 100 as illustrated in FIG. 9 may be output. When the second avatar object 150 and the attachment parts are determined on the viewing screen 100, the viewing user device 12B of the first viewing user may transmit a gift output request. Furthermore, the viewing screen 100 may include a button for transmitting a gift output request. When the first viewing user selects and operates the button, the viewing user device 12B transmits a gift output request.

[0096] Similarly, an attachment part selection unit 130 may be output on the distribution screen displayed on the distribution user device 12A. In this case, a notification of receiving a gift object is displayed on the distribution screen. The distribution user checks this notification and displays a selection unit similar to the attachment part selection unit 130 of the viewing screen 100. The distribution user selects an attachment part to use as an avatar object corresponding to receiving the gift object 135.

[0097] The attachment part selection unit 130 includes a category selection unit 131 and an attachment part display unit 132. The category selection unit 131 indicates each category of attachment parts. The attachment part display unit 132 indicates the attachment parts available to the user. The first viewing user selects a category and also selects the attachment parts output for each category. Or, the first viewing user cancels the selection of the attachment parts that have already been used. When the viewing user device 12B receives the selection of the attachment parts, it uses the selected attachment parts for the second avatar object. Also, when the viewing user device 12B receives the cancellation of the selection of the attachment parts, it sets the state not to use the attachment parts specified for the avatar object.

[0098] After the first viewing user finishes selecting the attachment parts, the save button 133 is selected. When the viewing user device 12B receives the selection of the save button 133, it transmits the part ID of the main body part of the second avatar object 150 and the part ID of the used attachment parts to the server 13.

[0099] Figure 10 shows a viewing screen 100 of a scene where a gift object is passed. The example in Figure 10 is the viewing screen 100 output when the viewing user is watching a video. The viewing screen 100 in Figure 10 shows an example where the position (viewpoint) of the virtual camera is set to an overhead viewpoint above the first avatar object 110 and the second avatar object 150. Not limited to the viewing user device 12B, the user device 12 draws a video of the virtual space seen from the set position of the virtual camera. The position of the virtual camera is not particularly limited. In the present embodiment, the position of the virtual camera is a common position among the distribution user device 12A, the viewing user device 12B of the first viewing user, and the viewing user device 12B of the second viewing user.

[0100] For example, the position of the virtual camera may be set to the position of the first avatar object 110, the position of the second avatar object 150, or other positions. The position of the first avatar object 110 is, for example, the position of the eyes of the first avatar object 110. The position of the second avatar object 150 is, for example, the position of the eyes of the second avatar object 150. When the user device 12 determines that the virtual camera is at the position of the first avatar object 110, the user device 12 outputs a first-person view video of the first avatar object 110 to the display 28. When the user device 12 determines that the virtual camera is at the position of the second avatar object 150, the user device 12 outputs a first-person view video of the second avatar object 150 to the display 28. The line of sight direction of the virtual camera, i.e., the visual center, may be constant or may be changed based on the operation of the operation unit 27 of the user.

[0101] The first avatar object 110, the gift object 135, and the second avatar object 150 are output on the viewing screen 100. The second avatar object 150 and the gift object 135 are associated with each other. Specifically, the second avatar object 150 is output so as to hold the gift object 135 by associating the hand part 150A with the gift object 135.

[0102] The viewing user device 12B draws the second avatar object 150 using automatic motion data. This automatic motion data is motion data that expresses the action of passing the gift object 135. The automatic motion data is an action that rotates the arm part 150B lowered on both sides of the second avatar object 150 upward by approximately 90 degrees. Thereby, an animation is output in which the gift object 135 held in the hand is extended forward and the gift object 135 is passed to the first avatar object 110.

[0103] On the other hand, while the automatic motion data is being used for the second avatar object 150, the tracking data of the first viewing user is used for the face part 150C (the first part) of the second avatar object 150. Thereby, the expression of the first viewing user is reflected in the second avatar object 150. When a non-display condition for making the second avatar object 150 non-display is satisfied, the distribution user device 12A and the viewing user device 12B make the second avatar object 150 non-display. An example of the non-display condition is that the second avatar object 150 has completed the operation. For example, the second avatar object 150 has passed the gift object 135 to the first avatar object 110. When the operation of the second avatar object 150 ends, the second avatar object 150 is made non-display. In the non-display state, the second avatar object 150 becomes non-display or moves outside the display area of the screen and exits the screen. The operation for the second avatar object 150 to exit uses the automatic motion data.

[0104] Not limited to the automatic motion data, for determining whether the operation of the second avatar object 150 is completed, (A) collision determination, and (B) determination of whether the position of the gift object 135 and the position of the first avatar object 110 are within a predetermined distance may be used. In the collision determination of (A), when the gift object 135 and a predetermined part of the first avatar object 110 associated with the gift object 135 are in contact, it is determined that the operation of the second avatar object 150 has ended. That is, it is determined that the non-display condition is satisfied. In (B), when the relative distance between the position of the gift object 135 and the position of the first avatar object 110 is within a predetermined distance, it is determined that the operation of the second avatar object 150 has ended. The non-display condition may be a condition corresponding to the gift object 135. For example, when the gift object 135 is of a type to which (A) and (B) are not applicable, the end of the reproduction of the animation of the gift object 135 may be the non-display condition. The gift object 135 of a type to which (A) and (B) are not applicable is, for example, an animation represented by one or more two-dimensional particles such as "fireworks", "rain", "flame", etc.

[0105] Note that before the second avatar object 150 is output to the video, a notification such as "Your avatar will be displayed" may be output. And after this notification is displayed, the second avatar object 150 may be output to the video. Alternatively, a notification such as "Will you appear?" and buttons for approving the appearance and canceling the appearance may be output. And when the first viewing user selects the approval button, the second avatar object 150 is output to the video, and when the cancel button is selected, the second avatar object 150 may not be output. That is, when the first viewing user selects the approval button, the viewing user device 12B of the first viewing user transmits a permission notification for outputting the second avatar object 150 to the server 13. When the server 13 receives the permission notification, it determines the timing for outputting the second avatar object 150. And the server 13 transmits a request for outputting the second avatar object 150 to the distribution user device 12A and each viewing user device 12B based on the timing for outputting the second avatar object 150. Also, the server 13 determines the timing for outputting the gift object, and based on the timing, transmits a request for outputting the gift object to the distribution user device 12A and each viewing user device 12B. The distribution user device 12A and each viewing user device 12B draw the second avatar object 150 based on the request for outputting the second avatar object 150 and the request for outputting the gift object.

[0106] Also, when the first viewing user selects the cancel button, the viewing user device 12B of the first viewing user transmits a cancellation notification for outputting the second avatar object 150 to the server 13. When the server 13 receives the cancellation notification, it determines the timing for outputting the gift object, and based on the timing, transmits a request for outputting the gift object to the distribution user device 12A and each viewing user device 12B. The distribution user device 12A and each viewing user device 12B draw the gift object 135 based on the drawing instruction for the gift object.

[0107] By automatically using the motion data to pass the gift object 135 to the second avatar object 150 in this way, the process of passing the gift object 135 can proceed smoothly. Also, by using the tracking data of the first viewer himself / herself for the facial part 150C of the second avatar object 150, the first viewer can easily obtain a sense of presence in the video.

[0108] FIG. 11 is a viewing screen 100 of a scene for outputting a gift object. The example of FIG. 11 shows the viewing screen 100 output when the first viewer is not viewing the video when the gift object is output.

[0109] On the viewing screen 100, the first avatar object 110 and the gift object 135 are output. In this case, the second avatar object 150 is not output on the viewing screen 100. The gift object 135 is output to the virtual space. The user device 12 may arrange the gift object 135 at a predetermined position such as the floor surface. Also, the user device 12 may draw the gift object 135 so as to drop it at a specific position in the virtual space.

[0110] In this way, even when the first viewer is not watching the video, the gift object 135 given by the first viewer is output. Therefore, the satisfaction of the distribution user and the first viewer can be enhanced.

[0111] The effects of the first embodiment will be described. (1-1) In the first embodiment, the user device 12 uses automatic motion data for the second avatar object corresponding to the first viewer. For this reason, it is possible to appropriately and smoothly output a scene or the like where drawing is not easy only with tracking data. Also, since the user device 12 uses the tracking data of the first viewer for the second avatar object in addition to the automatic motion data, the first viewer can obtain a sense of presence in the video. Therefore, the satisfaction of the viewing user with the video can be enhanced.

[0112] (1-2) In the first embodiment, at least some parts of the second avatar object 150 are drawn using the tracking data of the first viewing user, and at least some other parts of the second avatar object are drawn using the automatic motion data. Therefore, automatic motion data can be used for parts that are difficult to represent using tracking data. Also, for parts of the second avatar object 150 that are likely to reflect the intention of the first viewing user, it is possible to use the tracking data.

[0113] (1-3) In the first embodiment, the tracking data of the first viewing user is used for the first part such as the face part 150C of the second avatar object 150, and automatic motion data is used for the second part such as the hand part 150A and the arm part 150B. Therefore, while smoothly outputting the operation of passing the gift object 135, it is possible to reflect the expression of the first viewing user on the second avatar object 150. Also, by using the automatic motion data in this way, the processes of acquiring the tracking data and transmitting the tracking data by the viewing user device 12B of the first viewing user can be omitted. For this reason, it is possible to draw the second avatar object 150 that reflects the intention of the first viewing user and has individuality while suppressing the processing load of the viewing user device 12B of the first viewing user.

[0114] (1-4) In the first embodiment, when outputting the gift object 135, if the first viewing user is not continuing to watch the video, the second avatar object 150 is not output. Therefore, it is possible to prevent the second avatar object 150 corresponding to the user himself / herself from being output although the first viewing user is not watching.

[0115] (Second Embodiment) Next, a second embodiment of the information processing system will be described. In the second embodiment, it is different from the first embodiment in that the first viewing user can select an action. Hereinafter, the same parts as those in the first embodiment will be denoted by the same reference numerals, and the detailed description thereof will be omitted.

[0116] In this embodiment, the user device 12 draws the second avatar object 150 using the following (2) and (3). (2) Preset automatic motion data. (3) Among the motion data prepared in advance, the selected motion data selected by the first viewing user. With reference to FIG. 12, the process of the viewing user device 12B of the first viewing user drawing the second avatar object 150 and the gift object 135 will be described. In this embodiment, it is assumed that the first viewing user is viewing a video when the gift object 135 is output.

[0117] When the first viewing user selects a gift object, the control unit 20 of the viewing user device 12B of the first viewing user transmits the gift ID of the selected gift object and the user ID of the first viewing user to the server 13. The control unit 20 outputs the gift object 135 and the second avatar object 150 to the display 28 according to the operation of the first viewing user or based on the gift ID received from the server 13 (step S20).

[0118] Next, the control unit 20 outputs the emoticon selection unit to the video (step S21). The emoticon selection unit is a GUI for selecting the operation of the second avatar object 150. For example, the emoticon selection unit includes a plurality of selection buttons for expressions (hereinafter referred to as emoticons) of the second avatar object 150 such as "stretching both hands and passing" and "bowing and passing". The first viewing user selects one of the selection buttons. Note that the order of outputting the gift object 135 and the second avatar object 150 to the display 28 (step S20) and outputting the emoticon selection unit to the video (step S21) is not limited. It is also possible to perform the process of step S21 before the process of step S20.

[0119] The control unit 20 receives the emoticon selected by the first viewing user (step S22). Further, the control unit 20 transmits information indicating the selected emoticon to the server 13. The control unit 20 acquires the selection motion data corresponding to the emoticon selected by the first viewing user (step S23). The viewing user device 12B may acquire the selection motion data stored in the storage 22 in advance from the storage 22. At this time, the viewing user device 12B may receive the identification information and output request of the selection motion data from the server 13. In this case, the viewing user device 12B can set the timing of receiving the identification information and output request of the selection motion data as the timing of using the selection motion data for the second avatar object 150. Alternatively, the viewing user device 12B may transmit the identification information indicating the emoticon to the server 13 and then receive the selection motion data corresponding to the emoticon from the server 13.

[0120] Then, the viewing user device 12B uses the acquired selected motion data for the second avatar object 150 (step S24). For example, when the first viewing user selects an emotion of "stretching both hands and passing", the user device 12 uses the selected motion data of stretching both hands and passing the gift object 135 for the second avatar object 150. The viewing user device 12B and the distribution user device 12A used by the second viewing user receive the selected motion data from the server 13 or receive the identification information of the selected motion data together with the output request. Then, the viewing user device 12B and the distribution user device 12A used by the second viewing user use the acquired selected motion data for the second avatar object 150.

[0121] When the operation based on the selected motion data ends, the control unit 20 acquires automatic motion data (step S25). Then, the acquired automatic motion data is used for the second avatar object 150 (step S26). For example, the automatic motion data is "shaking hands". Thereby, the second avatar object 150 performs an operation of shaking hands with the hand part 150A overlapping the hand part of the first avatar object 110. After automatically using the automatic motion data, the viewing user device 12B exits the second avatar object 150. Note that the process related to the selected motion including outputting the emotion selection unit to the video (step S21), receiving the emotion selected by the first viewing user (step S22), acquiring the selected motion data corresponding to the emotion selected by the first viewing user (step S23), and using the acquired motion data for the second avatar object 150 (step S24), and the process related to the automatic motion including acquiring the automatic motion data (step S25) and using the acquired automatic motion data for the second avatar object 150 (step S26) may be performed in the reverse order. That is, the process related to the automatic motion may be performed before the process related to the selected motion.

[0122] The automatic motion data may be stored in each user device 12. In this case, the user device 12 receives the identification information of the automatic motion data from the server 13. Or, each user device 12 may receive the automatic motion data from the server 13.

[0123] FIG. 13 is a viewing screen 100 when the viewing user device 12B outputs a gift object. The example of FIG. 13 is the viewing screen 100 output when the first viewing user is viewing a video. The viewing screen 100 includes an emoticon selection unit 140.

[0124] On the viewing screen 100, a first avatar object 110, a gift object 135, and a second avatar object 150 are output. The second avatar object 150 and the gift object 135 are associated with each other.

[0125] The emoticon selection unit 140 is output in association with the second avatar object 150. Specifically, the emoticon selection unit 140 is output overlapping or in the vicinity of the second avatar object 150. The emoticon selection unit 140 includes selection buttons 140A and 140B. The selection button 140A is a button for selecting an action such as "stretching both hands to pass". The selection button 140B is a button for selecting an action such as "bowing and passing". Depending on which of the selection buttons 140A and 140B the first viewing user selects, the action of the second avatar object 150 is determined. When the first viewing user selects the selection button 140A, selection motion data for stretching both hands to pass the gift object 135 is used for the second avatar object 150. On the other hand, when the first viewing user selects the selection button 140B, selection motion data for bowing and passing the gift object 135 is used for the second avatar object 150.

[0126] When the viewing user device 12B and the distribution user device 12A complete the use of the selection motion data, they use the automatic motion data for the second avatar object 150. According to the second embodiment, in addition to at least the effects described in (1-4) of the first embodiment, the following effects can be obtained.

[0127] (2-1) In the second embodiment, the user device 12 uses the automatic motion data for the second avatar object 150 corresponding to the first viewing user. Therefore, it is possible to appropriately and smoothly output a scene or the like where drawing is not easy only with the tracking data. On the other hand, since the user device 12 uses the selected motion data selected by the first viewing user for the second avatar object 150 in addition to the automatic motion data, the first viewing user can obtain a sense of presence in the video. Therefore, the satisfaction of the first viewing user with the video can be increased. Furthermore, as long as such selected motion data is used, the processes of acquiring the tracking data and transmitting the tracking data by the viewing user device 12B of the first viewing user can be omitted. Therefore, it is possible to output a video with variations while suppressing the processing load of the viewing user device 12B of the first viewing user.

[0128] (2-2) In the second embodiment, the automatic motion data may be used for the second avatar object 150 and the selected motion data selected by the first viewing user may be used. In this case, while smoothly outputting the operation of passing the gift object 135, it is possible to reflect the intention of the first viewing user in the second avatar object 150.

[0129] (2-3) In the second embodiment, the emot selection unit 140 for selecting the selected motion data is output. Therefore, it becomes easier for the first viewing user to select the selected motion data.

[0130] (Third Embodiment) Next, a third embodiment of the information processing system will be described. In this embodiment, the user device 12 draws the second avatar object 150 using the following (1) and (3). (1) Tracking data of the first viewing user who sent a request to output a gift object. (3) Among the pre-prepared motion data, the selected motion data selected by the first viewing user.

[0131] Hereinafter, the same parts as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. With reference to FIG. 14, a process in which the viewing user device 12B draws the second avatar object 150 and the gift object 135 will be described. In this embodiment, it is assumed that the first viewing user is viewing a video when the gift object 135 is output.

[0132] The viewing user device 12B used by the first viewing user performs steps S30 to S34 in the same manner as steps S20 to S24 in the second embodiment. The viewing user device 12B and the distribution user device 12A used by the second viewing user draw the second avatar object 150 based on the selected motion data received from the server 13 or the identification information specifying the selected motion data. Note that the order of outputting the gift object 135 and the second avatar object 150 to the display 28 (step S30) and outputting the emoticon selection unit to the video (step S31) is not limited. It is also possible to perform the process of step S31 before the process of step S30.

[0133] Furthermore, the control unit 20 of the viewing user device 12B used by the first viewing user acquires the tracking data of the first viewing user (step S35). Also, the control unit 20 draws the second avatar object 150 based on the acquired tracking data (step S36). The distribution user device 12A and the viewing user device 12B used by the second viewing user receive the tracking data from the server 13 and draw the second avatar object 150 using the tracking data. As a result, the second avatar object 150 output on the viewing screen 100 operates based on the selection motion data first and then operates based on the motion of the first viewing user. Note that the process related to the selection motion including outputting the emoticon selection unit to the video (step S31), receiving the emoticon selected by the first viewing user (step S32), acquiring the selection motion data corresponding to the emoticon selected by the first viewing user (step S33), and using the acquired motion data for the second avatar object 150 (step S34), and the process of using the tracking data including acquiring the tracking data (step S35) and using the acquired tracking data for the second avatar object 150 (step S36) may be performed in the reverse order. That is, the process of using the tracking data may be performed before the process related to the selection motion.

[0134] In this way, by using the selection motion data prepared in advance for the second avatar object 150, the progress of the video can be smoothed. Also, since the first viewing user can select the operation of the second avatar object 150, the user can obtain a sense of presence in the video. Furthermore, in the scene of passing the gift object 135, since the second avatar object 150 holds the gift object 135, the operation may be restricted. Or, when using the tracking data with the second avatar object 150 holding the gift object 135, depending on the type of selection motion, it may look unnatural. However, in this embodiment, when the selection motion data is acquired in advance and the motion corresponding to the selection motion data is the operation of passing the gift object, after the association between the gift object 135 and the second avatar object 150 is released, the tracking data is used for the second avatar object 150, so that it can be unrestricted and have a natural motion.

[0135] According to the third embodiment, in addition to at least the effects described in (1-4) of the first embodiment and (2-3) of the second embodiment, the following effects can be obtained. (4-1) In the third embodiment, the user device 12 uses the selection motion data for the second avatar object 150 corresponding to the first viewing user. For this reason, it is possible to appropriately and smoothly output a scene or the like where drawing is not easy only with the tracking data. Also, the first viewing user can obtain a sense of presence in the video. Furthermore, since the user device 12 uses the tracking data of the first viewing user for the second avatar object 150 in addition to the selection motion data, the first viewing user can obtain a sense of presence in the video. For this reason, the satisfaction of the first viewing user with the video can be enhanced.

[0136] Each of the above embodiments can be implemented with the following modifications. The above embodiments and the following modification examples can be implemented in combination with each other within a technically consistent range. <User> · In each of the above embodiments, the distributing user means a user who transmits at least one of information related to video and information related to audio. For example, the distributing user is a user who performs video distribution alone or a collaborative distribution in which multiple people can participate. Or, the distributing user may be a user who hosts or holds a video chat or voice chat in which at least one of multiple people can participate and view, or an event (such as a party) in a virtual space in which at least one of multiple people can participate and view. That is, the distributing user can also be referred to as a host user, a hosting user, or a holding user.

[0137] On the other hand, the viewing user means a user who receives at least one of information related to video and information related to audio. However, the viewing user can be a user who not only receives the above information but can also react. For example, the viewing user is a user who views video distribution or collaborative distribution. Or the viewing user may be a user who participates in and / or views at least one of a video chat, a voice chat, or an event. Therefore, the viewing user can also be referred to as a guest user, a participating user, a listener, a viewer, a cheering user, etc.

[0138] · The information processing system in each of the above embodiments is a digital world in which many people can participate simultaneously and freely engage in activities such as communication, work, and play through character objects (avatars) at a level close to that of the real society, and can be used to provide the following Internet space (metaverse) in which social activities can transcend the gap between reality and virtuality.

[0139] In such a space, the user's avatar objects can freely walk around in the world and communicate with each other. Among a plurality of avatar objects in such a virtual space, one avatar object may be configured to be able to distribute a video as the first avatar object of the distributing user. That is, it may be possible to perform one-to-many video distribution in a many-to-many metaverse virtual space. In such a space, the distributing user and the viewing user may not be particularly distinguished.

[0140] <Operation and Drawing of the Second Avatar Object> · In the first embodiment, tracking data is used for the first part of the second avatar object 150, and automatic motion data is used for the second part. Instead of this, after using tracking data for the second avatar object 150, automatic motion data may be used. Or, after a period of using automatic motion data, tracking data may be used.

[0141] · In the second and third embodiments, the emotion selection unit 140 may be output when the emotion selection condition is satisfied. The emotion selection condition is, for example, that the sensor unit 24 of the viewing user device 12B used by the first viewing user does not detect an operation. When the first viewing user is confused about the operation to be taken, the sensor unit 24 does not detect an operation. In such a case, by outputting the emotion selection unit 140, the first viewing user can easily select an operation. Also, the emotion selection condition may be that the first viewing user has performed a predetermined operation. The predetermined operation is, for example, that the first viewing user has turned around, etc. Also, the emotion selection condition may be that an instruction to display on the screen of the emotion selection unit 140 has been given. In this way, when the first viewing user is in trouble corresponding when making an appearance in the video, the emotion selection unit 140 can be output.

[0142] · In each of the above embodiments, the second avatar object 150 was operated using the tracking data detecting the operation of the first viewing user or the selected motion data selected by the viewing user from among a plurality of preset motion data, and the automatic motion data automatically used for the second avatar object 150. In addition to this, the second avatar object 150 may be operated using the tracking data detecting the operation of the first viewing user and the selected motion data selected by the first viewing user from among a plurality of preset motion data, and the automatic motion data automatically used for the second avatar object 150. That is, the second avatar object 150 may be operated using all of (1) the tracking data of the first viewing user who transmitted the output request of the gift object, (2) the preset automatic motion data, and (3) the selected motion data selected by the first viewing user from among the prepared motion data.

[0143] · In each of the above embodiments, the second avatar object 150 was operated using the tracking data detecting the operation of the first viewing user or the selected motion data selected by the first viewing user from among a plurality of preset motion data, and the automatic motion data automatically used for the second avatar object 150. Instead of this, the second avatar object 150 may be operated using any one of (1) the tracking data of the first viewing user who transmitted the output request of the gift object, (2) the preset automatic motion data, and (3) the selected motion data selected by the first viewing user from among the prepared motion data.

[0144] · Gift management information associating the gift ID of the gift object 135 with the motion data for operating the second avatar object 150 may be stored in at least one of the server 13 and the user device 12.

[0145] In the second output control process, the user device 12 identifies the motion data based on the identification information of the motion data that operates the second avatar object 150 associated with the gift ID (identification information of the gift object), and uses the motion data to draw the gift object 135 and the second avatar object 150. The user device 12 may receive the gift ID from the server 13 and identify the identification information of the motion data associated with the gift ID based on the gift management information. Then, the second avatar object 150 may be drawn using the motion data corresponding to the identified identification information of the motion data. Alternatively, the user device 12 may receive the identification information of the motion data from the server 13 and draw the second avatar object 150 using the motion data associated with the identification information. Alternatively, the user device 12 may receive the motion data associated with the gift ID from the server 13 and draw the second avatar object 150 using the motion data. In this form, when drawing the gift object 135 and the second avatar object 150, at least one of the tracking data, the selected motion data, and the automatic motion data is used to draw the second avatar object 150.

[0146] The gift management information will be described. The gift ID included in the gift management information is associated with at least one of a plurality of operation data for each gift ID. The plurality of operation data are (1) the tracking data of the first viewing user who transmitted the output request of the gift object, (2) the preset automatic motion data, and (3) the prepared motion data in advance. The prepared motion data in advance in (3) is motion data that can be selected by the user as selected motion data. Alternatively, the gift ID included in the gift management information may be associated with the type of gift. The type of gift is associated with at least one of a plurality of operation data. Further, when the gift ID is associated with a plurality of the above (1) to (3), the gift ID is associated with the order in which they are used.

[0147] (2) and (3) The content of the operation data varies according to the gift object 135. For example, in the gift management information, when the gift object 135 is a "bouquet of flowers", the gift ID may be associated with the data of the operation of passing the gift object 135. Also, when the gift object 135 is a "snowball", the gift ID may be associated with the data of the operation of throwing the gift object 135. Or, in the gift management information, the gift object 135 of "bouquet of flowers" may be associated with a type indicating "pass". The type indicating "pass" may be associated with the data of the operation of passing the gift object 135. The gift object 135 of "snowball" may be associated with a type indicating "throw". The type indicating "throw" may be associated with the data of the operation of throwing the gift object 135.

[0148] The case where only the server 13 stores the gift management information and the user device 12 stores the operation data of (2) and (3) in the storage 22 in advance will be described. In this case, the server 13 determines the operation data corresponding to the gift ID received from the viewing user device 12B used by the first viewing user based on the gift management information. At this time, it is assumed that the server 13 determines that the operation data of (2) or (3) is associated with the gift ID. The server 13 transmits the identification information of the operation data of (2) or (3) to each user device 12. The user device 12 receives the identification information of the operation data from the server 13. Then, the user device 12 acquires the operation data corresponding to the received identification information from the storage 22. Also, the user device 12 uses the acquired operation data to draw the second avatar object 150 by the method described in each of the above embodiments.

[0149] The case where only the server 13 stores the gift management information and the user device 12 has not stored the operation data in the storage 22 in advance will be described. In this case, based on the gift management information, the server 13 transmits the operation data corresponding to the gift ID received from the viewing user device 12B used by the first viewing user to each user device 12. The server 13 may acquire the operation data of (2) or (3) from the storage 35. Alternatively, the server 13 may receive the operation data of (2) or (3) from the viewing user device 12B used by the first viewing user. Further, when the server 13 receives the tracking data from the viewing user device 12B used by the first viewing user, the server 13 transmits the tracking data to each user device 12. The user device 12 receives the operation data from the server 13. Further, the user device 12 uses the received operation data to draw the second avatar object 150 by the method described in each of the above embodiments.

[0150] The case where the user device 12 stores the gift management information in the storage 22 will be described. In this case, based on the gift management information, the server 13 transmits the gift ID received from the viewing user device 12B used by the first viewing user to each user device 12. The user device 12 specifies the operation data corresponding to the gift ID received from the server 13 based on the gift management information. When the operation data is the tracking data of (1), the user device 12 that has not acquired the tracking data receives the tracking data from the server 13. When the operation data is (2) or (3), the user device 12 acquires the specified operation data from the storage 22. Further, the user device 12 uses the operation data received from the server 13 or the acquired operation data to draw the second avatar object 150 by the method described in each of the above embodiments.

[0151] In addition to or instead of associating the gift ID with the identification information of the operation data, the gift management information may also associate the gift ID with avatar display information indicating whether to display the second avatar object 150. The avatar display information may be a flag indicating whether to display the second avatar object 150. Alternatively, when the identification information of the operation data is not associated with the gift ID in the server 13 or the user device 12, it may be determined not to display the second avatar object 150. When the user device 12 determines to display the second avatar object 150 based on the avatar display information, the gift object 135 and the second avatar object 150 are drawn. When the user device 12 determines not to display the second avatar object 150 based on the avatar display information, the gift object 135 is drawn and the second avatar object 150 is not displayed.

[0152] In addition, when the gift ID and the tracking data in (1) are associated as operation data, the gift management information may further associate the usage time, which is the time to use the tracking data for the gift ID. That is, as one of the non-display conditions of the second avatar object 150, in addition to or instead of the above (A) collision determination and (B) determination of whether the position of the gift object 135 is within a predetermined distance from the first avatar object 110, (C) the usage time may be set. When the user device 12 draws the second avatar object 150 using the tracking data, the second avatar object 150 is drawn using the tracking data only within the time indicated by the usage time. When the time to draw the second avatar object 150 reaches the usage time, the user device 12 exits the second avatar object 150 or makes the second avatar object 150 non-displayed.

[0153] Also, when the operation of the second avatar object 150 is defined in the gift management information in this way, the non-display condition of the second avatar object 150 may be that the operation of the second avatar object 150 has been completed. Whether the operation of the second avatar object 150 has been completed may be determined by the above (A) collision determination, (B) determination of whether the position of the gift object 135 and the position of the first avatar object 110 are within a predetermined distance, and (C) a predetermined usage time.

[0154] · In each of the above embodiments, the second avatar object 150 may be output using data with a lower rendering processing load than the first avatar object 110. For example, the second avatar object 150 may be data with a smaller number of polygons than the first avatar object 110, two-dimensional data, data obtained by integrating a plurality of parts, etc. Two-dimensional data is data for rendering two-dimensional objects. When the second avatar object 150 is rendered using two-dimensional data, the position of the virtual camera is fixed at a predetermined position. The predetermined position is a position where no discomfort is felt when rendering the three-dimensional first avatar object 110 and the two-dimensional second avatar object 150. In this aspect, the load on the rendering process of the user device 12 can be reduced. When rendering a plurality of second avatar objects 150, there is a problem that the processing load of the user device 12 increases. By reducing the processing load of each second avatar object 150 in this way, the processing load when rendering a plurality of second avatar objects 150 can also be reduced. Therefore, the opportunity to have a plurality of second avatar objects 150 appear in the video can be increased.

[0155] ·In each of the above embodiments, when the first viewing user who has sent a gift output request is watching a video, the second avatar object 150 and the gift object 135 are output, and when the first viewing user is not watching a video, the gift object 135 is output. Instead of this, the display mode of the gift object 135 may be changed according to whether the first viewing user who has sent the gift output request is watching a video or not. For example, when the first viewing user who has sent a gift output request of "snow ball" is watching a video, the second avatar object 150 and the snow ball which is the gift object 135 may be output so as to be thrown at the first avatar object 110. Also, when the first viewing user is not watching a video, as the gift object 135, an effect of making snow fall from above the head of the first avatar object 110 may be executed. In this case, the effect of making snow fall becomes the gift object. That is, at least a part of the color, shape, and movement of the gift object 135 may be changed according to whether the first viewing user who has sent the gift output request is watching a video or not. According to this aspect, the variations in the display mode of the gift object 135 can be increased.

[0156] ·In each of the above embodiments, when the first viewing user who has sent a gift output request is watching a video, the second avatar object 150 and the gift object 135 are output, and when the first viewing user is not watching a video, the gift object 135 is output. Instead of this, regardless of whether the first viewing user is watching a video or not, the second avatar object 150 may be displayed. That is, step S13 may be omitted.

[0157] · The motion data used for the first avatar object 110 and the second avatar object 150 is not limited to those described in the above embodiments. For example, when the first viewing user watches a video for the first time, motion data in which the first avatar object 110 and the second avatar object 150 shake hands may be used. When the first viewing user performs an operation of raising the arm with the palm facing forward, motion data of a high-five operation in which the first avatar object 110 and the second avatar object 150 align the palms of each other at a height near the head and slap each other may be used.

[0158] · In the first embodiment and the third embodiment, when drawing a scene in which the second avatar object 150 hands the gift object 135 to the first avatar object 110 using the tracking data, there is a possibility that the first viewing user does not perform an operation of handing the gift object 135 to the first avatar object 110. Or it is assumed that the first viewing user fails to hand the gift object 135 to the first avatar object 110. When the user device 12 determines that the gift object 135 and the first avatar object 110 are not associated even after a predetermined time has passed since the second avatar object 150 was displayed, the gift object 135 and the first avatar object 110 may be forcibly associated.

[0159] · In the second embodiment, the second avatar object 150 may be made to operate according to the type of operation of the first viewing user. For example, when the first viewing user taps the screen, the user device 12 draws the second avatar object 150 that performs a thumbs-up operation. Also, when the first viewing user taps the screen a plurality of times, the user device 12 draws the second avatar object 150 that claps.

[0160] <Operation of the First Avatar Object> · In the second and third embodiments, on the viewing screen, an emoticon selection unit 140 is output in association with the second avatar object 150. Instead of or in addition to this, as shown in FIG. 15, on the distribution screen 103, the emoticon selection unit 140 may be output in association with the first avatar object 110. The shown emoticon selection unit 140 includes selection buttons 140A and 140B. The selection button 140A is a button for selecting an operation such as "open immediately" for the gift object 135 passed from the second avatar object 150. The selection button 140B is a button for selecting an operation such as "place" for the gift object 135. Depending on which of the selection buttons 140A and 140B the distribution user selects, the operation of the first avatar object 110 is determined. When the distribution user selects the selection button 140A, selection motion data for opening the gift object 135, which is a box, is used for the first avatar object 110. On the other hand, when the distribution user selects the selection button 140B, selection motion data for placing the gift object 135 at a predetermined position such as the floor is used for the first avatar object 110. For this reason, it is possible to appropriately and smoothly output a scene where drawing is not easy only with tracking data. Also, in a specific scene such as a scene of receiving the gift object 135, the intention of the distribution user can be distributed to the video.

[0161] The distribution screen 103 shown in FIG. 15 includes a share button 116, a co-star confirmation button 137, a broadcaster ranking button 115, a setting button 138, and an audio button 160. By selecting the co-star confirmation button 137 during collaboration distribution, the profile information of the co-stars can be viewed. By selecting the setting button 138, distribution settings can be made. When the audio button 160 is selected, the broadcast user device 12A stops (mutes) the output of audio, or cancels the audio stop state. In addition, the distribution screen 103 includes a comment button 161, a shooting button 141, a game button 142, a video viewing button 143, and a gacha execution button 144. By operating the comment button 161, the broadcast user device 12A outputs an input form for inputting a comment. By operating the shooting button 141, the broadcast user device 12A captures the video. By operating the game button 142, the broadcast user device 12A outputs a list of games that can be played during the distribution of the video. A live game screen is output to the video. When the video viewing button 143 is operated, the broadcast user device 12A outputs a screen for selecting a video provided by another video broadcast service. When the gacha execution button 144 is operated, the broadcast user device 12A outputs a list of available lotteries (gacha).

[0162] <Drawing of gift object in response to a display request from a broadcasting user device> The broadcast user may use the broadcast user device 12A to make a request to output the gift object 135. When the second avatar object 150 is being output to the video, the broadcast user performs an operation to select the gift object. The broadcast user device 12A transmits a gift ID and a gift output request. Based on the gift ID and the gift output request, the viewing user device 12B outputs the gift object 135 corresponding to the gift ID in association with the second avatar object 150. According to this embodiment, the broadcast user can give the gift object 135 to the first viewing user. This can make the interaction between the broadcast user and the viewing user more active.

[0163] <Display Conditions for the Second Avatar Object> · In each of the above embodiments, when the viewing user device 12B transmits an output request for the gift object 135, each user device 12 draws the second avatar object 150. Instead of or in addition to this, when conditions other than the transmission of the output request for the gift object 135 are satisfied, the second avatar object 150 corresponding to the first viewing user may be drawn.

[0164] Examples of conditions other than the transmission of the output request for the gift object 135 are given below. The conditions only need to be other than the transmission of the output request for the gift object 135 and are not limited to those exemplified below. For example, the condition other than the transmission of the output request for the gift object 135 is that the value of the parameter associated with the first viewing user satisfies a predetermined condition regarding the parameter. A user who satisfies this predetermined condition can transmit a display request for the second avatar object 150. The parameter may be the number of mutual followers who mutually follow each other among users who use the video application. Alternatively, the parameter may be the ranking of the distribution user or the ranking of the viewing user. Also, the ranking may be calculated regardless of whether it is a distribution user or a viewing user. The ranking of the distribution user is determined based on the video distribution status or the situation in the video of the distribution user. The ranking of the viewing user is determined based on the actions of the viewing user.

[0165] An example of the ranking of the distribution user is the ranking of the distribution users based on the number of gift objects received among all distribution users who distribute videos. Also, the ranking is the ranking arranged in descending order of the number of gift objects received for each type of gift object such as "bouquet". An example of the ranking of the viewing user is the ranking of the viewing users based on the number of gifts sent.

[0166] For example, conditions other than the transmission of the output request for the gift object 135 are that the activity related to the user satisfies a predetermined condition. A user who satisfies this predetermined condition can send a display request for the second avatar object 150. For example, it may be that the amount of money charged by the user within the video application is equal to or more than a predetermined amount. Or, the predetermined condition may be that the number of messages posted by the user within the video application is equal to or more than a predetermined number. Or, the predetermined condition may be that the number of gift objects given by the user within the video application is equal to or more than a predetermined number. Or, the predetermined condition may be that the amount of money charged or the number of message posts in the video being viewed is equal to or more than a predetermined number.

[0167] For another example, it may be that the first viewer has posted a specific message. Or, it may be that the first viewer has viewed a video for the first time. Or, it may be that the first viewer has performed a predetermined operation. For example, when the first viewer posts a specific message such as "Nice to meet you", motion data in which the first avatar object 110 and the second avatar object 150 shake hands may be used.

[0168] · In each of the above embodiments, the second avatar object 150 corresponding to all the first viewing users who have sent a gift output request is drawn. In addition to this, conditions regarding the first viewing users who can display the second avatar object may be provided. Examples of the conditions regarding the first viewing users are given below, but the conditions may be any conditions regarding the first viewing users and are not limited to those exemplified below. For example, when a first viewing user who has followed the distribution user sends a gift output request, the user device 12 may draw the second avatar object 150 of that first viewing user. That is, it is sufficient that the identification information of the distribution user is included in the follow list stored in the viewing user device 12B or the like. Also, the conditions for the first viewing user who outputs the second avatar object 150 may be at least one of the following conditions instead of or in addition to mutually following the distribution user. For example, belonging to a fan club that supports the distribution user, being ranked above a predetermined rank in the support ranking of the distribution user, viewing the video for the first time, the cumulative viewing time of the first viewing user being a predetermined time or more or the number of viewing times being a predetermined number or more. Or, the conditions for the first viewing user who outputs the second avatar object 150 may be that the number of times of sending a gift output request is a predetermined number or more, the number of posted messages is a predetermined number or more, giving a gift object of a predetermined amount or more, and the charged amount reaching a reference value. Also, after the second avatar object 150 is output once in one distribution, the second avatar object 150 may not be output in that distribution session. According to this aspect, it is possible to suppress a situation where the number of second avatar objects to be output increases and there is not enough distribution time to output the second avatar object. Also, by restricting the number of times of drawing the second avatar object, the load on the drawing process of the user device 12 can be reduced.

[0169] <Position of the virtual camera> · In each of the above embodiments, the position of the virtual camera was set to be a common position among the distribution user device 12A, the viewing user device 12B of the first viewing user, and the viewing user device 12B of the second viewing user. Instead of or in addition to this, the position (viewpoint) of the virtual camera may be made different for each of the distribution user device 12A, the viewing user device 12B of the first viewing user, and the viewing user device 12B of the second viewing user.

[0170] The positions of the virtual camera that can be set by each user device 12 are as follows. (Position A) The position of the first avatar object 110 (Position B) The position of the second avatar object 150 (Position C) A position other than the first avatar object 110 and the second avatar object 150 A specific example of position A is the position of the eyes of the first avatar object 110. That is, it is a position where a first-person perspective video of the first avatar object 110 can be displayed. More specifically, it is one of the two eyes or the central position of both eyes. Alternatively, position A may be the top of the head of the first avatar object 110. The line-of-sight direction of the virtual camera is a direction in which a first-person perspective video of the first avatar object 110 can be displayed. The line-of-sight direction may be constant. Or the line-of-sight direction may be changed based on the operation of the operation unit 27 of the user.

[0171] A specific example of position B is the position of the eyes of the second avatar object 150. That is, it is a position where a first-person perspective video of the second avatar object 150 can be displayed. More specifically, it is one of the two eyes or the central position of both eyes. Alternatively, position B may be the top of the head of the second avatar object 150. The line-of-sight direction of the virtual camera is a direction in which a first-person perspective video of the second avatar object 150 can be displayed. The line-of-sight direction may be constant. Or the line-of-sight direction may be changed based on the operation of the operation unit 27 of the user.

[0172] A specific example of position C is a position overlooking the first avatar object 110 and the second avatar object 150. Also, position C may be set by the user at an arbitrary position. The line-of-sight direction of the virtual camera may be constant. Or the line-of-sight direction may be changed based on the operation of the operation unit 27 of the user.

[0173] The distribution user device 12A may set the virtual camera at position A and draw a video. Thereby, the distribution user device 12A can draw a video as if looking from the first avatar object 110 to the second avatar object 150. When the second avatar object 150 performs an operation of passing the gift object 135, the distribution user can watch a video with a sense of presence as if receiving the gift object 135 from the second avatar object 150.

[0174] The viewing user device 12B of the first viewing user may set the virtual camera at position B and draw a video. Thereby, this viewing user device 12B can draw a video as if looking from the second avatar object 150 to the first avatar object 110. When the second avatar object 150 performs an operation of passing the gift object 135, the first viewing user can watch a video with a sense of presence as if passing the gift object 135 to the first avatar object 110.

[0175] The viewing user device 12B of the second viewing user may set the virtual camera at position C and draw a video. Thereby, this viewing user device 12B can draw a video as if looking at the first avatar object 110 and the second avatar object 150.

[0176] Regardless of whether the virtual camera is set at any of positions A, B, or C, each user device 12 draws the second avatar object 150 using any one of the automatic motion data, selected motion data, and tracking data. Similarly, regardless of whether the virtual camera is set at any of positions A, B, or C, each user device 12 draws the first avatar object 110 using any one of the automatic motion data, selected motion data, and tracking data.

[0177] Also, the distributing user, the first viewing user, and the second viewing user may be able to set the camera position from among positions A, B, and C. Indicators such as icons of virtual cameras corresponding to positions A, B, and C respectively are displayed on the screen, and when each user selects one of those icons, the virtual camera may be changed to the selected position.

[0178] · The user device 12 may automatically set the position of the virtual camera according to the type of the gift object 135. When drawing the gift object 135, the user device 12 sets the position of the virtual camera based on the information regarding the position of the virtual camera associated with the gift object 135. For example, the user device 12 may set the position of the virtual camera so that the relative distance from the gift object 135 becomes longer or shorter according to the size of the gift object 135. When drawing a relatively large gift object 135, the position of the virtual camera is set so that the entire gift object 135 is included in the imaging range of the virtual camera. Also, the user device 12 may set the position of the virtual camera so that the relative distance from a predetermined position in the movement trajectory of the gift object 135 becomes longer or shorter according to the movement range of the gift object 135. When drawing the gift object 135 of "snowball", the user device 12 changes the position of the virtual camera so that the relative distance from the gift object 135 becomes longer. Then, at least a part of the movement trajectory of the gift object 135 is included in the imaging range of the virtual camera. Or the position of the virtual camera may be moved in accordance with the movement of the gift object 135.

[0179] <Gift output request by group> · In each of the above embodiments, the user device 12 may be configured to draw the second avatar objects 150 corresponding to the plurality of first viewing users when the plurality of first viewing users give gifts to the distribution user. For example, the viewing user device 12B is conducting a group chat in which a plurality of viewing users are chatting. One viewing user device 12B that makes a gift output request transmits a gift ID and a group ID to the server 13. Instead of the group ID, the user ID of the users belonging to the group may be transmitted. The server 13 acquires the part ID of the main body part and the part ID of the attached part of the avatar object of the users belonging to the group. Then, these data are transmitted to each user device 12 together with the gift ID. The user device 12 that has received the part ID and the gift ID draws the second avatar objects 150 of the first viewing users belonging to the group together with the gift object corresponding to the gift ID. Alternatively, the user device 12 draws the second avatar object 150 of the representative among the first viewing users belonging to the group together with the gift object corresponding to the gift ID. The representative may be the first viewing user who made the gift output request, or may be a viewing user predetermined within the group. When the representative is predetermined, the user ID of that user is recorded in the user device 12 or the server 13.

[0180] <Mode of Drawing Second Avatar Object> · Each user device 12 may be configured to draw a plurality of second avatar objects 150 based on an instruction transmitted from the server 13. The second avatar object 150 corresponds to the first viewing user who has performed an operation of transmitting a gift output request within a predetermined period.

[0181] For example, each user device 12 may draw a video in either a first mode of drawing one second avatar object 150 or a second mode of drawing a plurality of second avatar objects 150 based on an instruction transmitted from the server 13. The condition for changing from the first mode to the second mode is, for example, that the number of viewing users of the video in the first mode exceeds the maximum number of people in the first mode. Or, the distribution user has performed an operation to change from the first mode to the second mode or an operation to change from the second mode to the first mode.

[0182] The processing load of one second avatar object 150 in the second mode is smaller than the processing load of one second avatar object 150 in the first mode. For example, the drawing data of one second avatar object 150 in the second mode may be data with fewer polygons compared to one first avatar object 110 in the first mode. Or, while one first avatar object 110 in the first mode is drawn with three-dimensional data, one second avatar object 150 in the second mode may be drawn with two-dimensional data. Therefore, the processing load when drawing a plurality of second avatar objects 150 in the second mode can be reduced. The number of viewing users may be the cumulative number of viewing users including viewing users who have stopped viewing the video midway, or the number of viewing users who are currently viewing the video.

[0183] In the second mode, among all the viewing users, a predetermined number of second avatar objects 150 may be displayed. For example, when the number of first viewing users is 50 or more, the second avatar objects 150 of 20 first viewing users may be displayed. And, the second avatar objects 150 of all the first viewing users may be displayed in multiple batches.

[0184] The second avatar object 150 may be arranged in front of the first avatar object 110 located on the stage. Or, the second avatar objects 150 may be arranged so as to surround the first avatar object 110.

[0185] When the first viewing user performs an operation for transmitting a gift output request on the viewing user device 12B, the viewing user device 12B transmits the gift output request to the server 13 together with the user ID of the first viewing user, the gift ID, and the like. The server 13 transmits the gift output request or the like to each user device 12. At this time, the server 13 transmits a request to make the second avatar object 150 of the first viewing user more prominent than other second avatar objects 150 to the user device 12. For example, the second avatar object 150 is displayed next to the first avatar object 110. When the first avatar object 110 is located on the stage, the second avatar object 150 is drawn using motion data for drawing an operation of the second avatar object 150 rising onto the stage.

[0186] <Operation of Passing a Gift Object during Collaborative Distribution> The distribution user may perform collaborative distribution. Collaborative distribution is distribution in which a plurality of distribution users participate. Examples of collaborative distribution include first collaborative distribution and second collaborative distribution. In the first collaborative distribution method, the distribution starts and participates in the distribution of the video as a plurality of distribution users. The first avatar objects 110 of these distribution users are displayed in the video. In the second collaborative distribution method, there are a host user who is a distribution user and a guest user who is a distribution user. The host user starts the distribution of the video and permits the guest user to participate in the video. The guest user obtains permission from the host user during the distribution of the video. Thereby, the first avatar object 110 of the guest user is displayed in the video. The host user is the first distribution user, and the guest user is the second distribution user. There may be a plurality of second distribution users.

[0187] The first viewing user can cause a second avatar object 150 corresponding to the first viewing user to be displayed in the video in which the collaborative distribution is being performed. That is, a plurality of first avatar objects 110 corresponding to each distribution user and one or more second avatar objects 150 can be displayed in the video. The conditions for the first avatar object 110 and the second avatar object 150 to be displayed in the video and the conditions for ending the display of the video are different. Specifically, the first avatar object 110 is displayed when the video distribution starts or with the permission of the distribution user. The second avatar object 150 is displayed when the viewing user sends a gift display request. Further, the first avatar object 110 becomes invisible when the video distribution ends, when any of the distribution users performs an exit operation on the first avatar object 110, or when a predetermined time for displaying the first avatar object 110 has elapsed. On the other hand, the second avatar object 150 becomes invisible when the invisible condition for making the second avatar object 150 invisible is satisfied. An example of the invisible condition is that the second avatar object 150 has completed its operation. The operation of the second avatar object 150 may be associated with a gift ID.

[0188] When the second avatar object 150 is displayed in the video in which the collaborative distribution is being performed, the second avatar object 150 may give a gift object to the plurality of first avatar objects 110.

[0189] An example of a method for giving a gift object to a plurality of first avatar objects 110 will be described. The first viewing user performs an operation of designating the first avatar object 110. The viewing user device 12B of the first viewing user transmits the gift ID, user ID, gift output request, and identification information of the designated first avatar object 110 to the server 13. The server 13 transmits at least the gift ID, gift output request, and identification information of the designated first avatar object 110 to the user device 12. Further, the server 13 transmits a request to each user device 12 to draw the second avatar object 150 by designating at least one of the automatic motion data, selection motion data, or tracking data. As the method for drawing the second avatar object using the automatic motion data, selection motion data, or tracking data, any of the methods described in the above embodiments can be used.

[0190] Each user device 12 displays the second avatar object 150 and the gift object 135 in the video. Further, the second avatar object 150 is moved to a position near the designated first avatar object 110. This position is, for example, in front of the first avatar object 110. The user device 12 draws the second avatar object using at least one of the automatic motion data, selection motion data, and tracking data. Then, the gift object 135 is associated with the first avatar object 110.

[0191] Another example of a method for giving a gift object to a plurality of first avatar objects 110 will be described. When the number of gift output requests sent by the first viewing user reaches the number of first avatar objects 110 displayed in the video, the server 13 sends output requests for the second avatar object 150 and the gift object 135 to each user device 12. After the first viewing user selects one gift object 135, the gift button 122 is repeatedly operated the same number of times as the number of distribution users displayed in the video. Alternatively, the first viewing user repeatedly operates a combination of selection of one gift object and operation of the gift button 122 the same number of times as the number of first avatar objects 110. For example, when two distribution users participate in the collaborative distribution, the first viewing user operates the gift button 122 twice. Thereby, the viewing user device 12B of the first viewing user transmits the same number of gift output requests as the number of times the gift button 122 is operated, together with the gift ID and the user ID.

[0192] The server 13 receives a gift output request or the like. Further, the server 13 compares the number of gift output requests received within a predetermined time with the number of distribution users participating in the video being viewed by the first viewing user. When the number of gift output requests reaches the number of distribution users, a request to draw the second avatar object 150 by designating at least one of automatic motion data, selection motion data, or tracking data is sent to each user device 12. As a method of drawing the second avatar object using the automatic motion data, selection motion data, or tracking data, any method described in the above embodiments can be used.

[0193] When the gift object 135 associated with the second avatar object 150 is associated with the first avatar object 110, the user device 12 moves the position of the second avatar object 150 to a position near the next first avatar object 110 with which the gift object 135 is not associated. This position is, for example, in front of the first avatar object 110. When the user device 12 moves the second avatar object 150, it draws the second avatar object 150 using at least one of automatic motion data, selection motion data, or tracking data. When using tracking data, after the second avatar object 150 has completed its movement, the second avatar object 150 is drawn using the tracking data of the first viewing user received by the server 13. When the user device 12 moves the second avatar object 150 to the next position, it associates a new gift object 135 with the second avatar object 150. And it repeats the same process as when passing the gift object 135 to the first first avatar object 110. The user device 12 repeats the association of the gift object 135 with the first avatar object 110, the movement, and the association of the new gift object 135 with the second avatar object 150 until the gift object 135 is associated with all the first avatar objects 110.

[0194] Another example of a method for giving a gift object to a plurality of first avatar objects 110 will be described. In the collaborative distribution, the server 13 remembers the order in which the first avatar objects 110 receive the gift object 135. The order may be randomly determined by the server 13. Or the order may be determined based on the position of the first avatar object 110. Or the order may be determined based on the priority of the first avatar object 110. When the server 13 receives the gift ID and the gift output request, it determines the first avatar object 110 that is to receive the gift object 135 first based on the order. Then, it transmits at least the gift ID, the gift output request, the identification information of the first avatar object 110, etc. to the user device 12. Each user device 12 displays the second avatar object 150 and the gift object 135 at a position near the first avatar object 110 corresponding to the received identification information of the first avatar object 110. Also, when the server 13 newly receives the gift ID and the gift output request, it determines the first avatar object 110 that is to receive the gift object 135 second based on the order. Then, it transmits the gift ID, the gift output request, the identification information of the first avatar object 110, etc. to the user device 12. Each user device 12 displays the second avatar object 150 and the gift object 135 at a position near the second avatar object 110 corresponding to the received identification information of the first avatar object 110. In this way, the server 13 transmits the gift output request and the identification information of the first avatar object 110, etc. to the user device 12 until it receives the same number of gift IDs and gift output requests as the number of the first avatar objects 110 from one viewing user device 12B. The user device 12 displays the second avatar object 150 and the gift object 135 at a position near the first avatar object 110 corresponding to the received identification information of the first avatar object 110.

[0195] Also, when the second avatar object 150 passes the gift object 135 in the collaborative distribution in this way, other first avatar objects 110 other than the first avatar object 110 associated with the gift object 135 may be drawn using automatic motion. For example, the automatic motion is an operation in which another first avatar object 110 stares at the gift object 135.

[0196] <Configuration of Information Processing System> · Although the sensor unit 24 was assumed to acquire detection data obtained by detecting changes in the user's facial expression, head movement, and the user's relative position with respect to the sensor unit 24, at least one of these may be used. Further, the sensor unit 24 may acquire other detection data such as detection data obtained by detecting movements other than head movement in addition to or instead of at least one of the user's facial expression change, head movement, and user relative position.

[0197] · The virtual space output to the video may be an augmented reality (AR) space. For example, animations such as avatar objects and gift objects based on data transmitted from the distribution user device 12A may be superimposed on a real-world image captured by the camera of the viewing user device 12B and output. Alternatively, the distribution user device 12A may generate a video by superimposing animations such as avatar objects and gift objects on a real-world image captured by its own camera, encode the video, and transmit it to the server 13.

[0198] · In each of the above embodiments, the user device 12 is an information processing device such as a smartphone, a mobile phone, a tablet terminal, a personal computer, a console game machine, a wearable computer such as a head-mounted display, etc. Instead of this, the information processing system may be a system provided in a studio for video distribution. The information processing system includes an information processing device, a server, a sensor unit 24 worn on the body of the distribution user, a tracking system that detects the position of the sensor unit 24, an operation unit 27, a speaker 25, and a display 28. The tracking system may include a multi-axis laser emitter that emits a pulse laser beam for synchronization. The sensor unit 24 includes a sensor that detects the laser beam, and detects its own position and orientation while synchronizing with the synchronization pulse. As the sensor unit 24, for example, Vive Tracker (registered trademark) and Vive Base Station (registered trademark) provided by HTC Corporation (registered trademark) can be used.

[0199] · The control unit 20 of the user device 12 is configured to execute a first output control process and a second output control process by executing an application program. At least one of these may be executed by the server 13 or another device constituting the information processing system 11. Also, the control unit 30 of the server 13 is configured to execute a first transmission process, a gift reception process, and a second transmission process. At least one of these may be executed by the user device 12 or another device constituting the information processing system 11.

[0200] The technical idea and its operational effects grasped from the above-described embodiments and modification examples are described below. [1] On one or more computers, a first output control process for outputting a video including a first avatar object corresponding to a distribution user to a display, a reception process for receiving a selection operation for selecting a gift object by a viewing user who views the video, Gift sending process for sending the gift identification information of the selected gift object and the identification information of the viewing user to the server together with the output request of the gift object Second output control process for outputting the video based on the data for outputting a video including the selected gift object, the first avatar object, and the second avatar object corresponding to the viewing user to the display Based on the identification information of the motion data for operating the second avatar object associated with the identification information of the gift object, identify the motion data, and use the motion data to draw the gift object and the second avatar object

[0201] [A] On one or more computers First output control process for outputting a video including a first avatar object corresponding to the distribution user to the display Reception process for receiving a selection operation for selecting a gift object by a viewing user who views the video Gift sending process for sending the gift identification information of the selected gift object and the identification information of the viewing user to the server together with the output request of the gift object Second output control process for outputting the video based on the data for outputting a video including the selected gift object, the first avatar object, and the second avatar object corresponding to the viewing user to the display In the second output control process Program for drawing the second avatar object using at least one of the tracking data of the viewing user and the selected motion data selected by the viewing user and preset automatic motion data

[0202] [B]In the second output control process, at least some parts of the second avatar object are drawn using the tracking data, and at least some other parts of the second avatar object are drawn using the automatic motion data, and the program according to [A].

[0203] [C]In the second output control process, after drawing the second avatar object using the automatic motion data, the second avatar object is drawn using the selected motion data based on the input instruction of the viewing user, and the program according to [A].

[0204] [D]In the second output control process, a selection unit for selecting the selected motion data is output, and the second avatar object is drawn using the selected motion data corresponding to the input instruction of the viewing user received by the selection unit, and the program according to [A] or [C].

[0205] [E]When outputting the gift object, if the viewing user who requested the output of the gift object is continuing to view the video, the second avatar object corresponding to the viewing user and the gift object are output, and if the viewing user is not continuing to view the video, the second avatar object is not output and the gift object is output, and the program according to any one of [A] to [D].

[0206] [F]When outputting the gift object, the display mode of the gift object is changed depending on whether the viewing user who requested the output of the gift object is continuing to view the video or not, and the program according to any one of [A] to [E].

[0207] [G]In the second output control process, the second avatar object is output using data with a lower processing load for drawing than the first avatar object, and the program according to any one of [A] to [F].

[0208] [H] When the viewing user who has transmitted the gift identification information to the server satisfies a predetermined condition, execute the second output control process, the program according to any one of [A] to [G].

[0209] [I] When the value of the parameter associated with the viewing user satisfies the predetermined condition regarding the parameter, execute the second output control process, the program according to [H].

[0210] [J] When the activity of the viewing user satisfies the predetermined condition regarding the activity, execute the second output control process, the program according to claim [H]. [K] In the distribution mode for distributing the video, when outputting the gift object in response to an output request for the gift object transmitted from another viewing user device, use at least one of the tracking data of the distribution user who distributes the video and the selected motion data selected by the distribution user, and the preset automatic motion data to draw the first avatar object corresponding to the distribution user, the program according to any one of [A] to [J].

[0211] [L] While the second avatar object is being displayed, when the distribution user device used by the distribution user transmits an output request for the gift object, output the gift object in association with the second avatar object based on the output request, the program according to any one of [A] to [K].

[0212] [M] The selected motion data is the selected motion data selected by the viewing user from a plurality of preset motion data, the program according to any one of [A] to [L].

[0213] [O] Associate the object data selected by the operation input of the viewing user with the second avatar object. In the second output control process, a program according to any one of [A] to [M], which draws the second avatar object based on the selected object data.

[0214] [P]One or more computers perform a first output control process of outputting, to a display, a video including a first avatar object corresponding to a distribution user; a reception process of receiving a selection operation for selecting a gift object by a viewing user who views the video; a gift transmission process of transmitting, to a server together with an output request for the gift object, gift identification information of the selected gift object and identification information of the viewing user; and a second output control process of outputting, to the display, the video based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, wherein in the second output control process an information processing method of drawing the second avatar object by using at least one of tracking data of the viewing user and selection motion data selected by the viewing user, and preset automatic motion data.

[0215] [Q]One or more computers perform a first transmission process of transmitting, to at least a viewing user device used by a viewing user who views the video, data for outputting a video including a first avatar object corresponding to a distribution user; a gift reception process of receiving, from a first viewing user device used by the first viewing user who is the viewing user, gift identification information of a gift object and identification information of the first viewing user together with an output request for the gift object; A second transmission process that transmits at least to a distribution user device used by the distribution user and a second viewer device used by a second viewer other than the first viewer: the gift identification information, the gift object corresponding to the gift identification information, and an output request for outputting a video including the first avatar object and a second avatar object corresponding to the first viewer. The output request transmitted in the second transmission process is transmitted together with data used for rendering the second avatar object or identification information of the data, and the data used for rendering the second avatar object includes at least one of tracking data of the first viewer and selected motion data selected by the first viewer, and preset automatic motion data. An information processing method.

[0216] [R] A first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a distribution user to at least a viewer device used by a viewer who views the video. A gift receiving unit that receives, from a first viewer device used by the first viewer who is the viewer, gift identification information of a gift object and identification information of the first viewer together with an output request for the gift object. A second transmission unit that transmits at least to a distribution user device used by the distribution user and a second viewer device used by a second viewer other than the first viewer: the gift identification information, the gift object corresponding to the gift identification information, and an output request for outputting a video including the first avatar object and a second avatar object corresponding to the first viewer. The output request transmitted by the second transmission unit is transmitted together with data used for rendering the second avatar object or identification information of the data, and the data used for rendering the second avatar object includes at least one of tracking data of the first viewer and selected motion data selected by the first viewer, and preset automatic motion data. An information processing apparatus. [1]Cause one or more computers to perform a first output control process of outputting, to a display, a video including a first avatar object corresponding to a distribution user; perform a reception process of receiving a selection operation for selecting a gift object by a viewing user who views the video; perform a gift transmission process of transmitting, to a server together with an output request for the gift object, gift identification information of the selected gift object and identification information of the viewing user; perform a second output control process of outputting, to the display, the video based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user, wherein, in the second output control process, a program for drawing the second avatar object using at least one of tracking data of the viewing user and selection motion data selected by the viewing user and preset automatic motion data. [2] The program according to [1], wherein, in the second output control process, at least some parts of the second avatar object are drawn using the tracking data, and at least some other parts of the second avatar object are drawn using the automatic motion data. [3] The program according to [1], wherein, in the second output control process, after drawing the second avatar object using the automatic motion data, the second avatar object is drawn using the selection motion data based on an input instruction of the viewing user. [4] The program according to [1] or [3], wherein, in the second output control process, a selection unit for selecting the selection motion data is output, and the second avatar object is drawn using the selection motion data corresponding to an input instruction of the viewing user received by the selection unit. [5] When outputting the gift object, if the viewing user who requested the output of the gift object is continuing to view the video, the second avatar object and the gift object corresponding to the viewing user are output; if the viewing user is not continuing to view the video, the second avatar object is not output and the gift object is output. The program according to [1]. [6] When outputting the gift object, the display mode of the gift object is changed depending on whether the viewing user who requested the output of the gift object is continuing to view the video or not. The program according to [1] or [5]. [7] In the second output control process, the second avatar object is output using data with a lower rendering processing load than the first avatar object. The program according to [1]. [8] When the viewing user who sent the gift identification information to the server satisfies a predetermined condition, the second output control process is executed. The program according to [1]. [9] When the value of the parameter associated with the viewing user satisfies the predetermined condition regarding the parameter, the second output control process is executed. The program according to [8].

[10] When the activity of the viewing user satisfies the predetermined condition regarding the activity, the second output control process is executed. The program according to [8].

[11] In the distribution mode in which the video is distributed, when outputting the gift object in response to an output request for the gift object sent from another viewing user device, at least one of the tracking data of the distribution user who distributes the video and the selected motion data selected by the distribution user, and the preset automatic motion data are used to draw the first avatar object corresponding to the distribution user. The program according to [1].

[12] When the distribution user device used by the distribution user transmits an output request for the gift object while the second avatar object is being displayed, the gift object is output in association with the second avatar object based on the output request. The program according to [1].

[13] The selection motion data is the selection motion data selected by the viewing user from a plurality of preset motion data. The program according to [1].

[14] Associate the object data selected by the operation input of the viewing user with the second avatar object. In the second output control process, the second avatar object is drawn based on the selected object data. The program according to [1].

[15] One or more computers A first output control process for outputting a video including a first avatar object corresponding to a distribution user to a display; A reception process for receiving a selection operation for selecting a gift object by a viewing user who views the video; A gift transmission process for transmitting the gift identification information of the selected gift object and the identification information of the viewing user to a server together with an output request for the gift object; A second output control process for outputting the video based on data for outputting a video including the selected gift object, the first avatar object, and a second avatar object corresponding to the viewing user to the display. In the second output control process, An information processing method for drawing the second avatar object using at least one of the tracking data of the viewing user and the selection motion data selected by the viewing user and preset automatic motion data.

[16] One or more computers A first transmission process for transmitting data for outputting a video including a first avatar object corresponding to a distribution user to at least a viewing user device used by a viewing user who views the video; A gift reception process for receiving, from a first viewing user device used by the first viewing user who is the viewing user, gift identification information of a gift object and identification information of the first viewing user together with an output request for the gift object; A second transmission process for transmitting the gift identification information, an output request for outputting a video including the gift object corresponding to the gift identification information, the first avatar object, and a second avatar object corresponding to the first viewing user to at least a distribution user device used by the distribution user and a second viewing user device used by a second viewing user other than the first viewing user; The output request transmitted in the second transmission process is transmitted together with data used for rendering the second avatar object or identification information of the data, and the data used for rendering the second avatar object includes at least one of tracking data of the first viewing user and selected motion data selected by the first viewing user, and preset automatic motion data. An information processing method.

[17] A first transmission unit for transmitting data for outputting a video including a first avatar object corresponding to a distribution user to at least a viewing user device used by a viewing user who views the video; A gift reception unit for receiving, from a first viewing user device used by the first viewing user who is the viewing user, gift identification information of a gift object and identification information of the first viewing user together with an output request for the gift object; A second transmission unit that transmits, to at least the distribution user device used by the distribution user and the second viewing user device used by a second viewing user other than the first viewing user, the gift identification information, a video including the gift object corresponding to the gift identification information, the first avatar object, and the second avatar object corresponding to the first viewing user, and an output request for outputting the video The output request transmitted by the second transmission unit is transmitted together with data used for rendering the second avatar object or identification information of the data, and the data used for rendering the second avatar object includes at least one of tracking data of the first viewing user and selection motion data selected by the first viewing user, and preset automatic motion data. An information processing apparatus

Explanation of Signs

[0217] 11… Information processing system 12… User device 13… Server

Claims

1. One or more computers are caused to perform: a first output control process for outputting to a display a video including a first avatar object corresponding to a first user; a gift sending process for, when a second user who views the video satisfies a predetermined condition, transmitting, based on an operation of the second user, identification information of the second user to a server together with a request for output of a gift object; a second output control process for outputting to the display the video based on data for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user. A program.

2. The computer is further caused to perform: a reception process for receiving a selection operation of the gift object by the second user, and in the gift sending process, when the second user satisfies the predetermined condition, transmitting gift identification information of the selected gift object and a request for output of the gift object. The program according to claim 1.

3. The program according to claim 1, wherein the gift sending process is executed when a value of a parameter associated with the second user satisfies the predetermined condition regarding the parameter.

4. The program according to claim 1, wherein the gift sending process is executed when an activity of the second user satisfies the predetermined condition regarding the activity.

5. In the second output control process, the video including the second avatar object using at least one of tracking data of the second user, selection motion data selected by the second user, and preset automatic motion data is output to the display. The program according to claim 1.

6. In the second output control process, the video including the second avatar object using at least one of tracking data of the second user and selection motion data selected by the second user and preset automatic motion data is output to the display. The program according to claim 1. **Claim 7**: The program according to claim 1, wherein in the second output control process, the video including the second avatar object, in which some parts are drawn using the tracking data of the second user and other parts are drawn using the automatic motion data, is output to the display. **Claim 8**: The program according to claim 1, wherein in the second output control process, after drawing the second avatar object using preset automatic motion data, the video in which the second avatar object is drawn using the selected motion data selected by the second user is output to the display. **Claim 9**: The program according to claim 1, wherein in the second output control process, a selection unit for selecting motion data is output, and the video in which the second avatar object is drawn using the selected motion data corresponding to the input instruction of the second user received by the selection unit is output to the display. **Claim 10**: The program according to claim 1, wherein in the second output control process, when outputting the video including the gift object to the display, if the second user who requested the output of the gift object is continuing to watch the video, the video including the second avatar object and the gift object corresponding to the second user is output; if the second user is not continuing to watch the video, the video including the gift object is output without outputting the second avatar object. **Claim 11**: The program according to claim 1, wherein in the second output control process, when outputting the video including the gift object to the display, the video in which the display mode of the gift object is changed is output depending on whether the second user who requested the output of the gift object is continuing to watch the video or not. **Claim 12**: The program according to claim 1, wherein in the second output control process, the second avatar object is output using data with a lower processing load for drawing than the first avatar object.

13. The program according to claim 1, further causing the computer to execute a process of outputting a video in which the first avatar object corresponding to the first user is drawn using at least one of the tracking data of the distribution user who distributes the video and the selected motion data selected by the first user and preset automatic motion data when outputting the gift object in response to an output request for the gift object transmitted from another second user device in the distribution mode for distributing the video.

14. In the second output control process, when the first user device used by the first user transmits an output request for the gift object while the second avatar object is being displayed, the video including the gift object associated with the second avatar object is output based on the output request. The program according to claim 1.

15. In the second output control process, the program according to claim 1, outputs a video in which the second avatar object is drawn based on object data selected by an operation input of the second user.

16. One or more computers A first output control process for outputting a video including a first avatar object corresponding to a first user to a display; A gift sending process for transmitting the identification information of the second user to a server together with an output request for a gift object based on an operation of the second user when the second user who views the video satisfies a predetermined condition; An information processing method for executing a second output control process for outputting the video to the display based on data for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user.

17. One or more computers A first transmission process for transmitting at least data for outputting a video including a first avatar object corresponding to a first user to a second user device used by a second user who views the video; A gift receiving process for receiving the identification information of the second user from the second user device used by the second user who satisfies a predetermined condition together with an output request for a gift object. An information processing method for executing: a second transmission process of transmitting an output request for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user to at least a first user device used by the first user and a second user device used by the second user other than the second user satisfying the predetermined condition.

18. A first transmission unit that transmits data for outputting a video including a first avatar object corresponding to a first user to at least a second user device used by a second user who views the video. A gift reception unit that receives identification information of the second user from the second user device used by the second user satisfying a predetermined condition together with an output request for a gift object. An information processing system including: a second transmission unit that transmits an output request for outputting a video including the gift object, the first avatar object, and a second avatar object corresponding to the second user to at least a first user device used by the first user and a second user device used by the second user other than the second user satisfying the predetermined condition.

Images