Video application program, video object drawing method, video management program, video management method, server, and video distribution system

Abstract

To increase user satisfaction with a drawing function for objects requested to be displayed.SOLUTION: A moving image distribution system 11 includes: a display request receiving unit that receives a display request for an object based on an operation of a user viewing a moving image from a terminal used by the user together with the object's identification information; a group determination unit that determines a group to which the object that is a subject of the display request belongs; a management unit that adds the object's identification information to a waiting queue corresponding to the determined group; a determination unit that determines, among a plurality of drawing positions corresponding to the group, a drawing position at which an object at the head of the waiting queue should be drawn according to order assigned to each drawing position; and an execution unit that renders the object at the drawing position determined by the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video application program, a video object drawing method, a video management program, a video management method, a server, and a video distribution system. [Background technology]

[0002] 2. Description of the Related Art There has been known a video distribution system that distributes videos including animations of avatar objects based on the movements of a user (see, for example, Patent Document 1). In this type of video distribution system, a function has been proposed for displaying an arbitrary object in a video in response to a display request sent from a viewing user who is viewing the video. The arbitrary object includes a gift object given by the viewing user to an avatar object (or a distributor user who distributes the video). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-184689 A Summary of the Invention [Problem to be solved by the invention]

[0004] The display position of the gift object is often determined according to the type of the gift object. For example, as shown in FIG. 16, it is predetermined that the gift objects 1 to 3 are displayed in the first area 4, the second area 5, and the third area 6, respectively. As shown in FIG. 17, when a large number of display requests for the gift object 1 are made from the viewing user, the gift object may be displayed in a state where it is difficult to see or inconspicuous, such as one gift object 1 being hidden by another gift object 1 in the first area 4, or the gift object 1 becoming invisible immediately after being displayed. In this way, when the gift object is displayed in the video in a state unintended by the viewing user, the viewing user may be less satisfied with the gift giving. In addition, when the gift object given by the viewing user is displayed in a state where it is difficult to see or inconspicuous, the distribution user may also be less satisfied with the function of the video distribution system.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to increase a user's satisfaction with the rendering function for an object requested to be displayed. [Means for solving the problem]

[0006] A video application program that solves the above problem causes a computer to function as a receiving unit that receives identification information of an object transmitted by a first user device used by a first user watching a video, a determination unit that determines a drawing position at which the object should be drawn based on the received identification information, among a plurality of drawing positions corresponding to a group to which the object belongs, in accordance with the order assigned to each of the drawing positions, and an execution unit that renders the object at the drawing position determined by the determination unit.

[0007] A method for drawing objects in videos that solves the above problem includes a receiving step in which a computer receives identification information of an object transmitted by a first user device used by a first user watching the video, a determination step in which a drawing position at which the object should be drawn is determined, based on the received identification information, from among a plurality of drawing positions corresponding to a group to which the object belongs, in accordance with the order assigned to each of the drawing positions, and an execution step in which the object is rendered at the drawing position determined by the determination unit.

[0008] A video management program that solves the above problem causes a computer to function as a request receiving unit that receives an object display request and identification information of the object that is the subject of the display request transmitted by a first user device used by a first user watching the video, a group determination unit that accepts the object identification information and the object display request based on the operation of the user watching the video and determines the group to which the object belongs, a management unit that adds the object identification information to a waiting queue corresponding to the determined group, and a transmission unit that transmits the identification information of the object included in the waiting queue to a second user device used by a second user distributing the video.

[0009] A video management method that solves the above problem includes a request receiving step in which a computer receives an object display request and identification information of the object that is the subject of the display request transmitted by a first user device used by a first user watching the video, a group determination step in which the computer accepts the object identification information and the object display request based on an operation of the user watching the video and determines the group to which the object belongs, a management step in which the identification information of the object is added to a waiting queue corresponding to the determined group, and a transmission step in which the computer transmits a display request for the object included in the waiting queue to a second user device used by a second user distributing the video.

[0010] A server that solves the above problem includes a request receiving unit that receives a request to display an object and identification information of the object that is the subject of the display request transmitted by a first user device used by a first user watching a video, a group determination unit that accepts the identification information of the object and the request to display the object based on an operation of the user watching the video and determines the group to which the object belongs, a management unit that adds the identification information of the object to a waiting queue corresponding to the determined group, and a transmission unit that transmits a request to display the object included in the waiting queue to a second user device used by a second user distributing the video.

[0011] A video distribution system that solves the above problem includes a display request receiving unit that receives a display request for an object from a terminal used by a user watching a video together with identification information of the object based on operation of the user, a group determination unit that determines the group to which the object that is the subject of the display request belongs, a management unit that adds the identification information of the object to a waiting queue corresponding to the determined group, a determination unit that determines a drawing position at which the object at the head of the waiting queue should be drawn, among a plurality of drawing positions corresponding to the group, in accordance with the order assigned to each of the drawing positions, and an execution unit that renders the object at the drawing position determined by the determination unit. Effect of the Invention

[0012] According to the present invention, it is possible to increase a user's satisfaction with the rendering function for an object requested to be displayed. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram showing an embodiment of a video distribution system. [Diagram 2] FIG. 4 is a diagram showing the configuration of user management information in the embodiment. [Diagram 3] FIG. 13 is a diagram showing gift object information according to the embodiment. [Figure 4]5 is a schematic diagram showing the positional relationship between a user device and a user in real space in the embodiment. [Diagram 5] 10 is a flowchart showing the procedure of a process for displaying a gift object in the embodiment. [Figure 6] 10 is a flowchart showing the procedure of a process for displaying a gift object in the embodiment. [Figure 7] FIG. 4 is a conceptual diagram showing a rendering queue in the embodiment. [Figure 8] FIG. 11 is a conceptual diagram showing an example of a drawing position of a gift object in the embodiment. [Figure 9] FIG. 11 is a diagram of a gift object drawn in the same embodiment. [Figure 10] FIG. 11 is a conceptual diagram showing an example of a drawing position of a gift object in the embodiment. [Figure 11] FIG. 11 is a diagram of a gift object drawn in the same embodiment. [Figure 12] FIG. 4 is a diagram showing an example of a video viewing screen in the embodiment. [Figure 13] FIG. 4 is a diagram showing an example of a video viewing screen in the embodiment. [Figure 14] 13 is a conceptual diagram showing a modified example of the drawing position of the gift object. [Figure 15] 13 is a conceptual diagram showing a modified example of the drawing position of the gift object. [Figure 16] FIG. 13 is a conceptual diagram showing the drawing positions of conventional gift objects. [Figure 17] FIG. 13 is a schematic diagram showing a conventional drawing state of a gift object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of a video distribution system will be described with reference to the drawings. As shown in Fig. 1, a video distribution system 11 includes a user device 12 and a server 13. The video distribution system 11 is a system that displays videos on the user device 12 by transmitting and receiving data between a plurality of user devices 12 and the server 13 via a network 14. A video application program for viewing and distributing videos is installed on the user device 12. Hereinafter, a user who distributes a video is referred to as a "distributing user," and a user who views a video distributed by a distributing user is referred to as a "viewing user," and when there is no distinction between these, they are simply referred to as "users."

[0015] The user device 12 is a smartphone (multi-function phone terminal), a tablet terminal, a personal computer, a console game machine, a wearable computer such as a head-mounted display, and an information processing device other than these devices that can play back moving images.

[0016] The user device 12 includes a computer processor 20, a memory 21, a storage 22 (storage medium), and a communication interface (I / F) 23. The computer processor 20 is an arithmetic device that loads an operating system and other programs from the storage 22 or other storage into the memory 21 and executes instructions retrieved from the memory 21. The computer processor 20 is one or more arithmetic circuits such as a central processing unit (CPU), a graphic processing unit (GPU), or a neural network processing unit (NPU). The communication I / F 23 can transmit and receive data to and from the server 13 and other user devices 12 via a network 14. The network 14 includes various networks such as a local area network and the Internet. The communication I / F 23 is implemented as hardware, software, or a combination of these.

[0017] The memory 21 is a main storage device (recording medium) that can be directly read from and written to by the computer processor 20, and is composed of a semiconductor memory or the like. The storage 22 is an auxiliary storage device (recording medium), and is, for example, a storage medium such as a magnetic disk, an optical disk, or a semiconductor memory. The storage 22 can also be used in combination with a plurality of storages. The storage 22 stores a video application program 22A, avatar data 22B for drawing avatar objects, and object information 22C.

[0018] The video application program 22A is switchable, in response to a user's instruction, between a distribution mode in which the user distributes a video by himself / herself, and a viewing mode in which the user views a video distributed by another distribution user.

[0019] The avatar data 22B is three-dimensional model data for drawing an avatar object. The user device 12 acquires the avatar data 22B from the server 13 at a predetermined timing, such as when the video application program 22A is started. The avatar data includes data for drawing an avatar object body, texture data applied to the avatar object body, and the like. The data for drawing the avatar object body includes polygon data, skeletal data (bones) for expressing the movement of the avatar object, and the like. The avatar data 22B may include data for drawing a plurality of avatar object bodies. The texture data includes a plurality of parts data applicable to the avatar object. For example, a plurality of parts data is prepared for each category, such as "eyes," "eyebrows," "ears," and "clothes." The user selects the parts data and applies it to the avatar object body to create an avatar object. The combination of the avatar object body and the parts data selected by the user is recorded in the storage 22.

[0020] The object information 22C is data downloaded from the server 13 at a predetermined timing. The object information 22C is information about objects other than avatar objects. The objects other than avatar objects are distinguished according to the usage patterns of the users. Specifically, in addition to objects that the distribution user can display, the object information includes gift objects provided by the viewing user to the distribution user. The gift objects include effect objects, wearing objects, and normal objects. The effect objects are objects that affect the overall impression of the viewing screen of the distribution video, such as objects imitating confetti and objects imitating fireworks. The wearing objects are objects that are displayed on the display screen in association with a specific part of the avatar object. The wearing objects are, for example, accessories (hat, glasses, etc.), clothes (T-shirt, etc.), costumes that are attached to the wearing objects, and objects that can be attached to the avatar objects other than these. The wearing objects are provided by the viewing user and then attached to the avatar object by being selected by the distribution user. The normal objects are gift objects other than the wearing objects that are placed as objects in the virtual space, such as objects imitating stuffed toys and bouquets of flowers.

[0021] The object may also be an object that performs a physical behavior. Specifically, the object may simulate a physical behavior such as a fall due to gravity, a movement due to the application of a force, etc. A component (program) for simulating the physical behavior is associated with the physical behavior object in an executable manner.

[0022] The user device 12 also includes a sensor unit 24, a speaker 25, a microphone 26, an operation unit 27, and a display 28. At least one of the sensor unit 24, the speaker 25, the microphone 26, the operation unit 27, and the display 28 may be provided as the same device as the computer processor 20, or may be provided in a form connected to a device including the computer processor 20.

[0023] The sensor unit 24 is a group of sensors that detect face motions indicating changes in the user's facial expression and body motions indicating changes in the relative position of the user's body with respect to the sensor unit 24. Face motions include blinking, opening and closing of the mouth, and other movements. For example, the sensor unit 24 has at least one of an infrared camera that emits infrared rays toward the user and detects infrared rays reflected from the user's face, a camera that photographs the user's face, and an image processing unit that performs image processing on the data photographed by the camera. Alternatively, the sensor unit 24 may be a sensor other than the above, or may further be combined with sensors other than the above sensors, such as an acceleration sensor or a gyro sensor. Hereinafter, the detection data of face motion and the detection data of body motion are referred to as "motion data".

[0024] The speaker 25 converts voice data into voice and outputs it. The microphone 26 inputs voice spoken by the user and converts it into voice data. The operation unit 27 is a touch panel integrated with the display 28, operation buttons provided on the housing of the user device 12, a keyboard, a mouse, a controller, etc. The display 28 outputs various images in response to output instructions from the computer processor 20.

[0025] The computer processor 20 executes a video application program 22A stored in the storage 22, thereby functioning as an application management unit 20A, a motion data generation unit 20B, and a display control unit 20C. The application management unit 20A manages the video application program. The application management unit 20A inputs a user request input through the operation unit 27 or a request from the server 13, and outputs a request to the motion data generation unit 20B or the display control unit 20C according to the content of the request. The application management unit 20A also transmits requests and various data from the motion data generation unit 20B and the display control unit 20C to the server 13, and records various data received from the server 13 in the storage 22.

[0026] The motion data generating unit 20B acquires the motion data from the sensor unit 24. In addition, the motion data generating unit 20B transmits the acquired detection data to the server 13.

[0027] In the distribution mode, the display control unit 20C applies motion data to the avatar data 22B to generate an animation including an avatar object that moves in accordance with the movement of the distribution user. The display control unit 20C also performs rendering including the avatar object and objects other than the avatar object. The rendering here refers to a drawing process including acquisition of the position of the virtual camera, perspective projection, hidden surface removal (rasterization), and the like. The rendering may be at least one of these processes, or may include processes such as shading and texture mapping. The display control unit 20C also generates an animation using the rendered image. Then, the display control unit 20C outputs to the display 28 video data that is synthesized (superimposed) with a posted message transmitted from the user device 12 of the viewing user or a notification transmitted from the server 13, and the like.

[0028] Furthermore, the display control unit 20C synchronizes the audio data and the video data input from the microphone 26 and outputs them to the display 28. Furthermore, the display control unit 20C encodes the video data and the audio data. The application management unit 20A transmits the encoded data to the server 13.

[0029] In addition, in the viewing mode, the display control unit 20C receives encoded video data and audio data from the server 13. In addition, the display control unit 20C decodes the received data, synchronizes the video data and audio data, and outputs them to the display 28 and the speaker 25.

[0030] Next, the server 13 will be described. The server 13 is used by a service provider or the like that provides a service for distributing videos. The server 13 includes a computer processor 30, a memory 31, a storage 32, and a communication I / F 33. The computer processor 30 has a similar configuration to the computer processor 20 of the user device 12. The computer processor 30 may be configured by combining a plurality of arithmetic circuits. The memory 31, the storage 32, and the communication I / F 33 have the same configuration as the user device 12. The server 13 may be configured by a plurality of devices, and the computer processor 30, the memory 31, the storage 32, and the communication I / F 33 may be provided in a distributed manner in those devices.

[0031] A distribution program 32A is stored in the storage 32. The computer processor 20 executes the distribution program 32A to function as a distribution management unit 30A and a purchase processing unit 30B. The distribution program 32A corresponds to a video management program.

[0032] The distribution management unit 30A stores various data received from the user device 12 in the storage 32, and outputs a request to the purchase processing unit 30B based on a purchase request or the like received from the user device 12. Furthermore, the distribution management unit 30A acquires data requested by the user device 12 from the storage 32 or the like and transmits the data to the user device 12. Specifically, the distribution management unit 30A transmits a list of videos being distributed in response to a request from the user device 12. When the distribution management unit 30A receives identification information of a video requested from the list from the user device 12, it transmits a distribution request to a user device used by a distribution user of the requested video. In addition, the distribution management unit 30A receives posted messages and the like posted by viewing users to videos being distributed. Furthermore, the distribution management unit 30A transmits the received posted messages to the user device 12 of the distribution user and the user device 12 of the viewing user. In addition to the content of the message, the posted message includes identification information of the viewing user (such as the user's account name), the posting date and time, and the like. Along with the posted message, a notification message stating "User A has entered the room" will also be displayed.

[0033] The distribution management unit 30A receives a display request for an object on the video being viewed from the user device 12 used by the viewing user who views the video. The object for which a display request is made is an object displayed in a virtual space, and includes a gift object provided by the viewing user to the broadcasting user who broadcasts the video, an evaluation object that favorably evaluates the video, a support object for supporting the broadcasting user, and the like. The gift object may be one for which a display request can be made without paying a fee, or one for which a display request can be made by paying a fee. Alternatively, the gift object may be one for which a fee is paid when the gift object is displayed in response to the display request. The distribution management unit 30A transmits a display request for the gift object to the user device 12 of the broadcasting user together with the identification information of the gift object. The server 13 transmits a notification message such as "User B has gifted fireworks" to the user device 12 of the broadcasting user and the user device 12 of the viewing user at a predetermined timing, such as when the display request for the gift object is accepted by the server 13, or when a predetermined operation is performed by the broadcasting user in response to the display request.

[0034] The purchase processing unit 30B performs a process of purchasing objects in accordance with a user's operation. The purchase processing includes a purchase in which an object is obtained by paying a consideration (medium) such as coins, points, or tickets that can be used within a video application program. The purchase processing may also include a process of exchanging, selling, or transferring an object. The purchase processing unit 30B may also execute a lottery (gacha) in which a predetermined number of objects are selected from a plurality of objects upon payment of a consideration. The purchase processing unit 30B records the purchased object in the user device 12 or the server 13 as being owned by the user. The sales revenue of the purchasable objects is distributed to, for example, the distribution user and the service provider.

[0035] Next, there will be described various types of data stored in the storage 32 of the server 13. The storage 32 stores user management information 32B, avatar data 32C, and object information 32D.

[0036] User management information 32B is information about a user who uses a video application program. Avatar data 32C is master data for drawing an avatar object, and is transmitted to user device 12 in response to a request from user device 12.

[0037] The object information 32D is master data for drawing a gift object, and is transmitted to the user device 12 in response to a request from the user device 12. The object information 32D includes attribute information of the gift object, in addition to data for drawing the gift object, such as polygon data.

[0038] Next, the user management information 32B will be described with reference to Fig. 2. The user management information 32B is stored for each user. The user management information 32B includes user identification information (user ID and password), coins, points, etc. Coins are purchased by the user. Points are given to the user at a predetermined timing, such as when a broadcast is made, when an event is participated in, or when an object or the like is sold.

[0039] With reference to FIG. 3, the attribute information of objects contained in object information 32D will be described. This attribute information includes an object ID, a group, an order, and a drawing position. The object ID is identification information of a gift object. A group is a group to which a gift object belongs. A group is created in advance by a service provider or an administrator who operates the service. There are no particular limitations on the criteria for creating a group, and groups may be created for each category, type, or attribute of a gift object. Note that gift objects also include gift objects that are not associated with a group.

[0040] Furthermore, each group is associated with a plurality of drawing positions. The drawing positions are positions at which gift objects are drawn, and are indicated by virtual space coordinates. The drawing positions are expressed, for example, by three-dimensional coordinates in the virtual space. Furthermore, the drawing positions are assigned an order in which they are drawn. For example, "Hanging scroll 1", "Hanging scroll 2", etc. belong to "Group 1". Furthermore, "Group 1" is associated with "First drawing position" through "Fifth drawing position". For example, the "First drawing position" through "Fifth drawing position" are assigned the order of "1" through "5". The gift objects belonging to a group are placed at one drawing position selected from the plurality of drawing positions associated with the group.

[0041] Next, a method for distributing videos using the user device 12 will be described with reference to Fig. 4. The user device 12 used for distributing and viewing videos will be described as a smartphone. The broadcast user 100 faces the user device 12 so that at least his / her face is included in the detection range 101 of the sensor unit 24, and executes a broadcast application program. At this time, the broadcast user 100 sets the broadcast application program to a broadcast mode.

[0042] The user device 12 used by the broadcasting user 100 applies the motion data output from the sensor unit 24 to the avatar data 22B to generate an animation. A video of the virtual space including the avatar object is displayed on the display 28 of the user device 12. The angle of the virtual camera in the virtual space can be changed based on the user's operation. The position of the avatar object changes according to the relative distance L between the broadcasting user 100 and the sensor unit 24. The change in the relative distance L is detected as a change in distance in the positive x-axis direction, negative x-axis direction, positive z-axis direction, and negative z-axis direction in FIG. 4. The user device 12 encodes the generated animation and transmits the encoded data to the server 13.

[0043] The server 13 distributes the encoded data to the user device 12 used by the viewing user 102 via the server 13. The viewing user 102 performs an operation to give the gift object to the video, and the user device 12 transmits a display request for the gift object to the server 13. The server 13 transmits the display request for the gift object received from the user device 12 to the user device 12 of the broadcasting user 100. The user device 12 of the broadcasting user 100 transmits an animation including the gift object to the server 13 based on the display request.

[0044] (operation) Next, the processing procedure for displaying a gift object will be described with reference to Fig. 5 and Fig. 6. In this embodiment, Fig. 5 shows the processing procedure executed by the server, and Fig. 6 shows the processing procedure executed by the user device 12 of the broadcasting user. The server 13 receives identification information of a gift object and a display request for the gift object from the user device 12 in viewing mode (step S1).

[0045] The server 13 refers to the object information 32D based on the identification information of the gift object, and determines the group to which the gift object for which the display request is made belongs (step S2). Furthermore, the server 13 adds the identification information of the gift object to the waiting queue corresponding to the determined group (step S3).

[0046] Furthermore, when the server 13 adds a gift object to the waiting queue, it transmits a display request for the gift object at the head of the waiting queue to the user device 12 of the broadcasting user. At this time, the server 13 may transmit a list of the gift objects included in the waiting queue. This list also includes the order of the gift objects in the waiting queue.

[0047] 6, user device 12 of the broadcasting user renders objects such as a gift object and a character object at the head of the waiting line based on the display request received from server 13 (step S4). User device 12 also outputs the rendered animation as a moving image to display 28 together with the voice data, posted message, etc. received from server 13 (step S5).

[0048] Next, the processes in steps S3 and S4 will be described in detail with reference to FIGS. As shown in FIG. 7, when the server 13 receives a display request for gift objects such as "Hanging Scroll 1" and "Hanging Scroll 2" from the user device 12 of the viewing user, the server 13 adds the identification information 111, 112 of those gift objects to the rendering queue 110 of the group to which the gift objects belong. The server 13 adds the identification information 111, 112 to the rendering queue 110 in the order in which the display requests for the gift objects were received. Furthermore, when the server 13 receives another display request for a gift object such as "Hanging Scroll 3", the server 13 adds the identification information 113 to the end of the rendering queue 110. Furthermore, the server 13 transmits a display request for "Hanging Scroll 1" at the top of the rendering queue 110 to the user device 12 of the broadcasting user, and moves up the order of the other gift objects included in the rendering queue 110.

[0049] As shown in FIG. 8, the first drawing position 116 to the fifth drawing position 120 in the virtual space are set for the group to which the gift object "hanging scroll" belongs. The first drawing position 116 to the fifth drawing position 120 are provided at positions that do not interfere with the drawing of the avatar object 130 as viewed from the virtual camera. In the example of FIG. 8, the first drawing position 116 to the fifth drawing position 120 are arranged in a line. The drawing positions 116 to 120 are set with intervals between them, taking into consideration the size of the gift objects belonging to the group, so that the gift objects do not overlap each other and look good. Each of the first drawing position 116 to the fifth drawing position 120 is assigned a drawing order such as "1" to "5". For example, as shown in FIG. 8, the first order may be assigned to the center position, and the subsequent orders may be assigned alternately to the left and right. Alternatively, the drawing order may be assigned in ascending or descending order from left to right.

[0050] When a gift object is not displayed at any of the first drawing position 116 to the fifth drawing position 120, the user device 12 of the broadcasting user sets the position of the gift object of "Hanging Scroll 1" to the drawing position 116 assigned with the first number "1". Next, the user device 12 sets the position of the gift object of "Hanging Scroll 2" to the drawing position 117 assigned with the second number. Similarly, the user device 12 assigns the positions of the gift objects of "Hanging Scroll 3" and "Hanging Scroll 4" to the vacant drawing positions in order of priority. The "highest priority" drawing position here is the drawing position assigned with the smallest number among the vacant drawing positions. In addition, the user device 12 draws the gift objects 131 to 133 that do not belong to any group at the drawing positions set for them.

[0051] As shown in Fig. 9, the gift objects 140-141 are rendered at the drawing positions corresponding to the groups and drawn as animation, so that the gift objects 140-141 are displayed side by side in the horizontal direction of the screen without overlapping each other and look good. No gift object is displayed at the drawing position 120 at the right end in Fig. 9. When the user device 12 receives a display request for a new gift object belonging to the same group via the server 13, the gift object is displayed at the drawing position 120. When the gift objects 140-141 are arranged at the drawing positions 120, they may be made to fall at a constant speed from the top to the bottom of the screen.

[0052] Furthermore, the gift object exits from the drawing position when an exit condition is met. The exit condition is, for example, when the display time elapsed from the time when the display of the gift object started reaches an upper limit value. When the gift objects are displayed at all of the drawing positions 116 to 120, the user device 12 executes an effect of erasing the gift object or moving it from the drawing position when the elapsed time of any of the gift objects reaches an upper limit value. When the gift object exits from the drawing position, the distribution management unit 30A transmits a display request for the gift object at the head of the waiting queue for rendering to the user device 12 of the broadcast user. Alternatively, the user device 12 of the broadcast user renders the gift object at the head of the waiting queue at an empty drawing position based on the list.

[0053] 9, when the drawing position 120 with the lowest priority is vacant and the gift object 140 at the drawing position with the highest priority has left, a new gift object may be drawn at the position where the gift object 140 had been drawn, instead of at the drawing position 120. Alternatively, a new gift object may be drawn at the drawing position 120 where no gift object has been displayed for the longest time.

[0054] The exit condition may be other than the display time reaching the upper limit, for example, the broadcast user intentionally or unintentionally performing an operation to exit the gift object using the user device 12, the viewing user performing an operation to exit the gift object, the number of gift objects added to the waiting queue reaching a predetermined number, etc.

[0055] 10 and 11 are examples of arrangement of gift objects belonging to a group different from the group to which the above-mentioned gift objects 140-141 belong. In the example shown in FIG. 10, the first drawing position 151 to the sixth drawing position 156 are arranged in a plurality of rows. The first drawing position 151 to the sixth drawing position 156 are arranged in a staggered manner as viewed from the virtual camera. The first drawing position 151 to the sixth drawing position 156 are different in height position in each row in the direction from the bottom to the top of the screen (positive direction of the Y axis in FIG. 10). Specifically, the rows are higher in the order of the first drawing position 151, the second drawing position 152, and the third drawing position 153, and the fourth drawing position 154 to the sixth drawing position 156.

[0056] FIG. 11 shows a state where the gift objects 161 to 166, which are "tower cake 1" to "tower cake 6", are arranged at the first drawing position 151 to the sixth drawing position 156. If all the gift objects 161 to 166 are arranged side by side, they cannot be accommodated within the screen. In addition, it is difficult to display the gift objects 161 to 166 within the screen without overlapping each other. In response to this, by shifting the first drawing position 151 to the sixth drawing position 156 in a staggered manner as viewed from the camera position side, it is possible to minimize overlapping parts and display the gift objects 161 to 166 in an orderly manner. In addition, it is possible to display many gift objects 161 to 166 within the screen. By displaying the gift objects 161 to 166 within the screen in this manner with high density, it is possible to display as many gift objects given by viewing users as possible within the distribution time of the video. In addition, when an upper limit is set for the display time of one gift object, it is possible to extend the time for which each gift object is displayed as much as possible. In addition, by shifting tall objects such as the gift objects 161 to 166 in the height direction, the gift objects behind them become easier to see. Therefore, a viewing user who has given a gift object behind them can recognize his / her own gift object on the video, and thus becomes more satisfied with the gift.

[0057] Note that the number of drawing positions set for objects belonging to a group does not necessarily have to be multiple. For example, since only one "wallpaper" object can be displayed in the virtual space, one drawing position is set for the "wallpaper" object. When multiple "wallpaper" objects are gifted, those gift objects are added to the rendering queue for the "wallpaper" object group. Then, when a predetermined time has passed since the "wallpaper" object was drawn at the drawing position, it is erased or moved, and the first "wallpaper" object waiting in the queue is rendered.

[0058] Next, with reference to FIG. 12 and FIG. 13, a description will be given of screens displayed on the display 28 of the user device 12 used by the viewing user. 12, an avatar object 201 is displayed on viewing screen 200. Avatar object 201 changes its facial expression in response to changes in the broadcast user's facial expression. In addition, avatar object 201 moves based on changes in the relative position of the broadcast user with respect to user device 12.

[0059] Displayed on viewing screen 200 are posting field 170 for posting messages and messages 171 posted by viewing users using posting field 170 while viewing a video. Also displayed on viewing screen 200 are rating button 172 for giving a good rating to the video and gift button 173 for giving a gift object to the video. When the viewing user operates gift button 173, user device 12 accepts the input operation and displays a gift list, which will be described later.

[0060] When the server 13 receives a display request for gift objects transmitted from the user device 12 of the viewing user, the gift objects 140-143, 180 are displayed on the screen. The gift objects 140-143 that belong to a group are displayed at a drawing position corresponding to the group. The gift object 180 that does not belong to a group is displayed in the other area. Furthermore, the wearable object is displayed in association with a predetermined part of the avatar object 201. The gift object that does not belong to a group may be displayed at a position randomly selected from the predetermined area. In this way, the objects that belong to a group are displayed in a line according to the characteristics of the objects, while the objects that do not belong to a group are displayed in the other area. This reduces the processing load required for the user device 12 of the broadcasting user and the server 13 to determine the group and select the drawing position, and enables a wide variety of expressions.

[0061] 13 shows the gift selection screen 210 when the gift button 173 is selected. A gift list 211 is displayed on the gift selection screen 210. Selectable gift objects are displayed on the gift list 211 together with their corresponding fees. The viewing user performs an operation to select one gift object from the gift list 211. If the selected gift object is an object that requires a fee, the user operates the purchase button 212. As a result, identification information of the gift object and a display request are transmitted from the user device 12 to the server 13.

[0062] Next, the effects of this embodiment will be described. (1) When multiple gift objects belonging to a group are displayed on the screen at the same time, the gift objects are drawn in the order in which they are assigned to the drawing positions among the multiple drawing positions corresponding to the group. Therefore, even if a display request is made for multiple gift objects, the gift objects are displayed in a preset layout, so that the gift objects are not unintentionally overlapped or erased, and are displayed in a good-looking state. This can increase the satisfaction of the viewing user who requests the display of the gift objects and the satisfaction of the broadcasting user with the gift function.

[0063] (2) The gift object leaves the drawing position after a predetermined time has elapsed since the display started, and among the drawing positions where no object is displayed, a drawing position with a high display order priority is set as the drawing position for the new object. Therefore, even if the number of display requests for gift objects increases, it is possible to display as many objects as possible.

[0064] (3) When identification information of a new object is received, the drawing position of the object with the longest drawing time is set as the drawing position of the new object. Therefore, even if the number of display requests for gift objects increases, as many objects as possible can be displayed.

[0065] (4) The drawing positions corresponding to a group include drawing positions having different depth and height positions, so that even if the gift object is bulky, multiple gift objects can be displayed in an easily viewable manner.

[0066] (5) The gift object includes gift objects that do not belong to a group, and the gift objects that do not belong to a group are rendered in an area other than the drawing position corresponding to the group. This reduces the processing load on the user device 12 of the broadcast user and the server 13, and enables a wide variety of expressions.

[0067] The above embodiment can be modified as follows: The above embodiment and the following modifications can be combined with each other to the extent that no technical contradiction occurs. In the above embodiment, an example has been described in which gift objects of the same type or category, such as "Hanging Scroll 1" and "Hanging Scroll 2", are grouped, but the gift objects may be grouped based on other criteria. FIG. 14 shows avatar objects 130A and 130B and gift objects 220A to 220C, 221A to 221C, and 224 in a schematic manner. In FIG. 14, solid lines indicate objects, and dashed lines indicate areas. For example, when avatar objects 130A and 130B appear together in a video, display areas 220 and 221 of the gift objects may be set around avatar objects 130A and 130B. At this time, user device 12 determines which of avatar objects 130A and 130B the gift object requested to be displayed is associated with, and displays the gift object in the display area of ​​the associated avatar object. For example, in the case where a female avatar object 130A and a male avatar object 130B appear together, when a display request is made for a gift object 222 that is a "bag" for the avatar object 130A, the gift object 220A is displayed in a display area 220 set around the avatar object 130A. Also, when a display request is made for a gift object 222 that is a "shoes" for the avatar object 130B, the gift object 221A is displayed in a display area 221 set around the avatar object 130B. A plurality of drawing positions are set in the display areas 220 and 221, and the gift objects are rendered according to the order assigned to these drawing positions. When a display request is made for a gift object 224 that is not associated with either of the avatar objects 130A and 130B, the gift object 224 is displayed in an area 225 other than the display areas 220 and 221. Alternatively, the broadcast user corresponding to the avatar object 130A that is the main subject of the co-starring performs an operation to determine the display area.

[0068] In the above embodiment, the case where there is an upper limit on the display time of the gift object has been described, but the exit condition of the gift object may be different depending on the group. As shown in FIG. 15, the free object and the paid object may be displayed in different display areas 230, 231. The free object is paid for by points given in the video application, and the paid object is paid for by, for example, money or a currency such as points that can be used outside the video application. The free object may be given to the user at a predetermined timing without requiring a payment. Drawing positions 232 to 234 are set in the display area 230 of the free object. Drawing positions 235 to 239 are set in the display area 231 of the paid object. The upper limit Tmax2 of the display duration of the paid object displayed in the display area 231 may be longer than the upper limit Tmax1 of the display duration of the free object displayed in the display area 230 (Tmax2>Tmax1). The maximum number of paid objects displayed in the display area 231 may be greater than the maximum number of free objects displayed in the display area 230. By doing so, the display time of the paid object can be extended. Also, the display area 231 is arranged closer to the camera position than the display area 230. By doing so, it is expected that the motivation to give paid objects will be increased, and the viewing time and number of views of the video, and the distribution time and number of distributions of the video will increase.

[0069] In the above embodiment, the multiple drawing positions in the display area are assigned an order. Alternatively, the distribution management unit 30A may randomly select one of the multiple drawing positions in the display area and display the gift object at the selected drawing position.

[0070] In the above embodiment, the gift objects displayed at the drawing positions are made to exit from the drawing positions in the order in which the exit conditions are satisfied. Instead of or in addition to this, when the exit conditions are satisfied, the gift objects may be made to exit all at once from the drawing positions. Also, the gift objects may be kept displayed while the video is being distributed. For example, when the viewing user makes a display request for a "red ball" object, the user device 12 of the distribution user displays the "red ball" gift object in the corresponding drawing position in the display area in order or random order. When the viewing user selects a "white ball" object, the user device 12 of the distribution user displays the "white ball" gift object in the corresponding drawing position in the display area in order or random order. For a predetermined period of time, the gift objects displayed in the display area are not erased or made to exit from the display area. At this time, the distribution management unit 30A may count the number of gift objects for each display area and compare the number of gift objects for each display area to determine whether the game has won or lost or to perform an evaluation. Also, when a predetermined period of time has elapsed, all of the "red balls" and "white balls" within the display area may be made to exit all at once.

[0071] In the above embodiment, server 13 transmits the posted message to both user device 12 of the broadcasting user and user device 12 of the viewing user. Then, user device 12 combines (superimposes) the posted message with the animation and draws it. Alternatively, the posted message may be transmitted only to user device 12 of the broadcasting user. Then, user device 12 of the viewing user may receive video data in which the animation and the posted message have been combined by user device 12 of the broadcasting user.

[0072] In the above embodiment, the server 13 receives a display request for a gift object from the user device 12 and adds the identification information of the gift object to the waiting queue. Alternatively, the user device 12 of the broadcasting user may receive a display request for a gift object from the server 13 and manage the waiting queue.

[0073] In the above embodiment, a video application program for viewing and distributing videos is installed in the user device 12. Alternatively, the application program for viewing videos and the application program for distributing videos may be separate application programs. When distributing videos, the distribution application program is used to distribute the videos. When viewing videos, the viewing application program is used to view the videos. A display request for a gift object is made using the viewing application program.

[0074] In the above embodiment, the user device 12 of the broadcasting user encodes the video data and audio data, transmits the encoded data to the server 13, and the user device 12 of the viewing user displays the video on the display 28 based on the encoded data (video distribution method). Alternatively, the user device 12 of the broadcasting user may transmit the minimum data required to generate a video (video configuration data), such as audio data and motion data, to the server 13, and the server 13 may transmit the video configuration data to the other user devices 12, and each of the user devices 12 may generate a video (client rendering method). In the client rendering method, the server 13 transmits a display request for a gift object to each of the user devices 12. Alternatively, the server 13 may transmit a list of gift objects to each of the user devices 12. The user device 12 renders objects including avatar objects based on the display request or list, and generates a video. This omits the encoding process in the user device 12 of the broadcast user, and reduces the amount of communication between the user device 12 of the broadcast user and the server 13, and between the server 13 and the user device 12 of the viewing user.

[0075] Alternatively, the server 13 may render the avatar objects and the gift objects. In this case, the server 13 generates video data, encodes it, and distributes it to each user device 12.

[0076] Although the motion data generating unit 20B has been described as acquiring detection data detecting a change in the user's facial expression, head movement, and the user's relative position with respect to the sensor unit 24, at least one of these may be acquired. Furthermore, the motion data generating unit 20B may acquire other detection data, such as detection data detecting a movement other than the head movement, in addition to or instead of at least one of the change in the user's facial expression, head movement, and the user's relative position.

[0077] The virtual space displayed in the video may be an augmented reality (AR) space. For example, animations of an avatar object, a gift object, etc. may be superimposed on an image of the real world captured by the camera of the user device 12. Also, animations of a gift object, etc. may be superimposed on an image of the real world including the user captured by the camera of the user device 12.

[0078] In the above embodiment, the user device 12 is an information processing device such as a smartphone, a mobile phone, a tablet terminal, a personal computer, a console game machine, or a wearable computer such as a head-mounted display. Alternatively, the user device 12 may be a system including at least one of the information processing devices, an input device, and an output device. For example, the user device 12 may be a system including a console game machine including a computer processor 20 and a sensor unit 24, a controller as an operation unit 27, a speaker 25, a display 28, and the like, which are provided separately from the console game machine, and capable of mutually transmitting and receiving data. The user device 12 may also be a system including an information processing device such as a personal computer and a sensor unit 24, a controller as an operation unit 27, a speaker 25, a wearable display as a display 28, and the like, which are provided separately from the console game machine, and capable of mutually transmitting and receiving data. Alternatively, instead of the user device 12 that distributes videos, a system provided in a studio for video distribution and including an information processing device, a sensor unit 24, an operation unit 27, a speaker 25, and a display 28 may be used as the system for distributing videos.

[0079] In the above embodiment, the sensor unit 24 is provided on the user device 12 side, but at least a part of the sensor unit 24 may be attached to the body of the broadcast user or provided in the vicinity of the broadcast user. For example, the sensor unit 24 may be composed of an emitter that emits detection light such as infrared light at predetermined intervals, and a light receiver that is attached to the body of the broadcast user and receives the detection light and transmits the detection results to the computer processor 20. Alternatively, the sensor unit 24 may be composed of a camera or the like that detects a marker attached to the body of the broadcast user. [Additional Notes] The following additional notes are set forth in the claims of the original application of this application. [1] Computer, a receiving unit that receives identification information of an object transmitted by a first user device used by a first user who watches a video; a determination unit that determines a drawing position at which the object should be drawn, among a plurality of drawing positions corresponding to a group to which the object belongs, based on the received identification information, in accordance with an order assigned to each of the drawing positions; The determining unit determines whether the object is rendered at the determined drawing position. Video application program. [2] the execution unit causes the object to leave the drawing position after a predetermined time has elapsed since drawing of the object was started, The determination unit determines, among the drawing positions at which the object is not displayed, a drawing position having a high priority in the display order as a drawing position for a new object. 2. The video application program according to claim 1. [3] When identification information of a new object is received, the determination unit determines the drawing position of the object with the longest drawing time as the drawing position of the new object. 3. The video application program according to claim 1 or 2. [4] The plurality of drawing positions include positions having different depth positions and height positions in a direction away from a camera position of the video. 4. The video application program according to claim 1. [5] the objects include objects that do not belong to the group; The execution unit renders an object that does not belong to the group in an area other than a drawing position corresponding to the group. 5. The video application program according to claim 1. [6] The computer, a generator for generating an animation by rendering the object; The animation is encoded and transmitted to a video distribution server. 5. The video application program according to claim 1. [7] The computer, The position transmitting unit further functions as a position transmitting unit that transmits the drawing position determined by the determining unit to a video distribution server with a user device that is viewing the video as a destination. 5. The video application program according to claim 1. [8] The computer A receiving step of receiving identification information of an object transmitted by a first user device used by a first user who watches a video; a determining step of determining, based on the received identification information, a drawing position at which the object should be drawn, among a plurality of drawing positions corresponding to a group to which the object belongs, in accordance with an order assigned to each of the drawing positions; an execution step of rendering the object at the drawing position determined by the determination step; How to draw objects in video. [9] Computer, a request receiving unit that receives a display request for an object transmitted by a first user device used by a first user who watches a video, and identification information of the object that is the target of the display request; a group determination unit that receives identification information of an object and a display request for the object based on an operation of a user watching a video, and determines a group to which the object belongs; a management unit that adds identification information of the object to a waiting queue corresponding to the determined group; a transmitting unit that transmits identification information of the object included in the waiting queue to a second user device used by a second user who distributes the video; Video management program.

[10] The computer a request receiving step of receiving a display request for an object transmitted by a first user device used by a first user who watches a video, and identification information of the object that is the target of the display request; a group determination step of receiving identification information of an object and a display request for the object based on an operation of a user watching a video, and determining a group to which the object belongs; a management step of adding identification information of the object to a waiting queue corresponding to the determined group; a transmitting step of transmitting identification information of the object included in the waiting queue to a second user device used by a second user who distributes the video. How to manage videos.

[11] a request receiving unit that receives a display request for an object transmitted by a first user device used by a first user who watches a video, and identification information of the object that is the target of the display request; a group determination unit that receives identification information of an object and a display request for the object based on an operation of a user watching a video, and determines a group to which the object belongs; a management unit that adds identification information of the object to a waiting queue corresponding to the determined group; a transmission unit that transmits identification information of the object included in the waiting queue to a second user device used by a second user who distributes the video. server.

[12] a display request receiving unit that receives a display request for an object from a terminal used by the user based on an operation of the user watching the video together with identification information of the object; a group determination unit that determines a group to which the object that is the subject of the display request belongs; a management unit that adds identification information of the object to a waiting queue corresponding to the determined group; a determination unit that determines a drawing position at which the object at the head of the queue should be drawn, among a plurality of drawing positions corresponding to the group, in accordance with an order assigned to each of the drawing positions; an execution unit that renders the object at the drawing position determined by the determination unit; Video distribution system. [Explanation of symbols]

[0080] 11 Video distribution system, 12 User Device 13 servers.

Claims

1. On the computer, a receiving function for receiving, from a first user device used by a first user, identification information of an object that is provided to a second user and that can be displayed in a virtual space in which at least one avatar object can be displayed, the object being different from the avatar object; an execution function of rendering the object at a specific drawing position in the virtual space determined from at least one drawing position corresponding to a group to which the object belongs based on the received identification information; an exit function for causing the objects to exit all at once from the specific drawing position when a predetermined exit condition is met; A computer program that achieves this.

2. The computer program described in claim 1, wherein the specified exit condition is that the display time of the object has reached an upper limit value, the second user has intentionally or unintentionally performed an operation to exit the object, and / or the first user has performed an operation to exit the object.

3. The computer program described in Claim 1, characterized in that the specified exit conditions differ depending on the group.

4. The computer further comprises: A determination function is realized for determining whether or not the object belongs to a predetermined group based on the received identification information; The executive function is When it is determined that the object belongs to a predetermined group as a result of the determination by the determination function, the object is rendered at a drawing position corresponding to the predetermined group; The computer program product according to claim 1, characterized in that, when the determination function determines that the object does not belong to a specified group, the object is rendered in an area other than a specific drawing position corresponding to the specified group.

5. The computer program of claim 1, characterized in that the groups are created by category, type or attribute.

6. The computer program product of claim 1, characterized in that the group includes a group to which a paid object belongs.

7. The computer program described in Claim 1, characterized in that the execution function determines the specific drawing position so as to avoid objects belonging to the same group being displayed overlapping each other when viewed from the direction of the virtual camera.

8. A computer program as described in claim 1 or 7, characterized in that the execution function determines the specific drawing position according to the order assigned to each of a plurality of drawing positions corresponding to the group to which the object belongs.

9. A computer program as described in claim 1 or 7, characterized in that the execution function randomly determines the specific drawing position from among a plurality of drawing positions corresponding to the group to which the object belongs.

10. A computer program as described in claim 1 or 7, characterized in that the execution function determines the specific drawing position so as to avoid the object being displayed overlapping the second user's avatar object when viewed from the direction of the virtual camera.

11. The first user is a user participating in a first mode, The computer program product of claim 1 , wherein the second user is a user participating in a second mode.

12. The second mode is a mode for distributing video, The computer program product of claim 11 , wherein the first mode is a mode for watching the video.

13. The rendering position includes a position in which a depth position and a height position in a direction away from the virtual camera are different.

2. The computer program product of claim 1.

14. The computer further comprises: A generating function for generating an animation by rendering the object; and an encoding function for encoding the animation and transmitting it to a server.

2. The computer program product of claim 1.

15. The computer further comprises: A position transmission function is realized for transmitting the specific drawing position to a server with the first user device as a destination.

2. The computer program product of claim 1.

16. The computer a receiving step of receiving, from a first user device used by a first user, identification information of an object that is provided to a second user and that is displayable in a virtual space in which at least one avatar object can be displayed, the object being different from the avatar object; an execution step of rendering the object at a specific drawing position in the virtual space determined from at least one drawing position corresponding to a group to which the object belongs based on the received identification information; an exit step of causing the objects to exit the specific drawing position all at once when a predetermined exit condition is met; The object drawing method to perform.

17. Computer, a request receiving unit that receives, from a first user device used by a first user, a display request for an object that is provided to a second user and that is displayable in a virtual space in which at least one avatar object can be displayed, and identification information of the object that is the target of the display request, the object being different from the avatar object; a group determination unit that receives identification information of the object and a display request for the object and determines a group to which the object belongs; a management unit that adds identification information of the object to a waiting queue corresponding to the determined group; a second user device that transmits identification information of the object included in the waiting queue to a second user device used by the second user; Computer program.

18. The computer a request receiving step of receiving, from a first user device used by a first user, a display request for an object that is provided to a second user and that can be displayed in a virtual space in which at least one avatar object can be displayed, the object being different from the avatar object, and identification information of the object that is the target of the display request; a group determination step of receiving identification information of the object and a display request for the object and determining a group to which the object belongs; a management step of adding identification information of the object to a waiting queue corresponding to the determined group; a transmitting step of transmitting identification information of the object included in the waiting queue to a second user device used by the second user. Management method.

19. A request receiving unit that receives, from a first user device used by a first user, a display request for an object that is provided to a second user and that is displayable in a virtual space capable of displaying at least one avatar object and that is different from the avatar object, and identification information of the object that is the subject of the display request; a group determination unit that receives identification information of the object and a display request for the object and determines a group to which the object belongs; a management unit that adds identification information of the object to a waiting queue corresponding to the determined group; a transmission unit that transmits identification information of the object included in the waiting queue to a second user device used by the second user. server.

20. A display request receiving unit that receives a display request for an object that is provided to a second user and that is displayable in a virtual space capable of displaying at least one avatar object, the object being different from the avatar object, and identification information of the object from a first user terminal used by a first user; a group determination unit that determines a group to which the object that is the subject of the display request belongs; a management unit that adds identification information of the object to a waiting queue corresponding to the determined group; an execution unit that renders the object at the head of the queue at a specific drawing position determined from at least one drawing position corresponding to the group; Equipped Information processing system.

Images