program

Abstract

To provide a program for classifying comments posted by viewers in a live video distribution service into groups of similar comments and displaying groups including a larger number of similar comments as objects. [Solution] The program causes a computer to receive comments posted on a displayed video, classify the received comments into at least a first group and a second group in each of a first period and a second period which are set sequentially, calculate a first ratio of the number of comments classified into the first group to the total number of received comments, and switch and display a first object determined based on the first ratio in the first period to a first object determined based on the first ratio in the second period.

Description

[Technical field]

[0001] An embodiment of the present invention relates to a program for classifying and displaying comments posted in real time by viewers in a live video distribution service. [Background technology]

[0002] In recent years, various live video distribution services have become widespread along with the high performance and high functionality of information terminals such as smartphones and the high speed of communication via the Internet. For example, a live video distribution service is being provided that distributes a video of a distributor playing a game (i.e., a gameplay video) together with the distributor's audio commentary to the information terminals of many viewers in real time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-170093 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a live video distribution service, a distributor distributes a live video, and viewers who watch the live video can post comments on the live video. Comments posted by viewers are displayed on the information terminal of the distributor, so the distributor can check the comments posted by viewers and distribute a live video in response to the comments of the viewers, giving the viewers the impression that they are having a conversation and enabling closer communication with the viewers. However, in a live video distribution service, since many viewers post comments, it is often difficult for the distributor to check all the comments. In addition, it may be difficult for the distributor to instantly grasp the majority opinion supported or sympathized with by a larger number of viewers, or the current mainstream opinion, among all the comments posted by the viewers.

[0005] One embodiment of the present invention has an objective of providing a program in a live video distribution service that classifies comments posted by viewers into groups of similar comments and displays groups containing more similar comments as objects. [Means for solving the problem]

[0006] A program according to one embodiment of the present invention causes a computer to receive comments posted on a displayed video, classify the received comments into at least a first group and a second group in each of a first period and a second period which are set sequentially, calculate a first ratio of the number of comments classified into the first group to the number of received comments in each of the first period and the second period, determine a first object having a first display manner for identifying the first group when the first ratio is equal to or greater than a first threshold in each of the first period and the second period, and switch and display the first object determined based on the first ratio in the first period to the first object determined based on the first ratio in the second period.

[0007] The first display aspect may be the area, color, or movement of the first object.

[0008] A second ratio of the number of comments classified into a second group corresponding to the number of comments received in each of the first period and the second period may be calculated, and when the second ratio is equal to or greater than a second threshold in at least one of the first period and the second period, a second object having a second display manner for identifying the second group may be determined, and the first display manner and the second display manner may be determined according to the first ratio and the second ratio to display the first object and the second object.

[0009] The first display manner and the second display manner may be a first area of ​​a first object and a second area of ​​a second object, respectively, and the second area relative to the first area may correspond to a second ratio relative to the first ratio.

[0010] When the second object is not determined based on the second proportion in the first period, but is determined based on the second proportion in the second period, the second object may be displayed superimposed on the first object.

[0011] A first display manner of the first object in the first time period and the second time period may differ based on a first ratio.

[0012] The first display manner based on the first ratio may be the shade of a color of the first object or the size of text included in the first object.

[0013] The start of the second period may be before the end of the first period.

[0014] At least one of the first period and the second period may be started based on an instruction from a distributor who provides the video.

[0015] The distributor's instructions may be linked to a predetermined eye movement or facial movement by the distributor, or a predetermined keyword uttered by the distributor.

[0016] When the number of comments received in each of the first period and the second period is equal to or greater than a set value, each of the first period and the second period may be ended.

[0017] The first object includes text representing the first group, and the text may be determined from among pre-registered words.

[0018] The first object includes text representing the first group, and the text may be determined by extracting words within the comments that are classified into the first group.

[0019] The received comments may be classified into at least a first group and a second group using a deep learning model trained based on previously received comments.

[0020] The video may be a game broadcast video. Effect of the Invention

[0021] According to an embodiment of the present invention, in a live video distribution service, comments posted by viewers are classified into groups for each similar comment, and a group including a larger number of similar comments is displayed as an object on a display unit of a distributor's information terminal. That is, the displayed object reflects the majority opinion of the viewers. Even if there are multiple groups classified as the majority opinion, it is possible to grasp the mainstream majority opinion from the relative size of the multiple displayed objects by making the size of the object correspond to the number of comments in the group. In addition, the object is provided with text representing the comments in the group. Therefore, by visually checking the object, a distributor can easily grasp the majority opinion of the viewers and distribute a live video in response to the viewers' comments. This gives the viewer the impression that he or she is having a conversation with the distributor, and the distributor can communicate more closely with the viewer. In addition, by watching a live video that is in line with his or her comments, the viewer can feel that the live video is familiar to him or her. The live video distribution service becomes more popular due to the improvement in the sense of unity between the distributor and the viewer, and as a result, the number of viewers in the live video distribution service can be increased. [Brief description of the drawings]

[0022] [Figure 1] 1 is a block diagram showing a configuration of a live video distribution system according to an embodiment of the present invention. [Diagram 2] FIG. 5 is a schematic diagram illustrating a configuration of live video data 500 generated using a live video distribution system according to an embodiment of the present invention. [Diagram 3] 2 is a block diagram showing a configuration of a distribution server according to an embodiment of the present invention; FIG. [Figure 4] 1 is a block diagram showing a configuration of a distributor information terminal according to an embodiment of the present invention; [Diagram 5] 1 is a block diagram showing a configuration of a viewer information terminal according to an embodiment of the present invention; [Figure 6]FIG. 11 is a sequence diagram illustrating a live video distribution process according to an embodiment of the present invention. [Figure 7] 11 is a flowchart illustrating a live video data generation process according to an embodiment of the present invention. [Figure 8] 11 is a schematic diagram illustrating a process of classifying comment information into groups in the live video data generation process according to the embodiment of the present invention. FIG. [Figure 9] FIG. 11 is a schematic diagram illustrating a process of acquiring comment information in the live video data generation process according to the embodiment of the present invention. [Figure 10] FIG. 11 is a schematic diagram illustrating a process of generating an object in the live video data generation process according to the embodiment of the present invention. [Figure 11] 11 is a diagram illustrating an example of an object generated in the live video data generation process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, embodiments of the present invention will be described with reference to the drawings, etc. However, the embodiments of the present invention can be implemented in various forms without departing from the gist of the technical idea, and should not be construed as being limited to the embodiments described below.

[0024] In order to clarify the description, the drawings may be schematic in width, thickness, shape, etc., compared to the actual embodiment. In addition, in the drawings, components having the same or similar functions may be indicated by the same reference numerals or reference numerals with alphabets added thereto, and repeated description of the components may be omitted.

[0025] For the purposes of the specification and claims, the following terms are defined as follows:

[0026] A "program" refers to an instruction or group of instructions executed by a processor in a computer having a processor and memory. A "computer" is a general term referring to the entity that executes a program. For example, when a program is executed by a server (or a client), the "computer" refers to the server (or the client). Also, when a "program" is executed by distributed processing between a server and a client, the "computer" includes both the server and the client. In this case, the "program" includes "a program executed by a server" and "a program executed by a client." Similarly, when a "program" is processed in a distributed manner among multiple servers, the "computer" includes the multiple servers, and the "program" includes each program executed by each server.

[0027] The term "live video distribution service" refers to a service that distributes live video in real time over a network. The live video may be, for example, but is not limited to, a gameplay video.

[0028] "Distributor" refers to a person who provides live video for distribution in a live video distribution service.

[0029] "Viewer" refers to a person who watches live videos and posts comments in a live video distribution service.

[0030] [1. Overview of the live video distribution system 1000] FIG. 1 is a block diagram showing a configuration of a live video distribution system 1000 according to an embodiment of the present invention.

[0031] The live video distribution system 1000 includes a distribution server 200, a distributor's information terminal 300, and multiple viewer's information terminals 400 (400-1, 400-2, . . . , 400-n in FIG. 1). The distribution server 200, the distributor's information terminal 300, and the multiple viewer's information terminals 400 are communicatively connected via a network NW such as the Internet or a LAN. The network NW may be wired or wireless. The distributor's information terminal 300 and the viewer's information terminal 400 are clients that can receive services provided by the distribution server 200.

[0032] The distribution server 200 is an information processing device that provides a live video distribution service to the distributor information terminal 300 and the viewer information terminal 400, or provides various services related to the live video distribution service. The live video distribution service can be provided by executing a live video distribution program. The various services in the distribution server 200 may include, for example, a login process or a synchronization process when executing the live video distribution service. The various services may also include a comment posting function. The live video distribution program is recorded in a storage device included in the distribution server 200, a recording medium readable by the distribution server 200, or a database to which the distribution server 200 can be connected via the network NW. In FIG. 1, the distribution server 200 is illustrated as a single information processing device, but may be composed of multiple information processing devices.

[0033] The distributor's information terminal 300 is an information terminal used by a distributor. The viewer's information terminal 400 is an information terminal used by a viewer. The distributor's information terminal 300 and the viewer's information terminal 400 are each an information processing device such as a personal computer, a tablet, a smartphone, a stationary game machine, or a portable game machine. The distributor's information terminal 300 and the viewer's information terminal 400 can be communicatively connected to the distribution server 200 via the network NW. The distributor's information terminal 300 can execute a live video distribution program by communicatively connecting to the distribution server 200, and can transmit live video data corresponding to the live video to the distribution server 200. The viewer's information terminal 400 can execute a live video distribution program by communicatively connecting to the distribution server 200, and can receive live video data. The viewer's information terminal 400 can also allow a viewer who has viewed a live video to post a comment, and comment information including the comment to be posted can be transmitted from the viewer's information terminal 400 to the distribution server. In this way, in the live video distribution system 1000, by executing a live video distribution program, live video data can be transmitted in real time from the distributor's information terminal 300 to the distribution server 200, and the distribution server 200 can be accessed from the viewer's information terminal 400, the live video data transmitted from the distribution server 200 can be received in real time by the viewer's information terminal 400, and comment information can be transmitted in real time from the viewer's information terminal 400 to the distribution server 200.

[0034] In the live video distribution system 1000, various live videos can be distributed. For example, a distributor can distribute a live video in which the distributor makes a statement based on comments posted by viewers, together with a video of the distributor himself / herself. In addition, a distributor can distribute a live video (so-called gameplay video) in which the distributor gives a live commentary while reacting to comments posted by viewers, together with a video of a game screen in which the distributor plays the game. The game is, for example, an online game such as a social game or a mobile game, but is not limited to these.

[0035] The live video data corresponding to the live video provided by the distributor is generated in the distributor's information terminal 300, but a part of the live video data may be data acquired by communicating with another server via the network NW. For example, the game screen data in the game play video may not only be data captured by the distributor's information terminal 300, but also acquired game screen data transmitted from the game server via the network NW.

[0036] [2. Composition of Live Video Data 500] The configuration of live video data generated by the distributor information terminal 300 will be described with reference to FIG.

[0037] FIG. 2 is a schematic diagram illustrating the configuration of live video data 500 generated using a live video distribution system 1000 according to an embodiment of the present invention. For convenience, FIG. 2 illustrates the configuration of a live video in which the live video data 500 is displayed on a screen. That is, in a live video distribution service, a live video corresponding to the live video data 500 is provided to a viewer information terminal 400 and displayed. Note that the live video data 500 is generated in a distributor information terminal 300, but the live video provided to the viewer information terminal 400 may have a different display layout from that displayed on the distributor information terminal 300. In this case, the live video provided to the viewer information terminal 400 may also be provided to the distributor information terminal 300 and displayed.

[0038] As shown in FIG. 2, the live video data 500 includes a game play video 510, a distributor video 520, a comment display area 530, and an object display area 540. Although not shown in FIG. 2, the live video data 500 also includes audio data. The game play video 510 is a video that changes in real time according to the progress of the game played by the distributor. The distributor video 520 is a video that changes in real time according to the movement of the distributor. The distributor video 520 may be a video itself shot by the distributor, or may be a video (e.g., an avatar) that is processed based on a video shot by the distributor. The comment display area 530 is an area where comments posted by viewers are displayed sequentially. A certain number of comments including the latest comment are displayed in the comment display area 530. Therefore, comments posted before a certain number of comments are scrolled sequentially and disappear from the comment display area 530. The object display area 540 is an area in which objects 541 (first object 541-1, second object 541-2, and third object 541-3 in FIG. 8) are displayed. The objects 541 are obtained by classifying comments posted by viewers into groups of similar comments, and visually displaying groups containing more similar comments. Therefore, by visually checking the objects 541, the distributor can grasp the majority opinion or requests of the viewers. Although three objects are displayed in the object display area 540 in FIG. 2, the number of objects 541 is not limited to this. Details of the generation process and display mode of the objects 541 will be described later.

[0039] Although the live video data 500 including the gameplay video 510 and the broadcaster video 520 has been described with reference to FIG. 2, the live video data 500 may have a configuration including only one of the gameplay video 510 and the broadcaster video 520.

[0040] 3. Configuration of components of live video distribution system 1000 The configurations of the distribution server 200, the distributor information terminal 300, and the viewer information terminal 400 included in the live video distribution system 1000 will be described in detail with reference to FIGS.

[0041] [3-1. Configuration of distribution server 200] Fig. 3 is a block diagram showing a configuration of a distribution server 200 according to an embodiment of the present invention. As shown in Fig. 3, the distribution server 200 includes a control unit 210, a storage unit 220, and a communication unit 230. Note that the distribution server 200 may include further components.

[0042] The control unit 210 includes a processor (arithmetic processing device) such as a CPU (Central Processing Unit) and a storage device such as a RAM. The control unit 210 executes a program stored in the storage device by the processor, and realizes various functions related to the live video distribution service. For example, the control unit 210 can realize a live video data processing unit 211 and a comment information processing unit 212 as functions.

[0043] The live video data processing unit 211 acquires the live video data 500 transmitted from the distributor's information terminal 300, and transmits the acquired live video data 500 to the viewer's information terminal 400 that is communicatively connected to the distribution server 200. The live video data 500 acquired by the live video data processing unit 211 may be stored in the storage unit 220. The live video data processing unit 211 can also change or modify the configuration of the live video data 500 transmitted from the distributor's information terminal 300. For example, the live video data processing unit 211 can change and modify the configuration of the live video data 500 to a display layout for live video distribution. The live video data processing unit 211 can also transmit the acquired live video data 500 or the changed or modified live video data 500 to not only the viewer's information terminal 400 but also the distributor's information terminal 300.

[0044] The comment information processing unit 212 acquires the comment information transmitted from the viewer information terminal 400, and transmits the comment information to the distributor information terminal 300 and the viewer information terminal 400. The comment information acquired by the comment information processing unit 212 may be stored in the storage unit 220. The comment information transmitted to the distributor information terminal 300 and the viewer information terminal 400 may be all or a part of the acquired comment information. For example, when the comment information includes a viewer ID, a viewer nickname, and a comment, the comment information processing unit 212 can extract the comment from the acquired comment information and transmit the extracted comment to the distributor information terminal 300 and the viewer information terminal 400 as comment information. The comment information processing unit 212 can also perform a filtering process on the comment. Specifically, the comment information processing unit 212 can generate comment information to be transmitted to the distributor information terminal 300 and the viewer information terminal 400 after excluding comments including inappropriate or unpleasant expressions from the acquired comment information. The comment information transmitted to the distributor information terminal 300 and the comment information transmitted to the viewer information terminal 400 may be the same or different.

[0045] The storage unit 220 is a recording device (recording medium) capable of permanently retaining information and rewriting information, such as a non-volatile memory or a hard disk drive. For example, the storage unit can store the live video data 500 acquired by the live video data processing unit 211 or the comment information acquired by the comment information processing unit 212.

[0046] The communication unit 230 is a communication module that connects to the network NW under the control of the control unit 210 and transmits and receives data or information. If the network NW is wireless, the communication unit 230 may include a communication module for infrared communication or short-range wireless communication.

[0047] [2-2. Configuration of distributor information terminal 300] Fig. 4 is a block diagram showing a configuration of a distributor's information terminal 300 according to an embodiment of the present invention. As shown in Fig. 4, the distributor's information terminal 300 includes a control unit 310, a storage unit 320, a communication unit 330, a display unit 340, a sound input / output unit 350, an operation unit 360, and an imaging unit 370. Note that the distributor's information terminal 300 may include further components.

[0048] The control unit 310 has the same configuration as the control unit 210. However, the control unit 310 executes a program using a processor, and mainly realizes various functions related to a live video data generation process in a live video distribution service. For example, the control unit 310 can realize a comment information acquisition unit 311, a group classification unit 312, an object generation unit 313, and a live video data generation unit 314 as functions.

[0049] The comment information acquisition unit 311 acquires comment information transmitted from the distribution server 200. The comment information acquisition unit 311 may also perform a filtering process on the comments. Specifically, when the transmitted comment information includes a word (exclusion keyword) preset in the distributor information terminal 300 or a comment including an inappropriate or offensive expression, the comment information acquisition unit 311 can exclude such comment information from the acquired comment information.

[0050] The group classification unit 312 classifies the comment information acquired by the comment information acquisition unit 311 into groups for each similar comment.

[0051] The object generating unit 313 calculates the ratio of the number of comment information in a group to the total number of comment information acquired within a predetermined period. The object generating unit 313 also determines the ratio of the number of comment information in a group to the total number of comment information acquired within a predetermined period, and when there are multiple groups that satisfy the determination condition, generates an object 541 having a relative change corresponding to the ratio of the number of comment information in each group.

[0052] The live video data generator 314 generates live video data 500 including a game play video 510, a broadcaster video 520, a comment display area 530, and an object display area 540.

[0053] The functions of the comment information acquiring unit 311, the group classifying unit 312, the object generating unit 313, and the live video data generating unit 314 will be described in detail later.

[0054] The storage unit 320 has the same configuration as the storage unit 220. Data or information used in the live video data generation process is stored in the storage unit 320. For example, the storage unit 320 can store a threshold value, a setting value, an excluded keyword, comment information, or data related to the object 541.

[0055] The communication unit 330 has a similar configuration to the communication unit 230 .

[0056] The display unit 340 has a display area for displaying various display images (e.g., game play video 510, distributor video 520, comment display area 530, object display area 540, live video, etc.) according to the control of the control unit 310. The display unit 340 is, for example, a display device such as a liquid crystal display or an organic EL display.

[0057] The sound input / output unit 350 inputs and outputs sound data. The sound input / output unit 350 is composed of, for example, a microphone and a speaker. In this case, sound data is input by the microphone of the sound input / output unit 350, and sound is output by the speaker of the sound input / output unit 350. The sound input / output unit 350 can also be used to output game music, etc.

[0058] The operation unit 360 is an operation device that outputs a signal (e.g., a signal indicating a command or information) according to an operation by the broadcaster to the control unit 310. For example, the broadcaster can play a game or perform broadcasting operations using the operation unit 360. The operation unit 360 may be a touch sensor arranged on the surface of the display unit 340.

[0059] The imaging unit 370 is an imaging device (camera) that converts an image of an object into a signal. For example, a distributor can use the imaging unit 370 to capture a game screen in which the distributor is playing and the distributor himself / herself, and generate a game play video 510 and a distributor video 520.

[0060] [2-3. Configuration of viewer information terminal 400] Fig. 5 is a block diagram showing the configuration of a viewer information terminal 400 according to one embodiment of the present invention. As shown in Fig. 5, viewer information terminal 400 includes a control unit 410, a storage unit 420, a communication unit 430, a display unit 440, a sound input / output unit 450, and an operation unit 460. Note that viewer information terminal 400 may include further components.

[0061] The control unit 410 has the same configuration as the control unit 210. However, the control unit 410 executes a program by a processor and mainly controls the operation of a live video distribution service. The control unit 410 realizes various functions related to the comment generation process. For example, the control unit 410 can realize a comment information generation unit 411 as a function.

[0062] When a comment is input by the operation unit 460, the comment information generating unit 411 generates comment information including the comment. The comment information may include not only the comment but also a viewer ID or a viewer nickname for identifying the viewer. The comment information generated by the comment information generating unit 411 is transmitted to the distribution server 200.

[0063] The storage unit 420 has the same configuration as the storage unit 220. The storage unit 320 stores data or information used in the comment generation process. For example, the storage unit 420 can store a viewer ID or a viewer nickname.

[0064] The communication unit 430 has a similar configuration to the communication unit 230 .

[0065] Display unit 440 has the same configuration as display unit 340. Display unit 440 displays live video corresponding to live video data 500 transmitted from distribution server 200, allowing viewers to view the live video in real time.

[0066] Sound input / output unit 450 has a similar configuration to sound input / output unit 350. From the speaker of sound input / output unit 450, sound data included in the live video can be output.

[0067] Operation unit 460 has the same configuration as operation unit 360. A viewer can use operation unit 460 to input a comment while viewing the live video displayed on display unit 440.

[0068] [3. Live video distribution processing] [3-1. Overall flow of live video distribution processing] A live video distribution process executed in the live video distribution system 1000 will be described with reference to FIG.

[0069] Fig. 6 is a sequence diagram illustrating a live video distribution process according to an embodiment of the present invention. The sequence diagram illustrated in Fig. 6 includes steps S100 to S160. For convenience, steps S100 to S160 will be described in order below, but the steps in the live video distribution process may be executed in parallel or the order may be changed. Furthermore, the live video distribution process may include further steps.

[0070] In step S100, the live video data 500 is generated in the distributor information terminal 300. The live video data generation process in step S100 will be described later in detail.

[0071] In step S110, the generated live video data 500 is transmitted from the distributor information terminal 300 to the distribution server 200 in real time.

[0072] In step S120, the live video data 500 is transmitted in real time from the distribution server 200 to the viewer information terminal 400. As a result, a live video corresponding to the live video data is displayed on the display unit 440 of the viewer information terminal 400, so that the viewer can watch the live video. Although not shown, the live video data 500 may be transmitted in real time from the distribution server 200 to the distributor information terminal 300.

[0073] In step S130, comment information is generated in viewer information terminal 400. Specifically, when a viewer watching a live video inputs a comment by operating operation unit 460, comment information generating unit 411 generates comment information including the input comment.

[0074] In step S140, the generated comment information is transmitted from the viewer information terminal 400 to the distribution server 200. That is, the distribution server 200 can sequentially receive a plurality of pieces of comment information generated in one or a plurality of viewer information terminals 400. The comment information received by the distribution server 200 is stored in the storage unit 220.

[0075] In step S150, the comment information is sequentially transmitted from the distribution server 200 to the distributor's information terminal 300. As a result, the comments included in the comment information are displayed on the display unit 340 of the distributor's information terminal 300, so that the distributor can visually confirm the viewers' comments on the live video.

[0076] In step S160, the comment information is sequentially transmitted from the distribution server 200 to the viewer information terminal 400. As a result, the comments included in the comment information are displayed on the display unit 440 of the viewer information terminal 400, so that the viewer can visually recognize his or her own comments or those of others.

[0077] [3-2. Live video data generation process flow] The live video data generation process in step S100 will be described in detail with reference to FIGS.

[0078] Fig. 7 is a flowchart illustrating a live video data generation process according to an embodiment of the present invention. Each of Figs. 8(A) to 8(D) is a schematic diagram illustrating a process of acquiring comment information in a live video data generation process according to an embodiment of the present invention. Fig. 9 is a schematic diagram illustrating a process of classifying comment information into groups in a live video data generation process according to an embodiment of the present invention. Fig. 10 is a schematic diagram illustrating a process of generating an object 541 in a live video data generation process according to an embodiment of the present invention.

[0079] 7, the live video data generation process (S100) includes steps S101 to S107. Steps S101 to S107 will be described below in order with reference to FIGS. 8 to 10 as appropriate.

[0080] In step S101, the comment information acquisition unit 311 acquires comment information. When a viewer posts a comment (i.e., comment information is transmitted from the viewer information terminal 400 to the distribution server 200 (see step S140 in FIG. 6)), the distribution server 200 transmits the comment information to the distributor information terminal 300 (see step S160 in FIG. 6). Therefore, the comment information acquisition unit 311 can acquire the comment information received at the distributor information terminal 300. The comment information acquired by the comment information acquisition unit 311 is linked to the reception date and time or reception time at which the distributor information terminal 300 received the comment information, or the acquisition date and time or acquisition time acquired by the comment information acquisition unit 311, and is stored in the storage unit 320.

[0081] In step S102, the group classification unit 312 classifies the comment information acquired in step S101 into groups. Specifically, the group classification unit 312 determines the comments included in the comment information and classifies the comment information into groups in which the comments are similar. Any conventional method can be applied to classify the comment information. For example, the group classification unit 312 can determine the group to which the comment information belongs by dividing the comment into phrases and determining the similarity of one of the divided words. In addition, the group classification unit 312 can determine the group to which the comment information belongs by using a comment classification model obtained by learning using machine learning such as a neural network or deep learning, in which a group is output when a comment is input. In this case, the group classification unit 312 can also acquire the group to which the comment information belongs by communicating with an external server via the network NW.

[0082] The comment information is linked to the determined group and stored in the storage unit 320. For example, the comment information linked to the group may be stored as a database as shown in FIG. 8. In FIG. 8, the "reception time" when the distributor information terminal 300 received the comment information, the "viewer ID", the "viewer nickname", and the "comment" included in the received comment information, and the "group" determined in step S102 are shown as items in the database. For example, referring to FIG. 8, comments expressing the viewer's emotions such as "I'm shaking~" and "I'm moved!!!" are classified into the "emotional" group. In addition, negative comments such as "I can't do this anymore" are classified into the "impossible game" group. The comment information in the same group are similar to each other, and a word such as "emotional" or "impossible game" that represents a group can be said to be a text that encompasses the comments of multiple comment information in the group. Such text expressing the similarity of the group may be generated in the process of the group classification unit 312 classifying the comment information, or may be set in advance.

[0083] In addition, when classifying the groups, the group classification unit 312 may classify comment information including comments including excluded keywords or inappropriate or offensive expressions into one specific group. This can be called a filtering process by the group classification unit 312.

[0084] In step S103, the object generating unit 313 calculates the ratio of the number of comment information in the group to the total number of comment information acquired within a predetermined period (hereinafter, for convenience, referred to as the "comment information ratio"). The comment information ratio is calculated for each group. In addition, the comment information ratio is calculated for each predetermined period.

[0085] Here, the period for which the comment information ratio is calculated will be described with reference to Figures 9(A) to 9(D). Each of Figures 9(A) to 9(D) is an example of the period for which the comment information ratio is calculated.

[0086] In the example of Fig. 9(A), the first period T1, the second period T2, and the third period T3 are set in sequence without overlapping. Similar periods are set after the third period T3, but the description thereof is omitted here.

[0087] In the first period T1, the comment information ratio is calculated based on the comment information acquired between time t1 and time t2. Similarly, in the second period T2, the comment information ratio is calculated based on the comment information acquired between time t2 and time t3, and in the third period T3, the comment information ratio is calculated based on the comment information acquired between time t3 and time t4. Here, the time at which the comment information is acquired may be the time at which the comment information is received by the distributor information terminal 300, or may be the time at which the comment information is acquired by the comment information acquisition unit 311. For example, the object generation unit 313 can calculate the comment information ratio within each period based on the comment information associated with the "group" and "received time" shown in FIG. 8.

[0088] In the example of Fig. 9(A), the time corresponding to the length of the period is set in advance. As a result, each period is set for each set time, and the comment information ratio is calculated. Therefore, the first period T1, the second period T2, and the third period T3 have the same length.

[0089] In the example of Fig. 9(B), the first period T1, the second period T2, and the third period T3 are set in order while overlapping each other. Similar periods are set after the third period T3, but the description thereof is omitted here.

[0090] In the first period T1, the comment information ratio is calculated based on the comment information acquired between time t1 and time t3. In the second period T2, the comment information ratio is calculated based on the comment information acquired between time t2 before time t3 and time t5. In the third period T3, the comment information ratio is calculated based on the comment information acquired between time t4 before time t5 and time t7. As described above, the first period T1 and the second period T2 overlap, and the same comment information may be used in the calculation of the comment information ratio in the first period T1 and the second period T2. Similarly, the second period T2 and the third period T3 overlap, and the same comment information may be used in the calculation of the comment information ratio in the second period T2 and the third period T3.

[0091] In the example of Fig. 9(B), not only the first time corresponding to the length of the period, but also the second time (<first time) that overlaps the period is set in advance. As a result, the comment information ratio is calculated by starting the subsequent period before the end of the period, but the first period T1, the second period T2, and the third period T3 have the same length (first time).

[0092] As will be described in detail later, the calculated comment information ratio is used to generate the object 541. When the first period T1 and the second period T2 are independent as in the example of Fig. 9(A), the switching of the object 541 may be accompanied by a sudden change. However, when the first period T1 and the second period T2 overlap as in the example of Fig. 9(B), the switching of the object 541 can be made smoother.

[0093] In the example of Fig. 9(C), the first period T1 and the second period T2 are set in sequence without overlapping. A similar period is set after the second period T2, but the description is omitted here.

[0094] In the example of FIG. 9(C), the first period T1 and the second period T2 are set based on an object generation request. Specifically, when an object generation request for generating an object 541 is generated by the distributor operating the operation unit 360 at time t1, the comment information acquisition unit 311 starts acquiring comment information and the group classification unit 312 starts classifying the comment information. When an object generation request is generated again at time t2, the object generation unit 313 calculates the comment information ratio in the first period T1 based on the comment information acquired during the first period T1 from time t1 to time t2. Similarly, when an object generation request is further generated at time t3, the object generation unit 313 calculates the comment information ratio in the second period T2 based on the comment information acquired during the second period T2 from time t2 to time t3.

[0095] In the example of FIG. 9(C) described above, the start time and end time of the period are set based on an object generation request generated by an operation of the distributor's operation unit 360, so the length of each period is different. However, the object generation request may set only the disclosure time of the period. In this case, by setting a time corresponding to the length of the period in advance, the period started based on the object generation request ends after the preset time has elapsed. Since the length of each period is a preset time, each period has the same length.

[0096] In addition, a distributor who provides a live video may have difficulty operating the operation unit 360, for example, when the distributor is playing a game. For this reason, the object generation request may be generated based on the distributor's eye movement or face movement in the distributor video captured by the imaging unit 370. The object generation request may also be generated based on a keyword issued by the distributor input by the sound input / output unit 350. That is, the object generation request may be generated in conjunction with a predetermined action of the distributor or a predetermined keyword issued by the distributor.

[0097] In the example of FIG. 9(C), comment information can be obtained at the timing desired by the distributor, and object 541 can be generated.

[0098] In the example of Fig. 9(D), the first period T1 and the second period T2 are set in sequence without overlapping. A similar period is set after the second period T2, but the description thereof is omitted here.

[0099] In the example of Fig. 9(D), when acquisition of comment information starts at time t1, the number of acquired comments is counted, and the first period T1 ends at time t2 when the number of acquired comment information reaches a set value (i.e., is equal to or greater than the set value). In addition, the number of acquired comments is counted again from time t2, and the second period T2 ends at time t3 when the number of acquired comment information reaches a set value (i.e., is equal to or greater than the set value). The object generation unit 313 calculates the comment information ratios in the first period T1 and the second period T2 based on the comment information acquired in each of the first period T1 and the second period T2.

[0100] 9D, the upper limit of the number of comments to be acquired is preset as a set value, so that the number of comment information items acquired in the first period T1 and the second period T2 is the same, but the lengths of the first period T1 and the second period T2 are different.

[0101] 9(D), comment information is acquired without depending on time, and when the upper limit of comment information is acquired, object 541 is generated. Therefore, object 541 can be generated in accordance with the progress of the live video.

[0102] Although each of FIGS. 9(A) to 9(D) has been described above, these are merely examples of periods for which the comment information ratio is calculated, and the periods for which the comment information ratio is calculated are not limited to these.

[0103] In step S104, the object generating unit 313 judges whether the comment information ratio calculated in step S103 is equal to or greater than a threshold value. Here, the threshold value is a preset value, and the judgment in step S104 is performed for each comment information ratio of the group. When the comment information ratio is equal to or greater than the threshold value (step S104: YES), step S105 is executed. On the other hand, when the comment information ratio is less than the threshold value (step S104: NO), step S107 is executed. Although details will be described later, an object 541 is generated for a group whose comment information ratio is equal to or greater than the threshold value, and an object 541 is not generated for a group whose comment information ratio is less than the threshold value. A group with a high comment information ratio is a group with many similar comments from viewers. That is, by executing step S104, a group that reflects the majority opinion of viewers can be determined.

[0104] In step S104, a judgment is also made for a specific group (i.e., a group that includes comments with excluded keywords, etc.) classified by the filtering process by the group classification unit 312. However, even if the comment information ratio of the specific group is equal to or greater than a threshold, step S107 may be executed so as not to generate object 541.

[0105] In step S105, the object generation unit 313 calculates the object ratio of the group that satisfies the judgment condition in step S103 (hereinafter, for convenience, referred to as the "selected group"). The object generation unit 313 resets the total value of the comment information ratios of all selected groups to 100%, and calculates the ratio of the comment information ratios between the selected groups as the object ratio. In other words, the object ratio is a parameter that represents the magnitude of the comment information ratio between the selected groups, and a group with a large object ratio can be said to be a group with many similar comments from viewers among the selected groups.

[0106] Now, the object ratio will be described with reference to FIG.

[0107] In FIG. 10, when the threshold is set to 14% (see (1) in FIG. 10), three groups ("Emotional", "Seriously", and "Impossible Game") are selected. In this case, the object ratios of the "Emotional", "Seriously", and "Impossible Game" groups are calculated to be 60% (=45.0 / (45.0+15.4+14.3)×100), 21% (=15.4 / (45.0+15.4+14.3)×100), and 19% (=14.3 / (45.0+15.4+14.3)×100), respectively. When the threshold is set to 15% (see (2) in FIG. 10), two groups ("Emotional" and "Seriously") are selected. In this case, the object ratios for the "Emotional" and "Seriously" groups are calculated to be 75% (=45.0 / (45.0+15.4) x 100) and 25% (=15.4 / (45.0+15.4) x 100), respectively. Thus, even if the "Emotional" group is the selected group, the object ratio will differ depending on the number of selected groups. Note that the object ratio between selected groups corresponds to the ratio of comment information proportions between the selected groups, since the total denominator value at the time of calculation is the same.

[0108] In step S106, the object generating unit 313 generates an object 541 corresponding to the object ratio calculated in step S105. As described above, the group for which the object 541 is generated is a group in which the majority opinion of the viewers is reflected. The distributor can easily understand the majority opinion of the viewers by visually checking the object 541 without checking all the comments. In addition, the object 541 is generated so that the object ratio is expressed as a difference in the display mode. Therefore, even when a plurality of objects 541 are generated and displayed, the distributor can easily understand the group in which the majority opinion of the viewers is particularly large, based on the difference in the display mode of the object 541.

[0109] Here, the display mode of the generated object 541 will be described with reference to FIG.

[0110] FIG. 10 shows a first object 541-1 corresponding to the "emotional" group, a second object 541-2 corresponding to the "seriously" group, and a third object 541-3 corresponding to the "impossible" group (see (1) in FIG. 10). The first object 541-1, the second object 541-2, and the third object 541-3 each have a different area. Each object 541 has a circular shape with an area corresponding to the object ratio. Therefore, based on the size of the area of ​​the object 541, it is possible to grasp the group with a large number of majority opinions of the viewers. In addition, each object 541 is provided with text representing the group. That is, the first object 541-1, the second object 541-2, and the third object 541-3 are provided with the texts "emotional", "seriously", and "impossible" respectively.

[0111] In object 541, the size of the area indicates the proportion of comment information, and the text indicates representative words of similar comments. In other words, a large area of ​​object 541 means that there are many similar comments included in the group. Therefore, even when multiple objects 541 are generated and displayed, the distributor can easily grasp, in particular, the majority opinion and requests of the viewers by visually checking the size of the area of ​​object 541 and the text added to object 541. Then, the distributor can perform live video distribution based on the grasped majority opinion and requests.

[0112] Although the configuration in which the object ratio is converted to correspond to the area of ​​the object 541 has been described with reference to FIG. 10, the parameter of the object 541 to which the object ratio corresponds is not limited to the area. The object ratio may be converted to correspond to the radius of the object 541. Even in this case, since the multiple objects 541 are displayed in different sizes, it is possible to grasp a group with a large number of majority opinions of the viewers, based on the size of the object 541. Moreover, the display mode of the object 541 shown in FIG. 10 is one example, and the shape of the object 541 may be not only circular but also polygonal. Other display modes of the object 541 will be described later.

[0113] In step S107, the live video data generating unit 314 generates the live video data 500 (see FIG. 2). For example, the display unit 340 of the distributor information terminal 300 displays a game screen, a captured video of the distributor, received comment information, and a generated object 541. In this case, the live video data generating unit 314 can generate the live video data 500 using the image data of the display unit 340 captured by the imaging unit 370 and the sound data input by the sound input / output unit 350. The live video data generating unit 314 can also generate the live video data 500 according to a predetermined display layout using the image data captured individually. Furthermore, the live video data generating unit 314 can generate the live video data 500 using not only the image data but also acquired data such as game screen data.

[0114] When step S107 is executed, the live video data generation process (S100) ends. For example, the live video data 500 is as shown in Fig. 2. As described above, the live video data 500 is transmitted to the distribution server 200 and provided in the live video distribution service.

[0115] In the live video data generation process according to this embodiment, live video data 500 is transmitted in real time, but in the object display area 540 in the live video data 500, objects 541 corresponding to the comment information ratio are generated for each predetermined period, and the objects 541 generated for each period are displayed in a switching manner. As a result, the objects 541 are updated in accordance with the progress of the live video to be distributed, so that the distributor can always grasp the latest majority opinion of the viewers.

[0116] [4. Display of Object 541] The display mode of the object 541 is not limited to that shown in Fig. 10. Here, another example of the display mode of the object 541 will be described with reference to Fig. 11(A) to Fig. 11(C).

[0117] Each of Figs. 11(A) to 11(C) is an example of an object 541 generated in the live video data generation process according to an embodiment of the present invention.

[0118] 11(A) to 11(C), a first object 541-1 (the "Emotional" group) having a first object ratio, a second object 541-2 (the "Seriously?" group) having a second object ratio, and a third object 541-3 (the "Impossible Game" group) having a third object ratio are displayed. Here, the object ratios decrease in order from the first object ratio, the second object ratio, and the third object ratio.

[0119] In FIG. 11(A), a first object 541-1, a second object 541-2, and a third object 541-3 are shown in the object display area 540, and these objects 541 are controlled to move within the area of ​​the object display area 540. When the object 541 is moved, the object 541 may be controlled to bounce back due to collision between the objects 541 or collision between the object 541 and the boundary of the object display area 540. In addition, the moving speed of the object 541 may be changed according to the size of the object ratio. For example, the moving speed of the object 541 is slowed as the object ratio increases. This makes it easier to view the first object 541-1, which moves slowly, than the third object 541-3, which moves quickly, and makes it easier for the distributor to view the first object 541-1 and grasp the majority opinion of the viewers. In this way, one of the display modes of the object 541 is the movement of the object 541.

[0120] In Fig. 11(B), the colors of the first object 541-1, the second object 541-2, and the third object 541-3 are different from each other. For example, as the object ratio becomes smaller, the color of the object 541 is lightened, or the text added to the object 541 is made closer to the color of the background of the object 541. In this case, the first object 541-1 is easier to see than the third object 541-3, and the distributor can easily see the first object 541-1 and grasp the majority opinion of the viewers. In this way, the color of the object 541 can be given as one of the display modes of the object 541.

[0121] In FIG. 11(C), objects 541 are displayed in a superimposed manner. Specifically, a second object 541-2 is superimposed on a third object 541-3, and a first object 541-1 is further superimposed on the second object 541-2. That is, the first object 541-1 is displayed at the top. In this case, the distributor can easily grasp the majority opinion of the viewers by visually checking the object 541 displayed at the top. In this way, the order in which the objects 541 are superimposed can be cited as one of the display modes of the objects 541.

[0122] In the live video distribution service using the live video distribution system 1000 according to the present embodiment, comments posted by viewers are classified into groups for each similar comment, and a group including a larger number of similar comments is displayed as an object 541 on the display unit 340 of the information terminal for distributor 300. That is, the displayed object 541 reflects the majority opinion of the viewers. In addition, even if there are multiple groups classified as the majority opinion (when multiple objects 541 are generated), it is possible to grasp the mainstream majority opinion from the relative size of the multiple displayed objects by making the size of the object 541 correspond to the number of comments in the group (size based on the object ratio). In addition, the object 541 is attached with text representing the comments in the group. Therefore, by visually checking the object 541, the distributor can easily grasp the majority opinion of the viewers and distribute the live video in response to the comments of the viewers. This gives the viewer the impression that he or she is having a conversation with the distributor, and the distributor can communicate more closely with the viewer. In addition, the viewer can feel the live video as something familiar by watching the live video that is close to his or her comments. In this way, the live video distribution service becomes more exciting due to the increased sense of unity between the distributor and the viewers. In addition, depending on the live video distribution service, the tasks of the distributor may be complicated. For example, in the case of distribution of a gameplay video, the distributor must provide a live voice commentary while playing the game, and must check the comments posted by the viewers one by one and have a dialogue or reaction in response to the comments. However, the live video distribution system 1000 makes it easy to grasp the majority opinion of the viewers by using the object 541, and therefore can reduce the burden on the distributor in at least responding to the comments, among the tasks that are carried out simultaneously.

[0123] Although the present invention has been described above with reference to the drawings, the present invention is not limited to the above-mentioned embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, a person skilled in the art can add, delete, or modify components as appropriate based on the embodiment (including modifications), and the modifications are included in the scope of the present invention as long as they include the gist of the present invention. Furthermore, the modifications of the above-mentioned embodiment can be combined as appropriate as long as there are no contradictions between them.

[0124] Even if there are other effects and advantages different from those brought about by the aspects of the above-mentioned embodiment, those which are clear from the description in this specification or which can be easily predicted by a person skilled in the art are naturally understood to be brought about by the present invention. [Explanation of symbols]

[0125] 200: distribution server, 210: control unit, 211: live video data processing unit, 212: comment information processing unit, 220: storage unit, 230: communication unit, 300: information terminal for distributor, 310: control unit, 311: comment information acquisition unit, 312: group classification unit, 313: object generation unit, 314: live video data generation unit, 320: storage unit, 330: communication unit, 340: display unit, 350: sound input / output unit, 360: operation unit, 370: imaging unit, 400: information terminal for viewer, 410: control unit, 411: comment information generation unit, 420: storage unit, 430: communication unit, 440: display unit, 450: sound input / output unit, 460: operation unit, 500: live video data, 510: game play video, 520: Distributor video, 530: Comment display area, 540: Object display area, 541: Object, 541-1: First object, 541-2: Second object, 541-3: Third object, 1000: Live video distribution system

Claims

1. Receive comments posted on the displayed video, classifying the received comments into at least a first group and a second group during each of a first period and a second period that are sequentially set; calculating a first ratio of the number of comments classified into the first group to the number of comments received during each of the first period and the second period; determining a first object having a first display manner for identifying the first group when the first ratio is equal to or greater than a first threshold value during each of the first period and the second period; a program for causing a computer to execute a program for switching and displaying the first object determined based on the first ratio in the first period to the first object determined based on the first ratio in the second period, the program comprising: The start of the second period is before the end of the first period.

2. Receiving comments posted on a displayed video, classifying the received comments into at least a first group and a second group during each of a first period and a second period that are sequentially set; calculating a first ratio of the number of comments classified into the first group to the number of comments received during each of the first period and the second period; determining a first object having a first display manner for identifying the first group when the first ratio is equal to or greater than a first threshold value during each of the first period and the second period; a program for causing a computer to execute a program for switching and displaying the first object determined based on the first ratio in the first period to the first object determined based on the first ratio in the second period, the program comprising: When the number of the comments received in each of the first period and the second period is equal to or greater than a set value, each of the first period and the second period is terminated.

3. calculating a second ratio of the number of comments classified into the second group corresponding to the number of comments received during each of the first time period and the second time period; determining a second object having a second display manner for identifying the second group when the second ratio is equal to or greater than a second threshold during at least one of the first period and the second period; 3. The program according to claim 1, further comprising: determining the first display manner and the second display manner in accordance with the first ratio and the second ratio, and displaying the first object and the second object.

4. the first display manner and the second display manner are a first area of ​​the first object and a second area of ​​the second object, respectively; The program according to claim 3 , wherein the second area relative to the first area corresponds to the second ratio relative to the first ratio.

5. Receiving comments posted on a displayed video; classifying the received comments into at least a first group and a second group during each of a first period and a second period that are sequentially set; calculating a first ratio of the number of comments classified into the first group to the number of comments received during each of the first period and the second period; determining a first object having a first display manner for identifying the first group when the first ratio is equal to or greater than a first threshold value during each of the first period and the second period; a program for causing a computer to execute a program for switching and displaying the first object determined based on the first ratio in the first period to the first object determined based on the first ratio in the second period, the program comprising: calculating a second ratio of the number of comments classified into the second group corresponding to the number of comments received during each of the first time period and the second time period; determining a second object having a second display manner for identifying the second group when the second ratio is equal to or greater than a second threshold during at least one of the first period and the second period; determining the first display manner and the second display manner in accordance with the first ratio and the second ratio, and displaying the first object and the second object; When the second object is not determined based on the second ratio in the first period and the second object is determined based on the second ratio in the second period, the second object is displayed so as to be superimposed on the first object.

6. A program described in any one of claims 1, 2, and 5, wherein the first display mode is the area, color, or movement of the first object.

7. The program according to claim 1 , wherein the first display manner of the first object in the first period and the second period differs based on the first ratio.

8. The program according to claim 7 , wherein the first display aspect based on the first ratio is a shade of a color of the first object or a size of text included in the first object.

9. The program according to claim 1 , wherein at least one of the first period and the second period is started based on an instruction request from a distributor who provides the video.

10. 10. The program according to claim 9, wherein the instruction request from the distributor is linked to a predetermined eye movement or face movement by the distributor, or a predetermined keyword uttered by the distributor.

11. the first object includes text representing the first group; The program according to claim 1 , 2 , or 5 , wherein the text is determined from among words registered in advance.

12. the first object includes text representing the first group; The program of claim 1 , 2 , or 5 , wherein the text is determined by extracting words in the comments classified into the first group.

13. The program of claim 1 , 2 , or 5 , further comprising: classifying the received comments into at least the first group and the second group using a deep learning model trained based on previously received comments.

14. The program according to claim 1 , wherein the video is a gameplay video.

Images