Program

Abstract

To provide a program for classifying comments posted by viewers into groups of similar comments and displaying a group including more similar comments as an object in a live video distribution service.SOLUTION: Receiving comments posted for a displayed video, classifying the received comments into at least a first group and a second group in each of a first period and a second period that are sequentially set, and calculating a first ratio of the number of comments classified into the first group to the total number of the received comments; A first object determined based on a first ratio in a first time period is switched to a first object determined based on a first ratio in a second time period and displayed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] An embodiment of the present invention relates to a program for classifying and displaying comments posted in real time by viewers in a live video distribution service. [Background technology]

[0002] In recent years, various live video distribution services have become popular along with the increasing performance and functionality of information terminals such as smartphones and the increasing speed of communication over the Internet. For example, a live video distribution service is now being provided that distributes videos of a broadcaster playing a game (i.e., gameplay videos) along with the broadcaster's audio commentary to the information terminals of many viewers in real time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-170093 Summary of the Invention [Problem to be solved by the invention]

[0004] Live video streaming services allow broadcasters to stream live videos and allow viewers to post comments on the live videos. Comments posted by viewers are displayed on the broadcaster's information device, allowing the broadcaster to view the comments and broadcast live videos in response to the viewers' comments, giving the viewer the impression of a conversation and enabling closer communication with the viewer. However, with live video streaming services, many viewers post comments, making it often difficult for the broadcaster to check all of the comments. In addition, it can be difficult for the broadcaster to instantly grasp, among all the comments posted by viewers, the majority opinion that is supported or sympathized with by the most viewers, or the current mainstream opinion.

[0005] One embodiment of the present invention aims to provide a program for a live video distribution service that classifies comments posted by viewers into groups of similar comments and displays groups containing more similar comments as objects. [Means for solving the problem]

[0006] A program according to one embodiment of the present invention causes a computer to receive comments posted on a displayed video, classify the received comments into at least a first group and a second group in each of a first period and a second period that are set sequentially, calculate a first ratio of the number of comments classified into the first group to the number of received comments in each of the first period and the second period, determine a first object having a first display mode for identifying the first group when the first ratio is equal to or greater than a first threshold in each of the first period and the second period, and switch and display the first object determined based on the first ratio in the first period to the first object determined based on the first ratio in the second period.

[0007] The first display manner may be the area, color, or movement of the first object.

[0008] A second ratio of the number of comments classified into a second group corresponding to the number of comments received in each of the first period and the second period may be calculated, and when the second ratio is equal to or greater than a second threshold in at least one of the first period and the second period, a second object having a second display mode for identifying the second group may be determined, and the first display mode and the second display mode may be determined according to the first ratio and the second ratio to display the first object and the second object.

[0009] The first display manner and the second display manner may be a first area of ​​the first object and a second area of ​​the second object, respectively, and the second area relative to the first area may correspond to a second ratio relative to the first ratio.

[0010] When the second object is not determined based on the second ratio in the first period, but is determined based on the second ratio in the second period, the second object may be displayed so as to be superimposed on the first object.

[0011] The first display manner of the first object in the first time period and the second time period may differ based on a first ratio.

[0012] The first display manner based on the first ratio may be the shade of the color of the first object or the size of the text included in the first object.

[0013] The start of the second period may be before the end of the first period.

[0014] At least one of the first period and the second period may be started based on an instruction from a distributor who provides the video.

[0015] The instructions from the broadcaster may be linked to a predetermined eye movement or facial movement by the broadcaster, or a predetermined keyword uttered by the broadcaster.

[0016] When the number of comments received in each of the first period and the second period is equal to or greater than a set value, each of the first period and the second period may be ended.

[0017] The first object includes text representing the first group, and the text may be determined from among pre-registered words.

[0018] The first object includes text representing the first group, and the text may be determined by extracting words in the comments classified into the first group.

[0019] A deep learning model trained based on previously received comments may be used to classify the received comments into at least a first group and a second group.

[0020] The video may be a game broadcast video. [Effects of the Invention]

[0021] According to one embodiment of the present invention, in a live video streaming service, comments posted by viewers are classified into groups based on similar comments, and groups containing more similar comments are displayed as objects on the display unit of a broadcaster's information terminal. In other words, the displayed objects reflect the majority opinion of viewers. Even if multiple groups are classified as the majority opinion, the size of the objects corresponds to the number of comments in each group, making it possible to grasp the dominant majority opinion from the relative sizes of the multiple displayed objects. Furthermore, text representative of the comments in each group is attached to the objects. Therefore, broadcasters can easily grasp the majority opinion of viewers by visually checking the objects and broadcast live video in response to the viewers' comments. This gives viewers the impression that they are having a conversation with the broadcaster, allowing the broadcaster to communicate more closely with the viewers. Furthermore, by watching live video that is tailored to their own comments, viewers can feel closer to the live video. The increased sense of unity between broadcasters and viewers boosts the excitement of the live video streaming service, thereby increasing the number of viewers of the live video streaming service. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a block diagram showing a configuration of a live video distribution system according to an embodiment of the present invention. [Figure 2] 5 is a schematic diagram illustrating the configuration of live video data 500 generated using a live video distribution system according to an embodiment of the present invention. FIG. [Figure 3] FIG. 2 is a block diagram showing the configuration of a distribution server according to an embodiment of the present invention. [Figure 4] 1 is a block diagram showing a configuration of a distributor information terminal according to an embodiment of the present invention; [Figure 5] 1 is a block diagram showing a configuration of a viewer information terminal according to an embodiment of the present invention; [Figure 6]FIG. 10 is a sequence diagram illustrating a live video distribution process according to an embodiment of the present invention. [Figure 7] 10 is a flowchart illustrating a live video data generation process according to an embodiment of the present invention. [Figure 8] FIG. 10 is a schematic diagram illustrating a process of classifying comment information into groups in the live video data generation process according to an embodiment of the present invention. [Figure 9] FIG. 10 is a schematic diagram illustrating a process of acquiring comment information in a live video data generation process according to an embodiment of the present invention. [Figure 10] FIG. 10 is a schematic diagram illustrating a process of generating an object in a live video data generation process according to an embodiment of the present invention. [Figure 11] 10 is a diagram illustrating an example of an object generated in the live video data generation process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the present invention will be described with reference to the drawings, etc. However, the embodiments of the present invention can be implemented in various forms without departing from the gist of the technical idea thereof, and should not be construed as being limited to the embodiments described below.

[0024] In order to clarify the description, the drawings may be shown schematically in terms of width, thickness, shape, etc. compared to the actual embodiment. In addition, in the drawings, components having the same or similar functions are designated by the same reference numerals or reference numerals with an alphabetical character added thereto, and repeated description of those components may be omitted.

[0025] For purposes of the specification and claims, the following terms are defined as follows:

[0026] A "program" refers to an instruction or group of instructions executed by a processor in a computer equipped with a processor and memory. A "computer" is a general term referring to the entity that executes a program. For example, when a program is executed by a server (or a client), the "computer" refers to the server (or the client). When a "program" is executed by distributed processing between a server and a client, the "computer" includes both the server and the client. In this case, the "program" includes "a program executed by the server" and "a program executed by the client." Similarly, when a "program" is processed in a distributed manner among multiple servers, the "computer" includes the multiple servers, and the "program" includes each program executed by each server.

[0027] A "live video streaming service" refers to a service that distributes live video in real time over a network. The live video may be, for example, but is not limited to, gameplay video.

[0028] "Distributor" means a person who provides live video for distribution in a live video distribution service.

[0029] "Viewer" means a person who watches live videos and posts comments in a live video distribution service.

[0030] [1. Overview of the Live Video Distribution System 1000] FIG. 1 is a block diagram showing the configuration of a live video distribution system 1000 according to an embodiment of the present invention.

[0031] The live video distribution system 1000 includes a distribution server 200, a distributor information terminal 300, and multiple viewer information terminals 400 (400-1, 400-2, ..., 400-n in FIG. 1). The distribution server 200, the distributor information terminal 300, and the multiple viewer information terminals 400 are communicably connected via a network NW such as the Internet or a LAN. The network NW may be wired or wireless. The distributor information terminal 300 and the viewer information terminal 400 are clients that can receive services provided by the distribution server 200.

[0032] The distribution server 200 is an information processing device that provides a live video distribution service or various services related to the live video distribution service to the distributor information terminal 300 and the viewer information terminal 400. The live video distribution service can be provided by executing a live video distribution program. The various services in the distribution server 200 may include, for example, services such as login processing or synchronization processing when executing the live video distribution service. The various services may also include a comment posting function. The live video distribution program is recorded in a storage device included in the distribution server 200, a recording medium readable by the distribution server 200, or a database to which the distribution server 200 can communicate via the network NW. Although FIG. 1 illustrates the distribution server 200 as a single information processing device, it may also be composed of multiple information processing devices.

[0033] The distributor's information terminal 300 is an information terminal used by a distributor. The viewer's information terminal 400 is an information terminal used by a viewer. The distributor's information terminal 300 and the viewer's information terminal 400 are each an information processing device such as a personal computer, a tablet, a smartphone, a home video game console, or a portable video game console. The distributor's information terminal 300 and the viewer's information terminal 400 can be communicatively connected to the distribution server 200 via the network NW. The distributor's information terminal 300 communicates with the distribution server 200 to execute a live video distribution program and transmit live video data corresponding to the live video to the distribution server 200. The viewer's information terminal 400 communicates with the distribution server 200 to execute a live video distribution program and receive live video data. The viewer's information terminal 400 allows viewers who watch live video to post comments, and comment information including the posted comments can be transmitted from the viewer's information terminal 400 to the distribution server. In this way, in the live video distribution system 1000, by executing the live video distribution program, live video data can be transmitted in real time from the distributor's information terminal 300 to the distribution server 200, and the viewer's information terminal 400 can access the distribution server 200, receive the live video data transmitted from the distribution server 200 in real time at the viewer's information terminal 400, and transmit comment information from the viewer's information terminal 400 to the distribution server 200 in real time.

[0034] The live video distribution system 1000 can distribute various live videos. For example, a broadcaster can distribute a live video in which the broadcaster makes comments based on comments posted by viewers, along with a video of the broadcaster himself / herself. Furthermore, a broadcaster can also distribute a live video in which the broadcaster provides audio commentary while reacting to comments posted by viewers, along with a video of the broadcaster playing a game (so-called gameplay video). Games include, but are not limited to, online games such as social games or mobile games.

[0035] Live video data corresponding to live video provided by a broadcaster is generated by the broadcaster's information terminal 300, but part of the live video data may be data acquired by connecting to and communicating with another server via the network NW. For example, game screen data in a gameplay video may not only be data captured by the broadcaster's information terminal 300, but also game screen data acquired and transmitted from a game server via the network NW.

[0036] [2. Composition of Live Video Data 500] The structure of live video data generated by the distributor information terminal 300 will be described with reference to FIG.

[0037] FIG. 2 is a schematic diagram illustrating the configuration of live video data 500 generated using a live video distribution system 1000 according to an embodiment of the present invention. For convenience, FIG. 2 illustrates the configuration of live video data 500 displayed on a screen. That is, in a live video distribution service, live video corresponding to live video data 500 is provided to and displayed on a viewer information terminal 400. Note that while live video data 500 is generated on a distributor information terminal 300, the live video provided to the viewer information terminal 400 may have a different display layout from that displayed on the distributor information terminal 300. In this case, the live video provided to the viewer information terminal 400 may also be provided to and displayed on the distributor information terminal 300.

[0038] As shown in FIG. 2, the live video data 500 includes a gameplay video 510, a broadcaster video 520, a comment display area 530, and an object display area 540. Although not shown in FIG. 2, the live video data 500 also includes audio data. The gameplay video 510 is a video that changes in real time according to the progress of the game played by the broadcaster. The broadcaster video 520 is a video that changes in real time according to the broadcaster's movements. The broadcaster video 520 may be a video filmed by the broadcaster, or may be a video (e.g., an avatar) that has been processed based on a video filmed by the broadcaster. The comment display area 530 is an area where comments posted by viewers are sequentially displayed. A certain number of comments, including the most recent comment, are displayed in the comment display area 530. Therefore, comments posted before the certain number of comments are sequentially scrolled and disappear from the comment display area 530. The object display area 540 is an area where objects 541 (first object 541-1, second object 541-2, and third object 541-3 in FIG. 8) are displayed. The objects 541 are created by classifying comments posted by viewers into groups of similar comments, and visually displaying groups that include more similar comments. Therefore, by visually checking the objects 541, the broadcaster can grasp the majority opinion or requests of viewers. Although three objects are displayed in the object display area 540 in FIG. 2, the number of objects 541 is not limited to this. Details of the generation process and display mode of the objects 541 will be described later.

[0039] Note that, with reference to FIG. 2, the live video data 500 including the gameplay video 510 and the broadcaster video 520 has been described, but the live video data 500 may have a configuration including only one of the gameplay video 510 and the broadcaster video 520.

[0040] 3. Configuration of components of live video distribution system 1000 The detailed configurations of the distribution server 200, the distributor information terminal 300, and the viewer information terminal 400 included in the live video distribution system 1000 will be described with reference to FIGS.

[0041] [3-1. Configuration of distribution server 200] Fig. 3 is a block diagram showing the configuration of a distribution server 200 according to one embodiment of the present invention. As shown in Fig. 3, the distribution server 200 includes a control unit 210, a storage unit 220, and a communication unit 230. Note that the distribution server 200 may include further components.

[0042] The control unit 210 includes a processor (arithmetic processing device) such as a CPU (Central Processing Unit) and a storage device such as a RAM. The control unit 210 executes programs stored in the storage device using the processor to realize various functions related to the live video distribution service. For example, the control unit 210 can realize a live video data processing unit 211 and a comment information processing unit 212 as functions.

[0043] The live video data processing unit 211 acquires live video data 500 transmitted from the distributor's information terminal 300 and transmits the acquired live video data 500 to the viewer's information terminal 400, which is communicatively connected to the distribution server 200. The live video data 500 acquired by the live video data processing unit 211 may be stored in the storage unit 220. The live video data processing unit 211 can also change or modify the configuration of the live video data 500 transmitted from the distributor's information terminal 300. For example, the live video data processing unit 211 can change and modify the configuration of the live video data 500 to a display layout for live video distribution. The live video data processing unit 211 can also transmit the acquired live video data 500 or the changed or modified live video data 500 not only to the viewer's information terminal 400 but also to the distributor's information terminal 300.

[0044] The comment information processing unit 212 acquires comment information transmitted from the viewer information terminal 400 and transmits the comment information to the distributor information terminal 300 and the viewer information terminal 400. The comment information acquired by the comment information processing unit 212 may be stored in the storage unit 220. The comment information transmitted to the distributor information terminal 300 and the viewer information terminal 400 may be all or a part of the acquired comment information. For example, if the comment information includes a viewer ID, a viewer nickname, and a comment, the comment information processing unit 212 can extract the comment from the acquired comment information and transmit the extracted comment as comment information to the distributor information terminal 300 and the viewer information terminal 400. The comment information processing unit 212 can also perform comment filtering. Specifically, the comment information processing unit 212 can remove comments containing inappropriate or offensive language from the acquired comment information and generate comment information to be transmitted to the distributor information terminal 300 and the viewer information terminal 400. The comment information transmitted to the distributor information terminal 300 and the comment information transmitted to the viewer information terminal 400 may be the same or different.

[0045] The storage unit 220 is a recording device (recording medium) that can permanently store and rewrite information, such as a nonvolatile memory or a hard disk drive. For example, the storage unit can store live video data 500 acquired by the live video data processing unit 211 or comment information acquired by the comment information processing unit 212.

[0046] The communication unit 230 is a communication module that connects to the network NW and transmits and receives data or information under the control of the control unit 210. If the network NW is wireless, the communication unit 230 may include a communication module for infrared communication or short-range wireless communication.

[0047] [2-2. Configuration of Distributor Information Terminal 300] Fig. 4 is a block diagram showing the configuration of a distributor's information terminal 300 according to one embodiment of the present invention. As shown in Fig. 4, distributor's information terminal 300 includes a control unit 310, a storage unit 320, a communication unit 330, a display unit 340, a sound input / output unit 350, an operation unit 360, and an imaging unit 370. Note that distributor's information terminal 300 may include additional components.

[0048] The control unit 310 has the same configuration as the control unit 210. However, the control unit 310 executes a program using a processor to mainly realize various functions related to the live video data generation process in the live video distribution service. For example, the control unit 310 can realize the following functions: a comment information acquisition unit 311, a group classification unit 312, an object generation unit 313, and a live video data generation unit 314.

[0049] The comment information acquisition unit 311 acquires comment information transmitted from the distribution server 200. The comment information acquisition unit 311 may also perform a comment filtering process. Specifically, if the transmitted comment information includes a word (exclusion keyword) preset in the distributor information terminal 300 or a comment including an inappropriate or offensive expression, the comment information acquisition unit 311 can exclude such comment information from the acquired comment information.

[0050] The group classification unit 312 classifies the comment information acquired by the comment information acquisition unit 311 into groups of similar comments.

[0051] The object generation unit 313 calculates the ratio of the number of comment information in a group to the total number of comment information acquired within a predetermined period. Furthermore, the object generation unit 313 determines the ratio of the number of comment information in a group to the total number of comment information acquired within a predetermined period, and if there are multiple groups that satisfy the determination condition, generates an object 541 that corresponds to the ratio of the number of comment information in each group and has a relative change.

[0052] The live video data generator 314 generates live video data 500 including a gameplay video 510, a broadcaster video 520, a comment display area 530, and an object display area 540.

[0053] The functions of the comment information acquisition unit 311, group classification unit 312, object generation unit 313, and live video data generation unit 314 will be described in detail later.

[0054] The storage unit 320 has the same configuration as the storage unit 220. Data or information used in the live video data generation process is stored in the storage unit 320. For example, the storage unit 320 can store thresholds, setting values, excluded keywords, comment information, data related to the object 541, and the like.

[0055] The communication unit 330 has the same configuration as the communication unit 230 .

[0056] The display unit 340 has a display area that displays various display images (e.g., game play video 510, broadcaster video 520, comment display area 530, object display area 540, live video, etc.) in accordance with the control of the control unit 310. The display unit 340 is, for example, a display device such as a liquid crystal display or an organic EL display.

[0057] The sound input / output unit 350 inputs and outputs sound data. The sound input / output unit 350 is configured with, for example, a microphone and a speaker. In this case, sound data is input by the microphone of the sound input / output unit 350, and sound is output by the speaker of the sound input / output unit 350. The sound input / output unit 350 can also be used to output game music, etc.

[0058] The operation unit 360 is an operation device that outputs a signal (e.g., a signal indicating a command or information) according to an operation by the broadcaster to the control unit 310. For example, the broadcaster can use the operation unit 360 to play a game or perform broadcast operations. The operation unit 360 may be a touch sensor arranged on the surface of the display unit 340.

[0059] The imaging unit 370 is an imaging device (camera) that converts an image of an object into a signal. For example, a broadcaster can use the imaging unit 370 to capture a game screen in which the broadcaster is playing and the broadcaster himself / herself, and generate a gameplay video 510 and a broadcaster video 520.

[0060] [2-3. Configuration of viewer information terminal 400] Fig. 5 is a block diagram showing the configuration of a viewer information terminal 400 according to one embodiment of the present invention. As shown in Fig. 5, viewer information terminal 400 includes a control unit 410, a storage unit 420, a communication unit 430, a display unit 440, a sound input / output unit 450, and an operation unit 460. Note that viewer information terminal 400 may include further components.

[0061] The control unit 410 has the same configuration as the control unit 210. However, the control unit 410 executes a program using a processor, and mainly controls the operation of a live video distribution service. The control unit 410 realizes various functions related to the comment generation process. For example, the control unit 410 can realize a comment information generation unit 411 as a function.

[0062] When a comment is input via operation unit 460, comment information generation unit 411 generates comment information including the comment. The comment information may include not only the comment but also a viewer ID or a viewer nickname for identifying the viewer. The comment information generated by comment information generation unit 411 is transmitted to distribution server 200.

[0063] The storage unit 420 has the same configuration as the storage unit 220. The storage unit 320 stores data or information used in the comment generation process. For example, the storage unit 420 can store a viewer ID, a viewer nickname, or the like.

[0064] The communication unit 430 has the same configuration as the communication unit 230 .

[0065] Display unit 440 has the same configuration as display unit 340. Display unit 440 displays live video corresponding to live video data 500 transmitted from distribution server 200, allowing viewers to view the live video in real time.

[0066] The sound input / output unit 450 has the same configuration as the sound input / output unit 350. The speaker of the sound input / output unit 450 can output sound data included in the live video.

[0067] Operation unit 460 has the same configuration as operation unit 360. A viewer can use operation unit 460 to input comments while viewing the live video displayed on display unit 440.

[0068] [3. Live video streaming processing] [3-1. Overall flow of live video streaming processing] With reference to FIG. 6, a live video distribution process executed in the live video distribution system 1000 will be described.

[0069] Fig. 6 is a sequence diagram illustrating a live video distribution process according to an embodiment of the present invention. The sequence diagram shown in Fig. 6 includes steps S100 to S160. For convenience, steps S100 to S160 will be described in order below, but the steps in the live video distribution process may be executed in parallel or the order may be reversed. Furthermore, the live video distribution process may include additional steps.

[0070] In step S100, live video data 500 is generated in the distributor information terminal 300. Details of the live video data generation process in step S100 will be described later.

[0071] In step S110, the generated live video data 500 is transmitted from the distributor information terminal 300 to the distribution server 200 in real time.

[0072] In step S120, the live video data 500 is transmitted in real time from the distribution server 200 to the viewer information terminal 400. As a result, a live video corresponding to the live video data is displayed on the display unit 440 of the viewer information terminal 400, allowing the viewer to view the live video. Although not shown, the live video data 500 may also be transmitted in real time from the distribution server 200 to the distributor information terminal 300.

[0073] In step S130, comment information is generated in viewer information terminal 400. Specifically, when a viewer watching the live video inputs a comment by operating operation unit 460, comment information generation unit 411 generates comment information including the input comment.

[0074] In step S140, the generated comment information is transmitted from viewer information terminal 400 to distribution server 200. That is, distribution server 200 can sequentially receive multiple pieces of comment information generated in one or multiple viewer information terminals 400. The comment information received by distribution server 200 is stored in storage unit 220.

[0075] In step S150, the comment information is sequentially transmitted from the distribution server 200 to the distributor's information terminal 300. As a result, the comments included in the comment information are displayed on the display unit 340 of the distributor's information terminal 300, allowing the distributor to visually check the viewers' comments on the live video.

[0076] In step S160, the comment information is sequentially transmitted from the distribution server 200 to the viewer information terminal 400. As a result, the comments included in the comment information are displayed on the display unit 440 of the viewer information terminal 400, allowing the viewer to visually recognize their own or other people's comments.

[0077] [3-2. Live video data generation process flow] The live video data generation process in step S100 will be described in detail with reference to FIGS.

[0078] Fig. 7 is a flowchart illustrating a live video data generation process according to one embodiment of the present invention. Figs. 8(A) to 8(D) are each a schematic diagram illustrating a process for acquiring comment information in a live video data generation process according to one embodiment of the present invention. Fig. 9 is a schematic diagram illustrating a process for classifying comment information into groups in a live video data generation process according to one embodiment of the present invention. Fig. 10 is a schematic diagram illustrating a process for generating an object 541 in a live video data generation process according to one embodiment of the present invention.

[0079] 7, the live video data generation process (S100) includes steps S101 to S107. Steps S101 to S107 will be described below in order with reference to FIGS.

[0080] In step S101, the comment information acquisition unit 311 acquires comment information. When a viewer posts a comment (i.e., comment information is transmitted from the viewer information terminal 400 to the distribution server 200 (see step S140 in FIG. 6)), the comment information is transmitted from the distribution server 200 to the distributor information terminal 300 (see step S160 in FIG. 6). Therefore, the comment information acquisition unit 311 can acquire the comment information received at the distributor information terminal 300. The comment information acquired by the comment information acquisition unit 311 is associated with the reception date and time or reception time at which the distributor information terminal 300 received the comment information, or the acquisition date and time or acquisition time acquired by the comment information acquisition unit 311, and is stored in the storage unit 320.

[0081] In step S102, the group classification unit 312 classifies the comment information acquired in step S101 into groups. Specifically, the group classification unit 312 determines the comments included in the comment information and classifies the comment information into groups of similar comments. Any conventional method can be applied to classify the comment information. For example, the group classification unit 312 can divide the comment into phrases and determine the similarity of one of the divided words to determine the group to which the comment information belongs. The group classification unit 312 can also determine the group to which the comment information belongs by using a comment classification model obtained by learning using machine learning such as a neural network or deep learning, which outputs a group when a comment is input. In this case, the group classification unit 312 can also acquire the group to which the comment information belongs by connecting to an external server via a network NW for communication.

[0082] The comment information is linked to the determined group and stored in the storage unit 320. For example, the comment information linked to the group may be stored as a database as shown in FIG. 8. In FIG. 8, the database items include the "reception time" when the broadcaster information terminal 300 received the comment information, the "viewer ID," "viewer nickname," and "comment" included in the received comment information, and the "group" determined in step S102. For example, referring to FIG. 8, comments expressing the viewer's emotions, such as "I'm shaking!" and "I'm so moved!", are classified into the "emotional" group. Negative comments, such as "I can't do this anymore," are classified into the "impossible game" group. Comment information within the same group are similar to each other, and a word such as "emotional" or "impossible game" that represents a group can be considered to be text that encompasses the comments of multiple comment information within the group. Such text expressing the similarity of groups may be generated during the process of classifying comment information by the group classification unit 312, or may be set in advance.

[0083] When classifying the groups, the group classification unit 312 may classify comment information that includes excluded keywords or comments that include inappropriate or offensive expressions into one specific group. This can be called a filtering process by the group classification unit 312.

[0084] In step S103, the object generation unit 313 calculates the ratio of the number of comment information items in a group to the total number of comment information items acquired within a predetermined period (hereinafter, for convenience, referred to as "comment information ratio"). The comment information ratio is calculated for each group. The comment information ratio is also calculated for each predetermined period.

[0085] Here, the period for which the comment information ratio is calculated will be described with reference to Figures 9(A) to 9(D). Each of Figures 9(A) to 9(D) is an example of the period for which the comment information ratio is calculated.

[0086] 9A, the first period T1, the second period T2, and the third period T3 are set in order without overlapping. Similar periods are set after the third period T3, but the description thereof will be omitted here.

[0087] In the first period T1, the comment information ratio is calculated based on the comment information acquired between time t1 and time t2. Similarly, in the second period T2, the comment information ratio is calculated based on the comment information acquired between time t2 and time t3, and in the third period T3, the comment information ratio is calculated based on the comment information acquired between time t3 and time t4. Here, the time at which the comment information is acquired may be the time at which the comment information is received by the broadcaster information terminal 300, or the time at which the comment information is acquired by the comment information acquisition unit 311. For example, the object generation unit 313 can calculate the comment information ratio within each period based on the comment information associated with the "group" and "received time" shown in FIG. 8.

[0088] In the example of Figure 9(A), the time corresponding to the length of the period is set in advance. As a result, each period is set for each set time, and the comment information ratio is calculated. Therefore, the first period T1, the second period T2, and the third period T3 have the same length.

[0089] 9B, the first period T1, the second period T2, and the third period T3 are set in this order, overlapping each other. Similar periods are set after the third period T3, but a description thereof will be omitted here.

[0090] In the first period T1, the comment information ratio is calculated based on the comment information acquired between time t1 and time t3. In the second period T2, the comment information ratio is calculated based on the comment information acquired between time t2, which precedes time t3, and time t5. In the third period T3, the comment information ratio is calculated based on the comment information acquired between time t4, which precedes time t5, and time t7. As described above, the first period T1 and the second period T2 overlap, and the same comment information may be used in calculating the comment information ratio for the first period T1 and the second period T2. Similarly, the second period T2 and the third period T3 overlap, and the same comment information may be used in calculating the comment information ratio for the second period T2 and the third period T3.

[0091] 9(B), not only the first time corresponding to the length of the period, but also the second time (<first time) that the periods overlap is set in advance. As a result, the comment information ratio is calculated by starting a later period before the end of the period, but the first period T1, the second period T2, and the third period T3 have the same length (first time).

[0092] As will be described in detail later, the calculated comment information ratio is used to generate the object 541. When the first period T1 and the second period T2 are independent as in the example of Fig. 9(A), the object 541 may change suddenly. However, when the first period T1 and the second period T2 overlap as in the example of Fig. 9(B), the object 541 can change smoothly.

[0093] 9C, the first period T1 and the second period T2 are set in sequence without overlapping. A similar period is set after the second period T2, but the description thereof will be omitted here.

[0094] In the example of FIG. 9(C), the first period T1 and the second period T2 are set based on an object generation request. Specifically, at time t1, the distributor operates the operation unit 360 to generate an object generation request for generating an object 541. This causes the comment information acquisition unit 311 to start acquiring comment information and the group classification unit 312 to start classifying the comment information. At time t2, another object generation request is generated. The object generation unit 313 calculates the comment information ratio for the first period T1 based on the comment information acquired during the first period T1 from time t1 to time t2. Similarly, at time t3, another object generation request is generated. The object generation unit 313 calculates the comment information ratio for the second period T2 based on the comment information acquired during the second period T2 from time t2 to time t3.

[0095] In the example of FIG. 9(C) described above, the start time and end time of the period are set based on an object generation request generated by the distributor's operation using the operation unit 360, so the lengths of each period are different. However, the object generation request may set only the start time of the period. In this case, by setting a time corresponding to the length of the period in advance, the period started based on the object generation request will end after the predetermined time has elapsed. Since the length of each period is a predetermined time, each period has the same length.

[0096] Furthermore, a broadcaster who provides live video may find it difficult to operate the operation unit 360, for example, when the broadcaster is playing a game. Therefore, the object generation request may be generated based on the broadcaster's eye movement or facial movement in the broadcaster video captured by the imaging unit 370. The object generation request may also be generated based on a keyword uttered by the broadcaster that is input by the sound input / output unit 350. That is, the object generation request may be generated in conjunction with a predetermined action of the broadcaster or a predetermined keyword uttered by the broadcaster.

[0097] In the example of FIG. 9(C), comment information can be acquired at the timing desired by the distributor, and object 541 can be generated.

[0098] 9(D), the first period T1 and the second period T2 are set in order without overlapping. A similar period is set after the second period T2, but the description thereof will be omitted here.

[0099] 9(D), when acquisition of comment information starts at time t1, the number of acquired comments is counted, and the first period T1 ends at time t2 when the number of acquired comment information reaches a set value (i.e., is equal to or greater than the set value). Furthermore, the number of acquired comments is counted again from time t2, and the second period T2 ends at time t3 when the number of acquired comment information reaches a set value (i.e., is equal to or greater than the set value). The object generation unit 313 calculates the comment information ratios for the first period T1 and the second period T2 based on the comment information acquired in each of the first period T1 and the second period T2.

[0100] 9(D), the upper limit of the number of acquired comments is preset as a setting value, so that the number of pieces of comment information acquired in the first period T1 and the second period T2 is the same, but the lengths of the first period T1 and the second period T2 are different.

[0101] 9(D), comment information is acquired without depending on time, and when the upper limit of comment information is acquired, object 541 is generated. Therefore, object 541 can be generated in accordance with the progress of the live video.

[0102] Although each of FIGS. 9(A) to 9(D) has been described above, these are examples of periods for which the comment information ratio is calculated, and the periods for which the comment information ratio is calculated are not limited to these.

[0103] In step S104, the object generation unit 313 determines whether the comment information ratio calculated in step S103 is equal to or greater than a threshold. Here, the threshold is a preset value, and the determination in step S104 is performed for each comment information ratio of the group. When the comment information ratio is equal to or greater than the threshold (step S104: YES), step S105 is performed. On the other hand, when the comment information ratio is less than the threshold (step S104: NO), step S107 is performed. As will be described in detail later, an object 541 is generated for a group whose comment information ratio is equal to or greater than the threshold, and an object 541 is not generated for a group whose comment information ratio is less than the threshold. A group with a high comment information ratio is a group with many similar comments from viewers. In other words, by performing step S104, it is possible to determine a group that reflects the majority opinion of viewers.

[0104] In step S104, a judgment is also made for specific groups (i.e., groups that include comments with excluded keywords, etc.) classified by the filtering process by the group classification unit 312. However, even if the comment information ratio of a specific group is above a threshold, step S107 may be executed to prevent the generation of object 541.

[0105] In step S105, the object generation unit 313 calculates the object ratio of the group that satisfies the judgment condition of step S103 (hereinafter referred to as the "selected group" for convenience). The object generation unit 313 resets the total value of the comment information ratios of all selected groups to 100%, and calculates the ratio of the comment information ratios between the selected groups as the object ratio. In other words, the object ratio is a parameter that represents the magnitude of the comment information ratio between the selected groups, and a group with a large object ratio can be said to be a group with a large number of similar comments from viewers among the selected groups.

[0106] Here, the object ratio will be described with reference to FIG.

[0107] In FIG. 10, when the threshold is set to 14% (see (1) in FIG. 10), three groups ("Emotional," "Seriously," and "Impossible Game") are selected. In this case, the object ratios of the "Emotional," "Seriously," and "Impossible Game" groups are calculated to be 60% (= 45.0 / (45.0 + 15.4 + 14.3) × 100), 21% (= 15.4 / (45.0 + 15.4 + 14.3) × 100), and 19% (= 14.3 / (45.0 + 15.4 + 14.3) × 100), respectively. When the threshold is set to 15% (see (2) in FIG. 10), two groups ("Emotional" and "Seriously") are selected. In this case, the object ratios for the "Emotional" and "Seriously?" groups are calculated as 75% (= 45.0 / (45.0+15.4) x 100) and 25% (= 15.4 / (45.0+15.4) x 100), respectively. Thus, even if the "Emotional" group is the selected group, the object ratio will differ depending on the number of selected groups. Note that the object ratio between selected groups corresponds to the ratio of the comment information proportions between the selected groups, since the total denominator value used in the calculation is the same.

[0108] In step S106, the object generation unit 313 generates an object 541 corresponding to the object ratio calculated in step S105. As described above, the group for which the object 541 is generated is a group that reflects the majority opinion of viewers. The broadcaster can easily understand the majority opinion of viewers by visually checking the object 541 without having to check all of the comments. Furthermore, the object 541 is generated so that the object ratio is expressed as a difference in display mode. Therefore, even when multiple objects 541 are generated and displayed, the broadcaster can easily understand the group that has a particularly large majority opinion of viewers based on the difference in display mode of the object 541.

[0109] Here, the display mode of the generated object 541 will be described with reference to FIG.

[0110] FIG. 10 shows a first object 541-1 corresponding to the "emotional" group, a second object 541-2 corresponding to the "seriously?" group, and a third object 541-3 corresponding to the "impossible game" group (see (1) in FIG. 10). The first object 541-1, the second object 541-2, and the third object 541-3 each have a different area. Each object 541 has a circular shape with an area corresponding to the object ratio. Therefore, based on the size of the area of ​​the object 541, it is possible to identify the group that has a particularly large majority of viewers' opinions. Each object 541 is also accompanied by text representing the group. That is, the first object 541-1, the second object 541-2, and the third object 541-3 are accompanied by the text "emotional," "seriously?", and "impossible game," respectively.

[0111] In object 541, the size of the area indicates the proportion of comment information, and the text indicates representative words of similar comments. In other words, a larger area of ​​object 541 means that there are many similar comments in the group. Therefore, even when multiple objects 541 are generated and displayed, the broadcaster can easily grasp, in particular, the majority opinions and requests of viewers by visually checking the size of the area of ​​object 541 and the text attached to object 541. The broadcaster can then broadcast live video based on the grasped majority opinions and requests.

[0112] Note that, with reference to FIG. 10, a configuration has been described in which the object ratio is converted to correspond to the area of ​​the object 541, but the parameter of the object 541 to which the object ratio corresponds is not limited to the area. The object ratio may be converted to correspond to the radius of the object 541. Even in this case, since the multiple objects 541 are displayed at different sizes, it is possible to identify a group with a particularly large majority of the viewers' opinions based on the size of the object 541. Furthermore, the display form of the object 541 shown in FIG. 10 is one example, and the shape of the object 541 may be not only circular but also polygonal. Other display forms of the object 541 will be described later.

[0113] In step S107, the live video data generation unit 314 generates live video data 500 (see FIG. 2). For example, a game screen, a captured video of the broadcaster, received comment information, and a generated object 541 are displayed on the display unit 340 of the broadcaster information terminal 300. In this case, the live video data generation unit 314 can generate the live video data 500 using image data of the display unit 340 captured by the imaging unit 370 and sound data input by the sound input / output unit 350. The live video data generation unit 314 can also generate live video data 500 that matches a predetermined display layout using individually captured image data. Furthermore, the live video data generation unit 314 can generate the live video data 500 using not only the image data but also acquired data such as game screen data.

[0114] When step S107 is executed, the live video data generation process (S100) ends. For example, the live video data 500 is as shown in Fig. 2. As described above, the live video data 500 is transmitted to the distribution server 200 and provided in the live video distribution service.

[0115] In the live video data generation process according to this embodiment, live video data 500 is transmitted in real time, and objects 541 corresponding to the comment information ratio are generated in the object display area 540 of the live video data 500 for each predetermined period, and the objects 541 generated for each period are displayed in a switching manner. This allows the object 541 to be updated in accordance with the progress of the live video being distributed, allowing the distributor to always grasp the latest majority opinion of viewers.

[0116] [4. Display of Object 541] The display mode of the object 541 is not limited to that shown in Fig. 10. Here, another example of the display mode of the object 541 will be described with reference to Figs. 11(A) to 11(C).

[0117] Each of FIGS. 11(A) to 11(C) is an example of an object 541 generated in the live video data generation process according to one embodiment of the present invention.

[0118] 11(A) to 11(C) show a first object 541-1 (the "Emotional" group) having a first object ratio, a second object 541-2 (the "Seriously?" group) having a second object ratio, and a third object 541-3 (the "Impossible Game" group) having a third object ratio. Here, the object ratios decrease in order from the first object ratio to the second object ratio and then to the third object ratio.

[0119] In FIG. 11(A), a first object 541-1, a second object 541-2, and a third object 541-3 are shown in the object display area 540, and these objects 541 are controlled to move within the object display area 540. When the objects 541 are moved, they may be controlled so that they bounce back when the objects 541 collide with each other or with the boundary of the object display area 540. The movement speed of the objects 541 may also be changed depending on the object ratio. For example, the movement speed of the objects 541 may be slowed as the object ratio increases. This makes the first object 541-1, which moves slowly, more visible than the third object 541-3, which moves quickly. This makes it easier for the broadcaster to view the first object 541-1 and understand the majority opinion of viewers. As such, movement of the objects 541 is one display mode of the objects 541.

[0120] 11(B), the colors of first object 541-1, second object 541-2, and third object 541-3 are different from one another. For example, as the object ratio decreases, the color of object 541 is lightened, or the text attached to object 541 is made closer to the color of the background of object 541. In this case, first object 541-1 is easier to see than third object 541-3, making it easier for the broadcaster to see first object 541-1 and understand the majority opinion of viewers. In this way, the color of object 541 is one of the display modes of object 541.

[0121] In FIG. 11(C), objects 541 are displayed in a superimposed manner. Specifically, a second object 541-2 is superimposed on a third object 541-3, and a first object 541-1 is further superimposed on the second object 541-2. That is, the first object 541-1 is displayed at the top. In this case, the broadcaster can easily grasp the majority opinion of the viewers by visually checking the object 541 displayed at the top. In this way, the order in which the objects 541 are superimposed can be cited as one of the display modes of the objects 541.

[0122] In a live video distribution service using the live video distribution system 1000 according to this embodiment, comments posted by viewers are classified into groups based on similar comments, and groups with more similar comments are displayed as objects 541 on the display unit 340 of the broadcaster information terminal 300. In other words, the displayed objects 541 reflect the majority opinion of viewers. Even if multiple groups are classified as the majority opinion (multiple objects 541 are generated), the size of the objects 541 corresponds to the number of comments in the group (size based on the object ratio), making it possible to grasp the mainstream majority opinion from the relative sizes of the multiple displayed objects. Furthermore, the objects 541 are accompanied by text representative of the comments in the group. Therefore, by visually checking the objects 541, broadcasters can easily grasp the majority opinion of viewers and broadcast live video in response to the viewers' comments. This gives viewers the impression of having a conversation with the broadcaster, allowing the broadcaster to communicate more closely with the viewers. Furthermore, by watching live video that is tailored to their own comments, viewers can feel closer to the live video. In this way, the live video streaming service becomes more exciting by increasing the sense of unity between the streamer and the viewers. Furthermore, depending on the live video streaming service, the streamer's tasks may be complicated. For example, when streaming a gameplay video, the streamer must provide audio commentary while playing the game, and must also check the comments posted by viewers and engage in dialogue or reactions in response to the comments. However, the live video streaming system 1000 makes it easy to grasp the majority opinion of viewers using object 541, thereby reducing the streamer's burden in at least responding to comments, among the tasks that are simultaneously ongoing.

[0123] Although the present invention has been described above with reference to the drawings, the present invention is not limited to the above-described embodiments and can be modified as appropriate without departing from the spirit of the present invention. For example, if a person skilled in the art appropriately adds, deletes, or modifies components based on the embodiments (including modifications), such modifications are also included in the scope of the present invention as long as they incorporate the gist of the present invention. Furthermore, modifications of the above-described embodiments can be appropriately combined as long as they are not mutually inconsistent.

[0124] Even if there are other effects and advantages different from those brought about by the aspects of the above-described embodiments, those that are clear from the description in this specification or that can be easily predicted by a person skilled in the art are naturally understood to be brought about by the present invention. [Explanation of symbols]

[0125] 200: Distribution server, 210: Control unit, 211: Live video data processing unit, 212: Comment information processing unit, 220: Memory unit, 230: Communication unit, 300: Information terminal for distributor, 310: Control unit, 311: Comment information acquisition unit, 312: Group classification unit, 313: Object generation unit, 314: Live video data generation unit, 320: Memory unit, 330: Communication unit, 340: Display unit, 350: Sound input / output unit, 360: Operation unit, 370: Imaging unit, 400: Information terminal for viewer, 410: Control unit, 411: Comment information generation unit, 420: Memory unit, 430: Communication unit, 440: Display unit, 450: Sound input / output unit, 460: Operation unit, 500: Live video data, 510: Game play video, 520: Streamer video, 530: Comment display area, 540: Object display area, 541: Object, 541-1: First object, 541-2: Second object, 541-3: Third object, 1000: Live video distribution system

Claims

1. Receive comments posted on the displayed video, classifying the received comments into at least a first group and a second group during each of a first period and a second period that are sequentially set; calculating a first ratio of the number of comments classified into the first group to the number of comments received during each of the first period and the second period; determining a first object having a first display manner for identifying the first group when the first ratio is equal to or greater than a first threshold during each of the first period and the second period; A program for causing a computer to execute the following: switching and displaying the first object determined based on the first ratio during the first period with the first object determined based on the first ratio during the second period.

2. The program according to claim 1 , wherein the first display manner is an area, a color, or a movement of the first object.

3. calculating a second ratio of the number of comments classified into the second group to the number of comments received during each of the first period and the second period; determining a second object having a second display manner for identifying the second group when the second ratio is equal to or greater than a second threshold during at least one of the first period and the second period; The program according to claim 1 , wherein the first display manner and the second display manner are determined in accordance with the first ratio and the second ratio, and the first object and the second object are displayed by the determined first display manner and the second display manner.

4. the first display manner and the second display manner are a first area of ​​the first object and a second area of ​​the second object, respectively; The program according to claim 3 , wherein the second area relative to the first area corresponds to the second ratio relative to the first ratio.

5. 4. The program according to claim 3, wherein when the second object is not determined based on the second ratio in the first period and the second object is determined based on the second ratio in the second period, the second object is displayed so as to be superimposed on the first object.

6. The program according to claim 1 , wherein the first display manner of the first object in the first period and the second period differs based on the first ratio.

7. The program according to claim 6 , wherein the first display manner based on the first ratio is a shade of a color of the first object or a size of text included in the first object.

8. The program according to claim 1 , wherein the second period starts before the first period ends.

9. The program according to claim 1 , wherein at least one of the first period and the second period is started based on an instruction request from a distributor who provides the video.

10. 10. The program according to claim 9, wherein the instruction request from the distributor is linked to a predetermined eye movement or facial movement by the distributor, or a predetermined keyword uttered by the distributor.

11. The program according to claim 1 , wherein each of the first period and the second period is terminated when the number of comments received in each of the first period and the second period is equal to or greater than a set value.

12. the first object includes text representing the first group; The program according to claim 1 , wherein the text is determined from among pre-registered words.

13. the first object includes text representing the first group; The program of claim 1 , wherein the text is determined by extracting words from comments classified into the first group.

14. The program of claim 1 , further comprising: classifying the received comments into at least the first group and the second group using a deep learning model trained based on previously received comments.

15. The program according to claim 1 , wherein the video is a gameplay video.

Images