Information processing device, terminal, information processing system and information processing method
By enabling broadcasters to respond to user requests during live streaming and generating extracted video data, the system improves broadcaster satisfaction and content utility, addressing the need for increased participation in live streaming services.
Patent Information
- Application Number
- JP2023080613
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2043-05-16
AI Technical Summary
There is a need to improve the satisfaction level of broadcasters in live streaming services to increase the number of broadcasters and viewers, as current technologies do not adequately incentivize broadcasters.
The system allows broadcasters to accept and respond to user requests during live streaming, generating extracted video data with consistent content by instructing the start and end of replies, which can be edited or shared independently.
This approach enhances broadcaster satisfaction by creating high-value extracted video data, reducing reliance on external sources, and increasing the utility of live streaming content.
Smart Images

Figure 2024164898000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device capable of executing processes related to live distribution, a terminal capable of communicating with the information processing device, an information processing system including the information processing device and the terminal, and an information processing method by the information processing device. [Background technology]
[0002] In recent years, live streaming using services related to live streaming has become widespread. There are various aspects of live streaming, but in one aspect, a broadcaster (two or more people may be broadcasters together) and unspecified viewers participate, the broadcaster's behavior is filmed on the broadcaster's terminal (hereinafter referred to as the "broadcaster terminal"), and a video based on the results of filming on the broadcaster terminal is distributed in real time to the viewer's terminal (hereinafter referred to as the "viewer terminal"). In the following, when simply referring to "live streaming," unless otherwise specified, it is assumed to refer to such a form of live streaming. Various technologies have been proposed for services related to live streaming. For example, Patent Document 1 describes a technology that provides a broadcaster with advice information including useful information about a viewer in real time during live streaming to support the broadcaster. Also, for example, Patent Document 2 describes a technology that accepts votes from viewers during live streaming and draws a lottery for the votes, allowing the broadcaster to have a one-on-one conversation with the viewer related to the winning vote. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent No. 2022-075401 [Patent Document 2] Patent No. 2022-084214 Summary of the Invention [Problem to be solved by the invention]
[0004] With regard to services related to live streaming, there is a constant demand for improving the satisfaction of streamers, because improving streamer satisfaction will provide an incentive for users who want to become streamers to use the service, leading to an increase in the number of streamers using the service and an associated increase in viewers.
[0005] The present invention has been made to solve such problems, and has an object to improve the satisfaction of distributors with regard to live distribution services. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems, the present invention has a function of accepting requests posted by users to a noted user who may become a live broadcast distributor, a function of accepting instructions to start a reply to a request and an instruction to end the reply during a live broadcast by the noted user, and when an instruction to start a reply is given, a function of displaying one of the requests on a terminal of a viewer of the live broadcast, and a function of generating extracted video data that records video related to the live broadcast from the start to the end of the reply. Effect of the Invention
[0007] According to the present invention configured as described above, during live streaming, the distributor instructs the start of a reply, replies to the request displayed on the viewer's terminal in response to the instruction, and instructs the end of the reply after the reply is completed. By performing such operations by the distributor, extracted video data is generated in which video related to the live streaming during the reply period is recorded for each reply made by the distributor. The video recorded in this extracted video data is a video with consistent and cohesive content in that it corresponds to one reply. In recent years, not only live streaming but also videos extracted from parts of live streaming (sometimes called "cut-out videos") have been widely provided and viewed, and in light of this, the extracted video data can be said to be data with high utility value. According to the present invention, in order to generate such extracted video data with high utility value, the distributor does not need to edit video data in which the video of the entire live streaming is recorded or rely on others to generate the extracted video data, and the satisfaction of the distributor can be improved. In other words, according to the present invention, the satisfaction of the distributor can be improved with respect to services related to live streaming. [Brief description of the drawings]
[0008] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] 2 is a block diagram showing an example of a functional configuration of a terminal and an information processing server according to an embodiment of the present invention; FIG. [Diagram 3] FIG. 13 is a diagram showing a distributor's screen. [Figure 4] FIG. 2 is a diagram showing a viewer's screen. [Diagram 5] 1 is a flowchart showing an operation example of the information processing system. [Figure 6] FIG. 2 is a diagram showing the contents of records in various databases. [Figure 7] FIG. 11 is a diagram illustrating the processing of the information processing system during live distribution. [Figure 8] 1 is a flowchart showing an operation example of the information processing system. [Figure 9] FIG. 13 illustrates a normal box screen. [Figure 10] FIG. 13 illustrates a request input screen. [Figure 11] FIG. 13 is a diagram showing a distributor's box screen. [Figure 12] 1 is a flowchart showing an example of the operation of the information processing system. [Figure 13] FIG. 13 is a diagram showing the contents of reply log data. [Figure 14] FIG. 2 is a diagram showing a viewer's screen. [Figure 15] FIG. 13 is a diagram showing a distributor's screen. [Figure 16] 13 is a flowchart showing an example of the operation of the information processing server. [Figure 17] 11 is a diagram used to explain a process for generating extracted video data. FIG. [Figure 18] 1 is a flowchart showing an example of the operation of the information processing system. [Figure 19] FIG. 1 is a diagram showing a special viewing screen and a normal viewing screen. [Figure 20] 13A and 13B are diagrams showing a normal box screen and an additional point input screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to this embodiment. As shown in FIG. 1, the information processing system 1 is configured to include a terminal 2 and an information processing server 3 (information processing device). FIG. 1 shows an example of some of the countless terminals 2. Both the terminal 2 and the information processing server 3 can be connected to a network N including the Internet, a telephone network, and other communication networks. The terminal 2 and the information processing server 3 can communicate with each other via the network N.
[0010] The information processing system 1 according to the present embodiment has a function for realizing live streaming. In the present embodiment, live streaming means that a broadcaster (two or more people may be broadcasters together) and unspecified viewers participate, the broadcaster's behavior is filmed on the broadcaster's terminal 2, and a video based on the results of filming on the broadcaster's terminal 2 is distributed to the viewers' terminal 2 in real time. The information processing server 3 provides a platform for live streaming by providing a service related to live streaming. Hereinafter, the service related to live streaming provided by the information processing server 3 is referred to as the "service". In the present embodiment, a video means a combination of an image (still image, moving image) recognized visually and a sound recognized auditorily. In the following description, a person who can use the service is referred to as a "user". The user has registered an account on the service, and can log in to the account of the service using his / her own login information (for example, a combination of a login ID and a password).
[0011] In this embodiment, a user can be both a broadcaster and a viewer of live broadcasting. However, in the following, for clarity and simplicity of explanation, one of the users who can be a broadcaster is referred to as a "noted broadcaster" (noted user), and the operation of the information processing system 1 related to the broadcaster may be described with a focus on the noted broadcaster. The terminal 2 of the noted broadcaster is referred to as a broadcaster terminal 2L. The broadcaster terminal 2L refers to the terminal 2 of the noted broadcaster for convenience, and does not mean the terminal 2 used exclusively by the broadcaster. In the following, one of the users who can be a viewer is referred to as a "noted viewer" (user), and the operation of the information processing system 1 related to the viewer may be described with a focus on the noted viewer. The terminal 2 of the noted viewer is referred to as a viewer terminal 2A. The viewer terminal 2A refers to the terminal 2 of the noted viewer for convenience, and does not mean the terminal 2 used exclusively by the viewer.
[0012] The terminal 2 may have any functions necessary for a terminal 2 used by a user, such as information processing function, sound collection function, image capture function, sound emission function, display function, input reception function, communication function, and other functions. However, the viewer's terminal 2 does not necessarily need to have sound collection function and image capture function. Furthermore, the terminal 2 does not necessarily have to be a single device, and may be configured by combining multiple devices. The type of the terminal 2 is, for example, a smartphone, a tablet terminal other than a smartphone, a notebook computer, or a desktop computer connected to a camera.
[0013] The information processing server 3 is a server device that treats the terminal 2 as one of its clients. As described above, the information processing server 3 provides the present service. In FIG. 1 and FIG. 2 described later, the information processing server 3 is shown as one block, but the information processing server 3 does not have to be a single server device. For example, the information processing server 3 may be composed of multiple server devices, or may be part of a predetermined system.
[0014] FIG. 2A is a block diagram showing an example of a functional configuration of the terminal 2 and the information processing server 3. As shown in FIG. 2A, the terminal 2 includes a terminal control device 5. The terminal control device 5 includes a terminal control unit 6 and a terminal communication unit 7 as functional configurations. The terminal control device 5 also includes a terminal storage unit 8 as storage means. In the terminal 2, a microphone 9 having a sound collection function, a camera 10 having a shooting function, a speaker 11 having a sound emission function, a display device 12 having a display function, and an input device 13 having a function of accepting input are connected to the terminal control unit 6. As shown in FIG. 2A, the information processing server 3 includes a server control unit 14 and a server communication unit 15 as functional configurations. The information processing server 3 also includes a server storage unit 16 as storage means.
[0015] The above-mentioned functional blocks 6, 7, 14, and 15 can be configured by any of hardware, DSP (Digital signal Processor), and software. For example, when configured by software, the above-mentioned functional blocks 6, 7, 14, and 15 are actually configured with a CPU, RAM, ROM, and the like of a computer, and are realized by the operation of a program stored in a recording medium such as the RAM, ROM, a hard disk, or a semiconductor memory.
[0016] The terminal communication unit 7 communicates with devices connected to the network N in accordance with a predetermined protocol. In the following description, detailed description of communication between the terminal 2 and devices connected to the network N is omitted, assuming that the communication is appropriately performed by the terminal communication unit 7. The server communication unit 15 communicates with devices connected to the network N in accordance with a predetermined protocol. In the following description, detailed description of communication between the information processing server 3 and devices connected to the network N is omitted, assuming that the communication is appropriately performed by the server communication unit 15.
[0017] As shown in FIG. 2A, a dedicated application 17 (hereinafter referred to as the "dedicated application 17") related to the service is installed in the terminal 2. The dedicated application 17 is a dedicated application that is implemented with functions to provide a screen related to the service (hereinafter referred to as the "service screen"), exchange various information with the information processing server 3, and execute other processes related to the service. Regarding processes related to the service, the terminal control unit 6 basically executes the processes by using the functions of the dedicated application 17 (including the OS of the terminal 2, a Web application that the dedicated application 17 can use, and other programs that can cooperate with the dedicated application 17). Even if not specifically described, when executing some process, the terminal control unit 6 of the terminal 2 communicates with the server control unit 14 of the information processing server 3 as necessary, requests a response of necessary information from the server control unit 14 to obtain the information, or requests the server control unit 14 to perform necessary processing and obtains the processing result. The information processing server 3 stores all the information to be provided to the terminal control unit 6. The server control unit 14 is also implemented with a function to execute processing in response to a request from the terminal control unit 6.
[0018] FIG. 2B(B1) is a block diagram showing an example of the functional configuration of the distributor terminal 2L. As shown in FIG. 2B(B1), the distributor terminal 2L includes a terminal control device 5L, a microphone 9L, a camera 10L, a speaker 11L, a display device 12L, and an input device 13L. The terminal control device 5L includes a terminal control unit 6L, a terminal communication unit 7L, and a terminal storage unit 8L. FIG. 2B(B2) is a block diagram showing an example of the functional configuration of the viewer terminal 2A. As shown in FIG. 2B(B2), the viewer terminal 2A includes a terminal control device 5A, a microphone 9A, a camera 10A, a speaker 11A, a display device 12A, and an input device 13A. The terminal control device 5A includes a terminal control unit 6A, a terminal communication unit 7A, and a terminal storage unit 8A.
[0019] Next, the operation of the information processing system 1 will be explained by dividing it into scenes. For the sake of convenience, it is assumed below that the types of the distributor terminal 2L and the viewer terminal 2A are smartphones. The smartphone has a tablet-type housing, and a display panel and a touch panel are provided in a wide area on the front of the housing, and a built-in microphone, a built-in camera, and a built-in speaker are provided inside the housing. The built-in microphone of the smartphone functions as microphones 9L and 9A, the built-in camera functions as cameras 10L and 10A, the built-in speaker functions as speakers 11L and 11A, the display panel functions as display devices 12L and 12A, and the touch panel functions as input devices 13L and 13A.
[0020] <Basic operations during live streaming> First, we will explain the basic operation of the information processing system 1 when a noted distributor broadcasts a live stream and a noted viewer watches it. Below, we will first explain the screens displayed on the distributor terminal 2L and the viewer terminal 2A during the live stream, and then explain the operation of the information processing system 1.
[0021] FIG. 3 is a diagram showing a distributor-side screen 20 displayed on the distributor terminal 2L during live distribution. As shown in FIG. 3, a captured video image 21, which is a video image based on the shooting result of the camera 10L, is displayed as a background on the distributor-side screen 20. A comment list area 22 is provided in an area slightly to the lower left of the center of the distributor-side screen 20. As will be described later, in the comment list area 22, comment individual images 23 corresponding to comments posted by viewers are displayed in real time. A reply start button 24 is provided below the comment list area 22, and a request box confirmation button 25 is provided to the right of the reply start button 24. Matters related to the reply start button 24 and the request box confirmation button 25 will be described later. In addition, a distributor profile image 26 is displayed in the upper left of the distributor-side screen 20. In the distributor profile image 26, an icon image of the distributor and the user name of the distributor are displayed. An end button 27 is provided to the right of the distributor profile image 26. The end button 27 is a button selected when the distributor ends the live distribution.
[0022] FIG. 4 is a diagram showing a viewer-side screen 29 displayed on the viewer terminal 2A during live distribution. As shown in FIG. 4, the viewer-side screen 29 displays the captured video 21 as a background, similar to the distributor-side screen 20, and the comment list area 22, the distributor profile image 26, and the end button 30 are provided on the captured video 21. The end button 30 is a button that is selected when viewing the live distribution ends. On the other hand, the viewer-side screen 29 does not have the reply start button 24 and the request box confirmation button 25, but instead has a comment input field 31, a comment send button 32, and a request related button 33. Matters related to the comment input field 31, the comment send button 32, and the request related button 33 will be described later. Note that the example of the distributor-side screen 20 and the viewer-side screen 29 are simplified. It is obvious that the contents of the distributor-side screen 20 and the viewer-side screen 29 are not limited to the example.
[0023] 5 is a flowchart showing an information processing method by the distributor terminal 2L, the information processing server 3, and the viewer terminal 2A during live distribution. The flowchart FA shows the information processing method by the distributor terminal 2L, the flowchart FB shows the information processing method by the information processing server 3, and the flowchart FC shows the information processing method by the viewer terminal 2A.
[0024] To start live streaming, the noted broadcaster launches the dedicated app 17 on the broadcaster terminal 2L. The noted broadcaster logs in to the account for the service in question using the service screen displayed on the display device 12L by the terminal control unit 6L. The login may be performed automatically and securely. The noted broadcaster performs the necessary tasks to prepare to start live streaming as a broadcaster. As shown in the flowchart FA, the noted broadcaster uses the service screen to instruct the start of live streaming (step SA1). The terminal control unit 6L detects this instruction and notifies the server control unit 14 of the start of live streaming (step SA2). Hereinafter, the live streaming started here will be referred to as the "noteworthy live streaming" to distinguish it from general live streaming.
[0025] As shown in the flowchart FB, when the server control unit 14 of the information processing server 3 receives a notification of the start of the featured live stream, it starts measuring the elapsed time from the timing when the featured live stream started (step SB1). Furthermore, the server control unit 14 adds a record corresponding to the featured live stream to the live stream management database 35 stored in the server storage unit 16, and starts managing the featured live stream (step SB2). The live stream management database 35 is a database that manages each live stream, and a record is registered for each live stream. FIG. 6(A) shows the contents of one record in the live stream management database 35. As shown in FIG. 6(A), one record in the live stream management database 35 includes fields including a live stream ID <field>, a distributor ID <field>, a viewer ID <field>, a reply log <field>, and an entire live stream video <field>.
[0026] The live streaming ID <field> stores a live streaming ID, which is identification information for uniquely identifying the corresponding live streaming. The broadcaster ID <field> stores the user ID of the broadcaster of the corresponding live streaming. The user ID is identification information for uniquely identifying a user (account) of the service. The viewer ID <field> stores the user ID of the viewer of the corresponding live streaming. If there are multiple viewers, the user IDs of each of the multiple viewers are stored. The reply log <field> records reply log data 34. Matters related to the reply log data 34 will be described later. The entire live streaming video <field> stores the entire live streaming video data after the corresponding live streaming is completed. The entire live streaming video data is video data (video file) in which the video of the entire corresponding live streaming (= from the start to the end of the live streaming) is recorded. Matters related to the entire live streaming video data will be described later.
[0027] In step SB2, the server control unit 14 generates a unique live streaming ID and stores it in the live streaming ID <field> of the record to be added. The server control unit 14 also stores the user ID of the featured broadcaster in the broadcaster ID <field>, stores a null value in the viewer ID <field>, stores the initial reply log data 34 in the reply log <field>, and stores a null value in the entire live streaming video <field>.
[0028] Meanwhile, the noted viewer starts the dedicated application 17 of the viewer terminal 2A. The noted viewer logs in to the account of the noted service using the service screen provided by the terminal control unit 6A. The login may be performed automatically and securely. As shown in the flowchart FC, the noted viewer uses the service screen to instruct the start of viewing the noted live stream (step SC1). The terminal control unit 6A of the viewer terminal 2A detects this instruction and notifies the server control unit 14 of the start of viewing the noted live stream (step SC2).
[0029] As shown in the flowchart FB, when the server control unit 14 receives a notification that viewing of the featured live stream has begun, it additionally stores the user ID of the featured viewer in the viewer ID <field> of the record corresponding to the featured live stream in the live streaming management database 35 (step SB3). The viewer ID <field> for the featured live stream is a field that manages the viewers of the current featured live stream. In other words, when a viewer starts viewing the featured live stream, the user ID of that viewer is stored in the viewer ID <field>. Although details are omitted, when a viewer stops viewing the featured live stream, the server control unit 14 detects this and deletes the user ID of that viewer from the viewer ID <field>.
[0030] A viewer may instruct to start watching a live stream by various methods. For example, in the service, one user can follow another user. When the other user starts live streaming, a button indicating that the other user has started live streaming is displayed on a predetermined service screen provided to the one user. The one user (viewer) can instruct to start watching the live stream by the other user (distributor) by selecting this button. Also, for example, in the service, the viewer can cause the display device 12 to display a service screen displaying a list of live streams currently being performed. The viewer can instruct to start watching the live stream by selecting one live stream from the list. Also, for example, in the service, while watching one live stream, the viewer can switch the live stream to be viewed from the one live stream to another live stream currently being performed by performing a scroll operation on the screen or other predetermined operation. The other live stream may be determined randomly or based on some rule. The user can instruct the start of viewing the other live streaming by performing an operation to switch from the live streaming being viewed to the other live streaming. Although several examples have been given above as methods by which a viewer can instruct the start of viewing a live streaming, the method is of course not limited to these several examples.
[0031] During live distribution, the distributor terminal 2L, the information processing server 3, and the viewer terminal 2A perform the following processes to realize live distribution. FIG. 7 is a diagram showing the operation of the information processing system 1 during live distribution in a manner suitable for explanation, focusing on the processes of each device. As shown in FIG. 7, during live distribution, the camera 10L of the distributor terminal 2L continues to take pictures and the microphone 9L continues to collect sound. The terminal control unit 6L of the distributor terminal 2L inputs a signal based on the shooting result from the camera 10L and generates image data D1, and also inputs a signal based on the sound collection result from the microphone 9L and generates audio data D2 (process PA1). The terminal control unit 6L performs image processing on the image data D1 to apply face processing, processing using filters, decoration processing, and other processing to the video, and generates live distribution video data D3 (process PA2). Although details are omitted, the type of processing to be applied to the video is specified in advance by the distributor of interest. Furthermore, the terminal control unit 6L performs necessary audio processing on the audio data D2 to generate live distribution audio data D4 (process PA2).
[0032] The terminal control unit 6L performs encoding, compression, container storage, and other necessary processing on the live streaming video data D3 and the live streaming audio data D4 according to a protocol to generate live streaming data D5, and transmits the live streaming data D5 to the information processing server 3 according to the protocol (process PA3). The protocol used is, for example, "HTTP Live Streaming" or "Real Time Messaging Protocol" (obviously, other protocols may also be used).
[0033] The terminal control unit 6L also performs processing to add an additional image to the live streaming video data D3 to generate distributor-side screen data D6 (process PA4). The additional image is an image to be added to the captured video 21 on the distributor-side screen 20. As shown in FIG. 3, the additional image includes a comment individual image 23, a reply start button 24, a request box confirmation button 25, a distributor profile image 26, and an end button 27. The terminal control unit 6L drives the display device 12L based on the generated distributor-side screen data D6 to display the distributor-side screen 20 on the display device 12L (process PA5). The terminal control unit 6L also drives the speaker 11L based on the live streaming audio data D4 to emit audio (process PA5). The synchronization between the captured video 21 and the audio during display on the distributor-side screen 20 is appropriately performed by the terminal control unit 6L.
[0034] During live distribution, the server control unit 14 of the information processing server 3 continuously receives live distribution data D5 from the distributor terminal 2L (process PB1). The server control unit 14 refers to the live distribution management database 35 and the user management database 36 as necessary, and transmits the live distribution data D5 to the viewer terminal 2A in accordance with the protocol (process PB2).
[0035] The user management database 36 is a database for managing users (accounts), and a record is registered for each user. FIG. 6(B) shows the contents of one record in the user management database 36. As shown in FIG. 6(B), one record in the user management database 36 includes fields such as a user ID <field>, a user name <field>, an icon <field>, a point balance <field>, and a total acquired points <field>. The user ID <field> stores the user ID of the corresponding user. The user name <field> stores user name information indicating the user name of the corresponding user. The icon <field> stores icon image data which is image data of the icon of the corresponding user. The point balance <field> stores a value indicating the balance of the points (points) of the corresponding user (hereinafter referred to as the "point balance value"). The points are points that can be used in the service. The unit of the points is "pt". The user can acquire the points by purchase, by being awarded in a campaign, or by other means. The Total Earned Points <field> stores the total earned points value, which is the sum of the earned point values. Matters related to the earned point values will be described later. Note that the contents of the records in the user management database 36 are simplified. The records include information about the user's contact information (email address, etc.), login information, information about the payment method, and various other information about the user.
[0036] Furthermore, the server control unit 14 performs necessary processing such as extracting, decompressing, and decoding the received live distribution data D5 according to the protocol to generate live distribution video data D3 and live distribution audio data D4 (process PB3). The server control unit 14 cumulatively buffers the generated live distribution video data D3 and live distribution audio data D4 in the server storage unit 16 (process PB4).
[0037] During live distribution, the terminal control unit 6A of the viewer terminal 2A continuously receives the live distribution data D5 (process PC1). The terminal control unit 6A performs necessary processes such as extracting the live distribution data D5 from the container, decompressing, decoding, and other processes according to the protocol to generate live distribution video data D3 and live distribution audio data D4 (process PC2). The terminal control unit 6A performs a process of adding an additional image to the live distribution video data D3 to generate viewer side screen data D7 (process PC3). The additional image is an image added to the captured video data 21 on the viewer side screen 29. As shown in FIG. 4, the additional image includes a comment individual image 23, a comment input field 31, a comment send button 32, a request related button 33, a distributor profile image 26, and an end button 30. The terminal control unit 6A then drives the display device 12A based on the generated viewer side screen data D7 to display the viewer side screen 29 on the display device 12A (process PC4). The terminal control unit 6A also drives the speaker 11A to emit sound based on the live distribution sound data D4 (process PC4). The captured video image 21 and sound being displayed on the viewer side screen 29A are appropriately synchronized by the terminal control unit 6A.
[0038] Although an example of the operation of the information processing system 1 that realizes live streaming has been described above, the operation of the information processing system 1 is not limited to the exemplified operation. In other words, any operation may be performed by the information processing system 1 as long as live streaming is realized. Furthermore, all existing technologies related to realizing live streaming can be used for the operation of the information processing system 1. In the following description, it is assumed that during live streaming, live streaming is properly realized by the method described using FIG. 7, and a detailed description of the process for realizing live streaming will be omitted.
[0039] 5, as shown in the flowchart FA, when the noted distributor wants to end the live distribution, the distributor selects the end button 27 on the distributor side screen 20 (step SA3). When the terminal control unit 6L detects that the end button 27 has been selected, it notifies the information processing server 3 of the end of the live distribution (step SA4).
[0040] As shown in the flow chart FB, when the server control unit 14 receives a notification that the live streaming has ended, it stops measuring the elapsed time (step SB4). Next, the server control unit 14 generates entire live streaming video data in a predetermined file format based on the live streaming video data D3 and the live streaming audio data D4 buffered in the server storage unit 16 (step SB5). The entire live streaming video data is video data (video file) in which video of the entire live streaming (from start to end) of the featured live streaming is recorded. Next, the server control unit 14 stores the generated entire live streaming video data in the entire live streaming video <field> of the record corresponding to the featured live streaming in the live streaming management database 35 (step SB6).
[0041] The above is the basic operation of the information processing system 1 during live streaming. However, the operation of the information processing system 1 described is merely a simplified example, and different processes may be performed in each device of the information processing system 1 as long as the purpose is achieved. For example, the entire live streaming video data may be generated by the following method. That is, the data related to the live streaming may be received by a dedicated device different from the information processing server 3, and the entire live streaming video data may be generated by the dedicated device.
[0042] <Processing when a comment is posted by a featured viewer> Next, a brief description will be given of the operation of the information processing system 1 when a comment is posted by an interested viewer during live distribution. Note that in this embodiment, a comment is a different concept from a request, which will be described later, and the two are clearly distinguished from each other.
[0043] When posting a comment, the noted viewer inputs the comment in the comment input field 31 on the viewer side screen 29 (FIG. 4) and selects the comment send button 32. Comments and requests, which will be described later, are composed of characters (a concept that includes all usable objects, including emoticons, for example). When the terminal control unit 6A of the viewer terminal 2A detects that the comment send button 32 has been selected, it transmits information related to the input comment and information related to the noted viewer to the information processing server 3.
[0044] The server control unit 14 of the information processing server 3 receives the information and adds a record including various information related to the comment and the noted viewer to the comment management database 37 corresponding to the noted live stream. The comment management database 37 is a database that manages comments posted during live streams, and exists for each live stream. The server control unit 14 then references the user management database 36 and other data, and generates comment display information including information required to display the comment individual image 23 (see Figs. 3 and 4). The server control unit 14 then transmits the comment display information to the distributor terminal 2L and the viewer terminal 2A. The comment display information is transmitted not only to the distributor terminal 2L and the viewer terminal 2A, but also to the terminals 2 of all viewers who are watching the noted live stream.
[0045] When the terminal control unit 6L of the distributor terminal 2L receives the comment display information, it additionally displays the comment individual images 23 in the comment list area 22 of the distributor side screen 20 in a manner following the rules. As shown in FIG. 3, the comment individual images 23 include the user name and icon image of the attention viewer (the viewer who posted the comment in this example) and the content of the comment. The rule is that the existing comment individual images 23 are shifted upward in the comment list area 22, the existing comment individual images 23 that do not fit in the comment list area 22 are deleted, and new comment individual images 23 are displayed at the end of the comment list area 22. However, for the comment individual images 23 that do not fit in the comment list area 22, a certain amount of them may be made viewable by scrolling the comment list area 22. Similarly, when the terminal control unit 6A of the viewer terminal 2A receives the comment display information, it additionally displays the comment individual images 23 in the comment list area 22 of the viewer side screen 29 in a manner following the rules.
[0046] In this manner, in this embodiment, comments posted by viewers are displayed in real time on the terminals 2 of the distributor and the viewers.
[0047] <Processing when a request is posted by a featured viewer> Next, the operation of the information processing system 1 when a request is posted by a noted viewer during live distribution will be described. Fig. 8 is a flowchart showing the information processing method by the information processing server 3 and the viewer terminal 2A when a request is posted by the noted viewer. The flowchart FD shows the information processing method by the information processing server 3, and the flowchart FE shows the information processing method by the viewer terminal 2A.
[0048] A request is a request made in the hope of receiving a reply from the distributor. For example, a request may be a question to the distributor, or may be a request for the distributor to take some kind of action. A reply refers to any response made by a distributor to a request from a user. For example, a reply is a series of actions to answer a question related to a request, or may be an action taken in response to a request related to a request. As will be made clear later, requests are not displayed on the distributor's screen 20 in real time in response to a post, which is a point that makes them different from comments. Also, as will be made clear later, in certain cases, a distributor will exclusively reply to one request.
[0049] As shown in the flowchart FE, when posting a request, the attention viewer selects the request-related button 33 on the viewer-side screen 29 (step SE1). When the request-related button 33 is selected, the terminal control unit 6A of the viewer terminal 2A cooperates with the server control unit 14 of the information processing server 3 to pop up a normal box screen 38 (FIG. 9) on the display device 12A (step SE2). The process of step SE2 will be described in detail below.
[0050] In step SE2, the terminal control unit 6A notifies the server control unit 14 that the request-related button 33 has been selected. Upon receiving the notification, the server control unit 14 refers to the request management database 40 corresponding to the noted distributor. In this service, each user can post requests to users other than himself / herself. A request management database 40 exists for each user, and manages requests posted to each user. In other words, the request management database 40 corresponding to a certain user manages requests posted to that user. Furthermore, a record is registered in the request management database 40 corresponding to a certain user for each request posted to that user.
[0051] Fig. 6(C) shows the contents of one record in the request management database 40. As shown in Fig. 6(C), one record in the request management database 40 includes the following fields: request ID <field>, poster ID <field>, request posting date and time <field>, request content <field>, given points <field>, replied flag <field>, number of positive evaluations <field>, positive evaluation giver <field>, and reply priority <field>.
[0052] The request ID <field> stores a request ID, which is identification information for uniquely identifying the corresponding request. The poster ID <field> stores the user ID of the user who posted the corresponding request. Hereinafter, the user who posted the request may be referred to as the "request poster". The request posting date and time <field> stores request posting date and time information indicating the date and time when the request was posted. The request content <field> stores request content information indicating the content of the request. The granted point information <field> stores an granted point value indicating the value of the points granted to the corresponding request. Matters related to the granted point value will be described later. The replied flag <field> stores a replied flag, which is a flag indicating whether or not the user who received the request has replied to the corresponding request. When the replied flag is on, it indicates that a reply has already been made, and when it is off, it indicates that a reply has not yet been made. The number of positive evaluations <field> stores a value indicating the number (cumulative number) of positive evaluations given to the corresponding request (hereinafter referred to as the "number of positive evaluations"). Matters related to positive evaluations will be described later. The positive evaluation giver <field> stores the user ID of each user who gave a positive evaluation to the corresponding request (hereinafter referred to as the "positive evaluation giver"). The reply priority <field> stores reply priority information that indicates the reply priority of the corresponding request.
[0053] Matters related to reply priority will be described below. The server control unit 14 appropriately updates the reply priority information of each record in the request management database 40 in the following manner. In this embodiment, the reply priority is expressed in order from the top, such as 1, 2, 3, etc. That is, the server control unit 14 identifies records registered in the request management database 40 whose reply completed flag is off (=records corresponding to requests to which the user who received the request has not yet replied). Hereinafter, requests to which no reply has been made (=requests corresponding to records whose reply completed flag is off) are referred to as "unreplied requests". Next, the server control unit 14 sets reply priority for all unreplied requests such that the higher the assigned point value, the higher the reply priority, and if the assigned point value is the same, the more positive evaluations the higher the reply priority, and if the assigned point value and the number of positive evaluations are the same, the older the request posting date and time, the higher the reply priority.
[0054] For example, suppose that the assigned point value of the unresponded request R1 is "3,000pt" and the assigned point value of the unresponded request R2 is "5,000pt". In this case, the server control unit 14 sets the reply priority of the unresponded request R2 higher than that of the unresponded request R1. Also, suppose that the assigned points of the unresponded request R3 and the unresponded request R4 are the same, but the number of positive evaluations of the unresponded request R3 is greater than the number of positive evaluations of the unresponded request R4. In this case, the server control unit 14 sets the reply priority of the unresponded request R3 higher than that of the unresponded request R4. The server control unit 14 updates the reply priority information stored in the reply priority <field> for each record corresponding to each unresponded request according to the prioritization performed for the unresponded requests. The server control unit 14 sets the reply priority information of the record whose reply completed flag is on (= the record corresponding to the request for which the user who received the request has already replied) to a null value.
[0055] The server control unit 14 updates the reply priority information of each record in the request management database 40 whenever a factor occurs that may cause a change in the reply priority, such as when a record is added, when a record is deleted, or when the reply completed flag for any record changes from OFF to ON.
[0056] As described above, the server control unit 14 (control unit) has a function to set a reply priority order for unanswered requests so that the reply priority order (priority order) is higher as the assigned point value (assigned point value) is larger. The server control unit 14 also has a function to set a reply priority order for unanswered requests so that the reply priority order is higher as the assigned point value is larger and, if the assigned point value is the same, the number of assigned positive evaluations is larger.
[0057] The server control unit 14 refers to the request management database 40 corresponding to the noted distributor and generates display request information for each unresponded request. The display request information corresponding to one unresponded request includes the user name and icon image data of the poster of the unresponded request, request content information, the assigned point value, the number of positive evaluations, and other information required for displaying a request card 44 (described later). The server control unit 14 acquires this information for each unresponded request based on the request management database 40 and the user management database, and generates display request information. Next, the server control unit 14 transmits the display request information for each unresponded request to the terminal control unit 6A. The terminal control unit 6A uses the received display request information to pop up a normal box screen 38.
[0058] FIG. 9 is a diagram showing an example of the normal box screen 38. As shown in FIG. 9, an unresponded request count image 41 showing the number of unresponded requests at the present time is displayed at the top of the normal box screen 38. The terminal control unit 6A counts the number of unresponded requests based on the information received from the server control unit 14, and displays the unresponded request count image 41. A request card list area 42A is provided below the unresponded request count image 41 on the normal box screen 38, and a request creation button 43 is provided below the request card list area 42A. A request card 44 (information on a request) is displayed for each unresponded request in the request card list area 42A. That is, one request card 44 on the normal box screen 38 corresponds to one unresponded request. The request card 44 includes an image showing the user name of the corresponding request poster and an image showing the icon of the request poster. The request card 44 also includes a request content image 45 showing the content of the corresponding request, and an assigned point image 46 showing the assigned point value assigned to the corresponding request. The request card 44 also includes a positive evaluation object 47 .
[0059] The positive evaluation object 47 includes information indicating the number of positive evaluations. As described above, the number of positive evaluations is the number of positive evaluations given to the corresponding unanswered request. The positive evaluation object 47 also includes a mark 48 indicating whether or not the user (the noted viewer in this example) has given a positive evaluation. The mark is heart-shaped, and is filled in with a predetermined color when the user has given a positive evaluation, and is white when the user has not given a positive evaluation. The terminal control unit 6A inquires of the server control unit 14 about whether or not the user has given a positive evaluation for each request card 44, and controls the filling / whitening of the mark 48 based on the response to the inquiry.
[0060] The positive evaluation object 47 also functions as a button for giving a positive evaluation to a request and a button for canceling a positive evaluation when a positive evaluation has been given. When a viewer of interest has not given a positive evaluation to a request card 44, the viewer of interest can give a positive evaluation to the request corresponding to that request card 44 by selecting the positive evaluation object 47. When the viewer of interest selects the positive evaluation object 47, the server control unit 14 increments the number of positive evaluations in the corresponding record in the request management database 40 corresponding to the viewer of interest, in cooperation with the terminal control unit 6A. Furthermore, the server control unit 14 adds the user ID of the viewer of interest to the positive evaluation giver <field> of the corresponding record. Furthermore, the viewer of interest can cancel the positive evaluation given to the request corresponding to the request card 44 by selecting the positive evaluation object 47 when giving a positive evaluation to the request card 44. When the viewer of interest selects the positive evaluation object 47, the server control unit 14 increments the number of positive evaluations in the corresponding record in cooperation with the terminal control unit 6A. Furthermore, the server control unit 14 deletes the user ID of the viewer of interest from the positive evaluation giver <field> of the corresponding record.
[0061] In this way, the server control unit 14 has a function to detect when a user gives a positive evaluation to a request, and to execute processing to reflect the positive evaluation in the request management database 40 and other processes. This function corresponds to a function to accept the giving of a positive evaluation to a request.
[0062] The terminal control unit 6A displays the request card 44 based on the corresponding display request information. The terminal control unit 6A also changes the background color of the block in which the request content and the assigned point value of the request card 44 are displayed as follows. That is, the terminal control unit 6A sets the background color to "white" when the assigned point value is less than 2000pt, sets the background color to "yellow" when the assigned point value is 2000pt or more and less than 5000pt, and sets the background color to "red" when the assigned point value is 5000pt or more. The request card list area 42A is configured to be scrollable. The viewer can display all the request cards 44 by scrolling the request card list area 42A. The terminal control unit 6A arranges the request cards 44 in the request card list area 42A in order from top to bottom according to the reply priority order. The terminal control unit 6A recognizes the reply priority order of each of the request cards 44 by inquiring of the server control unit 14. In this way, the higher the reply priority order of the request card 44, the higher it is displayed in the request card list area 42A. By referring to the request card list area 42A, the attentive viewer can recognize the contents of the request, information about the request poster, the given point value, and information about the positive evaluation for each unprocessed request.
[0063] As will be apparent later, the higher the reply priority of a request, i.e., the higher the request card 44 displayed in the request card list area 42A, the earlier it will be the subject of a reply by the user who received the request. The higher the assigned point value, the higher the reply priority, and if the assigned point value is the same, the more positive evaluations, the higher the reply priority. Based on this, the attentive viewer has an incentive to give positive evaluations to request cards 44 for which he or she would like the distributor to reply at an earlier stage.
[0064] As described above, the terminal control unit 6A has a function of providing a normal box screen 38 (screen) on which the request cards 44 (information related to requests) are arranged according to reply priority (priority order). The terminal control unit 6A also has a function of providing a normal box screen 38 (screen) on which the request cards 44 are displayed in a list and on which positive evaluation objects 47 (objects for giving positive evaluations) are displayed in association with the request cards 44.
[0065] As shown in the flow chart FE, an attentive viewer who wishes to post a request refers to the normal box screen 38 and selects the request creation button 43 (step SE3). When detecting that the request creation button 43 has been selected, the terminal control unit 6A cooperates with the server control unit 14 to pop up a request input screen 50 (FIG. 10) on the display device 12A instead of the normal box screen 38 (step SE4). FIG. 10 is a diagram showing an example of the request input screen 50. As shown in FIG. 10, a request input field 51 is provided at the top of the request input screen 50. The request input field 51 is a field in which a request composed of characters is input. When the request input field 51 is selected, the terminal control unit 6A displays a software keyboard. The attentive viewer can input a request into the request input field 51 using the software keyboard. Note that the software keyboard is displayed by the terminal control unit 6 when inputting into any input field, not limited to the request input field 51.
[0066] A point balance image 52 is displayed below the request input field 51 on the request input screen 50. The terminal control unit 6A inquires of the server control unit 14 about the point balance value of the noted viewer, and displays the point balance image 52 based on the response to the inquiry. A point acquisition button 53 is provided to the right of the point balance image 52. The point acquisition button 53 is a button selected when increasing the point balance value by acquiring a specified amount of present points. When the point acquisition button 53 is selected, the terminal control unit 6A cooperates with the server control unit 14 to display a screen for acquiring the present points. The present points can be acquired by purchase or other means. The noted viewer uses the screen to acquire the present points by the specified amount. The server control unit 14 appropriately increases the point balance value of the corresponding record in the user management database 36 in response to the acquisition of the present points by the user. When the present points are acquired by purchase, payment, etc. is appropriately executed.
[0067] In the request input screen 50, an assigned point input field 54 is provided below the point balance image 52, an assigned point slider 55 is provided below the assigned point input field 54, and a maximum point assignment button 56 is provided below the assigned point slider 55. The assigned point input field 54, the assigned point slider 55, and the maximum point assignment button 56 are each an object for inputting an assigned point value to be assigned to a corresponding request. Hereinafter, these may be collectively referred to as "assigned point input objects."
[0068] The assigned point input field 54 is an input field in which the assigned point value can be entered as text. The assigned point slider 55 is a slider for entering an assigned point value between a lower limit value ("0pt" in this embodiment) and an upper limit value ("10,000pt" in this embodiment) by sliding the knob. The assigned point input field 54 and the assigned point slider 55 are linked. That is, when a certain value of an assigned point value is entered in the assigned point input field 54, the knob of the assigned point slider 55 moves to a position corresponding to that value. Also, when the assigned point value is set to a certain value by moving the knob of the assigned point slider 55, text indicating that value is displayed in the assigned point input field 54.
[0069] The maximum point allocation button 56 is a button for instructing input of an allocation point value that is "1 pt" higher than the "highest allocation point value among the current allocation point values of each unhandled request." As described above, for an unhandled request, the higher the allocation point value, the higher the reply priority. In light of this, if the attention viewer wishes to give the highest reply priority to the request he or she is about to post, he or she can simply input an allocation point value that gives the highest reply priority by selecting the maximum point allocation button 56. When the maximum point allocation button 56 is selected, the terminal control unit 6A identifies the highest allocation point value among the allocation point values of each unhandled request, and derives a value by adding "1 pt" to the highest allocation point value. The terminal control unit 6A displays the derived value in the allocation point input field 54, and moves the knob of the allocation point slider 55 to a position corresponding to the derived value.
[0070] The terminal control unit 6A increases the upper limit of the number of characters that can be input into the request input field 51 according to the assigned point value input into the assigned point input object, as the input assigned point value increases. In this embodiment, the upper limit is 100 characters when the value is less than 2000pt, the upper limit is 200 characters when the value is 2000pt or more and less than 5000pt, and the upper limit is 300 characters when the value is 5000pt or more. However, these are merely examples. The relationship between the assigned point value and the upper limit of the number of characters is clearly indicated on the assigned point slider 55. Moreover, the relationship is clearly indicated in association with the request input field 51. Due to this configuration, it is possible to give an incentive to a user who is about to post a request to grant a larger assigned point value.
[0071] A provisional priority order image 57 is displayed below the maximum point assignment button 56 on the request input screen 50. The provisional priority order image 57 is information indicating the reply priority order of the request when the assigned point value inputted in the assigned point input object is assigned to the request. The terminal control unit 6A cooperates with the server control unit 14 and executes the following process every time there is a change in the assigned point value inputted in the assigned point input object. That is, the terminal control unit 6A derives the reply priority order when the input assigned point value is assigned, taking into account the assigned point value of each unresponded request. The terminal control unit 6A updates the contents of the provisional priority order image 57 so as to indicate the derived reply priority order. The attentive viewer can adjust the assigned point value by referring to the reply priority order indicated in the provisional priority order image 57.
[0072] As described above, the terminal control unit 6A has a function of providing, when posting a request, a screen (request input screen 50) displaying an assignment point input object for inputting an assignment point value to be assigned to the request, and information (provisional priority order image 57) indicating the priority of the request when the assignment point value inputted in the object is assigned to the request. The terminal control unit 6A also has a function of providing, when posting a request, a request input screen 50 (screen) displaying a maximum points assignment button 56 (button) for inputting an assignment point value that will give the highest reply priority order (priority order) to the request. The terminal control unit 6A also has a function of providing, when posting a request, a request input screen 50 (screen) displaying a request input field 51 (input field) for inputting a request, and an assignment point input object (object) for inputting points to be assigned to the request, and a function of increasing the upper limit of the number of characters that can be entered in the request input field 51 as the assignment point value inputted in the assignment point input object increases.
[0073] A request send button 58 is provided below the provisional priority order image 57. The request send button 58 is a button that is selected when posting a request. As shown in flow chart FE, the attention viewer inputs the content of the request in the request input field 51 and the assigned point value in the assigned point input object, and selects the request send button 58 (step SE5). This completes the posting of the request. When the terminal control unit 6A detects that the request send button 58 has been selected, it transmits request notification information including at least request content information indicating the content of the request input in the request input field 51 and the assigned point value input in the assigned point input object to the server control unit 14 (step SE6).
[0074] When the request transmission button 58 is selected, the terminal control unit 6A cooperates with the server control unit 14 to determine whether the assigned point value input to the assigned point input object is within the point balance value range. If it is within the point balance value range, the terminal control unit 6A transmits request notification information. On the other hand, if it is not within the point balance value range, the terminal control unit 6A executes the following process without transmitting the request notification information. That is, the terminal control unit 6A displays a warning on the display device 12A to the effect that the request cannot be posted because the assigned point value is not within the point balance value range. Note that, together with the warning, the terminal control unit 6A may be configured to display a button for displaying a screen for acquiring the assigned point value, or to display the screen itself on the display device 12A.
[0075] When the request notification information is received, the server control unit 14 adds one record to the request management database 40 corresponding to the noted broadcaster (step SD1). In step SD1, the server control unit 14 stores a unique request ID in the request ID <field> of the record to be added. The server control unit 14 also stores the user ID of the noted viewer in the poster ID <field>. The server control unit 14 also stores request posting date and time information indicating the date and time when the request notification information was received in the request posting date and time <field>. However, the server control unit 14 may be configured to store request posting date and time information indicating another date and time that can be considered as the date and time when the request was posted (for example, the date and time when the request send button 58 was selected) instead of the date and time when the request notification information was received. The server control unit 14 also stores the received request content information in the request content <field>. The server control unit 14 also stores the received assigned point value in the assigned points <field>. The server control unit 14 also stores the reply completed flag with an off value in the reply completed flag <field>. The server control unit 14 also stores the initial value (zero) in the number of positive evaluations <field> and stores a null value in the positive evaluation giver <field>. The server control unit 14 also derives a reply priority and stores reply priority information indicating the derived reply priority in the request reply priority <field>. The server control unit 14 updates the reply priority information for all unanswered requests.
[0076] Furthermore, the server control unit 14 subtracts the received granted point value from the point balance value of the record corresponding to the noted viewer in the user management database 36 (step SD2).
[0077] As described above, the server control unit 14 has a function to detect when a request is posted by a user, register a corresponding record in the request management database 40, and execute other processes to reflect the posting of the request. This function corresponds to a function to accept a request posted from a user to a notable user who may become a live broadcaster. The server control unit 14 also has a function to detect when an assigned point value is assigned to a request, and execute processes to reflect the assigned point value in the request management database 40 and to reflect the assignment of other assigned point values. This function corresponds to a function to accept the assignment of point values to a request from the request poster when the request is posted.
[0078] Note that the normal box screen 38 and the request input screen 50 shown as examples are simplified. It goes without saying that the contents of the normal box screen 38 and the request input screen 50 are not limited to those shown as examples. For example, the normal box screen 38 and the request input screen 50 may be an integrated screen. Also, a configuration may be provided in which objects are provided for displaying Q&A, help information, and other information useful to the user. When the object is selected, the server control unit 14 cooperates with the terminal control unit 6A to display various information.
[0079] <Additional information regarding submitting requests> 8 has described an example of the operation of the information processing system 1 in the case where a noted viewer posts a request during live streaming. However, in the present service, a user can post requests to other users not only during the period when the other user is performing live streaming, but also during the period when live streaming is not being performed. In other words, the server control unit 14 accepts requests posted by users to the noted user regardless of whether live streaming is being performed or not.
[0080] Hereinafter, the operation of the information processing system 1 will be described in the case where the user U1 posts a request to the user U2 when the user U2 is not live streaming. The user U1 instructs the display of the normal box screen 38 (FIG. 9) for the user U2 on a predetermined present service screen. In response to the instruction, the terminal control unit 6 of the terminal 2 of the user U1 cooperates with the server control unit 14 to display the normal box screen 38 for the user U2 on the display device 12. The server control unit 14 appropriately provides information required for displaying the normal box screen 38 based on the request management database 40 corresponding to the user U2. Thereafter, the user U1 performs the same operation as the operation exemplified in FIG. 8 to post a request to the user U2. The server control unit 14 and the terminal control unit 6 execute the same process as the process described in FIG. 8 in response to the operation of the user U1. The instruction by the user U1 to display the normal box screen 38 for the user U2 can be performed in various ways.
[0081] <Processing when Request Box Confirmation Button 25 is selected> Next, the operation of the information processing system 1 when the request box confirmation button 25 is selected by a noted broadcaster during live broadcasting will be described. When it is detected that the noted broadcaster has selected the request box confirmation button 25, the terminal control unit 6L requests the server control unit 14 to respond with display request information for each of the unresponded requests. The server control unit 14 refers to the request management database 40, and generates and responds with display request information for each of the unresponded requests. The content of the display request information is as described above. Upon receiving the display request information for each of the unresponded requests, the terminal control unit 6L pops up a broadcaster box screen 60 on the broadcaster screen 20 based on the display request information.
[0082] Fig. 11 is a diagram showing a distributor's box screen 60. As shown in Fig. 11, an unanswered request count image 41 is displayed at the top of the distributor's box screen 60. To the right of the unanswered request count image 41, an obtainable points image 61 is displayed, which indicates the amount of points that would be obtained if all currently unanswered requests were replied to. As will be made clear later, when a featured distributor replies to a request, the featured distributor can obtain an obtained point value equivalent to the granted point value granted to that request.
[0083] A request card list area 42L is provided below the unresponded request count image 41 on the distributor box screen 60. In the request card list area 42A, a distributor request card 44L (information about the request) is displayed for each unresponded request. The distributor request card 44L displayed on the distributor box screen 60 differs from the request card 44 displayed on the normal box screen 38 in that it includes a card disposal button 62. When the card disposal button 62 is selected, the terminal control unit 6A pops up a screen that allows the user to select one of the following three actions:
[0084] The first action is to delete the distributor's request card 44L. When the deletion of the distributor's request card 44L is selected, the terminal control unit 6L deletes the corresponding distributor's request card 44L from the request card list area 42L. Furthermore, the server control unit 14 cooperates with the terminal control unit 6L to delete the "record corresponding to the distributor's request card 44L instructed to be deleted" in the request management database 40 corresponding to the noted distributor, and registers the record in another database that manages deleted requests. The second action is to block the user who posted the request. When the block is selected, the server control unit 14 cooperates with the terminal control unit 6L to create a state in which requests from the user who posted the request to the noted distributor are blocked. The third action is to notify the administrator. When the notification is selected, the server control unit 14 cooperates with the terminal control unit 6L to notify the administrator of the contents of the request and information about the request poster.
[0085] The terminal control unit 6L cooperates with the server control unit 14 to arrange the distributor request cards 44L in the request card list area 42L according to the reply priority order. By referring to the request card list area 42L, the noted distributor can grasp the contents of the unreplied requests and information about the request posters while taking into account the reply priority order. As described below, when the noted distributor selects the reply start button 24, the distributor request card 44L corresponding to the request with the highest reply priority order, that is, the distributor request card 44L displayed at the top when the distributor box screen 60 is displayed, becomes the target of the reply. Based on this, the noted distributor can check the contents of the request card 44 that will be the target of the reply when he or she selects the reply start button 24 by referring to the distributor box screen 60.
[0086] <Processing when a reply is made by a featured distributor> Next, we will explain the operation of the information processing system 1 when a reply is made by a noted broadcaster during live broadcasting. Fig. 12 is a flowchart showing the information processing method of the broadcaster terminal 2L, the information processing server 3, and the viewer terminal 2A when a reply is made by a noted broadcaster. Flowchart FF shows the information processing method by the broadcaster terminal 2L, flow chart FG shows the information processing method by the information processing server 3, and flow chart FH shows the information processing method by the viewer terminal 2A.
[0087] Here, in one live broadcast, the broadcaster can reply multiple times (however, it is not necessary to reply even once). For example, assume that the broadcaster makes two replies. In this case, the broadcaster selects the reply start button 24 when making the first reply, and selects the reply end button 63 when the first reply is completed. After that, the broadcaster selects the reply start button 24 when making the second reply, and selects the reply end button 63 when the second reply is completed. In the explanation using FIG. 12, the operation of the information processing system 1 will be explained by focusing on when one reply is made by the broadcaster of interest.
[0088] As shown in the flowchart FF, the noted distributor instructs the start of a reply by selecting the reply start button 24 (step SF1). When detecting that the reply start button 24 has been selected, the terminal control unit 6L notifies the server control unit 14 of the start of a reply (step SF2).
[0089] As shown in the flowchart FG, when the server control unit 14 receives a notification of the start of a reply, the server control unit 14 refers to the request management database 40 corresponding to the noted distributor, and identifies the unanswered request with the highest reply priority among the unanswered requests (step SG1). Hereinafter, the unanswered request with the highest reply priority is referred to as the "highest-ranked request."
[0090] Next, information regarding the start of a reply is recorded in the reply log data 34 of the record corresponding to the live streaming of interest in the live streaming management database 35 (step SG2). FIG. 13 is a diagram showing the contents of the reply log data 34 in a form suitable for explanation. In the reply log data 34, reply performance information RJ is recorded for each reply made in the live streaming. One piece of reply performance information RJ includes information indicating which reply it is, the request ID of the highest-ranked request, the start timing, and the end timing. In this embodiment, the start timing and the end timing are expressed as elapsed time from the timing when the live streaming started. However, the expression format of the start timing and the end timing is not limited to the format exemplified in this embodiment. The start timing and the end timing may be information that can be used to generate extracted video data, which will be described later.
[0091] In step SG2, the server control unit 14 executes the following process as an example. For example, assume that the reply started by selecting the reply start button 24 in step SF1 is the second reply. In this case, in step SG2, the server control unit 14 first recognizes the start timing. For example, assume that the elapsed time from the start of live streaming is "5:00". In this case, the server control unit 14 recognizes "5:00" as the start timing. Next, the server control unit 14 records information indicating that this is the second reply, the request ID of the highest-ranked request, and the start timing in the reply log data 34 according to the format.
[0092] Next, the server control unit 14 assigns the noted broadcaster earned points equal to the assigned point value assigned to the top request (step SG3). Specifically, the server control unit 14 increases the total earned points value of the record corresponding to the noted broadcaster in the user management database 36 by the assigned point value assigned to the top request. The total earned points value for a certain user is the cumulative value of the earned points value acquired by that user. Although details are omitted, the noted broadcaster can receive appropriate benefits in exchange for the earned points. As an example, the noted broadcaster can be awarded money equivalent to the amount of the earned points. The fact that users can receive appropriate benefits in exchange for their earned points in this way provides an incentive for users to reply.
[0093] Next, the server control unit 14 generates display request information for the highest request based on the record corresponding to the highest request in the request management database 40 corresponding to the noted distributor (step SG4). Next, the server control unit 14 transmits request display instruction information to the distributor terminal 2L and the viewer terminal 2A (step SG5). The request display instruction information includes display request information corresponding to the highest request, and is information that instructs the display of a request card 44 corresponding to the highest request.
[0094] As shown in the flowchart FH, when the terminal control unit 6A receives the request display instruction information, it pops up a request card 44 corresponding to the highest request on the viewer's screen 29 (step SH1). At that time, the terminal control unit 6A changes the color of the request card 44 according to the assigned point value. The relationship between the assigned point value and the color is as described above. FIG. 14 shows the request card 44 corresponding to the highest request displayed on the viewer's screen 29. Since the request card 44 corresponding to the highest request is displayed on the viewer's screen 29 in response to the start of a reply, the viewed viewer can accurately grasp the content of the corresponding request while the viewed broadcaster is replying.
[0095] As shown in the flowchart FF, when the terminal control unit 6L receives the request display instruction information, it pops up a request card 44 corresponding to the highest-ranked request on the broadcaster side screen 20 (step SF3). At that time, the terminal control unit 6A changes the color of the request card 44 according to the assigned point value. The relationship between the assigned point value and the color is as described above. By referring to the color of the request card 44, the noted broadcaster can intuitively grasp the assigned point value.
[0096] Furthermore, the server control unit 14 displays a reply end button 63 instead of the reply start button 24, and a discard button 64 instead of the request box confirmation button 25 (step SF4). FIG. 15 shows a state in which a request card 44 corresponding to the highest-ranked request is displayed on the distributor's screen 20. The discard button 64 is a button that instructs not to generate extracted video data (described later) for the reply being executed. When the terminal control unit 6L detects that the discard button 64 has been selected, it notifies the server control unit 14, and the server control unit 14 deletes the corresponding reply performance information RJ from the reply log data 34. In this case, when the reply end button 63 is selected by the noted distributor, the server control unit 14 does not record information indicating the end timing in the reply log data 34. By not recording the reply performance information RJ for a certain reply in the reply log data 34, extracted video data corresponding to that reply is not generated.
[0097] As described above, when the noted broadcaster instructs the broadcaster to start replying, the broadcaster request card 44L corresponding to the highest-ranked request is displayed on the display device 12L of the broadcaster terminal 2L. The broadcaster can understand the contents of the request by referring to the broadcaster request card 44L and can reply to the request. When the noted broadcaster starts replying, the noted broadcaster will exclusively respond to the request with the highest reply priority at that time. In other words, for a person who is about to post a request, the higher the reply priority of the request that the person is about to post, the higher the priority of the reply will be, and the more exclusively the person will be able to respond to it. For this reason, the present service can give users an incentive to raise the reply priority of requests.
[0098] The noted broadcaster replies to the request. For example, if the request is a question, the noted broadcaster answers the question. As shown in flowchart FF, when the noted broadcaster finishes replying, the broadcaster selects the reply end button 63 to instruct the end of the reply (flowchart SF5). When the terminal control unit 6L detects that the reply end button 63 has been selected, it notifies the server control unit 14 of the end of the reply (step SF6). Furthermore, the terminal control unit 6L stops displaying the request card 44, and displays the reply start button 24 and the request box confirmation button 25 on the broadcaster side screen 20 instead of the reply end button 63 and the discard button 64 (step SF7).
[0099] As shown in flow chart FG, when the server control unit 14 receives the notification of the end of the reply, it records information about the end of the reply in the reply log data 34 of the record corresponding to the noted live stream in the live streaming management database 35 (step SG6). More specifically, the server control unit 14 recognizes the end timing (the amount of time elapsed from the start of the live streaming to the present time) and records the recognized end timing in the current reply performance information RJ of the reply log data 34 according to a format. Furthermore, the server control unit 14 sends an instruction to the terminal control unit 6A to stop displaying the request card 44 (step SG7).
[0100] As shown in the flow chart FH, when the terminal control section 6A receives an instruction to stop displaying the request card 44, it stops displaying the request card 44 that has been displayed on the viewer's screen 29 (step SH2).
[0101] As described above, the server control unit 14 has a function to detect when the noted broadcaster instructs to start a reply, record information in the reply log data, transmit request information for display for the highest-ranked request, and execute other processes to reflect the instruction to start a reply. This function corresponds to a function to accept an instruction to start a reply to a request during live streaming by a noted user. The server control unit 14 also has a function to detect when the noted broadcaster instructs to end a reply, record information in the reply log data, instruct to stop displaying the request card, and execute other processes to reflect the instruction to end a reply. This function corresponds to a function to accept an instruction to end a reply to a request. Based on this, the server control unit 14 has a function to accept an instruction to start a reply to a request and an instruction to end a reply during live streaming by a noted broadcaster, and to display one of the requests on the terminal 2 of a viewer of the live streaming when an instruction to start a reply is given. The server control unit 14 also has a function of displaying, on the viewer's terminal 2, the request with the highest reply priority (priority) among the unanswered requests, when an instruction to start replying is given during live distribution.
[0102] In this embodiment, the reply ends only when the distributor selects the reply end button 63. In this regard, the following configuration may be used. That is, a limit may be set on the time from when the reply starts to when it ends, and when the limit is reached, the reply is forcibly ended and the processing from step SF6 onwards is executed.
[0103] <Operation when generating extracted video data> Next, a scene in which extracted video data is generated will be described. Extracted video data refers to video data (video file) that records video related to live streaming from the start to the end of one reply. As mentioned above, multiple replies may be made in one live streaming. In this service, extracted video data is automatically generated for each reply made.
[0104] Flowchart FI in FIG. 16 is a flowchart showing the operation of the information processing server 3 when generating extracted video data related to a featured live stream. In the following description, it is assumed that a featured broadcaster has replied multiple times in the featured live stream. The process of flow chart FI in FIG. 16 is started at a predetermined timing after the featured live stream ends. For example, the process is executed immediately after the featured live stream ends. Also, for example, the process is started as a batch process at a fixed time of day.
[0105] As shown in flow chart FI, the server control unit 14 refers to the record corresponding to the featured live stream in the live streaming management database 35, and acquires the entire live streaming video data related to the featured live stream (step SI1). The entire live streaming video data acquired here is video data in which the entire featured live stream is recorded. Next, the server control unit 14 acquires reply log data 34 of the record corresponding to the featured live stream (step SI2). The reply log data 34 acquired here records reply performance information RJ for each reply made in the featured live stream.
[0106] Next, the server control unit 14 executes an extracted video data generation process based on the entire live streaming video data and the reply log data 34 (step SI3). More specifically, the server control unit 14 generates video data by extracting a portion from the start timing to the end timing for each reply from the entire live streaming video data. One piece of video data corresponding to one reply generated here corresponds to "extracted video data". For example, assume that the content of the reply log data 34 is as shown in FIG. 17(A). Also assume that the entire length of the entire live streaming video data is "10:00". In this case, as shown in FIG. 17(B), the server control unit 14 extracts the portion "1:00-2:01" from the entire live streaming video data to generate one piece of extracted video data, extracts the portion "3:30-5:00" to generate one piece of extracted video data, and extracts the portion "7:01-9:01" to generate one piece of extracted video data. The extracted video data records a video related to the live streaming from the start to the end of the reply. Hereinafter, the moving image recorded in the extracted moving image data will be referred to as the "extracted moving image."
[0107] Although details are omitted, the server control unit 14 appropriately performs encoding to change the format to one suitable for streaming distribution, and other processing required to make the extracted video data available for streaming distribution, based on existing technology.
[0108] As described above, the server control unit 14 has a function of generating extracted video data in which video related to live streaming from the start to the end of a reply is recorded. The extracted video recorded in the extracted video data here is a video with consistent and cohesive content in that it corresponds to one reply. In recent years, not only live streaming but also videos extracted from parts of live streaming (sometimes called "cut-out videos") have been widely provided and viewed, and in light of this, the extracted video data can be said to be data with high utility value. This embodiment has a strong advantage in that such highly useful extracted video data is automatically generated.
[0109] Next, the server control unit 14 executes an appropriateness determination process (step SI4). The process executed by the server control unit 14 in step SI4 for one extracted video data will be described below. First, the server control unit 14 determines whether the extracted video data satisfies a preset inappropriateness condition. In this embodiment, one of the inappropriateness conditions is a "condition that is met when the length of the extracted video is shorter than the allowable range" (hereinafter referred to as a "time-related condition"). If the length of the extracted video is shorter than a predetermined threshold, the server control unit 14 determines that the time-related condition is met. If the length of the extracted video is shorter than the allowable range, it means that the noted broadcaster spent only a very short period of time replying, and the corresponding request may have been undesirable for the noted broadcaster, or the noted broadcaster may have mistakenly selected the reply end button 63.
[0110] One of the inappropriate conditions is a "condition that is met when the brightness of the extracted video is darker than the allowable range" (hereinafter referred to as a "brightness-related condition"). The server control unit 14 derives the average brightness of each frame of the extracted video, and further derives the average value of the average brightness of each frame. If the derived average value is smaller than a predetermined threshold, it is determined that the brightness-related condition is met. The threshold is set to a value such that the brightness-related condition is met when the extracted video continues to be nearly completely dark. If the brightness-related condition is met, it means that the extracted video continues to be nearly completely dark, and it is assumed that the featured broadcaster does not want to make it public. Note that the brightness-related condition exemplified in this embodiment is merely an example, and it may be met when the brightness of the extracted video is darker than the allowable range.
[0111] In the appropriateness determination process, when at least one of the time-related conditions and the brightness-related conditions is satisfied for a certain extracted video data, the server control unit 14 determines that the extracted video data satisfies the inappropriateness condition. The server control unit 14 deletes the extracted video data that satisfies the inappropriateness condition. The process of deleting the extracted video data corresponds to the process of discarding the extracted video data.
[0112] In this embodiment, the server control unit 14 is configured to delete the extracted video data if the inappropriateness condition is met after generating the extracted video data. In this regard, the server control unit 14 may be configured to determine whether the inappropriateness condition is met when generating the extracted video data, and not generate the extracted video data if the inappropriateness condition is met. The process of not generating the extracted video data corresponds to the process of discarding the extracted video data.
[0113] As described above, when generating extracted video data, the server control unit 14 has a function of discarding the extracted video data if the length of the recorded video is shorter than the acceptable range or if the brightness of the recorded video is darker than the acceptable range.
[0114] Next, the server control unit 14 registers each of the records corresponding to each of the generated extracted video data in the extracted video management database 65 stored in the server storage unit 16 (step SI5). The extracted video management database 65 is a database that manages each of the extracted video data, and a record is registered for each extracted video data. FIG. 6(D) shows the contents of one record in the extracted video management database 65. As shown in FIG. 6(D), one record in the extracted video management database 65 has the following fields: extracted video ID <field>, distributor ID <field>, poster ID <field>, request ID <field>, extracted video <field>, thumbnail image <field>, display request <field>, title <field>, and tag <field>.
[0115] The extracted video ID <field> stores an extracted video ID, which is identification information that uniquely identifies the corresponding extracted video data. The distributor ID <field> stores the user ID of the distributor of the corresponding live broadcast. The request ID <field> stores the request ID of the corresponding request (= the request to which the distributor replies in the extracted video based on the corresponding extracted video data). The extracted video <field> stores the corresponding extracted video data. The thumbnail image <field> stores image data of a thumbnail image based on the corresponding extracted video data (hereinafter referred to as "thumbnail image data"). The display request <field> records the display request information of the corresponding request. As described above, the display request information includes all the information necessary to display the request card 44. The title <field> stores information indicating the title of the corresponding extracted video data. The tag <field> stores tag information of the present tag (the specific value of the present tag). The present tag is a tag that is assigned to the content as meta information in the content available in the present service. The present tag is used for searching and classifying the content. This tag has the same function as a so-called hashtag.
[0116] In step SI5, when registering one record corresponding to one extracted video data, the server control unit 14 executes the following process. The server control unit 14 generates a unique extracted video ID and stores it in the extracted video ID <field>. The server control unit 14 also stores the user ID of the noted broadcaster in the broadcaster ID <field>. The server control unit 14 also stores the request ID of the corresponding request (the request ID recorded in the corresponding reply performance information RJ of the reply log data 34) in the request ID <field>. The server control unit 14 also stores the extracted video data in the extracted video <field>. The server control unit 14 also generates a thumbnail image based on the extracted video data and stores it in the thumbnail image <field>. The server control unit 14 also generates thumbnail image data by acquiring one of the frames constituting the extracted video data and superimposing a request card 44 based on the corresponding display request information. The server control unit 14 also stores the display request information of the corresponding request in the display request <field>. The server control unit 14 also sets a title according to the rules and stores information indicating the title in the title <field>. One example of the rule is that the date of live distribution and information indicating the number of replies are combined in a predetermined manner. Also, the server control unit 14 stores a null value in the tag <field>.
[0117] Next, the server control unit 14 executes the publishing process (step SI6). In this embodiment, the server control unit 14 executes the publishing process immediately after generating the extracted video data, but the server control unit 14 may execute the publishing process after a certain time has elapsed after generating the extracted video data. The publishing process is a process of constructing a state in which the extracted video can be viewed for each of the generated extracted video data. Hereinafter, making the corresponding extracted video available for viewing for a certain extracted video data may be expressed as "publishing the extracted video data". In addition, in the following, for a certain extracted video data, the user viewing the extracted video based on the extracted video data may be simply expressed as "the user views the extracted video data". Hereinafter, in this embodiment, the server control unit 14 provides the extracted video by streaming the extracted video data. The streaming distribution based on the extracted video data is performed using existing technology. In the following description, the streaming distribution itself will not be described in detail, assuming that the streaming distribution for realizing the provision of the extracted video is appropriately performed. However, the extracted video may be provided by a means other than streaming distribution.
[0118] Hereinafter, several examples of the publishing process will be described assuming that the publishing process is performed on extracted video data based on live streaming by a featured broadcaster. In this service, the server control unit 14 can cooperate with the terminal control unit 6 to provide an extracted video list screen for a specified specific user. The extracted video list screen for a certain user is a screen on which thumbnail images of extracted video data based on live streaming performed by the user as a broadcaster are displayed together with accompanying information. For example, user A can display the extracted video list screen for user B on the display device 12 of his / her own terminal 2 by performing a predetermined input involving the designation of user B. The extracted video list screen for user B is a screen on which thumbnail images of extracted video data based on live streaming performed by user B as a broadcaster are displayed in a list. When a thumbnail image (which may be accompanying information or a related object) on the extracted video list screen is selected, the extracted video data corresponding to the selected thumbnail image is streamed, and the extracted video is provided. Then, in the publishing process, the server control unit 14 configures the extracted video list screen related to the featured broadcaster to display thumbnail images related to each of the generated extracted video data, and creates a state in which the corresponding extracted video is provided when a thumbnail image is selected.
[0119] The service also provides a service (hereinafter referred to as the "specific viewing service") that allows the user to switch between extracted videos to be viewed by scrolling the screen (or by other operations). In the specific viewing service, switching between extracted videos is performed by the server control unit 14 in cooperation with the terminal control unit 6. Extracted video data corresponding to the next extracted video to be displayed in response to a switching instruction is selected by the server control unit 14 from the pooled extracted video data in a predetermined manner. The predetermined method is, for example, randomly or according to an algorithm that reflects the viewing user's preferences. In the disclosure process, the server control unit 14 adds each of the generated extracted video data to the pool, and creates a state in which extracted videos based on the generated extracted video data can be provided in the specific viewing service.
[0120] The service also allows for the search of extracted video data using tags. Extracted videos based on extracted video data found as a result of the search are provided according to the user's selection. In the publication process, the server control unit 14 creates a state in which the video can be searched by tag. Although several examples of the publication process have been described above, the publication process is not limited to these. In other words, the publication process only needs to create a state in which the extracted video based on the extracted video data can be viewed in the service.
[0121] As described above, the extracted video data is very useful data. In this embodiment, the generation and publication of the extracted video data is performed automatically. In other words, a distributor simply performs live streaming and then replies during the live streaming, and extracted video data based on the live streaming is automatically generated and published. Conventionally, extracted video data (extracted video) is called cut-out video data, and the generation and publication of the extracted video data was performed by the distributor himself or by entrusting it to someone else. In this embodiment, the generation and publication of the extracted video data does not need to be performed by the distributor himself or entrusted to someone else, which is highly convenient for the distributor.
[0122] After the publishing process, the server control unit 14 executes a notification process (step SI7). The notification process is a process of notifying the distributor, the request poster, and the positive evaluation giver that the extracted video data has been published for each of the extracted video data generated. In this embodiment, a box (hereinafter referred to as a "notification box") is set for each user to receive various notifications related to the service. Each user can display a screen on the display device 12 that lists notifications received in the notification box, and can further display details of a notification by selecting one notification from the screen. The server control unit 14 transmits a notification to the notification box of each of the distributor, the poster, and the positive evaluation giver for each of the extracted video data generated. The notification corresponding to one extracted video data includes at least the distributor's username, thumbnail image, and title. A link for transitioning to a screen for watching an extracted video based on the extracted video data may be added.
[0123] As described above, the server control unit 14 has a function of constructing a state in which the video recorded in the extracted video data can be viewed, and a function of notifying the poster of the request corresponding to the extracted video data and the user who gave a positive evaluation to the request corresponding to the extracted video data according to the construction of the state. According to this configuration, the request poster and the positive evaluation giver can each recognize the fact that the extracted video data has been made public by the notification, and can view the extracted video whenever they like. In particular, since the notification is given to the positive evaluation giver in this embodiment, when there is a request card 44 that interests each user, if each user gives a positive evaluation to the request card 44, when the corresponding extracted video data is made public, the user can recognize the fact and view the extracted video at a convenient time. In more detail, it is assumed that a certain user (assumed to be user U1) refers to the normal box screen 38 of another user (assumed to be user U2) who may be a distributor, and finds a request card 44 (request) that interests him / her and wonders how user U2 will respond. At this time, user U1 gives a positive evaluation to the request card 44. Then, after the live distribution where a reply was made to the request card 44 ends, the server control unit 14 automatically generates and publishes extracted video data corresponding to the request card 44, and notifies the user U1 of this fact. Even if user U1 misses the live distribution, he or she will know that the extracted video data corresponding to the request card 44 in which he or she was interested has been published, and can view it at his or her convenience.
[0124] <What happens when a user watches an extracted video> Next, the operation of the information processing system 1 when a user watches an extracted video will be described. The extracted video data generated as described above is made public. The user can use the service to watch the desired extracted video data. The server control unit 14 executes special processing when the user watches extracted video data based on a live broadcast in which the user was the broadcaster. In the following description, the broadcaster of the original live broadcast for a certain extracted video data is referred to as the "corresponding broadcaster." The server control unit 14 also executes special processing when the user watches extracted video data corresponding to a request posted by the user. In the following description, the corresponding request poster for a certain extracted video data is referred to as the "corresponding request poster."
[0125] 18 is a flowchart showing the operation of the information processing system 1 when viewing of the extracted video starts. Flowchart FJ shows an information processing method by the terminal 2, and flow chart FK shows an information processing method by the information processing server 3. In the following description, a user viewing the extracted video is referred to as a "viewing user", and the extracted video data viewed by the viewing user is referred to as "attention extracted data",
[0126] As shown in the flow chart FK, the server control unit 14 of the information processing server 3 starts streaming distribution of the attention extracted video data (step SK1). In conjunction with the start of streaming distribution, the server control unit 14 transmits display request information and tag information (multiple pieces of tag information if there are multiple pieces of tag information) of the record (hereinafter referred to as the "attention record") corresponding to the attention extracted video data in the extracted video management database 65 to the terminal control unit 6L (step SK2). Note that if no tag information is stored in the tag <field>, the tag information is not transmitted.
[0127] Furthermore, the server control unit 14 determines whether the viewing user is the relevant distributor or the relevant request poster (step SK3). The fact that the viewing user is the relevant distributor means that the attention extraction video data is based on live distribution performed by the viewing user as the distributor. Furthermore, the fact that the viewing user is the relevant request poster means that the attention extraction video data corresponds to a request posted by a viewer. The server control unit 14 determines whether the viewing user is the relevant distributor or the relevant request poster in the following manner. That is, the server control unit 14 makes the determination based on the match between the user ID of the viewing user and the user ID stored in the distributor ID <field> or the user ID stored in the poster ID <field> of the focused record.
[0128] If the viewing user is the relevant distributor or the relevant request poster (step SK3: YES), the server control unit 14 sends a permission notification to the terminal control unit 6 notifying that the addition of tag information is permitted (step SK4). On the other hand, if the viewing user is neither the relevant dividend recipient nor the relevant request poster (step SK3: NO), the server control unit 14 sends a non-permission notification to the terminal control unit 6 notifying that the addition of tag information is not permitted (step SK5).
[0129] As shown in flow chart FJ, the terminal control unit 6 receives each piece of information transmitted by the server control unit 14 (step SJ1). Next, the server control unit 14 determines whether a permission notice or a non-permission notice has been received (step SJ2). If a permission notice has been received (step SJ2: "permission notice"), the server control unit 14 displays a special viewing screen 66 on the display device 12 (step SJ3). On the other hand, if a non-permission notice has been received (step SJ2: "non-permission notice"), the server control unit 14 displays a normal viewing screen 67 on the display device 12 (step SJ4).
[0130] FIG. 19 (A) shows a special viewing screen 66, and (B) shows a normal viewing screen 67. The special viewing screen 66 in FIG. 19 (A) is displayed when the viewing user is the relevant distributor or the relevant request poster, and (B) is displayed when the viewing user is neither the relevant distributor nor the relevant request poster. As shown in FIG. 19, a distribution video 68 is displayed as a background on the special viewing screen 66. The distribution video 68 is a video displayed based on streaming distribution of the extracted video data, and corresponds to the captured video 21 during live distribution. A request card 44 is displayed slightly below the center of the distribution video 68. The terminal control unit 6 displays the request card 44 in the received display request information. In this way, the terminal control unit 6 has a function of displaying the video recorded in the extracted video data together with the content of the request (request card 44) corresponding to the extracted video data.
[0131] A tag display area 69 is provided below the request card 44 on the special viewing screen 66. Tag information attached to the corresponding extracted video data is displayed in the tag display area 69. Each piece of tag information is a selectable object, and when tag information is selected, the server control unit 14 searches for content (including the extracted video data) using the tag information as a key, and provides the search results.
[0132] On the special viewing screen 66, a tag add button 70 is displayed to the right of the tag display area 69. The tag add button 70 is a button for adding tag information. When the tag add button 70 is selected, the terminal control unit 6, in cooperation with the server control unit 14, displays a tag-related screen (not shown) that allows input of one or more pieces of tag information and an instruction to add the input tag information. When an instruction to add one or more pieces of tag information is given using the tag-related screen, the server control unit 14, in cooperation with the terminal control unit 6, additionally records the tag information instructed to be added in the tag <field> of the focused record. In this way, the relevant distributor or the relevant request poster can add tag information to the focused extraction video data. In other words, the server control unit 14 permits the relevant distributor and the relevant request poster to add tag information (meta information) to the focused extraction video data. In addition, the server control unit 14 has a function of detecting an instruction to add tag information from the relevant request poster and executing a process to reflect the instruction to the extracted video management database 65 and the addition of other tag information. This function corresponds to a function of accepting meta-information from the poster of a request.
[0133] 19B, the normal viewing screen 67 does not display the tag addition button 70, which is different from the special viewing screen 66. A user viewing the normal viewing screen 67 does not have a means for adding tag information and cannot add tag information to the attention extraction video data. In other words, a user who is neither the relevant distributor nor the relevant request poster cannot add tag information to the attention extraction video data.
[0134] As described above, the server control unit 14 has a function of allowing the poster of the request corresponding to the extracted video data to add meta information to the generated extracted video data after generating the extracted video data, and accepting the addition of meta information from the poster of the request. With this configuration, by posting a request, a user can obtain the privilege of being able to add tag information when extracted video data based on the request is generated and made public. This can provide an incentive to the user to post a request. In addition, the terminal control unit 6 has a function of displaying the video recorded in the extracted video data together with the content of the request (request card 44) corresponding to the extracted video data. With this configuration, a user who watches the extracted video can watch the extracted video while understanding the content of the request, resulting in high user satisfaction.
[0135] The means by which the relevant payee and the relevant request poster give tag information is not limited to the example. For example, when the server control unit 14 notifies the relevant payee and the relevant request poster that the extracted video data has been made public in the notification process of step SI7 of the flowchart FI in Fig. 16, it also provides a user interface for inputting tag information. The relevant payee and the relevant request poster may be configured to give tag information using the user interface.
[0136] Furthermore, the meta information that can be added is not limited to tag information. For example, it may be a title or subtitle of the extracted video data. Furthermore, the terminal control unit 6 may cooperate with the server control unit 14 to display an image related to a comment posted at that time when displaying the extracted video, in addition to the request card 44. Furthermore, the terminal control unit 6 may display the request card 44 (or information with equivalent content) in an area avoiding the delivery video image 68 on the screen for viewing the extracted video, rather than displaying the request card 44 superimposed on the delivery video image 68.
[0137] As described above, the server control unit 14 has a function to accept requests posted by users to a noted user who may become a live broadcast distributor, a function to accept instructions to start a reply to a request and an instruction to end the reply during a live broadcast by the noted user, and when notified of the start of a reply, to display one of the requests on the terminal 2 of a viewer of the live broadcast, and a function to generate extracted video data in which video related to the live broadcast from the start to the end of the reply is recorded.
[0138] According to this configuration, during live distribution, the distributor instructs the start of a reply, replies to the request displayed on the viewer's terminal in response to the instruction, and instructs the end of the reply after the reply is completed. This allows extracted video data to be generated in which a video related to the live distribution during the reply period is recorded for each reply made by the distributor. The video recorded in this extracted video data is a video with consistent and cohesive content in that it corresponds to one reply. In recent years, not only live distribution but also videos extracted from a part of a live distribution (so-called cut-out videos) have been widely provided and viewed, and in light of this, the extracted video data can be said to be data with high utility value. According to the present invention, in order to generate such extracted video data with high utility value, the distributor does not need to edit video data in which the video of the entire live distribution is recorded or rely on others to generate the extracted video data, and the satisfaction of the distributor can be improved. In other words, according to the present invention, the satisfaction of the distributor can be improved with respect to services related to live distribution.
[0139] <Modification> Next, a modified example will be described. In the modified example, the same elements as those in the above embodiment are given the same reference numerals. In the above embodiment, only the request poster can grant the granted point value to a certain request. On the other hand, in this embodiment, each user can add a desired additional point value to the granted point value of a desired request. Below, the operation of the information processing system 1 according to the modified example will be described using as an example a case where user U1 adds additional point value to a request already posted to user U2.
[0140] User U1 can instruct to display a normal box screen 38X corresponding to user U2 at any timing, regardless of whether user U2 is live streaming or not. In response to the instruction, the terminal control unit 6 of the terminal 2 of user U1 cooperates with the server control unit 14 to display the normal box screen 38X on the display device 12.
[0141] FIG. 20A is a diagram showing an example of a normal box screen 38X. As is clear from a comparison between FIG. 9 and FIG. 20A, in the normal box screen 38X, an add point button 72 is added in association with the granted point image 46 of the request card 44. The add point button 72 is a button selected when adding additional point values. When the user U1 selects the add point button 72, the terminal control unit 6 cooperates with the server control unit 14 to pop up and display an additional point input screen 73. The server control unit 14 provides information necessary for displaying the additional point input screen 73 based on the user management database 36, the request management database 40, and other data.
[0142] Fig. 20(B) is a diagram showing an example of an additional point input screen 73. As shown in Fig. 20(B), a point balance image 52 showing the balance of the present points of user U1 is displayed at the top of the additional point input screen 73. An additional point input field 74 is provided below the point balance image 52, and an additional point slider 75 is provided below this. The additional point input field 74 and the additional point slider 75 have the same functions as the granted point input field 54 and the granted point slider 55 of the request input screen 50 (Fig. 10), respectively, and detailed explanations will be omitted.
[0143] Below the additional point slider 75, an add maximum point button 76 is provided. The add maximum point button 76 is a button for instructing input of an additional point value that will make the "total point value" higher by "1 pt" than the "highest assigned point value among the currently assigned point values of each of the unhandled requests". The "total point value" means a value obtained by adding the currently assigned point value of the request and the additional point value. Hereinafter, the "highest assigned point value among the currently assigned point values of each of the unhandled requests" will be referred to as the "other request maximum value". For example, suppose that the other request maximum value is 3000 pt, and the currently assigned point value of the request to which the user U1 is trying to add additional point value is 1000 pt. In this case, when the user U1 selects the add maximum point button 76, "2001 pt" is input as the additional point value. In this case, the total point value becomes "3001 pt" (1000 pt + 2001 pt), which is higher by "1 pt" than the other request maximum value of "3000 pt".
[0144] Below the maximum point addition button 76, a total point image 77 is displayed. The total point image 77 displays the total of the currently granted point value of the corresponding request and the additional point value input by user U1. Below the total point image 77, a provisional priority order image 78 is displayed, which indicates what the reply priority of the request will be when the input additional point value is added. Below the provisional priority order image 78, a confirmation button 79 for confirming the input is provided.
[0145] When the Confirm button 79 is selected, the server control unit 14 works in cooperation with the terminal control unit 6 to increase the assigned point value in the assigned points <field> of the corresponding record in the request management database 40 of user U2 by the input additional point value. As described above, the assigned point value assigned to a request affects the reply priority of that request. In this way, when a user instructs the addition of additional point values to a request, the server control unit 14 has a function to detect this and to execute processing to reflect the addition of additional point values to the request management database 40 and other operations. This function corresponds to a function to accept the addition of point values to a request from users, not limited to the request submitter.
[0146] As described above, the server control unit 14 has a function of accepting the addition of additional point values (point values) to a request from users, not limited to the request poster, after the request has been posted. The terminal control unit 6 also has a function of providing a normal box screen 38X (screen) on which a list of request cards 44 (information about the request) is displayed and on which an add point button 72 (an object for inputting an additional point value to be added to the request) is displayed in association with the request card 44. The terminal control unit 6 also has a function of providing an additional point input screen 73 (screen) on which a provisional priority order image 78 (information) indicating the priority order of the request when the additional point value inputted into the object for inputting the additional point value is added to the request is displayed. The terminal control unit 6 also has a function of providing an additional point input screen 73 (screen) on which a maximum point addition button 76 (button) for inputting an additional point value that will give the highest priority to the request is displayed.
[0147] According to the configuration of this modified example, a user can add a desired additional point value to a desired request. Therefore, when a user wants another user to reply to a request that has already been posted by another user as soon as possible, or wants to increase the possibility of the other user replying to the request, the user can increase the reply priority of the request by adding an additional point value to the request.
[0148] The screens shown in the modified examples are merely examples, and screens with different contents may be provided. For example, the point addition button 72 may not be displayed within the request card 44, but may be displayed outside the request card 44 in association with the request card 44. Also, for example, instead of the point addition button 72, an object for directly inputting an additional point value, such as an additional point input field 74, may be provided.
[0149] Although one embodiment of the present invention (including modifications) has been described above, the above embodiment is merely one example of a specific embodiment of the present invention, and the technical scope of the present invention should not be interpreted as being limited thereby. In other words, the present invention can be embodied in various forms without departing from the gist or main characteristics thereof.
[0150] For example, the process of generating extracted video data is not limited to the process exemplified in the above embodiment. As an example, the server control unit 14 may be configured to record (save) in real time the video that is delivered during live streaming from when it receives notification of the start of a reply to when it receives notification of the end of the reply, and generate extracted video data based on the recording result. In other words, the specific process of generating extracted video data is not limited to the process exemplified in the above embodiment for the "function of generating extracted video data in which video related to live streaming from the start to the end of a reply is recorded" of the server control unit 14, and any process may be performed.
[0151] Also, for example, as described above, the information processing server 3 does not have to be a single device. In particular, the information processing server 3 may be configured by a combination of one or more server devices that realize live distribution and one or more other server devices. When the information processing server 3 is configured by multiple server devices, the server control units of the multiple server devices work together to function as a "control unit."
[0152] Also, for example, a validity period may be set for a request, and if a reply is not made after the validity period has elapsed, the server control unit 14 may discard the request. The discarding of a request may be performed, for example, by controlling a flag that manages whether or not the request has been discarded.
[0153] Furthermore, in the above embodiment, the server control unit 14 has a function for accepting comments posted by viewers during live distribution, but this function is not essential.
[0154] In addition, for example, in the above embodiment, a configuration may be adopted in which a web browser executes part or all of the processing of the dedicated application instead.
[0155] Also, for example, in the above embodiment, the server control unit 14 is configured to display the request (request card 44) with the highest reply priority on the viewer's terminal 2 in response to notification of the start of reply. In this regard, the server control unit 14 may be configured to select the request to be displayed in another manner. For example, the server control unit 14 may be configured to randomly determine one unprocessed request from among the unprocessed requests. Also, for example, the server control unit 14 may be configured to preferentially select requests posted earlier. In the case where the assigned point value does not affect the reply priority, the server control unit 14 may not have a function to accept the assignment of the assigned point value. In the case where the number of positive evaluations does not affect the reply priority, the server control unit 14 may not have a function to accept the assignment of positive evaluations.
[0156] The points in question were unique points circulated in the service in question. However, the specific content of the points in question is not limited to the content exemplified in this embodiment. For example, the points in question may be points used in services other than the service in question. For example, the points in question may be legal tender or cryptocurrency.
[0157] Also, for example, a part of the processes described as being executed by the server control unit 14 may be executed in cooperation with the server control unit 14 and an information processing mechanism of an external device capable of communicating with the information processing server 3. In this case, the information processing server 3 and the external device cooperate to function as an "information processing device." This also applies to the terminal control units 6, 6L, and 6A.
[0158] Also, for example, the embodiment may include the provision of a program executed by the computer of the terminal 2 or the information processing server 3, or the provision of a recording medium on which the program is recorded so as to be readable by the computer. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device. Specific examples of the recording medium include portable or fixed recording media such as flexible disks, HDDs (Hard DiSH Drives), CD-ROMs (Compact DiSH Read Only Memory), DVDs (Digital Versatile DiSH), Blu-ray (registered trademark) Discs, magneto-optical disks, flash memories, and card-type recording media.
[0159] Also, for example, with respect to the exemplary flowcharts, the order of processes may be changed, processes may be divided into smaller processes, processes may be added, or processes may be deleted, as long as the purpose can be achieved and no contradictions occur in the processes. [Explanation of symbols]
[0160] 1. Information Processing Systems 2. Terminal 3. Information processing server (information processing device) 6 Terminal control unit 14 Server control unit (control unit)
Claims
1. A function to accept requests from users to potential live streamers; a function of receiving an instruction to start a reply to the request and an instruction to end the reply during the live distribution by the noted user, and displaying one of the requests on a terminal of a viewer of the live distribution when the instruction to start the reply is received; a control unit having a function of generating extracted video data in which a video related to the live distribution from the start to the end of the reply is recorded; 23. An information processing apparatus comprising:
2. The control unit is When an instruction to start replying is given during the live distribution, the highest priority request among the unreplied requests, which are the requests to which the user of interest has not replied, is displayed on the terminal of the viewer.
2. The information processing apparatus according to claim 1,
3. The control unit is A function of accepting an assignment of a point value to the request from a poster of the request when the request is posted; and a function of setting the priority of the unresponded request such that the priority is higher as the assigned point value is larger.
3. The information processing apparatus according to claim 2.
4. The control unit is After the request is posted, a function is provided for accepting addition of the point value to the request from a user other than the person who posted the request.
4. The information processing apparatus according to claim 3.
5. The control unit is A function of accepting a positive evaluation of the request from the user; and a function of setting the priority of the unresponded request such that the priority is higher as the assigned point value is larger and, if the assigned point value is the same, the number of assigned positive evaluations is larger.
4. The information processing apparatus according to claim 3.
6. The control unit is When generating the extracted video data, if the length of the recorded video is shorter than an allowable range or if the brightness of the recorded video is darker than an allowable range, the extracted video data is discarded.
2. The information processing apparatus according to claim 1,
7. The control unit is and a function of permitting a poster of the request corresponding to the extracted video data to add meta information to the generated extracted video data after generating the extracted video data, and accepting the addition of the meta information from the poster of the request.
2. The information processing apparatus according to claim 1,
8. The control unit is A function of constructing a state in which the video recorded in the extracted video data can be viewed; and a function of notifying the poster of the request corresponding to the extracted video data and the person who gave the positive evaluation to the request corresponding to the extracted video data in response to the construction of the state.
6. The information processing apparatus according to claim 5,
9. A terminal capable of communicating with the information processing device according to any one of claims 2 to 5, a terminal control unit having a function of providing a screen on which information about the requests is arranged in accordance with the priority order; A terminal characterized by:
10. A terminal capable of communicating with the information processing device according to claim 3, a terminal control unit having a function of providing a screen on which an object for inputting the point value to be given to the request when the request is posted and information indicating what the priority of the request will be when the point value inputted to the object is given to the request are displayed A terminal characterized by:
11. A terminal capable of communicating with the information processing device according to claim 3, A terminal control unit having a function of providing a screen on which a button for inputting the point value that will give the highest priority to the request is displayed when the request is posted. A terminal characterized by:
12. A terminal capable of communicating with the information processing device according to claim 3, a function of providing, when posting the request, a screen displaying an input field for inputting the request and an object for inputting the point value to be assigned to the request; a terminal control unit having a function of increasing an upper limit of the number of characters that can be input into the input field as the point value input into the object increases; A terminal characterized by:
13. A terminal capable of communicating with the information processing device according to claim 4, and a terminal control unit having a function of providing a screen on which information about the request is displayed as a list and an object for inputting an additional point value to be added to the request in association with the information about the request is displayed. A terminal characterized by:
14. The terminal control unit a function of providing a screen on which information indicating what the priority of the request will be when the additional point value input to the object is added to the request is displayed 14. The terminal according to claim 13,
15. The terminal control unit a function of providing a screen on which a button for inputting the additional point value that will give the highest priority to the request is displayed 14. The terminal according to claim 13,
16. A terminal capable of communicating with the information processing device according to claim 5, A terminal control unit having a function of providing a screen on which information about the request is displayed as a list and an object for providing the positive evaluation in association with the information about the request is displayed. A terminal characterized by:
17. A terminal capable of communicating with the information processing device according to any one of claims 1 to 8, A terminal control unit having a function of displaying the video recorded in the extracted video data together with the content of the request corresponding to the extracted video data. A terminal characterized by:
18. An information processing system including an information processing device and a terminal capable of communicating with the information processing device, A function to accept requests from users to potential live streamers; a function of receiving an instruction to start a reply to the request and an instruction to end the reply during the live distribution by the focused user, and displaying one of the requests on the terminal of a viewer of the live distribution when the instruction to start the reply is received; and generating extracted video data in which a video related to the live distribution from the start to the end of the reply is recorded. An information processing system comprising:
19. a control unit of an information processing device receiving, during live streaming by a noted user who may become a distributor of the live streaming, an instruction to start a reply to a request posted to the noted user and an instruction to end the reply, and when the instruction to start the reply is received, displaying one of the requests on a terminal of a viewer of the live streaming; and generating extracted video data in which a video related to the live distribution from the start to the end of the reply is recorded by the control unit of the information processing device.
23. An information processing method comprising:
Citation Information
Patent Citations
Digital content distribution system, method and program
JP2017107305A
Server and program
JP2018023003A
System, method, and program for providing live video distribution service
JP2021081882A
System, method, and program for providing live distribution service
JP2022075401A
Systems and Methods for Digital Video Journaling
US20180048599A1
