System, communication method and program

Abstract

To allow multiple users who want to make sounds such as clapping sounds to share their sounds with each other.SOLUTION: A communication method, by which a first terminal and a second terminal can communicate with each other in real time via a server, includes: transmitting, by the first terminal, first event data instructing generation of a first sound to the server; transmitting, by the second terminal, second event data instructing generation of a second sound to the server; transmitting, by the server, data including the first event data and the second event data to the first terminal; and controlling, by the first terminal, generation of the first sound and the second sound, based on the data including the first event data and the second event data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system, a communication method, and a program for allowing a plurality of users located apart from one another to exchange information via a server. This application claims priority based on Japanese Patent Application No. 2020-171116, filed on October 9, 2020, the disclosure of which is incorporated herein in its entirety. [Background technology]

[0002] In situations where multiple users are enjoying a common activity, such as a play, musical performance, lecture, sporting event, or karaoke, sounds emitted by users, such as applause, are an effective way to liven up the atmosphere. Patent Document 1 discloses a technology for recording applause sounds together with a karaoke song on a recording medium and then playing the applause sounds together with the karaoke song from the recording medium. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 9-26796 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology described in Patent Document 1, the clapping sounds played along with the karaoke song are pre-recorded, fixed sounds, and are not sufficient as a means of livening up the atmosphere. Multiple users enjoying a common object may wish to share the clapping sounds each user makes in real time. However, with respect to objects such as plays, musical performances, lectures, and sports, multiple users who are located far from each other may enjoy viewing and enjoying the common object. In such cases, since the users are far from each other, they cannot share each other's clapping sounds.

[0005] The present invention has been made in view of the above circumstances, and an example of an object of the present invention is to provide a technical means that enables multiple users who wish to make sounds such as clapping to share their sounds with each other. [Means for solving the problem]

[0006] A system according to an embodiment of the present invention includes a first terminal, a second terminal, and a server, wherein the first terminal and the second terminal can communicate with each other in real time via the server, and the first terminal transmits first event data instructing the generation of a first sound to the server. sending the second terminal transmits second event data instructing generation of a second sound to the server, the server transmits data including the first event data and data including the second event data to the first terminal, and the first terminal controls generation of the first sound and the second sound based on the data including the first event data and the data including the second event data. the first event data includes information indicating a type of the first sound, and the second event data includes information indicating a type of the second sound, and controlling the generation of the first sound and the second sound includes controlling a manner in which the first sound is generated in accordance with the type of the first sound, and controlling a manner in which the second sound is generated in accordance with the type of the second sound. . A communication method according to an embodiment of the present invention includes: a first terminal and a second terminal capable of communicating in real time via a server; the first terminal transmitting first event data instructing generation of a first sound to the server; the second terminal transmitting second event data instructing generation of a second sound to the server; the server transmitting data including the first event data and data including the second event data to the first terminal; and the first terminal controlling generation of the first sound and the second sound based on the data including the first event data and the data including the second event data. the first event data includes information indicating a type of the first sound, and the second event data includes information indicating a type of the second sound, and controlling the generation of the first sound and the second sound includes controlling a manner in which the first sound is generated in accordance with the type of the first sound, and controlling a manner in which the second sound is generated in accordance with the type of the second sound. . A program according to an embodiment of the present invention causes a computer functioning as a first terminal in a system in which a first terminal and a second terminal can communicate in real time via a server to transmit first event data instructing the generation of a first sound to the server, receive data from the server including the first event data and second event data instructing the generation of a second sound, which data has been transmitted from the second terminal to the server, and control the generation of the first sound and the second sound based on the data including the first event data and the data including the second event data. the first event data includes information indicating a type of the first sound, and the second event data includes information indicating a type of the second sound, and controlling the generation of the first sound and the second sound includes controlling a manner in which the first sound is generated in accordance with the type of the first sound, and controlling a manner in which the second sound is generated in accordance with the type of the second sound. . [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a communication system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of a terminal according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing a display screen of the terminal shown in FIG. 2. [Figure 4] 3 is a diagram showing first event data generated in the terminal shown in FIG. 2. FIG. [Figure 5] FIG. 2 is a diagram illustrating a configuration of a server according to an embodiment of the present invention. [Figure 6] 5 is a flowchart showing the processing contents of the event reproducing units of the first terminal and the second terminal according to the embodiment of the present invention. [Figure 7] 4 is a time chart showing an example of operation according to an embodiment of the present invention. [Figure 8] 4 is a time chart showing an example of operation according to an embodiment of the present invention. [Figure 9] FIG. 10 is a diagram illustrating an example of operation according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] 1 is a block diagram showing a configuration of a communication system 1 according to an embodiment of the present invention. As shown in FIG. 1, the communication system 1 includes a first terminal 10_1 and a second terminal 10_2 connected to a network 30 such as the Internet, and a server 20.

[0010] In this communication system 1, a server 20 transmits video data (including audio data) of various live broadcast programs such as plays, musical performances, lectures, etc. to a first terminal 10_1 and a second terminal 10_2.

[0011] The first terminal 10_1 receives data from the server 20 and plays back the live broadcast program. During this playback, the first terminal 10_1 generates first event data EV_1 instructing the generation of clapping sounds in response to a user operation, and transmits the first event data EV_1 to the server 20. Similarly, the second terminal 10_2 receives data from the server 20 and plays back the live broadcast program. During this playback, the second terminal 10_2 generates second event data EV_2 instructing the generation of clapping sounds in response to a user operation, and transmits the second event data EV_2 to the server 20.

[0012] The server 20 generates third event data EV_3 including the first event data EV_1 received from the first terminal 10_1 and the second event data EV_2 received from the second terminal 10_2, and transmits this third event data EV_3 to the first terminal 10_1 and the second terminal 10_2.

[0013] Each of the first terminal 10_1 and the second terminal 10_2 generates clapping sounds in accordance with the third event data EV_3 received from the server 20. In this manner, in the communication system 1, the user of the first terminal 10_1 and the user of the second terminal 10_2 can share the clapping sounds that they have each instructed to generate while watching a common live broadcast program.

[0014] The above is an outline of the communication system 1 according to this embodiment. Although only two terminals 10_1 and 10_2 are shown in Fig. 1 to prevent the drawing from becoming complicated, there may be three or more terminals that receive a live broadcast program from the server 20 and transmit event data instructing the generation of applause sounds to the server 20.

[0015] 2 is a block diagram showing an example of the configuration of the first terminal 10_1 in this embodiment. Although not shown, the second terminal 10_2 also has a configuration similar to that of the first terminal 10_1. The first terminal 10_1 is a computer equipped with a communication function, such as a personal computer or a smartphone. As shown in FIG. 2, the first terminal 10_1 has a processor 100, an operation unit 110, a display unit 120, a sound input unit 130, a sound output unit 140, a storage unit 150, and a communication unit 160.

[0016] The processor 100 is a control center that controls each component of the first terminal 10_1. The operation unit 110 is a means for receiving operations to issue various instructions to the processor 100 and is composed of various controls such as a keyboard and a mouse. The display unit 120 is a means for displaying various information and is composed of, for example, a liquid crystal display panel. If the first terminal 10_1 is a terminal such as a smartphone, the operation unit 110 and the display unit 120 may be touch panels that combine their respective functions. The sound input unit 130 is a means for collecting sounds from the outside world and providing them to the processor 100 and is composed of a microphone 130-1. The sound output unit 140 is a means for outputting electrical signals provided by the processor 100 to the outside world as sound and is composed of a speaker 140-1. The storage unit 150 includes a volatile storage unit such as a RAM and a non-volatile storage unit such as a ROM and a hard disk. The volatile storage unit is used as a work area by the processor 100. The non-volatile storage unit stores various programs executed by the processor 100 and various control data used by the processor. The communication unit 160 is a means for controlling communication between the processor 100 and other devices connected to the network 30.

[0017] In this embodiment, when the first terminal 10_1 accesses the server 20 to start receiving video data of a live broadcast program, before starting to receive the video data, the first terminal 10_1 downloads a program playback program 151 from the server 20 and stores it in the volatile storage unit of the storage unit 150. The processor 100 executes this program playback program 151 to function as a program playback unit 101, an event generation unit 102, and an event playback unit 103.

[0018] The program playback unit 101 receives video data of a live broadcast program from the server 20 via the communication unit 160, and generates (plays) the live broadcast program based on the video data. More specifically, the program playback unit 101 displays an image indicated by the received video data on the display unit 120, and emits a sound indicated by the video data from the sound output unit 140.

[0019] During playback of a live broadcast program, the event generation unit 102 generates first event data EV_1 that instructs the generation of clapping sounds in response to an operation on the operation unit 110. In this embodiment, in order to receive various instructions such as an instruction to generate clapping sounds, the screen shown in Fig. 3 is displayed on the display unit 120.

[0020] 3, a program display area 121 displays an image of a live broadcast program being played back by the program playback unit 101. On the display screen of the display unit 120, to the right of the program display area 121, applause buttons 122a, 122b, 122c, and 122d, a first mute button 123a, and a second mute button 123b are displayed.

[0021] The clap buttons 122a, 122b, 122c, and 122d are respectively associated with claps a, b, c, and d, which have different sound waveforms or sound production modes. For example, the clap a may be a clap sound that is generated multiple times at short time intervals. The clap b may be a clap sound that is generated a small number of times at long time intervals. The user can issue an instruction to the event generation unit 102 to produce the desired clap sound by pointing (selecting) one of the clap buttons 122a to 122d with a pointing device such as a mouse.

[0022] 4 illustrates an example of first event data EV_1 generated by the event generating unit 102. As shown in FIG. 4, one piece of first event data EV_1 is text data including occurrence time data D1, sound type data D2, and terminal identification data D3. The first event data EV_1 does not include sound data of clapping sounds. Similarly, the second event data EV_1 and the third event data EV_3 do not include sound data of clapping sounds either.

[0023] The occurrence time data D1 is data indicating the time when a sound generation instruction was made, i.e., the time when one of the clap buttons 122a to 122d was selected. In this embodiment, the occurrence time of a sound generation instruction is the time on the time axis of the program progress provided by the server 20, i.e., the time within the program (the current time based on the start time of the program, the time indicating the elapsed time of the program).

[0024] Various means may be used to determine such in-program time. In a preferred embodiment, the server 20 transmits information indicating the start of the program to the first terminal 10_1 and the second terminal 10_2 at the start timing of live broadcast of the program. Each of the first terminal 10_1 and the second terminal 10_2 initializes its own in-program time by receiving this information. Thereafter, each of the first terminal 10_1 and the second terminal 10_2 updates its own in-program time as time passes.

[0025] In another preferred embodiment, when the server 20 broadcasts (transmits) a live broadcast program, it multiplexes the in-program time onto the video data of the live broadcast program and transmits the same to the first terminal 10_1 and the second terminal 10_2. Each of the first terminal 10_1 and the second terminal 10_2 extracts the in-program time data from the video data of the program currently being played back and uses it to generate the first event data EV_1. In this embodiment, the in-program time is determined in this manner. When any of the applause buttons 122a to 122d is selected, the event generating unit 102 generates occurrence time data D1 indicating the in-program time at that time.

[0026] The sound type data D2 is data indicating the type of clap sound that has been instructed to be produced. When the clap button 122a is selected, the sound type data D2 becomes data indicating clap sound a. When the clap button 122b is selected, the sound type data D2 becomes data indicating clap sound b. When the clap button 122c is selected, the sound type data D2 becomes data indicating clap sound c. When the clap button 122d is selected, the sound type data D2 becomes data indicating clap sound d.

[0027] The terminal identification data D3 is information for identifying a terminal at which a sound generation instruction has occurred. Since the first event data EV_1 is generated by a sound generation instruction to the first terminal 10_1, the terminal identification data D3 thereof is ID1, which is the identification information of the first terminal 10_1. Furthermore, since the second event data EV_2 is generated by a sound generation instruction to the second terminal 10_2, the terminal identification data D3 thereof is ID2, which is the identification information of the second terminal 10_2.

[0028] The first event data EV_1 generated by the event generating unit 102 is transmitted to the server 20 via the communication unit 160 and is also given to the event reproducing unit 103.

[0029] The event reproducing unit 103 has a first function of generating clapping sounds in accordance with the first event data EV_1 generated by the event generating unit 102, and a second function of generating clapping sounds in accordance with the third event data EV_3 received from the server 20 via the communication unit 160. These two functions are controlled based on the states of a first mute flag MUTE1 and a second mute flag MUTE2 set in the volatile storage unit of the storage unit 150.

[0030] The first mute flag (cancel flag, stop flag) MUTE1 is a flag that instructs the event reproducing unit 103 to mute (cancel, stop) (MUTE1=ON) or not (MUTE1=OFF) the generation of the clapping sound based on the first event data EV_1. When one of the clapping buttons 122a to 122d is selected and the first event data EV_1 is generated, it is determined whether or not to mute the generation of the clapping sound based on the first event data EV_1 according to the first mute flag MUTE1.

[0031] The second mute flag (cancel flag, abort flag) MUTE2 is a flag that indicates whether to mute (cancel, abort) (MUTE2=ON) or not (MUTE2=OFF) the generation of the clapping sound based on the third event data EV_3. When the first event data EV_1 generated by the event generating unit 102 is transmitted to the server 20 and then received from the server 20 as the third event data EV_3, it is determined whether to mute or not the generation of the clapping sound based on this third event data EV_3 according to the second mute flag MUTE2.

[0032] 3, the first mute button 123a lights up when the first mute flag MUTE1 is ON and goes out when it is OFF. The second mute button 123b lights up when the second mute flag MUTE2 is ON and goes out when it is OFF. The user can switch the first mute flag MUTE1 ON / OFF by selecting the first mute button 123a with a pointing device. The user can switch the second mute flag MUTE2 ON / OFF by selecting the second mute button 123b with a pointing device.

[0033] In the event reproduction unit 103, the relationship between the first mute flag MUTE1 and the first function is as follows: When the first mute flag MUTE1 is OFF, the event reproduction unit 103 generates a sound signal of the clapping sound specified by the first event data EV_1 at the time of receiving the first event data EV_1 from the event generation unit 102, and causes the sound output unit 140 to emit the clapping sound. When the first mute flag MUTE1 is ON, the event reproduction unit 103 does not emit the clapping sound specified by the first event data EV_1, even if it receives the first event data EV_1 from the event generation unit 102.

[0034] In the event reproducing unit 103, the relationship between the second mute flag MUTE2 and the second function is as follows: When the second mute flag MUTE2 is OFF, when the event reproducing unit 103 receives third event data EV_3 from the server 20, the event reproducing unit 103 causes the sound output unit 140 to emit the clapping sound specified by the third event data EV_3, regardless of whether the third event data EV_3 is the first event data EV_1 generated by the first event generating unit 102.

[0035] When the second mute flag MUTE2 is ON, the event reproducing unit 103, upon receiving the third event data EV_3 from the server 20, determines whether or not the terminal identification data D3 of the third event data EV_3 indicates the identification information ID1 of the first terminal 10_1, that is, whether or not the third event data EV_3 is the first event data EV_1 generated by the first terminal 10_1. If the third event data EV_3 does not correspond to the first event data EV_1, the event reproducing unit 103 causes the clapping sound specified by the third event data EV_3 to be produced. If the third event data EV_3 corresponds to the first event data EV_1, the event reproducing unit 103 does not cause the clapping sound specified by the third event data EV_3 to be produced.

[0036] Next, the timing control of the process by which the event playback unit 103 generates applause sounds based on the third event data EV_3 will be described. The event playback unit 103 controls the timing of generating applause sounds based on the third event data EV_3 based on the current time (in-program time) and the time at which the sound generation instruction indicated by the third event data EV_3 occurred. This is explained in more detail below.

[0037] The event playback unit 103 accumulates the third event data EV_3 received from the server 20 in the storage unit 150, and creates a list by sorting the accumulated third event data EV_3 in the order of occurrence time indicated by the occurrence time data D1. The event playback unit 103 then extracts each third event data EV_3 from the sorted list of the third event data EV_3 in the order of occurrence time, and generates clapping sounds in accordance with the extracted third event data EV_3. To avoid duplication, details of control of the timing of generation of clapping sounds based on this third event data EV_3 will be made clear in the description of the operation of this embodiment.

[0038] 5 is a block diagram showing the configuration of the server 20. The server 20 has a processor 200, an operation unit 210, a display unit 220, a sound input unit 230, a sound output unit 240, a storage unit 250, and a communication unit 260 similar to those of the first terminal 10_1 and the second terminal 10_2.

[0039] A program broadcasting program 251 is stored in the nonvolatile storage unit of the storage unit 250. The processor 200 executes this program broadcasting program 251 to function as a program broadcasting unit 201, an event merging unit 202, and an event reproducing unit 203.

[0040] The program broadcasting section 201 transmits the above-mentioned program reproducing program 151 to the first terminal 10_1 and the second terminal 10_2 that have requested to receive the live broadcast program, and then transmits the video data of the live broadcast program.

[0041] The event merging unit 202 merges the first event data EV_1 and the second event data EV_2 sent from the first terminal 10_1 and the second terminal 10_2 receiving the broadcast of the program, and generates third event data EV_3.

[0042] The event reproducing unit 203 is a means for controlling the generation of clapping sounds based on the third event data EV_3.

[0043] In this embodiment, the applause sound reproduction mode can be set in the server 20 or the set applause sound reproduction mode can be cancelled by operating the operation unit 210 or by an instruction from a terminal connected to the network 30. The applause sound reproduction mode is a mode in which the server 20 generates applause sounds in accordance with the first event data EV_1 and the second event data EV_2 transmitted from the first terminal 10_1 and the second terminal 10_2.

[0044] When this applause sound playback mode is set in the server 20, the event playback unit 203 causes the sound output unit 240 to emit applause sounds in accordance with the third event data EV_3. More specifically, the event playback unit 203 accumulates the third event data EV_3 in the storage unit 250 and sorts the accumulated third event data EV_3 in the order of occurrence times indicated by the occurrence time data D1. The event playback unit 203 then extracts third event data EV_3 from the sorted third event data EV_3 in the order of their occurrence times and generates applause sounds in accordance with the extracted third event data EV_3. More specifically, the sound input unit 230 is composed of multiple microphones, specifically, microphone 230-1 and microphone 230-2. The sound input unit 230 may also be composed of only one microphone, i.e., microphone 230-1. The sound output unit 240 is configured with a plurality of speakers, specifically, speakers 240-1 and 240-2. The sound output unit 240 may be configured with only one speaker, i.e., speaker 240-1. As a first example, a case where one microphone (microphone 230-1) and one speaker (speaker 240-1) are used will be described. In this case, the speaker 240-1 produces clapping sounds for each piece of third event data EV_3. The microphone 230-1 collects the clapping sounds produced by the speaker 240-1, and the sound input unit 230 generates sound data including the clapping sounds. The communication unit 260 transmits the sound data to the first terminal 10_1 and the second terminal 10_2. The first terminal 10_1 and the second terminal 10_2 reproduce the clapping sounds based on the sound data. Here, by generating a clapping sound from the speaker 240-1, a reverberation sound is generated. Therefore, the sound data includes a reverberation sound. Therefore, the first terminal 10_1 and the second terminal 10_2 can reproduce a natural clapping sound. Next, as a second example, a case where one microphone (microphone 230-1) and multiple speakers (speakers 240-1 and 240-2) are used will be described. In this case, the speakers 240-1 and 240-2 each generate a clapping sound according to the third event data EV_3.As another example, speaker 240-1 may emit only clapping sounds according to third event data EV_3 in which terminal identification data D3 indicates identification information ID1, and speaker 240-2 may emit only clapping sounds according to third event data EV_3 in which terminal identification data D3 indicates identification information ID2. Microphone 230-1 collects the clapping sounds emitted by speaker 240-1 and the clapping sounds emitted by speaker 240-2, and sound input unit 230 generates sound data including the clapping sounds. Communication unit 260 transmits the sound data to first terminal 10_1 and second terminal 10_2. First terminal 10_1 and second terminal 10_2 play back the clapping sounds based on the sound data. Here, by generating the clapping sounds from speakers 240-1 and 240-2, reverberation sounds corresponding to the difference in position between speakers 240-1 and 240-2 are generated. Therefore, the first terminal 10_1 and the second terminal 10_2 can reproduce clapping sounds including such reverberation sounds. Thus, users can share more natural clapping sounds. Next, as a third example, a case where multiple microphones (microphones 230-1 and 230-2) and multiple speakers (speakers 240-1 and 240-2) are used will be described. In this case, the sound input unit 230 generates sound data based on the sound picked up by the microphone 230-1 and sound data based on the sound picked up by the microphone 230-2. The communication unit 260 transmits the sound data based on the sound picked up by the microphone 230-1 to the first terminal 10_1. Furthermore, the communication unit 260 transmits the sound data based on the sound picked up by the microphone 230-2 to the second terminal 10_2. Therefore, the first terminal 10_1 and the second terminal 10_2 can respectively reproduce different clapping sounds in accordance with the difference in the positions of the microphones 230-1 and 230-2. In the first to third examples, the microphones 230-1 and / or 230-2 and the speakers 240-1 and / or 240-2 may be installed in locations where environmental noise enters. In this case, the sound data will include the environmental noise. Therefore, the first terminal 10_1 and the second terminal 10_2 can reproduce the environmental noise along with the clapping sounds. Therefore, users can share more natural clapping sounds.

[0045] Next, an example of the operation of this embodiment will be described. FIG. 6 is a flowchart showing the processing contents of the event reproducing unit 103 of the first terminal 10_1 and the second terminal 10_2. The event reproducing unit 103 repeatedly executes the processing shown in FIG. 6 while the program is being reproduced. As described above, the event reproducing unit 103 retrieves the sorted third event data EV_3 from the storage unit 150 in the order of their occurrence times and controls the generation of applause sounds. When the event reproducing unit 103 attempts to retrieve the third event data EV_3 with the earliest occurrence time remaining in the storage unit 150, it determines whether the time indicated by the occurrence time data D1 of that third event data EV_3 is within the allowable range TA of the current program time (step S1). If the result of this determination is negative, the event reproducing unit 103 removes that third event data EV_3 from the list and discards it (step S2). That is, the event reproducing unit 103 does not generate applause sounds based on that third event data EV_3.

[0046] On the other hand, if the result of the above determination is positive, the event reproducing unit 103 determines whether there is any third event data EV_3 that precedes the third event data EV_3 to be extracted and that produces clapping sounds (step S3). If there is no preceding third event data EV_3, the event reproducing unit 103 immediately extracts that third event data EV_3 from the list and produces clapping sounds based on that third event data EV_3 (step S4).

[0047] If there is preceding third event data EV_3, the event reproducing unit 103 calculates the time difference t2-t1 between the occurrence time t1 indicated by the occurrence time data D1 of the preceding third event data EV_3 and the occurrence time t2 indicated by the occurrence time data D1 of the third event data EV_3 to be extracted. Furthermore, the event reproducing unit 103 provisionally sets time t2', which is the time t2-t1 after the occurrence time t1' of the applause sound based on the preceding third event data EV_3, as the occurrence time of the applause sound based on the third event data EV_3 to be extracted (step S5).

[0048] Next, the event playback unit 103 determines whether the occurrence time t2 of the third event data EV_3 to be extracted is within the allowable range TA relative to the provisionally set occurrence time t2' (step S6). That is, the event playback unit 103 determines whether the time difference between the occurrence time t2 of the third event data EV_3 to be extracted and the provisionally set occurrence time t2' is within the allowable range TA. If the result of this determination is affirmative, the event playback unit 103 waits until the in-program time reaches the occurrence time t2', and at the occurrence time t2', extracts the third event data EV_3 from the list and generates applause sounds based on the third event data EV_3 (step S7).

[0049] On the other hand, if the occurrence time t2 of the third event data EV_3 to be extracted is not within the allowable range TA of the provisionally set occurrence time t2', the event replay unit 103 advances the occurrence time t2' so that the occurrence time t2 falls within the allowable range TA of the occurrence time t2' (step S8).The event replay unit 103 then waits until the in-program time reaches this advanced occurrence time t2', extracts the third event data EV_3 from the list at the occurrence time t2', and generates applause sounds based on the third event data EV_3 (step S9).

[0050] 7 and 8 are time charts showing an example of operation of this embodiment. In this example of operation, the transmission delay time from the server 20 to the first terminal 10_1 is longer than the transmission delay time from the server 20 to the second terminal 10_2. Therefore, the first terminal 10_1 starts playing back the live broadcast program later than the second terminal 10_2. Note that in FIGS. 6 and 7, to facilitate understanding of the operation of this embodiment, the transmission delays between the server 20 and each of the first terminal 10_1 and the second terminal 10_2 are exaggerated compared to the time intervals at which the first event data EV_1, the second event data EV_2, and the third event data EV_3 occur.

[0051] 7, in the second terminal 10_2, after the start of program playback, second event data EV_2 occurs at occurrence time (intra-program time) t1, and this second event data EV_2 becomes third event data EV_3 by passing through the server 20 and is transmitted to the first terminal 10_1. Occurrence time data D1 of this third event data EV_3 indicates the occurrence time t1.

[0052] In the first terminal 10_1, this third event data EV_3 is stored in the storage unit 150 at a time after the occurrence time (intra-program time) t1. In this operation example, there is no third event data EV_3 preceding this third event data EV_3. Furthermore, the occurrence time t1 indicated by the occurrence time data D1 of this third event data EV_3 is within the allowable range TA of the current occurrence time (intra-program time) t1'. Therefore, at the occurrence time (intra-program time) t1' immediately after the third event data EV_3 is stored in the storage unit 150, the event reproducing unit 103 retrieves this third event data EV_3 from the storage unit 150 and generates a clapping sound based on the third event data EV_3.

[0053] Thereafter, second event data EV_2 occurs at the second terminal 10_2 at occurrence time (intra-program time) t2, and this second event data EV_2 becomes third event data EV_3 by passing through the server 20 and is transmitted to the first terminal 10_1. Occurrence time data D1 of this third event data EV_3 indicates the occurrence time t2.

[0054] In the first terminal 10_1, this third event data EV_3 is stored in the storage unit 150 at a time after the occurrence time (in-program time) t2. Here, at time t1', a clap sound is generated based on the third event data EV_3 that precedes this third event data EV_3. Then, time t2', which is a time t2-t1 after time t1', is provisionally set as the occurrence time. In this operation example, the occurrence time t2 indicated by the occurrence time data D1 of the third event data EV_3 is within the allowable range TA of the provisionally set occurrence time t2'. Therefore, at the occurrence time t2', the event reproducing unit 103 retrieves this third event data EV_3 from the storage unit 150 and generates a clap sound based on the third event data EV_3.

[0055] 8, in the first terminal 10_1, first event data EV_1 occurs at occurrence time (intra-program time) t3, and this first event data EV_1 becomes third event data EV_3 through the server 20 and is transmitted to the second terminal 10_2. Occurrence time data D1 of this third event data EV_3 indicates the occurrence time (intra-program time) t3. Furthermore, in the first terminal 10_1, first event data EV_1 occurs at occurrence time (intra-program time) t4 after the occurrence time (intra-program time) t3, and this first event data EV_1 also becomes third event data EV_3 and is transmitted to the second terminal 10_2. Occurrence time data D1 of this third event data EV_3 indicates the occurrence time t4.

[0056] In the second terminal 10_2, the third event data EV_3 is accumulated in the storage unit 150 at a time after the occurrence time (in-program time) t3. The processing on this third event data EV_3 is similar to the processing performed on the third event data EV_3 received by the event reproducing unit 103 of the first terminal 10_1 at a time after the time t1 in the example of Fig. 7 .

[0057] In this operation example, the program playback start time of the second terminal 10_2 is earlier than that of the first terminal 10_1. Therefore, the time difference between the occurrence time indicated by the occurrence time data D1 of the third event data EV_3 received by the second terminal 10_2 and the in-program time at which the second terminal 10_2 receives the third event data EV_3 is greater than the time difference between the occurrence time indicated by the occurrence time data D1 of the third event data EV_3 received by the first terminal 10_1 and the in-program time at which the first terminal 10_1 receives the third event data EV_3. For this reason, in the second terminal 10_2, when temporarily setting the occurrence time of the applause sound based on the third event data EV_3, it is more likely that the occurrence time needs to be corrected than in the case of the first terminal 10_1.

[0058] In FIG. 8, when the second terminal 10_2 receives the third event data EV_3 including the occurrence time data D1 indicating the occurrence time t4, the time t4' that is t4 - t3 after the occurrence time t3' of the applause sound based on the preceding third event data EV_3 is temporarily set as the occurrence time of the applause sound based on the third event data EV_3 (occurrence time t4). However, in this operation example, the occurrence time t4 indicated by the third event data EV_3 is not within the allowable range TA with respect to the temporarily set occurrence time t4'. Therefore, in this operation example, the event playback unit 103 corrects the occurrence time t4' to a time earlier than normal, that is, to a time after the occurrence time t4 indicated by the third event data EV_3 by the allowable range (allowable time) TA. In this case, the applause sound based on the third event data EV_3 is pronounced at the occurrence time t4' that satisfies "t4' - t3' < t4 - t3".

[0059] As described above, in this embodiment, in the second terminal 10_2, the applause sound based on the pronunciation instruction is pronounced almost synchronously with the pronunciation instruction issued by the first terminal 10_1. Also, in the first terminal 10_1, the applause sound based on the pronunciation instruction is pronounced almost synchronously with the pronunciation instruction issued by the second terminal 10_2. Therefore, the users of the first terminal 10_1 and the second terminal 10_2 can share the applause sounds for which each has given a pronunciation instruction.

[0060] 7 and 8, the first event data EV_1 transmitted by the first terminal 10_1 passes through the server 20 to become third event data EV_3 and is transmitted to the first terminal 10_1. The second event data EV_2 transmitted by the second terminal 10_2 passes through the server 20 to become third event data EV_3 and is transmitted to the second terminal 10_2. In this way, the processing of the third event data EV_3 received by the first terminal 10_1 and the second terminal 10_2 differs depending on the ON / OFF state of the second mute flag MUTE2.

[0061] When the second mute flag MUTE2 is OFF, the event reproducing unit 103 of the first terminal 10_1 generates clapping sounds based on all the third event data EV_3, regardless of whether the terminal identification data D3 included in the third event data EV_3 indicates the identification information ID1 of the first terminal 10_1. The same applies to the second terminal 10_2.

[0062] When the second mute flag MUTE2 is ON, the event reproducing unit 103 of the first terminal 10_1 determines whether or not the terminal identification data D3 included in the third event data EV_3 indicates the identification information ID1 of the first terminal 10_1. The event reproducing unit 103 of the first terminal 10_1 does not produce clapping sounds for the third event data EV_3 for which the terminal identification data D3 indicates the identification information ID1, and produces clapping sounds only for the third event data EV_3 for which the terminal identification data D3 does not indicate the identification information ID1.

[0063] Fig. 9 is a diagram showing an example of the operation of the first terminal 10_1 in this case. Fig. 9 shows a list of third event data EV_3 received by the first terminal 10_1. When the second mute flag MUTE2 is ON, as shown in Fig. 9, the third event data EV_3 whose terminal identification data D3 indicates the identification information ID1 is excluded from the targets for generating the clapping sound. In the example of Fig. 9, "X" means exclusion.

[0064] The effect of this aspect can be easily understood by considering the following comparative example. In this comparative example, a server receives instructions to emit clapping sounds from multiple terminals, mixes these multiple clapping sounds, and transmits the resulting sound to the multiple terminals. Even in this comparative example, users of multiple terminals can share the clapping sounds emitted in accordance with the pronunciation instructions from each terminal. However, in this comparative example, when each of the multiple terminals emits clapping sounds in response to a pronunciation instruction from a user, it emits the clapping sounds twice, that is, once in response to the pronunciation instruction from the user and once in response to the instruction transmitted from the server. Such clapping sounds give the user a sense of discomfort.

[0065] In contrast, in the present embodiment, when the first mute flag MUTE1 is turned OFF and the second mute flag MUTE2 is turned ON in the first terminal 10_1, if a sound generation instruction is given to the first terminal 10_1, the first terminal 10_1 only generates claps based on this sound generation instruction at the time of the sound generation instruction, and even if the first event data EV_1 based on this sound generation instruction is returned to the first terminal 10_1 as third event data EV_3 via the server 20, the first terminal 10_1 does not generate claps based on this third event data EV_3. Therefore, according to the present embodiment, it is possible to avoid the situation in which the claps are generated twice, which occurs in the comparative example.

[0066] As described above, according to the present embodiment, the first terminal 10_1 transmits the first event data EV_1 instructing the generation of clapping sounds to the server 20. The second terminal 10_2 transmits the second event data EV_2 instructing the generation of clapping sounds to the server 20. The server 20 transmits the third event data EV_3 including the first event data EV_1 and the second event data EV_2 to the first terminal 10_1. The first terminal 10_1 generates clapping sounds in accordance with the third event data EV_3. This allows the user of the first terminal 10_1 to share the clapping sounds with the user of the second terminal 10_2. Furthermore, in the present embodiment, the server 20 transmits the third event data EV_3, which has a small amount of data, to the first terminal 10_1, and therefore the delay in data transmission from the server 20 to the first terminal 10_1 is shorter than when sound data of clapping sounds is transmitted. Therefore, the first terminal 10_1 can generate clapping sounds at an appropriate timing with little delay from the time when the sound generation instruction is issued at the second terminal 10_2.

[0067] Furthermore, in the above embodiment, the server 20 broadcasts (transmits) a program to the first terminal 10_1 and the second terminal 10_2. The first event data EV_1, the second event data EV_2, and the third event data EV_3 include occurrence time data D1 indicating the occurrence time of a sound generation instruction on the time axis along which the program progresses. The event reproducing unit 103 of the first terminal 10_1 controls the timing of generation of clapping sounds based on the third event data EV_3, based on the current time and the occurrence time of the sound generation instruction indicated by the occurrence time data D1 of the third event data EV_3. Therefore, according to the present embodiment, the event reproducing unit 103 of the first terminal 10_1 can generate clapping sounds based on the sound generation instruction in approximately synchronization with the generation of the sound generation instruction in the second terminal 10_2.

[0068] Furthermore, in the present embodiment, the event reproducing unit 103 of the first terminal 10_1 controls the sound generation process based on the event data corresponding to the first event data EV_1 transmitted from the first terminal 10_1 among the third event data EV_3, in accordance with an operation on the first terminal 10_1. Specifically, when the second mute flag MUTE2 is set to ON by an operation of the operation unit 110, the event reproducing unit 103 of the first terminal 10_1 excludes the event data corresponding to the first event data EV_1 transmitted from the first terminal 10_1 among the third event data EV_3 from the sound generation targets. Therefore, according to the present embodiment, it is possible to prevent the first terminal 10_1 from generating clapping sounds twice based on the same sound generation instruction.

[0069] Furthermore, in this embodiment, when the applause sound reproduction mode is set, the event reproduction unit 203 of the server 20 generates applause sounds in accordance with the third event data EV_3 generated by the event merging unit 202. Therefore, the user of the server 20 can share the applause sounds with each user of the first terminal 10_1 and the second terminal 10_2.

[0070] Although the embodiments of the present invention have been described above, the present invention is not limited to these. Another embodiment of the present invention will now be described.

[0071] (1) A terminal may be provided with a means for preventing the terminal from emitting clapping sounds that are undesirable to the user. For example, when the server 20 broadcasts a lecture program to each terminal, if a user emits clapping sounds at a time when many users are not clapping, such as during the period when the lecturer is speaking, the terminal will not emit clapping sounds for that user. This requires a means for identifying the terminal of the user emitting undesirable clapping sounds. The following aspects may be adopted for this means.

[0072] <Aspect 1> The event reproducing unit 103 displays a dot associated with each terminal on the display screen of the display unit 120. When generating a clapping sound in accordance with the third event data EV_3, the event reproducing unit 103 lights up a dot associated with the terminal identified by the terminal identification data D3 of the third event data EV_3. The user uses the pointing device to point out the dot that lights up at an undesirable timing. Thereafter, the event reproducing unit 103 excludes the third event data EV_3 including the terminal identification data D3 of the terminal corresponding to the pointed dot from the targets for generating the clapping sound.

[0073] <Aspect 2> The event playback unit 103 analyzes the timing of clapping sounds based on the third event data EV_3 for a certain period of time. Many clapping sounds occur close together at roughly the same time. On the other hand, there may be a minority of clapping sounds (less than a predetermined number of clapping instructions) that occur at times distant from the occurrence of the majority of clapping sounds. The device that generated the clapping instructions for such a minority of clapping sounds is identified. The third event data EV_3 including the terminal identification data D3 of that terminal is excluded from the targets for clapping sounds.

[0074] (2) In the above embodiment, the server 20 provides live broadcast programs to the first terminal 10_1 and the second terminal 10_2, but the present invention is not limited to such an embodiment. Alternatively, a recorded program may be provided in response to a request from the terminal.

[0075] In this embodiment, when recording a live broadcast program, the server 20 stores third event data EV_3, which is a combination of the first event data EV_1 received from the first terminal 10_1 and the second event data EV_2 received from the second terminal 10_2. Then, when the server 20 receives a recording / playback request from a terminal, the server 20 generates new third event data EV_3' by combining the event data instructing the generation of clapping sounds received from the terminal with the stored third event data EV_3, and transmits the new third event data EV_3' to the terminal. According to this embodiment, the user who requested recording / playback can share the clapping sounds with users who watched the live broadcast program.

[0076] (3) In the above embodiment, multiple users share the sound of applause, but the sound to be shared is not limited to the sound of applause. For example, when multiple users are watching a play such as kabuki, they may share the sound of shouts for the actors.

[0077] (4) In each of the first terminal 10_1 and the second terminal 10_2, the manner of sound generation processing based on the third event data EV_3, such as volume balance, positioning, tone, and duration, may be controlled by operating the operation unit 110 for each type of sound, i.e., applause sounds a, b, c, and d.

[0078] (5) In the above embodiment, the event reproducing unit 103 of the first terminal 10_1 excludes the event data corresponding to the first event data EV_1 transmitted from the first terminal 10_1 from the third event data EV_3 from the sound to be generated. However, the manner of controlling the sound to be generated based on the event data corresponding to the first event data EV_1 is not limited to this. For example, the volume of the clapping sound to be generated based on the event data corresponding to the first event data EV_1 may be reduced.

[0079] (6) The server 20 may transmit data including sound data (sound signal) of clapping sounds as the third event data EV_3 to the first terminal 10_1 and the second terminal 10_2. In this case, the first terminal 10_1 and the second terminal 10_2 generate clapping sounds based on the sound data of the clapping sounds. [Industrial Applicability]

[0080] The present invention may be applied to a communication method. [Explanation of symbols]

[0081] 10_1... Terminal 1 10_2... Terminal 2 20...Server 30...Network 100,200...processor 110,210……Operation unit 120,220……Display section 130,230...Sound input section 140,240...Sound output section 150,250...Storage section 160,260……Communications Department 151...Program playback program 101...Program playback section 102...Event generation unit 103,203...Event Playback Department 201...Broadcasting Department 202...Event Merging Department 121……Program display area 122a~122d... Applause button 123a……1st mute button 123b...Second mute button

Claims

1. A system comprising a first terminal, a second terminal, and a server, the first terminal and the second terminal are capable of communicating in real time via the server, the first terminal transmits first event data instructing the generation of a first sound to the server; the second terminal transmits second event data instructing the generation of a second sound to the server; the server transmits data including the first event data and data including the second event data to the first terminal; the first terminal controls generation of the first sound and the second sound based on data including the first event data and data including the second event data; the first event data includes information indicating a type of the first sound, and the second event data includes information indicating a type of the second sound; Controlling the generation of the first sound and the second sound includes: controlling a manner in which the first sound is generated in accordance with the type of the first sound; and controlling a manner in which the second sound is generated in accordance with the type of the second sound. system.

2. the first terminal and the second terminal are capable of playing common video data; The first terminal controls generation of the first sound and the second sound while the video data is being played back. The system of claim 1 .

3. the server transmits a program to the first terminal and the second terminal; the first event data includes information indicating a first time, which is an elapsed time of the program, as information indicating a time at which the first sound is to be generated; and the second event data includes information indicating a second time, which is an elapsed time of the program, as information indicating a time at which the second sound is to be generated; controlling the generation of the first sound and the second sound includes controlling timings of generating the first sound and the second sound based on a current time of the program being played back on the first terminal, the first time, and the second time.

3. The system according to claim 1 or claim 2.

4. Controlling the generation of the first sound and the second sound includes controlling the generation of the first sound in accordance with an operation on the first terminal. A system according to any one of claims 1 to 3.

5. transmitting the data including the first event data and the data including the second event data to the first terminal includes transmitting audio data of the first sound and audio data of the second sound from the server to the first terminal; Controlling the generation of the first sound and the second sound includes: generating the first sound based on audio data of the first sound; generating the second sound based on audio data of the second sound. A system according to any one of claims 1 to 4.

6. Controlling the generation of the first sound and the second sound includes controlling the manner in which the first sound and the second sound are generated in accordance with an operation on the first terminal. A system according to any one of claims 1 to 5.

7. A plurality of the first terminals are provided, The server analyzes, for a certain period of time, the timing of sounding of the first sound based on third event data, which is data including the first event data and data including the second event data, and excludes from the third event data the first event data corresponding to the first terminals whose sounding timings are separated by a certain amount or more and are less than a predetermined number of the first terminals. The system of claim 1 .

8. The server, storing third event data, which is data including the first event data and data including the second event data, in a storage unit; When a request for recording and playback is received, the event data instructing the generation of a sound received from the terminal that transmitted the request for recording and playback is merged with the third event data stored in the storage unit to generate new third event data, and the new third event data is transmitted to the terminal that transmitted the request for recording and playback. The system of claim 1 .

9. Each of the first terminal and the second terminal In response to the operation of the operation unit, at least one of volume balance, localization, timbre, and duration is controlled for each type of sound as a sound generation processing mode based on third event data, which is data including the first event data and the second event data. The system of claim 1 .

10. The first terminal and the second terminal are capable of communicating in real time via a server, transmitting, by the first terminal, first event data instructing the generation of a first sound to the server; transmitting second event data instructing the second terminal to generate a second sound to the server; transmitting data including the first event data and data including the second event data to the first terminal by the server; controlling, by the first terminal, generation of the first sound and the second sound based on data including the first event data and data including the second event data; This includes: the first event data includes information indicating a type of the first sound, and the second event data includes information indicating a type of the second sound; Controlling the generation of the first sound and the second sound includes: controlling a manner in which the first sound is generated in accordance with the type of the first sound; and controlling a manner in which the second sound is generated in accordance with the type of the second sound. Communication method.

11. A computer that functions as a first terminal in a system in which a first terminal and a second terminal can communicate in real time via a server, transmitting first event data instructing the generation of a first sound to the server; receiving, from the server, data including the first event data and data including second event data instructing generation of a second sound, the second event data having been transmitted from the second terminal to the server; Controlling generation of the first sound and the second sound based on data including the first event data and data including the second event data. Let it be done, the first event data includes information indicating a type of the first sound, and the second event data includes information indicating a type of the second sound; Controlling the generation of the first sound and the second sound includes: controlling a manner in which the first sound is generated in accordance with the type of the first sound; and controlling a manner in which the second sound is generated in accordance with the type of the second sound. program.

Images