COMMUNICATION METHOD, PROGRAM, AND COMMUNICATION SYSTEM

Abstract

To acquire, at a second communication site, sound data acquired sequentially at a first communication site and to recognize timing when the sound data is acquired at the first communication site even at the second communication site.SOLUTION: A program causes a computer to control notification processing performed by a notification device on the basis of timing information which is repetitively generated at a prescribed interval and to transmit the notification data and sound data acquired sequentially as packet data. The packet data includes notification data based on the presence or absence of timing information in a period when the sound data included in the packet data is acquired.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for controlling notification processing in a remote location. [Background technology]

[0002] When multiple users located far from each other play musical instruments together, it is important to synchronize the timing of the performance. For example, Patent Document 1 discloses a technique for synchronizing multiple metronomes located far from each other. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2001-154672 A Summary of the Invention [Problem to be solved by the invention]

[0004] One object of the present invention is to enable sound data sequentially acquired at a first communication point to be acquired at a second communication point, and also to enable the second communication point to recognize the timing at which the sound data was acquired at the first communication point. [Means for solving the problem]

[0005] According to one embodiment of the present invention, there is provided a communication method including controlling an alarm process executed by an alarm device based on timing information repeatedly generated at a predetermined interval, and transmitting alarm data and sequentially acquired sound data as packet data, the packet data including the alarm data based on the presence or absence of the timing information during a period in which the sound data included in the packet data was acquired.

[0006] According to one embodiment of the present invention, there is provided a communication method including receiving packet data including notification data and sound data based on the presence or absence of timing information that is repeatedly generated at predetermined intervals, playing the sound data included in the packet data, and controlling notification processing executed by an alarm device based on the notification data included in the packet data together with the sound data during a period in which the sound data is played.

[0007] According to an embodiment of the present invention, a program for causing a computer to execute the above-mentioned communication method may be provided, or a communication device for executing the above-mentioned communication method may be provided, or a communication system for executing both of the communication methods may be provided. Effect of the Invention

[0008] According to the present invention, sound data sequentially acquired at a first communication point can be acquired at a second communication point, and the timing at which the sound data was acquired at the first communication point can also be recognized at the second communication point. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a communication system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a data communication function according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a structure of packet data in one embodiment. [Figure 4] 1 is a flowchart illustrating a data transmission method according to an embodiment. [Diagram 5] 1 is a flowchart illustrating a data receiving method according to an embodiment. [Figure 6] 11 is a flowchart illustrating an interpolation process according to an embodiment. [Figure 7] FIG. 13 is a diagram illustrating a structure of packet data in a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A communication system according to an embodiment of the present invention will be described in detail below with reference to the drawings. The embodiment described below is an example of an embodiment of the present invention, and the present invention is not limited to these embodiments. In the drawings referred to in this embodiment, the same parts or parts having similar functions are given the same or similar symbols (symbols with A, B, etc. added after the numbers), and repeated explanations may be omitted.

[0011] [1. Communication system configuration] FIG. 1 is a diagram for explaining the configuration of a communication system in one embodiment. The communication system includes a server 1 connected to a network NW such as the Internet. The server 1 includes a control unit such as a CPU, a storage unit, and a communication unit, and the control unit executes a predetermined program to provide a service for realizing an ensemble between communication bases. The server 1 controls communication between a plurality of communication bases connected to the network NW, and executes processing required for a communication terminal 20 (communication device) at each communication base to realize P2P type communication. This processing may be realized by a known method. In FIG. 1, two communication bases T1 and T2 are illustrated as an example, but the number is not limited to this number, and more communication bases may exist. In the following description, when the communication bases T1 and T2 are not distinguished from each other, they are simply referred to as communication bases.

[0012] In this example, sounds corresponding to the performance (hereinafter referred to as performance sounds) between communication bases are transmitted and received by P2P type communication, allowing for an ensemble. According to the data communication function described below, by setting a metronome sound at a main communication base, the other communication bases can also receive the sound while maintaining the relationship between the performance sound at the main communication base and the metronome sound. The one main communication base and the other communication bases are set via server 1 based on information (user instructions, etc.) transmitted from the communication base. In the following explanation, it is assumed that communication base T1 is set as the main communication base.

[0013] A communication terminal 20 is disposed at each communication base. An electronic musical instrument 30, a sound collection device 40, an imaging device 50, a sound emission device 60, and a display device 70 are connected to the communication terminal 20. Although a communication terminal 20 is always present at each communication base, at least one of the electronic musical instrument 30, the sound collection device 40, the imaging device 50, the sound emission device 60, and the display device 70 does not have to be connected to the communication terminal 20. At least one of the electronic musical instrument 30, the sound collection device 40, the imaging device 50, the sound emission device 60, and the display device 70 may be configured as an integrated device with the communication terminal 20.

[0014] The electronic musical instrument 30 includes performance controls and a sound source that outputs sound data in response to operations on the performance controls. In this example, the electronic musical instrument 30 is an electronic piano having keys as performance controls. The sound data is output to the communication terminal 20. The sound data is data representing a sound waveform signal, and may be output to a sound emission device 60.

[0015] The sound collection device 40 has, for example, a microphone or an input terminal for a sound waveform signal, and outputs the sound input to the microphone or the sound waveform signal input to the input terminal as sound data to the communication terminal 20.

[0016] Imaging device 50 has, for example, a camera, and outputs video data corresponding to an image captured by the camera to communication terminal 20. In the following description, an image includes both a still image and a video image.

[0017] The sound emitting device 60 has, for example, a speaker, and outputs from the speaker a sound indicated by the sound data supplied from the communication terminal 20. The sound data supplied to the sound emitting device 60 may be sound data transmitted from another communication base, or may be sound data generated at the sound emitting device 60's own communication base.

[0018] The display device 70 has, for example, a liquid crystal display, and displays on the liquid crystal display an image indicated by the video data supplied from the communication terminal 20. The video data supplied to the display device 70 may be video data transmitted from another communication base, or may be video data generated at the display device 70's own communication base. The display device 70 may be a light-emitting unit including a light-emitting element such as an LED.

[0019] [2. Configuration of communication terminal] The communication terminal 20 includes a control unit 21, a storage unit 23, a communication unit 25, and an operation unit 27. The control unit 21 includes a CPU, a RAM, a ROM, etc. The control unit 21 executes a program stored in the storage unit 23 by the CPU, thereby performing processing according to instructions defined in the program. The program includes a program for performing processing to realize a data communication function. The data communication function includes a function to realize a data transmission method and a data reception method, which will be described later.

[0020] The storage unit 23 includes a storage device such as a non-volatile memory, and stores a program executed by the control unit 21. In addition, various data used in the communication terminal 20 is stored. This program may be provided to the communication terminal 20 in a state stored in a computer-readable recording medium such as a magnetic recording medium, an optical recording medium, a magneto-optical recording medium, or a semiconductor memory, as long as it is executable by a computer. In this case, the communication terminal 20 may be provided with a device for reading the recording medium. In addition, this program may be provided by downloading it via the communication unit 25.

[0021] The communication unit 25 includes a communication module, and is connected to the network NW to transmit and receive various data to and from external devices such as the server 1 and the communication terminals 20 at other communication points. Data transmitted between the communication terminals 20 at each communication point is transmitted as streaming data.

[0022] The operation unit 27 includes operation devices such as a mouse and a keyboard, receives user operations on the operation devices (hereinafter, may be simply referred to as user instructions), and outputs to the control unit 21 a signal according to the operation.

[0023] [3. Data communication function] Next, the data communication function will be described.

[0024] 2 is a diagram for explaining a data communication function in one embodiment. The data communication function 100 includes a data transmission unit 200, a data reception unit 300, and a notification control unit 500. The data transmission unit 200 executes a data transmission method described later. The data reception unit 300 executes a data reception method described later.

[0025] The data transmission unit 200 includes an acquisition unit 210 , a timing generation unit 220 , a packet generation unit 230 and a transmission unit 250 .

[0026] The acquisition unit 210, for example, sequentially acquires sound data from a device connected to the communication terminal 20. In this example, the acquisition unit 210 sequentially acquires sound data corresponding to sounds (hereinafter, referred to as performance sounds) generated by playing the electronic musical instrument 30. The acquisition unit 210 buffers a predetermined amount of sound data and then supplies it to the packet generation unit 230.

[0027] The timing generation unit 220 generates timing information based on a user instruction and supplies it to the packet generation unit 230 and the notification control unit 500. The timing information is repeatedly generated at a predetermined interval. This interval and the timing of starting generation are set by a user instruction, and in this example, correspond to the tempo for sounding a metronome sound and the start timing of the metronome sound. For example, when the tempo is set to "120", the timing generation unit 220 generates timing information every 0.5 seconds from the start timing.

[0028] The processing in the timing generation unit 220 is executed in the communication terminal 20 at the communication point T1 (main communication point) which is the source of the metronome sound, but is not executed in the communication terminal 20 at the communication point T2 which is not the source of the metronome sound.

[0029] The packet generator 230 packetizes the sound data acquired by the acquisition unit 210 to generate packet data. When packetizing the sound data, the packet generator 230 further packetizes the broadcast data using the timing information generated by the timing generator 220 to generate packet data. In this example, the communication protocol of this packet data is RTP (Real-time Transport Protocol). The structure of the packet data will be described.

[0030] Fig. 3 is a diagram for explaining the structure of packet data in one embodiment. As shown in Fig. 3, the payload PL following the RTP header RH includes sound data D1 arranged at a first data position and notification data D2 arranged at a second data position. The first data position is predetermined according to the number of samples of the sound data (for example, 64 samples).

[0031] The notification data D2 includes data corresponding to the timing information. In this example, the notification data D2 indicates whether or not timing information was generated by the packet generating unit 230 during the period when the sound data D1 included in the first data position was acquired by the acquiring unit 210. In other words, the notification data D2 includes data indicating the presence or absence of timing information during the period when the sound data D1 included in the first data position was acquired. In this example, the notification data D2 is "1" when there is timing information (when timing information is generated), and is "0" when there is no timing information (when timing information is not generated). Therefore, in packet data including a performance sound when a metronome sound is sounded, the notification data D2 is "1".

[0032] As described above, the sound data acquired by the acquisition unit 210 is buffered before being supplied to the packet generation unit 230. Therefore, the packet generation unit 230 sets the notification data to "1" when packetizing the sound data supplied from the acquisition unit 210 a time period corresponding to the amount of buffered sound data after the timing at which the timing information was generated. Adjustment of the time corresponding to the amount of buffered sound data does not have to be performed.

[0033] As described above, the communication terminal 20 at the communication point T2 does not execute the process in the timing generation unit 220. Therefore, timing information is not generated, and the broadcast data included in the packet data transmitted from the communication terminal 20 is "0." The packet data generated in the communication terminal 20 at the communication point T2 does not need to include the broadcast data.

[0034] 2, the description will continue. The transmitter 250 transmits the packet data generated by the packet generator 230. The destination is a communication terminal 20 at another communication point.

[0035] When the timing information is generated by the timing generating section 220, the notification control section 500 generates sound data indicative of a metronome sound and supplies it to the sound emitting device 60. As a result, the metronome sound is output from the sound emitting device 60. The outputted metronome sound is not included in the sound data output from the electronic musical instrument 30. In this way, the notification control section 500 controls the metronome sound generation process (notification process) executed by the sound emitting device 60 (notification device).

[0036] Therefore, when the metronome sound is being played when the acquisition unit 210 acquires sound data by performing operations on the electronic musical instrument 30, the packet data containing the sound data includes control data indicating that timing information has been generated.

[0037] The data receiving unit 300 includes a receiving unit 310 , a restoring unit 330 and a reproducing unit 350 .

[0038] The receiving unit 310 receives packet data transmitted from the communication terminal 20 at another communication point.

[0039] The restoration unit 330 restores the sound data and the notification data from the packet data received by the receiving unit 310. The restoration unit 330 supplies the restored sound data to the reproduction unit 350, generates timing information based on the notification data, and supplies it to the notification control unit 500. The restoration unit 330 supplies the timing information to the notification control unit 500 when the restoration unit 330 supplies the sound data included in the packet data in which the notification data is "1" to the reproduction unit 350.

[0040] As described above, the communication terminal 20 at the communication point T2 does not transmit packet data in which the notification data is "1" because the processing of the timing generation unit 220 is not executed. Therefore, in the communication terminal 20 at the communication point T1, the notification data restored by the restoration unit 330 does not become "1". That is, in the communication terminal 20 at the communication point T1, timing information to the notification control unit 500 is supplied from the timing generation unit 220 but not from the restoration unit 330. On the other hand, in the communication terminal 20 at the communication point T2, timing information to the notification control unit 500 is supplied from the restoration unit 330 but not from the timing generation unit 220.

[0041] The reproduction unit 350 buffers and reproduces the sound data restored by the restoration unit 330. The reproduced sound data is supplied to the sound emitting device 60. The sound emitting device 60 outputs a sound corresponding to the supplied sound data (hereinafter referred to as restored sound). As described above, the sound emitting device 60 is also supplied with sound data indicating a metronome sound from the notification control unit 500. Therefore, the sound emitting device 60 outputs the restored sound and the metronome sound. By transmitting packet data by the above-mentioned method, the temporal relationship between the restored sound and the metronome sound becomes the same as the temporal relationship between the performance sound and the metronome sound described above.

[0042] [4. Data transmission method] Next, a data transmission method executed by the data transmission unit 200 (control unit 21) of the data communication function 100 will be described. Each process of the data transmission method described below is started, for example, by a user instruction input to the operation unit 27. At this time, a main communication base (communication base T1) and another communication base (communication base T2) are set in advance as the source of the metronome sound. At communication base T1, a metronome sound is generated according to timing information in response to a user instruction.

[0043] 4 is a flowchart for explaining a data transmission method in one embodiment. When the control unit 21 generates timing information (step S110; Yes), it controls the notification process based on the timing information (step S112) and sets the notification data to "1" (step S114). Through this control, a metronome sound is generated according to the timing information, and the metronome sound is output from the sound emission device 60. When the control unit 21 does not generate timing information (step S110; No), it sets the notification data to "0" (step S116).

[0044] The control unit 21 acquires sound data output from the electronic musical instrument 30 when timing information is generated to generate a metronome sound (step S130). The control unit 21 associates the acquired sound data with notification data, packetizes the data, and generates packet data, which it then transmits to another communication point T2 (step S150). As a result, the notification data contains information based on whether or not timing information was generated during the period in which the sound data was acquired. That is, if timing information was generated during the period in which the sound data to be packetized was acquired, the notification data corresponding to the sound data will be "1," and if timing information was not generated during that period, the notification data corresponding to the sound data will be "0."

[0045] When the control unit 21 detects that an instruction to end the process has been input from the user (step S200; Yes), it ends the process of the data transmission method, and when the control unit 21 does not detect the instruction (step S200; No), it returns to step S110 and continues the process. This concludes the description of the data transmission method.

[0046] [5. How to receive data] Next, a description will be given of a data transmission method executed by the data receiving unit 300 (control unit 21) of the data communication function 100. Each process of the data transmission method described below is started by a user instruction input to the operation unit 27, for example.

[0047] 5 is a flowchart for explaining a data receiving method in one embodiment. While not receiving packet data (step S600; No), the control unit 21 executes an interpolation process (step S900) and waits for reception of packet data. The interpolation process will be described later. When the control unit 21 receives packet data (step S600; Yes), it restores sound data and notification data from the packet data (step S630) and buffers the restored sound data (step S650). The sound data reproduced by being buffered is supplied to the sound emitting device 60 and output from the sound emitting device 60 as sound.

[0048] If the restored notification data is "1" (step S700; Yes), the control unit 21 generates timing information and controls the notification process (step S750). Through this control, a metronome sound is generated according to the timing information, and the metronome sound is output from the sound emitting device 60. The control unit 21 controls the timing at which the metronome sound is output from the sound emitting device 60 to be delayed according to the buffer time of the sound data. As a result, the metronome sound is output when the restored sound is output from the sound emitting device 60 by the sound data corresponding to the notification data that is "1".

[0049] If the restored notification data is "0" (step S700; No), the control unit 21 does not generate timing information. If the control unit 21 detects that an instruction to end the process has been input by the user (step S800; Yes), it ends the process of the data transmission method, and if the instruction has not been detected (step S800; No), it returns to step S600 and continues the process. Next, the interpolation process (step S900) that is executed when no packet data is received in step S600 (step S600; No) will be described.

[0050] FIG. 6 is a flow chart for explaining the interpolation process in one embodiment. If the current time has not passed the prediction period (step S910; No), the control unit 21 ends the interpolation process and returns to step S600 to continue the process. The prediction period is a period during which timing information is predicted to be generated, and is determined based on the control history of past notification processes. The control history is, for example, the timing at which timing information was generated multiple times in the past. The control unit 21 predicts the next timing information generation timing from, for example, the average time of each interval (three periods) corresponding to the generation timing of four timing information. The prediction period is set as a period with a predetermined time width added thereto around the prediction timing.

[0051] If the current time has passed the predicted period (step S910; Yes), the control unit 21 generates timing information (step S950) and ends the interpolation process. This allows the communication terminal 20 at the communication point T2 to output the metronome sound from the sound emitting device 60 even if it does not receive packet data with the notification data being "1" due to packet loss or the like. Although the sound data corresponding to this metronome sound could not be restored due to packet loss, if there is sound data before and after it, the restored sound is hardly interrupted. In such a case, the metronome sound is generated without any gaps, so that the user does not feel uncomfortable. This concludes the explanation of the data transmission method.

[0052] According to the conventional technology, performance sounds sequentially acquired at a certain point (e.g., a first communication point) are transmitted as sound data in streaming format via a network to another communication point (e.g., a second communication point). Therefore, due to the influence of network delays, the timing at which the performance sounds are transmitted and then generated at the second communication point is delayed from the timing at which the performance sounds are generated at the first communication point. When the metronome sounds at all points simultaneously, a discrepancy occurs between the relationship between the performance sounds and the metronome sounds at the first communication point and the relationship between the performance sounds and the metronome sounds at the second communication point. This gives a sense of discomfort to a user at the second point when the user tries to play along with the performance sounds at the first communication point. Therefore, in an ensemble where musical instruments are played simultaneously at different points, it is not necessarily desirable to have the metronome sound at all communication points simultaneously.

[0053] On the other hand, it is possible to include not only the performance sound but also the metronome sound in the sound data transmitted from the first communication point. In this case, the performance sound and the metronome sound are mixed at the first communication point to generate sound data, which is then transmitted to the second communication point. Therefore, at the second communication point receiving this sound data, both the performance sound and the metronome sound that was sounding during the performance can be generated while maintaining the relative relationship between them. However, this sound data results in data in which the two sounds cannot be separated, which causes inconveniences such as the second communication point being unable to record only the performance sound at the first communication point.

[0054] In this way, according to the data communication function in one embodiment, while maintaining the relationship between the performance sound and the metronome sound at the communication point T1, the restored sound corresponding to the performance sound and the metronome sound can be output from the sound emitting device 60 at another communication point (communication point T2). Therefore, the sound data sequentially acquired at the first communication point T1 can be acquired at the second communication point T2, and the timing at which the sound data was acquired at the first communication point can also be recognized at the second communication point.

[0055] <Modification> Although one embodiment of the present invention has been described above, this embodiment can be modified in various ways as described below. Furthermore, the above-described embodiment and the modifications described below can be applied in combination with each other.

[0056] (1) In the above-described embodiment, the packet data is generated so that the sound data D1 and the notification data D2 are located at different data positions. However, the packet data may be generated in a different manner.

[0057] FIG. 7 is a diagram for explaining the structure of packet data in the modified example. For example, packet data including sound data D1 and notification data D2 may be generated by superimposing the notification data D2 on the performance sound (sound data D1) as data converted into a sound waveform in an inaudible frequency band. In this case, the packet generating unit 230 may packetize data in which the sound data D1 of the performance sound in the audible frequency band and the notification data D2 in the inaudible frequency band are superimposed, and place the data in the payload PL. The restoring unit 330 may separate the transmitted data into the sound data D1 and the notification data D2 by dividing the data into the audible frequency band and the inaudible frequency band. A known method may be used as a specific processing method for realizing the technique of superimposing information on the inaudible frequency band, and is disclosed in, for example, Japanese Patent Application Laid-Open No. 2010-55077.

[0058] (2) The information notified to the user based on the timing information is not limited to information that is recognized by hearing, such as a metronome sound, but may also be information that is recognized by sight. The information that is recognized by sight may be notified to the user by the display device 70, and may be, for example, a predetermined light emitted by a light-emitting element such as an LED, or a predetermined video played on the display. Since playing a video requires a longer processing time than playing a sound, it is preferable that the notification control unit 500 outputs video data (control data) for controlling the video to be displayed to the display device 70 before the period in which the sound data is played by the playback unit 350.

[0059] (3) The timing information may include beat information. The beat information is information indicating which beat the metronome sound to be sounded corresponds to. In this case, the beat is further set by a user instruction. This information does not have to indicate all beats, and may be information specifying only the first beat, for example.

[0060] The notification control unit 500 may generate metronome sounds with different sounds depending on the beats based on the beat information. For example, it is sufficient that the metronome sound generated for only the first beat is different from the other beats. The different sounds may be, for example, different loudnesses or different timbres. When visually recognized information is used as in the above-mentioned modification example (2), the colors or brightness may be different depending on the beats. The notification data D2 included in the packet data may indicate information indicating whether or not timing information has been generated by using the beat information included in the generated timing information.

[0061] (4) The notification data D2 may indicate the presence or absence of generation of timing information by other information. The generation of timing information (the presence of timing information) may be indicated, for example, by information specifying a more detailed data position of the sound data D1 in the packet data. When the number of samples of the sound data D1 becomes large and the duration of the performance sound included in the sound data D1 becomes long, it becomes necessary to specify the timing at which the metronome sound was sounded within the period of the performance sound depending on the interval of the metronome sound. In such a case, it is preferable to include information specifying a more detailed data position of the sound data D1 in the notification data D2 in order to specify the timing. The data position may be described as the number of samples counted from the first sample of the sound data D1.

[0062] If the period of the performance sound becomes longer and multiple metronome sounds are sounded, the notification data D2 may include information indicating the number of metronome sounds sounded during the period of the performance sound in the sound data D1 (the number of times timing information was generated) and information specifying the data position of each metronome sound. For example, if two metronome sounds are sounded during the period of the performance sound in the sound data D1, the notification data D2 includes information that the metronome sound is sounded twice, and the data positions of the first and second sounds.

[0063] (5) The broadcast data D2 may be arranged in the header instead of in the payload PL of the packet data.

[0064] (6) In the data receiving method described with reference to FIGS. 5 and 6, the interpolation process does not have to be performed.

[0065] The above is a description of the modified example.

[0066] As described above, according to one embodiment of the present invention, there is provided a communication method including controlling an alarm process executed by an alarm device based on timing information repeatedly generated at a predetermined interval, transmitting alarm data and sound data acquired sequentially as packet data, and transmitting the packet data including the alarm data based on the presence or absence of the timing information during a period in which the sound data included in the packet data was acquired. It can also be configured as follows.

[0067] Transmitting the packet data includes generating packet data by making the position of the sound data and the position of the notification data in the packet data different from each other.

[0068] Transmitting the packet data includes superimposing the notification data on the sound data as data converted into a sound waveform in a different frequency band from that of the sound data.

[0069] The notification data includes information for specifying a data position of the sound data included in the packet data when the timing information was acquired.

[0070] The notification data includes information indicating the number of times the timing information was generated during the period in which the sound data included in the packet data was acquired.

[0071] The timing information defines beat information, and the notification data includes the beat information corresponding to the timing information in the period.

[0072] According to one embodiment of the present invention, there is provided a communication method including receiving packet data including notification data and sound data based on the presence or absence of timing information repeatedly generated at a predetermined interval, reproducing the sound data included in the packet data, and controlling notification processing executed by a notification device based on the notification data included in the packet data together with the sound data during a period in which the sound data is reproduced.

[0073] The notification process includes outputting a predetermined sound.

[0074] The notification process includes emitting a predetermined light.

[0075] The notification process includes playing a predetermined video.

[0076] Controlling the notification process includes outputting control data for the notification process to the notification device prior to a period during which the sound data is reproduced.

[0077] If the packet data is not received within a period of time specified based on a past control history of the notification process, the notification device is controlled to execute a predetermined notification process after the period of time has elapsed. [Explanation of symbols]

[0078] 1...server, 20...communication terminal, 21...control unit, 23...storage unit, 25...communication unit, 27...operation unit, 30...electronic musical instrument, 40...sound collection device, 50...imaging device, 60...sound emission device, 70...display device, 100...data communication function, 200...data transmission unit, 210...acquisition unit, 220...timing generation unit, 230...packet generation unit, 250...transmission unit, 300...data reception unit, 310...reception unit, 330...restoration unit, 350...playback unit, 500...notification control unit

Claims

1. Controlling a notification process executed by the notification device based on timing information repeatedly generated at a predetermined interval; Transmitting the notification data and the sound data acquired sequentially as packet data, the packet data including the notification data based on the presence or absence of the timing information during the period in which the sound data included in the packet data was acquired. A communication method comprising:

2. The communication method according to claim 1 , wherein transmitting the packet data includes generating packet data by making a position of the sound data and a position of the notification data in the packet data different from each other.

3. The communication method according to claim 1 , wherein the transmitting of the packet data includes superimposing the notification data on the sound data as data obtained by converting the notification data into a sound waveform having a different frequency band from that of the sound data.

4. 4. The communication method according to claim 1, wherein the notification data includes information for specifying a data position of sound data included in the packet data when the timing information was acquired.

5. The communication method according to claim 4 , wherein the notification data includes information indicating a number of times the timing information was generated during a period in which the sound data included in the packet data was acquired.

6. The timing information defines beat information, The communication method according to claim 1 , wherein the notification data includes beat information corresponding to the timing information in the period.

7. Controlling a notification process executed by the notification device based on timing information repeatedly generated at a predetermined interval; Transmitting the notification data and the sound data acquired sequentially as packet data, the packet data including the notification data based on the presence or absence of the timing information during the period in which the sound data included in the packet data was acquired. A program for causing a computer to execute the following.

8. 8. The program according to claim 7, wherein transmitting the packet data includes generating packet data by making a position of the sound data and a position of the notification data in the packet data different from each other.

9. 8. The program according to claim 7, wherein transmitting the packet data includes superimposing the notification data on the sound data as data obtained by converting the notification data into a sound waveform having a different frequency band from that of the sound data.

10. 10. The program according to claim 7, wherein the notification data includes information for specifying a data position of sound data included in the packet data when the timing information was acquired.

11. The program according to claim 10 , wherein the notification data includes information indicating a number of times the timing information was generated during a period in which the sound data included in the packet data was acquired.

12. The timing information defines beat information, 12. The program according to claim 7, wherein the notification data includes beat information corresponding to the timing information in the period.

13. receiving packet data including notification data and sound data based on the presence or absence of timing information that is repeatedly generated at a predetermined interval; Reproducing sound data contained in the packet data; During the period in which the sound data is reproduced, a notification process executed by a notification device is controlled based on notification data included in the packet data together with the sound data. A program for causing a computer to execute the following.

14. The program according to claim 13 , wherein the notification process includes outputting a predetermined sound.

15. 15. The program according to claim 13, wherein the notification process includes emitting a predetermined light.

16. 16. The program according to claim 13, wherein the notification process includes playing a predetermined video.

17. The program according to claim 16 , wherein controlling the notification process includes outputting control data for the notification process to the notification device before a period during which the sound data is reproduced.

18. A program as described in any one of claims 13 to 17, which controls the notification device to execute a predetermined notification process after a period specified based on a control history of the notification process in the event that the packet data is not received during the period specified based on a control history of the notification process in the past.

19. A first communication device and a second communication device are provided, The first communication device is a first notification control unit that controls a notification process executed by the first notification device based on timing information that is repeatedly generated at a predetermined interval; a data transmitting unit that transmits, as packet data, notification data and sound data that is sequentially acquired, and transmits the packet data including the notification data based on the presence or absence of the timing information during a period in which the sound data included in the packet data is acquired; having The second communication device is a data receiving unit that receives the packet data and reproduces sound data included in the packet data; a second notification control unit that controls a notification process executed by a second notification device based on notification data included in the packet data together with the sound data during a period in which the sound data is reproduced; having Communication systems.

Images