Electronic device, output program and output method

Abstract

In the present invention, when a MIDI message is received later than a scheduled output time, the MIDI message is immediately played, and the reception timing of the MIDI message and a time-stamp thereof are reset to a reference time and a reference time-stamp. As a result, because a time difference in terms of playing timing between the subsequent MIDI message and the MIDI message associated with the reference time and the reference time-stamp can be determined based on a time difference between the time-stamps of these MIDI messages, a listener's sense of unnaturalness due to the playing timing of the subsequent MIDI message can be suppressed. In addition, an initial MIDI message is output instantly after the reception thereof such that a time lag between the start of transmission of the MIDI message at a portable terminal 50 and the start of playing at a synthesizer 1 can be suppressed. As a result, the occurrence of a latency between MIDI messages can be suppressed.

Description

[Technical Field]

[0001] The present invention relates to an electronic device, an output program, and an output method. [Background technology]

[0002] Patent Document 1 discloses an electronic musical instrument 10 that receives MIDI messages via wireless communication from a device 30 that is a mobile terminal and produces sound from the received MIDI messages. Specifically, the device 30 transmits to the electronic musical instrument 10 a timestamp indicating the timing of execution of the MIDI message along with the MIDI message.

[0003] When the electronic musical instrument 10 receives the first MIDI message from the device 30, the sound of that MIDI message is produced at a time obtained by adding a predetermined offset time to the time of receipt of the MIDI message, and this time is designated as the "reference time." Subsequent MIDI messages are produced at a time obtained by adding the time difference between the timestamp of the first MIDI message and the timestamp of the MIDI message to be produced to the reference time.

[0004] In this way, in the electronic musical instrument 10, an offset time is provided between receiving a MIDI message and sounding a tone, so that even if jitter (fluctuation) or latency (delay) occurs in the communication of the MIDI message, a tone can be sounded while maintaining the time difference with the previously sounded MIDI message. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2022-96035 (for example, paragraphs 0073-0079, Figures 7C and 7D) Summary of the Invention [Problem to be solved by the invention]

[0006] Here, in the electronic musical instrument 10 of Patent Document 1, in FIG. 7C, the subsequent MIDI message is sounded at a time (hereinafter referred to as the "scheduled output time") obtained by adding the time difference between the timestamp of the MIDI message sounded at the reference time based on the initial MIDI message and the timestamp of the initial MIDI message.

[0007] For example, if MIDI message M1 to be sounded is received later than its scheduled output time due to jitter or latency, that MIDI message M1 will be sounded immediately after it is received. Furthermore, if the immediately following MIDI message M2 is received earlier than its scheduled output time, MIDI message M2 will be sounded at its scheduled output time. Therefore, if the reception of MIDI message M1 is delayed, the time difference between the sounding of MIDI message M1 and the sounding of MIDI message M2 will be shorter than the time difference between these timestamps, and the listener may feel that MIDI message M2 is sounding earlier, which creates a sense of discomfort.

[0008] Furthermore, in Figure 7D, the first MIDI message is not sounded until it has waited the offset time after arriving at the receiving device. The scheduled output times of subsequent MIDI messages are then set to a reference time based on the sounding timing of the first MIDI message plus the time difference between the timestamp of the MIDI message to be sounded and the timestamp of the first MIDI message. This allows the sounding intervals between each MIDI message to be maintained even if jitter occurs. However, setting the offset time to a fixed value can result in the risk of excessive latency, which would otherwise be avoidable.

[0009] The present invention has been made to solve the above-mentioned problems, and aims to provide an electronic device, an output program, and an output method that can suppress the listener's discomfort with the timing of sound output and can also suppress the occurrence of latency, even when outputting a MIDI message received from an external device. [Means for solving the problem]

[0010] In order to achieve this object, the electronic device of the present invention comprises: a performance information receiving means for receiving performance information from an external device; a TS receiving means for receiving from the external device a timestamp relating to the execution timing of the performance information received by the performance information receiving means; and a reference timing setting means for, when a piece of performance information is received by the performance information receiving means, setting a timing based on the reception timing of the piece of performance information as a reference timing. After the performance information in 1 above and a scheduled output timing setting means for setting a timing obtained by adding a time difference between a time stamp received by the TS receiving means corresponding to the other performance information and a time stamp received by the TS receiving means corresponding to the one performance information to the reference timing set by the reference timing setting means as a scheduled output timing, which is a timing for outputting the other performance information received by the performance information receiving means. When the scheduled output timing set by the scheduled output timing setting means is before the reception timing of the other performance information, the reference timing setting means sets the timing based on the reception timing of the other performance information as the reference timing. When the reference timing is set by the reference timing setting means based on the reception timing of the other performance information, the output scheduled timing setting means sets, as the output scheduled timing of subsequent performance information received by the performance information receiving means after the other performance information, a timing obtained by adding a time difference between a time stamp received by the TS receiving means corresponding to the subsequent performance information and a time stamp received by the TS receiving means corresponding to the other performance information to the reference timing based on the reception timing of the other performance information. do.

[0011] The output program of the present invention is a program for causing a computer to execute a process of outputting received performance information, and includes: a performance information receiving step of receiving performance information from an external device; a TS receiving step of receiving from the external device a timestamp relating to the execution timing of the performance information received in the performance information receiving step; and a reference timing setting step of, when one piece of performance information is received in the performance information receiving step, setting a timing based on the reception timing of the one piece of performance information as a reference timing. After the performance information in 1 aboveand a planned output timing setting step for setting a timing obtained by adding a time difference between a timestamp received in the TS receiving step corresponding to the other performance information and a timestamp received in the TS receiving step corresponding to the one performance information to the reference timing set in the reference timing setting step as a planned output timing, which is a timing to output the other performance information received in the performance information receiving step, and the reference timing setting step sets a timing based on the reception timing of the other performance information as the reference timing when the planned output timing set in the planned output timing setting step is before the reception timing of the other performance information. and when the reference timing is set based on the reception timing of the other performance information by the reference timing setting step, the output planned timing setting step sets, as the output planned timing of subsequent performance information received in the performance information receiving step after the other performance information, a timing obtained by adding a time difference between a timestamp received in the TS receiving step corresponding to the subsequent performance information and a timestamp received in the TS receiving step corresponding to the other performance information to the reference timing based on the reception timing of the other performance information. do.

[0012] The output method of the present invention includes a performance information receiving step of receiving performance information from an external device, a TS receiving step of receiving from the external device a timestamp relating to the execution timing of the performance information received in the performance information receiving step, and a reference timing setting step of, when one piece of performance information is received in the performance information receiving step, setting a timing based on the reception timing of the one piece of performance information as a reference timing. After the performance information in 1 above and a planned output timing setting step for setting a timing obtained by adding a time difference between a timestamp received in the TS receiving step corresponding to the other performance information and a timestamp received in the TS receiving step corresponding to the one performance information to the reference timing set in the reference timing setting step as a planned output timing, which is a timing to output the other performance information received in the performance information receiving step, wherein the reference timing setting step sets a timing based on the reception timing of the other performance information as the reference timing when the planned output timing set in the planned output timing setting step is before the reception timing of the other performance information. and when the reference timing is set based on the reception timing of the other performance information by the reference timing setting step, the output planned timing setting step sets, as the output planned timing of subsequent performance information received in the performance information receiving step after the other performance information, a timing obtained by adding a time difference between a timestamp received in the TS receiving step corresponding to the subsequent performance information and a timestamp received in the TS receiving step corresponding to the other performance information to the reference timing based on the reception timing of the other performance information. do. [Brief explanation of the drawings]

[0013] [Figure 1](a) is an external view of a synthesizer and a portable terminal, (b) is a diagram showing the timing of MIDI messages sent from the portable terminal, and (c) is a diagram showing the timing of receiving and sounding MIDI messages in the synthesizer. [Figure 2] FIG. 2 is a functional block diagram of a synthesizer. [Figure 3] 1A is a block diagram showing the electrical configuration of a synthesizer, and FIG. 1B is a diagram showing a schematic diagram of a delay output buffer. [Figure 4] 10A is a flowchart of the main processing of the synthesizer, and FIG. 10B is a flowchart of the timer processing. [Figure 5] (a) is a diagram explaining the offset time in the second embodiment, (b) is a diagram showing the timing of a MIDI message sent from a mobile terminal in the second embodiment, (c) is a diagram showing the timing of receiving and producing a MIDI message by a synthesizer in the second embodiment, and (d) is a diagram schematically showing the flash ROM in a synthesizer in the second embodiment. [Figure 6] 10 is a flowchart of main processing of a synthesizer according to the second embodiment. [Figure 7] 10A is a flowchart of the main processing of the mobile terminal in the modified example, and FIG. 10B is a diagram illustrating the advancement of the reference time in the modified example. DETAILED DESCRIPTION OF THE INVENTION

[0014] Preferred embodiments will now be described with reference to the accompanying drawings. An overview of a synthesizer 1 according to this embodiment will be described with reference to Fig. 1(a). Fig. 1(a) is a diagram illustrating an overview of the synthesizer 1 and a portable terminal 50.

[0015] Synthesizer 1 is a device (electronic musical instrument or electronic device in a broad sense) that produces musical tones based on a performance by user H or MIDI (Musical Instruments Digital Interface) messages, which are performance information received from an external device such as mobile terminal 50 (described later). Synthesizer 1 is provided with keyboard 2 that acquires operation information from operations by user H, and setting buttons 3 through which user H inputs various settings. Synthesizer 1 is also connected to mobile terminal 50 via wireless communication such as Wi-Fi or Bluetooth (both registered trademarks).

[0016] The mobile terminal 50 is an information processing device (computer) that performs processing in accordance with instructions input by the user H. The mobile terminal 50 is configured to be able to transmit MIDI messages to an external device in accordance with instructions from the user H. Here, with reference to FIGS. 1(b) and 1(c), the transmission of MIDI messages by the mobile terminal 50 and the generation of sounds by the synthesizer 1 in response to MIDI messages received from the mobile terminal 50 will be described.

[0017] Fig. 1(b) is a diagram showing the timing of MIDI messages transmitted from the mobile terminal 50. The horizontal axis in Fig. 1(b) represents time, and the same applies to Fig. 1(c) and Figs. 5 and 7 described below. As shown in Fig. 1(b), MIDI messages are transmitted from the mobile terminal 50 at transmission timings P0 to P5, which correspond to times T0 to T5.

[0018] For simplicity of explanation, in this embodiment, the time between time T0 and time T1, the time between time T1 and time T2, the time between time T2 and time T3, the time between time T3 and time T4, and the time between time T4 and time T5 are each set to a fixed time (e.g., 100 milliseconds), but the intervals between each time may be different. Also, the number of MIDI messages transmitted from mobile terminal 50 is not limited to six, from transmission timings P0 to P5, and may be less than six or more than six.

[0019] Additionally, together with each MIDI message, a timestamp consisting of information about the time at which that MIDI message is transmitted is also transmitted to synthesizer 1. For example, the timestamp of a MIDI message transmitted at timing P0 contains information about time T0, and the timestamp of a MIDI message transmitted at timing P3 contains information about time T3. As will be described in more detail below, the timestamps are used to control the timing at which the corresponding MIDI message is sounded in synthesizer 1.

[0020] Such MIDI messages and corresponding timestamps transmitted from portable terminal 50 are received by synthesizer 1, and the corresponding musical tones are generated from synthesizer 1. The timing of receiving and generating MIDI messages in synthesizer 1 will be described with reference to Fig. 1(c).

[0021] 1(c) is a diagram showing the timing of reception and generation of MIDI messages in synthesizer 1. MIDI messages transmitted from portable terminal 50 at transmission times P0 to P5 are received by synthesizer 1 at reception times Pr0 to Pr5, respectively. Reception times Pr0 to Pr5 correspond to times Tr1 to Tr5, respectively.

[0022] These MIDI messages are transmitted from portable terminal 50 to synthesizer 1 via wireless communication, which can cause communication fluctuations (jitter) and communication delays (latency) depending on the communication conditions. As a result, there is a time variance in the reception timings Pr0 to Pr5 compared to the transmission timings P0 to P5 shown in FIG. 1(b).

[0023] For example, reception timing Pr1 is delayed due to latency. As a result, the time difference Tr1-Tr0 between reception timing Pr1 and the immediately preceding reception timing Pr0 is greater than the time difference T1-T0 between the corresponding transmission timing P1 and the transmission timing P0. On the other hand, reception timing Pr2 fluctuates due to jitter. As a result, the time difference Tr2-Tr1 between reception timing Pr2 and the immediately preceding reception timing Pr1 is smaller than the time difference T2-T1 between the corresponding transmission timing P2 and the transmission timing P1.

[0024] In this way, there is a time variance between the reception timings Pr0 to Pr5 and the transmission timings P0 to P5, so even if a MIDI message is sounded at the reception timings Pr0 to Pr5, it will not be sounded with a time difference that conforms to the original transmission timings P0 to P5 transmitted by the mobile terminal 50. Therefore, in this embodiment, a reference time that serves as a reference for the timing at which these messages are sounded is set in accordance with the reception timings Pr0 to Pr5, thereby suppressing the deviation from the time difference that conforms to the original transmission timings P0 to P5.

[0025] Specifically, when synthesizer 1 receives the first MIDI message (i.e., reception timing Pr0) from portable terminal 50, time Tr0 corresponding to reception timing Pr0 is set as scheduled output time Ts0, which is the time at which the MIDI message is to be sounded. In other words, scheduled output time Ts0 is set as sounding timing Ps0 at which the first MIDI message is to be sounded.

[0026] In synthesizer 1, the corresponding MIDI message is generated when the set scheduled output time is before the current time. Therefore, for the first MIDI message, time Tr0 corresponding to reception timing Pr0 is set as scheduled output time Ts0, so the message is generated immediately after reception. By generating the first MIDI message immediately after reception, the time lag between the start of MIDI message transmission at portable terminal 50 and the start of generation at synthesizer 1 can be reduced. This reduces the discomfort felt by the listener when the MIDI message generation starts.

[0027] In this embodiment, the "first MIDI message" refers to the MIDI message received for the first time from the mobile terminal 50 after the synthesizer 1 is turned on, and the MIDI message received from the mobile terminal 50 after a predetermined time (e.g., 15 seconds) has elapsed since the synthesizer 1 received the last MIDI message from the mobile terminal 50.

[0028] In this way, the time Tr0, Ts0 when the first MIDI message is received and sounded is set as the reference time (reference timing) that serves as the basis for the sounding timing of subsequent MIDI messages, and the timestamp of the first MIDI message is set as the reference timestamp that serves as the basis for the sounding timing.

[0029] The sounding timing of the subsequent MIDI message is set to the time obtained by adding the time difference between the timestamp of the subsequent MIDI message and the timestamp of the initial MIDI message, which is the reference time, to the time Tr0, Ts0, which is the reference time.

[0030] This allows the time difference between the sounding timing of the first MIDI message set at the reference time and the sounding timing of subsequent MIDI messages to be made equal to the time difference in the timestamp of the corresponding MIDI message transmitted from the mobile terminal 50, so that subsequent MIDI messages can be sounded while maintaining an appropriate time difference from the first MIDI message set at the reference time. As will be described in detail later, the reference time and reference timestamp are reset based on the temporal relationship between the reception timing of the MIDI message and the scheduled output time.

[0031] The second MIDI message following the first MIDI message is received at time Tr1, which is the reception timing Pr1. The scheduled output time Ts1 of this second MIDI message is set to the reference time (i.e., time Tr0 of the reception timing Pr0 of the first MIDI message) plus the time difference T1-T0 between the transmission timing P1 of the second MIDI message, time T1, and the reference timestamp (i.e., the timestamp of the first MIDI message).

[0032] However, the scheduled output time Ts1 set in this way is set to a time before the time Tr1 of the reception timing Pr1 of the second MIDI message. A MIDI message is sounded when its scheduled output time is before the current time, so the second MIDI message is sounded immediately at the time Tr1, which is the reception timing Pr1. This time is set as the scheduled output time Ts1', and is also set as the sounding timing Ps1 of the second MIDI message.

[0033] In other words, the second MIDI message is sounded later than the scheduled output time Ts1, which is set based on the reference time and reference timestamp based on the first MIDI message. This prevents the second MIDI message, which is received later than the initially expected scheduled output time Ts1, from being sounded even later.

[0034] However, if the second MIDI message is delayed in this way, and the third MIDI message is set to a scheduled output time while the reference time and reference timestamp remain based on the first MIDI message, the time difference between the second MIDI message being sounded and the scheduled output time of the third MIDI message will be shorter than the time difference T2-T1 between the corresponding transmission timings P1 and P2. This could cause the listener to feel that the third MIDI message was sounded early, causing a sense of discomfort when the third MIDI message is sounded.

[0035] Therefore, in this embodiment, the scheduled output time is before the reception timing, and the reception timing of the MIDI message that has been delayed is reset to the reference time, and the timestamp of that MIDI message is newly reset to the reference timestamp.

[0036] For the second MIDI message, the time Tr1, Ts1' when the second MIDI message was received and sounded is set as the new reference time, and the timestamp of the second MIDI message is set as the reference timestamp. The scheduled output times of the third and subsequent MIDI messages are set using the reference time and reference timestamp reset based on the second MIDI message.

[0037] The scheduled output time Ts2 of the third MIDI message following the second MIDI message is set to the reference time (i.e., the time Tr1, Ts1' of the reception timing Pr1 of the second MIDI message) plus the time difference T2-T1 between the time T2 of the transmission timing P2 of the third MIDI message and the reference timestamp (i.e., the timestamp of the second MIDI message).

[0038] The third MIDI message is received at time Tr2 of reception timing Pr2. Since this time Tr2 is earlier than the set scheduled output time Ts2, the third MIDI message is sounded at the scheduled output time Ts2, which becomes the sounding timing Ps2. This allows the third MIDI message to be sounded while maintaining the time difference T2-T1 with the transmission timing P1 of the second MIDI message.

[0039] The fourth MIDI message following the third MIDI message is received later than the scheduled output time Ts3 of the fourth MIDI message, just like the second MIDI message described above. Therefore, just like the second MIDI message described above, the fourth MIDI message is sounded at time Tr3 of sounding timing Ps3, the time Tr3, Ts3' when the fourth MIDI message was received and sounded is reset to the reference time, and the timestamp of the fourth MIDI message is set to the reference timestamp.

[0040] Similarly, the fifth MIDI message following the fourth MIDI message is received earlier than the scheduled output time Ts4 of the fifth MIDI message, so it is sounded at the scheduled output time Ts4. Similarly, the sixth MIDI message following the fifth MIDI message is received earlier than the scheduled output time Ts5 of the sixth MIDI message, so it is sounded at the scheduled output time Ts5.

[0041] In this way, if a MIDI message is received at a timing later than the scheduled output time obtained by adding the time difference between the timestamp of the MIDI message and the reference timestamp to the reference time, the MIDI message is immediately sounded, and the reception timing and timestamp of the MIDI message are reset to the reference time and reference timestamp.

[0042] The scheduled output time of the subsequent MIDI message is then set based on the reset reference time and reference timestamp. This allows the time difference between the sounding timing of the subsequent MIDI message and the sounding timing of the MIDI message used to reset the reference time and reference timestamp to be based on the time difference between the timestamps of these MIDI messages, thereby reducing any discomfort felt by the listener with respect to the sounding timing of the subsequent MIDI message.

[0043] Next, the function of synthesizer 1 will be described with reference to Fig. 2. Fig. 2 is a functional block diagram of synthesizer 1. As shown in Fig. 2, synthesizer 1 has performance information receiving means 300, TS receiving means 301, reference timing setting means 302, and scheduled output timing setting means 303.

[0044] The performance information receiving means 300 is a means for receiving a MIDI message from the mobile terminal 50, and is realized by the CPU 10, which will be described later in Fig. 3. The TS receiving means 301 is a means for receiving a timestamp related to the execution timing of the MIDI message received by the performance information receiving means 300 from the mobile terminal 50, and is realized by the CPU 10.

[0045] The reference timing setting means 302 is a means for setting a reference time to a timing based on the reception timing of one MIDI message when the performance information receiving means 300 receives the one MIDI message at a predetermined timing, and is realized by the CPU 10. The scheduled output timing setting means 303 is a means for setting a scheduled output time, which is the timing at which another MIDI message received by the performance information receiving means 300 is to be sounded (output), by adding the time difference between the timestamp received by the TS receiving means 301 corresponding to the other MIDI message and the timestamp received by the TS receiving means 301 corresponding to the one MIDI message to the reference time set by the reference timing setting means 302, and is realized by the CPU 10.

[0046] Furthermore, when the scheduled output time set by the scheduled output timing setting means 303 occurs before the reception timing of another MIDI message, the reference timing setting means 302 sets the reference time to the timing based on the reception timing, and further, when the performance information receiving means 300 receives the first MIDI message, sets the reference time to the reception timing of that first MIDI message. Furthermore, the scheduled output timing setting means 303 sets the reception timing of the first MIDI message to the scheduled output timing of that first MIDI message.

[0047] A MIDI message is received from the mobile terminal 50, and a timestamp related to the execution timing of the received MIDI message is also received. When one MIDI message is received at a predetermined timing, the timing based on the reception timing of that one MIDI message is set as the reference time. Then, the estimated output time for sounding another MIDI message is set to the reference time plus the time difference between the timestamp corresponding to that other MIDI message and the timestamp corresponding to the one MIDI message. Here, if the estimated output time is before the reception timing of the corresponding MIDI message, the timing based on the reception timing is set as the reference time.

[0048] In other words, if a delayed MIDI message is received later than its scheduled output time, the reference time is reset based on that delayed MIDI message. The scheduled output time of the subsequent MIDI message received after that delayed MIDI message is then set based on the reset reference time. This allows the time difference between the sounding timing of the subsequent MIDI message and the sounding timing of the delayed MIDI message to be based on the time difference between these timestamps, reducing any discomfort felt by listeners with regard to the sounding timing of the subsequent MIDI message.

[0049] Furthermore, when the first MIDI message is received by the first performance information receiving means 300, the MIDI message is instantly generated. This reduces the time lag between when the MIDI message starts being transmitted from the mobile terminal 50 and when the MIDI message starts being generated by the synthesizer 1, thereby reducing the latency of the MIDI message.

[0050] Next, the electrical configuration of synthesizer 1 will be described with reference to Fig. 3. Fig. 3(a) is a block diagram showing the electrical configuration of synthesizer 1. Synthesizer 1 has CPU 10, flash ROM 11, RAM 12, RTC 13 that keeps track of the date and time, keyboard 2 and setting buttons 3 described above, sound source 14, DSP (Digital Signal Processor) 15, and wireless communication device 20 for wireless communication with mobile terminal 50, all of which are connected via bus line 16. DSP 15 is connected to DAC (Digital Analog Converter) 17, DAC 17 is connected to amplifier 18, and amplifier 18 is connected to speaker 19.

[0051] The CPU 10 is a computing device that controls each unit connected via a bus line 16. The flash ROM 11 is a rewritable nonvolatile memory and stores a musical sound output program 11a. When the musical sound output program 11a is executed by the CPU 10, the main processing shown in FIG. 4(a) is executed.

[0052] The RAM 12 is a memory that stores various work data, flags, etc. in a rewritable manner when the CPU 10 executes programs such as the musical sound output program 11a, and has a delay output buffer 12a, a reference time memory 12b in which the above-mentioned reference time is stored, and a reference TS memory 12c in which the above-mentioned reference time stamp is stored. The delay output buffer 12a will be described with reference to Figure 3(b).

[0053] 3(b) is a diagram schematically illustrating the delay output buffer 12a. As shown in FIG. 3(b), the delay output buffer 12a stores the above-mentioned scheduled output times and the MIDI messages corresponding to the scheduled output times, in association with each other.

[0054] Returning to FIG. 3(a), the sound source 14 is a device that outputs waveform data according to information input from the CPU 10. The DSP 15 is a calculation device that processes the waveform data input from the sound source 14. The DAC 17 is a conversion device that converts the waveform data input from the DSP 15 into analog waveform data. The amplifier 18 is an amplification device that amplifies the analog waveform data output from the DAC 17 with a predetermined gain. The speaker 19 is an output device that emits (outputs) the analog waveform data amplified by the amplifier 18 as musical tones.

[0055] Next, the processing executed by CPU 10 of synthesizer 1 will be described with reference to Fig. 4. Fig. 4(a) is a flowchart of the main processing of synthesizer 1. The main processing is processing that is executed when synthesizer 1 is powered on. The main processing first checks whether a MIDI message and a timestamp have been received from portable terminal 50 via wireless communication device 20 (S1).

[0056] In the process of S1, if a MIDI message and timestamp are received (S1: Yes), the received MIDI message and timestamp are acquired (S2). After the process of S2, it is confirmed whether the received MIDI message is the first MIDI message (S3).

[0057] In the process of S3, if the received MIDI message is the first MIDI message (S3: Yes), the scheduled output time is set to the current time, i.e., the timing of receiving the MIDI message (S4). The current time is obtained from the RTC 13. After the process of S4, the scheduled output time set in the process of S4 is stored as the reference time in the reference time memory 12b, and the timestamp obtained in the process of S2 is stored as the reference timestamp in the reference TS memory 12c (S5).

[0058] On the other hand, if the received MIDI message is not the first MIDI message in the process of S3 (S3: No), the scheduled output time is set to the reference time in the reference time memory 12b plus the time difference between the timestamp acquired in the process of S2 and the reference timestamp in the reference TS memory 12c (S6). After the process of S6, it is confirmed whether the current time (i.e., the reception timing) is after the scheduled output time set in the process of S6 (S7). If the scheduled output time set in the process of S6 is earlier than the current time (S7: Yes), the MIDI message was received later than the scheduled output time, and the processes from S4 onwards are executed.

[0059] After the process of S5, or in the process of S7, if the scheduled output time in the process of S6 is later than the current time (S7: No), the scheduled output time set in S4 and S6 and the MIDI message acquired in the process of S2 are added to the delay output buffer 12a (S8). The MIDI message added to the delay output buffer 12a is sounded by the timer process described later in FIG. 4(b).

[0060] In the process of S1, if the MIDI message and timestamp are not received (S1: No), or after the process of S8, other processes of the synthesizer 1 (S9) are executed, and the processes from S1 onwards are repeated.

[0061] Next, the timer processing will be described with reference to Fig. 4(b). Fig. 4(b) is a flowchart of the timer processing. The timer processing is a process executed by the CPU 10 of the synthesizer 1 at regular intervals (for example, every 50 milliseconds).

[0062] The timer process first checks whether there is a MIDI message in the delay output buffer 12a whose scheduled output time is before the current time (S20). If there is a MIDI message in the delay output buffer 12a whose scheduled output time is before the current time (S20: Yes) in the process of S20, the corresponding MIDI message is generated (S21). Specifically, waveform data corresponding to the corresponding MIDI message is acquired from the sound source 14, and the acquired waveform data is output to the DSP 15, DAC 17, amplifier 18, and speaker 19, whereby a musical tone is generated.

[0063] After the process of S21, the MIDI message sounded in the process of S21 and the scheduled output time corresponding to that MIDI message are deleted from the delay output buffer 12a (S22). In the process of S20, if there is no MIDI message in the delay output buffer 12a whose scheduled output time is earlier than the current time (S20: No), or after the process of S22, the timer process ends.

[0064] Next, a second embodiment will be described with reference to Figures 5 and 6. In the synthesizer 1 of the first embodiment described above, if the scheduled output time of a MIDI message is before the reception timing of that MIDI message, the reception timing is set as the reference time, and the timestamp corresponding to that MIDI message is set as the reference timestamp.

[0065] In addition, synthesizer 100 of the second embodiment acquires an offset time, which is the time difference between the reception timing and sound generation timing of each MIDI message, and adds the longest offset time among the acquired offset times to the reception timing of the first MIDI message that will be generated next, to determine the scheduled output time and reference time of the first MIDI message. The same reference numerals are used to designate the same parts as those of the first embodiment, and their explanations will be omitted.

[0066] Figure 5(a) is a diagram illustrating offset times in the second embodiment. Figure 5(a) illustrates an example of the same timing as the MIDI message reception and sound generation described above in Figure 1(c). In Figure 5(a), if the reception timing is before the scheduled output time, such as for the third MIDI message or the fifth and sixth MIDI messages, a time lag occurs between the reception of the MIDI message and the generation of sound.

[0067] For example, for the third MIDI message, a time F2 occurs between the scheduled output time Ts2 and the time Tr2 of the reception timing Pr2; for the fifth MIDI message, a time F4 occurs between the scheduled output time Ts4 and the time Tr4 of the reception timing Pr4; and for the sixth MIDI message, a time F5 occurs between the scheduled output time Ts5 and the time Tr5 of the reception timing Pr5.

[0068] Times F2, F4, and F5 are considered to be "delay times" caused by delays in receiving the second and fourth MIDI messages, which cause the sound to be produced from the original scheduled output times Ts1 and Ts3 (see Figure 1(c)) to Ts1' and Ts3'. In other words, time F4, which is the longest of times F2, F4, and F5, is considered to be the longest possible delay in receiving and producing this series of MIDI messages.

[0069] Therefore, in the second embodiment, the time difference between the scheduled output time and the reception timing of a series of MIDI messages is obtained, and the longest time FT (time F4 in FIG. 5(a)) is added to the reception timing of the first MIDI message in the next series of MIDI messages. Hereinafter, the time difference between the scheduled output time and the reception timing will be referred to as the "offset time."

[0070] The offset times of a series of MIDI messages are acquired until the synthesizer 100 is powered off or until a predetermined time (e.g., 15 seconds) has elapsed since the last MIDI message was received from the mobile terminal 50, and the longest offset time FT is set from the acquired offset times.

[0071] Fig. 5(b) is a diagram showing the timing of MIDI messages transmitted from portable terminal 50 in the second embodiment, and Fig. 5(c) is a diagram showing the timing of MIDI message reception and sound generation by synthesizer 100 in the second embodiment. Synthesizer 100 receives MIDI messages at reception times Pr0 to Pr5 as shown in Fig. 5(a) and generates sound, and then is powered off. At this time, "offset time F4" is set as the longest offset time FT.

[0072] After that, after the power is turned on again, MIDI messages are transmitted from the mobile terminal 50 at transmission timings P30 to P35 (times T30 to T35) as shown in Fig. 5(b). At this time, similar to the first embodiment, the timestamps of the times T30 to T35 at which the MIDI messages were transmitted are also transmitted together with the MIDI messages.

[0073] At these transmission times P30 to P35, the MIDI messages and corresponding timestamps transmitted from the portable terminal 50 are received by the synthesizer 100 and sounded. At this time, as shown in FIG. 5(c), the first MIDI message is sounded at time Ts30, which is the longest offset time FT added to time Tr30, which is the reception timing Pr30 (i.e., time Ts30 is set as the scheduled output time of the first MIDI message), and this time Ts30 is set as the reference time. Furthermore, the timestamp of the first MIDI message is set as the reference timestamp. Subsequent MIDI messages are sounded at times obtained by adding the time difference between the timestamp of the received MIDI message and the reference timestamp to the set reference time, as in the first embodiment described above.

[0074] The reference time is the time Tr30, which is the timing Pr30 of receiving the first MIDI message, plus the longest offset time FT, which is the longest possible delay in receiving and sounding the previous series of MIDI messages. In other words, because the reference time takes the longest offset time FT into consideration, even if the reception of a subsequent MIDI message is delayed, the scheduled output time can be prevented from occurring before the reception timing.

[0075] This allows MIDI messages to be stably generated in accordance with the time difference based on the time stamp transmitted by the mobile terminal 50, thereby reducing the listener's discomfort with the timing of the generation of the messages.

[0076] Next, the electrical configuration of the second embodiment will be described with reference to Fig. 5(d). Fig. 5(d) is a diagram schematically showing flash ROM 11 in synthesizer 100 of the second embodiment. In addition to the musical tone output program 11a, flash ROM 11 in synthesizer 100 of the second embodiment has longest offset time memory 11b in which the longest offset time FT is stored.

[0077] Next, the main processing of synthesizer 100 of the second embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart of the main processing of synthesizer 100 of the second embodiment. In the processing of S3, if the received MIDI message is the first MIDI message (S3: Yes), instead of the processing of S4 described above, the expected output time is set to the time obtained by adding the longest offset time FT in longest offset time memory 11b to the current time (S30). After the processing of S30, the processing of S5 described above is executed.

[0078] In the process of S7, if the scheduled output time set in the process of S6 is before the current time (S7: Yes), the scheduled output time is set to the current time, i.e., the timing of receiving the MIDI message (S31). The process of S31 is the same as the process of S4 described above.

[0079] On the other hand, in the process of S7, if the scheduled output time set in the process of S6 is later than the current time (S7: No), the offset time is set to the time from the scheduled output time set in the process of S6 to the current time (S32). After the process of S32, it is confirmed whether the offset time set in the process of S32 is longer than the longest offset time FT in the longest offset time memory 11b (S33).

[0080] In the process of S33, after the process of S32, if the offset time set in the process of S32 is longer than the longest offset time FT in the longest offset time memory 11b (S33: Yes), the offset time set in the process of S32 is stored in the longest offset time memory 11b (S34).

[0081] In the process of S33, if the offset time set in the process of S32 is equal to or shorter than the longest offset time FT in the longest offset time memory 11b (S33: No), or after the process of S34, the processes of S8 and onwards are executed.

[0082] The above has been explained based on the above embodiment, but it can be easily imagined that various improvements and modifications are possible.

[0083] In the above embodiment, a MIDI message for producing sound (hereinafter referred to as a "sound production MIDI message") is transmitted from mobile terminal 50, and a reference time and a reference timestamp are set along with the production of the sound production MIDI message in synthesizer 1,100. However, the present invention is not limited to this, and before transmitting the sound production MIDI message from mobile terminal 50, a dummy MIDI message that does not produce sound production and does not include information such as timbre or scale may be transmitted, and the reference time and the reference timestamp may be set in synthesizer 1,100 using the received dummy MIDI message.

[0084] Specifically, in the main processing of portable terminal 50 in the modified example shown in FIG. 7(a), first, wireless communication with synthesizer 1,100 is established (S50), and counter variable N is set to 1 (S51). After the processing of S51, a dummy MIDI message and a timestamp of the time when the dummy MIDI message is to be transmitted are transmitted to synthesizer 1,100 (S52). After the processing of S52, a wait processing is performed (S53). Note that the wait processing of the processing of S53 may be a wait processing for a fixed time (for example, 100 milliseconds) or a wait processing for a random time.

[0085] After the process of S53, 1 is added to the counter variable N (S54), and it is confirmed whether the added counter variable N is greater than 10 (S55). In the process of S55, if the counter variable N is 10 or less (S55: No), the processes from S52 onwards are repeated.

[0086] In the process of S52, synthesizer 1,100 receives the dummy MIDI message transmitted from portable terminal 50, and sets the reference time and reference timing by the processes of S1 to S8 described above in Figures 4 and 6. At this time, if the target MIDI message in the process of S21 in Figure 4(b) is a dummy, the MIDI message does not contain information such as tone color or scale, and therefore no sound is produced, but the display means of synthesizer 1,100 may display a message indicating that a dummy MIDI message has been received.

[0087] On the other hand, if the counter variable N is greater than 10 in the process of S55 (S55: Yes), other processes (S56) of the mobile terminal 50 are repeated. The other processes of S56 include a process of transmitting the sounding MIDI message and its timestamp to the synthesizer 1,100.

[0088] In this way, a dummy MIDI message is transmitted from portable terminal 50 before transmitting a sound generation MIDI message, and the reference time and reference timestamp are set by the dummy MIDI message in synthesizer 1,100 before receiving the sound generation MIDI message. This allows synthesizer 1,100 to receive and generate the sound generation MIDI message using a reference time and reference timestamp that take into account communication conditions such as jitter and latency between synthesizer 1,100 and portable terminal 50 from the beginning. This prevents the scheduled output time from occurring before the reception timing during the generation of a series of MIDI messages, causing the generation timing to deviate from the timing based on the time difference between the timestamps, thereby ensuring stable generation of sound generation for MIDI messages.

[0089] In the processes of S52 to S55, the number of times that dummy MIDI messages are sent is not limited to 10, and may be more or less than 10. Furthermore, dummy MIDI messages are not limited to those sent before sounding MIDI messages, and for example, dummy MIDI messages may be sent before sounding MIDI messages and between sounding MIDI messages.

[0090] In the above embodiment, if the scheduled output time of the MIDI message falls before the reception time, the reference time is reset to the reception time. However, this is not limited to this. The reference time may also be reset to a time earlier than the reception time. For example, as shown in Figure 7(b), if the scheduled output time Ts1 of the MIDI message falls before time Tr1, which is the reception time Pr1, the reference time may be set to a time earlier than time Tr1 by the time difference ΔT between time Tr1 and the scheduled output time Ts1 multiplied by a predetermined coefficient (e.g., 0.25).

[0091] In this way, by moving the reference time forward from the reception timing, it is possible to prevent the latency of subsequent MIDI messages from becoming unnecessarily long due to large jitter (time delay), which rarely occurs. Furthermore, by basing the amount by which the reference time is moved forward on the time difference ΔT between the reception timing Pr1 and the scheduled output time Ts1, it is possible to prevent the reference time from being set unnecessarily early, such as before the scheduled output time Ts1. This reduces the frequency with which subsequent MIDI messages are received after the scheduled output time, allowing MIDI messages to be played stably.

[0092] The time by which the reference time is advanced is not limited to the time based on the time difference ΔT between the reception timing and the scheduled output time, but may be, for example, the smaller of a fixed time (30 milliseconds) and the time difference ΔT.

[0093] In the above embodiment, in the process of S7 in Fig. 4, if the scheduled output time set in the process of S6 falls before the reception timing, the reception timing is reset to the reference time and the timestamp of that MIDI message is reset as the reference timestamp, but this is not limited to this. For example, regardless of the temporal relationship between the scheduled output time and the reception timing, the reception timing may always be set to the reference time and the timestamp of that MIDI message may always be set as the reference timestamp.

[0094] In the above embodiment, the time stamp of a MIDI message transmitted from mobile terminal 50 is set to the time at which the MIDI message is transmitted, but this is not limited to this. For example, the time stamp may be set to the time at which synthesizer 1,100 receiving the MIDI message will produce a sound, or the time stamp may be set to the timing (execution timing) related to the output of the MIDI message or other events.

[0095] In the second embodiment, the reference time is set to the time when the first MIDI message is received plus the longest offset time FT obtained when the previous MIDI message was received and sounded. However, the time added to the time when the first MIDI message is received when setting the reference time is not limited to the longest offset time FT. For example, a time longer or shorter than the longest offset time FT may be added. Furthermore, the reference time may be the average or median of the offset times obtained when the previous MIDI message was received and sounded, or any other time related to the offset time.

[0096] Alternatively, the reference time may be set by adding a time shorter than the longest offset time FT to the timing of receiving the first MIDI message, such as by multiplying the longest offset time FT by a predetermined coefficient less than 1, or by subtracting a fixed value from the longest offset time FT. This prevents the latency of subsequent MIDI messages from becoming unnecessarily long due to large jitter (time delay) that rarely occurs.

[0097] In the above embodiment, MIDI messages are used as an example of performance information handled by the synthesizer 1100, but this is not limiting, and information other than MIDI messages may also be used as performance information.

[0098] In the above embodiment, a MIDI message is generated in the process of S21 in Fig. 4(b), but this is not limiting. For example, in addition to generating a MIDI message, or instead of generating a MIDI message, the MIDI message may be transmitted to another electronic device via the wireless communication device 20, or the MIDI message may be output in another manner.

[0099] In the above embodiment, synthesizer 1,100 and portable terminal 50 are connected by wireless communication, but the communication method is not limited to this. For example, synthesizer 1,100 and portable terminal 50 may be connected by wired communication, or may be connected by another communication method.

[0100] In the above embodiment, the musical sound output program 11a is stored in the flash ROM 11 of the synthesizer 1100 and is run on the synthesizer 1100. However, this is not necessarily limited to this, and the musical sound output program 11a may be run on other electronic musical instruments such as an electronic piano, electronic organ, or electronic drum, or on other computers or electronic devices such as a PC (personal computer), mobile terminal, or tablet terminal.

[0101] Furthermore, although the external device that sends MIDI messages to the synthesizer 1,100 is configured as a mobile terminal 50, this is not limited to this, and the external device may be configured as another computer such as a PC or tablet terminal, or may be configured as an electronic musical instrument such as a synthesizer. [Explanation of symbols]

[0102] 1,100 Synthesizers (electronic devices) 11a Musical sound output program (output program) 11b Longest offset time memory (part of offset time storage means) 50 Mobile devices (external devices) Pr1~Pr35 reception timing S1, S2: Performance information receiving means, TS receiving means, performance information receiving step, TS receiving step S5 Reference timing setting means, reference timing setting step S4, S6: Output timing schedule setting means, output timing schedule setting step S32 Offset time acquisition means S33, S34 Part of offset time storage means

Claims

1. a performance information receiving means for receiving performance information from an external device; a TS receiving means for receiving from the external device a timestamp relating to the execution timing of the performance information received by the performance information receiving means; a reference timing setting means for setting a reference timing based on the reception timing of one piece of performance information when the performance information receiving means receives the piece of performance information; an expected output timing setting means for setting a timing obtained by adding a time difference between a time stamp received by the TS receiving means corresponding to the other performance information and a time stamp received by the TS receiving means corresponding to the first performance information to the reference timing set by the reference timing setting means, as an expected output timing for outputting other performance information received by the performance information receiving means after the first performance information, the reference timing setting means sets the reference timing to a timing based on the reception timing of the other performance information when the scheduled output timing set by the scheduled output timing setting means is before the reception timing of the other performance information, When the reference timing setting means has set the reference timing based on the reception timing of the other performance information, the output planned timing setting means sets, as the output planned timing of subsequent performance information received by the performance information receiving means after the other performance information, a timing obtained by adding a time difference between a timestamp received by the TS receiving means corresponding to the subsequent performance information and a timestamp received by the TS receiving means corresponding to the other performance information to the reference timing based on the reception timing of the other performance information.

2. the reference timing setting means sets the timing of receiving the first performance information as the reference timing when the performance information receiving means receives the first performance information, 2. The electronic device according to claim 1, wherein the output scheduled timing setting means, when the performance information receiving means receives the first performance information, sets the reception timing of the first performance information as the output scheduled timing of the first performance information.

3. an offset time acquisition means for acquiring an offset time, which is a time difference between the scheduled output timing set by the scheduled output timing setting means and the timing of receiving performance information corresponding to the scheduled output timing; an offset time storage means for storing a time based on the offset time acquired by the offset time acquisition means as a delay offset time, the reference timing setting means, when receiving initial performance information from the performance information receiving means, sets the reference timing to a timing obtained by adding the delay offset time stored in the offset time storage means to a receiving timing of the initial performance information, 3. The electronic device according to claim 2, wherein when the performance information receiving means receives the first performance information, the output scheduled timing setting means sets the output scheduled timing of the first performance information to a timing obtained by adding the delay offset time stored in the offset time storage means to the reception timing of the first performance information.

4. 4. The electronic device according to claim 3, wherein the offset time storage means stores the longest offset time among the offset times acquired by the offset time acquisition means as the delay offset time.

5. an offset time acquisition means for acquiring an offset time, which is a time difference between the scheduled output timing set by the scheduled output timing setting means and the timing of receiving performance information corresponding to the scheduled output timing; an offset time storage means for storing a time based on the offset time acquired by the offset time acquisition means as a delay offset time, the reference timing setting means, when receiving initial performance information by the performance information receiving means, sets the reference timing to a timing obtained by adding a time shorter than the delay offset time stored in the offset time storage means to a reception timing of the initial performance information, 3. The electronic device according to claim 2, wherein when the performance information receiving means receives the first performance information, the output scheduled timing setting means sets the output scheduled timing of the first performance information to a timing obtained by adding a time shorter than the delay offset time stored in the offset time storage means to the reception timing of the first performance information.

6. The first performance information is dummy performance information that does not include information regarding sound production, 2. The electronic device according to claim 1, wherein the other performance information is performance information that includes information related to sound production.

7. 2. The electronic device according to claim 1, wherein the reference timing setting means, when receiving other performance information by the performance information receiving means, sets the reference timing to a timing earlier than the reception timing of the other performance information.

8. 8. The electronic device according to claim 7, wherein, when the scheduled output timing set by the scheduled output timing setting means is before the reception timing of the other performance information, the reference timing setting means sets the reception timing of the other performance information to a timing advanced by an amount of time based on a time difference between the reception timing and the scheduled output timing.

9. An output program that causes a computer to execute a process of outputting received performance information, a performance information receiving step for receiving performance information from an external device; a TS receiving step for receiving, from the external device, a timestamp relating to the execution timing of the performance information received in the performance information receiving step; a reference timing setting step of setting a timing based on a reception timing of one piece of performance information as a reference timing when the performance information receiving step receives the one piece of performance information; a scheduled output timing setting step of setting a timing obtained by adding a time difference between a time stamp received in the TS receiving step corresponding to the other performance information and a time stamp received in the TS receiving step corresponding to the first performance information to the reference timing set in the reference timing setting step as a scheduled output timing for outputting other performance information received in the performance information receiving step after the first performance information, the reference timing setting step sets a timing based on the reception timing of the other performance information as the reference timing when the scheduled output timing set in the scheduled output timing setting step is before the reception timing of the other performance information, The output program is characterized in that, when the reference timing is set based on the reception timing of the other performance information by the reference timing setting step, the planned output timing of subsequent performance information received in the performance information receiving step after the other performance information is set to a timing obtained by adding the time difference between the timestamp received in the TS receiving step corresponding to the subsequent performance information and the timestamp received in the TS receiving step corresponding to the other performance information to the reference timing based on the reception timing of the other performance information.

10. a performance information receiving step for receiving performance information from an external device; a TS receiving step for receiving, from the external device, a timestamp relating to the execution timing of the performance information received in the performance information receiving step; a reference timing setting step of setting a timing based on a reception timing of one piece of performance information as a reference timing when the performance information receiving step receives the one piece of performance information; an expected output timing setting step for setting a timing obtained by adding a time difference between a time stamp received in the TS receiving step corresponding to the other performance information and a time stamp received in the TS receiving step corresponding to the first performance information to the reference timing set in the reference timing setting step, as an expected output timing for outputting other performance information received in the performance information receiving step after the first performance information, the reference timing setting step sets a timing based on the reception timing of the other performance information as the reference timing when the scheduled output timing set in the scheduled output timing setting step is before the reception timing of the other performance information, The output method is characterized in that, when the reference timing is set based on the reception timing of the other performance information by the reference timing setting step, the planned output timing of subsequent performance information received in the performance information receiving step after the other performance information is set to a timing obtained by adding the time difference between the timestamp received in the TS receiving step corresponding to the subsequent performance information and the timestamp received in the TS receiving step corresponding to the other performance information to the reference timing based on the reception timing of the other performance information.

Images