Electronic device, output program, and output method

Abstract

When an MIDI message is received later than a predetermined output time, the MIDI message is made to pronunciate immediately, and the reception timing and timestamp of the MIDI message are reset as a reference time and a reference timestamp. Thus, the time difference between the pronunciation timing of the subsequent MIDI message and the reference time point and the reference timestamp can be made to be based on the time difference of the timestamps of the MIDI messages, and thus the sense of discomfort of the listener with respect to the pronunciation timing of the subsequent MIDI message can be suppressed. Furthermore, by outputting the first MIDI message immediately after reception, it is possible to suppress a delay in time from the start of transmission of the MIDI message by the portable terminal 50 to the start of pronunciation by the combiner 1. As a result, the occurrence of a delay in the MIDI message can be suppressed.

Description

Technical Field

[0001] The present invention relates to an electronic device, an output program, and an output method. Background Art

[0002] In Patent Document 1, the following electronic musical instrument 10 is disclosed: receiving a Musical Instrument Digital Interface (MIDI) message from a device 30 including a portable terminal via wireless communication, and causing the received MIDI message to sound. Specifically, the MIDI message and a timestamp of a moment related to the execution timing of the MIDI message are sent from the device 30 to the electronic musical instrument 10 together.

[0003] In the electronic musical instrument 10, when the first MIDI message is received from the device 30, the MIDI message sounds at a moment obtained by adding a predetermined offset time to the moment when the MIDI message is received, and further this moment is set as the "reference moment". The subsequent MIDI messages sound at a moment obtained by adding the time difference between the timestamp of the first MIDI message and the timestamp of the MIDI message to be sounded to the reference moment.

[0004] Thus, in the electronic musical instrument 10, a delay of the offset time is set from when the MIDI message is received until it sounds, and thus it is possible to perform sounding such that even when jitter (fluctuation) or delay (lag) occurs in the communication of the MIDI message, the time difference from the previously sounded MIDI message can be maintained.

[0005] Prior Art Documents

[0006] Patent Documents

[0007] Patent Document 1: Japanese Patent Laid-Open No. 2022-96035 (for example, paragraphs 0073 - 0079, Figure 7 C, Figure 7 D) Summary of the Invention

[0008] Problems to be Solved by the Invention

[0009] Here, in the electronic musical instrument 10 of Patent Document 1, in Figure 7 C, the subsequent MIDI messages sound at a moment (hereinafter referred to as "output scheduled moment") obtained by adding the time difference between the timestamp of the MIDI message to be sounded and the timestamp of the first MIDI message to the reference moment based on the first MIDI message.

[0010] For example, if a MIDI message M1 to be pronounced is received later than the output predetermined time due to jitter or delay, the MIDI message M1 is pronounced immediately after being received. Further, if a MIDI message M2 immediately following the MIDI message M1 is received earlier than the output predetermined time, the MIDI message M2 is pronounced at the output predetermined time. Therefore, due to the reception lag of the MIDI message M1, the time difference from the pronunciation of the MIDI message M1 to the pronunciation of the MIDI message M2 is shorter than the time difference of the time stamps thereof, and there is a concern that the listener may feel a sense of disharmony such as the pronunciation timing of the MIDI message M2 being fast.

[0011] Furthermore, in Figure 7 D, after the first MIDI message arrives at the receiving side, it waits only for the offset time and then pronounces. Then, the output predetermined time of the subsequent MIDI messages is set to the time after adding the time difference between the time stamp of the MIDI message to be pronounced and the time stamp of the first MIDI message to the reference time based on the pronunciation timing of the first MIDI message. Thus, even if jitter occurs, the interval between the pronunciations of each MIDI message can be maintained. However, if the offset time is set to a fixed value, there is a problem point that there is a concern about an excessive delay that could have been avoided.

[0012] The present invention is made to solve the above problem points, and an object thereof is to provide an electronic device, an output program, and an output method that can suppress the sense of disharmony of the listener with respect to the pronunciation timing and can suppress the occurrence of delay even when outputting a MIDI message received from an external device.

[0013] Technical means for solving the problem

[0014] To achieve the above object, the electronic device of the present invention includes: a performance information receiving unit that receives performance information from an external device; a time stamp (TS) receiving unit that receives a time stamp related to the execution timing of the performance information received by the performance information receiving unit from the external device; a reference timing setting unit that, when a performance information is received at a predetermined timing by the performance information receiving unit, sets the timing based on the reception timing of the one performance information as a reference timing; and an output predetermined timing setting unit that sets, as an output predetermined timing, which is the timing for outputting other performance information received by the performance information receiving unit, the timing after adding the time difference between the time stamp received by the TS receiving unit corresponding to the other performance information and the time stamp received by the TS receiving unit corresponding to the one performance information to the reference timing set by the reference timing setting unit. When the output predetermined timing set by the output predetermined timing setting unit is before the reception timing of the other performance information, the reference timing setting unit sets the timing based on the reception timing as the reference timing.

[0015] The output program of the present invention is a program that causes a computer to execute a process of outputting received performance information, and causes the computer to execute: a performance information reception step of receiving performance information from an external device; a TS reception step of receiving a time stamp related to the execution timing of the performance information received in the performance information reception step from the external device; a reference timing setting step of setting, when a piece of performance information is received at a specified timing through the performance information reception step, the timing based on the reception timing of the piece of performance information as a reference timing; and an output predetermined timing setting step of setting, as an output predetermined timing which is the timing of outputting other performance information received in the performance information reception step, the timing obtained by adding the time difference between the time stamp received in the TS reception step corresponding to the other performance information and the time stamp received in the TS reception step corresponding to the piece of performance information to the reference timing set in the reference timing setting step. When the output predetermined timing set in the output predetermined timing setting step is before the reception timing of the other performance information, the reference timing setting step sets the timing based on the reception timing as the reference timing.

[0016] In addition, the output method of the present invention includes: a performance information reception step of receiving performance information from an external device; a TS reception step of receiving a time stamp related to the execution timing of the performance information received in the performance information reception step from the external device; a reference timing setting step of setting, when a piece of performance information is received at a specified timing through the performance information reception step, the timing based on the reception timing of the piece of performance information as a reference timing; and an output predetermined timing setting step of setting, as an output predetermined timing which is the timing of outputting other performance information received in the performance information reception step, the timing obtained by adding the time difference between the time stamp received in the TS reception step corresponding to the other performance information and the time stamp received in the TS reception step corresponding to the piece of performance information to the reference timing set in the reference timing setting step. When the output predetermined timing set in the output predetermined timing setting step is before the reception timing of the other performance information, the reference timing setting step sets the timing based on the reception timing as the reference timing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 (a) is an external view of a synthesizer and a portable terminal, (b) is a diagram showing the timing of MIDI messages transmitted from the portable terminal, and (c) is a diagram showing the timing of reception and sound generation of MIDI messages in the synthesizer.

[0018] Figure 2 is a functional block diagram of the synthesizer.​​

[0019] Figure 3 (a) is a block diagram showing the electrical structure of the synthesizer, and (b) is a diagram schematically showing the delay output buffer.

[0020] Figure 4 (a) is a flowchart of the main processing of the synthesizer, and (b) is a flowchart of the timer processing.

[0021] Figure 5 (a) is a diagram explaining the offset time of the second embodiment, (b) is a diagram showing the timing of the MIDI message transmitted from the portable terminal in the second embodiment, (c) is a diagram showing the timing of receiving and sounding the MIDI message of the synthesizer in the second embodiment, and (d) is a diagram schematically showing the flash ROM in the synthesizer of the second embodiment.

[0022] Figure 6 is a flowchart of the main processing of the synthesizer in the second embodiment.

[0023] Figure 7 (a) is a flowchart of the main processing of the portable terminal in the modification, and (b) is a diagram explaining the advancement of the reference time in the modification. Detailed Embodiments

[0024] Hereinafter, preferred embodiments will be described with reference to the accompanying drawings. Refer to Figure 1 (a) explains the outline of the synthesizer 1 of the present embodiment. Figure 1 (a) is a diagram explaining the outline of the synthesizer 1 and the portable terminal 50.

[0025] The synthesizer 1 is a device (electronic musical instrument or general electronic device) that generates musical sounds based on performance information, i.e., Musical Instruments Digital Interface (MIDI) messages, received from the performance of the user H or external devices such as the portable terminal 50 described later. A keyboard 2 for acquiring operation information based on the operation of the user H and a setting button 3 for the user H to input various settings are provided in the synthesizer 1. In addition, the synthesizer 1 is connected to the portable terminal 50 via wireless communication such as Wi-Fi and Bluetooth (both are registered trademarks).

[0026] The portable terminal 50 is an information processing device (computer) that performs processing corresponding to an instruction input from the user H. It is configured to be able to transmit MIDI messages corresponding to an instruction from the user H to an external device. Here, refer to Figure 1 ​​​​​(b) and (c) are used to illustrate the transmission of MIDI messages using the portable terminal 50 and the sounding of the MIDI messages received from the portable terminal 50 by the synthesizer 1.

[0027] Figure 1 (b) is a diagram showing the timing of MIDI messages transmitted from the portable terminal 50. Figure 1 The horizontal axis in (b) represents time, and as will be described later, Figure 1 (c) and Figure 5 、 7 are the same. As shown in Figure 1 (b), MIDI messages are transmitted from the portable terminal 50 at transmission timings P0 to P5 when the time is from T0 to T5.

[0028] For simplicity of explanation, in the present embodiment, the time between time T0 and time T1, the time between time T1 and time T2, the time between time T2 and time T3, the time between time T3 and time T4, and the time between time T4 and time T5 are each set to a fixed time (for example, 100 milliseconds), but the intervals between each time may also be different. In addition, the MIDI messages transmitted from the portable terminal 50 are not limited to these six transmission timings P0 to P5, and may be less than six or more than six.

[0029] In addition, the MIDI messages and the timestamps including information on the time when the MIDI messages are transmitted are also transmitted to the synthesizer 1 respectively. For example, information related to time T0 is set in the timestamp of the MIDI message at transmission timing P0, and information related to time T3 is set in the timestamp of the MIDI message at transmission timing P3. Details will be described later, but the timestamp is used to control the timing of sounding the corresponding MIDI message in the synthesizer 1.

[0030] The MIDI messages transmitted from the portable terminal 50 and the corresponding timestamps are received by the synthesizer 1, and the corresponding musical tones are sounded from the synthesizer 1. Refer to Figure 1 (c) to describe the reception and sounding timing of MIDI messages in the synthesizer 1.

[0031] Figure 1 (c) is a diagram showing the reception and sounding timing of MIDI messages in the synthesizer 1. The MIDI messages transmitted from the portable terminal 50 at transmission timings P0 to P5 are received by the synthesizer 1 at reception timings Pr0 to Pr5 respectively. Reception timings Pr0 to Pr5 correspond to times Tr1 to Tr5 respectively.

[0032] These MIDI messages are transmitted from the portable terminal 50 to the synthesizer 1 via wireless communication, so fluctuations in communication, i.e., "jitter", or delays in communication, i.e., "latency", occur according to the communication conditions. Therefore, the reception timings Pr0 to Pr5 also have a time deviation compared to the Figure 1 transmission timings P0 to P5 described in (b).

[0033] For example, the reception timing Pr1 is delayed due to latency. Thus, the time difference Tr1 - Tr0 between the reception timing Pr1 and the previous reception timing Pr0 is greater than the time difference T1 - T0 between the corresponding transmission timings P1 and P0. On the other hand, the reception timing Pr2 fluctuates due to jitter. Thus, the time difference Tr2 - Tr1 between the reception timing Pr2 and the previous reception timing Pr1 is less than the time difference T2 - T1 between the corresponding transmission timings P2 and P1.

[0034] In this way, since the reception timings Pr0 to Pr5 have a time deviation compared to the transmission timings P0 to P5, even if the MIDI messages are made to sound at the reception timings Pr0 to Pr5 directly, they will not sound at the time differences based on the original transmission timings P0 to P5 transmitted from the portable terminal 50. Therefore, in the present embodiment, by setting a reference time that serves as the reference for the timing at which these are made to sound corresponding to the reception timings Pr0 to Pr5, the deviation from the time difference based on the original transmission timings P0 to P5 is suppressed.

[0035] Specifically, when the synthesizer 1 receives the first MIDI message (i.e., the reception timing Pr0) from the portable terminal 50, the moment Tr0 corresponding to the reception timing Pr0 is set as the moment at which the MIDI message is made to sound, i.e., the output predetermined moment Ts0. That is, the output predetermined moment Ts0 is set as the sounding timing Ps0 for the first MIDI message.

[0036] In the synthesizer 1, when the set output predetermined moment is before the current moment, the corresponding MIDI message is made to sound. Therefore, for the first MIDI message, since the moment Tr0 corresponding to the reception timing Pr0 is set as the output predetermined moment Ts0, the message is made to sound immediately after reception. By making the first MIDI message sound immediately after reception, the time lag from the start of transmission of the MIDI message using the portable terminal 50 to the start of sounding using the synthesizer 1 can be suppressed. Thereby, the sense of disharmony of the listener regarding the start of sounding of the MIDI message can be suppressed.

[0037] In addition, in the present embodiment, the so-called "initial MIDI message" refers to a MIDI message received for the first time from the portable terminal 50 after the power of the synthesizer 1 is turned on, and a MIDI message received from the portable terminal 50 after a predetermined time (for example, 15 seconds) or more has elapsed since the synthesizer 1 received the last MIDI message from the portable terminal 50.

[0038] In this way, the times Tr0 and Ts0 at which the initial MIDI message is received and sounded are set as the reference times (reference timings) that serve as the reference for the sounding timings of subsequent MIDI messages, and the timestamp of the initial MIDI message is set as the reference timestamp that serves as the reference for the sounding timing.

[0039] The sounding timings of subsequent MIDI messages are set to the times obtained by adding the time difference between the timestamp of the subsequent MIDI message and the timestamp of the initial MIDI message (which is the reference timestamp) to the reference times Tr0 and Ts0 (which are the reference times).

[0040] As a result, the time difference between the sounding timing of the initial MIDI message set as the reference time and the sounding timing of the subsequent MIDI message can be made equal to the time difference between the timestamps of the corresponding MIDI messages transmitted from the portable terminal 50. Therefore, the subsequent MIDI messages can be sounded while maintaining an appropriate time difference from the initial MIDI message set as the reference time. The detailed situation will be described later, but the reference time and the reference timestamp are reset according to the time relationship between the reception timing of the MIDI message and the output scheduled time.

[0041] The second MIDI message immediately following the initial MIDI message is received at the time Tr1 which is the reception timing Pr1. The output scheduled time Ts1 of the second MIDI message is set to the time obtained by adding the time difference T1 - T0 between the transmission timing P1 (the time T1) of the second MIDI message and the reference timestamp (the timestamp of the initial MIDI message) to the reference time (i.e., the time Tr0 which is the reception timing Pr0 of the initial MIDI message).

[0042] Incidentally, the output scheduled time Ts1 set in this way is set to a time before the time Tr1 which is the reception timing Pr1 of the second MIDI message. Since the MIDI message is sounded when the output scheduled time is before the current time, the second MIDI message is immediately sounded at the time Tr1 which is the reception timing Pr1. This time is set as the output scheduled time Ts1' and is set as the sounding timing Ps1 of the second MIDI message.

[0043] That is, the second MIDI message is sounded later than the output scheduled time Ts1 set based on the reference time and reference timestamp of the initial MIDI message. Thus, it is possible to suppress a situation where the sounding of the second MIDI message received after the initially envisioned output scheduled time Ts1 is further delayed.

[0044] However, after the sounding of the second MIDI message is delayed in this way, if the output scheduled time is set for the third MIDI message while keeping the reference time and reference timestamp based on the state of the initial MIDI message, the time difference from the sounding of the second MIDI message to the output scheduled time of the third MIDI message is shorter than the time difference T2 - T1 between the corresponding transmission timings P1 and P2. Thus, there is a concern that the listener may feel that the third MIDI message is sounded too quickly, creating a sense of disharmony with the sounding of the third MIDI message.

[0045] Therefore, in the present embodiment, the output scheduled time is reset to the reference time at the reception timing of the MIDI message whose reception timing is before the reception timing and whose sounding is delayed, and the timestamp of the MIDI message is reset to a new reference timestamp.

[0046] In the second MIDI message, the time Tr1 and time Ts1' after receiving and sounding the second MIDI message are set as the new reference time, and the timestamp of the second MIDI message is set as the reference timestamp. The output scheduled times of the third and subsequent MIDI messages are set using the reference time and reference timestamp reset based on the second MIDI message.

[0047] The output scheduled time Ts2 of the third MIDI message immediately following the second MIDI message is set as the time after adding the time difference T2 - T1 between the time T2 of the transmission timing P2 of the third MIDI message and the reference timestamp (i.e., the timestamp of the second MIDI message) to the reference time (i.e., the time Tr1 and time Ts1' at the reception timing Pr1 of the second MIDI message).

[0048] In addition, the third MIDI message is received at the time Tr2 of the reception timing Pr2. Since the time Tr2 is set to a timing earlier than the set output scheduled time Ts2, the third MIDI message is sounded at the output scheduled time Ts2, and the output scheduled time Ts2 is set as the sounding timing Ps2. Thus, it is possible to maintain the time difference T2 - T1 from the transmission timing P1 of the second MIDI message and sound the third MIDI message.

[0049] Similarly to the second MIDI message, the fourth MIDI message immediately following the third MIDI message is received at a timing later than the output scheduled time Ts3 of the fourth MIDI message. Therefore, similarly to the second MIDI message, the fourth MIDI message is sounded at the sounding timing Ps3 at the time Tr3. After the fourth MIDI message is received and sounded, the time Tr3 and the time Ts3' are reset to the reference time, and the timestamp of the fourth MIDI message is set to the reference timestamp.

[0050] In addition, similarly to the third MIDI message, the fifth MIDI message immediately following the fourth MIDI message is received at a timing earlier than the output scheduled time Ts4 of the fifth MIDI message, and thus is sounded at the output scheduled time Ts4. Similarly, the sixth MIDI message immediately following the fifth MIDI message is received at a timing earlier than the output scheduled time Ts5 of the sixth MIDI message, and thus the sixth MIDI message is sounded at the output scheduled time Ts5.

[0051] Thus, in the case where a MIDI message is received at a timing later than the output scheduled time after adding the time difference between the timestamp of the MIDI message and the reference timestamp to the reference time, the MIDI message is immediately sounded, and the reception timing and the timestamp of the MIDI message are reset to the reference time and the reference timestamp.

[0052] Then, the output scheduled times of the subsequent MIDI messages of the MIDI message are set based on the reset reference time and reference timestamp. Thereby, the time difference between the sounding timings of the subsequent MIDI messages and the sounding timing of the MIDI message used in the resetting of the reference time and reference timestamp can be made based on the time difference of the timestamps of these MIDI messages, and thus the sense of incongruity of the listener with respect to the sounding timing of the subsequent MIDI messages can be suppressed.

[0053] Next, refer to Figure 2 to describe the functions of the synthesizer 1. Figure 2 is a functional block diagram of the synthesizer 1. As Figure 2 shown, the synthesizer 1 includes a performance information receiving unit 300, a TS receiving unit 301, a reference timing setting unit 302, and an output scheduled timing setting unit 303.

[0054] The performance information receiving unit 300 is a unit that receives MIDI messages from the portable terminal 50 and is composed of Figure 3It is implemented by the central processing unit (CPU) 10 described later. The TS reception unit 301 is a unit that receives a timestamp related to the execution timing of the MIDI message received by the performance information reception unit 300 from the portable terminal 50, and is implemented by the CPU 10.

[0055] The reference timing setting unit 302 is a unit that, when a MIDI message is received at a specified timing by the performance information reception unit 300, sets the timing based on the reception timing of the one MIDI message as the reference time, and is implemented by the CPU 10. The output predetermined timing setting unit 303 is a unit that sets, as the pronunciation (output) timing of other MIDI messages received by the performance information reception unit 300, i.e., the output predetermined time, the timing after adding the time difference between the timestamp received by the TS reception unit 301 corresponding to the other MIDI message and the timestamp received by the TS reception unit 301 corresponding to the one MIDI message to the reference time set by the reference timing setting unit 302, and is implemented by the CPU 10.

[0056] In addition, when the output predetermined time set by the output predetermined timing setting unit 303 is before the reception timing of other MIDI messages, the reference timing setting unit 302 sets the timing based on the reception timing as the reference time, and further, when the first MIDI message is received by the performance information reception unit 300, sets the reception timing of the first MIDI message as the reference time. In addition, the output predetermined timing setting unit 303 sets the reception timing of the first MIDI message as the output predetermined timing of the first MIDI message.

[0057] Receives MIDI messages from the portable terminal 50 and receives timestamps related to the execution timing of the received MIDI messages. When a MIDI message is received at a specified timing, sets the timing based on the reception timing of the one MIDI message as the reference time. Then, as the output predetermined time for sounding other MIDI messages, is set to the timing after adding the time difference between the timestamp corresponding to the other MIDI message and the timestamp corresponding to the one MIDI message to the reference time. Here, when the output predetermined time is before the reception timing of the corresponding MIDI message, sets the timing based on the reception timing as the reference time.

[0058] That is, in the case where there is a delayed MIDI message received at a timing later than the output predetermined time, the reference time is reset based on the delayed MIDI message. Further, the output predetermined time of the subsequent MIDI received after the delayed MIDI message is set based on the reset reference time. As a result, the time difference between the pronunciation timings of the subsequent MIDI messages and the pronunciation timing of the delayed MIDI message can be made based on the time difference of these timestamps, and thus the sense of discomfort of the listener with respect to the pronunciation timing of the subsequent MIDI messages can be suppressed.

[0059] In addition, in the case where the first MIDI message is received by the performance information receiving unit 300, the MIDI message is pronounced immediately. As a result, the time lag from the start of transmission of the MIDI message from the portable terminal 50 to the start of pronunciation of the MIDI message using the synthesizer 1 can be suppressed, and thus the delay of the MIDI message can be suppressed.

[0060] Next, refer to Figure 3 the electrical configuration of the synthesizer 1 will be described. Figure 3 (a) is a block diagram showing the electrical configuration of the synthesizer 1. The synthesizer 1 includes a CPU 10, a flash read only memory (ROM) 11, a random access memory (RAM) 12, a real time clock (RTC) 13 that measures the date and time, the keyboard 2 and the setting button 3, a sound source 14, a digital signal processor (DSP) 15, and a wireless communication device 20 for wireless communication with the portable terminal 50, which are respectively connected via a bus 16. A digital analog converter (DAC) 17 is connected to the DSP 15, an amplifier 18 is connected to the DAC 17, and a speaker 19 is connected to the amplifier 18.

[0061] The CPU 10 is an arithmetic device that controls each unit connected via the bus 16. The flash ROM 11 is a rewritable nonvolatile memory and has a tone output program 11a. When the tone output program 11a is executed by the CPU 10, the main process of Figure 4 (a) is executed.

[0062] The RAM 12 is a memory that can rewrite and store various workpiece data, flags, etc. when the CPU 10 executes programs such as the tone output program 11a, and has a delay output buffer 12a, a reference time memory 12b that stores the reference time, and a reference TS memory 12c that stores the reference timestamp. Refer to Figure 3(b) Description of the delay output buffer 12a.

[0063] Figure 3 (b) is a diagram schematically showing the delay output buffer 12a. As Figure 3 (b) shows, the output scheduled time and the MIDI message corresponding to the output scheduled time are respectively stored in the delay output buffer 12a.

[0064] Return Figure 3 (a). The sound source 14 is a device that outputs waveform data corresponding to the information input from the CPU 10. The DSP 15 is an arithmetic device for performing arithmetic processing on the waveform data input from the sound source 14. The DAC 17 is a conversion device that converts the waveform data input from the DSP 15 into analog waveform data. The amplifier 18 is an amplification device that amplifies the analog waveform data output from the DAC 17 with a specified gain. The speaker 19 is an output device that plays (outputs) the analog waveform data amplified by the amplifier 18 in the form of musical sounds.

[0065] Next, refer to Figure 4 to describe the processing executed by the CPU 10 of the synthesizer 1. Figure 4 (a) is a flowchart of the main processing of the synthesizer 1. The main processing is the processing executed when the power of the synthesizer 1 is turned on. Regarding the main processing, first, it is confirmed whether a MIDI message and a time stamp are received from the portable terminal 50 via the wireless communication device 20 (S1).

[0066] In the process of S1, when a MIDI message and a time stamp are received (S1: Yes), the received MIDI message and time stamp are acquired (S2). After the process of S2, it is confirmed whether the received MIDI message is a first MIDI message (S3).

[0067] In the process of S3, when the received MIDI message is a first MIDI message (S3: Yes), the current time, that is, the reception timing of the MIDI message, is set as the output scheduled time (S4). In addition, the current time is acquired from the RTC 13. After the process of S4, the output scheduled time set in the process of S4 is stored in the reference time memory 12b as a reference time, and the time stamp acquired in the process of S2 is stored in the reference TS memory 12c as a reference time stamp (S5).

[0068] On the other hand, in the process of S3, when the received MIDI message is not the initial MIDI message (S3: No), the time obtained by adding the time difference between the timestamp acquired in the process of S2 and the reference timestamp in the reference TS memory 12c to the reference time in the reference time memory 12b is set as the output scheduled time (S6). After the process of S6, it is confirmed whether the current time (i.e., the reception timing) is after the output scheduled time set in the process of S6 (S7). In the process of S7, when the output scheduled time set in the process of S6 is before the current time (S7: Yes), since the MIDI message is received later than the output scheduled time, the processes below S4 are executed.

[0069] After the process of S5, or when the output scheduled time in the process of S6 is later than the current time in the process of S7 (S7: No), the MIDI messages set in S4 and S6 and the MIDI message acquired in the process of S2 are added to the delay output buffer 12a (S8). The MIDI messages added to the delay output buffer 12a are sounded in the Figure 4 timer process described later in (b).

[0070] In the process of S1, when no MIDI message and timestamp are received (S1: No), or after the process of S8, other processes of the synthesizer 1 are executed (S9), and the processes below S1 are repeated.

[0071] Next, the timer process will be described with reference to Figure 4 (b). Figure 4 (b) is the flowchart of the timer process. The timer process is a process executed every certain time (e.g., 50 milliseconds) in the CPU 10 of the synthesizer 1.

[0072] Regarding the timer process, first, it is confirmed whether there is a MIDI message in the delay output buffer 12a whose output scheduled time is before the current time (S20). In the process of S20, when there is a MIDI message in the delay output buffer 12a whose output scheduled time is before the current time (S20: Yes), the corresponding MIDI message is sounded (S21). Specifically, the waveform data corresponding to the corresponding MIDI message is obtained from the sound source 14, and the obtained waveform data is output to the DSP 15, DAC 17, amplifier 18, and speaker 19, thereby sounding in the form of music.

[0073] After the process of S21, the MIDI message that has been sounded in the process of S21 and the output scheduled time corresponding to the MIDI message are deleted from the delay output buffer 12a (S22). In the process of S20, when there is no MIDI message in the delay output buffer 12a whose output scheduled time is before the current time (S20: No), or after the process of S22, the timer process ends.

[0074] Next, refer to Figure 5 、 6 The second embodiment will be described. In the synthesizer 1 of the first embodiment, when the output scheduled time of the MIDI message is before the reception timing of the MIDI message, the reception timing is set as the reference time, and the time stamp corresponding to the MIDI message is set as the reference time stamp.

[0075] In the synthesizer 100 of the second embodiment, in addition to obtaining the above content, the offset time, which is the time difference between the reception timing and the sounding timing of each MIDI message, is also obtained. The value obtained by adding the longest offset time among the obtained offset times to the reception timing of the next first MIDI message is set as the output scheduled time and the reference time of the first MIDI message. The same parts as those in the first embodiment are denoted by the same reference numerals, and their descriptions are omitted.

[0076] Figure 5 (a) is a diagram for explaining the offset time in the second embodiment. In Figure 5 (a), the case where the reception and sounding timings of the MIDI message described in Figure 1 (c) are equal is illustrated. In Figure 5 (a), when the reception timing is before the output scheduled time, such as in the third MIDI message, the fifth MIDI message, or the sixth MIDI message, there is a time lag from the reception of the MIDI message to the sounding.

[0077] For example, in the third MIDI message, a time F2 is generated between the output scheduled time Ts2 and the time Tr2 of the reception timing Pr2. In the fifth MIDI message, a time F4 is generated between the output scheduled time Ts4 and the time Tr4 of the reception timing Pr4. In the sixth MIDI message, a time F5 is generated between the output scheduled time Ts5 and the time Tr5 of the reception timing Pr5.

[0078] The times F2, F4, and F5 are set as the "lag times", which are caused by the reception lags of the second MIDI message and the fourth MIDI message that are earlier than these, resulting in the sounding being delayed from the original output scheduled times Ts1 and Ts3 (refer to Figure 1(c)) It is generated by adjusting backward to the output predetermined times Ts1' and Ts3'. That is, the longest time among times F2, F4, and F5, which is time F4, is set as the longest possible lag time in the reception and pronunciation of this series of MIDI messages.

[0079] Therefore, in the second embodiment, the time difference between the output predetermined time and the reception timing in a series of MIDI messages is obtained, and the longest time FT (time F4 in Figure 5 (a)) is added to the reception timing of the first MIDI message in the next series of MIDI messages. Hereinafter, the time difference between the output predetermined time and the reception timing will be referred to as the "offset time".

[0080] The offset time of a series of MIDI messages is obtained until the power of the synthesizer 100 is turned off, or until a specified time (e.g., 15 seconds) has elapsed after receiving the last MIDI message from the portable terminal 50, and the longest offset time FT is set from the obtained offset times.

[0081] Figure 5 (b) is a diagram showing the timing of MIDI messages transmitted from the portable terminal 50 in the second embodiment. Figure 5 (c) is a diagram showing the timing of reception and pronunciation of MIDI messages of the synthesizer 100 in the second embodiment. As Figure 5 (a) shows, the synthesizer 100 receives and pronounces MIDI messages at reception timings Pr0 to Pr5 and then the power is turned off. At this time, the "offset time F4" is set as the longest offset time FT.

[0082] After that, after the power is turned on again, as Figure 5 (b) shows, the portable terminal 50 transmits MIDI messages at transmission timings P30 to P35 (times T30 to T35). At this time, similar to the first embodiment, the MIDI messages and the timestamps of times T30 to T35 when the MIDI messages were transmitted are also transmitted together.

[0083] The MIDI messages and the corresponding timestamps transmitted from the portable terminal 50 at such transmission timings P30 to P35 are received by the synthesizer 100 and pronounced. At this time, as Figure 5As shown in (c), the first MIDI message is sounded at time Ts30 which is the time Tr30 of the reception timing Pr30 plus the longest offset time FT (i.e., time Ts30 is set as the output scheduled time of the first MIDI message), and the time Ts30 is set as the reference time. Furthermore, the timestamp of the first MIDI message is set as the reference timestamp. Similar to the first embodiment, subsequent MIDI messages are sounded at the time obtained by adding the time difference between the timestamp of the received MIDI message and the reference timestamp to the set reference time.

[0084] Here, the reference time is the time obtained by adding the longest offset time FT to the time Tr30 which is the reception timing Pr30 of the first MIDI message, and the longest offset time FT is set as the longest time that may be delayed in the reception and sounding of the previous series of MIDI messages. That is, the longest offset time FT is considered in advance at the reference time, so even if the reception of subsequent MIDI messages is delayed, it is possible to suppress the output scheduled time from being before the reception timing.

[0085] Thereby, it is possible to stably sound MIDI messages based on the time difference of the timestamps transmitted by the portable terminal 50, and thus it is possible to suppress the sense of incongruity of the listener with respect to the sounding timing.

[0086] Next, refer to Figure 5 (d) to describe the electrical configuration of the second embodiment. Figure 5 (d) is a diagram schematically showing the flash ROM 11 in the synthesizer 100 of the second embodiment. In the flash ROM 11 of the synthesizer 100 of the second embodiment, in addition to having the tone output program 11a, it also has a longest offset time memory 11b that stores the longest offset time FT.

[0087] Next, refer to Figure 6 to describe the main processing of the synthesizer 100 of the second embodiment. Figure 6 It is a flowchart of the main processing of the synthesizer 100 in the second embodiment. In the process of S3, when the received MIDI message is the first MIDI message (S3: Yes), instead of the process of S4, the time obtained by adding the longest offset time FT of the longest offset time memory 11b to the current time is set as the output scheduled time (S30). After the process of S30, the process of S5 is executed.

[0088] In the process of S7, when the output scheduled time set in the process of S6 is before the current time (S7: Yes), the current time, that is, the reception timing of the MIDI message, is set as the output scheduled time (S31). The process of S31 is the same as the process of S4.

[0089] On the other hand, in the process of S7, when the output scheduled time set in the process of S6 is after the current time (S7: No), the time from the output scheduled time set in the process of S6 to the current time is set as the offset time (S32). After the process of S32, it is confirmed whether the offset time set in the process of S32 is longer than the longest offset time FT in the longest offset time memory 11b (S33).

[0090] In the process of S33, when the offset time set in the process of S32 after the process of S32 is longer than the longest offset time FT in the longest offset time memory 11b (S33: Yes), the offset time set in the process of S32 is saved in the longest offset time memory 11b (S34).

[0091] In the process of S33, when the offset time set in the process of S32 is less than or equal to the longest offset time FT in the longest offset time memory 11b (S33: No), or after the process of S34, the processes from S8 and below are executed.

[0092] As described above based on the embodiment, it can be easily inferred that various improvements and changes can be made.

[0093] In the above embodiment, a MIDI message for pronunciation (hereinafter referred to as "pronunciation MIDI message") is sent from the portable terminal 50, and a reference time and a reference timestamp are set in the synthesizer 1 and the synthesizer 100 together with the pronunciation of the pronunciation MIDI message. However, it is not limited to this. Before sending the pronunciation MIDI message from the portable terminal 50, a virtual MIDI message that does not include information such as timbre or scale and does not perform pronunciation may be sent, and the received virtual MIDI message may be used in the synthesizer 1 and the synthesizer 100 to set the reference time and the reference timestamp.

[0094] Specifically, in Figure 7In the main process of the portable terminal 50 in the modified example shown in (a), first, wireless communication with the synthesizer 1 and the synthesizer 100 is established (S50), and 1 is set as the counter variable N (S51). After the process of S51, a virtual MIDI message and a timestamp at the moment of sending the virtual MIDI message are sent to the synthesizer 1 and the synthesizer 100 (S52). After the process of S52, a waiting process is performed (S53). In addition, the waiting process in the process of S53 can be a waiting process for a certain time (for example, 100 milliseconds) or a waiting process for a random time.

[0095] After the process of S53, 1 is added to the counter variable N (S54), and it is confirmed whether the counter variable N after the addition is greater than 10 (S55). In the process of S55, when the counter variable N is 10 or less (S55: No), the processes from S52 and below are repeated.

[0096] Through the process of S52, in the synthesizer 1 and the synthesizer 100 that receive the virtual MIDI message sent from the portable terminal 50, through Figure 4 , 6 the processes of S1 to S8 described in are used to set the reference time or reference timing. At this time, when the MIDI message that is the object in the process of S21 in Figure 4 (b) is virtual, since the MIDI message does not contain information such as timbre or scale, it will not sound, but the main idea of receiving the virtual MIDI message can also be displayed on the display units of the synthesizer 1 and the synthesizer 100.

[0097] On the other hand, when the counter variable N is greater than 10 in the process of S55 (S55: Yes), other processes of the portable terminal 50 are repeated (S56). The other processes of S56 also include the process of sending the sounding MIDI message and its timestamp to the synthesizer 1 and the synthesizer 100.

[0098] In this way, a virtual MIDI message is sent before the sounding MIDI message is sent from the portable terminal 50, and before the synthesizer 1 and the synthesizer 100 receive the sounding MIDI message, the reference time or reference timestamp is set through the virtual MIDI message. Thus, in the synthesizer 1 and the synthesizer 100, the sounding MIDI message can be received and sounded using the reference time or reference timestamp that has considered the communication conditions such as jitter or delay between the synthesizer 1 and the synthesizer 100 and the portable terminal 50 from the beginning. Therefore, in the sounding of a series of MIDI messages, the situation where the output predetermined time is before the reception timing and the sounding timing deviates from the timing based on the time difference of the timestamp can be suppressed, so the sounding of the MIDI message can be stably performed.

[0099] In addition, in the processes of S52 to S55, the number of times of sending virtual MIDI messages is not limited to 10, and can be 10 or more or 10 or less. Further, the virtual MIDI messages are not limited to being sent before the sounding MIDI messages. For example, in addition to sending virtual MIDI messages before the sounding MIDI messages, virtual MIDI messages can also be sent during the gaps between the sounding MIDI messages.

[0100] In the above-described embodiment, when the scheduled output time of the MIDI message is before the reception timing, the reference time is reset to the reception timing, but it is not limited thereto. The reference time can also be reset to a time earlier than the reception timing. For example, as Figure 7 (b) shows, when the scheduled output time Ts1 of the MIDI message is before the time Tr1 which is the reception timing Pr1, the reference time can also be set to the time after the time Tr1 by the time difference ΔT between the time Tr1 and the scheduled output time Ts1 multiplied by a prescribed coefficient (e.g., 0.25).

[0101] In this way, by advancing the reference time from the reception timing, it is possible to suppress a situation where the delay of subsequent MIDI messages is unnecessarily increased due to a large jitter (in the time lag direction) that occurs only occasionally. Further, by making the time for advancing the reference time based on the time difference ΔT between the reception timing Pr1 and the scheduled output time Ts1, it is possible to suppress the set reference time from becoming an unnecessarily early timing that is earlier than the scheduled output time Ts1. As a result, it is possible to suppress the frequency at which the reception timing of subsequent MIDI messages is after the scheduled output time, and thus it is possible to make the MIDI messages sound stably.

[0102] In addition, the time for advancing the reference time is not limited to the time based on the time difference ΔT between the reception timing and the scheduled output time. For example, it can also be the smaller one of a fixed time (30 milliseconds) and the time difference ΔT.

[0103] In the above-described embodiment, in Figure 4 the process of S7, when the scheduled output time set in the process of S6 is before the reception timing, the reception timing is reset to the reference time, and the time stamp of the MIDI message is reset to the reference time stamp, but it is not limited thereto. For example, regardless of the time relationship between the scheduled output time and the reception timing, the reception timing can always be set to the reference time, and the time stamp of the MIDI message can be set to the reference time stamp.

[0104] In the above-described embodiment, the timestamp of the MIDI message transmitted from the portable terminal 50 is set to the time when the MIDI message is transmitted, but it is not limited thereto. For example, the timestamp may be set to the time when the sound is produced in the synthesizer 1 or the synthesizer 100 that receives the MIDI message, or the timestamp may be set to a timing (execution timing) related to the output and others of the MIDI message.

[0105] In the second embodiment, as the reference time, the time obtained by adding the longest offset time FT obtained in the reception and sound production of the previous MIDI message to the reception timing of the first MIDI message is set. However, the content added to the reception timing of the first MIDI message in the setting of the reference time is not limited to the longest offset time FT. For example, it may be a time longer than the longest offset time FT or a time shorter than the longest offset time FT. In addition, it may be the average value or the median value of the offset times obtained in the reception and sound production of the previous MIDI message, or other times related to the offset time.

[0106] In addition, the reference time may be set by adding a time shorter than the longest offset time FT obtained by multiplying the longest offset time FT by a prescribed coefficient less than 1, or subtracting a fixed value from the longest offset time FT, etc., to the reception timing of the first MIDI message. Thereby, it is possible to suppress a situation where the delay of subsequent MIDI messages is unnecessarily increased due to a large jitter (in the time lag direction) that occurs only occasionally.

[0107] In the above-described embodiment, the MIDI message is exemplified as the performance information processed in the synthesizer 1 and the synthesizer 100, but it is not limited thereto. Information other than the MIDI message may be used as the performance information.

[0108] In the above-described embodiment, in Figure 4 the process of S21 in (b), the MIDI message is made to produce sound, but it is not limited thereto. For example, together with making the MIDI message produce sound, instead of producing sound, the MIDI message may be transmitted to other electronic devices via the wireless communication device 20, or the MIDI message may be output in other forms.

[0109] In the above-described embodiment, the synthesizer 1 and the synthesizer 100 are connected to the portable terminal 50 by wireless communication, but the communication method is not limited thereto. For example, the synthesizer 1 and the synthesizer 100 may be connected to the portable terminal 50 by wired communication, or may be connected by other communication methods.

[0110] In the described embodiment, the musical tone output program 11a is stored in the flash ROM 11 of the synthesizer 1 and the synthesizer 100 and run on the synthesizer 1 and the synthesizer 100. However, it is not necessarily limited thereto. The musical tone output program 11a may also be run using other electronic musical instruments such as an electronic piano, an electronic organ, or an electronic drum, or may be run on other computers or electronic devices such as a personal computer (PC), a portable terminal, or a tablet terminal.

[0111] In addition, the portable terminal 50 constitutes an external device that sends MIDI messages to the synthesizer 1 and the synthesizer 100. However, it is not limited thereto. An external device may be constituted by other computers such as a PC or a tablet terminal, or may be constituted by an electronic musical instrument such as a synthesizer.

[0112] Explanation of Reference Numerals in the Drawings

[0113] 1, 100: Synthesizer (electronic device)

[0114] 11a: Musical tone output program (output program)

[0115] 11b: Longest offset time memory (a part of the offset time storage unit)

[0116] 50: Portable terminal (external device)

[0117] Pr1 to Pr35: Reception timing

[0118] S1, S2: Performance information reception unit, TS reception unit, performance information reception step, TS reception step

[0119] S5: Reference timing setting unit, reference timing setting step

[0120] S4, S6: Output scheduled timing setting unit, output scheduled timing setting step

[0121] S32: Offset time acquisition unit

[0122] S33, S34: A part of the offset time storage unit

Claims

1. An electronic device, characterized in that it includes: a performance information receiving unit that receives performance information from an external device; a timestamp receiving unit that receives from the external device a timestamp related to the execution timing of the performance information received by the performance information receiving unit; a reference timing setting unit that, when a performance information is received at a specified timing by the performance information receiving unit, sets the timing based on the reception timing of the one performance information as the reference timing; and an output predetermined timing setting unit that, as the output predetermined timing for outputting other performance information received by the performance information receiving unit, sets the timing after adding the time difference between the timestamp received by the timestamp receiving unit corresponding to the other performance information and the timestamp received by the timestamp receiving unit corresponding to the one performance information to the reference timing set by the reference timing setting unit, when the output predetermined timing set by the output predetermined timing setting unit is before the reception timing of the other performance information, the reference timing setting unit sets the timing based on the reception timing as the reference timing.

2. The electronic device according to claim 1, characterized in that when the performance information receiving unit receives the first performance information, the reference timing setting unit sets the reception timing of the first performance information as the reference timing, when the performance information receiving unit receives the first performance information, the output predetermined timing setting unit sets the reception timing of the first performance information as the output predetermined timing of the first performance information.

3. The electronic device according to claim 2, characterized in that it includes: an offset time acquisition unit that acquires the time difference between the output predetermined timing set by the output predetermined timing setting unit and the reception timing of the performance information corresponding to the output predetermined timing, that is, the offset time; and an offset time storage unit that stores the time based on the offset time acquired by the offset time acquisition unit as the delay offset time, when the performance information receiving unit receives the first performance information, the reference timing setting unit sets the timing after adding the delay offset time stored by the offset time storage unit to the reception timing of the first performance information as the reference timing, when the performance information receiving unit receives the first performance information, the output predetermined timing setting unit sets the timing after adding the delay offset time stored by the offset time storage unit to the reception timing of the first performance information as the output predetermined timing of the first performance information.

4. The electronic device according to claim 3, characterized in that the offset time storage unit stores the longest offset time among the offset times acquired by the offset time acquisition unit as the delay offset time.

5. The electronic device according to claim 3 or 4, characterized in that it includes: An offset time acquisition unit acquires a time difference, i.e., an offset time, between an output predetermined timing set by the output predetermined timing setting unit and a reception timing of performance information corresponding to the output predetermined timing. And An offset time storage unit stores, as a delayed offset time, a time based on the offset time acquired by the offset time acquisition unit. When the performance information reception unit receives the first performance information, the reference timing setting unit sets, as a reference timing, a timing after adding a time shorter than the delayed offset time stored by the offset time storage unit to the reception timing of the first performance information. When the performance information reception unit receives the first performance information, the output predetermined timing setting unit sets, as the output predetermined timing of the first performance information, a timing after adding a time shorter than the delayed offset time stored by the offset time storage unit to the reception timing of the first performance information.

6. The electronic device according to claim 1, wherein the performance information reception unit receives, from the external device, virtual performance information that does not have information related to pronunciation before performance information having information related to pronunciation. The timestamp reception unit receives, from the electronic device, a timestamp related to an execution timing of the virtual performance information received by the performance information reception unit. As an output predetermined timing for outputting the other performance information, the output predetermined timing setting unit sets a timing after adding a time difference between the timestamp received by the timestamp reception unit corresponding to the other performance information and the timestamp received by the timestamp reception unit corresponding to the virtual performance information to the reference timing set by the reference timing setting unit. When the output predetermined timing set by the output predetermined timing setting unit is before the reception timing of the virtual performance information, the reference timing setting unit sets, as a reference timing, a timing based on the reception timing.

7. The electronic device according to claim 1, wherein when the performance information reception unit receives other performance information, the reference timing setting unit sets, as a reference timing, a timing earlier than the reception timing of the other performance information.

8. The electronic device according to claim 7, wherein when the output predetermined timing set by the output predetermined timing setting unit is before the reception timing of the corresponding other performance information, the reference timing setting unit sets, as a reference timing, a timing after advancing the reception timing by a time based on a time difference between the reception timing and the output predetermined timing.

9. An output program causes a computer to execute a process of outputting received performance information. The output program is characterized in that it causes the computer to execute: a performance information reception step of receiving performance information from an external device; a timestamp reception step of receiving, from the external device, a timestamp related to an execution timing of the performance information received in the performance information reception step. Reference timing setting step, when a performance information is received at a specified timing through the performance information receiving step, setting the timing based on the reception timing of the one performance information as the reference timing; and Output predetermined timing setting step, as the output timing of other performance information received in the performance information receiving step, i.e., the output predetermined timing, setting the timing after adding the time difference between the timestamp received in the timestamp receiving step corresponding to the other performance information and the timestamp received in the timestamp receiving step corresponding to the one performance information to the reference timing set in the reference timing setting step, when the output predetermined timing set in the output predetermined timing setting step is before the reception timing of the other performance information, the reference timing setting step sets the timing based on the reception timing as the reference timing.

10. An output method, characterized in that it includes: Performance information receiving step, receiving performance information from an external device; Timestamp receiving step, receiving from the external device a timestamp related to the execution timing of the performance information received in the performance information receiving step; Reference timing setting step, when a performance information is received at a specified timing through the performance information receiving step, setting the timing based on the reception timing of the one performance information as the reference timing; and Output predetermined timing setting step, as the output timing of other performance information received in the performance information receiving step, i.e., the output predetermined timing, setting the timing after adding the time difference between the timestamp received in the timestamp receiving step corresponding to the other performance information and the timestamp received in the timestamp receiving step corresponding to the one performance information to the reference timing set in the reference timing setting step, when the output predetermined timing set in the output predetermined timing setting step is before the reception timing of the other performance information, the reference timing setting step sets the timing based on the reception timing as the reference timing.

Images