Device, system, method and program

Abstract

To easily measure wireless performance in real time. [Solution] An electronic device according to one aspect of the present disclosure has a receiving unit that receives measurement data including identification number information and counter value information, and a transmitting unit that transmits return data that is configured by updating the counter value information of the measurement data based on the number of times the measurement data is received.

Description

[Technical Field]

[0001] The present disclosure utilizes wireless communication. Rude The present invention relates to a device, a system, a method and a program. [Background technology]

[0002] The Musical Instrument Digital Interface (MIDI (registered trademark)) is widely used as a format for exchanging data between devices for playing electronic musical instruments. Meanwhile, with the widespread use of wireless communication devices in recent years, technologies for transmitting and receiving MIDI wirelessly are being considered. For example, MIDI over Bluetooth LE (or BLE MIDI), which transmits and receives MIDI using Bluetooth (registered trademark) Low Energy (Bluetooth LE), is being considered.

[0003] Also, a dongle device has been proposed that can realize BLE MIDI simply by inserting it into the MIDI terminal of an existing electronic musical instrument (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-179141 Summary of the Invention [Problem to be solved by the invention]

[0005] MIDI data itself does not contain any time-related data, so the timing at which the device receives it determines the timing at which the instrument is controlled (to produce sound, etc.).

[0006] However, if the wireless communication environment is poor (for example, communication quality is poor), the time it takes for transmitted data to arrive may vary. Furthermore, Bluetooth LE transmits data intermittently to reduce power consumption. Since it is recommended that the timing difference between multiple intermittent transmissions be 15 ms (milliseconds) or less, even in a good wireless environment, a data delay (fluctuation) of 15 ms may occur.

[0007] BLE MIDI and MIDI 2.0 can use timestamps, and jitter can be reduced by synchronizing with timestamps regardless of the timing of reception. However, to achieve sufficient improvement, the receiving device must have a large data buffer, which increases the quantitative delay (latency) and impairs real-time performance. If the trade-off between jitter and latency cannot be achieved, the user's quality of experience will be reduced, but no such method has been considered.

[0008] Therefore, the present disclosure provides a method for easily measuring wireless performance between electronic devices and devices in real time in order to achieve a favorable trade-off between jitter and latency. Rude One of the objects of the present invention is to provide a device, system, method and program. [Means for solving the problem]

[0009] A device according to one aspect of the present disclosure includes: News a transmitting unit configured to transmit measurement data including the measurement data to an electronic device or a communication dongle connected to the electronic device; and a receiving unit configured to update counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or to receive return data in which information regarding the reception time at which the electronic device or the communication dongle received the measurement data is set. and a control unit, wherein when the measurement data includes counter value information, the transmission unit transmits information informing the electronic device or the communication dongle of the start of measurement before transmitting the measurement data, and then the control unit resets the counter value of the counter value information. . [Effects of the Invention]

[0010] According to one aspect of the present disclosure, wireless performance measurements between electronic devices and devices can be easily performed in real time. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of a system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the appearance of the electronic musical instrument 10 according to an embodiment. [Figure 3] FIG. 3 is a diagram showing an example of the hardware configuration of the control system 100 of the electronic musical instrument 10 according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a hardware configuration of the dongle 20 according to an embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a hardware configuration of the device 30 according to an embodiment. [Figure 6] FIG. 6 is a sequence diagram of a jitter control method according to an embodiment. [Figure 7] 7A-7D are conceptual diagrams of correction based on timestamps and offsets. [Figure 8] FIG. 8 is a sequence diagram of a wireless performance measurement method according to an embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a format of measurement data. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, identical parts are designated by the same reference numerals. Since identical parts have the same names, functions, etc., detailed description thereof will not be repeated.

[0013] (system) Fig. 1 is a diagram showing an example of a schematic configuration of a system according to an embodiment. The system 1 shown in Fig. 1 includes an electronic musical instrument 10, a dongle 20, and a device 30. The system 1 may also be called a MIDI transmission (or playback or execution) system, a wireless performance evaluation system, an electronic musical instrument system, or the like.

[0014] The electronic musical instrument 10 is a device for receiving input from a user via controls such as a keyboard and switches, and for controlling performance, etc. The electronic musical instrument 10 may be a device that has the function of generating sounds in accordance with performance information such as MIDI data. The device may be an electronic musical instrument (such as an electronic piano or synthesizer), an analog musical instrument equipped with sensors and configured to have functions equivalent to those of an electronic musical instrument, or an electronic device that does not have controls (such as a keyboard) for performance (such as an electronic device similar to device 30 described below). Note that in this disclosure, the term "electronic device" may refer to both the electronic musical instrument 10 and other electronic devices. In other words, in this disclosure, the term "electronic musical instrument 10" may be interchangeable with the term "electronic device 10."

[0015] The dongle 20 is connected to the electronic musical instrument 10 and relays direct communication with the device 30. In this disclosure, the dongle 20 is described assuming that it is connected (attached) to a Universal Serial Bus (USB) terminal of the electronic musical instrument 10, but this is not limited thereto. If an interface other than USB is used, the USB in this disclosure may be interpreted as referring to that interface. The dongle 20 may also be called an electronic musical instrument communication device.

[0016] The device 30 is an electronic device that communicates with the electronic musical instrument 10 (or the dongle 20). The device 30 may be a mobile terminal (mobile communication terminal) such as a mobile phone, smartphone, or tablet terminal, or may be a fixed communication terminal such as a personal computer (PC), server, television, or game console. In other words, the device 30 in this disclosure may be interchangeably referred to as a communication device, a communication apparatus, a terminal apparatus, or the like.

[0017] <Electronic Musical Instrument> 2 is a diagram showing an example of the appearance of an electronic musical instrument 10 according to an embodiment. The electronic musical instrument 10 may include a switch (button) panel 140b, a keyboard 140k, a display 150d, and a speaker 150s.

[0018] The switch panel 140b may include switches for operating the volume setting, sound source, tone setting, song (accompaniment) selection, song playback start / stop, song playback settings (tempo, etc.), etc.

[0019] The keyboard 140k may have a plurality of keys as performance operators. The keys may be called performance operators, pitch operators, timbre operators, direct operators, etc.

[0020] The display 150d may display lyrics, musical scores, various setting information, etc. The speaker 150s may be used to emit sounds generated by playing.

[0021] The electronic musical instrument 10 may be capable of generating and converting at least one of MIDI messages (events) and Open Sound Control (OSC) messages. In this disclosure, "MIDI" may refer to any of the MIDI 1.0 standard, the MIDI 2.0 standard, and modified / extended MIDI standards. In this disclosure, the terms MIDI message and MIDI data may be interchangeable. A MIDI message may refer to multiple (e.g., several bytes) of MIDI data that function as a single command.

[0022] The electronic musical instrument 10 may communicate with a network (such as the Internet) via at least one of wired and wireless communication (e.g., Long Term Evolution (LTE), 5th generation mobile communication system New Radio (5G NR), Wi-Fi (registered trademark), Bluetooth, etc.).

[0023] FIG. 3 is a diagram showing an example of the hardware configuration of the control system 100 of the electronic musical instrument 10 according to an embodiment.

[0024] A central processing unit (CPU) 101, a ROM (read-only memory) 102, a RAM (random access memory) 103, a sound source 104, a key scanner 106 to which the switch (button) panel 140b and keyboard 140k in Figure 2 are connected, a USB interface 107, and an LCD controller 108 to which a Liquid Crystal Display (LCD) as an example of the display 150d in Figure 2 is connected, are all connected to a system bus 109.

[0025] A timer 110 (which may also be called a counter) for controlling performance may be connected to the CPU 101. The timer 110 may be used, for example, to count the progress of an automatic performance in the electronic musical instrument 10. The CPU 101 may also be called a processor, and may include an interface with peripheral circuits, a control circuit, an arithmetic circuit, a register, etc.

[0026] CPU 101 executes a control program stored in ROM 102 while using RAM 103 as a work memory, thereby performing control operations of electronic musical instrument 10 shown in Fig. 2. In addition to the control program and various fixed data, ROM 102 may also store vocal data, accompaniment data, and song data including these.

[0027] A key scanner (scanner) 106 constantly scans the key-on / key-off states of the keyboard 140k in FIG. 2, the switch operation states of the switch panel 140b, and the like, and issues an interrupt to the CPU 101 to notify it of state changes.

[0028] The LCD controller 108 is an integrated circuit (IC) that controls the display state of an LCD, which is an example of the display 150d.

[0029] The sound source 104 generates a digital sound source signal (for example, waveform data of a musical instrument sound) corresponding to the sound to be generated (note-on) based on note-on / off data input from the CPU 101 via the key scanner 106, and outputs the signal to a Digital to Analog (D / A) converter 111. The sound source 104 may also perform processes such as envelope control of the sound to be generated. The sound source 104 may also perform voice synthesis processing to generate a synthetic voice signal.

[0030] The D / A converter 111 converts the input digital signal into an analog signal and outputs it to the amplifier 112. The amplifier 112 may amplify the input signal and output it from a speaker 150s or an output terminal (not shown).

[0031] The USB interface 107 exchanges signals in accordance with the USB standard with the dongle 20, which is physically connected to the electronic musical instrument 10 using an external connection terminal (for example, a connector). The USB interface 107 may include a USB connector (USB connection terminal).

[0032] When the signal input from the USB interface 107 includes MIDI format data (MIDI data), the CPU 101 may perform playback processing using the sound source 104. Furthermore, the CPU 101 may generate MIDI data based on key input information (e.g., note on / off) acquired from the key scanner 106, output the MIDI data to the USB interface 107, and transmit the data to the device 30 via the dongle 20.

[0033] <Dongle> 4 is a diagram illustrating an example of the hardware configuration of the dongle 20 according to an embodiment. The dongle 20 includes, for example, a USB interface unit 201, an RF unit 202, and an antenna unit 203. Note that the drawings in this disclosure merely illustrate configurations used for explanation, and it will be understood by those skilled in the art that the dongle 20 may include configurations such as a power supply that are not shown.

[0034] The USB interface unit 201 relays signals conforming to the USB standard (for example, signals including MIDI data) between the USB interface 107 of the electronic musical instrument 10 and the RF unit 202. In other words, the USB interface unit 201 has a function of bridging (converting) between signals used in the RF unit and signals conforming to the USB standard.

[0035] For example, the USB interface unit 201 acquires the original data from a signal (packet) transmitted via the USB interface 107 of the electronic musical instrument 10, and transfers it to the RF unit 202. The USB interface unit 201 also includes the data transferred from the RF unit 202 in a USB packet and transmits it to the USB interface 107 of the electronic musical instrument 10.

[0036] The RF unit 202 realizes transmission and reception of signals using wireless communication (for example, Bluetooth, Wi-Fi, etc.).

[0037] The RF unit 202 may perform transmission processing such as channel coding (which may include error correction coding), modulation, mapping, filtering, inverse fast Fourier transform (IFFT), and digital-to-analog conversion on a bit string (e.g., MIDI data) to be transmitted and transferred from the USB interface unit 201, and output a baseband signal. The RF unit 202 may perform modulation to a radio frequency band, filtering, amplification, and the like on the baseband signal, and transmit the radio frequency band signal via the antenna unit 203.

[0038] Meanwhile, the RF unit 202 may perform amplification, filtering, demodulation to a baseband signal, etc. on the radio frequency band signal received by the antenna unit 203. The RF unit 2002 may apply reception processing such as analog-to-digital conversion, Fast Fourier Transform (FFT) processing, filtering, demapping, demodulation, and decoding (which may include error correction decoding) to the acquired baseband signal, acquire transmitted data (e.g., MIDI data), and transfer the data to the USB interface unit 201.

[0039] The RF unit 202 may simultaneously perform the process of converting the received radio signal into the transmitted data and the process of converting the bit string to be transmitted into the radio signal.

[0040] The antenna section 203 can be configured from at least one of antennas described based on common understanding in the technical field to which the present disclosure relates, such as a pattern antenna, a chip antenna, a dipole antenna, an omni-antenna, a whip antenna, etc.

[0041] In the present disclosure, other data (for example, audio data for the accompaniment of a song) may be transmitted and received simultaneously with MIDI via a USB interface or wirelessly. Furthermore, the USB interface unit 201 of the present disclosure may be interpreted as any interface unit capable of connecting to and communicating with the electronic musical instrument 10. In this case, "USB" in the present disclosure may be interpreted as the name of any interface unit, and "a signal conforming to the USB standard" may be interpreted as a signal conforming to the standard to which the any interface unit conforms.

[0042] <device> 5 is a diagram illustrating an example of the hardware configuration of the device 30 according to an embodiment. The device 30 may be physically configured as a computer device including a processor 301, a memory 302, a storage 303, a communication device 304, an input device 305, an output device 306, a bus 307, and the like.

[0043] Each function of device 30 is realized by loading specific software (programs) onto hardware such as processor 301 and memory 302, causing processor 301 to perform calculations and control communication via communication device 304, reading and / or writing of data in memory 302 and storage 303, etc.

[0044] The processor 301 controls the entire computer by running, for example, an operating system, and may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control unit, an arithmetic unit, a register, and the like.

[0045] The processor 301 also reads programs (program codes), software modules, data, etc. from at least one of the storage 303 and the communication device 304 into the memory 302, and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the embodiments of the present disclosure.

[0046] The memory 302 is a computer-readable recording medium and may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically EEPROM (EEPROM), a random access memory (RAM), or other suitable storage medium. The memory 302 may also be referred to as a register, a cache, a main memory, or the like. The memory 302 may store executable programs (program codes), software modules, and the like for implementing a method according to one embodiment.

[0047] Storage 303 is a computer-readable recording medium and may be, for example, at least one of a flexible disk, a floppy disk, a magneto-optical disk (e.g., a compact disk (CD-ROM, etc.), a digital versatile disk, a Blu-ray disc), a removable disk, a hard disk drive, a smart card, a flash memory device (e.g., a card, stick, key drive), a magnetic stripe, a database, a server, or other suitable storage medium. Storage 303 may also be referred to as an auxiliary storage device.

[0048] The communication device 304 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0049] The input device 305 is an input device (for example, a keyboard, a mouse, etc.) that receives input from the outside. The output device 306 is an output device (for example, a display, a speaker, etc.) that outputs to the outside. The input device 305 and the output device 306 may be integrated into one device (for example, a touch panel).

[0050] Furthermore, each device such as the processor 301 and the memory 302 is connected by a bus 307 for communicating information. The bus 307 may be configured as a single bus, or may be configured as different buses between the devices.

[0051] These system configurations and device configurations are merely examples and are not limited to these. For example, the number of each circuit included is not limited to these. Each device may have a configuration that does not include some circuits (mechanisms), or may have a configuration in which the function of one circuit is realized by multiple circuits, or may have a configuration in which the functions of multiple circuits are realized by one circuit.

[0052] Furthermore, the electronic musical instrument 10, dongle 20, and device 30 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by such hardware. For example, the CPU 101 may be implemented by at least one of these pieces of hardware.

[0053] The block diagrams described above show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, there are no particular limitations on the means by which each functional block is realized. That is, each functional block may be realized by a single physically coupled device, or may be realized by two or more physically separate devices connected by wire or wirelessly.

[0054] For example, although only one processor 301 is shown, there may be multiple processors. Furthermore, the processes may be performed by one processor, or the processes may be performed by two or more processors simultaneously, sequentially, or using other techniques. Furthermore, each functional block may be implemented by one or more chips.

[0055] (Latency and Jitter Control Method) A latency and jitter control method according to an embodiment of the present disclosure is described below.

[0056] There is a trade-off between latency and jitter. The inventors have discovered that while data control is performed on the receiving side (electronic musical instrument 10 side), the optimal setting value for the receiving buffer largely depends on the sending side (device 30 side), such as how the application software on the sending side (device 30 side) handles MIDI data, the data structure of the MIDI data, and the performance of device 30.

[0057] Therefore, the inventors came up with the idea of ​​a control method that uses manufacturer-configurable / extensible messages in the MIDI data format to enable the application software of the device 30 to remotely set the electronic musical instrument 10's timestamp control method and optimal offset values ​​for buffering in the electronic musical instrument 10 in real time. This configuration makes it possible to optimize the balance between jitter and latency. Furthermore, the application software of the device 30 can adjust the optimal balance between latency and jitter without the user being aware of it.

[0058] Although the present embodiment discloses a method using system exclusive messages as extensible messages, it is also possible to use NRPNs (Non Registered Parameter Numbers) included in control change messages or assignable controller messages of the MIDI 2.0 standard. In other words, the system exclusive messages in this disclosure may be interchangeably referred to as control change messages, NRPNs, assignable controller messages, etc.

[0059] 6 is a diagram showing an example of a flowchart of a jitter control method according to an embodiment. In the description of the present disclosure, the device 30 may be read as an "application of the device 30" and vice versa.

[0060] The device 30 determines an offset time for buffering MIDI data (step S101). The device 30 may set the offset time to a preset value, a value input by the user, or may determine the offset time based on measurements, which will be described later. For example, if the device 30 determines based on the measurements that the jitter (or delay) between the device 30 and the electronic musical instrument 10 is relatively small (or almost zero), the device 30 may determine the offset time to be a relatively small value (or zero).

[0061] The offset time may be interchangeably referred to as the offset amount, offset, etc. Hereinafter, the offset time will be simply referred to as the offset. The offset may also be referred to as the time related to buffering of MIDI data.

[0062] In step S101, the device 30 may also determine whether or not to apply timestamp processing to the electronic musical instrument 10. For example, if the device 30 determines based on the measurement that the jitter (or delay) between the device 30 and the electronic musical instrument 10 is relatively small (or almost zero), it may determine that timestamp processing is not to be applied, and otherwise it may determine that timestamp processing is to be applied.

[0063] The device 30 transmits information about the determined offset to the electronic musical instrument 10 (step S102). This information about the offset (which may be referred to as offset information) may explicitly indicate the offset time itself (e.g., 15 ms, 30 ms, etc.), or may indicate an index value associated with the offset time. The correspondence between this index value and the offset time may be predefined or may be set. For example, the correspondence may indicate that index=0 indicates offset time=0 (or no timestamp processing), index=1 indicates offset time=15 ms, etc.

[0064] The electronic musical instrument 10 sets the offset based on the received offset information (step S103). If the electronic musical instrument 10 does not receive offset information, it may determine that the offset is a predetermined value (a default value, for example, 0).

[0065] Thereafter, the device 30 transmits the MIDI data to the electronic musical instrument 10 via wireless communication (for example, using BLE) (step S104). The electronic musical instrument 10 buffers the received MIDI data based on the offset set in step S103, and executes (for example, plays, sounds) the MIDI data (step S105). The execution process of step S105 may include any MIDI data-related processing, such as processing a note-on message that starts sound production, processing a note-off message that ends sound production, processing a message that turns a pedal on / off, etc.

[0066] In step S102, information regarding whether or not time stamp processing is performed (which may be called time stamp processing information) may be transmitted instead of or together with the offset information.

[0067] In BLE-MIDI and MIDI 2.0, a timestamp (time information) may be added when transmitting original MIDI data. If the timestamp processing information indicates "Yes," the electronic musical instrument 10 performs correction based on the timestamp in the execution process of step S105. If not, even if a timestamp is added to the received MIDI data, the electronic musical instrument 10 does not perform correction based on the timestamp (the electronic musical instrument 10 ignores the timestamp and executes the process).

[0068] The timestamp may also be referred to as information regarding the execution timing of a MIDI message. In the present disclosure, the terms timestamp and timestamp information may be used interchangeably. The timestamp may be included in the MIDI data, or may be notified to the electronic musical instrument 10 as information separate from the MIDI data. In the latter case, timestamp control based on the present disclosure can be applied even when receiving MIDI 1.0 data, which does not include timestamp information in the standard.

[0069] 7A-7D are conceptual diagrams of correction based on timestamps and offsets.

[0070] Figure 7A shows an example of the timing of notes being played as intended. In the figure, the notes are arranged in the following order: note 0, which is an eighth note long, note 1, which is a quarter note long, and note 2, which is an eighth note long. ts0, ts1, and ts2 correspond to timestamps indicating the times when notes 0, 1, and 2 are played, respectively. Note 1 is played ts1-ts0 after note 0, and note 2 is played ts2-ts0 after note 0.

[0071] Figures 7B-7D show the timing of notes when jitter occurs after the data in Figure 7A is transmitted. The st0, st1, and st2 at the top of each figure indicate the arrival times of notes 0 and 2 at the receiving end, respectively. Each timing is affected by jitter, so st1-st0 ≠ ts1-ts0, and st2-st0 ≠ ts2-ts0. The positions of the notes at the bottom of each figure indicate the actual times of notes.

[0072] 7B shows an example of the sounding timing when no correction based on the timestamp is performed. In this case, each note is sounded at its arrival time, so the sounding timing of note 0-2 does not maintain the intervals ts0, ts1, and ts2.

[0073] 7C shows an example of sounding timing when correction based on a timestamp is performed without applying an offset. In this case, the sounding timing of a note that arrives earlier than a reference is corrected. This reference may be, for example, sounding timing based on a timestamp based on the arrival time of a certain note, or sounding timing based on a timestamp based on a certain time (for example, the start time of a song or performance (MIDI playback, etc.)).

[0074] In the case of Figure 7C, note 2 is corrected to its original pronunciation timing (the pronunciation timing is adjusted to time ts2-ts0 after st0) because the difference in arrival time (st2-st0) relative to note 0 is smaller than the difference in timestamp (ts2-ts0) (i.e., note 2 arrived earlier than the scheduled pronunciation timing).

[0075] On the other hand, note 1 cannot be corrected and is pronounced immediately after arrival because the difference in arrival time (st1-st0) relative to note 0 is greater than the difference in timestamp (ts1-ts0) (i.e. note 1 arrived later than the scheduled timing).

[0076] In the case of FIG. 7C, the intervals between ts0, ts1, and ts2 are not maintained for the sounding timings of some notes.

[0077] 7D shows an example of sounding timing when correction based on a timestamp is performed with an offset applied. In this case, the sounding timing of a note that arrives earlier than the reference is corrected. Also, the sounding timing of a note that arrives later than the reference by an offset or less is corrected. This reference may be sounding timing based on a timestamp based on the arrival time of a certain note, as in FIG. 9C, or sounding timing based on a timestamp based on a certain time (for example, the start time of a song or performance (MIDI playback, etc.)).

[0078] In the case of Figure 7D, note 0 arrives at st0 and waits the offset time before being pronounced. Note 1 arrives at st1, but is not pronounced immediately; it waits the timestamp difference (ts1-ts0) from the pronunciation timing of note 0 before being pronounced. Note 2 arrives at st2, but is not pronounced immediately; it waits the timestamp difference (ts2-ts0) from the pronunciation timing of note 0 before being pronounced.

[0079] In the case of FIG. 7D, the intervals ts0, ts1, and ts2 are maintained for the sounding timing of the notes.

[0080] <Format of offset information and timestamp processing information> At least one of the offset information and the timestamp processing information transmitted in step S102 described above may be notified to the electronic musical instrument 10 using a MIDI System Exclusive (SysEx) message, an NRPN (Non Registered Parameter Number) included in a control change message, or the like.

[0081] Both the offset information and the timestamp processing information may be notified in one SysEx message (or NRPN), or the offset information and the timestamp processing information may be notified separately in multiple SysEx messages (or NRPN).

[0082] When offset information is notified by a SysEx message, the SysEx message may include information (bit string) indicating that the offset information is included and information (bit string) indicating the content of the offset information (e.g., the offset time itself).

[0083] When timestamp processing information is notified by a SysEx message, the SysEx message may include information (bit string) indicating that it includes timestamp processing information, and information (bit string) indicating the content of the timestamp processing information (e.g., whether or not to perform timestamp processing).

[0084] This information may be transmitted using, for example, MIDI data bytes (e.g., data bytes in a SysEx message). In the present disclosure, when any information is transmitted using MIDI data bytes, the information must be configured taking into consideration that the most significant bit (MSB) of the data byte is 0. In other words, the transmitting device 30 assigns the information to each data byte in 7-bit units and transmits it, and the receiving electronic musical instrument 10 removes the MSB of each data byte of the received information and combines them to obtain the original information.

[0085] For example, if the offset information to be transmitted is two bytes and is "0000000011111111," the device 30 first separates it into seven-bit chunks to obtain three bit strings: "0000000," "0111111," and "11." The device 30 then adds '0' to the beginning of each bit string (and applies bit padding if the bit string is less than one byte), obtaining three data bytes: "00000000," "00111111," and "01100000," which may then be transmitted in a SysEx message.

[0086] The number of bits of the offset information / timestamp processing information to be notified may be defined in advance or may be notified from the device 30. For example, the offset information may be expressed in 4, 8, 16, or 32 bits, and the timestamp processing information may be expressed in 1 bit.

[0087] When reporting offset information using the NRPN assigned to control numbers 98 and 99 of a control change message, values ​​indicating that the information is offset can be set in the MSB and LSB of the NRPN (these can be set arbitrarily by the manufacturer), and the content of the offset information (for example, the offset time itself) can be set in the data entry of that NRPN and transmitted. Timestamp processing information can also be transmitted using NRPN. Offset information and timestamp processing information can also be transmitted using assignable controller messages, which are newly defined in MIDI 2.0 as messages corresponding to NRPNs.

[0088] (Wireless performance measurement method) Depending on the environment surrounding the electronic musical instrument 10 and the device 30, the quality of the wireless communication between them may fluctuate, which may affect jitter and latency. For this reason, it is preferable to be able to easily measure the performance of the wireless communication between the electronic musical instrument 10 and the device 30 in real time.

[0089] The present inventors have devised a data format and measurement method for this purpose.

[0090] FIG. 8 is a sequence diagram of a wireless performance measurement method according to an embodiment.

[0091] The device 30 transmits measurement data to the electronic musical instrument 10 (step S201). The transmission in step S201 may be a single transmission, or multiple transmissions at certain transmission intervals (e.g., 7.5 ms, 15 ms, etc.) over a certain period (e.g., 1 s).

[0092] The electronic musical instrument 10 corrects the counter value for each of the received measurement data and returns the corrected measurement data to the device 30 (step S202). The corrected measurement data may also be called return data, reply data, report data, etc.

[0093] The device 30 calculates the performance of the wireless communication between the electronic musical instrument 10 and the device 30 based on the amount of measurement data transmitted in step S201, the time of transmission, the time of reception of the returned data in step S202, etc. (step S203). The performance of the wireless communication may correspond to at least one of, for example, throughput per unit time and round-trip delay time.

[0094] In this example, the device 30 is the transmitting side and the electronic musical instrument 10 is the receiving side, but the electronic musical instrument 10 may be the transmitting side and the device 30 may be the receiving side.

[0095] Before step S201, information informing the other of the start of measurement may be transmitted from either the device 30 or the electronic musical instrument 10. The transmission and reception of this information may trigger the device 30 and the electronic musical instrument 10 to reset the identification number, counter value, etc., described below, or perform time synchronization (time adjustment).

[0096] Furthermore, the number of times the measurement data is transmitted (the number of transmitted data), the transmission interval, the transmission period (measurement period), etc. may be predefined in the device 30, or may be determined by the device 30 based on input from the user. Information regarding the number of times the measurement data is transmitted, the transmission interval, the transmission period, etc. may be notified to the electronic musical instrument 10 from the device 30.

[0097] During this measurement, device 30 may simultaneously measure the received signal strength indicator (RSSI), which may be an instantaneous RSSI or an average / minimum / maximum RSSI over a certain period of time.

[0098] <Measurement data / Reply data format> The measurement data transmitted in step S201 above may be notified to the electronic musical instrument 10 using at least one of a MIDI SysEx message, a control change message, and the like.

[0099] Figure 9 shows an example of the format of measurement data. This format corresponds to a SysEx message, and begins with "F0" indicating the start of a SysEx message, and ends with "F7" indicating the start of a SysEx message. Other information required for a SysEx message is omitted.

[0100] When measurement data is notified by a SysEx message, the SysEx message may include identification number information (bit string) indicating the ordinal number of the data, and counter value information (bit string) indicating the value of a certain counter.

[0101] This information may be communicated using, for example, MIDI data bytes (for example, data bytes in a SysEx message).

[0102] For example, the identification number information and counter value information may be expressed in 4, 8, 16, or 32 bits.

[0103] In step S201, when the device 30 transmits the ith (i is an integer) measurement data among N consecutive measurement data (N is an integer), the device 30 sets a value of, for example, i-1 as the identification number information and transmits it. In other words, for one measurement, the identification number starts from 0, and the device 30 sets and transmits a value that is counted up for each data. In other words, the identification number may mean "the number of times measurement data has been transmitted - 1." Furthermore, when the device 30 transmits any measurement data, the device 30 sets an arbitrary value (for example, 0) as the counter value and transmits it.

[0104] In step S202, the electronic musical instrument 10 increments its own counter value by +1 each time it receives measurement data. Note that the electronic musical instrument 10 may also set the counter value to -1 when starting measurement. In other words, the counter value of the electronic musical instrument 10 may represent "the number of times measurement data has been received -1."

[0105] Also, in step S202, the electronic musical instrument 10 may leave the identification number information of the received measurement data as is, and set (update) its own counter value as the counter value information, and then return the result to the device 30 as return data.

[0106] That is, the measurement data and the reply data may have the same format. In order to reduce the impact of differences in data size on communication (to improve measurement accuracy), it is preferable that the measurement data and the reply data have the same size (number of bits).

[0107] If the identification number and counter value in the received return data match, the device 30 can determine that there is no loss in the measurement data that it has transmitted so far (that it has been transmitted and received correctly).

[0108] The device 30 may derive the round trip time (RTT) between the device 30 and the electronic musical instrument 10 based on the time of transmission of measurement data and the time of reception of return data of the same identification number. The device 30 may also derive the communication throughput between the device 30 and the electronic musical instrument 10 based on the communication volume of measurement data / return data per unit time. The RTT and communication throughput may be instantaneous values, or may be average / maximum / minimum values ​​over a certain period of time.

[0109] The receiving side (electronic musical instrument 10) may record the deviations of the measurement data transmitted at equal intervals, and calculate the jitter involved in transmission from the device 30 to the electronic musical instrument 10 based on these deviations. The electronic musical instrument 10 may transmit the calculated jitter to the device 30. The electronic musical instrument 10 may transmit information related to the jitter using a SysEx message.

[0110] In step S202, the electronic musical instrument 10 may return the received measurement data to the device 30 as return data by leaving the identification number information unchanged and setting information about the reception time instead of the counter value information. The information about the reception time preferably has the same number of bits (size) as the counter value information. The device 30 may calculate the jitter from the received return data.

[0111] The electronic musical instrument 10 may return return data including the counter value information and information regarding the reception time to the device 30. The electronic musical instrument 10 may return return data including the identification number information, the counter value information, and information regarding the reception time to the device 30.

[0112] The measurement method described above makes it easy to measure wireless performance when, for example, MIDI and other traffic (such as audio data) are flowing simultaneously between the electronic musical instrument 10 and the device 30, thereby enabling jitter and latency to be controlled appropriately in a variety of environments.

[0113] (Variation) In each of the above-described embodiments, the dongle 20 may be omitted. In this case, the electronic musical instrument 10 only needs to have the function of wirelessly communicating with the device 30 (for example, the RF unit 202 and antenna unit 203 in FIG. 4).

[0114] In each of the above-described embodiments (particularly in FIGS. 6 and 8, etc.), the operation of the electronic musical instrument 10 may be performed by the dongle 20. For example, the offset in step S103 in FIG. 6 may be set in the dongle 20, in which case the buffering in step S105 may be performed by the dongle 20, and the electronic musical instrument 10 may receive the MIDI buffered for the offset time by the dongle 20 and immediately execute the MIDI data.

[0115] 8 may be performed by the dongle 20. In this case, it is expected that the performance of only the wireless portion can be measured more precisely.

[0116] In each of the above-described embodiments (particularly in FIGS. 6 and 8, etc.), the operation of the electronic musical instrument 10 may be performed by the dongle 20. For example, the offset in step S103 in FIG. 6 may be set in the dongle 20, in which case the buffering in step S105 may be performed by the dongle 20, and the electronic musical instrument 10 may receive the MIDI buffered for the offset time by the dongle 20 and immediately execute the MIDI data.

[0117] In the above embodiment, the electronic musical instrument 10 is an example of a keyboard-like instrument, but is not limited to this. The electronic musical instrument 10 may be any instrument that allows the user to specify the timing of sound generation, such as an electric violin, electric guitar, drums, or trumpet.

[0118] Furthermore, the electronic musical instrument 10 is not limited to a so-called musical instrument (such as a keyboard), but may also be interpreted as a mobile phone, a smartphone, a tablet terminal, a personal computer (PC), a television, or the like.

[0119] As described above, by using the electronic musical instrument 10, dongle 20, device 30, etc. of the present disclosure, it is possible to achieve a suitable trade-off between jitter and latency.

[0120] The electronic device 10 (for example, the electronic musical instrument 10; the same applies hereinafter) may have a receiving unit (for example, the USB interface 107) that receives measurement data including identification number information and counter value information, and a transmitting unit (for example, the USB interface 107) that transmits return data configured by updating the counter value information of the measurement data based on the number of times the measurement data has been received. With this configuration, wireless communication measurement for the electronic device can be easily performed.

[0121] The receiver may receive a plurality of pieces of measurement data transmitted at a certain transmission interval (for example, 15 ms), and the transmitter may transmit the return data for each of the plurality of pieces of measurement data. With this configuration, it is possible to suitably measure the throughput, jitter, etc. of the wireless communication over a certain period of time.

[0122] The measurement data may be a system exclusive message, a control change message, or an assignable controller message of Musical Instrument Digital Interface (MIDI (registered trademark)). With this configuration, measurements suitable for wireless communication between devices that can use MIDI (for example, BLE-MIDI) can be performed.

[0123] Furthermore, the receiving unit may receive the measurement data via a dongle (dongle 20) physically connected to the electronic device 10, and the transmitting unit may transmit the return data via the dongle. With this configuration, even if the electronic device 10 does not have a function for direct wireless communication with the device 30, it is possible to measure wireless communication between them via the dongle 20.

[0124] The electronic device 10 may also include a receiver that receives Musical Instrument Digital Interface (MIDI (registered trademark)) messages, timestamp information (e.g., USB interface 107) related to the execution timing of the MIDI messages, and offset information related to buffering of the MIDI messages, and a controller (e.g., CPU 101) that controls the execution timing of the MIDI messages based on the timestamp information, based on the offset information. This configuration allows for a favorable trade-off between jitter and latency to be adjusted for wireless communication for the electronic device.

[0125] The receiver may receive the offset information included in a MIDI system exclusive message, a control change message, or an assignable controller message. This configuration enables control suitable for wireless communication between devices that can use MIDI (e.g., BLE-MIDI).

[0126] The control unit may determine whether to control the execution timing based on the timestamp information based on a MIDI system exclusive message, a control change message, or an assignable controller message. This configuration enables control suitable for wireless communication between devices that can use MIDI (e.g., BLE-MIDI).

[0127] Furthermore, the receiving unit may receive the timestamp information, the offset information, and the MIDI message via a dongle (dongle 20) physically connected to the electronic device 10. With this configuration, when the electronic device 10 does not have a function for direct wireless communication with the device 30, it is possible to achieve a suitable trade-off between jitter and latency in wireless communication between them via the dongle 20.

[0128] Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Furthermore, the order of the processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless inconsistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the specific order presented.

[0129] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0130] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0131] When used in this disclosure, the terms "include," "including," and variations thereof are intended to be inclusive, similar to the term "comprising." Furthermore, when used in this disclosure, the term "or" is not intended to be an exclusive or.

[0132] In the present disclosure, "A / B" may mean "at least one of A and B."

[0133] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0134] Although the invention according to the present disclosure has been described in detail above, it is clear to those skilled in the art that the invention according to the present disclosure is not limited to the embodiments described in the present disclosure. The invention according to the present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the invention as defined by the description of the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not impose any limiting meaning on the invention according to the present disclosure.

Claims

1. a transmitter that transmits measurement data including counter value information to the electronic device or a communication dongle connected to the electronic device; a receiving unit configured to update counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or to receive return data in which information regarding the reception time when the electronic device or the communication dongle received the measurement data is set; A control unit; and When the measurement data includes counter value information, the transmitting unit sends information to the electronic device or the communication dongle to notify the start of measurement before sending the measurement data, and then the control unit resets the counter value of the counter value information.

2. a transmitter that transmits measurement data including counter value information to the electronic device or a communication dongle connected to the electronic device; a receiving unit configured to update counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or to receive return data in which information regarding the reception time when the electronic device or the communication dongle received the measurement data is set; A control unit; and When the measurement data includes counter value information, the control unit receives information from the electronic device or the communication dongle notifying the start of measurement before the receiving unit transmits the measurement data, and then resets the counter value of the counter value information.

3. 3. The device according to claim 1, wherein the control unit calculates communication performance between the electronic device and the communication dongle based on the amount of measurement data transmitted and the transmission time, and the return data received from the electronic device.

4. a transmitter that transmits measurement data including counter value information to the electronic device or a communication dongle connected to the electronic device; a receiving unit configured to update counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or to receive return data in which information regarding the reception time when the electronic device or the communication dongle received the measurement data is set; A control unit; and The control unit is a device that calculates communication performance between the electronic device or the communication dongle based on the amount of data transmitted and the transmission time of the measurement data and the return data received from the electronic device.

5. The device of claim 4 , wherein the control unit determines an offset for buffering Musical Instrument Digital Interface (MIDI) data according to the communication performance.

6. 6. The device according to claim 1, wherein the measurement data is one of a MIDI system exclusive message, a control change message, and an assignable controller message.

7. a transmitter that transmits measurement data including counter value information to the electronic device or a communication dongle connected to the electronic device; a receiving unit configured to update counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or to receive return data in which information regarding the reception time when the electronic device or the communication dongle received the measurement data is set; and The measurement data is one of a MIDI system exclusive message, a control change message, and an assignable controller message.

8. A system including an electronic device and a device, The device comprises: a transmitter that transmits measurement data including counter value information to the electronic device or a communication dongle connected to the electronic device; a receiving unit configured to update counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or to receive return data in which information regarding the reception time when the electronic device or the communication dongle received the measurement data is set; a control unit; The electronic device includes: a receiving unit that receives the measurement data directly or via the communication dongle; a transmitter configured to update counter value information of the measurement data based on the number of times the measurement data has been received, or to transmit return data in which information regarding the reception time at which the electronic device or the communication dongle received the measurement data is set; When the measurement data includes counter value information, the control unit of the device either sends information to the electronic device or the communication dongle to notify the start of measurement before the transmitting unit sends the measurement data, and then resets the counter value of the counter value information, or receives information from the electronic device or the communication dongle to notify the start of measurement before the receiving unit sends the measurement data, and then resets the counter value of the counter value information.

9. The system according to claim 8 , wherein the control unit calculates communication performance between the electronic device or the communication dongle based on the amount of data transmitted and the transmission time of the measurement data and the return data received from the electronic device.

10. The system according to claim 9 , wherein the control unit determines an offset for buffering MIDI data according to the communication performance.

11. transmitting measurement data including counter value information to the electronic device or a communication dongle connected to the electronic device; receiving return data configured by updating counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or including information regarding the reception time when the electronic device or the communication dongle received the measurement data; If the measurement data includes counter value information, a step of sending information to the electronic device or the communication dongle notifying the start of measurement before sending the measurement data and then resetting the counter value of the counter value information, or receiving information from the electronic device or the communication dongle notifying the start of measurement before sending the measurement data and then resetting the counter value of the counter value information.

12. On your device's computer, transmitting measurement data including the counter value information to the electronic device or a communication dongle connected to the electronic device; receiving return data configured by updating counter value information of the measurement data by the electronic device or the communication dongle based on the number of times the electronic device or the communication dongle has received the measurement data, or including information regarding the reception time when the electronic device or the communication dongle received the measurement data; When the measurement data includes counter value information, a program that sends information to the electronic device or the communication dongle to notify the start of measurement before sending the measurement data, and then resets the counter value of the counter value information, or that receives information from the electronic device or the communication dongle to notify the start of measurement before sending the measurement data, and then resets the counter value of the counter value information.

Images