Sound output system

Abstract

To faithfully reproduce sound of a performance at the time of performance, and to drive actuators based on the sound of the performance to accurately reproduce the performance at the time of performance.SOLUTION: A sound output system according to one embodiment includes: a speaker device that outputs sound in accordance with supplied sound data; and an operation device that includes one or more operators, a drive unit that drives the one or more operators based on performance data supplied in synchronization with the sound data, and a drive control unit that controls the drive unit. According to the sound output system, performance sound during the performance can be faithfully reproduced, and the actuators can be driven based on the sound of the performance to accurately reproduce the performance during the performance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a sound output system.

Background Art

[0002] There is known a so-called automatic-playing piano that sounds based on MIDI data and drives a keyboard without striking the strings. For example, Patent Document 1 discloses an automatic-playing piano that can perform synchronous performance via a network. Patent Document 2 discloses a performance operation device that is operated based on performance operation information in the MIDI standard transmitted via a network.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the technologies disclosed in the above Patent Documents 1 to 3, automatic performance of a keyboard device can be performed based on MIDI data. On the other hand, since the performance sound during performance is once converted into MIDI data, there is a problem that the faithful reproducibility of the performance sound itself during performance is low.

[0005] One object of the present disclosure is to faithfully reproduce the performance sound during performance, drive an operator based on the performance sound, and accurately reproduce the performance during performance.

Means for Solving the Problems

[0006] According to one embodiment of the present disclosure, a sound output system is provided, comprising: a speaker device that outputs sound corresponding to supplied sound data; an operating device including one or more operators; a drive unit that drives the one or more operators based on performance data supplied in synchronization with the sound data; and a drive control unit that controls the drive unit. [Effects of the Invention]

[0007] According to one embodiment of the present disclosure, the sound of a performance can be faithfully reproduced, and the controls can be driven based on the sound of the performance to accurately reproduce the performance. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram of a sound output system according to one embodiment of the present disclosure. [Figure 2] This is a block diagram showing the configuration of a speaker device according to one embodiment of the present disclosure. [Figure 3] A perspective view showing an example of an operating device according to one embodiment of this disclosure. [Figure 4] This is a diagram illustrating the internal structure of the keyboard mechanism. [Figure 5] This is a block diagram showing the configuration of the control device for the keyboard. [Figure 6] This figure shows the configuration of a supply unit according to one embodiment of the present disclosure. [Figure 7] This is a block diagram showing the functional configuration of the control unit of the supply section. [Figure 8] This block diagram shows the configuration of a server according to one embodiment of the present disclosure. [Figure 9] This is a block diagram showing the configuration of the performance data generation unit. [Figure 10] This is a block diagram showing the configuration of a speaker device according to a modified embodiment of one embodiment of the present disclosure. [Modes for carrying out the invention]

[0009] Hereinafter, an audio output system in one embodiment of the present disclosure will be described in detail with reference to the drawings. The embodiments shown below are examples of embodiments of the present disclosure, and the present disclosure is not to be construed as being limited to these embodiments. In the drawings referenced in this embodiment, the same parts or parts having similar functions are denoted by the same or similar reference numerals (simply a number followed by A, B, etc.), and repeated descriptions may be omitted.

[0010] [Overall configuration of the sound output system] Figure 1 is a schematic diagram of a sound output system 10 according to one embodiment of the present disclosure. The sound output system 10 includes a speaker device 100, a keyboard device 200, a supply unit 300, and a server 400.

[0011] [1. Speaker System Configuration] The speaker device 100 outputs sound according to sound data. The sound data is supplied to the speaker device 100 from the supply unit 300. The sound data is data representing an audio signal that has undergone reverberation removal processing from music data that indicates the content of the sound produced by the music. Figure 2 is a block diagram showing the configuration of the speaker device 100. Referring to Figure 2, the speaker device 100 includes a sound data acquisition unit 101, an equalizer (EQ) 103, a D / A converter (DAC) 105, an amplification unit 107, and a speaker unit 109.

[0012] The sound data acquisition unit 101 acquires sound data from the supply unit 300. The sound data acquisition unit 101 supplies the acquired sound data to the equalizer 103. The equalizer 103 adjusts the frequency characteristics of the sound data supplied from the sound data acquisition unit 101 and outputs it to the D / A converter 105.

[0013] The D / A converter 105 acquires sound data from the equalizer 103, adjusts the frequency characteristics, converts it from a digital signal to an analog signal, and outputs it to the amplification unit 107. The amplification unit 107 amplifies the analog-converted sound data according to the set amplification ratio and outputs it to the speaker unit 109.

[0014] The speaker unit 109 is a member that radiates sound. The speaker unit 109 may be an electromagnetic speaker unit, but is not limited thereto. The speaker unit 109 emits sound based on the sound data supplied from the amplifier unit 107. The sound data may be supplied as streaming data.

[0015] [2. Configuration of Keyboard Device] The keyboard device 200 includes a plurality of keys as one or more operators, a key drive unit that drives the plurality of keys based on performance data, and a key drive control unit that controls the operation of the drive unit. The performance data is MIDI data. The performance data includes performance event information including note-on and note-off defined on a time axis determined according to tempo and duration (gate time), and pitch information indicating the pitch of the sounding content. The pitch information corresponds to the key number. The performance data corresponds to the sound data and is supplied from the supply unit 300 to the keyboard device 200 in synchronization with the sound data supplied to the speaker device 100.

[0016] FIG. 3 is a perspective view showing an example of the keyboard device 200 in the present embodiment. The keyboard device 200 is a keyboard instrument having a keyboard on which a plurality of keys 202 for performing operations by a performer are arranged on its front surface, and a pedal 203. Although there are a plurality of pedals of the keyboard device 200, the pedal 203 indicates a damper pedal. Further, the keyboard device 200 has a control device 210 having an operation panel 213 on the front portion, and a touch panel 260 provided on the score stand portion.

[0017] User instructions can be input to the control device 210 by operating the operation panel 213 and the touch panel 260. The keyboard device 200 has a plurality of operation modes. The keyboard device 200 controls the operation of each component of the keyboard device 200 based on the operation mode set based on the user's instructions. The operation modes include a mode in which sound generated without driving the keys 202 is generated from the speaker device 100. Details of each operation mode will be described later.

[0018] Figure 4 is a diagram illustrating the internal structure of the keyboard device 200. In Figure 4, the configurations corresponding to each key 202 are shown focusing on the configurations corresponding to one key 202 (a white key in this example), and the configurations corresponding to other keys 202 are omitted from the description.

[0019] At the rear end of each key 202 (the back side of the key 202 as seen from the user playing the keyboard device 200), there is a key drive unit 230 that drives the key 202 using a solenoid. The key drive unit 230 drives the solenoid in accordance with a key control signal based on performance data supplied from the supply unit 300. The key drive unit 230 reproduces the state when the user presses a key by driving the solenoid to raise the plunger, and reproduces the state when the user releases a key by lowering the plunger.

[0020] Each hammer 204 is provided corresponding to each key 202. When a key 202 is pressed, force is transmitted via the action mechanism 245, causing the hammer to move and strike the string 5, which is provided corresponding to each key 202. The strings 205 are sound-producing elements that produce sound as a result of the strike from the hammers 204. Each string 205 has a vibration frequency corresponding to each key 202.

[0021] The damper 208 is moved by the damper operating mechanism 280. The damper operating mechanism 280 moves the damper 208 to control the contact state between the damper 208 and the string 205 according to the amount the key 202 is pressed and the pedal 203 is pressed. This control of the contact state means moving the damper 208 within a range from a position where the damper 208 and the string 205 are in contact and the vibration of the string 205 is suppressed (vibration damping position) to a position where the string 205 is released from the damper 208 (release position).

[0022] In this embodiment, it is described that all 88 keys 202 of the keyboard device 200 each have a damper 208. However, the keyboard device 200 may have a structure like a typical piano, with dampers 208 corresponding to 66 or 70 keys from the lowest pitch, and keys 202 with higher pitches not having dampers 208.

[0023] The stopper 240 is a component that, when the setting applied to the operating mode is a predetermined setting, collides with the hammer shank to prevent the hammer 204 from striking the string 205 before the string is struck. The stopper 240 moves to either a position where it collides with the hammer shank (hereinafter referred to as the blocking position) or a position where it does not collide with the hammer shank (hereinafter referred to as the retraction position) in response to a stopper control signal from the control device 210.

[0024] The stopper drive unit 244 may be a motor that is driven in response to a stopper control signal from the control device 210 when the setting applied to the operating mode is a predetermined setting. The stopper drive unit 244 moves the stopper 240 to either the blocking position or the retracted position.

[0025] The key sensor 222 is located at the bottom of each key 202 and outputs a detection signal to the control device 210 according to the behavior of the key 202. In this example, the key sensor 222 detects the amount of pressure applied to the key 202 in a continuous manner and outputs a detection signal to the control device 210 indicating the detection result. Alternatively, instead of outputting a detection signal according to the amount of pressure applied to the key 202, the key sensor 222 may output a detection signal indicating that the key 202 has passed a specific pressing position. The specific pressing position is any position within the range from the rest position to the end position of the key 202, and it is desirable that there be multiple such positions. The detection signal output by the key sensor 222 can be any signal that allows the control device 210 to recognize the behavior of the key 202.

[0026] The hammer sensor 224 is provided in correspondence with each hammer 204 and outputs a detection signal to the control device 210 according to the behavior of the hammer 204. In this embodiment, the hammer sensor 224 detects the movement speed of the hammer 204 immediately before striking the string 205 and outputs a detection signal indicating the detection result to the control device 210. This detection signal does not have to indicate the movement speed of the hammer 204 itself, and may be a detection signal in another form, for example, so that the movement speed is calculated in the control device 210. The detection signal output by the hammer sensor 224 can be any signal that allows the control device 210 to recognize the behavior of the hammer 204.

[0027] Each pedal sensor 223 is provided for each pedal 203 and outputs a detection signal to the control device 210 according to the behavior of the pedal 203. In this example, the pedal sensor 223 detects the amount of pedal 203 pressed and outputs a detection signal indicating the detection result to the control device 210. Instead of outputting a detection signal according to the amount of pedal 203 pressed, the pedal sensor 223 may output a detection signal indicating that the pedal 203 has passed a specific pressed position. The specific pressed position is any position within the range from the rest position to the end position of the pedal, and may be detectable at multiple locations. The detection signal output by the pedal sensor 223 can be any signal that allows the control device 210 to recognize the behavior of the pedal 203.

[0028] The pedal drive unit 233 is provided in correspondence with each pedal 203 and drives the corresponding pedal 203 to press in response to the pedal control signal from the control device 210. This mechanically reproduces the same situation as when a performer presses down on the pedal 203.

[0029] It is sufficient that the control device 210 can determine the timing of the hammer 204 striking the string 205 (key-on timing), the striking speed (velocity), and the timing of the damper 208's vibration suppression for the string 205 (key-off timing) for each key 202 (key number) based on the detection signals output from the key sensor 222, pedal sensor 223, and hammer sensor 224. Therefore, the key sensor 222, pedal sensor 223, and hammer sensor 224 may output the results of detecting the behavior of the key 202, pedal 203, and hammer 204 as detection signals different from those described above.

[0030] The soundboard 207 is connected to the sound ribs 275 and the bridge 206, and the vibrations of each string 205 are transmitted to the soundboard 207 via the bridge 206.

[0031] Figure 5 is a block diagram showing the configuration of the control device 210. The control device 210 includes a control unit 211, a storage unit 212, an operation panel 213, a communication unit 214, a key drive control unit 611, a stopper drive control unit 612, a pedal drive control unit 613, and an interface 216. Each of these components is connected via a bus 217. The control device 210 may also include a signal output unit 215.

[0032] The control unit 211 includes an arithmetic unit such as a CPU (Central Processing Unit), a memory device such as ROM (Read Only Memory), and a memory device such as RAM (Random Access Memory). Based on a control program stored in the memory device, the control unit 211 controls each component of the control device 210 and each component connected to the interface 216. In this example, the control unit 211 executes the control program to make the control device 210 and some of the components connected to the control device 210 function as a keyboard instrument. Control signals are used to control each component connected to the control device 210. For example, these include the key control signal, stopper control signal, and pedal control signal mentioned above. The control device 210 controls the keyboard device 200 according to the set operating mode. The operating modes that can be set for the keyboard device 200 will be described later.

[0033] The memory unit 212 stores various information, including setting information, music data, sound data, and performance data. Music data is data that represents audio signals indicating the content of the music's sound. The format of the music data is various encoding formats, such as WAV or MP3. The music data includes audio information obtained when the music was recorded and may include the sound content of one or more instruments. As mentioned above, sound data is data that represents audio signals after reverberation removal processing has been performed on the music data. Performance data is MIDI data that includes performance event information and pitch information. The performance data corresponds to the sound data supplied to the speaker device 100, and the performance data and sound data are synchronized with each other. Here, synchronization between the performance data and sound data means that the performance data and sound data are played back in sync along the time axis. The performance data is played back along a time axis based on tempo and duration time, and the sound data is played back in sync with the timestamp. Setting information indicates various settings used during the execution of the control program. For example, setting information includes information such as the operating mode set by the user and the settings applied in each operating mode.

[0034] The control panel 213 has operation buttons and other features that accept user input. When user input is received via these buttons, an operation signal corresponding to the input is output to the control unit 211. The touch panel 260, connected to the interface 216, has a display screen such as a liquid crystal display, and a touch sensor that accepts user input is provided on the surface of the display screen. This display screen shows various information, such as a settings screen and sheet music for selected songs, which can be changed by various settings controlled by the control unit 211 via the interface 216. When user input is received via the touch panel 260, an operation signal corresponding to the input is output to the control unit 211 via the interface 216. In other words, instructions from the user to the control device 210 are input by operations received by the control panel 213 or the touch panel 260.

[0035] The communication unit 214 is an interface that communicates with other devices such as the speaker device 100 and the supply unit 300 via wireless, wired, or other means. This interface may be connected to a disk drive that reads various data recorded on recording media such as DVDs (Digital Versatile Disks) and CDs (Compact Disks) and outputs the read data, or it may be connected to semiconductor memory, or an external device such as a server may be connected via a network. The data input to the control device 210 via the communication unit 214 may be, for example, performance data, music data, or the control program described above.

[0036] The sound source unit 215 generates and outputs an audio signal based on instructions from the control unit 211. Although not shown in the figures, the sound source unit 215 includes an equalizer unit for adjusting the frequency distribution of the audio signal and an amplification unit for amplifying the audio signal. The sound source unit 215 may generate an audio signal according to, for example, music data or detection signals from the key sensor 222, pedal sensor 223, and hammer sensor 224. The sound source unit 215 may also include a decoding unit (not shown) for decoding music data encoded in various formats. The audio signal generated by the sound source unit 215 is output to a terminal to which headphones or the like are connected. Furthermore, if the keyboard device 200 includes a vibrator, the audio signal may be converted into a drive signal for driving the vibrator.

[0037] The key drive control unit 611 generates a key control signal based on the performance data. The generated key control signal is output to the key drive unit 230 via the interface 216. The key drive control unit 611 may sequentially generate key control signals based on the performance data and output them to the key drive unit 230. The key drive control unit 611 may also generate a key control signal based on the detection signal of the key sensor 222, or it may generate a key control signal based on the music data.

[0038] The stopper drive unit 612 generates a stopper control signal based on the performance data and predetermined settings applied to the operating mode. The generated stopper control signal is output to the stopper drive unit 244 via the interface 216.

[0039] The pedal drive control unit 613 generates a pedal control signal based on the performance data if the performance data contains information for driving the pedal. The generated pedal control signal is output to the pedal drive unit 233 via the interface 216. The pedal drive control unit 613 may also generate the pedal control signal based on the detection signal from the pedal sensor 223, or it may generate the pedal control signal based on the music data.

[0040] Interface 216 is an interface that connects the control device 210 to each external component. In this example, the components connected to interface 216 are the key sensor 222, pedal sensor 223, hammer sensor 224, key drive unit 230, stopper drive unit 240, touch panel 260, and pedal drive unit 233. Interface 216 outputs detection signals from the key sensor 222, pedal sensor 223, and hammer sensor 224, and operation signals from the touch panel 260, to the control unit 211. Interface 216 also outputs a key control signal to the key drive unit 230, a stopper control signal to the stopper drive unit 240, and a pedal control signal to the pedal drive unit 233.

[0041] [3. Explanation of Operating Modes] The operating modes that can be set for the keyboard unit 200 are described below. The keyboard unit 200 is set by selecting one of several operating modes. Each of the operating modes, including manual playing mode and automatic playing mode, can be set by the user of the keyboard unit 200 via the touch panel 260 or the control panel 213. Each operating mode and the settings that apply to it are described below.

[0042] <Manual Play Mode> The manual playing mode is used when the performer operates the keys 202 of the keyboard device 200 to play. In manual playing mode, the normal setting or the mute setting is applied to the keyboard device 200.

[0043] The default setting is the setting used when the keyboard device 200 is played as an acoustic piano. When the default setting is applied, the stopper 240 is moved to the retracted position, and the hammer 204 strikes the string 205.

[0044] On the other hand, the mute setting in manual playing mode is the setting used when playing the keyboard device 200 as an electronic piano. When the mute setting is applied in manual playing mode, the stopper 240 is moved to the blocking position, and the hammer 204's strike of the string 205 is blocked by the stopper 240. In the mute setting, the sound source unit 215 generates an audio signal based on the detection signals from the key sensor 222, the pedal sensor 223, and the hammer sensor 224.

[0045] <Automatic Playback Mode> The automatic performance mode is a mode in which, instead of the performer operating the keys 202 of the keyboard device 200, the key drive unit 230 drives the keys 202 based on musical data or performance data. In automatic performance mode, the automatic performance setting or mute setting is applied to the keyboard device 200.

[0046] The automatic playing setting is a mode that drives the keyboard device 200 as a normal automatic playing piano. When the automatic playing setting is applied, the key drive control unit 611 generates a key control signal based on the playing data. The key control signal is output to the key drive unit 230, and the key 202 is driven by the key drive unit 230. Meanwhile, the stopper 240 is moved to the blocking position, and the hammer 204 is prevented from striking the string 205 by the stopper 240. Instead, the sound source unit 215 generates an audio signal based on the playing data.

[0047] The mute setting in automatic performance mode is a mode in which, when the keyboard device 200 is driven as an automatic performance piano, the sound source unit 215 does not generate an audio signal, and instead outputs sound from the speaker device 100. When the mute setting is applied in automatic performance mode, the key drive control unit 611 generates a key control signal based on the performance data. As described above, the performance data is supplied from the supply unit 300 to the keyboard device 200 in synchronization with the sound data supplied to the speaker device 100. The key control signal is output to the key drive unit 230, and the key 202 is driven by the key drive unit 230. Meanwhile, the stopper 240 is moved to the blocking position, and the hammer 204's strike of the string 205 is blocked by the stopper 240. In the mute setting in automatic performance mode, a sound corresponding to the sound data is emitted from the speaker device 100 in synchronization with the driving of the key 202 in the keyboard device 200. Furthermore, the speaker device 100 may emit a sound corresponding to music data, rather than a sound corresponding to sound data. For example, when playing a sound corresponding to music data, if there is an unnaturalness in the reverberation, the speaker may emit a sound corresponding to sound data; otherwise, it may emit a sound corresponding to music data. When emitting music data, the sound data can also be omitted.

[0048] [4. Configuration of the Supply Unit] Returning to Figure 1, let's describe the supply unit 300. The supply unit 300 supplies sound data to the speaker device 100 and performance data synchronized with the sound data to the keyboard device 200. The performance data is supplied to the key drive control unit 611 and the pedal drive control unit 613 of the keyboard device 200.

[0049] Figure 6 is a block diagram showing the configuration of the supply unit 300. The supply unit 300 includes a control unit 311, a storage unit 312, an operation unit 313, and a communication unit 314. The supply unit 300 may also be a mobile terminal device such as a smartphone. Each component of the supply unit 300 is connected to the others by a bus 315.

[0050] The control unit 311 includes an arithmetic unit such as a CPU (Central Processing Unit) 601, a RAM (Random Access Memory) 602, and a storage device such as a ROM (Read Only Memory) 603. The CPU 601 controls each component of the supply unit 300 based on a control program stored in the ROM 603. The ROM 603 readably stores various computer programs executed by the CPU 601, various table data referenced by the CPU 601 when executing a predetermined computer program, etc. The RAM 602 is used as working memory to temporarily store various data generated when the CPU 601 executes a predetermined computer program. Alternatively, the RAM 602 may be used as memory to temporarily store the computer program being executed and related data.

[0051] The storage unit 312 stores sound data acquired via the communication unit 314. The storage unit 312 may also store performance data acquired from the server 400, which will be described later. In this case, the sound data and performance data are stored in association with each other so that they can be played back synchronously. The storage unit 312 may also store music data. The control unit 311 reads the sound data and the performance data associated with the sound data from the storage unit 312 based on the user's music playback instruction input to the operation unit 313. The control unit 311 supplies the sound data to the speaker device 100 and the performance data to the keyboard device 200 in a synchronous manner via the communication unit 314. In this embodiment, the storage unit 312 is described as being part of the supply unit 300, but this disclosure is not limited thereto. For example, the storage unit 312 may be implemented by an external storage device or the storage unit of an external server. In this case, the supply unit 300 and the external device are connected via a network, and the supply unit 300 reads the sound data stored in the external device and the performance data associated with the sound data based on the user's music playback instruction input to the operation unit 313.

[0052] The operation unit 313 consists of operation buttons, a touch panel, etc., that accept user input. When user input is received by the operation unit 313, an operation signal corresponding to the input operation is output to the control unit 311. The operation signal includes, for example, song selection information that specifies the song the user wants, instrument selection information that specifies the desired instrument sound, and song playback instructions to support the playback of the song.

[0053] The communication unit 314 is an interface that communicates with other devices wirelessly, wired, or otherwise. This interface may be connected to a disk drive that reads various data recorded on recording media such as DVDs (Digital Versatile Disks) and CDs (Compact Disks) and outputs the read data, or it may be connected to semiconductor memory, or an external device such as a server may be connected via a network. The supply unit 300 can obtain desired music data such as MP3s from digital sound sources recorded on recording media such as CDs or from external servers, etc., via the communication unit 314 according to the music selection information. The music data may be audio signals that include the sound of one or more instruments being played. The supply unit 300 also supplies the sound data read out in response to the user's music playback instruction to the speaker device 100 and the performance data to the keyboard device 200 via the communication unit 314.

[0054] Before transmitting sound data to the speaker device 100 and performance data to the keyboard device 200, the control unit 311 may adjust the timing of the start of sound generation based on sound data relative to the timing of the start of key operation based on performance data. Specifically, the control unit 311 may perform a delay process that delays the sound data relative to the performance data by a predetermined time. This predetermined time may be a set time, for example, 0.5 seconds, or a time set by the user via the operation unit 313 or the like.

[0055] Figure 7 is a block diagram showing the functional configuration of the control unit 311. The control unit 311 includes a data acquisition unit 701 and an adjustment unit 703. The functions of the data acquisition unit 701 and the adjustment unit 703 described below may be performed by the CPU 601 of the control unit 311.

[0056] The data acquisition unit 701 reads and acquires sound data and performance data associated with the sound data from the storage unit 312 based on user instructions input to the operation unit 313. The data acquisition unit 701 outputs the acquired sound data and performance data to the adjustment unit 703.

[0057] The adjustment unit 703 receives sound data and performance data, and adjusts the timing of the start of sound production based on the sound data relative to the timing of the start of key operation based on the performance data. Specifically, the adjustment unit 703 performs a delay process that delays the sound data relative to the performance data by a predetermined time. The delay process may include, for example, inserting a silent period corresponding to a predetermined period at the beginning of the sound data in order to delay the timing of the start of sound production in the speaker device 100 by a predetermined time relative to the timing of the start of key operation in the keyboard device 200 based on the performance data. Alternatively, the delay process may include delaying the timing of the start of transmission of sound data to the speaker device 100 by a predetermined time relative to the timing of the start of transmission of performance data to the keyboard device 200. Alternatively, the delay process may include shifting the start time of the performance event forward by a predetermined time relative to the start time of sound production based on the sound data. In this case, the timing of the performance event information included in the performance data, which is defined on a time axis determined according to the tempo and duration, may be shifted forward by a predetermined time. As mentioned above, the predetermined time is also called the delay time, and may be a pre-set time, for example, 0.5 seconds. Furthermore, the user can change the delay time via the control unit 313 depending on the music.

[0058] In the case of an acoustic piano, an interval is observed between the time a key is pressed and the time it takes for the sound to actually be produced. This interval corresponds to the time it takes for the hammer to move in response to the key press and for the hammer to strike the string corresponding to the pressed key. The adjustment unit 703 delays the sound data relative to the performance data, thereby improving the reproducibility of the acoustic piano's sound in the sound output system 10. Furthermore, in an acoustic piano, the interval between the time a key is pressed and the time it is actually produced varies depending on the key press speed. In an acoustic piano, the slower the key press speed, the longer the interval. Therefore, by changing the time the sound data is delayed relative to the performance data, i.e., the delay time, according to the dynamics of the sound included in the music, the reproducibility of the acoustic piano's sound in the sound output system 10 can be further improved. In addition, in the keyboard device 200, an interval is observed between the note-on included in the performance data and the time when the corresponding key 202 is actually driven by the key drive unit 230. This interval could cause the sound corresponding to the key 202 to be emitted from the speaker device 100 before the key 202 is driven. The adjustment unit 703 delays the sound data relative to the performance data, thereby preventing the sound corresponding to the key 202 from being emitted from the speaker device 100 before the key 202 is driven, and allowing for a more natural performance to be reproduced.

[0059] [5. Server Configuration] Returning to Figure 1, let's describe the server 400. The supply unit 300 transmits the acquired music data to the server 400. At this time, the supply unit 300 may also supply instrument specification information to the server 400 along with the music data. The server 400 processes the music data acquired from the supply unit 300 and generates sound data. Based on the generated sound data, the server 400 automatically generates performance data in MIDI format and supplies the generated performance data to the supply unit 300, associating it with the sound data.

[0060] Figure 8 is a block diagram showing the configuration of server 400. Server 400 includes a control unit 411, a storage unit 412, a performance data generation unit 413, and a communication unit 414. Each component of server 400 is connected to the others by a bus 415.

[0061] The control unit 411 includes an arithmetic unit such as a CPU (Central Processing Unit), a storage device such as ROM (Read Only Memory), and RAM (Random Access Memory). The control unit 411 controls each component of the server 400 based on a control program stored in the recording device. In this example, the control unit 411 executes an automatic generation function of performance data based on sound data by executing the control program.

[0062] The memory unit 412 stores music data and instrument specification information acquired via the communication unit 414. The music data and instrument specification information are supplied to the performance data generation unit 413, which will be described later. The memory unit 412 may also store performance data generated by the performance data generation unit 413. In this case, the sound data and performance data are stored in association with each other.

[0063] The performance data generation unit 413 generates performance data based on sound data. Performance data is data corresponding to sound data, such as MIDI data. If the musical data includes the sound content of one or more instruments, the performance data generation unit 413 generates performance data based on the sound content of a predetermined instrument desired by the user, based on instrument specification information.

[0064] Figure 9 is a block diagram showing the configuration of the performance data generation unit 413. The performance data generation unit 413 includes an instrument sound selection unit 911, a reverberation removal unit 912, and a data generation unit 913.

[0065] If the music data acquired from the supply unit 300 includes the sound content of one or more instruments, the music data is supplied to the instrument sound selection unit 911 along with instrument specification information. Based on the instrument specification information, the instrument sound selection unit 911 extracts music data from the music data that indicates the sound content of a predetermined instrument. The instrument sound selection unit 911 supplies the music data extracted based on the instrument specification information to the reverberation removal unit 912. However, if the music data acquired from the supply unit 300 includes the sound content of only one instrument, the extraction process of predetermined music data by the instrument sound selection unit 911 is omitted. In this case, the music data acquired from the supply unit 300 may be supplied directly to the reverberation processing unit 912.

[0066] The reverberation removal unit 912 removes reverberation components from the acquired music data to generate sound data. Specifically, the reverberation removal unit 912 analyzes the sound data and removes echoes, reverb, noise, and other unclear components (for example, sound components other than sounds with strong attacks). The reverberation removal unit 912 supplies the reverberation-removed sound data to the data generation unit 913.

[0067] The data generation unit 913 generates performance data based on the acquired sound data. As described above, the performance data is control data that defines the performance content with sound production and sound stopping control according to the progression of time, and is MIDI data that includes performance information such as pitch information that specifies the pitch of the sound content and period information that defines the duration of sound production. The data generation unit 913 outputs the sound data and the performance data generated based on the sound data, associating them with each other. The sound data and performance data output from the data generation unit 913 are stored in the storage unit 412. Alternatively, the sound data and performance data output from the data generation unit 913 may be stored as associations with each other in an external storage device.

[0068] The supply unit 300 described above reads sound data and corresponding performance data from the server 400 or an external storage device in response to an operation input by the user. The supply unit 300 synchronizes the acquired sound data and performance data and supplies them to the speaker device 100 and the keyboard device 200, respectively.

[0069] As described above, in the sound output system 10 according to this embodiment, when the keyboard device 200 is set to automatic playing mode and silent mode, the keyboard device 200 drives the keys 202 by the key drive unit 230 without generating any string striking sounds, and at the same time, the speaker device 100 outputs sound based on sound data, which is an audio signal. The sound data includes audio information obtained at the time of recording. Therefore, the sound output system 10 can reproduce the accurate performance sound at the time of recording. The user can confirm the accurate performance sound at the time of recording and see the operation of the keys 202 and pedal 203 in the keyboard device 200.

[0070] Furthermore, because the sound based on the sound data from which the reverberation component has been removed is output from the speaker device 100, reverberation corresponding to the space in which the sound is emitted is added to the sound. As a result, the user can enjoy a more natural sound that is appropriate for that space.

[0071] [Differentiation] Although one embodiment of the present disclosure has been described above, the present invention can be implemented in various forms as follows.

[0072] (1) The keyboard device 200 may be a sound-producing device that includes controls other than keys. Examples of such sound-producing devices include drums, cymbals, and wind instruments. It may also be a device that includes only controls and does not have a sound-producing function. All sound-producing devices and devices that do not have a sound-producing function may be considered as operating devices.

[0073] (2) In the embodiments described above, the supply unit 300 was described as a device independent of the keyboard device 200. However, the supply unit 300 may be included in the keyboard device 200. In this case, the keyboard device 200 supplies sound data to the speaker device 100 in synchronization with the performance data.

[0074] (3) In the embodiments described above, the server 400 was described as a device independent of the supply unit 300. However, the server 400 may be included in the supply unit 300. In this case, the functions of the server 400 are performed by the supply unit 300. The supply unit 300 generates sound data from music data and generates performance data based on the sound data. The supply unit 300 synchronizes the sound data with the performance data and supplies it to the speaker device 100, and synchronizes the performance data with the sound data and supplies it to the keyboard device 200.

[0075] (4) In the embodiments described above, the server 400 was described as a device independent of the keyboard device 200. However, the server 400 may be included in the keyboard device 200. In this case, the functions of the server 400 are performed by the keyboard device 200. The keyboard device 200 generates sound data from musical data and generates performance data based on the sound data. The keyboard device 200 synchronizes the sound data with the performance data and supplies it to the speaker device 100.

[0076] (5) The supply unit 300 and the server 400 may both be included in the keyboard device 200.

[0077] (6) In the embodiment described above, if the musical data includes the sound content of one or more instruments, the instrument sound selection unit 911 of the server 400 extracts musical data for a predetermined instrument according to the instrument specification information, and the extracted musical data is subjected to reverberation removal processing to generate sound data and performance data. The generated performance data and the corresponding sound data are stored in association and supplied to the supply unit 300. However, the sound data supplied from the supply unit 300 to the speaker device 100 may be musical data that includes the sound content of one or more instruments. In other words, the musical data before processing in the instrument sound selection unit 911 and the reverberation removal unit 912 may be supplied to the supply unit 300 as sound data associated with the performance data. The user can confirm the operation of the controls of the desired instrument on the keyboard device 200 and enjoy an ensemble of multiple instruments including the desired instrument.

[0078] (7) When the user selects automatic playback mode and mute setting in the keyboard device 200, the supply unit 300 may output song data and instruction signals to the server 400 to read song data and generate sound data and performance data. In this case, when the supply unit 300 detects that the user has selected automatic playback mode and mute setting, it may query the keyboard device 200 for song specification information and instrument specification information.

[0079] (8) When the supply unit 300 reads out song data based on the operation input by the user, the keyboard device 200 may automatically change its operating mode to automatic playback mode and mute setting. In this case, the user may input song specification information and instrument specification information via the operation unit 313 of the supply unit 300.

[0080] (9) In the embodiments described above, it was explained that the supply unit 300 performs a delay process that delays the sound data relative to the performance data by a predetermined time. However, this delay process may be performed in the speaker device 100. Figure 10 is a block diagram showing the configuration of the speaker device 100A according to this modified example. The configuration of the speaker device 100A is substantially the same as the configuration of the speaker device 100 shown in Figure 2, except that it includes an adjustment unit 102.

[0081] The sound data acquisition unit 101 acquires delay time information from the supply unit 300 along with the sound data, in order to delay the sound data relative to the performance data by a predetermined time, and supplies it to the adjustment unit 102. Based on the acquired sound data and delay time information, the adjustment unit 102 performs a delay process to delay the sound data relative to the performance data by a predetermined time. Here, the delay process may include a process to insert a silent period corresponding to a predetermined period at the beginning of the sound data in order to delay the timing of the start of sound generation from the speaker device 100A by a predetermined time. The adjustment unit 102 supplies the delayed sound data to the equalizer 103. In this modified example, the timing at which the adjustment unit 102 performs the delay process on the sound data is not limited to before the frequency characteristics of the sound data are adjusted by the equalizer 103. The timing at which the delay process is performed is after the sound data acquisition unit 101 acquires the sound data and delay time information, and before the speaker unit 109 acquires the sound data amplified by the amplification unit 107.

[0082] (10) In the embodiments described above, the supply unit 300 was shown to perform a delay process that delays the sound data relative to the performance data by a predetermined time. However, the delay process may be performed in the server 400. For example, the delay process may be performed in the performance data generation unit 413. In this case, the performance data generation unit 413 obtains delay time information along with the music data from the supply unit 300. The data generation unit 913 of the performance data generation unit 413 performs a delay process that delays the sound data relative to the performance data by a predetermined time based on the delay time. The delay process here is the same as the delay process performed by the adjustment unit 703 described above.

[0083] (11) In the embodiments described above, the speaker device 100 sequentially emits sound based on sound data supplied from the supply unit 300, and the keyboard device 200 sequentially generates key control signals based on performance data supplied from the supply unit 300. However, the disclosure is not limited thereto. The supply unit 300 may receive a data transmission instruction from the user and transmit sound data to the speaker device 100 and performance data to the keyboard device 300. The speaker device 100 may store the sound data supplied from the supply unit 300 as a single data file and, upon receiving a music playback instruction from the user via the supply unit 300, begin emitting sound based on the sound data. Similarly, the keyboard device 200 may store the performance data supplied from the supply unit 300 as a single data file and, upon receiving a music playback instruction from the user via the supply unit 300, begin generating key control signals based on the performance data.

[0084] In this case, if the sound data is delayed relative to the performance data by a predetermined time, the supply unit 300 may delay the timing of sending the music playback instruction to the speaker device 100 by a predetermined time from the timing of sending the music playback instruction to the keyboard device 200. Alternatively, the supply unit 300 may shift the timing of sending the music playback instruction to the keyboard device 200 to a predetermined time earlier than the timing of sending the music playback instruction to the speaker device 100.

[0085] (12) In the embodiments described above, the keys 202 in the keyboard device 200 are driven by the key drive unit 230. However, the disclosure is not limited thereto. For example, instead of the configuration in which the keys 202 themselves are driven, or in addition to the configuration in which the keys 202 themselves are driven, the illumination of a light-emitting part built into the key may be controlled based on performance data.

[0086] The embodiments and modifications described above as one embodiment of this disclosure can be combined as appropriate, insofar as they do not contradict each other. Furthermore, any configurations based on the embodiments, in which a person skilled in the art has added, deleted, or modified components, or added, omitted, or modified processes, are also included in the scope of the invention, as long as they retain the gist of this disclosure. [Explanation of symbols]

[0087] 10...Sound output device, 100, 100A...Speaker device, 101...Sound data acquisition unit 101, 103...Equalizer, 105...D / A converter, 107...Amplification unit, 109...Speaker unit, 200...Operation device (keyboard device), 211...Control unit, 212...Storage unit, 213...Operation panel, 214...Communication unit, 215...Sound source unit, 216...Interface, 217...Bus, 222...Key sensor, 223...Pedal sensor, 230...Key drive unit, 233...Pedal drive unit, 244...Stopper drive unit, 260...Touch panel, 300...Supply unit, 311...Control unit, 312...Storage unit, 313...Operation unit, 314...Communication unit, 400...Server, 411...Control unit, 412...Storage unit, 413...Performance data generation unit, 414...Communication unit, 611...Key drive control unit, 612...Stopper drive control unit, 613...Pedal drive control unit, 911...Instrument sound selection unit, 912...Reverberation processing unit, 913...Data generation unit

Claims

1. A speaker device that outputs sound according to the supplied sound data, An operating device including one or more controls, a drive unit that drives the one or more controls based on performance data supplied in synchronization with the sound data, and a drive control unit that controls the drive unit, A performance data generation unit that generates performance data based on the acquired sound data, A sound output system equipped with [specific features / features].

2. The sound output system according to claim 1, wherein the operating device includes a supply unit that supplies the sound data to the speaker device and the performance data to the drive control unit.

3. The sound output system according to claim 1, further comprising a supply unit that supplies the sound data to the speaker device and the performance data to the drive control unit.

4. The system further includes a storage unit for associating and storing the sound data and the performance data. The sound output system according to claim 2 or 3, wherein the supply unit reads the sound data and the performance data from the storage unit.

5. The sound output system according to claim 1, wherein the sound data is a performance sound including one or more instrument sounds, and the performance data generation unit generates the performance data based on a predetermined instrument sound among the sounds corresponding to the sound data.

6. The sound output system according to claim 1, further comprising a reverberation processing unit that removes reverberation components from the sound corresponding to the acquired sound data.

7. The sound output system according to claim 1, further comprising an adjustment unit that performs delay processing to delay the sound data relative to the performance data.

8. The operating device is a keyboard device that includes multiple keys as one or more operating elements, The aforementioned keyboard device is Multiple hammers that are linked to each of the aforementioned multiple keys, A stopper prevents the aforementioned multiple hammers from striking the string, A stopper drive unit that drives the stopper, The stopper drive unit is controlled by a stopper drive control unit, Includes, The sound output system according to claim 1, wherein the stopper drive control unit controls the stopper drive unit to prevent the stopper from striking the string while the drive unit drives the plurality of keys, in synchronization with the output of sound corresponding to the sound data from the speaker device.

Images