Sound generation control device

Abstract

To use a measurement signal including information on a depression amount of a key for performance support.SOLUTION: A sound generation control device includes: first and second acquisition parts; first and second specification parts; and first and second generation parts. The first acquisition part acquires a sound generation control signal. The second acquisition part acquires a key operation signal. The first specification part specifies a performance prediction section based on music data and the sound generation control signal. The second specification part specifies a prediction pitch and a sound generation prediction period based on the key operation signal. When the sound generation control signal is acquired, the first generation part generates a first instruction signal for allowing a sound source device to generate a sound signal when a pitch in the sound generation control signal is included in a pitch to be played and sound generation intensity is less than a prescribed value. The second generation part generates a second instruction signal for driving a keyboard device so that a sound generation body produces sound when the sound generation control signal is not acquired during the sound generation prediction period.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a method for assisting performance.

Background Art

[0002] Keyboard instruments such as electronic pianos generate control signals including sound emission intensity and sound emission timing using measurement signals obtained by measuring movements such as keys. The sound source device generates a sound signal by receiving the control signal. The measurement signal is also being considered for other uses. For example, in order to realize ensemble performance with a performer via a network with low latency, instead of transmitting the control signal generated on the transmission side, a technique has been proposed in which the measurement signal is transmitted to another communication base and the control signal is generated on the receiving side. This technique is disclosed in, for example, Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The measurement signal contains a lot of information such as the history of the key depression amount. In order to effectively utilize such information, there is also a demand for utilization methods in other applications.

[0005] One of the objects of the present invention is to use a measurement signal including information on the key depression amount for performance support.

Means for Solving the Problems

[0006] The pronunciation control device in one embodiment includes a first acquisition unit, a second acquisition unit, a first identification unit, a second identification unit, a first generation unit, and a second generation unit. The first acquisition unit acquires a pronunciation control signal including a pitch and a pronunciation intensity that are pronounced corresponding to a key pressed in a keyboard device including a key and a sounding body that sounds when the key is pressed. The second acquisition unit acquires a key operation signal including a key and the amount of depression of the key. The first identification unit identifies a performance prediction section based on the piece data and the pronunciation control signal in which the pitch constituting the piece and the pronunciation timing of the pitch are defined. The second identification unit identifies a predicted pitch and a pronunciation prediction period related to the pronunciation predicted based on the key operation signal. When the first generation unit acquires the pronunciation control signal, if the pitch in the pronunciation control signal is included in the performance target pitch corresponding to the performance prediction section and the pronunciation intensity corresponding to the pitch is less than a predetermined value, the first generation unit generates a first instruction signal for causing a sound source device to generate a sound signal corresponding to the pitch. The second generation unit generates a second instruction signal for driving the keyboard device so that the sounding body corresponding to the predicted pitch sounds when the pronunciation control signal corresponding to the predicted pitch is not acquired during the pronunciation prediction period.

[0007] The pronunciation control method in one embodiment is executed in a control device that acquires a pronunciation control signal including a pitch and a pronunciation intensity that are pronounced corresponding to a key pressed in a keyboard device including a key and a sounding body that sounds when the key is pressed, and a key operation signal including a key and the amount of depression of the key. The pronunciation control method includes identifying a performance prediction section based on the piece data and the pronunciation control signal in which the pitch constituting the piece and the pronunciation timing of the pitch are defined, identifying a predicted pitch and a pronunciation prediction period related to the pronunciation predicted based on the key operation signal, generating a first instruction signal for causing a sound source device to generate a sound signal corresponding to the pitch when the pitch in the pronunciation control signal is included in the performance target pitch corresponding to the performance prediction section and the pronunciation intensity corresponding to the pitch is less than a predetermined value when the pronunciation control signal is acquired, and generating a second instruction signal for driving the keyboard device so that the sounding body corresponding to the predicted pitch sounds when the pronunciation control signal corresponding to the predicted pitch is not acquired during the pronunciation prediction period.

Advantages of the Invention

[0008] According to the present invention, a measurement signal including information on the key depression amount can be used for performance support.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0010] Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings. The following embodiments are examples, and the present invention is not construed as being limited to these embodiments.

[0011] <Embodiment> [Summary] The automatic-playing piano in one embodiment includes a function to assist the user's performance. In this example, the performance assistance function includes functions such as accompanying following the user's performance, generating accompaniment sounds by the user pressing keys, and enabling even a user with a weak key-pressing force to perform. The automatic-playing piano in one embodiment can also be referred to as a performance assistance system that realizes the performance assistance function.

[0012] [Automatic-Playing Piano] FIG. 1 is a diagram for explaining the configuration of an automatic-playing piano in one embodiment. FIG. 2 is a diagram for explaining the internal configuration of an automatic-playing piano in one embodiment. The automatic-playing piano 1 includes a keyboard instrument 10, a control unit 20, a sensor 30, and a driving device 40.

[0013] The keyboard instrument 10 is an example of a keyboard device and corresponds to, for example, a grand piano. The keyboard instrument 10 includes a plurality of keys 12. The keyboard instrument 10 includes hammers 14, strings 15, and dampers 18 provided corresponding to each key 12. The keyboard instrument 10 includes a plurality of pedals 13. The plurality of pedals 13 are, for example, a damper pedal, a shift pedal, and a sostenuto pedal. The keyboard instrument 10 further includes a soundboard 17 through which the vibration of the strings 15 is transmitted via the bridges 16, etc.

[0014] In FIG. 2, the configurations provided corresponding to the key 12 and the pedal 13 are shown by focusing on each configuration provided corresponding to one key 12 and pedal 13. Therefore, descriptions of the respective configurations provided corresponding to other keys 12 and other pedals 13 are omitted.

[0015] The sensor 30 includes a key sensor 32, a pedal sensor 33, and a hammer sensor 34. The key sensor 32 is provided corresponding to each key 12 and outputs a measurement signal corresponding to the behavior of the key 12 to the control unit 20. In this example, the key sensor 32 outputs a measurement signal corresponding to the depression amount of the key 12 (which may be referred to as the position of the key 12) to the control unit 20. The position of the key 12 may be measured as a continuous quantity (fine resolution), or may be measured by detecting that the key 12 has passed through a predetermined position. The position where the key 12 is detected may be a plurality of positions within the range from the rest position to the end position.

[0016] The hammer sensor 34 is provided corresponding to each hammer 14 and outputs a measurement signal corresponding to the behavior of the hammer 14 to the control unit 20. In this example, the hammer sensor 34 measures the position (rotation amount) of the hammer shank immediately before the hammer 14 strikes the string 15, and outputs a measurement signal corresponding to the measurement result to the control unit 20. The position of the hammer shank may be measured as a continuous quantity (fine resolution), or may be measured by detecting that the hammer shank has passed through a predetermined position. The position where the hammer shank is detected may be a plurality of positions within the range immediately before the hammer 14 strikes the string 15.

[0017] The pedal sensor 33 is provided corresponding to each pedal 13 and outputs a measurement signal corresponding to the behavior of the pedal 13 to the control unit 20. In this example, the pedal sensor 33 outputs a measurement signal corresponding to the position (depression amount) of the pedal 13 to the control unit 20. The position of the pedal 13 may be detected as a continuous quantity (fine resolution), or may be detected by the fact that the pedal 13 has passed through a predetermined position. The position where the pedal 13 is detected may be a plurality of positions within the depression range of the pedal 13 (the range from the rest position to the end position).

[0018] The drive device 40 includes a key drive device 42, a pedal drive device 43, a stopper 44, a vibrator 47, and a damper drive device 48. The key drive device 42 is provided corresponding to each key 12 and is driven to press the key 12 based on the control from the control unit 20. Thereby, the same situation as when the performer presses the key 12 is mechanically reproduced. The pedal drive device 43 is provided corresponding to each pedal 13 and is driven to press the pedal 13 based on the control from the control unit 20. Thereby, the same situation as when the performer steps on the pedal 13 is mechanically reproduced. The damper drive device 48 is provided corresponding to each damper 18 and is driven to separate the damper 18 from the string 15 based on the control from the control unit 20. The damper drive device 48 may be configured to drive all the dampers 18 simultaneously.

[0019] The stopper 44 is driven based on the control from the control unit 20 so as to be in either a position where it collides with the hammer shank (blocking position) or a position where it does not collide with the hammer shank (retracted position). When the stopper 44 is in the blocking position, even if the key 12 is pressed, the movement of the hammer shank is restricted and the hammer 14 does not strike the string 15. When the stopper 44 is in the retracted position, when the key 12 is pressed, the hammer 14 interlocked with the key 12 strikes the string 15. When the string 15 is struck, the keyboard instrument 10 generates a sound. When the performance support function described below is realized, the stopper 44 is controlled to be in the retracted position. The performance support function may be realized in a state where the stopper 44 is controlled to be in the blocking position and the sound generation using the sound source device 25 is realized.

[0020] In this example, the vibrator 47 is supported by a support portion connected to the straight support column 19 so as to contact the surface of the soundboard 17 on the opposite side of the portion where the piece 16 is disposed. The vibrator 47 vibrates the soundboard 17 based on the control from the control unit 20. For example, when a drive signal including a piano sound is supplied from the control unit 20 to the vibrator 47, the vibrator 47 applies vibrations corresponding to the drive signal to the soundboard 17. As a result, a piano sound is emitted from the soundboard 17. A plurality of vibrators 47 may be arranged to contact the soundboard 17. Instead of the vibrator 47 that vibrates the soundboard 17, a speaker that emits sound may be used.

[0021] The sound production by the keyboard instrument 10 includes a case where the hammer 14 strikes the string 15 which is the sound-producing body, and a case where the vibrator 47 vibrates the soundboard 17 which is the sound-producing body. Therefore, it can also be said that the keyboard instrument 10 includes a sound-producing device that generates a string sound by driving the key 12, and a sound-producing device that generates sound from the soundboard 17 by driving the vibrator 47.

[0022] The configuration of the control unit 20 will be described. In this example, the control unit 20 is attached to the keyboard instrument 10.

[0023] FIG. 3 is a diagram for explaining the configuration of the control unit in one embodiment. The control unit 20 includes a control device 21, a storage device 22, an operation device 23, a communication device 24, a sound source device 25, and an interface 26. These components are connected via a bus 27.

[0024] The control device 21 is an example of a computer including a processor such as a CPU and a storage device such as a RAM. The control device 21 executes the program stored in the storage device 22 using the CPU (processor), and realizes in the control unit 20 the functions for executing various processes. The functions realized in the control unit 20 include the performance support function described later. By this performance support function, each part of the control unit 20 and each component connected to the interface 26 are controlled in various ways. That is, the performance support function is an example of the functions realized by the sound generation control method executed by the control device 21.

[0025] The storage device 22 is a device such as a non-volatile memory and a hard disk drive. The storage device 22 includes a program storage area 22a and a music data storage area 22b. The program storage area 22a stores the program executed in the control device 21 and various data required when executing this program. The music data storage area 22b stores music data.

[0026] FIG. 4 is a diagram for explaining the configuration of music data in an embodiment. The music data Md is, for example, data described in the MIDI format. The music data Md indicates each sound constituting the music, and is data that defines the pitch and the sounding timing corresponding to each sound. The music data Md includes performance schedule data Mp, first accompaniment data Mt, and second accompaniment data Mb. Each track is distinguished by a track, for example.

[0027] The performance schedule data Mp is data indicating the part to be performed by the user, and includes, for example, data defining each sound constituting the melody of the music. The first accompaniment data Mt and the second accompaniment data Mb include data defining each sound constituting the accompaniment of the music. The first accompaniment data Mt is data defining the sounds to be sounded at the same timing as the user's performance sound among the accompaniment sounds. The second accompaniment data Mb is data defining the sounds to be made to follow the user's performance among the accompaniment sounds.

[0028] Returning to FIG. 3, the description will continue. The operation device 23 has operation buttons and the like for receiving user operations. When a user operation is received by this operation button, an operation signal corresponding to the operation is output to the control device 21. The operation device 23 may have a display screen. In this case, the operation device 23 may be a touch panel in which a touch sensor is combined with the display screen. The operation device 23 receives, for example, an operation for specifying a piece of music that the user intends to play.

[0029] The communication device 24 is a communication module that communicates with other devices such as a server and a smartphone by wireless, wired, or the like.

[0030] The sound source device 25 generates a sound signal under the control from the control device 21. The sound signal is used for a drive signal or the like for driving the vibrator 47. In this example, the sound signal includes a signal indicating the sound of a piano. The sound source device 25 is controlled, for example, by a tone generation control signal described in the MIDI format such as note on, note off, note number, and velocity. Although the signal for generating the sound signal in the sound source device 25 is generated in the performance support function described later, it may be generated so as to generate a sound signal indicating the sound of a piano according to the performance content of the user.

[0031] The interface 26 is an interface that connects the control unit 20 and each external component. Each component connected to the interface 26 includes, as described above, the sensor 30 and the drive device 40 in this example. The interface 26 outputs the signal output from the sensor 30 to the control device 21. The interface 26 outputs a signal for driving each device to the drive device 40. The signal for driving each device in the drive device 40 is generated in the performance support function described later. The above-described sound source device 25 may be a component connected to the control unit 20 via the interface 26.

[0032] [Performance Support Function] Next, the performance support function realized by the control device 21 executing a program will be described. The configuration for realizing the performance support function is not limited to the case where it is realized by executing a program, and at least a part of the configuration may be realized by hardware. The configuration for realizing the performance support function may be realized by a device (for example, a computer on which this program is installed) connected to the interface 26 instead of the control unit 20.

[0033] FIG. 5 is a diagram for explaining the configuration of the performance support function in one embodiment. The performance support function 100 includes a control signal acquisition unit 110, an operation signal acquisition unit 120, a position identification unit 130, an accompaniment generation unit 140, a synchronization sound generation unit 200, and a performance assistance unit 300.

[0034] The control signal acquisition unit 110 acquires a tone generation control signal Sc based on the measurement signal output from the hammer sensor 34. The tone generation control signal Sc is event information including information such as, for example, a note number, note on, and velocity, and includes information described in, for example, the MIDI format. That is, the tone generation control signal Sc includes information on the sound to be generated. In the case of a note-on event, the tone generation control signal Sc includes, in this example, the pitch (note number) and the tone generation intensity (velocity).

[0035] When the measurement signal indicates that the hammer 14 strikes the string 15, the control signal acquisition unit 110 generates a note-on, a note number indicating the pitch corresponding to the hammer 14, and a tone generation intensity corresponding to the speed of the hammer 14, thereby acquiring the tone generation control signal Sc corresponding to the note-on event. The tone generation control signal Sc indicating a note-off event is generated based on the measurement signal output from the key sensor 32.

[0036] The operation signal acquisition unit 120 acquires a key operation signal Sp based on the measurement signal output from the key sensor 32. The key operation signal Sp includes, for example, a note number indicating the pitch corresponding to the key 12 and the depression amount of the key 12. When it is indicated by the measurement signal that the key 12 has been depressed, the operation signal acquisition unit 120 acquires the key operation signal Sp by generating the note number and the depression amount. A specific example of the method for generating the depression amount in the key operation signal Sp will be described later.

[0037] As will be described later, the performance support function 100 controls the drive device 40. The acquired key operation signal Sp is acquired when it corresponds to the key 12 depressed by the user. The operation signal acquisition unit 120 does not acquire the key operation signal Sp based on the measurement signal generated when the key 12 is depressed by this control.

[0038] The position specifying unit 130 specifies the position (hereinafter referred to as the performance position Pp) in the piece being played by the user based on the sound generation control signal Sc and the piece data Md. This piece data Md is data corresponding to a piece specified in advance by the user. A known method may be used to specify the performance position Pp. The performance position Pp may be specified using the sound generation control signal Sc including the virtual sound generation control signal Sv described later.

[0039] The accompaniment generation unit 140 reads information corresponding to the performance position Pp from the second accompaniment data Mb and generates a drive instruction signal Sb based on the read information. The information read from the second accompaniment data Mb corresponds to a position (hereinafter referred to as the read position) after a predetermined time has elapsed from the performance position Pp in this example. The drive instruction signal Sb is a signal for driving the key 12 so that the string 15 corresponding to the pitch in the read information sounds. Therefore, the key drive device 42 in the drive device 40 drives the key 12 to be depressed based on the drive instruction signal Sb. At this time, the damper 18 may also be driven based on the second accompaniment data Mb. The damper 18 may be driven by the damper drive device 48 or may be driven by depressing the pedal 13 by the pedal drive device 43.

[0040] When the reading position is specified as a timing slightly after the performance position Pp, the influence of the delay for driving in the key driving device 42 is reduced. Even when the influence of the delay is small, it is less likely to cause a sense of discomfort in the performance that the accompaniment is sounded slightly earlier than the melody.

[0041] The synchronous sound generation unit 200 reads information corresponding to the performance position Pp from the first accompaniment data Mt, and generates a drive instruction signal St based on the read information. The synchronous sound generation unit 200 generates the drive instruction signal St at the timing specified from the key operation signal Sp. The key operation signal Sp used here is a signal corresponding to the key 12 of the pitch specified based on the performance schedule data Mp and the performance position Pp.

[0042] The drive instruction signal St is a signal for driving the key 12 so that the string 15 corresponding to the pitch in the read information is sounded. Therefore, in the drive device 40, the key drive device 42 drives to press the key 12 based on the drive instruction signal St. At this time, the damper 18 may also be driven based on the information read from the first accompaniment data Mt.

[0043] According to the drive instruction signal St output from the synchronous sound generation unit 200, the key 12 corresponding to the sound defined in the first accompaniment data Mt is driven at the same timing as the performance sound of the user. The sound generated by driving the key 12 in this way may be referred to as a synchronous sound in the following description. The configuration realized in the synchronous sound generation unit 200 will be described later.

[0044] The performance assistance unit 300 generates a drive instruction signal Sa or a sound emission instruction signal Ss using performance schedule data Mp, a performance position Pp, a sound production control signal Sc, and a key operation signal Sp. The drive instruction signal Sa is a signal for driving the key 12 so that the string 15 is sounded when the user presses the key 12 to be played but fails to sound the string 15. The sound emission instruction signal Ss is a signal for causing the sound source device 25 to generate a sound signal of a pitch corresponding to the string 15 when the user presses the key 12 to be played and the sound is too soft. The sound signal generated in the sound source device 25 is used to drive the vibrator 47 and is emitted as sound through the soundboard 17. The configuration realized in the performance assistance unit 300 will be described later.

[0045] [Synchronous sound generation unit] Subsequently, the configuration for realizing the synchronous sound generation unit 200 will be described with reference to FIG. 6.

[0046] FIG. 6 is a diagram for explaining the configuration of the synchronous sound generation unit in one embodiment. The synchronous sound generation unit 200 includes a section specifying unit 210, a predicted sound specifying unit 220, a performance target specifying unit 230, a target determination unit 240, a reproduction target specifying unit 250, a drive instruction generation unit 260, and a virtual generation unit 290.

[0047] The section specifying unit 210 specifies a performance prediction section Pd1 based on the performance position Pp. The performance prediction section Pd1 corresponds to a predetermined period including the performance position Pp. The performance prediction section Pd1 is defined as a range of the first time Tp on the front side and the second time Tf on the rear side with respect to the performance position Pp. The first time Tp is shorter than the second time Tf, but they may be the same or longer. Also, the first time Tp may be "0" and the performance position Pp may be the start point of the performance prediction section Pd1. The performance prediction section Pd1 may or may not include the above-described reading position. The reading position may be the end point of the performance prediction section Pd1.

[0048] The predicted sound specifying unit 220 predicts the sound to be produced corresponding to each key 12 based on the key operation signal Sp. More specifically, the predicted sound specifying unit 220 predicts the change in the pressing amount of the subsequent key 12 based on the pressing amount of the key 12 sequentially acquired, and determines whether the string 15 is struck by the hammer 14 by a predetermined calculation. When the predicted sound specifying unit 220 determines that the string 15 is struck, it predicts that the sound corresponding to that key 12 will be produced. When the predicted sound specifying unit 220 predicts that a sound will be produced, it outputs information indicating the pitch of that sound (hereinafter referred to as predicted pitch information).

[0049] The performance target specifying unit 230 specifies the performance target pitch from the information included in the performance prediction section Pd1 in the performance schedule data Mp. Since the performance schedule data Mp is data indicating the part that the user should perform as described above, the performance target pitch corresponds to the pitch corresponding to the key 12 that the user should perform in the performance prediction section Pd1. The performance target pitch may indicate a plurality of pitches.

[0050] When the predicted pitch information is output from the predicted sound specifying unit 220, the target determination unit 240 determines whether the predicted pitch information is included in the performance target pitch. When the predicted pitch information is included in the performance target pitch, the target determination unit 240 outputs a reproduction instruction.

[0051] Since the performance target pitch indicates the pitch to be performed in the performance prediction section Pd1, when the user starts pressing the key 12 corresponding to the progress of the song, it is determined that the predicted pitch information is included in the performance target pitch. On the other hand, when the user accidentally presses the key 12 that has nothing to do with the progress of the song, it means that the predicted pitch information is not included in the performance target pitch.

[0052] The reproduction target specifying unit 250 specifies the pitch of the sound to be reproduced, that is, the pitch from the accompaniment sound, from the information included in the first accompaniment data Mt within the performance prediction section Pd1, and buffers the note events related to that pitch. As the performance prediction section Pd1 moves, the buffered note events are updated. The note events may include information for driving the pedal 13 or the damper 18.

[0053] When a reproduction instruction is output from the target determination unit 240, the reproduction target specifying unit 250 outputs the buffered note events. However, when a reproduction instruction is output from the target determination unit 240, if the same note events as the buffered note events were output at a predetermined time before that output timing, the reproduction target specifying unit 250 does not output those note events. This prevents the phenomenon where, when the user continuously presses the same key 12, the same note events occur even though they are not defined in the first accompaniment data Mt.

[0054] Based on the note events output from the reproduction target specifying unit 250, the drive instruction generation unit 260 outputs a drive instruction signal St to the drive device 40. Using the drive instruction signal St, the key drive device 42 starts to drive to press the key 12 corresponding to the note event, and then, in response to the pressing of the key 12, the string 15 sounds. By generating the drive instruction signal St in this way, in conjunction with the pressing of the key 12 by the user, the key 12 corresponding to the accompaniment according to the first accompaniment data Mt is also pressed. As a result, the sounds (performance sound and synchronized sound) corresponding to both presses can be made to occur almost simultaneously.

[0055] When the target determination unit 240 outputs a reproduction instruction, the virtual generation unit 290 generates a virtual pronunciation control signal Sv. The pronunciation control signal Sv indicates the pronunciation predicted by the predicted sound identification unit 220. That is, the pronunciation control signal Sv includes information corresponding to the predicted pitch. The pronunciation control signal Sv is provided to the position identification unit 130 and used for identifying the performance position Pp in the position identification unit 130. After the pronunciation control signal Sv is generated, when the key 12 is further pressed to generate a pronunciation control signal Sc, the pronunciation control signal Sc is not used for identifying the performance position Pp in the position identification unit 130, but the pronunciation control signal Sv is used. Therefore, the pronunciation control signal Sv is also used for identifying the performance prediction section Pd1.

[0056] The virtual generation unit 290 may generate the pronunciation control signal Sv not when the target determination unit 240 outputs a reproduction instruction, but after a predetermined time of the reproduction instruction (for example, the time assumed to reach the actual pronunciation from the reproduction instruction). The virtual generation unit 290 may not exist.

[0057] [Performance assistance unit] Subsequently, the configuration for realizing the performance assistance unit 300 will be described with reference to FIG. 7.

[0058] FIG. 7 is a diagram for explaining the configuration of the performance assistance unit in an embodiment. The performance assistance unit 300 includes a section identification unit 310, a predicted sound identification unit 320, a performance target identification unit 330, a drive determination unit 350, a drive instruction generation unit 360, a speed determination unit 370, and a pronunciation instruction generation unit 380.

[0059] The section identification unit 310 identifies a performance prediction section Pd2 based on the performance position Pp. Similar to the performance prediction section Pd1 described above, the performance prediction section Pd2 corresponds to a predetermined period including the performance position Pp. The performance prediction section Pd2 may be the same section as the performance prediction section Pd1 or may be different. When the performance prediction section Pd1 and the performance prediction section Pd2 are the same section, the section identification unit 210 and the section identification unit 310 may be realized by one configuration.

[0060] The predicted sound specifying unit 320 predicts the sound to be produced corresponding to each key 12 based on the key operation signal Sp. The prediction method is the same as that in the predicted sound specifying unit 220. When the predicted sound specifying unit 320 predicts that a sound will be produced, it outputs predicted pitch information. The predicted sound specifying unit 220 and the predicted sound specifying unit 320 may be realized in one configuration. The prediction method in the predicted sound specifying unit 320 may be different from that in the predicted sound specifying unit 220. For example, the predicted sound specifying unit 320 may be set so that it is easier to predict that a sound will be produced than in the predicted sound specifying unit 220. That is, according to the predicted sound specifying unit 320, even if the depression of the key 12 is weaker, it may be predicted that a sound will be produced.

[0061] Similar to the performance target specifying unit 230 described above, the performance target specifying unit 330 specifies the performance target pitch from the information included in the performance prediction section Pd2 in the performance schedule data Mp. The performance target specifying unit 230 and the performance target specifying unit 330 may be realized in one configuration.

[0062] When the predicted pitch information is output from the predicted sound specifying unit 320, the drive determination unit 350 determines whether the control signal acquisition unit 110 has acquired the sound generation control signal Sc by the depression of the key 12 corresponding to the predicted pitch during the sound generation prediction period. The sound generation prediction period is, for example, the period until a predetermined time has elapsed after the predicted pitch information is output. When the drive determination unit 350 determines that the target sound generation control signal Sc has not been acquired during the sound generation prediction period, it outputs a note event for generating the predicted pitch at a predetermined intensity.

[0063] When the drive determination unit 350 determines that the target sound generation control signal Sc has not been acquired during the sound generation prediction period and the predicted pitch is included in the performance target pitch, it may output a note event for generating the predicted pitch at a predetermined intensity. When the sound generation intensity corresponding to the performance target pitch is defined in the performance schedule data Mp, that sound generation intensity may be set as the sound generation intensity of the note event. The sound stop event may be generated after a predetermined time has elapsed after the note event is output, or may be generated according to the note-off event corresponding to the performance target pitch.

[0064] Based on the note event output from the drive determination unit 350, the drive instruction generation unit 360 outputs a drive instruction signal Sa to the drive device 40. The key drive device 42 in the drive device 40 drives to press the key 12 based on the drive instruction signal Sa.

[0065] The situation where a note event is output from the drive determination unit 350 corresponds to a situation where the user starts pressing the key 12 but the key 12 returns before reaching the position where sound is produced. In this case, the key 12 is pressed by the drive instruction signal Sa. Therefore, even for a user with a weak force to press the key 12 and unable to move it to the end position, if the key 12 can be moved slightly, the key 12 is driven to reach the sound production, so the performance is assisted.

[0066] Furthermore, when the predicted pitch is included in the performance target pitch, if a note event is output from the drive determination unit 350, the performance can also be assisted only when the key 12 to be played according to the progress of the music is pressed.

[0067] When the pronunciation control signal Sc indicating the pitch included in the performance target pitch is acquired during the pronunciation prediction period, the speed determination unit 370 outputs a note event when the pronunciation intensity indicated by the pronunciation control signal Sc is less than a predetermined value. The note event includes information for pronouncing the pitch indicated by the pronunciation control signal Sc at a predetermined intensity. The pronunciation intensity indicated by the note event may be constant regardless of the pronunciation intensity indicated by the acquired pronunciation control signal Sc. The pronunciation intensity indicated by the note event may vary according to the pronunciation intensity indicated by the pronunciation control signal Sc, may have a positive correlation, or may have a negative correlation. When the pronunciation intensity corresponding to the performance target pitch is defined in the performance schedule data Mp, that pronunciation intensity may be set as the pronunciation intensity of the note event. The pronunciation stop event may occur after a predetermined time has elapsed after the note event is output, or may occur according to the note-off event corresponding to the performance target pitch.

[0068] Based on the note event output from the speed determination unit 370, the pronunciation instruction generation unit 380 outputs a pronunciation instruction signal Ss to the sound source device 25. The sound source device 25 generates a sound signal based on the pronunciation instruction signal Ss.

[0069] The situation where a note event is output from the speed determination unit 370 corresponds to a situation where the string 15 produces a small sound because the force with which the user presses the key 12 is weak. In this case, a sound corresponding to the sound signal generated by the sound source device 25 according to the pronunciation instruction signal Ss, that is, a piano sound with the same pitch as the string 15, is emitted from the soundboard 17. As a result, the sound produced by the string 15 and the sound produced by the soundboard 17 are combined, and the overall pronunciation intensity can be increased. Therefore, even for a user who presses the key 12 with a weak force and the string 15 can only produce a small sound, the pronunciation intensity can be reinforced by the vibration of the soundboard 17, so the performance is assisted.

[0070] Since the pronunciation is controlled in various ways by the performance support function 100 in this way, the control device 21 is an example of a pronunciation control device for realizing the performance support function 100. The above is the description of the performance support function 100.

[0071] [Example of key operation signal] An example of a method for generating a key operation signal Sp when the operation signal acquisition unit 120 acquires the key operation signal Sp based on the measurement signal output from the key sensor 32 will be described. In this example, the key sensor 32 measures the range from the rest position to the end position as a continuous quantity (fine resolution). In this example, when the key 12 is at the rest position, it is measured as 0 mm, and when the key 12 is at the end position, it is measured as 10 mm.

[0072] In this example, the key operation signal Sp includes information indicating the section where the key 12 exists and the note number corresponding to the key 12. The section corresponds to the range obtained by dividing the range from the rest position to the end position into 1-mm intervals. In the following description, the information indicating the section is defined as section (X) if the position of the key 12 is X mm or more and less than X + 1 mm.

[0073] For example, when the key 12 is in the rest position, it is measured as 0 mm, so it becomes the section (0). When the key 12 is in the end position, it is measured as 10 mm, so it becomes the section (10). The operation signal acquisition unit 120 acquires such a key operation signal Sp by generating it. A method for generating the key operation signal Sp executed in the operation signal acquisition unit 120 will be described. In the following description, the key operation signal Sp(Z) indicates that the key 12 exists in the section (Z). Here, Z is any integer from "0" to "10".

[0074] FIG. 8 is a flowchart for explaining the process of a method for generating a key operation signal in an embodiment. When the performance support function 100 is realized, the process flow of the method for generating the key operation signal by the operation signal acquisition unit 120 starts corresponding to each key 12. The operation signal acquisition unit 120 samples the measurement signal at predetermined time intervals.

[0075] The operation signal acquisition unit 120 refers to the measurement signal and waits until the key 12 moves from the section (0) to the section (1) (step S101; No). When the key 12 moves to the section (1) (step S101; Yes), the operation signal acquisition unit 120 generates the key operation signal Sp(1) (step S103).

[0076] Subsequently, the operation signal acquisition unit 120 refers to the measurement signal and waits until the key 12 moves from the section (1) to the section (0) or moves through multiple sections (two or more sections in this example) (step S111; No, step S121; No). When the key 12 moves from the section (1) to the section (0) (step S111; Yes), the operation signal acquisition unit 120 generates the key operation signal Sp(0) (step S113) and waits again until the key 12 moves from the section (0) to the section (1) (step S101; No).

[0077] When the key 12 moves between sections in two steps (step S121; Yes), the operation signal acquisition unit 120 generates a key operation signal Sp(X) corresponding to the moved section (X) (step S123). Moving between sections in two steps means, when the key operation signal Sp(Y) was last generated, moving to a section other than section (Y + 1), section (Y), and section (Y - 1). For example, if it moves to section (Y + 1) and then to section (Y), the section has only been moved twice, and the key 12 is in the same section. Therefore, in this case, it is not considered to have moved between sections in two steps.

[0078] The operation signal acquisition unit 120 generates a key operation signal Sp(X) corresponding to the moved section (X) (step S123), and then waits again until the key 12 moves from section (1) to section (0) or moves between sections in two or more steps (step S111; No, step S121; No). The above is the explanation of the method for generating the key operation signal.

[0079] FIG. 9 is a diagram showing the key operation signal generated for the operation example of the key. FIG. 9 is a graph showing the depression amount of the key 12 on the vertical axis and time on the horizontal axis. The black dots correspond to the positions of the key 12 indicated by the sampled measurement signals. The line connecting each point shows the change in the depression amount of the key 12 over time. The information in the graph shows the timing at which the key operation signal Sp is output and its content.

[0080] The tone generation control signal Sc is also shown below the graph for reference. The description of "ON" indicates the timing at which the tone generation control signal Sc indicating a note-on event is generated based on the measurement signal from the hammer sensor 34. In this example, the note-on event does not use the measurement signal from the key sensor 32. When the automatic-playing piano 1 does not use the hammer sensor 34, the timing of note-on may be determined using the measurement signal from the key sensor 32.

[0081] The description of "OFF" indicates the timing at which a sound production control signal Sc indicating a note-off event is generated based on the measurement signal from the key sensor 32. The note-off event is generated when the key depression amount of the key 12 changes to 4.5 mm or less after the note-on event is generated. An example where the note-off event is generated regardless of the sampling timing of the measurement signal is shown in FIG. 9, but it may also be generated based on the value at the sampling timing.

[0082] [Method for generating synchronized sound] Each process of the method for generating synchronized sound in the synchronized sound generation unit 200 will be described with reference to FIG. 10.

[0083] FIG. 10 is a flowchart for explaining the process of the method for generating synchronized sound in one embodiment. When the performance support function 100 is realized, the processing flow of the method for generating synchronized sound is started by the synchronized sound generation unit 200. The synchronized sound generation unit 200 starts specifying a performance prediction section Pd1 based on the performance position Pp (step S201). The synchronized sound generation unit 200 continues to update the performance prediction section Pd1 following the performance position Pp. The synchronized sound generation unit 200 waits until it acquires the key operation signal Sp (step S203; No). When the synchronized sound generation unit 200 acquires the key operation signal Sp (step S203; Yes), it specifies the predicted pitch based on the key operation signal Sp (step S205).

[0084] When the predicted pitch is included in the performance target pitch specified based on the performance prediction section Pd1 (step S211; Yes), the synchronized sound generation unit 200 generates a drive instruction signal St (step S215). When the predicted pitch is not included in the performance target pitch (step S211; No), and after generating the drive instruction signal St, the synchronized sound generation unit 200 waits again until it acquires the key operation signal Sp (step S203; No).

[0085] [Performance assistance method] Each process of the performance assistance method in the performance assistance unit 300 will be described with reference to FIG. 11.

[0086] FIG. 11 is a flowchart for explaining the processing of the performance assistance method in one embodiment. When the performance support function 100 is realized, the processing flow of the performance assistance method is started by the performance assistance unit 300. The performance assistance unit 300 starts specifying the performance prediction section Pd2 based on the performance position Pp (step S301). The performance assistance unit 300 continues to update the performance prediction section Pd2 following the performance position Pp. The performance assistance unit 300 waits until it acquires the key operation signal Sp (step S303; No). When the performance assistance unit 300 acquires the key operation signal Sp (step S303; Yes), it specifies the predicted pitch based on the key operation signal Sp (step S305).

[0087] The performance assistance unit 300 waits until it acquires the pronunciation control signal Sc that satisfies a predetermined condition or until the pronunciation prediction period elapses (step S311; No, step S313; No). As described above, the pronunciation control signal Sc that satisfies the predetermined condition is the pronunciation control signal Sc indicating the pitch included in the performance target pitch. The pronunciation prediction period is the period until a predetermined time elapses after the predicted pitch is specified.

[0088] When the performance assistance unit 300 does not acquire the pronunciation control signal Sc that satisfies the predetermined condition until the pronunciation prediction period elapses (step S313; Yes), it generates the drive instruction signal Sa (step S315) and waits again until it acquires the key operation signal Sp (step S303; No).

[0089] When the performance assistance unit 300 acquires the pronunciation control signal Sc that satisfies the predetermined condition before the pronunciation prediction period elapses (step S311; Yes), and when the pronunciation intensity is less than a predetermined value (step S321; Yes), it generates the pronunciation instruction signal Ss (step S323) and then waits again until it acquires the key operation signal Sp (step S303; No). When the pronunciation intensity is not less than the predetermined value (step S321; No), the performance assistance unit 300 waits again until it acquires the key operation signal Sp (step S303; No).

[0090] <Modification Example> The present invention is not limited to the above-described embodiments, and includes various other modifications. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Some modifications will be described below.

[0091] (1) The keyboard instrument 10 in the automatic-playing piano 1 is not limited to an acoustic piano such as a grand piano, and may be an electronic keyboard instrument.

[0092] (2) The control unit 20 does not have to be a device attached to the keyboard instrument 10, and may be, for example, a personal computer, a tablet computer, a smartphone, or the like.

[0093] (3) The performance support function 100 includes the accompaniment generation unit 140, the synchronized sound generation unit 200, and the performance assistance unit 300 as a configuration for generating a drive instruction signal and a sound generation instruction signal, but may have at least one configuration.

[0094] (4) The performance assistance for the key 12 realized by the performance assistance unit 300 may be executed as performance assistance for pressing the pedal 13.

Explanation of Reference Numerals

[0095] 1: Automatic playing piano, 10: Keyboard instrument, 12: Keys, 13: Pedals, 14: Hammers, 15: Strings, 16: Pieces, 17: Cymbals, 18: Dampers, 19: Upright posts, 20: Control unit, 21: Control device, 22: Memory device, 23: Operating device, 24: Communication device, 25: Sound source device, 26: Interface, 27: Bus, 30: Sensors, 32: Key sensors, 33: Pedal sensors, 34: Hammer sensors, 40: Driving devices, 42: Key driving devices, 43: Pedal driving devices, 44: Stoppers, 47: Vibrators, 48: Damper driving devices, 100: Performance support function, 110: Control signal acquisition unit, 120: Motion signal acquisition unit, 130: Position identification unit, 140: Accompaniment generation unit, 200: Synchronized sound generation unit, 210: Section identification unit, 220: Predicted sound identification unit, 230: Performance target identification unit, 240: Target determination unit, 250: Playback target identification unit, 260: Driving instruction generation unit, 290: Virtual generation unit, 300: Performance assistance unit, 310: Section identification unit, 320: Predicted sound identification unit, 330: Performance target identification unit, 350: Driving determination unit, 360: Driving instruction generation unit, 370: Speed determination unit, 380: Sound production instruction generation unit

Claims

1. In a keyboard device including keys and a sounding body that sounds when a key is pressed, a first acquisition unit that acquires a sound generation control signal including a pitch and a sound generation intensity that are generated corresponding to the pressed key; a second acquisition unit that acquires a key operation signal including the key and the key depression amount; a first specifying unit that specifies a performance prediction section based on music data in which the pitch constituting a piece of music and the sounding timing of the pitch are defined and the sound generation control signal; a second specifying unit that specifies a predicted pitch related to the sound generation predicted based on the key operation signal and a sound generation prediction period; when the sound generation control signal is acquired, if the pitch in the sound generation control signal is included in the performance target pitch corresponding to the performance prediction section and the sound generation intensity corresponding to the pitch is less than a predetermined value, a first generation unit that generates a first instruction signal for causing a sound source device to generate a sound signal corresponding to the pitch; a second generation unit that generates a second instruction signal for driving the keyboard device so that the sounding body corresponding to the predicted pitch sounds when the sound generation control signal corresponding to the predicted pitch is not acquired during the sound generation prediction period; A sound generation control device including:

2. The music data includes performance schedule data and first accompaniment data, The performance target pitch includes the pitch included in the performance prediction section in the performance schedule data, When the predicted pitch is included in the performance target pitch, further including a third generation unit that generates a third instruction signal for driving the keyboard device so that the sounding body corresponding to the pitch specified based on the performance prediction section and the first accompaniment data sounds; The sound generation control device according to claim 1.

3. Further including a virtual generation unit that virtually generates the sound generation control signal corresponding to the predicted pitch when the third instruction signal is generated, When the sound generation control signal corresponding to the predicted pitch is not acquired from the keyboard device within a predetermined period after the first acquisition unit generates the third instruction signal, the first specifying unit further specifies the performance prediction section using the virtually generated sound generation control signal; The sound generation control device according to claim 2.

4. The music data further includes second accompaniment data, The first specifying unit specifies the performance prediction section so as to include a performance position specified based on the music data and the sound generation control signal. A fourth generation unit that generates a fourth instruction signal for driving the keyboard device so that the sounding body corresponding to the pitch defined in the second accompaniment data sounds according to the performance position. The sound production control device according to claim 3.

Images