Information processing device, performance device, method and program

Abstract

To enable users to intuitively understand how musical notes change during pedal operation.SOLUTION: An information processing device comprises a display unit and at least one processor. When the at least one processor acquires first performance information corresponding to an operation on a pedal of a performance device, an image simulating movement of dampers and hammers corresponding to the pedal operation is displayed on the display unit.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The disclosure of this specification relates to an information processing device, a performance device, a method, and a program. [Background technology]

[0002] BACKGROUND ART Electronic keyboard instruments are known that display the amount of pedal depression on a display unit (see, for example, Patent Document 1).

[0003] The electronic keyboard instrument described in Patent Document 1 detects the amount of pedal depression and displays the detected amount of depression alongside a model amount of depression, allowing the user to understand the appropriate amount of depression by comparing these amounts of depression. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-256871 Summary of the Invention [Problem to be solved by the invention]

[0005] With the electronic keyboard instrument described in Patent Document 1, even if the user can grasp the appropriate amount of pedal depression, it is difficult for the user to intuitively understand the principle by which musical tones change when the pedal is operated (changes in resonance, softness, etc.) In other words, the electronic keyboard instrument described in Patent Document 1 has room for improvement in terms of enabling the user to intuitively understand how musical tones change when the pedal is operated.

[0006] The embodiments of the present disclosure have been made in consideration of the above circumstances, and their purpose is to provide an information processing device, a performance device, a method, and a program that allow a user to easily and intuitively understand how musical tones change when pedals are operated. [Means for solving the problem]

[0007] An information processing device according to an embodiment of the present disclosure includes a display unit and at least one processor, and when the at least one processor acquires first performance information corresponding to an operation on a pedal of a performance device, the display unit displays an image simulating the movements of a damper and a hammer corresponding to the operation on the pedal. [Effects of the Invention]

[0008] According to one embodiment of the present disclosure, an information processing device, a performance device, a method, and a program are provided that allow a user to intuitively understand how musical tones change when a pedal is operated. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the appearance of an electronic musical instrument according to an embodiment of the present disclosure. [Figure 2] 1 is a block diagram illustrating a configuration of an electronic musical instrument according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a block diagram illustrating a configuration of an information processing device according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 7] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 9] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 10]FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram illustrating an example of an image displayed on a screen of an information processing device according to an embodiment of the present disclosure. [Figure 12] 1 is a flowchart of a process executed by a processor of an information processing device according to an embodiment of the present disclosure. [Figure 13] FIG. 10 is a diagram showing an example of an image displayed on a screen of an information processing device according to a first modification of the present disclosure. [Figure 14] FIG. 10 is a diagram showing an example of an image displayed on a screen of an information processing device according to a second modification of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following description relates to an information processing device, a performance device, a method, and a program according to an embodiment of the present disclosure. Common or corresponding elements are denoted by the same or similar reference numerals, and duplicate descriptions will be appropriately simplified or omitted.

[0011] 1 is an example of a musical performance device, such as an electronic keyboard, but the electronic musical instrument 1 may also be an electronic keyboard instrument other than an electronic keyboard, such as an electronic piano.

[0012] The performance device according to the present disclosure is not limited to the electronic musical instrument 1. The performance device may be, for example, an information processing device on which a musical instrument app that reproduces the electronic musical instrument 1 is installed. Illustratively, the performance device may be a smartphone, tablet terminal, laptop PC (Personal Computer), portable game console, or PDA (Personal Digital Assistant) on which such a musical instrument app is installed.

[0013] That is, the performance device according to the present disclosure may be an information processing device 3 (such as a smartphone, tablet terminal, or laptop) on which a musical instrument app is installed. In this case, the user can play a piece of music by, for example, operating the keyboard and pedals displayed on the musical instrument app.

[0014] The electronic musical instrument 1 is an example of a computer. As shown in Fig. 2, the electronic musical instrument 1 includes, as its hardware configuration, a processor 10, a RAM (Random Access Memory) 11, a flash ROM (Read Only Memory) 12, an external connection interface 13, a keyboard 14, a pedal unit 15, pedal terminals 16, a switch panel 17, a key scanner 18, an LCD (Liquid Crystal Display) unit 19, a tone generator LSI (Large Scale Integration) 20, a D / A converter 21, and an amplifier 22. The various components of the electronic musical instrument 1 are connected via a bus 23.

[0015] The processor 10 reads out the programs and data stored in the flash ROM 12. The processor 10 controls the electronic musical instrument 1 in an integrated manner by using the RAM 11 as a work area.

[0016] The processor 10 may be, for example, a single processor or a multi-processor, and includes at least one processor. When multiple processors are included, the processor 10 may be packaged as a single device, or may be configured as multiple devices that are physically separated within the electronic musical instrument 1. The processor 10 may be called, for example, a control unit, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or an MCU (Micro Controller Unit).

[0017] The RAM 11 temporarily stores data and programs, and stores various programs and data read from the flash ROM 12.

[0018] The flash ROM 12 is a non-volatile semiconductor memory such as a flash memory, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically Erasable Programmable ROM). The flash ROM 12 stores a control program 12A. The processor 10 executes the control program 12A to perform various processes of the electronic musical instrument 1 according to an embodiment of the present disclosure.

[0019] The external connection interface 13 is, for example, an interface that, under the control of the processor 10, inputs and outputs MIDI data (MIDI messages) to and from an external MIDI (Musical Instrument Digital Interface) device in a serial format.

[0020] The keyboard 14 has 61 keys that are the first performance operators. Specifically, the keyboard 14 has 36 white keys and 25 black keys. Each key is associated with a different pitch. The electronic musical instrument 1 produces musical tones in response to the depression of a key on the keyboard 14. The number of keys on the keyboard 14 is not limited to 61. The keyboard 14 may have other numbers of keys, such as 88 keys or 76 keys.

[0021] It should be noted that any reference to an element using a designation such as "first," "second," etc., used in this disclosure does not generally limit the quantity or order of those elements. These designations are used for convenience to distinguish between two or more elements. Thus, reference to a first and a second element does not imply, for example, that only two elements are employed, that the first element must precede the second element, etc.

[0022] The pedal unit 15 is connected to the pedal terminal 16. The pedal unit 15 includes three pedals that are second performance controls. Specifically, the pedal unit 15 includes a damper pedal 15A, a sostenuto pedal 15B, and a soft pedal 15C. When the user presses a key on the keyboard 14 while depressing a pedal, the electronic musical instrument 1 performs sound generation processing by adding a sound effect associated with the pedal that is being depressed to the musical tone.

[0023] The switch panel 17 includes various controls for operating the electronic musical instrument 1. The various controls include controls corresponding to various functions such as power, recording, play / stop, tone adjustment, tone selection, and sound effects (vibrato, pitch bend, etc.).

[0024] The key scanner 18 monitors key presses and releases on the keyboard 14. For example, when the key scanner 18 detects a key press by the user, it outputs a key press event to the processor 10. The key press event includes information about the pitch of the key related to the key press (key number). The key number is also called a key number, a MIDI (Musical Instrument Digital Interface) key, or a note number. The pitch is also called a note.

[0025] In this embodiment, a separate means is provided for measuring the key pressing speed (velocity), and the velocity measured by this means is also included in the key pressing event. For example, multiple contact switches are provided for each key. The velocity is measured based on the difference in the time that each contact switch remains conductive when the key is pressed. Velocity can be considered a value that indicates the strength of the key pressing operation, and also a value that indicates the loudness (volume) of the musical sound.

[0026] The LCD unit 19 includes an LCD and a driver. When the driver drives the LCD in accordance with a control signal from the processor 10, a screen corresponding to the control signal is displayed. The LCD may be replaced with another type of display device, such as an organic EL (Electro Luminescence) display.

[0027] The waveform data is stored in flash ROM 12 or another memory (not shown). This waveform data is loaded into RAM 11 during startup of electronic musical instrument 1 so that musical tones are quickly generated in response to key presses. When key scanner 18 detects a key press, processor 10 instructs sound source LSI 20 to read corresponding waveform data from the waveform data loaded into RAM 11. The waveform data to be read is determined, for example, by the tone color selected by the user and the key press event.

[0028] The tone generator LSI 20 generates musical tones based on waveform data read from the RAM 11 under the instruction of the processor 10. The tone generator LSI 20 has, for example, 128 generator sections and can simultaneously generate up to 128 musical tones. In this embodiment, the processor 10 and the tone generator LSI 20 are configured as separate processors, but in another embodiment, the processor 10 and the tone generator LSI 20 may be configured as a single processor.

[0029] The digital musical sound data generated by the sound source LSI 20 is converted into an analog signal by a D / A converter 21, amplified by an amplifier 22, and output from, for example, a line-out terminal. For example, the musical sound is reproduced by a speaker connected to the line-out terminal.

[0030] 1, the information processing device 3 is placed on a music stand 2. The information processing device 3 is an example of a computer. As shown in FIG. 3, the information processing device 3 includes a processor 30, a flash ROM 31, an HMI (Human Machine Interface) 32, and a communication interface 33.

[0031] The processor 30 is an element including a CPU, RAM, ROM, etc. The processor 30 loads various programs, including a control program 31A stored in a flash ROM 31, onto the RAM, which is a work area, and controls the information processing device 3 in accordance with the loaded programs.

[0032] The processor 30 is, for example, a single processor or a multi-processor, and includes at least one processor. When multiple processors are included, the processor 30 may be packaged as a single device, or may be configured as multiple devices that are physically separated within the information processing device 3. The processor 30 may be called a "control unit."

[0033] The flash ROM 31 is a non-volatile semiconductor memory that serves as a secondary storage device or auxiliary storage device. The flash ROM 31 stores programs and data, including a control program 31A, that the processor 30 uses to perform various processes. The processor 30 executes the control program 31A, thereby executing various processes of the information processing device 3 according to an embodiment of the present disclosure.

[0034] The HMI 32 includes, for example, a touch panel, a mechanical key switch, a capacitive non-contact key switch, a membrane key switch, and buttons. When a user operates the HMI 32, a signal indicating the operation is output to the processor 30. The processor 30 controls the information processing device 3 based on the input signal.

[0035] The touch panel includes a screen 32A configured with an LCD (Liquid Crystal Display) panel, an organic EL (Electro Luminescence) panel, an LED (Light Emitting Diode) panel, or the like. The HMI 32 displays, for example, an image acquired by the processor 30 executing the control program 31A on the screen 32A. In other words, the HMI 32 is an example of a display unit that displays an image. The image displayed on the HMI 32 is a still image or a moving image.

[0036] The communication interface 33 includes a wireless and wired unit for communicating with an external device. The information processing device 3 is communicably connected to the external device (e.g., the electronic musical instrument 1) via a cable connected to the external connection interface 13 or wirelessly.

[0037] The communication interface 33 includes an interface conforming to the MIDI standard. The processor 30 accepts input of performance information received from an electronic musical instrument 1, which is an example of an external device, via the communication interface 33. The input performance information is stored in, for example, the RAM of the processor 30.

[0038] When a user operates the pedals on a grand piano with the lid open, they can directly see the movement of the dampers and hammers in response to the pedal operation. By directly seeing the movement of the dampers and hammers, the user can easily understand how the mechanism and the musical tone change (the resonance, softness, etc.) when the pedal is operated.

[0039] In contrast, unlike a grand piano, the electronic musical instrument 1 does not have strings, and hammers do not physically strike the strings when the pedals are operated. Therefore, even if the internal structure is visible, it is difficult for users to understand how the pedals change and how the musical tone changes when the pedals are operated. Furthermore, electronic musical instruments 1 often lack the cover seen on grand pianos, making the internal structure inaccessible to users. Many users continue practicing without understanding the principles behind the pedals. Beginners, in particular, have difficulty understanding the role and usage of each pedal. Intermediate players who are somewhat proficient at keyboard playing may understand the role of each pedal but remain unfamiliar with its principles. As a result, they may not be able to grasp its usage and apply it effectively (and in some cases, their progress may halt). Additionally, beginners and intermediate players often do not understand the specific situations and principles behind each pedal. Unfamiliar with the principles, they often continue practicing without even using the pedals. Some users operate the pedals as if they were switches to sustain a particular note or to soften the sound. If you operate the pedal as if it were a switch, it will be difficult to increase the expressiveness of your playing.

[0040] Therefore, the control program 31A stored in the flash ROM 31 causes the processor 30 to execute the following process: when it acquires first performance information (e.g., a control change message) corresponding to an operation on the pedal unit 15, it causes the HMI 32 (an example of a display unit) to display an image that simulates the movement of the damper and hammer corresponding to the operation on the pedal unit 15.

[0041] By viewing such images, the user can intuitively understand how the mechanism and tone change when the pedal is operated. From another perspective, the user can intuitively connect visual information (such as the movement of the damper) with the change in tone. This makes it easier for the user to grasp the feel of pedal operation, and improves the expressiveness of performance when using the pedal.

[0042] For example, the image 100 shown in Fig. 4 is displayed on the screen 32A of the HMI 32. Hammers H1 to H61 corresponding to the pitches C2 to C7 (in other words, the keys of the keyboard 14) are displayed in a row on the image 100. Above each of the hammers H1 to H61, a string S corresponding to each key of the keyboard 14 is displayed, and furthermore, dampers D1 to D61 corresponding to each of the hammers H1 to H61 are displayed in a row. For convenience, the hammers H1 to H61 are collectively referred to as "hammers H," and the dampers D1 to D61 are collectively referred to as "dampers D."

[0043] In this way, image 100 displays dampers D, hammers H, and strings S corresponding to the number of keys (61 in this case) on keyboard 14. In image 100, dampers D, hammers H, and strings S are shown in schematic figures to clearly convey the movements of dampers D and hammers H to the user.

[0044] For convenience, image 100 shows three strings S for each pitch. Generally, a grand piano has fewer strings in the lower range. For example, only one string is strung for each pitch in the lowest octave. Therefore, image 100 may also show only one or two strings S for pitches in the lower range, in keeping with the grand piano.

[0045] In image 100, letters indicating the pitch are written on each hammer H as auxiliary information. Instead of or in addition to the pitch, a key number may be written on each hammer H. Such auxiliary information may also be written on the damper D. For the sake of simplicity in image 100, such auxiliary information does not have to be displayed.

[0046] 5 show the movements of the damper D and hammer H from when the user presses a key until when the user releases it, with the pedal not being depressed. For example, when the key of pitch C#2 is pressed, the processor 10 of the electronic musical instrument 1 instructs the sound source LSI 20 to generate a musical tone of pitch C#2, and outputs a note-on message corresponding to this key pressing operation to the information processing device 3.

[0047] The processor 30 of the information processing device 3 creates an animation showing the movements of the damper D and the hammer H based on the input note-on message. In the created animation, the hammer H2 corresponding to the pitch C#2 rises. A little while after the start of the hammer H2, the damper D2 starts to rise and separates from the string S (see image 100a in FIG. 5).

[0048] The raised hammer H2 strikes the string S. The moment the hammer H2 strikes the string S, it falls (see image 100b in Figure 5). After falling, the hammer H2 returns to its original position. By watching the hammer H2 strike the string S, which has been released from the pressure applied by the damper D2, the user can intuitively understand how the musical note C#2 is produced.

[0049] To allow the user to more intuitively understand, the string S struck by the hammer H may be vibrated in the video. Furthermore, as shown in image 100b of FIG. 5, an effect EF may be displayed on the vibrating string S. In the example of FIG. 5, an effect EF is displayed by surrounding the vibrating string S with a rectangle. This rectangular shape may be displayed in a display form that indicates vibration (for example, by periodically flashing, periodically changing color, etc.). The effect EF is not limited to this. The effect EF may also be an effect in a different display form, such as periodically flashing the string S or periodically changing the color of the string S.

[0050] When the finger is released from the key of pitch C#2, the processor 10 of the electronic musical instrument 1 instructs the sound source LSI 20 to mute the musical tone of pitch C#2 that is currently being sounded, and outputs a note-off message corresponding to this key-release operation to the information processing device 3.

[0051] The processor 30 of the information processing device 3 creates a video showing the movements of the damper D and hammer H based on the input note-off message. In the created video, the damper D2 descends (see image 100c in FIG. 5) and presses down on the string S from above (see image 100 in FIG. 4). By watching the damper D2 press down on the string S from above, the user can intuitively understand how the musical note of pitch C#2 disappears.

[0052] 6 show the movement of the damper D when the damper pedal 15A is depressed. For example, when the damper pedal 15A is depressed, the processor 10 of the electronic musical instrument 1 transmits a control change message (hold message) corresponding to the operation of the damper pedal 15A to the sound source LSI 20, and also outputs this hold message to the information processing device 3.

[0053] The processor 30 of the information processing device 3 creates a video showing the movement of the dampers D based on the input hold message. In the created video, all of the dampers D rise from the string S according to the value indicating the amount of depression of the damper pedal 15A included in the hold message. The smaller the amount of depression, the smaller the amount of rise of the dampers D (see image 101a in FIG. 6), and the greater the amount of depression, the greater the amount of rise of the dampers D (see image 101b in FIG. 6). When the damper pedal 15A is released, all of the dampers D descend and press down on the string S from above (see image 100 in FIG. 4).

[0054] By observing how damper D releases the pressure on string S in all registers as damper pedal 15A is depressed, the user can intuitively understand that musical tones will resonate and sustain in all registers. By observing how damper D presses string S in all registers as damper pedal 15A is released, the user can intuitively understand that musical tones will resonate and sustain in all registers.

[0055] Images 102a to 102c shown in FIG. 7 show the movements of the damper D and hammer H from when the user presses the key until when the user releases it, with the sostenuto pedal 15B depressed.

[0056] For example, when the sostenuto pedal 15B is depressed while the key of pitch C#2 is pressed, the processor 10 of the electronic musical instrument 1 transmits a control change message (sostenuto message) corresponding to the operation of the sostenuto pedal 15B to the sound source LSI 20, and also outputs this sostenuto message to the information processing device 3. As long as the sostenuto pedal 15B is depressed, the designated musical tone of pitch C#2 resonates and sustains.

[0057] When the sostenuto pedal 15B is depressed (for example, when the value indicating the depression amount of the sostenuto pedal 15B included in the sostenuto message is between 64 and 127), the function of the sostenuto pedal 15B is turned on, and the sound effect of the sostenuto pedal 15B is added to the musical tone. When the sostenuto pedal 15B is released (for example, when the above value is between 0 and 63), the function of the sostenuto pedal 15B is turned off, and the sound effect of the sostenuto pedal 15B is not added to the musical tone.

[0058] The processor 30 of the information processing device 3 creates a moving image showing the movement of the damper D based on the input sostenuto message. In the created moving image, the damper D2 corresponding to the designated note is raised and fixed at a predetermined position (hereinafter referred to as the "predetermined raised position") (see image 102a in FIG. 7). In the moving image, the damper D2 may be raised according to a value indicating the depression amount of the sostenuto pedal 15B, which is included in the sostenuto message, as in the case of the damper pedal 15A.

[0059] For example, when the key of pitch B6 is pressed and released in succession with damper D2 in the raised state, the processor 10 of the electronic musical instrument 1 outputs a note-on message and a note-off message to the information processing device 3 in succession.

[0060] The processor 30 of the information processing device 3 creates a video showing the movements of the damper D and hammer H based on the note-on and note-off messages input in sequence. In the created video, the hammer H60 strikes the string S, which is released from the pressure applied by the damper D60 (see image 102b in FIG. 7). Next, the hammer H60 falls. Furthermore, the damper D60 descends (see image 102c in FIG. 7), pressing down on the string S from above. During this time, the damper D2 corresponding to the note specified by the sostenuto pedal 15B (the musical note with a pitch of C#2) remains fixed at a predetermined raised position.

[0061] By observing how damper D2 is fixed in a predetermined raised position, the user can intuitively understand that the musical tone of pitch C#2 will resonate and sustain while the sostenuto pedal 15B is depressed. By observing how damper D60 descends and presses down on string S from above at the timing of key release, the user can intuitively understand that the musical tone of pitch B6 corresponding to the released key will disappear without sustaining.

[0062] 8 show the movements of the damper D and hammer H from when the user presses the key until when the user releases it with the soft pedal 15C depressed. When the soft pedal 15C is depressed, the processor 10 of the electronic musical instrument 1 transmits a control change message (soft message) corresponding to the operation of the soft pedal 15C to the sound source LSI 20, and also outputs this soft message to the information processing device 3.

[0063] When the soft pedal 15C is depressed (for example, when the value indicating the depression amount of the soft pedal 15C included in the soft message is between 64 and 127), the function of the soft pedal 15C is turned on, and the sound effect of the soft pedal 15C is added to the musical sound. When the soft pedal 15C is released (for example, when the above value is between 0 and 63), the function of the soft pedal 15C is turned off, and the sound effect of the soft pedal 15C is not added to the musical sound.

[0064] Based on the input soft message, the processor 30 of the information processing device 3 creates a video showing the movement of the hammer H. In the created video, the hammer H slides a predetermined amount relative to the string S (see image 103a in FIG. 8).

[0065] For example, when the key of pitch C#2 is pressed and released in sequence while the hammer H is in a sliding state, the processor 10 of the electronic musical instrument 1 outputs a note-on message and a note-off message to the information processing device 3 in sequence.

[0066] The processor 30 of the information processing device 3 creates a video showing the movements of the damper D and hammer H based on the note-on and note-off messages input in sequence. In the created video, the hammer H2 strikes the string S that has been released from the pressure applied by the damper D2 (see image 103b in FIG. 8). Because the hammer H has slid from its initial position, only two of the three strings S are struck by the hammer H2. Next, the hammer H2 falls. Furthermore, the damper D2 descends (see image 103c in FIG. 8) and presses down on the string S from above.

[0067] The user can intuitively understand that the musical sound becomes softer (or, from another perspective, the musical sound becomes weaker) by observing how the number of struck strings S decreases as the hammer H slides. In the example of Fig. 8, the rectangular effect EF surrounds two of the three strings S. Therefore, the user can easily understand that the number of struck strings S has decreased from three to two.

[0068] 9 shows the movement of the hammers H when the user presses a key with both the damper pedal 15A and the soft pedal 15C depressed. In image 104a, as the damper pedal 15A is depressed, the dampers D release the pressure on the strings S in all registers. Furthermore, as the soft pedal 15C is depressed, all of the hammers H slide from their initial positions.

[0069] By observing how, as both the damper pedal 15A and the soft pedal 15C are depressed, the damper D releases the pressure on the strings S and the number of strings S struck by the hammer H2 decreases as the hammer H2 slides, the user can intuitively understand that a softer-than-normal musical tone will resonate and sustain regardless of which key is pressed.

[0070] 10 shows the movement of the hammers H when the user presses a key with both the sostenuto pedal 15B and the soft pedal 15C depressed. In image 105a, as the sostenuto pedal 15B is depressed, the damper D2 corresponding to the designated note releases the pressure on the string S. Furthermore, as the soft pedal 15C is depressed, all of the hammers H slide from their initial positions.

[0071] By observing how, as both the sostenuto pedal 15B and the soft pedal 15C are depressed, the damper D2 corresponding to the designated note releases the pressure on the string S and the number of strings S struck by the hammer H2 decreases as the hammer H2 slides, the user can intuitively understand that a softer-than-normal musical tone resonates and sustains only when the key corresponding to the designated note is pressed.

[0072] Images 106a and 106b shown in FIG. 11 show the movements of the damper D and the hammer H when only the damper pedal 15A, out of the damper pedal 15A and the sostenuto pedal 15B, is released from depression.

[0073] In image 106a, as the damper pedal 15A is depressed, all dampers D are fixed in their predetermined raised positions. In image 106b, as the damper pedal 15A is released, the dampers D descend and return to pressing down on the strings S. However, only the damper D2 corresponding to the key that was pressed when the sostenuto pedal 15B was depressed (i.e., the damper D2 corresponding to the designated note) remains fixed in its predetermined raised position.

[0074] By observing that the damper D2 continues to be fixed at the predetermined raised position, the user can intuitively understand that even after the damper pedal 15A is released, as long as the sostenuto pedal 15B is depressed, only the designated musical tone will resonate and sustain.

[0075] When performance information (e.g., a MIDI message) is input, the processor 30 of the information processing device 3 starts executing the process shown in the flowchart of FIG. 12. Note that the order of the steps in the flowchart shown in this embodiment may be changed as long as it is consistent. For example, although the present disclosure presents the processing of various steps using an exemplary order, it is not limited to this presented order. Also, the steps in the flowchart shown in this embodiment may be executed in parallel or in parallel as long as it is consistent.

[0076] The processor 30 determines whether the input MIDI message is a note-on message (step S101). If the input MIDI message is a note-on message (step S101: YES), the processor 30 creates an animation in which the damper D and hammer H corresponding to the note number included in the note-on message move, and displays the animation on the screen 32A of the HMI 32 (step S102).

[0077] For example, when the key corresponding to the pitch C#2 is pressed without pedal operation, a video including images 100a to 100b in FIG. 5 is displayed on the screen 32A of the HMI 32. Here, the frame rate of the video is, for example, 30 fps (frames per second). Between the images 100a and 100b, frame images showing the intermediate movements of the damper D and hammer H are displayed multiple times. Therefore, the damper D and hammer H are displayed with smooth movement on the screen 32A.

[0078] For example, when the key corresponding to pitch B6 is pressed with the sostenuto pedal 15B depressed, a video including the image 102b in Fig. 7 is displayed on the screen 32A of the HMI 32. That is, a video is displayed on the screen 32A showing the damper D2 corresponding to the designated note and the hammer H2 moving, with the damper D2 corresponding to the pitch B6 fixed at a predetermined raised position.

[0079] If the input MIDI message is a note-off message (step S101: NO, step S103: YES), the processor 30 creates an animation in which the damper D corresponding to the note number included in the note-off message moves, and displays it on the screen 32A of the HMI 32 (step S104).

[0080] For example, when the finger is released from the key corresponding to the pitch C#2 without pedal operation, a video including image 100c in Fig. 5 is displayed on screen 32A of HMI 32. That is, a video is displayed on screen 32A showing damper D2 corresponding to pitch C#2 gradually descending from a predetermined raised position to a position where it presses down on string S from above.

[0081] For example, when the soft pedal 15C is depressed and the finger is released from the key corresponding to the pitch C#2, a video including the image 102c in Fig. 8 is displayed on the screen 32A of the HMI 32. That is, a video is displayed on the screen 32A showing the damper D2 corresponding to the pitch C#2 gradually descending from a predetermined raised position to a position where it presses down on the string S from above, with the hammer H out of alignment.

[0082] In this way, when the processor 30 acquires second performance information (e.g., a note-on message or a note-off message) corresponding to keyboard operation on the electronic musical instrument 1 (an example of a performance device), it causes the HMI 32 (an example of a display unit) to display an image simulating the movement of the damper D and hammer H corresponding to this keyboard operation.

[0083] By watching the video displayed on the screen 32A of the HMI 32, the user can intuitively understand how musical tones are produced and extinguished when the keyboard is operated.

[0084] If the input MIDI message is a control change message (step S101: NO, step S103: NO, step S105: YES), the processor 30 creates a video corresponding to the control change message (in other words, an image corresponding to the pedal operation) and displays it on the screen 32A of the HMI 32 (step S106).

[0085] In the case of a hold message, a video is displayed on the screen 32A of the HMI 32, showing all dampers D rising to a position corresponding to the depression amount of the damper pedal 15A (see, for example, images 101a to 101b in FIG. 6). In the case of a sostenuto message, a video is displayed on the screen 32A, showing the dampers D corresponding to the designated note rising to a predetermined rising position (see, for example, image 102a in FIG. 7 and image 105a in FIG. 10). In the case of a soft message, a video is displayed on the screen 32A, showing all hammers H sliding a predetermined amount (see, for example, image 103a in FIG. 8, image 104a in FIG. 9, and image 105a in FIG. 10).

[0086] In this way, when the processor 30 acquires first performance information (e.g., a hold message) in response to the operation of the damper pedal 15A, it causes the HMI 32 (an example of a display unit) to display an image of all dampers D being raised relative to the strings S. When the processor 30 acquires first performance information (a sostenuto message) in response to the operation of the sostenuto pedal 15B, it causes the HMI 32 to display an image of the dampers D corresponding to the keys pressed by the user on the electronic musical instrument 1 (an example of a performance device) being raised relative to the strings S. When the processor 30 acquires first performance information (a soft message) in response to the operation of the soft pedal 15C, it causes the HMI 32 to display an image of the hammers H being slid a predetermined amount relative to the strings S.

[0087] By watching the video displayed on screen 32A of HMI 32, the user can intuitively understand how the mechanism changes the musical tone (the resonance, softness, etc.) when the pedal is operated. This makes it easier for the user to get a feel for pedal operation (how to use the different pedals, timing, amount of depression, etc.), and improves the expressiveness of performance when using the pedals.

[0088] If the message does not correspond to any of the note-on message, note-off message, and control change message (step S101: NO, step S103: NO, step S105: NO), processor 30 ends the processing shown in FIG. 12 without creating a moving image.

[0089] The above is a description of exemplary embodiments of the present disclosure. The embodiments of the present disclosure are not limited to those described above, and various modifications are possible within the scope of the technical idea of ​​the present disclosure. For example, the embodiments of the present disclosure also include appropriate combinations of embodiments explicitly shown as examples in the specification or obvious embodiments.

[0090] In the above embodiment, a moving image is displayed on the screen 32A of the HMI 32. However, in another embodiment, a still image may be displayed on the screen 32A. As an example, when the damper pedal 15A is depressed, the image 101b in FIG. 6 may be displayed as a still image on the screen 32A. In this case, the image processing load on the processor 30 is reduced compared to when a moving image is displayed. By viewing such a still image, the user can intuitively understand that the musical tone resonates and sustains across all registers.

[0091] In the above embodiment, the parts (dampers D, hammers H, strings S) are displayed aligned in a row in the horizontal direction, but the display format of the parts is not limited to this. For example, the parts corresponding to the white keys and the parts corresponding to the black keys may be displayed offset in the vertical direction. In this case, the user can more easily grasp the correspondence between the operated keys and the operating parts.

[0092] In the above embodiment, the components corresponding to the white keys and the components corresponding to the black keys are displayed without distinction, but the display form of the components is not limited to this. For example, the components corresponding to the white keys and the components corresponding to the black keys may be displayed in different forms (colors, shapes, etc.). In this case, too, the user can more easily grasp the correspondence between the operated keys and the operating components.

[0093] A visual effect may be added to the image. For example, a component corresponding to a pressed or released key may be displayed in a form (color, shape, etc.) different from that of other components. In this case, too, the user can more easily grasp the correspondence between the operated key and the operating component.

[0094] Images showing the movement of the parts may be displayed on the screen of the LCD unit 19. That is, the processor 10 of the electronic musical instrument 1 may perform the same processing as the processor 30 of the information processing device 3. In this case, the electronic musical instrument 1 operates as both a performance device according to the present disclosure and an information processing device according to the present disclosure. Therefore, it is not necessary to connect the information processing device 3 to the electronic musical instrument 1.

[0095] An example of an image displayed on the screen 32A of the HMI 32 in the first modification of the present disclosure will be described with reference to Fig. 13. In the image 107a shown in Fig. 13, all of the hammers H slide from their initial positions as the soft pedal 15C is depressed. In the image 107a, the string S1 that is not struck even when the hammer H rises and the string S2 that is struck when the hammer H rises are displayed in different colors. In yet another modification, the strings S1 and S2 may be displayed in different shapes.

[0096] For example, due to screen size limitations, the amount of lateral displacement of the hammer H caused by the soft pedal 15C may appear negligible on the display. Therefore, in Modification 1, the processor 30 displays, in an image in which the hammer H is slid relative to the strings S, the strings S2 that are struck by the hammer H in response to key depressions on the electronic musical instrument 1 (an example of a performance device), and the strings S1 that are not struck by the hammer H, in different forms. By visually recognizing the strings S1 and S2 displayed in different forms, the user can instantly visually grasp that the number of struck strings S2 decreases as the soft pedal 15C is depressed, and can more intuitively understand that the musical tone becomes softer.

[0097] An example of an image displayed on the screen 32A of the HMI 32 in Modification 2 of the present disclosure will be described with reference to Fig. 14. Modification 2 describes a case where the electronic musical instrument 1 plays back music from the built-in library stored in the flash ROM 12. For convenience, the lowest and highest pitches among the pitches constituting the music from the built-in library to be played back in Modification 2 will be referred to as the "lowest pitch" and the "highest pitch," respectively.

[0098] 14 includes not only diagrams showing the parts (dampers D, hammers H, and strings S) but also an overall view F1. The overall view F1 is a diagram that schematically shows the entire keyboard 14. In the overall view F1, individual keys are omitted from the illustration in order to clearly show the range of the entire keyboard 14 from the lowest pitch key to the highest pitch key in the piece of music.

[0099] For example, processor 30 searches the music data to be played in advance to detect the lowest pitch (e.g., pitch C3) and the highest pitch (e.g., pitch C5). Processor 30 then displays an overall view F1 on screen 32A of HMI 32. Specifically, processor 30 displays the range of keys from the lowest pitch to the highest pitch (pitches C4 to C5) of the entire keyboard 14 in a different color from the remaining range (pitches C2 to B3 and pitches C#5 to C7). Processor 30 further displays not all key parts, but only parts within the range from the lowest pitch to the highest pitch (dampers D25 to D37, hammers H25 to H37, and strings S within that range) on screen 32A.

[0100] In this way, in Modification 2, the number of parts displayed on screen 32A can be reduced. Therefore, for example, each part can be displayed in a larger size. Because visibility is improved, the user can more intuitively understand, for example, how the mechanism and how the musical tone changes when the pedal is operated. Furthermore, the user can grasp, through the overall view F1, where the part is located within the whole and how it moves. [Explanation of symbols]

[0101] 1: Electronic musical instrument, 3: Information processing device, 15: Pedal unit, 15A: Damper pedal, 15B: Sostenuto pedal, 15C: Soft pedal, 30: Processor, HMI: 32, D: Damper, H: Hammer, S: String

Claims

1. A display unit; at least one processor; when the at least one processor acquires first performance information corresponding to an operation on a pedal of the performance device, the at least one processor causes the display unit to display an image simulating movements of a damper and a hammer corresponding to the operation on the pedal; Information processing device.

2. The pedal includes a damper pedal, when the at least one processor acquires the first performance information corresponding to an operation of the damper pedal, the at least one processor causes the display unit to display an image of the dampers corresponding to all keys being raised relative to the strings. The information processing device according to claim 1 .

3. the pedals include a sostenuto pedal; and when the at least one processor acquires the first performance information corresponding to an operation of the sostenuto pedal, the at least one processor displays on the display unit an image of the damper corresponding to the key pressed by the user on the performance device being raised relative to the string. The information processing device according to claim 1 .

4. the pedals include soft pedals; when the at least one processor acquires the first performance information corresponding to the operation of the soft pedal, it causes the display unit to display an image of the hammer sliding a predetermined amount relative to the string. The information processing device according to claim 1 .

5. the at least one processor displays, in an image in which the hammer is slid relative to the strings, strings that are struck by the hammer in response to a key depression operation on the performance device and strings that are not struck by the hammer in different forms. The information processing device according to claim 4 .

6. Among the pitches that make up a piece of music, the lowest and highest pitches are called the lowest pitch and highest pitch, respectively. The at least one processor Detecting the lowest pitch and the highest pitch of the music piece; displaying on the display unit the dampers and the hammers corresponding to all the keys included in the keyboard of the performance device, from the detected lowest pitch key to the detected highest pitch key; The information processing device according to claim 1 .

7. and when the at least one processor acquires second performance information corresponding to a keyboard operation on the performance device, the processor causes the display unit to display an image simulating movements of the damper and the hammer corresponding to the keyboard operation. The information processing device according to claim 1 .

8. An information processing device according to any one of claims 1 to 7; The keyboard and The pedal. performance equipment.

9. a computer that, upon receiving first performance information corresponding to an operation on a pedal of the performance device, displays an image on a display unit that simulates the movement of a damper and a hammer corresponding to the operation on the pedal; method.

10. a computer that, upon receiving first performance information corresponding to an operation on a pedal of the performance device, displays an image on a display unit that simulates the movement of a damper and a hammer corresponding to the operation on the pedal; program.

Images