Program, method, information processing apparatus, and image display system

Abstract

To precisely determine the end of performance, for example.SOLUTION: A program allows an information processing device to perform at least processing for determining output timing of data outputted after the end of performance on the basis of an interval between first performance operation and second performance operation.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a program, a method, an information processing apparatus, and an image display system.

Background Art

[0002] Electronic musical instruments such as digital keyboards are equipped with a processor and a memory, and can be said to be a kind of embedded computer with a keyboard. There are also known models that can be connected to an information processing device such as a tablet via an interface such as USB (Universal Serial Bus) and utilize various expansion functions. For example, a technology has been developed to analyze MIDI (Musical Instrument Digital Interface) data generated by playing an electronic musical instrument and create and display a moving image that changes with the performance and a still image (picture) that reflects the content of the performance (see, for example, Patent Document 1).

[0003] Practicing musical instruments is difficult, and many people get bored halfway and give up. In order to increase the motivation to practice not only for advanced players but also for those who are at the beginning of musical instrument performance, attention has been focused on technologies that visualize music performance and create visual effects. As can be seen by accessing Non-Patent Document 1, according to this type of technology, a moving image is dynamically generated / displayed along with the performance, so that music can be enjoyed from a new perspective.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] For example, in a technique for visualizing a performance, a moving image (first image) that is displayed in real time during the performance and a summary image (second image: final picture) that is displayed after the performance ends are generated by a computer. Among these, it is difficult to determine how to decide the timing for displaying the final picture. Currently, when a certain amount of time (judgment time) has elapsed since the last note-off, it is determined that the performance has ended and the final picture is displayed. However, for a child with a poor performance, it takes all their energy to find the next key, resulting in a long key press interval, and the final picture may appear even though the performance is still continuing.

[0007] If the performance ends halfway through while the user is playing with all their might, they will be disappointed. If the judgment time is lengthened to avoid this, the user will be stressed because they have to wait until the final picture appears after the performance ends. For example, when an unspecified number of users play like street piano, it is desirable to be able to appropriately set the judgment time according to the performance situation.

[0008] Therefore, an object of the present invention is to provide a program, method, information processing apparatus, and image display system that can accurately determine the end of a performance, thereby making it more enjoyable to play.

Means for Solving the Problems

[0009] To achieve the above object, in one embodiment of the present invention, a program causes an information processing apparatus to determine that a performance made by a plurality of performance operations is not musically appropriate update so that the determination period related to the determination of the end of the performance becomes longer, and determine that the performance has ended based on a comparison between the elapsed time from the last note-off and the updated determination periodDetermine the output timing of the image data after the end of the performance.

Advantages of the Invention

[0010] According to the present invention, for example, it becomes possible to accurately determine the end of a performance, and thereby, it becomes possible to enjoy performing even more.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Modes for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. <Configuration> FIG. 1 is a schematic diagram showing an example of an image display system according to an embodiment. The image display system shown in FIG. 1 draws an image (picture) in real time according to the performance of a user (performer). This type of image display system analyzes performance data acquired from an electronic musical instrument or the like that can output the performance of the user as performance data (for example, MIDI data), and generates an image based on the result.

[0013] In FIG. 1, the image display system includes an electronic musical instrument, an information processing device, and a display device. The electronic musical instrument generates performance data (for example, MIDI data) from the performance of the user, and outputs the performance data to the information processing device. The information processing device analyzes the received performance data and generates image data. The information processing device is, for example, a tablet or a personal computer (PC). The display device displays the image generated by the information processing device.

[0014] FIG. 2 is a diagram showing an example of an image display system in which a tablet is combined with a keyboard instrument. This system includes a digital keyboard 1 and a tablet 3 that can be connected to the digital keyboard 1. The digital keyboard 1 is an electronic keyboard instrument such as an electronic piano, a synthesizer, or an electronic organ, for example.

[0015] The digital keyboard 1 includes a plurality of keys 10 arranged on the keyboard, a display unit 14, an operation unit 18, and a music stand MS. As shown in FIG. 2, the tablet 3 connected to the digital keyboard 1 can be placed on the music stand MS to display a musical score or used as a user interface. The key 10 is an operator for the performer to specify the pitch. When the performer presses or releases the key 10, the digital keyboard 1 produces and silences the sound corresponding to the specified pitch. Pressing and releasing the key are examples of performance operations. Each can be regarded as an individual performance operation, or a set of pressing and releasing the key can be regarded as one performance operation. Alternatively, only the key press can be regarded as an individual performance operation and counted, or only the key release can be regarded as a performance operation. For example, an event that triggers the generation of performance data can be regarded as a performance operation. All actions that generate performance data can be regarded as performance operations, or only actions that generate a specific type of performance data (such as note-on, note-off, etc.) can be regarded as performance operations.

[0016] The display unit 14 includes, for example, a liquid crystal display (LCD) with a touch panel, and displays messages and the like associated with the operations of the operation unit 18 of the performer. When the display unit 14 has a touch panel function, the display unit 14 can undertake part of the functions of the operation unit 18. The operation unit 18 has operation buttons, dials, etc. for the performer to perform various settings and the like. The user can operate the operation buttons, dials, etc. to perform various setting operations such as volume adjustment.

[0017] FIG. 3 is a block diagram showing an example of the digital keyboard 1 according to the embodiment. The digital keyboard 1 includes a USB interface (I / F) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a display unit 14, a display controller 15, an LED (Light Emitthing Diode) controller 16, a keyboard 17, an operation unit 18, a key scanner 19, a MIDI interface (I / F) 20, a system bus 21, a CPU (Central Processing Unit) 22, a timer 23, a sound source 24, a digital / analog (D / A) converter 25, a mixer 26, a D / A converter 27, a voice synthesis LSI 28, and an amplifier 29. Here, the sound source 24 and the voice synthesis LSI 28 are realized as, for example, a DSP (Digital Signal Processor).

[0018] The CPU 22, the sound source 24, the voice synthesis LSI 28, the USB interface 11, the RAM 12, the ROM 13, the display controller 15, the LED controller 16, the key scanner 19, and the MIDI interface 20 are connected to the system bus 21.

[0019] The CPU 22 is a processor that controls the digital keyboard 1. That is, the CPU 22 reads the program stored in the ROM 13 into the RAM 12 as a working memory and executes it to realize various functions of the digital keyboard 1. The CPU 22 operates according to the clock supplied from the timer 23. The clock is used, for example, to control the sequence of automatic performance and automatic accompaniment.

[0020] The ROM 13 stores programs, various setting data, automatic accompaniment data, etc. The automatic accompaniment data may include preset rhythm patterns, chord progressions, bass patterns, or melody data such as obligato. The melody data may include pitch information of each sound, pronunciation timing information of each sound, etc.

[0021] The pronunciation timing of each note may be the interval time between each pronunciation or the elapsed time since the start of the automatic performance piece. The tick is often used as the unit of time. A tick is a unit based on the tempo of a piece used in a general sequencer. For example, if the resolution of the sequencer is 480, 1 / 480 of the time of a quarter note is 1 tick.

[0022] The automatic accompaniment data is not limited to ROM13 and may be stored in an information storage device or an information storage medium (not shown). The format of the automatic accompaniment data may conform to the file format for MIDI.

[0023] The display controller 15 is an IC (Integrated Circuit) that controls the display state of the display unit 14. The LED controller 16 is, for example, an IC. The LED controller 16 lights the keys of the keyboard 17 according to an instruction from the CPU 22 to navigate the performance of the performer.

[0024] The key scanner 19 constantly monitors the key press / release state of the keyboard 17 and the switch operation state of the operation unit 18. Then, the key scanner 19 transmits the states of the keyboard 17 and the operation unit 18 to the CPU 22.

[0025] The MIDI interface 20 inputs MIDI data (performance data, etc.) from an external device such as the MIDI device 4 or outputs MIDI data to an external device. The digital keyboard 1 can exchange MIDI data and music files with an external device using an interface such as USB (Universal Serial Bus). The received MIDI data is passed to the sound source 24 via the CPU 22. The sound source 24 produces sound according to the timbre, volume (velocity), timing, etc. specified by the MIDI data.

[0026] Note that MIDI data (MIDI messages) can represent all kinds of information related to the performance of a piece of music, such as the pitch number corresponding to key 10, timbre number, information representing timings such as note-on and note-off, intensity information called velocity, or various control information.

[0027] The sound source 24 is, for example, a so-called GM sound source compliant with the GM (General MIDI) standard. This type of sound source can change the timbre by giving a program change as a MIDI message included in the MIDI data. Also, if a control change is given, the default effects can be controlled.

[0028] The sound source 24 has, for example, the ability to simultaneously produce a maximum of 256 voices. The sound source 24 reads out musical tone waveform data from, for example, a waveform ROM (not shown) and outputs it as digital musical tone waveform data to the D / A converter 211. The D / A converter 211 converts the digital musical tone waveform data into an analog musical tone waveform signal.

[0029] When the voice synthesis LSI 28 is given text data of lyrics and information regarding pitch as voice data from the CPU 22, it synthesizes voice data corresponding to the same and outputs it to the D / A converter 25. The D / A converter 25 converts the voice data into an analog voice waveform signal.

[0030] The mixer 26 mixes the analog musical tone waveform signal and the analog voice waveform signal to generate an output signal. This output signal is amplified by the amplifier 29 and output from an output terminal such as a speaker or a headphone out.

[0031] The tablet 3 is connected to the system bus 21 via the USB interface 11. The tablet 3 can acquire MIDI data (performance data) generated by playing the digital keyboard 1 via the USB interface 11.

[0032] Furthermore, a memory medium (not shown) or the like may also be connected to the system bus 21 via the USB interface 11. Examples of the memory medium include a USB memory, a flexible disk drive (FDD), a hard disk drive (HDD), a CD-ROM drive, and a magneto-optical disk (MO) drive. When a program is not stored in the ROM 106, a program can be stored in the memory medium and read into the RAM 105, so that the CPU 111 can execute the same operations as when the program is stored in the ROM 106.

[0033] FIG. 4 is a functional block diagram showing an example of the tablet 3. The tablet 3 is a portable information processing device, and an application for generating and outputting an image reflecting performance using the digital keyboard 1 is installed. The tablet 3 may also include a sequencer or the like that receives MIDI data from the digital keyboard 1 and plays back music data.

[0034] The tablet 3 mainly includes an operation unit 31, a display unit 32, a communication unit 33, a sound output unit 34, a memory 35, and a control unit 36 (CPU). Each unit (operation unit 31, display unit 32, communication unit 33, sound output unit 34, memory 35, and control unit 36) is communicably connected via a bus 37, and necessary data can be exchanged between the units.

[0035] The operation unit 31 includes, for example, switches such as a power switch for turning the power on and off. The display unit 32 includes a liquid crystal monitor with a touch panel and displays an image. Since the display unit 32 also has a touch panel function, it can serve as one end of the operation unit 31.

[0036] The communication unit 33 includes a wireless unit or a wired unit for communicating with other devices. In the embodiment, for example, it is wired to the digital keyboard 1 via a USB cable or the like, so that the tablet 3 can exchange various digital data with the digital keyboard 1. The sound output unit 34 includes a speaker, an earphone jack, etc., and reproduces and outputs analog voices and musical sounds, or outputs an audio signal.

[0037] The control unit 36 includes a processor such as a CPU and controls the tablet 3. The CPU of the control unit 36 executes various processes and the like according to the control program stored in the memory 35 and the installed applications.

[0038] The memory 35 includes a ROM 40 and a RAM 50. The ROM 40 stores, for example, a program 41 executed by the control unit 36 and various data tables. In particular, in the embodiment, the determination period T related to the determination of the end of performance is stored in the storage area 42 of the ROM 40.

[0039] The RAM 50 stores the data necessary for operating the program 41. The RAM 50 also functions as a data created by the control unit 36, MIDI data sent from the digital keyboard 1, and a temporary storage area for deploying an application. In the embodiment, the RAM 50 stores, in addition to the performance data 50a including MIDI data, character data 50b, first image data 50c, and second image data 50d.

[0040] Incidentally, in the embodiment, the program 41 includes a music analysis routine 41a, a first image creation routine 41b, a second image creation routine 41c, and an output control routine 41d.

[0041] The music analysis routine 41a acquires each performance data successively generated in response to the performance of the digital keyboard 1 and stores it in the RAM 50 as performance data 50a. Further, the music analysis routine 41a performs music analysis based mainly on the pitch data included in the performance data 50a, and performs tonality determination, chord type determination, note name determination, etc. of the piece.

[0042] Note that the method of music analysis or the method for determining tonality, chord type, etc. is not particularly limited. For example, the method disclosed in the specification of Patent No. 3211839 can be used.

[0043] Based on the results of music analysis, the first image creation routine 41b creates moving image data that is displayed in real time during performance. The created moving image data is temporarily stored in the RAM 50 as the first image data 50c, and then immediately read out and displayed on the display unit 32.

[0044] Based on the results of music analysis, the second image creation routine 41c creates a still image that is displayed as a summary after the performance ends. The moving image data of the created still image is temporarily stored in the RAM 50 as the second image data 50d, and then output (read out) at an appropriate timing and displayed on the display unit 32.

[0045] The output control routine 41d determines the timing for outputting the second image data based on the timing when each performance data is generated from the digital keyboard 1 or the interval between each timing when the performance data is acquired.

[0046] <Function> Next, the function of the above configuration will be described. Hereinafter, it will be described assuming that the tablet 3 is communicably connected to the digital keyboard 1. Also, it is assumed that an application for displaying an image on the display unit 32 (FIG. 4) of the tablet 3 is activated on the tablet 3.

[0047] FIG. 5 is a flowchart showing an example of the processing procedure of the tablet 3. In FIG. 5, the control unit 36 (CPU) of the tablet 3 waits for the input of performance data from the digital keyboard 1 (step S1). If there is an input of performance data in step S1 (YES), the control unit 36 executes a performance determination process (step S2). In step S2, the control unit 36 determines, based on the acquired performance data, for example, the key of the piece being performed (e.g., 24 keys from C major to B minor), the chord type (e.g., Major, minor, sus4, aug, dim, 7th, etc.), the tempo, etc. The determination results obtained here are reflected in the first image.

[0048] FIG. 6 is a diagram showing a musical score example. When a performance such as that shown in FIG. 6 is performed, as shown in FIG. 7, characters of flower (1), leaf (2), ladybug (3), butterfly (4), etc. are successively arranged in the order of do, re, mi, fa..., resulting in the first image. When the end of the performance is determined, as shown in FIG. 8, each character is arranged, for example, on a spiral orbit, resulting in the second image. Returning to FIG. 5 to continue the explanation. The control unit 36 generates the first image based on the result of the performance determination process and outputs it to the display unit 32 (step S3).

[0049] Next, the control unit 36 performs a process of updating the determination period T based on the result of the performance determination process (step S4). FIG. 9 is a flowchart showing an example of the processing procedure in step S4. When the determination period update process in step S4 is called, a software interrupt occurs. Then the control unit 36 first sets the initial value T0 to the determination period T (step S41). As the initial value T0, for example, 5 seconds is set. Next, the control unit 36 calculates the maximum value Tmax of the intervals between the most recent notes (step S42). That is, in this step, the control unit 36 acquires the note-on times of X (e.g., 4) sounds going back from the most recent note-on time, calculates the time intervals for each, and obtains the maximum value Tmax.

[0050] Next, the control unit 36 compares the determination period T with Tmax (step S43). If (T < Tmax) is FALSE (NO), that is, if there is no note interval that exceeds the determination period T in the most recent note intervals, then T remains T0 (step S44), and the processing procedure returns (returns) to the calling source.

[0051] On the other hand, if (T < Tmax) is TRUE (YES) in step S43, that is, if there is a note interval that exceeds the determination period T in the most recent note intervals, then the product of Tmax and the determination period update coefficient α is substituted into T (step S45), and the processing procedure returns (returns) to the calling source. Here, the value of the coefficient α can be 1.1, which is equivalent to making the determination period T longer than the default. Also, the value of the coefficient α is updated to a different value according to the performance situation.

[0052] Returning to FIG. 5 to continue the explanation. If there is no input of performance data in step S1 (NO), or when step S4 ends, the control unit 36 performs an end determination (step S5). In the embodiment, the end determination is performed by comparing the determination period T as a reference value with the elapsed time from the last note-off. That is, when the elapsed time from the last note-off becomes longer than the determination period T, it is determined that the performance has ended (YES). If it is NO in step S5, the processing procedure returns to step S1 again until a YES determination is made.

[0053] During the performance, the processing of steps S1 to S5 is repeated, and when the performance ends, it becomes YES in step S5. Then, the control unit 36 creates a second image reflecting the analysis result of the accumulated performance data 50a and displays and outputs it to the display unit 32 (step S6).

[0054] FIG. 10 is a diagram for explaining the calculation of the note interval in step S42 of FIG. 9. In the embodiment, the "note interval" means the period from the previous note-on to the next note-on. At this time, sounds (notes) that temporally overlap are treated as one lump of sound (note).

[0055] For example, when pressing multiple keys together like chord strumming, strictly speaking, the note-on times of each note often deviate slightly. As shown in Fig. 10, even if the C, E, and G notes of the C chord with C, E, and G as the constituent notes deviate slightly, if the deviation amount is within the preset value, these are grouped together, and the occurrence of note-on and note-off is counted as once each. For example, the key-pressing time of the first note in a group is set as note-on, and the key-releasing time of the last note is set as note-off. Then, the period until the note-on of the next note is set as the note interval. The note-on of a single note can be counted as the actual key-pressing time of that note.

[0056] Fig. 11 is a flowchart showing an example of a processing procedure related to the update of the determination period update coefficient α as a reference value for determining the output timing of the second image data. In Fig. 11, the control unit 36 counts the number of notes N in the past few seconds (e.g., 8 seconds) from the current time (step S7), and compares it with a preset threshold value N1 (e.g., 5 notes) (step S8). If N is greater than the threshold value N1 (YES), 1.1 is substituted into α (step S10). On the other hand, if N is less than or equal to the threshold value N1 (NO), a value larger than 1.1, for example, 1.5, is substituted into α (step S9).

[0057] In step S8, if it is determined that the number of notes generated in the performance in the past few seconds is small, this means that the performance may have been unstable or is likely to be performed slowly. Therefore, in such a case, α is set to a larger value so that the determination period T becomes longer. Conversely, if the number of notes in the past few seconds is large, α is set to a smaller value.

[0058] That is, when the number of performance data generated or acquired within the set period does not reach the threshold value, the control unit 36 delays the output timing of the second image data more than when the number of performance data reaches the threshold value. Note that the rank of the threshold value may be set not only to N1 but also to a plurality of values such as N1, N2, N3... and α may be gradually changed in several steps.

[0059] <Effect> As described above, in the embodiment, based on the result of music analysis of performance data, the timing of outputting the second image is controlled for each performance. For example, when playing a piece with a fast tempo, the second image is output at a timing earlier than when playing a piece with a slow tempo, starting from the stop of the performance. Conversely, when a beginner is playing slowly at a slow tempo, the time from the stop of the performance until the second image is displayed becomes longer.

[0060] By doing so, it becomes possible to output the second image not at a fixed timing but at a good timing after the end of the performance. That is, it is possible to prevent situations such as the final picture appearing even though the performance has not ended, or conversely, the final picture not appearing for a long time even though the performance has ended, which would be disappointing.

[0061] That is, according to the embodiment, it becomes possible to accurately determine the end of the performance. Therefore, it is possible to provide a program, an electronic device, a method, and an image display system that improve the experiential value of the technology for visualizing music performance, and make it more enjoyable to play or practice musical instruments without diminishing the user's motivation to practice.

[0062] Note that the present invention is not limited to the above embodiment.

[0063] <Modification Example 1> When a certain degree of stability of the performance can be expected, in step S42 of FIG. 9, instead of the maximum value of the most recent note intervals, the average value may be used.

[0064] <Modification Example 2> As a condition for updating the determination for a predetermined time in step S42, instead of the maximum value (or average value) of the most recent note intervals, an index of performance instability may be used. As the index, the result of non-musical determination, or the instability of the tempo, etc. may be used.

[0065] For example, when the chord cannot be determined by music analysis (when the determination fails), when the number of times the chord cannot be determined exceeds the specified number, or when detecting the simultaneous pressing of five or more adjacent white keys, etc., it is possible to determine that the performance operation is non-musical based on combinations of pitch data detected at almost the same timing, such as combinations of pitch data detected at a certain time length, etc.

[0066] For example, if non-musical determination is used, regarding the update of the determination period T, a condition (Condition A) such as "when non-musical determination occurs a specified number of times (for example, 3 times) or more within the most recent several beats (for example, 8 beats)" can be considered. Take the logical product (AND) of this Condition A and the "determination based on the calculated value of the most recent note interval (Condition B)" in Step S42, and update the determination period T when this is true.

[0067] <Modification Example 3> More directly, if the tempo of the piece is used as a condition for determining the end of the performance, the following can be considered. That is, the control unit 36 determines the tempo of the performance based on the performance data acquired from the digital keyboard 1. Then, when the determined tempo is the second tempo slower than the first tempo, the output timing of the second image data is determined so that the output timing determined for the second tempo is later than the output timing determined for the first tempo.

[0068] <Modification Example 4> Even more directly, as a form of determining the end of the performance, there is also a method of checking the actions of the performer. That is, a camera or the like capable of detecting the actions of the performer is installed, and for example, by determining that the performer has stood up from the chair, the final picture (the second image) may be displayed regardless of the time elapsed since the last note-off.

[0069] That is, in this embodiment, the processor 36 of the information processing apparatus (display apparatus) 3 determines the output timing of data to be output after the end of performance based on the interval between the first performance operation and the second performance operation on the keyboard 17 of the electronic musical instrument 1 by the user. As one embodiment, this interval is calculated based on the acquisition timing of note data acquired on the information processing apparatus 3 side in response to the first performance operation on the electronic musical instrument 1 side and the acquisition timing of note data acquired on the information processing apparatus 3 side in response to the second performance operation after the first performance operation on the electronic musical instrument 1 side. That is, the interval between the first performance operation and the second performance operation may be calculated by any method.

[0070] The invention described in the claims first attached to the application form of this application is appended below. The item numbers of the claims described in the appendix are as in the claims first attached to the application form of this application. [Appendix] [Claim 1] An information processing apparatus, determines the output timing of data to be output after the end of performance based on the interval between the first performance operation and the second performance operation, A program for executing the process. [Claim 2] The data to be output after the end of the performance includes image data, The program according to claim 1. [Claim 3] has a reference value for determining the output timing, changes the reference value based on the interval, The program according to any one of claims 1 or 2. [Claim 4] determines the tempo based on at least the first performance data generated based on the first performance operation and the second performance data generated based on the second performance operation, when the determined tempo is a second tempo slower than the first tempo, determines the output timing so that the output timing determined in the case of the second tempo is later than the output timing determined in the case of the first tempo, The program according to any one of claims 1 to 3. <Claim 5> Determine whether the performance is musically appropriate, Based on the determination result of whether it is musically appropriate, determine the output timing, the program according to any one of claims 1 to 4. <Claim 6> When the number of performance data generated or acquired within the set period does not reach the threshold, delay the output timing more than when it reaches the threshold, The program according to any one of claims 1 to 5. <Claim 7> An information processing apparatus, Based on the interval between the first performance operation and the second performance operation, determine the output timing of the data to be output after the performance ends, A method for executing the process. <Claim 8> Based on the interval between the first performance operation and the second performance operation, determine the output timing of the data to be output after the performance ends, An information processing apparatus that executes the process. <Claim 9> Comprising an electronic musical instrument and a display device, The electronic musical instrument, Successively generate performance data in response to the performance, Transmit the generated performance data to the display device, The display device, Acquire each of the performance data, Based on the acquired performance data, determine the output timing of the image data to be output after the performance ends, Based on the determined output timing, display the image data, An image display system that executes the process.

Description of the reference numerals

[0071] 1…Digital keyboard, 3…Tablet, 4…MIDI device, 10…Keys, 11…USB interface, 12…RAM, 13…ROM, 14…Display unit, 15…Display controller, 16…LED controller, 17…Keyboard, 18…Operation unit, 19…Key scanner, 20…MIDI interface, 21…System bus, 22…CPU, 23…Timer, 24…Sound source, 25, 27…D / A converter, 26…Mixer, 29…Amplifier, 31…Operation unit, 32…Display unit, 33…Communication unit, 34…Sound output unit, 35…Memory, 36…Control unit, 37…Bus, 40…ROM, 41…Program, 41a…Music analysis routine, 41b…First image creation routine, 41c…Second image creation routine, 41d…Output control routine, 42…Storage area, 50…RAM, 50a…Performance data, 50b…Character data, 50c…First image data, 50d…Second image data, 105…RAM, 106…ROM, 111…CPU, 211…D / A converter.

Claims

1. When an information processing apparatus determines that a performance made by a plurality of performance operations is not musically appropriate, it updates the determination period related to the determination of the end of the performance so as to be longer, and determines the output timing of image data after the end of the performance by determining that the performance has ended based on a comparison between the elapsed time from the last note-off and the updated determination period. A program for executing the process.

2. When it is impossible to determine the chord by music analysis from the performance operation, when the number of times the chord cannot be determined exceeds a specified number, or when simultaneous pressing of five or more adjacent white keys is detected, it is determined that the performance is not musically appropriate. The program according to claim 1.

3. When it is determined that the performance is not musically appropriate and either the maximum value of the intervals of the most recent notes or the average value of the intervals of the most recent notes among the intervals of the plurality of performance operations is longer than the determination period as a reference value, the determination period is updated to be longer, and the end of the performance is determined based on a comparison between the elapsed time from the last note-off and the updated determination period. The program according to claim 1 or 2.

4. When an information processing apparatus determines that a performance made by a plurality of performance operations is not musically appropriate, it updates the determination period related to the determination of the end of the performance so as to be longer, and determines the output timing of image data after the end of the performance by determining that the performance has ended based on a comparison between the elapsed time from the last note-off and the updated determination period. A method for executing the process.

5. When it is determined that a performance made by a plurality of performance operations is not musically appropriate, the determination period related to the determination of the end of the performance is updated to be longer, and the output timing of image data after the end of the performance is determined by determining that the performance has ended based on a comparison between the elapsed time from the last note-off and the updated determination period. An information processing apparatus for executing the process.

6. Comprising an electronic musical instrument and a display device, wherein the electronic musical instrument transmits performance data generated in response to a plurality of performance operations to the display device, and the display device updates the determination period related to the determination of the end of the performance so as to be longer when it is determined that a performance made by a plurality of performance operations is not musically appropriate. ​ ​ Determine the output timing of the image data after the end of the performance by determining that the performance has ended based on a comparison between the elapsed time since the last note-off and the updated determination period, and display the image data based on the determined output timing. An image display system that executes the processing.

Images