Method for an information processing apparatus, information processing apparatus, and image display system

Abstract

Provided are a method for an information processing apparatus, an information processing apparatus, and an image display system. An embodiment of the present invention causes an information processing apparatus to perform at least the following processing: determining an output timing of data output after the end of a performance, based on an interval between a first performance operation and a second performance operation.

Description

Technical Field

[0001] The present invention relates to a method for an information processing apparatus, an information processing apparatus, and an image display system. Background Technology

[0002] The following technologies are under development: technology for analyzing MIDI (Musical Instrument Digital Interface) data generated by playing electronic musical instruments, and for creating and displaying motion graphics that change according to the performance, as well as still images (pictures) that reflect the content of the performance.

[0003] For example, see Japanese Patent Application Publication No. 2019-101168.

[0004] Practicing musical instruments is very arduous, and many people get bored and give up halfway through. In order to increase the desire to practice not only for experts but also for beginners, making musical performances visual and creating visual effects is an effective way to improve performance.

[0005] When moving images are dynamically generated / displayed along with the performance, music can be appreciated from a new perspective. Summary of the Invention

[0006] In one example of a technology that visualizes performance, a computer generates a motion image displayed in real time during the performance (first image) and a summarized image displayed after the performance (second image: final picture). Determining the timing for displaying the final picture is challenging. Currently, if a certain amount of time elapses from the last note's end (determination time), the performance is considered over, and the final picture is displayed. However, for children or others who are not good at playing, the keystroke intervals may be longer, and the final picture may appear even though the performance is still ongoing.

[0007] If the final image appears midway through a frantic performance, it will be very disappointing. Extending the judgment time to avoid this would increase the time required from the end of the performance to the final image appearing, thus adding pressure to the user.

[0008] In order to provide a platform for a large number of unspecified users to play, similar to a street piano, it is desirable to be able to set the judgment time appropriately based on the playing situation.

[0009] In one embodiment of the present invention, the processor of the information processing device determines the output timing of the data to be output after the performance ends based on the interval between the first performance operation and the second performance operation.

[0010] According to the present invention, for example, the end of the performance can be accurately determined, thereby allowing for further enjoyment of the performance. Attached Figure Description

[0011] Figure 1 This is a diagram illustrating an example of an image display system implemented in this way.

[0012] Figure 2 This diagram illustrates an example of a keyboard instrument combined with a tablet computer's image display system.

[0013] Figure 3 This is a block diagram illustrating an example of a numeric keypad 1 implemented in this way.

[0014] Figure 4 This is a function block diagram representing an example of a tablet computer 3.

[0015] Figure 5 This is a flowchart illustrating an example of the processing procedure of tablet computer 3.

[0016] Figure 6 It is a diagram representing a musical score example.

[0017] Figure 7 It means according to Figure 6 The first image is an example of a musical score.

[0018] Figure 8 It means according to Figure 6 The second image is an example of a musical score.

[0019] Figure 9 It means Figure 5 A flowchart of an example of the processing procedure in step S4.

[0020] Figure 10 It is used for explanation Figure 9 The diagram shows the calculation of the note interval in step S42.

[0021] Figure 11 This is a flowchart illustrating an example of the processing steps related to updating the coefficient α during the decision period. Detailed Implementation

[0022] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0023] <Structure>

[0024] Figure 1 This is a schematic diagram illustrating an example of an image display system implemented in this way. Figure 1 The image display system shown depicts images (pictures) in real time in conjunction with the user's (performer's) performance. This image display system parses performance data obtained from electronic musical instruments or similar devices that can output the user's performance data (e.g., MIDI data) and generates images based on the results.

[0025] exist Figure 1 In this context, the image display system includes electronic musical instruments, information processing devices, and display devices.

[0026] Electronic musical instruments generate performance data (e.g., MIDI data) based on the user's playing and output this data to an information processing device. The information processing device parses the received performance data and generates image data. This device can be, for example, a tablet computer or a personal computer (PC). A display device shows the image generated by the information processing device.

[0027] Figure 2 This diagram illustrates an example of a tablet computer-based image display system combined with a keyboard instrument. The system includes a numeric keypad 1 and a tablet computer 3 that can be connected to the numeric keypad 1. The numeric keypad 1 is, for example, an electronic keyboard instrument such as an electronic piano, synthesizer, or electronic organ.

[0028] In addition to the multiple keys 10 arranged on the keyboard, the numeric keypad 1 also includes a display unit 14, an operation unit 18, and a music stand MS. For example... Figure 2 As shown, the tablet computer 3 connected to the numeric keypad 1 can be placed on the music stand MS to display sheet music or used as a user interface.

[0029] Key 10 is the operator used by the performer to specify the pitch. By pressing / releasing key 10, the numeric keypad 1 sounds or silences the note corresponding to the specified pitch. Pressing and releasing keys are examples of performance operations. They can be captured individually as performance operations, or as a combination of pressing and releasing keys. Alternatively, only pressing keys can be captured as individual performance operations for counting, or only releasing keys can be captured as a performance operation. For example, events that trigger the generation of performance data can be captured as performance operations. All actions that generate performance data can be captured as performance operations, or only actions that generate a specific type of performance data (note on, note cessation, etc.) can be captured as performance operations.

[0030] The display unit 14 may include, for example, a liquid crystal display (LCD) with a touch panel, which displays messages accompanying the performer's operation of the operation unit 18. When the display unit 14 has a touch panel function, the display unit 14 can perform one of the functions of the operation unit 18.

[0031] The control unit 18 includes operation buttons, dials, etc., for the performer to make various settings. The user can operate the operation buttons or dials to perform various settings such as volume adjustment.

[0032] Figure 3This is a block diagram illustrating an example of a digital keyboard 1 implemented in this way. The digital keyboard 1 includes a USB interface (I / F) 11, RAM (Random Access Memory) 12, ROM (Read Only Memory) 13, a display unit 14, a display controller 15, an LED (Light Emitting Diode) controller 16, a keyboard 17, an operation unit 18, a key scanner 19, a MIDI interface (I / F) 20, a system bus 21, a CPU (Central Processing Unit) 22, a timer 23, a sound source 24, a digital-to-analog (D / A) converter 25, a mixer 26, a D / A converter 27, a sound synthesizer LSI 28, and an amplifier 29. Here, the sound source 24 and the sound synthesizer LSI 28 are implemented, for example, as a DSP (Digital Signal Processor).

[0033] CPU22, sound source 24, sound synthesis LSI28, USB interface 11, RAM12, ROM13, display controller 15, LED controller 16, key scanner 19 and MIDI interface 20 are connected to system bus 21.

[0034] CPU 22 is the processor that controls the numeric keypad 1. Specifically, CPU 22 reads the program stored in ROM 13 into RAM 12, which serves as working memory, and executes it to implement various functions of the numeric keypad 1. CPU 22 operates according to a clock provided by timer 23. The clock is used, for example, to control sequences of automatic playing and automatic accompaniment.

[0035] ROM13 stores programs, various setting data, automatic accompaniment data, etc. Automatic accompaniment data may include preset rhythm patterns, chord progressions, base patterns, or obligato melody data. Melody data may include pitch information for each note and timing information for each note's articulation.

[0036] The timing of each note's sound can be the interval between notes or the elapsed time since the beginning of an automatic piece. The unit of time is mostly tick. A tick is a unit used in general sequencers, based on the tempo of a piece of music. For example, if the sequencer's resolution is 480, then 1 / 480th of the time of a quarter note is 1 tick.

[0037] Automatic accompaniment data is not limited to ROM13, but can also be stored in an information storage device or medium not shown. The format of the automatic accompaniment data can follow the file format used by MIDI.

[0038] The display controller 15 is an IC (Integrated Circuit) that controls the display state of the display unit 14. The LED controller 16 is, for example, an IC. The LED controller 16 illuminates the keys of the keyboard 17 according to instructions from the CPU 22, guiding the player's performance.

[0039] The key scanner 19 constantly monitors the key press / release status of the keyboard 17 and the on / off operation status of the operation unit 18. Furthermore, the key scanner 19 transmits the status of the keyboard 17 and the operation unit 18 to the CPU 22.

[0040] The MIDI interface 20 inputs MIDI data (performance data, etc.) from external devices such as the MIDI device 4, or outputs MIDI data to external devices. The numeric keypad 1 can use an interface such as USB (Universal Serial Bus) to send and receive MIDI data or music files with external devices. The received MIDI data is delivered to the sound source 24 via the CPU 22. The sound source 24 emits sounds according to the timbre, volume (velocity), timing, etc., specified by the MIDI data.

[0041] In addition to information such as pitch number and timbre number corresponding to key 10, MIDI data (MIDI messages) can also represent timing information such as note triggering and note termination, intensity information called velocity, or various control information, and all other information related to the performance of the music.

[0042] Sound source 24 is, for example, a so-called GM sound source that conforms to the GM (General MIDI) standard. This type of sound source allows for timbre changes by providing program changes as MIDI messages included in the MIDI data. Furthermore, by providing control changes, predetermined effects can be controlled.

[0043] The sound source 24 has the capability to simultaneously emit up to 256 sounds. For example, the sound source 24 reads musical tone waveform data from a waveform ROM (not shown) and outputs it as digital musical tone waveform data to the D / A converter 211. The D / A converter 211 converts the digital musical tone waveform data into analog musical tone waveform signals.

[0044] When the CPU 22 provides the text data of the lyrics and pitch-related information as vocal data, the sound synthesis LSI 28 synthesizes the corresponding vocal sound data and outputs it to the D / A converter 25. The D / A converter 25 converts the sound data into an analog sound waveform signal.

[0045] Mixer 26 combines analog musical waveform signals and analog audio waveform signals to generate an output signal. This output signal is amplified by amplifier 29 and output from an output terminal such as a speaker or headphone output.

[0046] The tablet computer 3 is connected to the system bus 21 via USB interface 11. The tablet computer 3 can obtain MIDI data (performance data) generated by playing the digital keyboard 1 via USB interface 11.

[0047] Furthermore, storage media (not shown) can also be connected to the system bus 21 via the USB interface 11. Examples of storage media include USB memory, floppy disk drive (FDD), hard disk drive (HDD), CD-ROM drive, and magneto-optical disk (MO) drive. If no program is stored in ROM 106, by pre-storing the program in a storage medium and reading it into RAM 105, the CPU 111 can perform the same operations as if the program were stored in ROM 106.

[0048] Figure 4 This is a functional block diagram illustrating an example of a tablet computer 3. The tablet computer 3 is a portable information processing device equipped with an application for generating and outputting images reflecting performance using the numeric keypad 1. Additionally, the tablet computer 3 may also include a sequencer that receives MIDI data from the numeric keypad 1 and reproduces the musical data.

[0049] The tablet computer 3 mainly includes an operation unit 31, a display unit 32, a communication unit 33, an audio output unit 34, a memory 35, and a control unit 36 ​​(CPU). Each unit (operation unit 31, display unit 32, communication unit 33, audio output unit 34, memory 35, and control unit 36) can be communicatively connected via a bus 37, and can send and receive necessary data between each unit.

[0050] The operation unit 31 includes, for example, a power switch or similar switch for turning the power on / off. The display unit 32 is equipped with an LCD monitor with a touch panel for displaying images. The display unit 32 also has a touch panel function, thus it can serve as one end of the operation unit 31.

[0051] The communication unit 33 includes a wireless unit or a wired unit for communication with other devices. In one embodiment, for example, it is wired to the numeric keypad 1 via a USB cable, thereby enabling the tablet computer 3 to send and receive various digital data with the numeric keypad 1.

[0052] The audio output unit 34 is equipped with a speaker, headphone jack, etc., and reproduces and outputs analog sound, music, or outputs audio signals.

[0053] The control unit 36 ​​is equipped with a processor such as a CPU, and is responsible for controlling the tablet computer 3. The CPU of the control unit 36 ​​performs various processes according to the control program stored in the memory 35 or the installed application program.

[0054] The memory 35 includes ROM 40 and RAM 50.

[0055] ROM 40 stores, for example, the program 41 executed by the control unit 36, various data tables, etc. In particular, in the embodiment, the determination period T related to the determination of the end of the performance is stored in the storage area 42 of ROM 40.

[0056] RAM 50 stores the data required to operate program 41. Furthermore, RAM 50 also functions as data generated by control unit 36, MIDI data sent from digital keyboard 1, and a temporary storage area for application deployment. In this embodiment, RAM 50 stores performance data 50a containing MIDI data, character data 50b, first image data 50c, and second image data 50d.

[0057] Furthermore, in the implementation, program 41 includes a music analysis routine 41a, a first image creation routine 41b, a second image creation routine 41c, and an output control routine 41d.

[0058] Music parsing routine 41a retrieves the performance data generated sequentially based on the playing of numeric keypad 1 and stores it in RAM 50 as performance data 50a. Furthermore, music parsing routine 41a primarily performs music parsing based on the pitch data contained in performance data 50a, determining the tonality, chord types, and note names of the music.

[0059] Furthermore, there are no particular limitations on the methods for music analysis or for determining tonality, chord types, etc. For example, the methods disclosed in Japanese Patent No. 3211839 can be used.

[0060] The first image creation routine 41b generates motion image data to be displayed in real time during the performance based on the results of music analysis. The generated motion image data is temporarily stored in RAM 50 as first image data 50c, and then immediately read out and displayed on display unit 32.

[0061] The second image creation routine 41c creates a still image as a summary display after the music analysis is completed. The motion image data of the created still image is temporarily stored in RAM 50 as second image data 50d, and then output (read out) and displayed on display unit 32 at an appropriate time.

[0062] Output control routine 41d determines the timing for outputting the second image data based on the timing of generating each performance data from the digital keyboard 1, or the interval of each timing of acquiring the performance data.

[0063] <Function>

[0064] Next, the function of the above structure will be explained. In the following description, it is assumed that the tablet computer 3 can be communicatively connected to the numeric keypad 1. Furthermore, it is assumed that the display unit 32 of the tablet computer 3 ( Figure 4 The application that displays images starts on the tablet 3.

[0065] Figure 5 This is a flowchart illustrating an example of the processing procedure of tablet computer 3. Figure 5 In this process, the control unit 36 ​​(CPU) of the tablet computer 3 waits for input of performance data from the numeric keypad 1 (step S1). If performance data is input in step S1 (yes), the control unit 36 ​​performs performance determination processing (step S2). In step S2, the control unit 36 ​​determines the performance based on the acquired performance data, such as the key of the piece being played (e.g., 24 keys from C major to B minor), the type of chord (e.g., Major, minor, sus4, aug, dim, 7th, etc.), and the beat. The determination result obtained here is reflected in the first image.

[0066] Figure 6 It is a diagram representing a musical score example. For example, when performing... Figure 6 When playing as shown, as Figure 7 As shown, following the order of Do Re Mi Fa…, the roles of flower (1), leaf (2), ladybug (3), and butterfly (4) are arranged in sequence to form the first image. When the performance ends, as shown… Figure 8 As shown, the characters are arranged on, for example, a spiral track, forming the second image.

[0067] return Figure 5 Continuing the explanation. The control unit 36 ​​generates a first image based on the result of the performance determination processing and outputs it to the display unit 32 (step S3).

[0068] Next, the control unit 36 ​​updates the judgment period T based on the result of the performance judgment processing (step S4).

[0069] Figure 9This is a flowchart illustrating an example of the processing in step S4. A soft interrupt occurs when the decision period update process in step S4 is invoked. Then, the control unit 36 ​​first sets an initial value T0 during the decision period T (step S41). For example, 5 seconds is set as the initial value T0. Next, the control unit 36 ​​calculates the maximum value Tmax of the most recent note interval (step S42). That is, in this step, the control unit 36 ​​obtains the note trigger times of X (e.g., 4) notes traced back from the latest note trigger time, calculates the time interval of each note, and determines its maximum value Tmax.

[0070] Next, the control unit 36 ​​compares the determination period T and Tmax (step S43). If T is smaller than Tmax (T < Tmax), it is false (FALSE). That is, if the nearest note interval does not exceed the determination period T, then T = T0 remains unchanged (step S44), and the processing returns to the calling source (return).

[0071] On the other hand, if (T < Tmax) is true in step S43 (TRUE), that is, if the nearest note interval exceeds the decision period T, then the value of Tmax multiplied by the decision period update coefficient α is substituted into T (step S45), and the process returns to the calling source (return). Here, the value of coefficient α can be 1.1, which is equivalent to making the decision period T longer than the default value. In addition, the value of coefficient α is updated to different values ​​depending on the performance situation.

[0072] return Figure 5 Continuing the explanation. If no performance data is input in step S1 (No), or if step S4 ends, the control unit 36 ​​performs an end determination (step S5). In this embodiment, the end determination is performed by comparing the determination period T, which serves as a reference value, with the elapsed time since the last note. That is, if the elapsed time since the last note is longer than the determination period T, the performance is determined to be over (Yes). If the result in step S5 is no, the process returns to step S1 until the result is yes.

[0073] During the performance, the processing steps S1 to S5 are repeated. When the performance ends, step S5 is "yes". In this way, the control unit 36 ​​creates a second image reflecting the analysis result of the accumulated performance data 50a and displays it on the display unit 32 (step S6).

[0074] Figure 10 It is used for explanation Figure 9 The diagram shows the calculation of the note interval in step S42. In this implementation, the "note interval" refers to the period from the triggering of the previous note to the triggering of the next note. At this time, notes that overlap in time are treated as a group of notes.

[0075] For example, when pressing multiple keys in a concentrated manner, such as in chord playing, strictly speaking, the timing of the notes on each key is mostly slightly staggered. For instance... Figure 10 As shown, even if the C, E, and G notes of a C chord are slightly offset, if the offset is within a predetermined value, they are grouped together, and the occurrence of the note trigger and note cessation are counted as 1 each. For example, the key press of the first note in a group is set as the note trigger, and the key release of the last note is set as the note cessation. Furthermore, the interval until the next note triggers is set as the note interval. As the name suggests, the note trigger of a note can be counted as the key press of that note.

[0076] Figure 11 This is a flowchart illustrating an example of the processing related to the updating of the coefficient α during the determination period, which serves as a reference value for determining the output timing of the second image data. Figure 11 In step S7, the control unit 36 ​​counts the number of notes N over the past few seconds (e.g., 8 seconds) since the current time point and compares it with a predetermined threshold N1 (e.g., 5 notes) (step S8). If N is greater than the threshold N1 (yes), then 1.1 is substituted into α (step S10). On the other hand, if N is less than or equal to the threshold N1 (no), then a value greater than 1.1, such as 1.5, is substituted into α (step S9).

[0077] If, in step S8, it is determined that the number of notes produced during the performance in the past few seconds was low, this likely indicates that the performance is unstable or is being played slowly. Therefore, in this case, α is set to a larger value to lengthen the determination period T. Conversely, if the number of notes produced in the past few seconds was high, α is set to a smaller value.

[0078] That is, if the number of performance data generated or acquired by the control unit 36 ​​within the set period does not reach the threshold, the output timing of the second image data is delayed compared to the case where the number of performance data reaches the threshold. Furthermore, the threshold level can be set not only to N1, but also to multiple values ​​such as N1, N2, N3, etc., allowing α to gradually change in several stages.

[0079] <Effect>

[0080] As described above, in this implementation, the timing of outputting the second image is controlled based on the results of music analysis of the performance data during each performance. For example, when playing a fast-paced piece, the second image is output at a timing earlier than when playing a slow-paced piece. Conversely, when a beginner plays slowly at a slow tempo, the time from the end of the performance to the display of the second image becomes longer.

[0081] Because of this, the second image can be output at a good time after the performance ends, rather than at a fixed time. That is, it avoids the disappointing situation where the final image appears even though the performance has not ended, or conversely, the final image never appears even though the performance has ended.

[0082] That is, according to the implementation method, the end of the performance can be reliably determined. Therefore, a program, electronic device, method, and image display system can be provided that enhances the experiential value of technology that visualizes musical performances and further enhances the enjoyment of playing or practicing an instrument without diminishing the user's desire to practice.

[0083] Furthermore, the present invention is not limited to the embodiments described above.

[0084] <Variation Example 1>

[0085] While expecting a certain degree of stability in performance, Figure 9 In step S42, the maximum value of the nearest note interval can also be used instead of the average value.

[0086] <Variation Example 2>

[0087] As a condition for updating the specified time in step S42, the maximum (or average) value of the most recent note interval can be replaced by an indicator of performance instability. Alternatively, non-musical judgment results or rhythmic instability can be used as the indicator.

[0088] For example, in cases where chords cannot be determined through music analysis (determination failure), in cases where the number of times chords cannot be determined exceeds a certain number, or by detecting simultaneous pressing of more than 5 adjacent white keys, combinations of pitch data detected at approximately the same time, or combinations of pitch data detected multiple times within a certain time interval, it can be determined that the performance operation is non-musical.

[0089] For example, if non-musical judgments are used, the update of the judgment period T can take into account conditions such as "the non-musical judgments have occurred a specified number of times (e.g., 3 times) or more within the most recent number of beats (e.g., 8 beats)" (Condition A).

[0090] The logical AND operation of condition A and the "determination based on the operation value of the nearest note interval (condition B)" in step S42 can be performed, and the determination period T can be updated if the result is true.

[0091] <Variation Example 3>

[0092] More directly, if the beat of the music is set as the condition for determining the end of the performance, the following case is considered. That is, the control unit 36 ​​determines the beat of the performance based on the performance data obtained from the digital keypad 1. Then, if the determined beat is a second beat that is slower than the first beat, the output timing of the second image data is determined such that the output timing determined in the case of the second beat is later than the output timing determined in the case of the first beat.

[0093] <Variation Example 4>

[0094] As a more direct way to determine the end of a performance, there is also a method of confirming the performer's actions. That is, a camera or similar device that can detect the performer's actions can be set up, for example, by determining whether the performer stands up from the chair, and the final image (second image) can be displayed without relying on the time elapsed since the last note ended.

[0095] That is, in this embodiment, the processor 36 of the information processing device (display device) 3 determines the output timing of the data after the performance ends based on the interval between the user's first and second performance operations on the keyboard 17 of the electronic musical instrument 1. As one embodiment, this interval is calculated based on the acquisition timing of note data obtained on the information processing device 3 side according to the first performance operation on the electronic musical instrument 1 side, and the acquisition timing of note data obtained on the information processing device 3 side according to the second performance operation following the first performance operation on the electronic musical instrument 1 side. In other words, the interval between the first and second performance operations can be calculated by any method.

Claims

1. A method for an information processing apparatus, wherein, The information processing device updates the determination period related to the determination of the end of performance based on the interval of multiple performance operations, including the first performance operation and the second performance operation, or the determination result of whether the performance performed by the multiple performance operations is musically appropriate. The end of the performance is determined by comparing the elapsed time since the last note was played with the updated determination period. If the performance is determined to be over, the timing for outputting the data after the performance ends is determined.

2. The method according to claim 1, wherein, The data includes image data.

3. The method according to claim 1 or 2, wherein, The information processing device determines whether to update the determination period based on a comparison between one of the maximum value and the average value of the latest note interval in the interval and a determination period used as a reference value. The baseline value is changed based on the interval.

4. The method according to claim 1, wherein, The beat is determined based at least on the first performance data generated based on the first performance operation and the second performance data generated based on the second performance operation. If the determined beat is a second beat that is slower than the first beat, the output timing is determined such that the timing determined in the case of the second beat is delayed compared to the timing determined in the case of the first beat.

5. The method according to claim 1, wherein, If the amount of performance data generated or acquired within a set period does not reach a threshold, the output timing is delayed compared to the case where the threshold is reached.

6. The method according to claim 1, wherein, If the chord cannot be determined by music analysis based on the performance operation, the number of times the chord cannot be determined exceeds the specified number, or if five or more adjacent white keys are detected to be pressed at the same time, then the performance is determined to be musically inappropriate.

7. The method according to claim 1, wherein, If the performance is determined to be musically inappropriate, and either the maximum value of the latest note interval or the average value of the latest note interval exceeds the determination period used as a reference value during the intervals of the plurality of performance operations, the determination period is updated by extending the determination period.

8. An information processing apparatus, wherein, have: Input / output interfaces; and At least one processor, The at least one processor performs the following processing: The first performance data corresponding to the first performance operation and the second performance data corresponding to the second performance operation are obtained through the input / output interface. Based on the obtained first performance data and second performance data, the interval between the first performance operation and the second performance operation is obtained. Based on the obtained interval or the determination result of whether the performance performed by the multiple performance operations is musically appropriate, update the determination period related to the performance end determination. The end of the performance is determined by comparing the elapsed time since the last note was played with the updated determination period. If the performance is determined to be over, the timing for outputting the data after the performance ends is determined.

9. The information processing apparatus according to claim 8, wherein, The data includes image data.

10. The information processing apparatus according to claim 8 or 9, wherein, Based on a comparison between one of the maximum value and the average value of the latest note interval within the specified intervals and a determination period serving as a baseline value, a decision is made as to whether to update the determination period. The baseline value is changed based on the interval.

11. The information processing apparatus according to claim 8, wherein, Based on the first performance data and the second performance data, the beat is determined. If the determined beat is a second beat that is slower than the first beat, the output timing is determined such that the output timing determined in the case of the second beat is delayed compared to the output timing determined in the case of the first beat.

12. The information processing apparatus according to claim 8, wherein, If the amount of performance data generated or acquired within a set period does not reach a threshold, the output timing is delayed compared to the case where the threshold is reached.

13. The information processing apparatus according to claim 8, wherein, If the chord cannot be determined by music analysis based on the performance operation, the number of times the chord cannot be determined exceeds the specified number, or if five or more adjacent white keys are detected to be pressed at the same time, then the performance is determined to be musically inappropriate.

14. The information processing apparatus according to claim 8, wherein, If the performance is determined to be musically inappropriate, and either the maximum value of the latest note interval or the average value of the latest note interval exceeds the determination period used as a reference value during the intervals of the plurality of performance operations, the determination period is updated by extending the determination period.

15. An image display system, wherein, Equipped with electronic musical instruments and display devices, The electronic musical instrument The first performance data generated based on the first performance operation is sent to the display device. The second performance data generated based on the second performance operation is sent to the display device. The display device, Obtain the first performance data and the second performance data. Based on the obtained first performance data and second performance data, the interval between the first performance operation and the second performance operation is obtained. Based on the obtained interval or the determination result of whether the performance performed by the multiple performance operations is musically appropriate, update the determination period related to the performance end determination. The end of the performance is determined by comparing the elapsed time since the last note was played with the updated determination period. If the performance is deemed to have ended, the timing for outputting the image data after the performance ends is determined. The image data is displayed based on the determined output timing.

Images