Display information generation method, program, information processing apparatus, electronic musical instrument, and electronic musical instrument system

Abstract

To reduce display of inappropriate data.SOLUTION: A control part of an information processor generates display information for displaying a piano roll corresponding to reproduction of a musical piece based on a series of performance information on a display part in each generation cycle on the basis of the performance information. At that time, the control unit determines a terminal end of a generation range of the display information in the musical piece for each generation cycle on the basis of note distribution in the musical piece.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a display information generating method, a program, an information processing device, an electronic musical instrument, and an electronic musical instrument system. [Background technology]

[0002] Conventionally, there is known a piano roll display device that displays a piano roll according to an automatic performance based on performance information (see, for example, Patent Document 1). In a piano roll, for example, the horizontal direction indicates the arrangement of the keys (pitch), the vertical direction indicates time, and a bar corresponding to the timing and length of key presses is displayed on the vertical axis corresponding to the pitch of the pressed note. There are also piano rolls in which the horizontal and vertical directions are reversed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-279470 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, display information for displaying a piano roll on a screen is generated at a predetermined cycle based on performance information of a predetermined range (length) and registered in a display buffer, and the piano roll based on the display information registered in the display buffer is displayed sequentially on the screen as the music progresses. However, when a boundary between pieces of display information generated at different cycles is displayed on the screen, a problem occurs in the display of the boundary, resulting in an inappropriate display.

[0005] An object of the present invention is to reduce the display of irrelevant data. [Means for solving the problem]

[0006] In order to solve the above problem, the display information generation method of the present invention includes a generation step in which a computer divides display information into multiple parts to display an image based on feature information on a display unit, and the generation step is controlled so that the range of the divided display information to be generated is variable according to the feature information. [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce the display of inappropriate data. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an electronic musical instrument system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of performance information. [Figure 3] FIG. 1 is a diagram showing an example of a piano roll. [Figure 4] 10 is a flowchart showing the flow of a piano roll display process executed by a control unit of the information processing device in FIG. [Figure 5] 5 is a flowchart showing the flow of the song analysis process executed in step S3 of FIG. 4. [Figure 6] 6 is a flowchart showing the process of analyzing tempo information executed in step S31 of FIG. 5. [Figure 7] 6 is a flowchart showing the flow of the note information analysis process executed in step S32 of FIG. 5. [Figure 8] 5 is a flowchart showing the flow of the display information generation process started in step S4 of FIG. [Figure 9] 10 is a flowchart showing the flow of a generation range update process executed in step S44 of FIG. 8. [Figure 10] 10A and 10B are diagrams illustrating examples of index values ​​from the end to the upper limit of a reference generating range, and the end of the generating range to be determined. [Figure 11]10A and 10B are diagrams illustrating examples of index values ​​from the end to the upper limit of a reference generating range, and the end of the generating range to be determined. [Figure 12] FIG. 10 is a diagram illustrating an example of the configuration of display information. [Figure 13] 10A and 10B are diagrams for explaining the display range of the piano roll and the generation range of display information at a certain time point and after a generation period has elapsed from that time point. [Figure 14] 5 is a flowchart showing the flow of the display process started in step S5 of FIG. 4. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are subject to various limitations that are technically preferable for implementing the present invention. Therefore, the technical scope of the present invention is not limited to the following embodiments and illustrated examples.

[0010] 1, the electronic musical instrument system 100 is configured to include an information processing device 1 and an electronic musical instrument 2. The information processing device 1 and the electronic musical instrument 2 are communicatively connected via a wired or wireless connection so as to be able to send and receive data.

[0011] The information processing device 1 is a device that displays a piano roll in response to the playback of a piece of music based on performance information (feature information) of the piece of music selected by a user. The information processing device 1 can be, for example, a tablet PC (Personal Computer), a notebook PC, a smartphone, or the like.

[0012] As shown in FIG. 1, the information processing device 1 is configured to include a control unit 11, a memory unit 12, an input unit 13, a display unit 14, a communication unit 15, and an output unit 16, and each unit is connected by a bus 17.

[0013] The control unit 11 is a computer including at least one processor such as a CPU (Central Processing Unit) and a RAM (Random Access Memory), and controls each unit of the information processing device 1. Specifically, the CPU of the control unit 11 reads out a designated program from among the system programs and various application programs stored in the storage unit 12, loads it into the RAM, and executes various processes in cooperation with the loaded program. For example, when an instruction to launch the piano roll display application 121 stored in the storage unit 12 is given by operating the input unit 13, the CPU of the control unit 11 executes a piano roll display process, which will be described later.

[0014] The storage unit 12 is configured by a non-volatile semiconductor memory, a hard disk drive (HDD), etc. The storage unit 12 stores the system program of the information processing device 1, various application programs, and data required for executing the programs. The storage unit 12 is not limited to being built into the information processing device 1, and may include an external recording medium that is detachable from the information processing device 1, such as an external HDD or USB memory.

[0015] In this embodiment, the storage unit 12 stores a piano roll display application 121 for the control unit 11 to execute a piano roll display process, which will be described later. The piano roll display application 121 is a program code or source code for executing the piano roll display process.

[0016] The storage unit 12 also stores a series of performance information (e.g., MIDI (Musical Instrument Digital Interface) data) for playable music pieces. The performance information includes note information, tempo information, and beat information. As shown in FIG. 2, beat information and tempo information are arranged at the beginning of the series of performance information. The beat information is information indicating the beat, such as 4 / 4 beat or 3 / 4 beat. The beat information determines the length of the measure and the beat. For example, in 4 / 4 beat, one beat is defined as 480 ticks, which is a quarter note, and one measure is defined as 480 × 4 = 1920 ticks. The tempo information is, for example, BPM (Beat Per Minute). After the beat information and tempo information at the beginning, note information, tempo information, beat information, and the like are arranged according to the music piece. Note information indicates the pitch, timing, length, etc. of a note, and as shown in Figure 2, it consists of the onset time (tick), note number (pitch), velocity value, length (tick), channel (for identifying the part), etc. Tempo information consists of the time (tick), BPM, etc. Beat information consists of the time (tick), beat, etc.

[0017] The input unit 13 is composed of push button switches, a touch panel attached to the display unit 14, etc. The input unit 13 detects the operation of the push button switches or the touch operation on the screen by the user, and outputs an operation signal to the control unit 11.

[0018] The display unit 14 is configured with at least one of displays such as an LCD (Liquid Crystal Display) and an EL (Electro Luminescence) display, and performs various displays according to display information instructed by the control unit 11.

[0019] The communication unit 15 includes a wired unit or a wireless unit for communicating with external devices such as the electronic musical instrument 2, and transmits and receives data to and from the external devices. In this embodiment, the communication unit 15 transmits and receives data to and from the electronic musical instrument 2 via a communication interface (in this embodiment, a MIDI interface) not shown.

[0020] The output unit 16 includes a sound source, a D / A converter, an amplifier, a speaker, etc., and outputs sound based on instructions from the control unit 11.

[0021] 1, the electronic musical instrument 2 is, for example, a keyboard instrument. The electronic musical instrument 2 is configured to include a control unit 21, a storage unit 22, a keyboard 23, an input unit 24, a display unit 25, a communication unit 26, an output unit 27, and pedals 28, and each unit is connected via a bus 29.

[0022] The control unit 21 is a computer that includes at least one processor such as a CPU and a DSP (Digital Signal Processor), RAM, etc., and controls each part of the electronic musical instrument 2. Specifically, the CPU of the control unit 21 reads out a designated program from among the system programs and various programs stored in the storage unit 22, expands it in the RAM, and executes various processes in cooperation with the expanded program. For example, the control unit 21 causes the output unit 27 to output (sound) musical instrument sounds based on the operation of the keyboard 23 and pedals 28. The control unit 21 also causes the output unit 27 to output (sound) musical sounds based on performance information input from the information processing device 1 via the communication unit 26.

[0023] The storage unit 22 is configured with a non-volatile semiconductor memory, a hard disk drive (HDD), etc. The storage unit 22 stores the system program of the electronic musical instrument 2, various programs, and data required for executing the programs. The storage unit 22 is not limited to being built into the electronic musical instrument 2, but may also include an external recording medium that is detachable from the electronic musical instrument 2, such as an external HDD or USB memory.

[0024] The keyboard 23 includes a plurality of keys (performance operators) and a detection unit that detects pressed / released keys, and outputs information indicating the pitch, velocity, and timing of the keys pressed / released by the user to the control unit 21.

[0025] The input unit 24 is configured with at least one of hardware keys such as push button switches and software keys such as a touch panel attached to the display unit 25. The input unit 24 outputs an operation signal to the control unit 21 in response to a user's operation of the push button switch or a touch operation on the screen.

[0026] The display unit 25 is configured with a display such as an LCD (Liquid Crystal Display), and performs various displays according to display information instructed by the control unit 21.

[0027] The communication unit 26 includes a wired unit or a wireless unit for communicating with external devices such as the information processing device 1, and transmits and receives data to and from the external devices. In this embodiment, the communication unit 26 transmits and receives data to and from the information processing device 1 via a communication interface (a MIDI interface in this embodiment) not shown.

[0028] The output unit 27 includes a sound source unit, a D / A converter, an amplifier, a speaker, etc. In accordance with instructions from the control unit 21, the output unit 27 reads waveform data stored in advance in a waveform ROM provided in the sound source unit or generates waveform data, and outputs musical instrument sounds based on the waveform data from the speaker via the D / A converter and amplifier.

[0029] The pedal 28 includes, for example, a pedal such as a damper pedal and a detector that detects whether the pedal is ON or OFF, and outputs pedal ON / OFF information to the control unit 21.

[0030] Next, the operation of the electronic musical instrument system 100 of this embodiment will be described. In the information processing device 1, when an instruction to start the piano roll display application 121 is given via the input unit 13, the control unit 11 starts the piano roll display application 121 and establishes a communication connection with the electronic musical instrument 2 via the communication unit 15. Once the communication connection with the electronic musical instrument 2 is established, the control unit 11, in cooperation with the piano roll display application 121, causes the display unit 14 to display a display screen (not shown) showing a selection field for a song to be played, and starts piano roll display processing.

[0031] The piano roll display process is a process for displaying a piano roll 30 on the display unit 14 in response to the playback of a musical piece. The displayed piano roll 30 is a graphic, as shown in FIG. 3 , in which the horizontal coordinate S indicates the position (pitch) of each key on the keyboard 23, the vertical coordinate L indicates time, and bars 31 corresponding to the timing and duration of key presses are displayed on the vertical axis corresponding to the pitch of the pressed note. The lower the vertical coordinate L, the earlier the playback timing. The bars 31 are displayed in different colors depending on the part (left hand, right hand). The bars 31 are displayed as if they are moving (scrolling) from top to bottom in accordance with the progression of the musical piece, i.e., the passage of time. Under the control of the control unit 11 of the information processing device 1 or the control unit 21 of the electronic musical instrument 2, the sound corresponding to the pitch of the key corresponding to the bar 31 is played (output) when it reaches the position (bottom) of the keyboard diagram 32. In other words, the faster the tempo of the musical piece, the faster the bar 31 moves downward. When a song is selected by operating the input unit 13, the control unit 11 displays a play start button on the display unit 14. When an instruction to start playback is input by operating the input unit 13 (when the play start button is pressed), the control unit 11 displays a play stop button on the display unit 14. The images "9" and "10" on the left side of the piano roll 30 indicate the bar numbers of the selected song, and are set to be displayed in accordance with the time coordinates at the beginning of the bar.

[0032] The piano roll display process will be described below with reference to Figures 4 to 14. First, the control unit 11 performs initialization necessary for the operation of the piano roll display process (step S1). For example, the control unit 11 initializes memory areas such as a display buffer, variables, an index value table, and the like used in the piano roll display process.

[0033] Next, the control unit 11 determines whether or not a song to be played has been selected by the user's operation on the input unit 13 (step S2). If it is determined that a song to be played has not been selected by the user's operation on the input unit 13 (step S2; NO), the control unit 11 returns to step S2.

[0034] When it is determined that a piece of music to be played has been selected by operating the input unit 13 (step S2; YES), the control unit 11 executes a music analysis process (step S3).

[0035] 5, in the song analysis process, the control unit 11 first executes a tempo information analysis process (step S31), and then executes a note information analysis process (step S32). These steps S31 and S32 are more efficient when the control unit 11 processes multiple lines in thread units at once, rather than completing the process for each line along the horizontal direction S in FIG.

[0036] The tempo information analysis process is a process of reading performance information of the selected song, analyzing the tempo information, and setting an optimal display information generation cycle based on the tempo information. The display information generation cycle is a cycle at which display information for displaying the piano roll 30 on the display unit 14 is generated and registered (written) in the display buffer. Note that information required for piano roll display, such as tempo information for the longest section (described later) based on the song performance information, may be linked to the song in advance and stored in the storage unit 12. In this case, when the user selects a song in step S2, the control unit 11 of the information processing device 1 can read the tempo information for the longest section (described later) linked to the selected song from the storage unit 12, thereby omitting steps S311 to S317 (described later).

[0037] Conventionally, the display information generation cycle and the display information generation range for each generation cycle have been fixed, regardless of tempo. Here, the display information generation range indicates the number of ticks from the current playback position for which display information is prepared (the range for which display information is generated). However, if the display information generation cycle and generation range are fixed, wasteful processing and memory consumption may occur. For example, if a fixed generation range is set longer to generate all the display information at once in order to continuously display a fast roll on the piano roll 30 for a fast-tempo piece of music, the longer the generation range, the greater the memory usage. Therefore, at least one of a large-capacity memory and a high-speed control unit 11 is required to ensure sufficient free memory space. Such memory and control circuitry may be over-specified for slow tempos, resulting in high costs for the information processing device 1. Furthermore, because the generation cycle is fixed, if the generation process is set to keep up with a fast tempo, the process will be completed in a short time if the tempo is slow, resulting in wasteful processing. Therefore, in this embodiment, the generation cycle and range of display information are set based on the tempo of the song so that the display information is generated a necessary and sufficient number of times so that the display information can be generated in time with the progress of the song.

[0038] 6, in the tempo information analysis process, first, the control unit 11 sets a reference generation period of the display information (step S311). For example, the control unit 11 sets a predetermined reference generation period (for example, a 1-second period (0.5 bars) in a 4 / 4 time signature at BPM 120).

[0039] Next, the control unit 11 reads one piece of performance information (step S312) and determines whether it is the end of the music piece (EOF) (step S313). If it is determined that it is not the end of the music piece (step S313; NO), the control unit 11 determines whether the read performance information is tempo information (step S314). If it is determined that the read performance information is tempo information (step S314; YES), the control unit 11 calculates the length (in ticks) of the section played at the tempo of the tempo information (step S315). For example, if the read tempo information is BPM 120, the control unit 11 searches the performance information and calculates the length of the section played at BPM 120 in ticks. If the section played at BPM 120 is divided into multiple sections, the control unit 11 adds up the lengths of the multiple sections. The calculated section length is stored in RAM in association with the tempo information. After calculating the section length, the control unit 11 reads the next piece of performance information (step S316) and returns to step S313.

[0040] On the other hand, if it is determined in step S313 that the music piece has reached its end (step S313; YES), the control unit 11 determines the tempo of the longest section in the music piece as the tempo of the music piece (step S317).

[0041] Next, the control unit 11 applies the tempo change made by the user to the tempo of the music (step S318). That is, the control unit 11 adds or subtracts the tempo change value set by the user to the tempo of the music to obtain the tempo of the music. The control unit 11 then optimizes the generation cycle of the display information based on the tempo of the music (step S319). Note that the tempo change made by the user is optional, so it does not necessarily have to be changed.

[0042] In step S319, the control unit 11 optimizes the generation cycle based on the tempo of the music piece so that the faster the calculated tempo, the longer the generation cycle. For example, the control unit 11 uses "a generation cycle of 1 second at 120 BPM" set as the reference generation cycle as a reference, and increases the generation cycle by 0.5 seconds for every 30 increase in the tempo of the music piece, and decreases the generation cycle by 0.5 seconds for every 30 decrease in the tempo of the music piece. Here, since the movement speed of the bar 31 is fast in music pieces with fast tempos, it is preferable to keep the generation range for each cycle wide to match the progression of the music piece. Therefore, in this embodiment, the faster the tempo of the music piece, the wider the generation range, as will be described in detail later. When the generation range is wide, a short generation cycle would result in unnecessary generation processing, so to avoid this, the generation cycle is lengthened as the tempo becomes faster.

[0043] After completing the tempo information analysis process, the control unit 11 executes note information analysis (step S32). Conventional information processing devices generate display information in batches. Therefore, when a boundary between display information generated in different but consecutive periods is displayed on the screen, display issues can occur in the display of the note at the boundary. For example, if a note played repeatedly at a certain pitch is the boundary, the bars 31 that should be represented as two separate notes at the boundary of the display information generation range may be merged due to a display issue, causing the boundary to disappear and appear as if they were a single note, preventing the user from correctly recognizing the note length. Therefore, the note information analysis process analyzes the performance information, calculates an index value every 1 / 2 beat (240 ticks) indicating the degree to which the bar is suitable for the boundary of the generation range, and creates an index value table. In other words, the note information analysis process is an analysis process for adjusting the variably set generation range.

[0044] As shown in FIG. 7, in the note information analysis process, the control unit 11 first determines a reference generation range based on a generation period determined based on tempo information of the music piece, and sets the value (tick) of the reference generation range to the performance information position (step S320). The performance information position is a variable for storing the position of the performance information to be processed. The reference generation range is a range that serves as a reference for the generation range of display information that is generated collectively to display multiple lines of piano roll 30 images along the vertical direction L. The reference generation range is calculated based on the generation period and other factors, for example, using the following (Equation 1). Since the faster the tempo, the longer the generation period, the wider the reference generation range becomes. Reference generation range = generation cycle × length of one beat (480 ticks in 4 / 4 time) × number of beats per measure × 2 (Equation 1)

[0045] Next, the control unit 11 determines whether the performance information position is the end of the song (step S321). If it is determined that the performance information position is not the end of the song (step S321; NO), the control unit 11 determines whether the note distribution at the performance information position in the song satisfies a predetermined condition by the processes of steps S322 to S330, and calculates an index value indicating the degree to which the position is suitable as a boundary of a generation range of display information based on the determination result.

[0046] Here, the position that is least affected by a problem with note display when used as a boundary of the display information generation range is considered to be more suitable as a boundary of the generation range. Therefore, for example, conditions for note distribution that would be affected by a problem with note display when used as a boundary of the display information generation range are defined in advance, and in the processing of steps S322 to S330, the above-mentioned index value is calculated based on whether the note distribution at the performance information position satisfies each of the conditions. Specifically, in the processing of steps S322 to S330, the control unit 11 determines whether the note distribution at the performance information position satisfies the following predefined conditions (1) to (4), and adds an index value if it is determined that the note distribution satisfies the conditions. The minimum value of the index value is 0, and the smaller the value, the more suitable the position is for a boundary of the generation range. (1) It is not a rest period (there is a note) (2) There is a note of the same pitch immediately before and after (3) There are notes of the same length immediately before and after (4) There are more than the specified number of notes at the same time.

[0047] The conditions used to calculate the index value are not limited to the following (1) to (4). For example, other conditions may be included, or not all of (1) to (4) may be included. Specifically, instead of (2), a condition that there is a note of the same pitch immediately after the note may be applied, and instead of (3), a condition that there is a note of the same duration immediately after the note may be applied. Furthermore, the index value may be calculated by weighting each condition.

[0048] In step S322, the control unit 11 sets the index value of the performance information position to 0 as the initial value (step S322). Next, the control unit 11 determines whether the performance information position is within a rest section (step S323). A rest section is a section where there are no notes. If it is determined that the performance information position is not within a rest section (step S323; NO), the control unit 11 adds 1 to the index value of the performance information position (step S324) and proceeds to step S325. If it is determined that the performance information position is within a rest section (step S323; YES), the control unit 11 proceeds to step S325.

[0049] In step S325, the control unit 11 determines whether there is a note of the same pitch immediately before or after the performance information position (step S325). If it is determined that there is a note of the same pitch immediately before or after the performance information position (step S325; YES), the control unit 11 adds 1 to the index value of the performance information position (step S326) and proceeds to step S327. If it is determined that there is no note of the same pitch immediately before or after the performance information position (step S325; NO), the control unit 11 proceeds to step S327.

[0050] In step S327, the control unit 11 determines whether there is a note of the same duration immediately before or after the performance information position (step S327). If it is determined that there is a note of the same duration immediately before or after the performance information position (step S327; YES), the control unit 11 adds 1 to the index value of the performance information position (step S328) and proceeds to step S329. If it is determined that there is no note of the same duration immediately before or after the performance information position (step S327; NO), the control unit 11 proceeds to step S329.

[0051] In step S329, the control unit 11 determines whether the number of on notes at the performance information position is equal to or greater than a predetermined threshold (for example, 4) (step S329). If it is determined that the number of on notes at the performance information position is equal to or greater than the predetermined threshold (step S329; YES), the control unit 11 adds 1 to the index value of the performance information position (step S330) and proceeds to step S331. If it is determined that the number of on notes at the performance information position is not equal to or greater than the predetermined threshold (step S329; NO), the control unit 11 proceeds to step S331.

[0052] In step S331, the control unit 11 stores the calculated index value in the index value table in RAM in association with the performance information position (step S331). Next, the control unit 11 moves the performance information position to the next 1 / 2 beat (step S332), and returns to step S321.

[0053] If it is determined in step S321 that the performance information position is the end of the song (step S321; YES), the control unit 11 terminates the note information analysis process and proceeds to step S4 in FIG. 4. Conventionally, image generation processing has been performed for each fixed generation range (e.g., the reference generation range in this embodiment). This disrupts image continuity at the boundary between two adjacent generation ranges on the time axis, potentially leading to misunderstandings due to inappropriate display. In this embodiment, to prevent inappropriate display, the control unit 11 creates an index value table for adjusting the boundary position so that it shifts appropriately in accordance with the performance tempo and the note distribution at the performance information position. Note that, since it is determined whether the note distribution at the performance information position, which is matched with the initial tempo of a song, satisfies the specified conditions, index value information corresponding to the determined information may be stored in advance in the storage unit 12 for each song. If the user does not change the tempo in step S318, index value information corresponding to the selected song may be read from the storage unit 12, and index values ​​may be written to the index value table based on the read index value information, thereby omitting part or all of the note information analysis process. 4 before determining in step S321 that the performance information position is the end of the song. In other words, the display information generation process and display process from step S4 onward may start before the analysis of the entire song is completed. In this way, when the song analysis process is performed in parallel with the display information generation process and display process, the displayed image may be generated based on display information corresponding to the index value table generated just before display.

[0054] In step S4 of the main flow in FIG. 4, the control unit 11 starts a thread for display information generation processing (step S4), and then starts a thread for display processing (step S5). The thread for display processing is started before the completion of the display information generation processing. After starting the thread for display processing, the control unit 11 determines whether an instruction to start playback has been input by operating the input unit 13 (step S6). If it is determined that an instruction to start playback has not been input by operating the input unit 13 (step S6; NO), the control unit 11 returns to step S6. If it is determined that an instruction to start playback has been input by operating the input unit 13 (step S6; YES), the control unit 11 starts a playback processing thread (step S7). Next, the control unit 11 determines whether an instruction to stop playback has been input by operating the input unit 13 or whether playback has ended (step S8). If it is determined that an instruction to stop playback has been input by operating the input unit 13 or that playback has ended (step S8; YES), the control unit 11 stops the playback processing thread (step S9) and returns to step S2.

[0055] 8, in the display information generation process started in step S4, the control unit 11 first calculates a reference generation range (step S41). The reference generation range can be calculated by the above-mentioned (Equation 1). Next, the control unit 11 acquires the current playback position (tick) of the song (step S42) and determines whether it is the end of the song (step S43). If it is determined that the current playback position is not the end of the song (step S43; NO), the control unit 11 executes a generation range update process (step S44). Here, immediately after starting the thread for the display information generation process, playback has not yet started. Until playback starts, in step S42, the first tick of the song is acquired as the current playback position.

[0056] As shown in Fig. 9, in the generation range update process, the control unit 11 first references the index value table and determines whether the index value at the end of the reference generation range is 0 (step S441). In this process, the end of the reference generation range is the end of the reference generation range (reference range) whose start point is the current playback position. If it is determined that the index value at the end of the reference generation range is 0 (step S441; YES), the control unit 11 determines the end of the reference generation range as the end of the generation range of the display information (step S451), and proceeds to step S45 in Fig. 8.

[0057] On the other hand, if it is determined that the index value of the end of the reference generation range is not 0 (step S441; NO), the control unit 11 stores the end of the reference generation range as a candidate end of the display information generation range and stores its position and the index value of that position in RAM (step S442).

[0058] Next, the control unit 11 determines the upper limit that can be set as the generation range of the display information based on the generation cycle and the current tempo (step S442). The upper limit that can be set as the generation range is determined based on the generation cycle. For example, if the current tempo (BPM) is 150, the number of beats per second is 150 beats / 60 sec = 2.5 beats / sec, that is, 2.5 beats per second. If the generation cycle is 1.0 second, the number of beats per generation cycle is 2.5 beats, and the playback position advances by 2.5 beats in one generation cycle. Therefore, the upper limit that can be set as the generation range is "current playback position + reference generation range + 2.5 beats," and the generation range is set to be wider than the display range of the piano roll 30 as a single still image actually displayed on the display unit 14.

[0059] Next, the control unit 11 moves to the next performance information position in the index value table (step S444), and determines whether the current performance information position in the index value table has exceeded the upper limit (step S445). If it is determined that the current performance information position in the index value table has not exceeded the upper limit (step S445; NO), the control unit 11 determines whether the index value of the current performance information position is 0 (step S446). If it is determined that the index value of the current performance information position is 0 (step S446; YES), the control unit 11 determines the current performance information position as the end of the generation range (step S447), and moves to step S45 of FIG. 8.

[0060] If the control unit 11 determines that the index value of the current performance information position is not 0 (step S446; NO), it determines whether or not the index value of the current performance information position is smaller than the index value of the end candidate (step S448). If the control unit 11 determines that the index value of the current performance information position is smaller than the index value of the end candidate (step S448; YES), it overwrites the position and index value of the end candidate stored in RAM with the current performance information position as the end candidate (step S449), and returns to step S444. If the control unit 11 determines that the index value of the current performance information position is equal to or larger than the index value of the end candidate (step S448; NO), it returns to step S444.

[0061] In step S445, if it is determined that the current performance information position has exceeded the upper limit of the generation range (step S445; YES), the control unit 11 determines the end candidate as the end of the generation range (step S450), and proceeds to step S45 in FIG.

[0062] For example, as shown in Fig. 10, if the index value at the end of the reference generating range is +2 and the index values ​​for every 0.5 beat thereafter are +2, +0, +2, +2, ..., the end of the generating range will be one beat after the end of the reference generating range.Also, as shown in Fig. 11, if the index value at the end of the reference generating range is +1 and the index values ​​for every 0.5 beat thereafter are +2, +2, +2, +2, +1, ..., the end of the generating range will be the end of the reference generating range.

[0063] Returning to FIG. 8, the control unit 11 deletes the display information (see FIG. 13) that is registered in the display buffer and that is outside the display range of the piano roll 30 on the display unit 14 (step S45).

[0064] Next, the control unit 11 acquires the first piece of performance information of the generation range determined in the generation range update process, the first piece of performance information for which display information has not yet been registered in the display buffer (step S46). Next, the control unit 11 determines whether the acquired performance information extends beyond the end of the generation range (step S47). If it is determined that the acquired information does not extend beyond the end of the generation range (step S47; NO), the control unit 11 generates display information based on the acquired performance information and registers it in the display buffer (step S48).

[0065] As shown in FIG. 12 , the display information includes an object number, an object type, color information, display position information x, display position information y, width, length, and the like. The object number is an ID for identifying the display information. The object type is information indicating the type of display information, such as note, bar line, or pedal. The color information is information for specifying the color of the display information. The color information is used, for example, to color-code the right-hand part and the left-hand part. The display position information x is a value indicating the x-coordinate of the display position in the horizontal direction S of the display screen of an object displayed based on the display information. For example, if the acquired performance information is note information, the display position information x is determined based on the note number of the note information. The display position information y is a value indicating the y-coordinate of the display position in the vertical direction L of the display screen of an object specified by the display information. For example, if the performance information is note information, the display position information y is determined based on the onset time of the note information and the tick of the current playback position. The width is information indicating the width of the object displayed based on the display information. For example, if the acquired performance information is note information, the width is the width of the bar 31. The length is information indicating the length of the display information. For example, if the acquired performance information is note information, the length indicates the length of the bar 31 determined based on the length of the note.

[0066] Next, the control unit 11 acquires the next performance information (step S49) and returns to step S47. The control unit 11 repeatedly executes steps S47 to S49 until the acquired performance information exceeds the end of the generation range.

[0067] If it is determined in step S47 that the acquired performance information has exceeded the end of the generation range (step S47; YES), the control unit 11 adjusts the generation cycle (step S50) and returns to step S42. In step S50, the control unit 11 adjusts the generation cycle of the display information to the set generation cycle, for example, by waiting until the elapsed time from the start of step S42 in the display information generation process becomes the generation cycle.

[0068] The generation cycle may be changed in response to changes in tempo information during playback. For example, the average tempo for each measure may be tabulated, and when the tempo information changes by a predetermined value or more (for example, by about 30) from a predetermined reference value, the generation cycle may be updated to match the tempo information.

[0069] In step S43, if it is determined that the current playback position has reached the end of the music (step S43; YES), the control unit 11 ends the display information generation process. In the display information generation process, when an object (e.g., bar 31) displayed in response to playback moves (scrolls) within the display range of the piano roll 30 displayed on the display unit 14, the display information generation range is set to include display information for the object (e.g., bar 31) that should be displayed within the display range of the piano roll 30 by the next generation cycle. The display information for the generation range is generated and registered in the display buffer. The left side of FIG. 13 shows a display information generation range A that includes the display range of the piano roll 30 at a certain point in time (e.g., the start of playback). The right side of FIG. 13 shows a previous generation range A, corresponding to the generation range A on the left side of FIG. 13, for the piano roll 30 after a generation cycle has elapsed from the certain point in time, and a display information generation range C, corresponding to the previous generation range A and updated based on the index value table, above the previous generation range A. The previous generation range A includes a deletion range D that is deleted from the current display buffer over time, a portion of the current generation range B located within the display range, and another portion of the current generation range B located above the display range. In other words, the generation range A and the generation range C correspond to the current generation range B and the deletion range D. As shown in Fig. 13, when a generation cycle has elapsed, display information that has scrolled downward from the display range is deleted from the display buffer, and display information that has not yet been generated within the current generation range B is generated. More specifically, the generation range C of display information corresponding to one index value table is a range that combines one line along the horizontal direction S with multiple lines along the vertical direction L, and the number of lines is determined by the addition of the reference generation range and index values ​​corresponding to the tempo information described above. Note that the position of the boundary (N+1), which marks the end of generation range C after a generation cycle has elapsed from a certain point on the right side of Fig. 13, follows the generation range update process shown in Fig. 9, and therefore does not necessarily have to coincide with the position of boundary N at a certain point on the left side of Fig. 13, which is determined by the end of the previous generation range.

[0070] Next, a specific display process in the thread of the display process started in step S5 will be described with reference to Fig. 14. The display process is a process that calculates how far the playback position has advanced since the previous display cycle, updates the display position information y of all display information in the display buffer based on the calculation result, and updates the display of the piano roll 30 displayed on the display unit 14. This process is always executed during playback after playback is started by operating the input unit 13 (step S6; YES).

[0071] 14, in the display process, the control unit 11 first acquires tick information of the current playback position (step S61), and determines whether it is the end of the music (step S62). If it is determined that the current playback position is not the end of the music (step S62; NO), the control unit 11 acquires display information of the display buffer (step S63). The control unit 11 acquires the display buffers registered in the display buffer in order from the top.

[0072] Next, the control unit 11 determines whether updating of the display positions of all display information in the display buffer has been completed (step S64). For example, if the control unit 11 was able to acquire display information from the display buffer, it determines that updating of the display positions of all display information has not been completed. If the control unit 11 was unable to acquire display information from the display buffer, it determines that updating of the display positions of all display information has been completed.

[0073] If it is determined that updating of the display positions for all display information in the display buffer has not been completed (step S64; NO), the control unit 11 updates the display position information y of the acquired display information based on the difference between the tick information of the current playback position and the tick information of the previous playback position (step S65). Next, the control unit 11 acquires the next display information registered in the display buffer (step S66) and returns to step S64.

[0074] In step S64, when it is determined that updating of the display positions for all display information in the display buffer has been completed (step S64; YES), the control unit 11 updates the display of the piano roll 30 on the display unit 14 (step S67). That is, the control unit 11 displays the display information having the display position information x and the display position information y within the display range of the piano roll 30 on the display unit 14 at the corresponding display positions. This allows scrolling of the piano roll 30.

[0075] Next, the control unit 11 adjusts the display cycle (step S68). In step S68, for example, when the frame rate is 30 fps, the control unit 11 waits for the transition to the next frame until the elapsed time from the start of step S61 reaches a predetermined display cycle (e.g., 33 ms), thereby adjusting the display cycle of the piano roll 30 to the set display cycle. In addition, the control unit 11 stores the tick information of the current playback position in RAM as the tick information of the previous playback position.

[0076] When the adjustment of the display cycle is completed, the control unit 11 returns to step S61. The control unit 11 repeats the processes of steps S61 to S68 until it determines in step S62 that the playback position has reached the end of the song. If it determines in step S62 that the playback position has reached the end of the song (step S62; YES), the control unit 11 ends the display process.

[0077] In the playback process initiated in step S7 along with the display of the piano roll 30 controlled by the display process, the control unit 11 plays the selected music piece based on the performance information of the selected music piece. For example, the control unit 11 executes the following process at a predetermined cycle (e.g., 1 ms). That is, the control unit 11 converts the elapsed time since the start of the playback process into ticks based on the current tempo information in the performance information of the selected music piece, and stores this in RAM as tick information of the current playback position. In the display information generation process and display process, the control unit 11 acquires this information as the current playback position. The control unit 11 also determines whether there has been a tempo change based on the performance information of the music piece, and if it determines that there has been a tempo change, it updates the current tempo information in RAM. The control unit 11 also determines whether there has been a beat change based on the performance information, and if it determines that there has been a beat change, it updates the current beat information in RAM. The control unit 11 also determines whether a note-on timing has been reached based on the performance information, and if it determines that the note-on timing has been reached, it plays the note that corresponds to the note-on timing. That is, based on the note information of the note, the control unit 11 causes the output unit 27 of the electronic musical instrument 2 to output the note via the communication unit 15. Furthermore, when the control unit 11 determines that the note-off timing has been reached based on the performance information, the control unit 11 causes the output unit 27 to mute the note that has reached the note-off timing via the communication unit 15.

[0078] The control unit 11 repeats the processes of steps S2 to S9 until an instruction to close the piano roll display application 121 is input by operating the input unit 13. When an instruction to close the piano roll display application 121 is input by operating the input unit 13, the piano roll display process ends.

[0079] As described above, the control unit 11 of the information processing device 1 generates display information for each generation cycle based on the performance information, for displaying a piano roll on the display unit 14 in response to the playback of a piece of music based on a series of performance information. In this process, the control unit 11 determines the end of the generation range of the display information for the piece of music for each generation cycle based on the note distribution in the piece of music. This makes it possible to avoid setting the end of the generation range at a point where the note distribution is inappropriate, thereby reducing the impact of improper note display in the piano roll. For example, this reduces improper display, such as when bars 31 representing two notes are combined and displayed as if they were one note, preventing the user from correctly recognizing the note length.

[0080] For example, the control unit 11 determines whether or not the note distribution satisfies a predetermined condition at each position at a predetermined time interval (e.g., every 1 / 2 beat) in at least a part of the music piece based on the performance information, and determines the end of the range for generating display information based on the determination result. Therefore, the control unit 11 can determine the end of the range for generating display information based on whether the note distribution at each position at a predetermined time interval satisfies a predetermined condition.

[0081] For example, the control unit 11 determines whether the note distribution satisfies a plurality of predetermined conditions at each position at a predetermined time interval (e.g., every half beat) in at least a portion of the music piece based on the performance information, calculates an index value indicating the degree to which each of the positions is suitable as a boundary of the display information generation range based on the determination result, and determines the end of the range based on the calculated index value. Thus, the control unit 11 can determine the point suitable as a boundary of the display information generation range as the end of the generation range.

[0082] For example, the control unit 11 determines for each generation cycle whether the index value calculated for the end position of the reference generation range, which has the current playback position of the music as its starting point, is an index value that indicates the highest degree of suitability for the boundary of the generation range of the display information, and if it determines that the index value indicates the highest degree, it determines the end of the reference generation range as the end of the generation range. Therefore, if the end of the reference generation range is suitable for the boundary of the generation range of the display information, it can determine the end of the reference generation range as the end of the generation range.

[0083] Furthermore, for example, the control unit 11 determines the position after the end of the reference generation range that has a higher index value than the index value calculated for the end position of the reference generation range and is closest to the end of the reference generation range as the end of the generation range of the display information. Therefore, if the end of the reference generation range is not suitable as the boundary of the generation range of the display information, another suitable location can be determined as the end of the generation range.

[0084] Furthermore, the control unit 11 determines the reference generation range based on the generation cycle of the display information, and therefore can determine the reference generation range according to the generation cycle.

[0085] Furthermore, the control unit 11 determines the generation cycle based on the tempo of the music, and therefore, the display information can be generated at a generation cycle that corresponds to the tempo of the music.

[0086] The contents of the above embodiment and modifications are preferred examples of the display information generating method, program, information processing device, electronic musical instrument, and electronic musical instrument system according to the present invention, and are not limited to these.

[0087] For example, in the above embodiment, the information processing device 1 and the electronic musical instrument 2 are separate entities. However, the functions of the present invention implemented by the control unit 11 of the information processing device 1 may be provided in the electronic musical instrument 2. The piano roll 30 may be displayed on the display unit 25 of the electronic musical instrument 2. While the above embodiment describes an information processing device for displaying the piano roll 30, the information processing device may also be applied to displaying performances of musical instruments other than the piano (e.g., percussion instruments, wind instruments). The playback (sound generation) of musical pieces may be performed by the output unit 16 of the information processing device 1. While the above embodiment uses musical piece performance information as feature information, necessary feature information may be extracted from images in other applications, such as tapping controls in accordance with a time-varying display, instead of displaying musical piece performances, and the range for generating display information may be adjusted to be variable. In the above embodiment, the process proceeds to step S4 after the tempo information analysis process (step S31) and the note information analysis process (step S32) are completed. However, the playback process may be performed in parallel with the tempo information analysis process (step S31) and the note information analysis process (step S32). In this case, just before the display process, an index value table is created according to the tempo information and note information for only the newly generated generation range, an image for the newly generated generation range is generated according to the created index value table, and then the display process for the image for the newly generated generation range is executed. By performing such processes in parallel, the index value table for one piece of music can be subdivided and processed in stages, thereby reducing memory storage capacity and processing speed. Furthermore, in the above embodiment, the information processing device 1 automatically plays the musical piece according to the images on the piano roll 30 so that the musical piece is played automatically without the user (performer) performing normal playing operations on the keyboard 23 of the electronic musical instrument 2. However, instead, the piano roll 30 may be displayed to assist the user in playing the musical piece without automatic playing, and the user may generate the musical piece's playing sounds by performing normal playing operations on the keyboard 23 of the electronic musical instrument 2 at timings that match the images while checking the images on the piano roll 30.

[0088] Furthermore, the method of calculating the index value in the above embodiment is merely an example, and is not limited to this. For example, the index value may be calculated so that it is larger as it becomes more suitable for the boundary of the generation range.

[0089] In the above embodiment, an example has been disclosed in which a semiconductor memory such as a ROM or a hard disk is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. Other computer-readable media may include a hard disk, an SSD, and a portable recording medium such as a CD-ROM. Furthermore, a carrier wave may also be used as a medium for providing data for the program according to the present invention via a communication line.

[0090] Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the above-described embodiments, but is defined by the claims. Furthermore, the technical scope of the present invention also includes equivalents to the claims that are not related to the essence of the present invention. [Explanation of symbols]

[0091] 100 Electronic musical instrument system, 1 Information processing device, 11 Control unit, 12 Memory unit, 121 Piano roll display application, 13 Input unit, 14 Display unit, 15 Communication unit, 16 Output unit, 17 Bass, 2 Electronic musical instrument, 21 Control unit, 22 Memory unit, 23 Keyboard, 24 Input unit, 25 Display unit, 26 Communication unit, 27 Output unit, 28 Pedal, 29 Bass

Claims

1. The computer a generating step of dividing and generating display information for displaying an image based on the feature information on a display unit; the generating step is controlled so that a range for generating the divided display information is varied in accordance with the feature information. Display information generation method.

2. The characteristic information is performance information of the music piece. The display information generating method according to claim 1 .

3. the generating step determines whether or not a note distribution satisfies a predetermined condition for each position at a predetermined time interval in at least a portion of the music piece based on the performance information, and determines the range based on the determination result. The display information generating method according to claim 2 .

4. The generating step determines whether or not a note distribution satisfies a plurality of conditions for each position at a predetermined time interval in at least a portion of the musical piece based on the performance information, calculates an index value indicating the degree to which each of the positions is suitable for a boundary of the range based on the determination result, and determines the range based on the calculated index value. The display information generating method according to claim 2 .

5. the generating step determines whether the index value calculated for the end position of a reference range for each of the divided display information is an index value indicating the highest level, and if it is determined that the index value is an index value indicating the highest level, determines the end of the reference range to be an end of the range. The display information generating method according to claim 4 .

6. The generating step determines, as the end of the range, a position after the end of the reference range where the index value is higher than the index value calculated for the end position of the reference range and is closest to the end of the reference range. The display information generating method according to claim 5 .

7. 5. The display information generating method according to claim 4, wherein the plurality of conditions include at least one of the following conditions (1) to (4): (1) It is not a rest period. (2) There is a note of the same pitch immediately before and after (3) There are notes of the same length immediately before and after (4) There are more than the specified number of notes at the same time.

8. The computer determining the reference range based on the tempo of the music piece; The display information generating method according to claim 5 .

9. On the computer, generating display information for displaying an image based on the feature information on a display unit by dividing the display information into a plurality of parts; a range for generating the divided display information is controlled so as to be variable in accordance with the feature information; A program for executing a process.

10. a control unit that divides display information into a plurality of parts and generates the display information for displaying an image based on the feature information on a display unit, and controls the range of the divided display information to be generated so as to be variable in accordance with the feature information; Information processing device.

11. The information processing device according to claim 10; an output unit that outputs sound; An electronic musical instrument comprising:

12. The information processing device according to claim 10; an electronic musical instrument having an output unit that outputs sound; An electronic musical instrument system comprising:

Images