Accompaniment generation method, accompaniment generation program, accompaniment generation device, and accompaniment generation system

Abstract

An accompaniment generation method according to the present invention comprises: acquiring musical score image data SD representing a musical score 5; analyzing musical piece information included in the musical score 5 from the acquired musical score image data SD; and generating, on the basis of the analysis result of the musical piece information, an accompaniment to be added to the musical piece represented by the musical score 5. An accompaniment generation program according to the present invention executes the accompaniment generation method on a computer.

Description

Accompaniment generation method, accompaniment generation program, accompaniment generation device, and accompaniment generation system

[0001] The present invention relates to an accompaniment generation method for generating accompaniment for a musical piece, an accompaniment generation program for executing the accompaniment generation method, an accompaniment generation device, and an accompaniment generation system.

[0002] Accompaniment is sometimes used when playing an instrument or singing. For example, musicians can practice playing an instrument along with the accompaniment and enjoy playing an instrument. Similarly, singers and ordinary people can practice singing along with the accompaniment and enjoy singing.

[0003] Patent Document 1 listed below discloses a device that takes a musical score as input, interprets the music, and outputs performance information.

[0004] Patent No. 3177991

[0005] The aim of the above-mentioned patent document 1 is to provide a well-balanced interpretation of the sound based on the dynamic signals read from the musical score. It would be beneficial for users if a service useful for musicians and singers could be provided based on the information read from the musical score.

[0006] The object of the present invention is to provide additional information to a piece of music by interpreting the score, thereby creating added value that cannot be obtained by simply referring to the score.

[0007] An accompaniment generation method according to one aspect of the present invention includes acquiring sheet music image data representing a musical score, analyzing music information contained in the score from the acquired sheet music image data, and generating an accompaniment to be added to the music represented by the sheet music based on the analysis results of the music information.

[0008] An accompaniment generation program according to another aspect of the present invention is a program that causes a computer to execute an accompaniment generation method, which includes acquiring sheet music image data representing a musical score, analyzing musical information contained in the musical score from the acquired sheet music image data, and generating an accompaniment to be added to the musical piece represented by the sheet music based on the analysis results of the musical information.

[0009] An accompaniment generation device according to another aspect of the present invention includes a score image data acquisition unit that acquires score image data representing a score, a music information analysis unit that analyzes music information contained in the score from the acquired score image data, and an accompaniment generation unit that generates accompaniment to be added to the music represented by the score based on the analysis results of the music information.

[0010] According to the present invention, by interpreting the musical score, additional information is provided to the music piece, thereby creating added value that cannot be obtained by simply referring to the musical score.

[0011] FIG. 1 is a diagram showing a state in which a user is playing an electronic musical instrument to an accompaniment generated by an accompaniment generation method according to the present embodiment. FIG. 2 is a block diagram of an accompaniment generation device according to the embodiment. FIG. 3 is a functional block diagram of an accompaniment generation device according to the embodiment. FIG. 4 is a diagram showing the concept and data structure of accompaniment style data. FIG. 5 is a flowchart showing an accompaniment generation method according to the embodiment. FIG. 6 is a flowchart showing an accompaniment generation method according to the embodiment. FIG. 7 is a flowchart showing an accompaniment generation method according to the embodiment. FIG. 8 is a functional block diagram of an accompaniment generation device according to Variation 1.

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An accompaniment generation method, an accompaniment generation program, an accompaniment generation device, and an accompaniment generation system according to embodiments of the present invention will now be described with reference to the accompanying drawings.

[0013] {1. Method of Using the Accompaniment Generation Device} Fig. 1 is a diagram showing a user 4 playing an electronic musical instrument 1 along with an accompaniment generated by an accompaniment generation method according to this embodiment. The user 4 imports a musical score 5 into the electronic musical instrument 1 in advance as image data, which causes the electronic musical instrument 1 to automatically generate accompaniment data. The user 4 plays the electronic musical instrument 1 while playing back the accompaniment data. The user 4 can enjoy playing the electronic musical instrument 1 along with the accompaniment. The user 4 can enjoy playing like a piano version of karaoke.

[0014] 2. Configuration of the Electronic Musical Instrument} FIG. 2 is a block diagram of an electronic musical instrument 1 equipped with an accompaniment generation device 200 according to this embodiment. In this embodiment, the electronic musical instrument 1 is described as an electronic keyboard instrument. The electronic musical instrument 1 according to this embodiment has the capability of reading and playing back MIDI (Musical Instrument Digital Interface) data. As shown in FIG. 2 , the electronic musical instrument 1 includes performance controls 11, a storage device 12, a sound source 13, a sound system 14, a control unit 2, a user interface 3, and an external interface 15. The control unit 2 includes a CPU (Central Processing Unit) 21, a RAM (Random Access Memory) 22, and a ROM (Read Only Memory) 23. The user interface 3 includes controls 31 and a display 32. The control unit 2 functions as the accompaniment generation device 200 according to this embodiment.

[0015] The performance controller 11 includes a keyboard 111 and is connected to the bus 20. The keyboard 111 has an arrangement of multiple keys. The keyboard 111 may have, for example, 61 or 88 keys. The keyboard 111 of the performance controller 11 may be an image of a keyboard displayed on the screen of a touch panel display, which will be described later.

[0016] The storage device 12 includes a storage medium such as a hard disk, optical disk, magnetic disk, or memory card. This storage device 12 stores computer programs such as a program PG, music score image data SD, accompaniment style data ASD, identification information ID, accompaniment data AD, and control programs. The program PG is a program for executing the accompaniment generation process according to the present invention.

[0017] The CPU 21, RAM 22, and ROM 23 are connected via a bus 20. The RAM 22 is, for example, a volatile memory, and is used as a work area for the CPU 21 and temporarily stores various data. The ROM 23 is, for example, a non-volatile memory, and stores various programs, setting data, etc. The CPU 21 executes programs stored in the storage device 12 or the ROM 23 while using the RAM 22 as a work area.

[0018] The sound source 13 is connected to the bus 20 and outputs audio data (sound signals) based on pitches specified by operating the keyboard 111. The audio data is sampling data (e.g., PCM (Pulse Code Modulation) data) that indicates the waveform of a sound. Hereinafter, the audio data output by the sound source 13 will be referred to as "sound signals." The sound source 13 stores sound signals for all pitches in advance. The sound system 14 includes a digital-to-analog (D / A) conversion circuit, an amplifier, and a speaker. The sound system 14 converts the sound signals provided by the sound source 13 into analog sound signals and generates sounds based on the analog sound signals.

[0019] The controls 31 include on / off switches, rotary switches, slide switches, and the like, and are connected to the bus 20. The controls 31 are used to perform various settings, including power on / off, volume adjustment, and settings for applying various effects to the audio signals output from the electronic musical instrument 1. The controls 31 may also be used for performance operations. The display 32 includes, for example, a liquid crystal display and is connected to the bus 20. The display 32 displays the title of a piece of music, musical scores, or various other information. In this embodiment, the display 32 also displays operation menus for specifying a method for analyzing music information, an operation menu for specifying an accompaniment style, and the like. The display 32 may be a touch panel display. In this case, the keyboard 111 or some or all of the controls 31 may be displayed on the display 32. The user 4, who is the performer, can perform various operations by operating the display 32.

[0020] The external interface 15 is an interface for connecting to an external storage medium ED or a network. The CPU 21 can access a storage medium ED such as a CD-ROM, DVD, MD, or USB storage memory via the external interface 15.

[0021] {3. Configuration of the Accompaniment Generation Device} Next, the functions of the accompaniment generation device 200 of this embodiment will be described. FIG. 3 is a block diagram showing the functional configuration of the accompaniment generation device 200 according to this embodiment. The accompaniment generation device 200 is realized by the CPU 21 executing the program PG while using hardware resources such as the RAM 22. As shown in FIG. 3, the accompaniment generation device 200 includes a score image data acquisition unit 201, a music information analysis unit 202, an accompaniment generation unit 203, an analysis method reception unit 204, and an accompaniment style setting unit 205. Each of the functional units 201 to 205 is realized by the CPU 21 executing the program PG while using hardware resources such as the RAM 22.

[0022] The sheet music image data acquisition unit 201 acquires sheet music image data SD stored in the storage device 12. The sheet music image data SD is, for example, image data such as PDF (Portable Document Format). The user stores the sheet music image data SD in the storage device 12 by scanning a sheet music (musical score) in advance, or based on electronically distributed sheet music data. For example, sheet music image data SD obtained by converting an SMF (Standard MIDI File) into an electronic musical score may be used. Alternatively, the sheet music image data acquisition unit 201 may acquire the sheet music image data SD based on a sheet music read by a scanner connected via the external interface 15.

[0023] The music information analysis unit 202 analyzes music information from the score image data SD acquired by the score image data acquisition unit 201. The music information analysis unit 202 acquires information (text information, note information) about the music score recorded in the score image data SD by performing image analysis of the music score image data SD. The music information analysis unit 202 analyzes the music represented by the music score recorded in the score image data based on the acquired information about the music score.

[0024] The accompaniment generation unit 203 automatically generates accompaniment to be added to the music piece represented by the score recorded in the score image data SD based on the analysis results of the music information analysis unit 202. The accompaniment generation unit 203 stores the generated accompaniment as accompaniment data AD in the storage device 12. The accompaniment generation unit 203 uses accompaniment style data ASD to set the style of the accompaniment to be added to the music piece. The accompaniment style data ASD is data in which the accompaniment style is set for each section. The user can arrange the melody, atmosphere, etc. of the accompaniment by applying various types of accompaniment style data ASD to the accompaniment data AD. The accompaniment style data ASD will be described in detail later. In this embodiment, the accompaniment data AD is composed of SMF (Standard MIDI File) and control data for controlling the accompaniment style data ASD.

[0025] The analysis method receiving unit 204 specifies the analysis method for the music information in the music information analysis unit 202. The analysis method receiving unit 204 receives the analysis method specification from the user via the user interface 3 (the operators 31 or the display 32 functioning as a touch panel). The analysis method receiving unit 204, for example, displays an analysis method selection menu on the display 32 and receives the analysis method for the music information in response to a user operation using the operators 31 or the display 32. The analysis method receiving unit 204 provides the received analysis method specification information to the music information analysis unit 202. Note that the accompaniment generation device 200 may not be configured to include the analysis method receiving unit 204. Alternatively, the accompaniment generation device 200 may be configured without the analysis method receiving unit 204 if a single analysis method is used or if the analysis method is automatically set.

[0026] The accompaniment style setting unit 205 sets the accompaniment style data ASD to be used by the accompaniment generation unit 203. The accompaniment style setting unit 205 accepts an accompaniment style designation operation from the user via the user interface 3 (the operators 31 or the display 32 functioning as a touch panel). The accompaniment style setting unit 205, for example, displays a list menu of accompaniment styles on the display 32 and accepts the designation of the accompaniment style in response to a user operation using the operators 31 or the display 32. The accompaniment style setting unit 205 provides the accepted accompaniment style designation information (designation information of the accompaniment style data ASD) to the accompaniment generation unit 203.

[0027] {4. Accompaniment Style Data} Fig. 4 is a diagram illustrating the concept and data structure of accompaniment style data ASD. Fig. 4 shows an example of accompaniment style data ASD. The accompaniment style data ASD is a style to be applied to each section of a piece of music in the time direction, and defines elements that change over time. As shown in Fig. 4, one or more accompaniment style data ASD are prepared for each category, such as jazz or rock. Such categories may be arranged hierarchically. For example, big band and modern jazz may be provided as sub-categories of jazz.

[0028] Each accompaniment style data ASD in FIG. 4 includes accompaniment element data for an "intro" section, a "main" section, a "fill-in" section, and an "ending" section. A section represents a portion of a musical piece in the time direction. The accompaniment element data for each section includes multiple variations of accompaniment element data with different atmospheres or levels of excitement. The "intro" section, "main" section, "fill-in" section, and "ending" section are represented by the letters "I," "M," "F," and "E," respectively. Furthermore, the variations of the "intro," "main," and "ending" sections are represented by letters such as "A" (normal (quiet)), "B" (slightly flashy), "C" (flashy), and "D" (very flashy), depending on the atmosphere or level of excitement.

[0029] Because "fill-in" sections are used as transitions between other sections, their variations are represented by a combination of two letters that correspond to the change in mood or intensity between the preceding and following sections. For example, "AC" corresponds to a change from "quiet" to "flashy." In this case, it would be natural for the variation of the preceding section to be "A" and the variation of the following section to be "C," but the choice of variation is arbitrary.

[0030] Each accompaniment element data is represented by a combination of an alphabet representing the type of section and an alphabet representing the variation. For example, the section of accompaniment element data MA is "main" and the variation is "A." The section of accompaniment element data FAB is "fill-in" and the variation is "AB."

[0031] Each accompaniment element data includes accompaniment pattern data for multiple tracks (accompaniment parts), as well as reference chord information and pitch conversion rules (pitch conversion table information, range, re-sounding rules when changing chords, etc.). The accompaniment pattern data is a string of notes in MIDI format or phrase data in audio format, and can be converted to any pitch based on the reference chord information and pitch conversion rules. Therefore, multiple accompaniment pattern data can be used to generate accompaniment data AD to be added to a song.

[0032] {5. Accompaniment Generation Processing Method} Next, an accompaniment generation method according to this embodiment will be described. Figures 5 to 8 are flowcharts showing the accompaniment generation method according to this embodiment. The flowcharts shown in Figures 5 to 8 show processing that is realized by the CPU 21 executing the program PG while utilizing hardware resources such as the RAM 22. In other words, the flowcharts shown in Figures 5 to 8 show processing that is executed by the accompaniment generation device 200, which includes the score image data acquisition unit 201, music information analysis unit 202, accompaniment generation unit 203, analysis method acceptance unit 204, and accompaniment style setting unit 205 shown in Figure 3.

[0033] 5 begins, the user stores sheet music image data SD in the storage device 12. For example, the user creates sheet music image data SD by scanning sheet music that the user owns, and stores the sheet music image data SD in the storage device 12. Alternatively, the user may obtain electronic sheet music data from a website or the like, and store the electronic sheet music data in the storage device 12 as sheet music image data SD by converting the electronic sheet music data or the like.

[0034] In step S1 , the sheet music image data acquisition unit 201 acquires sheet music image data SD representing a sheet music stored in the storage device 12 .

[0035] In step S2, the music information analysis unit 202 analyzes music information contained in the score from the score image data SD acquired by the score image data acquisition unit 201. For example, the music information analysis unit 202 analyzes music information by acquiring information such as the music's notes, time signature, BPM (beats per minute), repeats, volume markings, double lines, or rehearsal marks from the score image data SD. Music information is information for playing the music represented on the score according to the score, and includes information such as scale, tempo, dynamics, and repetition. The music information analysis unit 202 also acquires an identification information ID that identifies the music based on the music name or music information acquired from the score image data SD. For example, the music information analysis unit 202 acquires the identification information ID by connecting to a server on the Internet based on the analyzed information. It may not be possible to acquire the music's identification information ID, so acquiring the identification information ID is optional.

[0036] Methods for obtaining the identification information ID of a song include a method of outputting the song (MIDI, Audio) by interpreting the music based on the sheet music, connecting to a server on the Internet and obtaining the identification information ID based on the song output, a method of connecting to a server on the Internet and obtaining the identification information ID based on hash (one-way function) data of the song output, and a method of connecting to a server on the Internet and obtaining the identification information ID based on hash (one-way function) data of the sheet music image data SD.

[0037] 6 is a flowchart showing the details of step S2 (music information analysis processing). First, in step S21, the music information analysis unit 202 determines whether or not an analysis method has been specified. The user can specify the analysis method using the user interface 3 (the controls 31 or the display 32 functioning as a touch panel). The user specifies an analysis method such as beat or BPM, for example. The music information analysis unit 202 may, for example, display an analysis method selection menu on the display 32, allowing the user to select a preferred analysis method.

[0038] If the designation of the analysis method is accepted in step S21, the music information analysis unit 202 executes analysis processing in accordance with the accepted analysis method in step S22.

[0039] If no designation of an analysis method is accepted in step S21, the music information analysis unit 202 executes a default analysis process in step S23. Alternatively, the analysis method may be determined based on the acquired identification information ID of the music. For example, an analysis method corresponding to the identification information ID may be acquired by connecting to a server on the Internet. The server may store statistical information on which analysis method other users have used for each identification information ID. The analysis method corresponding to the identification information ID acquired from the server may be set as the default analysis method. In this case, even if the user does not designate an analysis method, an appropriate analysis method for the music is selected, which is highly convenient.

[0040] Next, in step S24, the music information analysis unit 202 provides the analysis result of the music information, which is the result of the analysis process, to the accompaniment generation unit 203.

[0041] Referring again to Fig. 5, in step S3, the accompaniment generation unit 203 generates accompaniment to be added to the music represented by the score based on the analysis results of the music information. As described above, the accompaniment generation unit 203 generates accompaniment data AD consisting of data in SMF format and control data for controlling the accompaniment style data ASD.

[0042] 7 is a flowchart showing the details of step S3 (accompaniment generation processing). First, in step S31, the accompaniment generation unit 203 generates SMF format data constituting the accompaniment data AD based on the analysis results of the music information obtained from the music information analysis unit 202. Next, in step S32, the accompaniment generation unit 203 determines the accompaniment style data ASD to be applied to the accompaniment.

[0043] The accompaniment generation unit 203 applies to the accompaniment an accompaniment style that changes over time, which is a style that is applied to each section in the time direction of the music piece. This changes the style for each section, preventing the accompaniment from becoming monotonous. Alternatively, the accompaniment generation unit 203 also determines which of the types (A, B, ...) for each section described with reference to FIG. 4 should be assigned to each style. Note that the processing of steps S31 and S32 in FIG. 7 may be reversed. The type (A, B, ...) is set (determined inflection, excitement level) based on criteria corresponding to the style (or different criteria).

[0044] 8 is a flowchart showing the details of the process of step S32 (deciding accompaniment style data). First, in step S321, the accompaniment generating section 203 determines whether or not the identification information ID of the music piece has been given.

[0045] If it is determined in step S321 that the identification information ID of the song has been provided, then in step S322 the accompaniment generation unit 203 determines the accompaniment style data ASD to be applied to the accompaniment based on the identification information ID. For example, the accompaniment generation unit 203 acquires information about the song from a server on the Internet based on the identification information ID, and determines the accompaniment style data ASD to be applied based on the acquired information. Alternatively, the accompaniment generation unit 203 determines the accompaniment style data ASD based on the identification information ID and recommended information stored on the server.

[0046] As a method for determining the accompaniment style data ASD based on the identification information ID, the accompaniment generation unit 203 may present to the user multiple accompaniment style data ASD based on recommendation information stored in the server. In this case, the accompaniment style is determined by the user selecting their preferred accompaniment style data ASD from the multiple pieces of recommended information. Alternatively, the recommended information stored in the server may be popularity information from other users. The accompaniment generation unit 203 may present a popularity ranking of styles selected by other users for songs with the same identification information ID.

[0047] If it is determined in step S321 that the identification information ID of the music piece has not been given, then in step S323, the accompaniment generating unit 203 determines whether or not the user has selected an accompaniment style.

[0048] If it is determined in step S323 that the user has selected an accompaniment style, the accompaniment generation unit 203 applies the selected accompaniment style to the accompaniment in step S324. As described above, the accompaniment style setting unit 205 accepts the user's specification of the accompaniment style data ASD via the user interface 3. This allows the user to apply an accompaniment style that suits their preferences. The accompaniment style setting unit 205 also accepts a change of accompaniment style via the user interface 3. If the user wishes to change the style of the generated accompaniment, they can instruct the user to change the accompaniment style via the user interface 3. For example, even if the accompaniment style data ASD is determined based on the identification information ID in step S322, the accompaniment style can be changed according to the user's preferences.

[0049] If it is determined in step S323 that the user has not selected an accompaniment style, then in step S325 the accompaniment generation unit 203 sets an accompaniment style to be applied to the accompaniment based on the analysis results of the music information acquired from the music information analysis unit 202. Alternatively, the accompaniment generation unit 203 applies a preset standard accompaniment style to the accompaniment.

[0050] The accompaniment generation unit 203 includes in the accompaniment data AD data that controls the accompaniment style data ASD determined in step S322, S324, or S325, and stores the generated accompaniment data AD (SMF data and data that controls the accompaniment style data ASD) in the storage device 12.

[0051] In this manner, the accompaniment generation device 200, accompaniment generation method, and accompaniment generation program of the present embodiment automatically generate an accompaniment for a musical piece represented by the musical score 5 based on the musical score image data SD that visualizes the musical score 5. The user 4 can enjoy playing the electronic musical instrument 1 while playing back the generated accompaniment. The user 4 can use the accompaniment to practice playing the electronic musical instrument 1. The user 4 can enjoy playing the electronic musical instrument 1 in accordance with the accompaniment while listening to the accompaniment. For example, the user can enjoy playing a rich accompaniment even with a score without accompaniment (such as a guitar riff score or a score for singing and playing), a TAB score, or a score with a simple accompaniment. The accompaniment generation device 200 of the present embodiment can add new value to the electronic musical instrument 1. The accompaniment generated in this embodiment includes various musical pieces that are matched to the performance of the electronic musical instrument 1. For example, a main melody with different climaxes depending on the style may be used as the accompaniment. Furthermore, the accompaniment generated in this embodiment is intended to be generated over the entire time span of a song. For example, the accompaniment is intended to be generated in a continuous manner from near the beginning of the song (near the intro) to near the final measure (near the ending) (of course, when we say "overall," it also includes parts that include silent sections).

[0052] {6. Modifications} (Modification 1) Next, Modification 1 of the present embodiment will be described. Fig. 9 is a functional block diagram of an accompaniment generation device 200 according to Modification 1. As shown in Fig. 9, the accompaniment generation device 200 according to Modification 1 includes a candidate style presenting unit 206 in addition to the components shown in Fig. 3. The components other than the candidate style presenting unit 206 are the same as those described above. The candidate style presenting unit 206 is a functional unit that presents, as candidate styles, accompaniment styles suited to a piece of music represented by a musical score.

[0053] The candidate style presentation unit 206 presents candidate styles whose main meter matches the main meter of the song based on the analysis results of the song information. The song information analysis unit 202 acquires the main meter of the song by analyzing the score image data SD. For example, the song information analysis unit 202 counts the number of bars for each beat in the song. Then, it determines the beat with the most bars (the longest) as the main meter of the song. The candidate style presentation unit 206 selects only accompaniment styles whose main meter matches the main meter from the multiple accompaniment style data ASD stored in the storage device 12 and presents them to the user. For example, the candidate style presentation unit 206 displays a list menu of candidate styles on the display 32 and accepts the user's designation of the accompaniment style desired by the user in response to a user operation using the controls 31 or the display 32. The user can select a preferred accompaniment style from candidate styles whose main meter matches the main meter, thereby preventing the selection of an accompaniment style that does not match the song. The accompaniment generating section 203 determines the accompaniment style data ASD to be applied based on the accompaniment style selected from the candidate styles.

[0054] (Variation 2) Next, Variation 2 of this embodiment will be described. For example, some pop piano pieces contain odd time signatures. Even when playback is started by synchronizing the SMF and accompaniment style data ASD that make up the accompaniment data AD, the rhythms may not match at the point where an odd time signature occurs. Typically, preset accompaniment styles built into electronic musical instruments have a fixed time signature. Therefore, in Variation 2 of this embodiment, playback is reset (stop and start) every measure. However, resetting every measure results in an accompaniment that feels unchanging and repetitive. Therefore, in Variation 2, preferably, an accompaniment style that matches the main time signature of the piece is applied as a general rule, and the accompaniment style is reset at points where the time signature differs from the main time signature. Note that odd time signatures can be determined from score information or music information.

[0055] (Variation 3) Next, Variation 3 of this embodiment will be described. For example, if an accompaniment based on an accompaniment style is simply looped for a piece of music lasting several minutes, the accompaniment may sound monotonous to the user. Therefore, in Variation 3, the accompaniment generation unit 203 adds section changes and fill-ins to the accompaniment data AD. Even if changes to the accompaniment are made at appropriate times based solely on the passage of time, the accompaniment may end up sounding unnatural and ignoring the original composition and progression of excitement. Therefore, the accompaniment generation unit 203 in Variation 3 acquires information about the compositional divisions of the music and the excitement of the music (information about the intonation of the music) from the score information, and adds appropriate excitement to the accompaniment according to the composition of the music.

[0056] The accompaniment generating unit 203 adds section changes and fill-ins to the accompaniment data AD based on, for example, rehearsal marks, double lines, repeat symbols, or volume symbols.

[0057] Rehearsal marks are obtained from symbols enclosed in a rectangle on the musical score. Rehearsal marks are usually made up of capital letters or the like. Double lines indicate divisions in the musical composition, allowing for the creation of a musical score with a sense of excitement in the accompaniment, in line with the composition intended by the score creator. Repeat symbols also coincide with divisions in the musical composition in many musical pieces, and can therefore be used as points where changes are made to the accompaniment. Volume symbols also coincide with divisions in the musical composition in many musical pieces, and can therefore be used as points where changes are made to the accompaniment. Volume symbols are an example of "information regarding intonation" in this invention. As "information regarding intonation," the accompaniment generation unit 203 may use the number of notes (density) of the singing part.

[0058] (Variation 4) Next, Variation 4 of this embodiment will be described. Variation 4 is an example of a method for specifying the above-mentioned analysis method. Some musical scores are written at a tempo that is double or half the original tempo of a piece of music. For example, there are cases where the tempo is written at a tempo that matches the user's performance level. When accompaniment data AD generated from such a score is played back, an accompaniment is played back at a tempo that differs from the tempo assumed by the original piece of music. Therefore, Variation 4 makes it possible to change the accompaniment tempo to a different tempo, such as double or half, in response to a user's specification.

[0059] For example, if the accompaniment generation unit 203 wants to set the accompaniment style to double speed for the melody, it converts the SMF constituting the accompaniment data AD so that the length of the melody in the time direction is halved. Conversely, if the accompaniment style to half speed for the melody, it converts the SMF constituting the accompaniment data AD so that the length of the melody in the time direction is doubled.

[0060] In Modification 4, the accompaniment generation unit 203 also converts compound time signatures. Compound time signatures are triplet-based time signatures, such as 6 / 8, 9 / 8, and 12 / 8. Analysis of composite time signature scores reveals that when 6 / 8, 9 / 8, and 12 / 8 time signatures are associated with quarter notes, their time lengths are the same as those of triplet, 4.5, and 6 beats, respectively. However, because compound time signatures are based on triplets, using the analysis results as is would not match the accompaniment to a musical piece. Therefore, the accompaniment generation unit 203 converts the melody of the compound time signature section in the SMF that constitutes the accompaniment data AD to 2 / 3 times its original length. If compound time signature accompaniment styles are available, they can be used. However, because many accompaniment styles are created as triplet systems in 4 beats, the conversion function provided by Modification 4 would be convenient.

[0061] (Variation 5) Next, Variation 5 of this embodiment will be described. In Variation 5, the accompaniment generation unit 203 synchronizes the accompaniment in the intro and ending of a piece of music. For example, a simple intro with a playback section of only one measure is set in the accompaniment style data ASD. For example, the same time signature as the time signature of the beginning of the piece of music is applied to this intro. Here, as explained in Variation 4, if the tempo of the accompaniment style is set to double speed or half speed, the speed of the intro section can be converted accordingly.

[0062] The accompaniment style data ASD also includes a simple, standard ending. For example, if a song uses syncopation in its ending, the timing of the song and the accompaniment may not match. Therefore, the accompaniment generation unit 203 resets the accompaniment style at the end of the final measure, and then generates the accompaniment data AD so that the ending is played. When the accompaniment ends in time with the final note, the user is given the impression that the accompaniment has finished playing in sync with the score.

[0063] 7. Other Embodiments In the above embodiment, the data for controlling the accompaniment style is generated as separate data from the SMF. In other embodiments, the data for controlling the accompaniment style may be embedded in the SMF rather than being generated as separate data. For example, the accompaniment style may be embedded in the MIDI data as note information.

[0064] 6, in step S22, the music information analysis unit 202 executes analysis processing according to the analysis method received from the user. In another embodiment, the music information analysis unit 202 may access a server on the Internet and obtain information about the analysis method from the server. For example, the server collects information from a large number of users and stores data about the optimal analysis method.

[0065] In the embodiment described in Fig. 8, in step S322, the accompaniment generation unit 203 determines the accompaniment style to be applied to the accompaniment based on the identification information ID. In other embodiments, the accompaniment generation unit 203 may determine the accompaniment style based on the identification information ID and the tempo of the music obtained from the analysis results of the music. As mentioned above, some musical scores are written at a tempo that is twice or half the original tempo of the music. Therefore, when searching for an accompaniment style using the identification information ID, tempo information may also be used to improve search accuracy.

[0066] 8, in step S323, the accompaniment generation unit 203 uses the accompaniment style selected by the user via the user interface 3. The user interface 3, which allows the user to set or change the accompaniment style, can present the user with a list of accompaniment styles. This list is displayed based on, for example, the preset accompaniment styles stored in the electronic musical instrument 1 connected to the accompaniment generation device 200.

[0067] In the above embodiment, an example has been described in which the user 4, who is an instrument player, plays the electronic musical instrument 1 along with the accompaniment generated by the accompaniment generation method. The user may also sing along with the accompaniment generated by the accompaniment generation method of the present embodiment. The user can practice singing along with the accompaniment or enjoy karaoke along with the accompaniment.

[0068] In the above embodiment, the electronic musical instrument 1 is described as being equipped with the accompaniment generation device 200. In another embodiment, the accompaniment generation device 200 may be equipped in an information terminal such as a tablet. In this case, the user reads the musical score using the tablet. The tablet is connected to the electronic musical instrument, and the accompaniment generated on the tablet is played on the electronic musical instrument. Alternatively, the tablet may function as an electronic musical instrument and play the accompaniment on the tablet. The user can perform musical operations on the virtual musical instrument displayed on the tablet's display.

[0069] {8. Features and Effects of the Embodiment} (1) The accompaniment generation method according to the present embodiment includes acquiring sheet music image data SD representing a sheet music score, analyzing music information contained in the sheet music 5 from the acquired sheet music image data SD, and generating an accompaniment to be added to the music represented by the sheet music 5 based on the analysis results of the music information. An accompaniment not included in the sheet music is generated based on the music information. A user can obtain an accompaniment for the music represented by the sheet music by imaging the sheet music. For example, the user can play while listening to the accompaniment.

[0070] (2) In the above accompaniment generation method, the analysis may include accepting a designation of an analysis method, and the analysis may involve analyzing music information based on the designation. This allows the user to designate a method of analyzing music information, thereby enabling the generation of accompaniment that suits the user's preferences.

[0071] (3) In the above accompaniment generation method, the accompaniment may be generated in a style that is applied to each section of the music based on the analysis of the music information, and the accompaniment may be applied in a time-varying style that changes over time. Because the style changes for each section, the accompaniment does not become monotonous.

[0072] Here, a section is a unit obtained by dividing a song in the time direction, and each unit separated by a section change is processed to be determined as a section such as an intro, main, fill-in, or ending.

[0073] (4) In the above accompaniment generation method, the identification information ID of the music piece represented by the music score 5 is obtained based on the music score image data SD, and the accompaniment is generated by applying an accompaniment style that changes over time to the accompaniment based on the identification information ID, which is a style that is applied to each section of the music piece in the time direction. Since an accompaniment style that suits the music piece is applied, it is possible to generate an accompaniment that matches the music piece.

[0074] (5) The above accompaniment generation method may include presenting a user interface for setting and / or changing the accompaniment style to be applied to the accompaniment. Since the accompaniment style according to the user's preference is applied, it is possible to generate accompaniment that matches the user's taste.

[0075] (6) In the above accompaniment generation method, the method may include presenting candidate styles whose main beats match those of the music piece based on the analysis results of the music piece information, and generating the accompaniment may involve determining the accompaniment style to be applied based on a style selected from the candidate styles. Since the accompaniment style is determined from the candidate styles whose main beats match, an accompaniment style that matches the music piece is applied.

[0076] (7) In the above accompaniment generation method, the accompaniment style may include information about the main time signature of the music piece, and the accompaniment style may be reset in measures of the music piece where the time signature differs from the main time signature. This prevents mismatch between the music piece and the accompaniment style, even for music pieces with irregular time signatures.

[0077] (8) In the above accompaniment generation method, the accompaniment may be generated by adding additional elements to the accompaniment based on information about divisions and / or intonation of the music piece. This can prevent the accompaniment from becoming monotonous.

[0078] (9) In the above accompaniment generating method, the user interface for changing the tempo of the accompaniment may include an interface for changing the tempo of the accompaniment. Since an accompaniment style is applied according to the tempo, it is possible to generate an accompaniment that matches the music piece or that is preferred by the user.

[0079] (10) In the above accompaniment generating method, generating the accompaniment may include synchronizing the accompaniment with the music piece.

[0080] (11) The accompaniment generation program according to this embodiment is a program that causes a computer to execute an accompaniment generation method, which acquires score image data SD representing a score 5, analyzes music information contained in the score 5 from the acquired score image data SD, and generates an accompaniment to be added to the music represented by the score 5 based on the analysis of the music information. An accompaniment that was not included in the score is generated based on the music information. A user can obtain an accompaniment for the music represented by the score by imaging the score.

[0081] (12) The accompaniment generation device 200 according to this embodiment includes a score image data acquisition unit 201 that acquires score image data SD representing a score 5, a music information analysis unit 202 that analyzes music information contained in the score 5 from the acquired score image data SD, and an accompaniment generation unit 203 that generates an accompaniment to be added to the music represented by the score 5 based on the analysis results of the music information. An accompaniment that was not included in the score is generated based on the music information. A user can obtain an accompaniment for the music represented by the score by imaging the score.

[0082] (13) The accompaniment generation system of this embodiment is an accompaniment generation system comprising an information terminal and an electronic musical instrument, wherein the information terminal includes a score image data acquisition unit 201 that acquires score image data SD representing a musical score 5, a music information analysis unit 202 that analyzes music information contained in the musical score 5 from the acquired score image data SD, and an accompaniment generation unit 203 that generates an accompaniment to be added to the music represented by the musical score 5 based on the analysis results of the music information, and the information terminal is connected to the electronic musical instrument, and the accompaniment is played on the electronic musical instrument.

[0083] 1: Electronic musical instrument 3: User interface 4: User 5: Music score 12: Storage device 31: Operator 32: Display 200: Accompaniment generation device 201: Music score image data acquisition unit 202: Music information analysis unit 203: Accompaniment generation unit 204: Analysis method reception unit 205: Accompaniment style setting unit 206: Candidate style presentation unit AD: Accompaniment data ASD: Accompaniment style data PG: Program SD: Music score image data

Claims

1. A method for generating accompaniment, comprising: acquiring sheet music image data representing a sheet music; analyzing music information contained in the sheet music from the acquired sheet music image data; and generating accompaniment to be added to the music represented by the sheet music based on the analysis results of the music information.

2. An accompaniment generating method according to claim 1, further comprising accepting a designation of an analysis method, wherein said analyzing comprises analyzing said music information based on the designation accepted.

3. The accompaniment generation method described in claim 1, wherein the generation of the accompaniment is performed by applying an accompaniment style that changes over time to the accompaniment, which is a style that is applied to each section of the music in the time direction based on the analysis results of the music information.

4. The accompaniment generation method of claim 1, wherein identification information of the music piece represented by the music score is obtained based on the music score image data, and the generation of the accompaniment involves applying to the accompaniment an accompaniment style that changes over time, which is a style that is applied to each section of the music piece in the time direction based on the identification information.

5. The accompaniment generating method according to claim 3, further comprising presenting a user interface for setting and / or changing the accompaniment style to be applied to the accompaniment.

6. The accompaniment generating method according to claim 3, further comprising presenting candidate styles whose main beat matches that of the music based on the analysis results of the music information, and generating the accompaniment involves determining the accompaniment style to be applied based on a style selected from the candidate styles.

7. An accompaniment generating method according to claim 3, wherein the accompaniment style includes information regarding the main time signature of the music piece, and the accompaniment style is reset at a bar in the music piece whose time signature differs from the main time signature.

8. The accompaniment generating method according to claim 1, wherein generating the accompaniment comprises adding additional elements to the accompaniment based on information about divisions and / or intonations of the music piece.

9. The accompaniment generating method according to claim 5, wherein said modification user interface includes an interface for modifying the tempo of said accompaniment.

10. The accompaniment generating method according to claim 1, wherein generating the accompaniment includes synchronizing the accompaniment with the music piece.

11. A program for causing a computer to execute an accompaniment generation method, the accompaniment generation method including: acquiring sheet music image data representing a musical score; analyzing musical information contained in the musical score from the acquired sheet music image data; and generating an accompaniment to be added to the musical piece represented by the sheet music based on the analysis results of the musical information.

12. An accompaniment generation device comprising: a score image data acquisition unit that acquires score image data representing a score; a music information analysis unit that analyzes music information contained in the score from the acquired score image data; and an accompaniment generation unit that generates accompaniment to be added to the music represented by the score based on the analysis results of the music information.

13. An accompaniment generation system comprising an information terminal and an electronic musical instrument, wherein the information terminal comprises: a score image data acquisition unit that acquires score image data representing a score; a music information analysis unit that analyzes music information contained in the score from the acquired score image data; and an accompaniment generation unit that generates an accompaniment to be added to the music represented by the score based on the analysis results of the music information; and wherein the information terminal is connected to the electronic musical instrument, and the accompaniment is played on the electronic musical instrument.

Images