Accompaniment generation method, accompaniment generation apparatus, and accompaniment generation system

Abstract

To generate an added value which cannot be obtained only by referring to a musical score by giving additional information to a musical piece by interpreting the musical score.SOLUTION: The accompaniment generation method includes acquiring the musical score image data SD representing the musical score 5, analyzing the musical piece information included in the musical score 5 from the acquired musical score image data SD, and generating the accompaniment to be added to the musical piece represented by the musical score 5 based on the analysis result of the musical piece information. The accompaniment generation program executes the accompaniment generation method on a computer.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an accompaniment generation method for generating accompaniment for a piece of music, an accompaniment generation program for executing the accompaniment generation method, an accompaniment generation device, and an accompaniment generation system. [Background technology]

[0002] Accompaniment is sometimes used when playing an instrument or singing. For example, musicians can practice playing an instrument along with accompaniment and enjoy playing an instrument. Similarly, singers and ordinary people can practice singing along with accompaniment and enjoy singing.

[0003] Patent Document 1 below discloses a device that takes a musical score as input, interprets the music, and outputs performance information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 3177991 Summary of the Invention [Problem to be solved by the invention]

[0005] The aim of Patent Document 1 is to provide a well-balanced interpretation of acoustics based on dynamic signals read from musical scores. It would be beneficial for users if a service useful to musicians and singers could be provided based on information read from musical scores.

[0006] The object of the present invention is to provide additional information to a piece of music by interpreting the score, thereby creating added value that cannot be obtained by simply referring to the score. [Means for solving the problem]

[0007] An accompaniment generation method according to one aspect of the present invention includes acquiring sheet music image data representing a musical score, analyzing music information contained in the score from the acquired sheet music image data, and generating an accompaniment to be added to the music represented by the sheet music based on the analysis results of the music information.

[0008] An accompaniment generation program according to another aspect of the present invention is a program that causes a computer to execute an accompaniment generation method, which includes acquiring sheet music image data representing a musical score, analyzing musical information contained in the musical score from the acquired sheet music image data, and generating an accompaniment to be added to the musical piece represented by the sheet music based on the analysis results of the musical information.

[0009] An accompaniment generation device according to another aspect of the present invention includes a score image data acquisition unit that acquires score image data representing a score, a music information analysis unit that analyzes music information contained in the score from the acquired score image data, and an accompaniment generation unit that generates accompaniment to be added to the music represented by the score based on the analysis results of the music information. [Effects of the Invention]

[0010] According to the present invention, by interpreting the musical score and providing additional information to the music, it is possible to create added value that cannot be obtained by simply referring to the musical score. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing a user playing an electronic musical instrument to an accompaniment generated by the accompaniment generating method according to the present embodiment. FIG. [Figure 2] 1 is a block diagram of an accompaniment generation device according to an embodiment. [Figure 3] 1 is a functional block diagram of an accompaniment generation device according to an embodiment. [Figure 4] FIG. 2 is a diagram showing the concept and data structure of accompaniment style data. [Figure 5] 1 is a flowchart showing an accompaniment generating method according to an embodiment. [Figure 6]1 is a flowchart showing an accompaniment generating method according to an embodiment. [Figure 7] 1 is a flowchart showing an accompaniment generating method according to an embodiment. [Figure 8] 1 is a flowchart showing an accompaniment generating method according to an embodiment. [Figure 9] FIG. 10 is a functional block diagram of an accompaniment generating device according to a first modified example. DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An accompaniment generation method, an accompaniment generation program, an accompaniment generation device, and an accompaniment generation system according to embodiments of the present invention will now be described with reference to the accompanying drawings.

[0013] {1. How to use the accompaniment generation device} FIG. 1 is a diagram showing a user 4 playing an electronic musical instrument 1 along with an accompaniment generated by an accompaniment generation method according to this embodiment. The user 4 imports a musical score 5 as image data into the electronic musical instrument 1 in advance, which causes the electronic musical instrument 1 to automatically generate accompaniment data. The user 4 plays the electronic musical instrument 1 while playing back the accompaniment data. The user 4 can enjoy playing the electronic musical instrument 1 along with the accompaniment. The user 4 can enjoy playing like a piano version of karaoke.

[0014] {2. Electronic Instrument Structure} FIG. 2 is a block diagram of an electronic musical instrument 1 equipped with an accompaniment generation device 200 according to this embodiment. In this embodiment, the electronic musical instrument 1 will be described as an electronic keyboard instrument. The electronic musical instrument 1 according to this embodiment has the ability to read and play MIDI (Musical Instrument Digital Interface) data. As shown in FIG. 2, the electronic musical instrument 1 includes performance controls 11, a storage device 12, a sound source 13, a sound system 14, a control unit 2, a user interface 3, and an external interface 15. The control unit 2 includes a CPU (Central Processing Unit) 21, a RAM (Random Access Memory) 22, and a ROM (Read Only Memory) 23. The user interface 3 includes controls 31 and a display 32. The control unit 2 functions as the accompaniment generation device 200 according to this embodiment.

[0015] The performance controller 11 includes a keyboard 111 and is connected to the bus 20. The keyboard 111 has an arrangement of multiple keys. The keyboard 111 may have, for example, 61 or 88 keys. The keyboard 111 of the performance controller 11 may be an image of a keyboard displayed on the screen of a touch panel display, which will be described later.

[0016] The storage device 12 includes a storage medium such as a hard disk, optical disk, magnetic disk, or memory card. This storage device 12 stores computer programs such as the program PG, music score image data SD, accompaniment style data ASD, identification information ID, accompaniment data AD, and control programs. The program PG is a program for executing the accompaniment generation process according to the present invention.

[0017] The CPU 21, RAM 22, and ROM 23 are connected via a bus 20. The RAM 22 is, for example, a volatile memory, and is used as a work area for the CPU 21, and temporarily stores various data. The ROM 23 is, for example, a nonvolatile memory, and stores various programs, setting data, etc. The CPU 21 executes programs stored in the storage device 12 or the ROM 23, using the RAM 22 as a work area.

[0018] The sound source 13 is connected to the bus 20 and outputs audio data (sound signals) based on pitches specified by operating the keyboard 111. The audio data is sampling data (e.g., PCM (Pulse Code Modulation) data) that indicates the waveform of a sound. Hereinafter, the audio data output by the sound source 13 will be referred to as sound signals. The sound source 13 stores sound signals for all pitches in advance. The sound system 14 includes a digital-to-analog (D / A) conversion circuit, an amplifier, and a speaker. The sound system 14 converts the sound signals provided by the sound source 13 into analog sound signals and generates sounds based on the analog sound signals.

[0019] The controls 31 include on / off switches, rotary switches, slide switches, and the like, and are connected to the bus 20. The controls 31 are used to perform various settings, including power on / off, volume adjustment, and settings for applying various effects to the audio signals output from the electronic musical instrument 1. The controls 31 may also be used for performance operations. The display 32 includes, for example, a liquid crystal display and is connected to the bus 20. The display 32 displays the title of a piece of music, musical scores, and various other information. In this embodiment, the display 32 also displays operation menus for specifying a method for analyzing music information, an operation menu for specifying an accompaniment style, and the like. The display 32 may be a touch panel display. In this case, the keyboard 111 or some or all of the controls 31 may be displayed on the display 32. The user 4, who is the performer, can perform various operations by operating the display 32.

[0020] The external interface 15 is an interface for connecting to an external storage medium ED or a network. The CPU 21 can access a storage medium ED such as a CD-ROM, DVD, MD, or USB storage memory via the external interface 15.

[0021] {3. Configuration of the accompaniment generation device} Next, the functions of the accompaniment generation device 200 of this embodiment will be described. Fig. 3 is a block diagram showing the functional configuration of the accompaniment generation device 200 according to this embodiment. The accompaniment generation device 200 is realized by the CPU 21 executing the program PG while using hardware resources such as the RAM 22. As shown in Fig. 3, the accompaniment generation device 200 includes a score image data acquisition unit 201, a music information analysis unit 202, an accompaniment generation unit 203, an analysis method reception unit 204, and an accompaniment style setting unit 205. Each of the functional units 201 to 205 is realized by the CPU 21 executing the program PG while using hardware resources such as the RAM 22.

[0022] The sheet music image data acquisition unit 201 acquires sheet music image data SD stored in the storage device 12. The sheet music image data SD is, for example, image data such as PDF (Portable Document Format). The user stores the sheet music image data SD in the storage device 12 by scanning a sheet music (musical score) in advance, or based on electronically distributed sheet music data. For example, sheet music image data SD that is an electronic musical score converted from an SMF (Standard MIDI File) may be used. Alternatively, the sheet music image data acquisition unit 201 may acquire the sheet music image data SD based on a sheet music read by a scanner connected via the external interface 15.

[0023] The music information analysis unit 202 analyzes music information from the score image data SD acquired by the score image data acquisition unit 201. The music information analysis unit 202 acquires information (text information, note information) about the music score recorded in the score image data SD by performing image analysis of the music score image data SD. The music information analysis unit 202 analyzes the music represented by the music score recorded in the score image data based on the acquired information about the music score.

[0024] Based on the analysis results of the music information analysis unit 202, the accompaniment generation unit 203 automatically generates accompaniment to be added to the music represented by the music score recorded in the music score image data SD. The accompaniment generation unit 203 stores the generated accompaniment as accompaniment data AD in the storage device 12. The accompaniment generation unit 203 uses accompaniment style data ASD to set the style of the accompaniment to be added to the music. The accompaniment style data ASD is data in which the accompaniment style is set for each section. The user can arrange the melody, atmosphere, etc. of the accompaniment by applying various types of accompaniment style data ASD to the accompaniment data AD. The accompaniment style data ASD will be explained in detail later. In this embodiment, the accompaniment data AD is composed of SMF (Standard MIDI File) and control data for controlling the accompaniment style data ASD.

[0025] The analysis method receiving unit 204 specifies the analysis method for the music information in the music information analysis unit 202. The analysis method receiving unit 204 receives the analysis method specification from the user via the user interface 3 (the operators 31 or the display 32 functioning as a touch panel). The analysis method receiving unit 204, for example, displays an analysis method selection menu on the display 32 and receives the analysis method for the music information in response to a user operation using the operators 31 or the display 32. The analysis method receiving unit 204 provides the received designation information for the analysis method to the music information analysis unit 202. Note that the accompaniment generation device 200 may not be equipped with the analysis method receiving unit 204. Alternatively, the accompaniment generation device 200 may be equipped with no analysis method receiving unit 204 if a single analysis method is used or if the analysis method is automatically set.

[0026] The accompaniment style setting unit 205 sets accompaniment style data ASD to be used by the accompaniment generation unit 203. The accompaniment style setting unit 205 accepts an accompaniment style designation operation from the user via the user interface 3 (the operators 31 or the display 32 functioning as a touch panel). For example, the accompaniment style setting unit 205 displays a list menu of accompaniment styles on the display 32, and accepts the designation of the accompaniment style in response to a user operation using the operators 31 or the display 32. The accompaniment style setting unit 205 provides the accepted accompaniment style designation information (designation information of the accompaniment style data ASD) to the accompaniment generation unit 203.

[0027] {4. Accompaniment Style Data} FIG. 4 is a diagram illustrating the concept and data structure of accompaniment style data ASD. FIG. 4 shows an example of accompaniment style data ASD. The accompaniment style data ASD is a style to be applied to each section of a piece of music in the time direction, and defines elements that change over time. As shown in FIG. 4, one or more accompaniment style data ASD are prepared for each category, such as jazz or rock. Such categories may be arranged hierarchically. For example, big band and modern jazz may be provided as sub-categories of jazz.

[0028] Each accompaniment style data ASD in Figure 4 includes accompaniment element data for an "intro" section, a "main" section, a "fill-in" section, and an "ending" section. A section represents a portion of a piece of music in the time direction. The accompaniment element data for each section includes multiple variations of accompaniment element data that differ in atmosphere or excitement. The "intro" section, "main" section, "fill-in" section, and "ending" section are represented by the letters "I," "M," "F," and "E," respectively. Furthermore, the variations of the "intro," "main," and "ending" sections are represented by the letters "A" (normal (quiet)), "B" (slightly flashy), "C" (flashy), and "D" (very flashy), etc., according to the atmosphere or excitement.

[0029] Because "fill-in" sections are used as transitions between other sections, their variations are represented by a combination of two letters that correspond to the change in mood or intensity between the preceding and following sections. For example, "AC" corresponds to a change from "quiet" to "exciting." In this case, it would be natural for the preceding section's variation to be "A" and the following section's variation to be "C," but the choice of variation is arbitrary.

[0030] Each accompaniment element data is represented by a combination of an alphabet representing the section type and an alphabet representing the variation. For example, the section of accompaniment element data MA is "main" and the variation is "A." The section of accompaniment element data FAB is "fill-in" and the variation is "AB."

[0031] Each accompaniment element data includes accompaniment pattern data for multiple tracks (accompaniment parts), as well as reference chord information and pitch conversion rules (pitch conversion table information, range, re-pronunciation rules when changing chords, etc.). The accompaniment pattern data is a string of notes in MIDI format or phrase data in audio format, and can be converted to any pitch based on the reference chord information and pitch conversion rules. Therefore, multiple accompaniment pattern data can be used to generate accompaniment data AD to be added to a song.

[0032] {5. Accompaniment Generation Processing Method} Next, an accompaniment generation method according to this embodiment will be described. Figures 5 to 8 are flowcharts showing the accompaniment generation method according to this embodiment. The flowcharts shown in Figures 5 to 8 show processing that is realized by the CPU 21 executing the program PG while using hardware resources such as the RAM 22. In other words, the flowcharts shown in Figures 5 to 8 show processing that is executed by the accompaniment generation device 200, which includes the score image data acquisition unit 201, music information analysis unit 202, accompaniment generation unit 203, analysis method reception unit 204, and accompaniment style setting unit 205 shown in Figure 3.

[0033] 5, the user stores the music score image data SD in the storage device 12. For example, the user creates the music score image data SD by scanning a sheet of music that the user owns, and stores the music score image data SD in the storage device 12. Alternatively, the user obtains electronic music score data from a website or the like, and stores the electronic music score data in the storage device 12 as the music score image data SD by performing data conversion or the like on the electronic music score data.

[0034] In step S1, the music score image data acquisition unit 201 acquires music score image data SD representing a music score stored in the storage device 12.

[0035] In step S2, the music information analysis unit 202 analyzes music information contained in the score from the score image data SD acquired by the score image data acquisition unit 201. For example, the music information analysis unit 202 analyzes music information by acquiring information such as the music's notes, time signature, BPM (Beats Per Minute), repeats, volume markings, double lines, or rehearsal marks from the score image data SD. Music information is information for playing the music represented on the score according to the score, and includes information such as scale, tempo, dynamics, and repetition. The music information analysis unit 202 also acquires an identification information ID that identifies the music based on the music name or music information acquired from the score image data SD. For example, the music information analysis unit 202 acquires the identification information ID by connecting to a server on the Internet based on the analyzed information. There may be cases where it is not possible to acquire the music's identification information ID, and acquiring the identification information ID is optional.

[0036] Methods for obtaining the identification information ID of a song include: outputting the song (MIDI, Audio) using musical interpretation based on the sheet music, connecting to a server on the Internet and obtaining the identification information ID based on the song output; connecting to a server on the Internet and obtaining the identification information ID based on hash (one-way function) data of the song output; and connecting to a server on the Internet and obtaining the identification information ID based on hash (one-way function) data of the sheet music image data SD.

[0037] 6 is a flowchart showing the details of the processing of step S2 (music information analysis processing). First, in step S21, the music information analysis unit 202 determines whether or not it has received a designation of an analysis method. The user can designate the analysis method using the user interface 3 (the controls 31 or the display 32 functioning as a touch panel). The user designates an analysis method such as beat or BPM, for example. The music information analysis unit 202 may, for example, display a selection menu of analysis methods on the display 32, allowing the user to select a preferred analysis method.

[0038] If the designation of the analysis method is accepted in step S21, the music information analysis unit 202 executes analysis processing in accordance with the accepted analysis method in step S22.

[0039] If no designation of an analysis method is accepted in step S21, the music information analysis unit 202 executes a default analysis process in step S23. Alternatively, the analysis method may be determined based on the acquired identification information ID of the music. For example, an analysis method corresponding to the identification information ID may be acquired by connecting to a server on the Internet. The server may store statistical information on which analysis method other users have used for each identification information ID. The analysis method corresponding to the identification information ID acquired from the server may be set as the default analysis method. In this case, even if the user does not designate an analysis method, an appropriate analysis method for the music is selected, which is highly convenient.

[0040] Next, in step S24, the music information analysis unit 202 provides the accompaniment generation unit 203 with the analysis result of the music information, which is the result of the analysis process.

[0041] Referring again to Figure 5, in step S3, the accompaniment generation unit 203 generates an accompaniment to be added to the music represented by the score based on the analysis results of the music information. As described above, the accompaniment generation unit 203 generates accompaniment data AD that is made up of SMF format data and control data that controls the accompaniment style data ASD.

[0042] 7 is a flowchart showing the details of the processing of step S3 (accompaniment generation processing). First, in step S31, the accompaniment generation unit 203 generates SMF format data constituting the accompaniment data AD based on the analysis result of the music information acquired from the music information analysis unit 202. Next, in step S32, the accompaniment generation unit 203 determines the accompaniment style data ASD to be applied to the accompaniment.

[0043] The accompaniment generation unit 203 applies to the accompaniment an accompaniment style that changes over time, which is a style that is applied to each section in the time direction of the music piece. This changes the style for each section, preventing the accompaniment from becoming monotonous. Alternatively, the accompaniment generation unit 203 also determines which type of section-by-section type (A, B, ...) described with reference to FIG. 4 should be assigned to each style. Note that the processing of steps S31 and S32 in FIG. 7 may be reversed. The type (A, B, ...) is set (determined as to intonation and excitement) based on (or different from) criteria according to the style.

[0044] 8 is a flowchart showing the details of the process of step S32 (deciding accompaniment style data). First, in step S321, the accompaniment generating unit 203 determines whether or not identification information ID of the music piece has been given.

[0045] If it is determined in step S321 that the identification information ID of the song has been given, then in step S322 the accompaniment generation unit 203 determines the accompaniment style data ASD to be applied to the accompaniment based on the identification information ID. For example, the accompaniment generation unit 203 acquires information about the song from a server on the Internet based on the identification information ID, and determines the accompaniment style data ASD to be applied based on the acquired information. Alternatively, the accompaniment generation unit 203 determines the accompaniment style data ASD based on the identification information ID and recommended information stored in the server.

[0046] As a method of determining the accompaniment style data ASD based on the identification information ID, the accompaniment generation unit 203 may present to the user multiple accompaniment style data ASD based on recommendation information stored in the server. In this case, the accompaniment style is determined by the user selecting their preferred accompaniment style data ASD from the multiple pieces of recommendation information. Alternatively, the recommendation information stored in the server may be popularity information from other users. The accompaniment generation unit 203 may present a popularity ranking of styles selected by other users for songs with the same identification information ID.

[0047] If it is determined in step S321 that the identification information ID of the music piece has not been given, then in step S323, the accompaniment generating unit 203 determines whether or not the user has selected an accompaniment style.

[0048] If it is determined in step S323 that the user has selected an accompaniment style, the accompaniment generation unit 203 applies the selected accompaniment style to the accompaniment in step S324. As described above, the accompaniment style setting unit 205 accepts, via the user interface 3, a specification of the accompaniment style data ASD desired by the user. This allows the user to apply an accompaniment style according to their preferences. The accompaniment style setting unit 205 also accepts, via the user interface 3, a change of the accompaniment style. If the user wishes to change the style of the generated accompaniment, the user can instruct the change of the accompaniment style via the user interface 3. For example, even if, in step S322, the accompaniment style data ASD is determined based also on the identification information ID, the accompaniment style can be changed according to the user's preferences.

[0049] If it is determined in step S323 that the user has not selected an accompaniment style, in step S325 the accompaniment generation unit 203 sets the accompaniment style to be applied to the accompaniment based on the analysis results of the music information acquired from the music information analysis unit 202. Alternatively, the accompaniment generation unit 203 applies a preset standard accompaniment style to the accompaniment.

[0050] The accompaniment generation unit 203 includes in the accompaniment data AD the data that controls the accompaniment style data ASD determined in step S322, S324, or S325. The accompaniment generation unit 203 stores the generated accompaniment data AD (SMF data and data that controls the accompaniment style data ASD) in the storage device 12.

[0051] As described above, the accompaniment generation device 200, accompaniment generation method, and accompaniment generation program of the present embodiment automatically generate an accompaniment for a musical piece represented by the musical score 5 based on the musical score image data SD obtained by imaging the musical score 5. The user 4 can enjoy playing the electronic musical instrument 1 while playing back the generated accompaniment. The user 4 can use the accompaniment to practice playing the electronic musical instrument 1. The user 4 can enjoy playing the electronic musical instrument 1 in accordance with the accompaniment while listening to the accompaniment. For example, the user can enjoy playing a rich accompaniment even with a score without accompaniment (such as a guitar riff score or a score for singing and playing), a TAB score, or a score with a simple accompaniment. The accompaniment generation device 200 of the present embodiment can add new value to the electronic musical instrument 1. The accompaniment generated in the present embodiment includes various musical pieces to be matched with the performance of the electronic musical instrument 1. For example, a main melody with different climaxes depending on the style may be used as the accompaniment. The accompaniment generated in the present embodiment is intended to be generated over the entire time span of a musical piece. For example, the accompaniment is intended to be generated in a continuous manner from near the beginning of the song (near the intro) to near the final measure (near the ending) (of course, when we say "overall," it also includes parts that include silent sections).

[0052] {6. Modifications} (Variation 1) Next, a first modification of this embodiment will be described. Fig. 9 is a functional block diagram of an accompaniment generation device 200 according to the first modification. As shown in Fig. 9, the accompaniment generation device 200 according to the first modification includes a candidate style presenting unit 206 in addition to the components shown in Fig. 3. The components other than the candidate style presenting unit 206 are the same as those described above. The candidate style presenting unit 206 is a functional unit that presents, as candidate styles, accompaniment styles suited to a piece of music represented by a musical score.

[0053] The candidate style presentation unit 206 presents candidate styles whose main meter matches the main meter of the song based on the analysis results of the song information. The song information analysis unit 202 acquires the main meter of the song by analyzing the score image data SD. For example, the song information analysis unit 202 counts the number of bars for each beat in the song. Then, it determines the beat with the most bars (longest) as the main meter of the song. The candidate style presentation unit 206 selects only accompaniment styles whose main meter matches from the multiple accompaniment style data ASD stored in the storage device 12 and presents them to the user. For example, the candidate style presentation unit 206 displays a list menu of candidate styles on the display 32 and accepts the user's designation of the accompaniment style desired by the user in response to a user operation using the controls 31 or the display 32. The user can select a preferred accompaniment style from candidate styles whose main meter matches, thereby preventing the user from selecting an accompaniment style that does not match the song. The accompaniment generating unit 203 determines the accompaniment style data ASD to be applied based on the accompaniment style selected from the candidate styles.

[0054] (Variation 2) Next, a second variation of this embodiment will be described. For example, some pop piano pieces contain odd time signatures. Even when playback is started by synchronizing the SMF and accompaniment style data ASD that make up the accompaniment data AD, the rhythms may not match at the point where an odd time signature occurs. Typically, preset accompaniment styles built into electronic musical instruments have a fixed time signature. Therefore, in a second variation of this embodiment, playback is reset (stop and start) every measure. However, resetting every measure results in an accompaniment that feels unchanging and repetitive. Therefore, in the second variation, preferably, an accompaniment style that matches the main time signature of the piece is applied as a general rule, and the accompaniment style is reset at points where the time signature differs from the main time signature. Note that odd time signatures can be determined from score information or music information.

[0055] (Variation 3) Next, a third variation of this embodiment will be described. For example, if an accompaniment based on an accompaniment style is simply looped for a piece of music lasting several minutes, the accompaniment may sound monotonous to the user. Therefore, in the third variation, the accompaniment generation unit 203 adds section changes and fill-ins to the accompaniment data AD. Even if changes are made to the accompaniment at appropriate timings based solely on the passage of time, the accompaniment may end up sounding unnatural and ignoring the original composition and progression of excitement. Therefore, the accompaniment generation unit 203 according to the third variation acquires information about the division positions of the composition of the music and the excitement of the music (information about the intonation of the music) from the score information, and adds appropriate excitement to the accompaniment according to the composition of the music.

[0056] The accompaniment generating unit 203 adds section changes and fill-ins to the accompaniment data AD based on, for example, rehearsal marks, double lines, repeat symbols, or volume symbols.

[0057] Rehearsal marks are obtained from symbols enclosed in a rectangle on the musical score. Rehearsal marks are usually made up of capital letters or the like. Double lines indicate divisions in the musical composition, allowing for the addition of excitement to the accompaniment according to the composition intended by the score creator. Repeat symbols also coincide with divisions in the musical composition in many pieces of music, so they can be used as points to introduce changes to the accompaniment. Volume symbols also coincide with divisions in the musical composition in many pieces of music, so they can be used as points to introduce changes to the accompaniment. Volume symbols are an example of "information regarding intonation" in this invention. As "information regarding intonation," the accompaniment generation unit 203 may use the number of notes (density) of the singing part.

[0058] (Variation 4) Next, a fourth variation of this embodiment will be described. The fourth variation is an example of a method for specifying the above-mentioned analysis method. Some musical scores are written at a tempo that is double or half the original tempo of the music piece. For example, there are cases where the tempo is written at a tempo that matches the user's performance level. When accompaniment data AD generated from such a musical score is played back, the accompaniment is played back at a tempo that is different from the tempo assumed by the original music piece. Therefore, in the fourth variation, the tempo of the accompaniment can be changed to a different tempo, such as double or half, in response to a user's specification.

[0059] For example, if the accompaniment generation unit 203 wants to set the accompaniment style to double speed for the melody, it converts the SMF constituting the accompaniment data AD so that the length of the melody in the time direction is halved. Conversely, if the accompaniment style wants to set the accompaniment style to half speed for the melody, it converts the SMF constituting the accompaniment data AD so that the length of the melody in the time direction is doubled.

[0060] In Modification 4, the accompaniment generation unit 203 also converts compound time signatures. Compound time signatures are triplet-based time signatures, such as 6 / 8, 9 / 8, and 12 / 8. Analysis of composite time signature scores reveals that when 6 / 8, 9 / 8, and 12 / 8 time signatures are associated with quarter notes, their time lengths are the same as those of triplet, 4.5, and sextuple, respectively. However, because compound time signatures are based on triplets, using the analysis results as is would not match the accompaniment to a musical piece. Therefore, the accompaniment generation unit 203 converts the melody of the compound time signature section in the SMF that constitutes the accompaniment data AD to 2 / 3 times its original length. If compound time signature accompaniment styles are available, they can be used. However, since most accompaniment styles are created as triplet systems in four beats, the conversion function provided by Modification 4 would be convenient.

[0061] (Variation 5) Next, a fifth variation of this embodiment will be described. In the fifth variation, the accompaniment generation unit 203 synchronizes the accompaniment in the intro and ending of a piece of music. For example, a simple intro with a playback section of only one bar is set in the accompaniment style data ASD. For example, the same time signature as that of the beginning of the piece of music is applied to this intro. Here, as explained in the fourth variation, if the tempo of the accompaniment style is set to double speed or half speed, the speed of the intro section can be converted accordingly.

[0062] The accompaniment style data ASD also includes a simple, standard ending. For example, if a song uses syncopation in its ending, the timing of the song and the accompaniment may not match. Therefore, the accompaniment generation unit 203 resets the accompaniment style at the end of the final measure, and then generates the accompaniment data AD so that the ending is played. When the accompaniment ends on the final note, the user is given the impression that the accompaniment has finished playing in perfect time with the score.

[0063] {7. Other Embodiments} In the above embodiment, the data for controlling the accompaniment style is generated as separate data from the SMF. In other embodiments, the data for controlling the accompaniment style may be embedded in the SMF rather than being generated as separate data. For example, the accompaniment style may be embedded in the MIDI data as note information.

[0064] 6, in step S22, the music information analysis unit 202 executes analysis processing according to the analysis method received from the user. In another embodiment, the music information analysis unit 202 may access a server on the Internet and obtain information about the analysis method from the server. For example, the server collects information from a large number of users and stores data about the optimal analysis method.

[0065] In the embodiment described in Fig. 8, in step S322, the accompaniment generation unit 203 determines the accompaniment style to be applied to the accompaniment based on the identification information ID. In other embodiments, the accompaniment generation unit 203 may determine the accompaniment style based on the identification information ID and the tempo of the music obtained from the analysis results of the music. As mentioned above, some musical scores are written at a tempo that is twice or half the original tempo of the music, so when searching for an accompaniment style using the identification information ID, tempo information may also be used to improve search accuracy.

[0066] In the embodiment described in Fig. 8, in step S323, the accompaniment generation unit 203 uses the accompaniment style selected by the user via the user interface 3. The user interface 3, which allows the user to set / change the accompaniment style, can present a list of accompaniment styles to the user. This list is displayed based on, for example, the preset accompaniment styles stored in the electronic musical instrument 1 connected to the accompaniment generation device 200.

[0067] In the above embodiment, an example has been described in which a user 4, who is an instrument player, plays an electronic musical instrument 1 along with an accompaniment generated by an accompaniment generation method. The user may also sing along with the accompaniment generated by the accompaniment generation method of the present embodiment. The user can practice singing along with the accompaniment or enjoy karaoke along with the accompaniment.

[0068] In the above embodiment, the electronic musical instrument 1 is described as being equipped with the accompaniment generation device 200. In another embodiment, the accompaniment generation device 200 may be equipped in an information terminal such as a tablet. In this case, the user reads the musical score using the tablet. The tablet is connected to the electronic musical instrument, and the accompaniment generated on the tablet is played on the electronic musical instrument. Alternatively, the tablet may have the functionality of an electronic musical instrument and play the accompaniment on the tablet. The user can perform musical operations on a virtual musical instrument displayed on the tablet's display.

[0069] {8. Features and Effects of the Embodiments} (1) The accompaniment generation method according to this embodiment includes acquiring sheet music image data SD representing a sheet music score, analyzing music information contained in the sheet music 5 from the acquired sheet music image data SD, and generating an accompaniment to be added to the music represented by the sheet music 5 based on the analysis results of the music information. An accompaniment not included in the sheet music is generated based on the music information. A user can obtain an accompaniment for the music represented by the sheet music by imaging the sheet music. For example, the user can play while listening to the accompaniment.

[0070] (2) In the above accompaniment generation method, the analysis may include accepting a designation of an analysis method, and the analysis may analyze the music information based on the accepted designation. This allows the user to designate the analysis method of the music information, making it possible to generate an accompaniment that suits the user's preferences.

[0071] (3) In the above accompaniment generation method, the accompaniment may be generated based on the analysis results of the music information, in which a style is applied to each section in the time direction of the music, and an accompaniment style that changes over time may be applied to the accompaniment. Since the style changes for each section, the accompaniment does not become monotonous.

[0072] Here, a section is a unit into which a song is divided in the time direction, and each unit separated by a section change is processed to be determined as a section such as intro, main, fill-in, or ending.

[0073] (4) In the above accompaniment generation method, the identification information ID of the music piece represented by the music score 5 is obtained based on the music score image data SD, and the accompaniment is generated by applying an accompaniment style that changes over time to the accompaniment based on the identification information ID, which is a style that is applied to each section of the music piece in the time direction. Since an accompaniment style that suits the music piece is applied, it is possible to generate an accompaniment that matches the music piece.

[0074] (5) The above accompaniment generation method may include presenting a user interface for setting and / or changing the accompaniment style to be applied to the accompaniment. Since the accompaniment style according to the user's preference is applied, it is possible to generate accompaniment that matches the user's taste.

[0075] (6) In the above accompaniment generation method, the method may include presenting candidate styles whose main beats match the main beat of the music piece based on the analysis results of the music piece information, and generating the accompaniment may involve determining the accompaniment style to be applied based on a style selected from the candidate styles. Since the accompaniment style is determined from the candidate styles whose main beats match, an accompaniment style that matches the music piece is applied.

[0076] (7) In the above accompaniment generation method, the accompaniment style may include information about the main time signature of the music piece, and the accompaniment style may be reset in measures of the music piece where the time signature differs from the main time signature. This prevents mismatch between the music piece and the accompaniment style, even for music pieces with irregular time signatures.

[0077] (8) In the above accompaniment generation method, the accompaniment may be generated by adding additional elements to the accompaniment based on information about divisions and / or intonation of the music piece. This can prevent the accompaniment from becoming monotonous.

[0078] (9) In the above accompaniment generation method, the change user interface may include an interface for changing the tempo of the accompaniment. Since an accompaniment style is applied according to the tempo, it is possible to generate an accompaniment that matches the music piece or that is preferred by the user.

[0079] (10) In the above accompaniment generating method, generating the accompaniment may include synchronizing the accompaniment with the music piece.

[0080] (11) The accompaniment generation program according to this embodiment is a program that causes a computer to execute an accompaniment generation method, which acquires score image data SD representing a score 5, analyzes music information contained in the score 5 from the acquired score image data SD, and generates an accompaniment to be added to the music represented by the score 5 based on the analysis of the music information. An accompaniment that was not included in the score is generated based on the music information. A user can obtain an accompaniment for the music represented by the score by imaging the score.

[0081] (12) The accompaniment generation device 200 according to this embodiment includes a score image data acquisition unit 201 that acquires score image data SD representing a score 5, a music information analysis unit 202 that analyzes music information contained in the score 5 from the acquired score image data SD, and an accompaniment generation unit 203 that generates an accompaniment to be added to the music represented by the score 5 based on the analysis results of the music information. An accompaniment that was not included in the score is generated based on the music information. A user can obtain an accompaniment for the music represented by the score by imaging the score.

[0082] (13) The accompaniment generation system of this embodiment is an accompaniment generation system comprising an information terminal and an electronic musical instrument, wherein the information terminal includes a score image data acquisition unit 201 that acquires score image data SD representing a musical score 5, a music information analysis unit 202 that analyzes the music information contained in the musical score 5 from the acquired score image data SD, and an accompaniment generation unit 203 that generates an accompaniment to be added to the music represented by the musical score 5 based on the analysis results of the music information, and the information terminal is connected to the electronic musical instrument, and the accompaniment is played on the electronic musical instrument. [Explanation of symbols]

[0083] 1: Electronic instruments 3: User Interface 4: User 5: Music score 12:Storage device 31: Operator 32: Display 200:Accompaniment generation device 201: Music score image data acquisition unit 202: Music Information Analysis Unit 203: Accompaniment generation section 204: Analysis Method Reception Department 205: Accompaniment style setting section 206: Candidate style presentation section AD: Accompaniment data ASD: Accompaniment Style Data PG: Program SD: Music score image data

Claims

1. Acquire music score image data representing the music score; Analyzing music information contained in the score from the acquired score image data; an accompaniment generating method including generating an accompaniment to be added to the music represented by the score based on the analysis result of the music information.

2. Including accepting a designation of an analysis method, The accompaniment generating method according to claim 1 , wherein said analyzing step analyzes said music information based on a received specification.

3. The accompaniment generation method according to claim 1, wherein the generation of the accompaniment involves applying an accompaniment style that changes over time to the accompaniment, the accompaniment style being applied to each section of the music in the time direction based on the analysis results of the music information.

4. acquiring identification information of the music piece represented by the music score based on the music score image data; The accompaniment generation method according to claim 1, wherein the generation of the accompaniment involves applying an accompaniment style to the accompaniment that changes over time, the accompaniment style being applied to each section of the music piece in the time direction based on the identification information.

5. The accompaniment generating method according to claim 3 , further comprising presenting a user interface for setting and / or changing the accompaniment style to be applied to the accompaniment.

6. presenting candidate styles whose main beats match those of the music based on the analysis results of the music information; The accompaniment generating method according to claim 3 , wherein the step of generating the accompaniment comprises determining the accompaniment style to be applied based on a style selected from the candidate styles.

7. the accompaniment style includes information about the main time signature of the piece of music; The accompaniment generating method according to claim 3 , wherein the accompaniment style is reset in a measure of the music piece whose time signature differs from the main time signature.

8. The accompaniment generating method according to claim 1 , wherein generating the accompaniment comprises adding additional elements to the accompaniment based on information about divisions and / or intonations of the music piece.

9. The accompaniment generating method according to claim 5 , wherein the modification user interface includes an interface for modifying the tempo of the accompaniment.

10. The accompaniment generating method according to claim 1 , wherein generating the accompaniment includes synchronizing the accompaniment with the music piece.

11. A program for causing a computer to execute an accompaniment generation method, The accompaniment generation method includes: Acquire music score image data representing the music score; Analyzing music information contained in the score from the acquired score image data; an accompaniment generation program that generates an accompaniment to be added to the music represented by the score based on the analysis result of the music information.

12. a music score image data acquisition unit that acquires music score image data representing a music score; a music information analysis unit that analyzes music information included in the music score from the acquired music score image data; an accompaniment generation unit that generates an accompaniment to be added to the music represented by the score based on the analysis result of the music information; An accompaniment generating device comprising:

13. An accompaniment generation system comprising an information terminal and an electronic musical instrument, The information terminal a music score image data acquisition unit that acquires music score image data representing a music score; a music information analysis unit that analyzes music information included in the music score from the acquired music score image data; an accompaniment generation unit that generates an accompaniment to be added to the music represented by the score based on the analysis result of the music information; Including, The information terminal is connected to the electronic musical instrument, and the accompaniment is reproduced on the electronic musical instrument.

Images