Information processing device for drawing musical score, musical score generation method and program
The described method enhances musical score design freedom and readability by using a staff container to separate musical symbols from the background, addressing the limitations of existing software in creating visually appealing and readable scores.
Patent Information
- Application Number
- JP2023074118
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
AI Technical Summary
Existing musical score creation software lacks the ability to create visually appealing designs while maintaining readability, as adding background images reduces visibility of musical symbols and requires significant manual effort due to the varying note positions and large number of notes.
An information processing device and method that extracts positional information of staff areas and notes, generates a staff container to encompass notes, and distinguishes musical symbols from the background, allowing for customizable designs without compromising readability.
Enables musical scores with high design freedom and maintained readability through automated processes, reducing manual work and enhancing visual appeal.
Smart Images

Figure 2024158708000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, a method and a program for generating musical scores, and more particularly to an information processing device, a method and a program for rendering musical scores with a high degree of freedom and design. [Background technology]
[0002] In recent years, sheet music has attracted people's attention not only for its musical expression but also for its design, and there are many designs using musical notes and symbols other than sheet music. There are also sheet music collectors, and there are sheet music that are drawn with an awareness of visual beauty and interest. In addition, with the spread of performance video posting, there are an increasing number of cases where individuals and artists provide composed and arranged sheet music, and sheet music is beginning to create added value and value as a branding element.
[0003] While the visual appeal of sheet music is recognized, the majority of published sheet music is in black and white and has a uniform design. The main reason for this is that the use of background images significantly reduces the visibility of musical symbols such as notes, making them unsuitable for performance, so they are inevitably designed with a white background. Another reason is that the drawing range of notes varies vertically, so the objects drawn in the sheet music need to be custom-designed for each piece of music, which is time-consuming. Another reason is that the number of notes in a sheet music is huge, so designing depending on the position of each individual note requires a huge amount of work, which is very difficult for people to do by hand.
[0004] Currently available musical scores are polarized into traditional black-and-white scores that emphasize readability and non-readable art works. Several music notation software programs, such as finale (Non-Patent Document 1) and MuseScore (Non-Patent Document 2), are known for creating musical scores, but they are for creating black-and-white scores that emphasize readability. Various fonts for notation are provided in music notation software, but there are few design elements other than font selection, and they are not sufficient in terms of the design of the musical scores that can be created. In addition, non-readable art works are usually excerpts of a single page, and in the case of a representation method in which a drawing object other than the musical score is overlaid as in Non-Patent Document 3, part of the musical score is hidden and the musical score information is often lost. Alternatively, in those with a drawn background as in Non-Patent Document 4, the musical score information is preserved, but the readability tends to be low.
[0005] In addition, Japanese Patent Application Laid-Open No. 2009-153068 (Patent Document 1), which is related to the creation of musical scores, discloses a technique for easily creating musical scores that allow the user to recognize the continuity of the melody. Patent Document 1 discloses a configuration in which a user sets a unit such as a bar for processing image data of the musical score, divides image data of the musical score for each unit, judges whether identical image data are consecutive in the unit, and, if it is judged that identical image data are consecutive, adds information such as information for coloring a background image to the image data judged to be consecutive so that the user can easily recognize the consecutive melody in the musical score represented by the image data. Patent Document 1 also mentions a method of expressing notes by coloring them with colors assigned to each scale, a method of drawing arrows on the musical score, and a method of expressing performance symbols related to the speed and strength of the performance, or a method of expressing the background by coloring it with a color different from the color of the notes. However, the prior art of Patent Document 1 and the prior art mentioned in Patent Document 1 are limited to changing the color of notes or the color of background parts in units of bars, etc., and are not disclosed from the perspective of solving the problem of the readability of the musical score decreasing when there is a background as described above. In addition, since the prior art of Patent Document 1 is based on image matching, only the parts that match the melody specified by the input parameters or the parallel movement on the scale are subject to image processing. On the other hand, since a musical pattern (motif) in musical expression is generally subjected to slight corrections each time a musical pattern appears, it was not sufficient from the perspective of extracting a musical pattern in musical expression. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] JP 2009-153068 A [Non-patent literature]
[0007] [Non-Patent Document 1] finale,[online],[Searched on April 5, 2023],Internet<URL:https: / / www.finalemusic.jp / > [Non-Patent Document 2] MuseScore, [online], [searched on April 5, 2023], Internet<https: / / musescore.org / ja> [Non-Patent Document 3] Etsy,[online],[Searched on April 5, 2020],<URL:https: / / www.etsy.com / jp / listing / 572404942 / mary-poppins-silhouette-disney> [Non-Patent Document 4] Etsy,[online],[Searched on April 5, 2020],<URL:https: / / www.etsy.com / jp / listing / 267200594 / original-watercolor-pen-artwork-on> Summary of the Invention [Problem to be solved by the invention]
[0008] The present disclosure has been made in consideration of the above-mentioned shortcomings in the conventional technology, and aims to provide an information processing device, a music score generation method, and a program that are capable of creating music scores with a highly flexible design while maintaining the readability of the music score information with a minimal amount of manual work. [Means for solving the problem]
[0009] In order to solve the above problems, the present disclosure provides an information processing device for generating an image of a musical score based on musical score data, the information processing device having the following characteristics. The information processing device includes an extraction unit that extracts position information of a staff area and position information of a plurality of notes based on musical score data including at least information representing pitches. The information processing device also includes a container drawing unit that draws a staff container that encompasses one or more outside-staff portions of the plurality of notes that extend outside the staff area and the staff area, and that visually distinguishes musical symbols placed inside from the background, based on the position information of the staff area and the position information of the plurality of notes. The information processing device further includes an image generation unit that generates an image of the musical score in which the staff container and a plurality of musical symbols based on the musical score data are drawn.
[0010] The present disclosure also provides a score generation method for generating an image of a score based on score data, the method having the following characteristics. The score generation method includes a step in which one or more computers extract position information of a staff area and position information of a plurality of notes based on the score data including information representing at least pitches. The score generation method also includes a step in which one or more computers draw a staff container based on the position information of the staff area and the position information of the plurality of notes, the staff container including one or more outside-staff portions of the plurality of notes that extend outside the staff area and the staff area, and visually distinguishing musical symbols placed inside from the background. The score generation method further includes a step in which one or more computers generate an image of the score in which the staff container and the plurality of musical symbols based on the score data are drawn.
[0011] Furthermore, the present disclosure provides a program readable and executable by a processor for realizing an information processing device having the above-mentioned features. Effect of the Invention
[0012] With the above configuration, it is possible to create sheet music with a high degree of freedom in design while maintaining the readability of the sheet music information with a small amount of manual work. [Brief description of the drawings]
[0013] [Figure 1] 1 is a hardware block diagram showing a computer on which a musical score creation application according to an embodiment of the present invention runs. [Diagram 2] 1 is a functional block diagram of a musical notation system according to an embodiment of the present invention. [Diagram 3] 4 is a flow chart illustrating a process for creating musical scores according to an embodiment of the present invention. [Figure 4] FIG. 4 illustrates data structures used in the music notation process according to an embodiment of the present invention. [Diagram 5] 5A to 5C are diagrams for explaining the staff container generation process in the musical score creation process according to an embodiment of the present invention. [Figure 6] 11A to 11C are diagrams illustrating a modified example of the staff container generation process in the musical score creation process according to an embodiment of the present invention. [Figure 7] 5A to 5C are diagrams for explaining a note pattern highlighting process in the musical score creation process according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] The following describes an embodiment of the present invention, but the embodiment of the present invention is not limited to the embodiment described below. Note that in the embodiment described below, an information processing device is used as an example to run an application (music notation creation application) that generates an image of a musical score based on musical score data in response to a user's operation and displays the generated image of the musical score.
[0015] Before describing the details of the music notation application, the hardware configuration of an information processing device 10 on which the music notation application runs will be described below with reference to Fig. 1. In a specific embodiment, the information processing device 10 shown in Fig. 1 is implemented in a general-purpose computer such as a desktop computer or a laptop computer. The information processing device 10 includes a central processing unit (CPU) 12, a ROM 16 that stores a BIOS (Basic Input Output System) and the like, and a RAM (Random Access Memory) 16 that provides an execution memory space that enables the CPU 12 to process programs.
[0016] The CPU 12 is connected to a storage 18 such as a hard disk drive (HDD) or a solid state drive (SSD) via an internal bus 28. The storage 18 stores an operating system (OS) 30 and a music notation creation application program 32, which will be described in detail later. The OS may be any OS, such as Windows (registered trademark), UNIX (registered trademark), ChromeOS (registered trademark), or MacOS (registered trademark). The CPU 12 communicates with an input device 22, such as a keyboard or a mouse, via an appropriate interface such as a universal serial bus (USB) to receive input from a user. The information processing device 10 further includes a display device 20, which displays a graphical user interface for creating music scores and an image of the generated music score. The CPU 12 is also connected to a network interface (I / F) 24 via the internal bus 28, thereby connecting the information processing device 10 to a network such as the Internet.
[0017] The information processing device 110 reads out the OS 30 and the music score creation application program 32 stored in a storage device such as the ROM 16 or the storage 18, and expands them in the memory area of the RAM 14. This allows each functional unit and each process described below to be realized under the OS. Note that in the embodiment described, a general-purpose computer such as a desktop computer has been used as an example of the information processing device, but the present invention is not limited to this, and may be implemented in a mobile information terminal such as a smartphone or tablet computer using an appropriate OS such as Android (registered trademark), iOS (registered trademark), or iPadOS (registered trademark).
[0018] A musical score creation function according to an embodiment of the present invention will be described below with reference to Figs. 2 to 7. Fig. 2 is a functional block diagram showing the overall configuration of a musical score creation system 100 according to this embodiment. The musical score creation system 100 is realized by executing a musical score creation application program 32 on an information processing device 10 shown in Fig. 1. The musical score creation system 100 shown in Fig. 2 is configured to receive target musical score data 102 as input and output a musical score design image 106, and includes various components (software modules) and data structures.
[0019] The input target score data 102 is data describing the target score, and includes information such as the pitch (key) and note value (length) of the notes, and the horizontal position in the score. The target score data 102 is not particularly limited, and may be a score file for MusicXML (Music eXtensible Markup Language), MIDI (Musical Instrument Digital Interface), or other score creation applications (e.g., Finale (registered trademark) and MuseScore (registered trademark)). Alternatively, the target score data 102 may be data obtained by scanning a manuscript of the score with a scanner. From the viewpoint of extracting the musical pattern described later, it is preferable that the target score data 102 is provided as score data such as MusicXML describing the score, rather than as image data.
[0020] The musical score creation system 100 includes, as its constituent components, an information extraction unit 120 and a drawing unit 140. The information extraction unit 120 is configured to extract various information from the target musical score data 102, and includes, as its subcomponents, a position information extraction unit 122, a musical idea information extraction unit 124, and a note pattern matching unit 126.
[0021] Based on the target score data 102, the position information extraction unit 122 extracts position information of various musical symbols included in the score to be drawn, and outputs it to the drawing unit 140 as position information 113. Here, the musical symbols from which position information is extracted include a plurality of notes, staves, clefs, accidentals, and brackets connecting a plurality of staves based on information representing pitches (keys) included in the score data. The position information of the musical symbols includes information on the position of the musical symbols and the positions of parts of the musical symbols (for example, the positions of the note heads, stems, and tails that constitute the notes). Examples of position information include the coordinates, length, and direction of the center, top end, bottom end, left end, and right end of the musical symbols or parts thereof. The position of the notes is determined relative to the position of the staff, and more specifically, the notes are placed on the staff at a vertical position according to the pitch and a specified horizontal position. In other words, the position information of the notes has both musical position information and physical position information on the score image.
[0022] The musical score creation system 100 has, as data structures, musical idea extraction rules 111 and musical pattern matching rules 112. The musical idea extraction rules 111 define rules for assigning musical idea attributes to predetermined units such as bars. The musical pattern matching rules 112 define musical patterns (motifs) to be applied to multiple notes.
[0023] The musical idea information extraction unit 124 performs statistical processing based on the target musical score data 102, assigns musical idea attributes indicating the corresponding musical idea to a predetermined unit that matches any of the multiple rules in the musical idea extraction rules 111, and outputs the information to the drawing unit 140 as musical idea information 114. Here, the predetermined unit may be a bar unit as exemplified above, but is not particularly limited thereto, and may be a page unit, a staff line unit, or a set of notes.
[0024] The tone pattern matching unit 126 matches the target musical score data 102 with a plurality of tone patterns in the tone pattern matching rules 112, extracts a set of notes that match any of the plurality of tone patterns, and outputs the information to the drawing unit 140 as tone pattern information 115. More specifically, the tone pattern matching unit 126 matches the target musical score data with a plurality of tone patterns based on one or both of (1) a specification of an allowable range for the difference in pitch between a plurality of notes and (2) a specification of a similarity or a difference based on the pitch with respect to the tone pattern of a specified sample. The user may specify whether to use (1) the specification of the allowable range and (2) the specification of the similarity or the difference, or to use both. The tone patterns defined in the tone pattern matching rules 112 include (1) a predefined tone pattern (specified tone pattern), (2) a tone pattern that frequently appears in the target musical score data 102 (frequent tone patterns), and (3) a tone pattern similar to the above-mentioned specified tone pattern or the frequent tone pattern (similar tone patterns). Among these musical patterns, the musical pattern designated by the user is the subject of extraction.
[0025] In a preferred embodiment, the musical score creation system 100 may further include a frequent musical pattern extraction unit 130 and a similar musical pattern generation unit 132 in order to generate the musical pattern matching rule 112. The musical pattern matching rule 112, the frequent musical pattern extraction unit 130, and the similar musical pattern generation unit 132 will be described in detail later.
[0026] The drawing unit 140 draws the musical score based on the input musical symbol position information 113, musical mood information 114, and musical pattern information 115, and generates a musical score design image 106. The drawing unit 140 is configured to include, as subcomponents, a container drawing unit 141, a highlight drawing unit 142, a background drawing unit 143, a musical symbol drawing unit 144, and an image synthesis unit 145.
[0027] The container drawing unit 141 draws a staff container that includes the staff area and the part of the musical symbol that extends outside the staff area, based on the position information 113 of each musical symbol that is input. Here, the staff container is an object other than a musical symbol for visually distinguishing the musical symbol placed inside it from the background (e.g., background image or pattern), and is generated based on generation logic and generation parameters specified by the user. In addition, preferably, the staff container itself is also visually distinguished from the background. The staff container specifies at least one of the following: placement of an image (e.g., filling with a pixel value with high brightness such as white, or a pattern that does not interfere with the readability of the musical symbol) in the foreground of the background image and the background of the musical symbol, and application of image processing (e.g., adjustment to increase the brightness of the background image in the staff container area) to the area corresponding to the staff container in the background image. What kind of processing (at least one of image placement and application of image processing) is performed on the staff container may be specified by a default value or may be specified by the user. The staff container is generated to give freedom to the background image while avoiding interference with the musical symbols placed inside, and preferably, its outline is defined so as not to include, produce, or be dominant in any line segments that are approximately parallel to the staff lines, in order not to reduce the readability of the staff lines and the musical symbols placed along the staff lines.
[0028] In a preferred embodiment, the container drawing unit 141 may draw a staff container based on the position information 113 of each musical symbol as well as on the musical mood information 114 attached to a predetermined unit. For example, the container drawing unit 141 can change the processing (image placement, application of image processing) for an area in the staff container corresponding to a predetermined unit, such as a measure, based on the musical mood attribute attached to the predetermined unit. For example, depending on the musical mood information 114, it can change the pixel value of the fill (for example, change the background color of the middle part (B) of a ternary form (ABA) from the color of the parts (A) before and after it) or change the degree of brightness adjustment to be applied (make the middle part (B) brighter than the parts (A) before and after it). Details of drawing the staff container will be described later.
[0029] The highlight drawing unit 142 draws a musical pattern highlight that modifies a plurality of notes included in the set of notes based on the musical pattern information 115 and the position information of each note included in the set of notes to which the musical pattern information 115 has been added. Here, the musical pattern highlight is an effect for modifying and highlighting a set of notes, preferably while avoiding deterioration of readability, and is generated based on a generation logic and a generation parameter that are pre-specified by a user. The musical pattern highlight may specify a change in the image itself of a plurality of notes that match a predetermined musical pattern (e.g., changing the font of the notes), an arrangement of images based on the note heads, note stems, or lines connecting the note heads or note stems of a plurality of notes included in the set of notes (e.g., an arrangement of images or figures centered on the note heads of a series of notes), or application of image processing. For example, as the musical pattern highlight, a concentric graphic object can be arranged around each of a plurality of notes that match a predetermined musical pattern (e.g., around the note heads of the notes). In a preferred embodiment, the highlight drawing unit 142 can also draw mood highlights that modify a plurality of musical symbols included in a predetermined unit based on mood attributes and position information of a plurality of musical symbols included in the predetermined unit to which the mood attributes are attached. The mood highlights are similar to the form highlights except that the object of modification is a predetermined unit to which the mood attributes are attached, and may include image placement or image processing based on the positions of a plurality of musical symbols included in the predetermined unit.
[0030] The background drawing unit 143 draws a background image of the score based on the user's designation, and generates an image layer of the background image. The generation of the background image by the background drawing unit 143 is not particularly limited, but may be performed, for example, by enlarging or reducing the designated image to fit the size of one page of the score, by tiling the designated image on one page of the score, or by drawing an object (shape or pattern) based on a parameter designated by the user. Alternatively, the background drawing unit 143 may generate an image layer of the background image by drawing an object based on background drawing logic designated by the user. Note that the background image does not exclude cases where it is filled with white or a color with high brightness, but it is preferable to target an image that has a high degree of design freedom other than white or a single color.
[0031] The musical symbol drawing unit 144 draws a plurality of musical symbols (including the above-mentioned staves, notes, etc.) based on the musical score data based on the extracted musical symbol position information 113 and other basic information specified by the user, and generates an image layer of the musical symbols.
[0032] The image synthesis unit 145 superimposes and synthesizes the image layer of the generated background image, the staff container described above, and, if necessary, the image layers of the figure highlights, the melody highlights, and the music symbols, to generate the final score design image 106. The staff container may be placed between the image layer of the background image and the image layer of the music symbols. The figure highlights and the melody highlights may be placed between the image layer of the background image and the image layer of the music symbols, or may be placed in the foreground of the image layer of the music symbols to the extent that the readability of the music symbols is not deteriorated. The image synthesis unit 145 constitutes the image generation unit according to this embodiment.
[0033] Alternatively, if a staff container specifies the application of image processing to a background image, the image processing may be applied to an area of the background image that corresponds to the staff container, or to an object that constitutes the background image and that is overlapped by the staff container. For example, if the background image specifies a polka dot pattern in which disk objects are arranged at random sizes and positions, a process may be applied that relatively brightens the color of disk objects that are overlapped by the staff container.
[0034] The score design image 106 is a still image of a musical score generated by the image synthesis unit 145. The score design image 106 is configured in various image or document formats such as bitmap, JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), GIF (Graphics Interchange Format), PDF (Portable Document Format), etc. The purpose of using the score design image 106 is not particularly limited, and it may be printed on a transfer medium such as paper or a sheet, displayed on a display screen of a personal computer, a smartphone, a tablet, a dedicated electronic score terminal, etc., or inserted into video data.
[0035] As described above, the score design image 106 may be a still image, but is not necessarily limited to a still image. For example, the score design image 106 may have a background image configured as an animation, and the score design image 106 may also be configured with multiple frames. Even in this case, the staff container can visually distinguish the moving background from the musical symbols. When the score design image 106 is configured with multiple frames, the musical figure highlight may have movement, and the staff container may also have changes in shape (the shape changes slightly between frames) or image (for example, the fill color or pattern changes slightly between frames) within a range that does not deteriorate the readability of the musical symbols. In addition, when inserting a score design image into a video or animation with music, the musical figure highlight can start animation in accordance with the timing at which the associated musical note is played.
[0036] The UI unit 190 provides a user interface for the user U to perform input operations to create the above-mentioned score design image 106. Operations for creating the score design image 106 include specifying the target score data 102 to be loaded, specifying a background image (specifying image data and specifying how to arrange it), as well as operations for setting the generation logic and generation parameters of the staff container described above, generation logic and generation parameters of the pattern and musical mood highlights, and specifying the pattern and musical mood to be extracted. When the creation of the score design image 106 is completed, the UI unit 190 also displays it so that the user U can view it.
[0037] Each component constituting the musical score creation system 100 shown in FIG. 2 can be realized on a single computer by a processor such as the CPU 12 of the single computer reading out the program 32 of the musical score creation application from the storage 18 and expanding it in the execution space of the RAM 14. However, the present invention is not limited to such an embodiment. In other embodiments, the components shown in FIG. 2 may be distributed and implemented on multiple computers. In the above description, the musical score creation system 100 has been described as being realized by a musical score creation application executed on a computer operated by a user. However, in other embodiments, the musical score creation system 100 may be configured as a server that provides a user interface that can be displayed and operated on a web browser to a client accessing remotely, and provides a web application that generates an image of a musical score based on specified target musical score data in response to a request via a network and responds with the generated musical score design image 106. In that case, the musical score creation function may be provided as a cloud service such as SaaS (Software as a Service).
[0038] The score image creation process according to the embodiment of the present invention will be described in more detail below with reference to the images generated, with reference to Fig. 3 to Fig. 7. Fig. 3 is a flowchart showing the score creation process executed by the information processing device 10 according to the embodiment of the present invention. Each process shown in Fig. 3 is executed, for example, by a processor in charge of all or corresponding components of the score creation system 100 shown in Fig. 2.
[0039] 3 starts at step S100 in response to an instruction to generate a score design image specifying target score data 102. It is assumed that the target score data 102, background image, staff container generation logic and parameters, highlight generation logic and parameters, and the notes to be extracted are set in advance as user and system defaults. In step S101, the processor reads note pattern matching rules 112 and musical mood extraction rules 111 for the processing described below.
[0040] In step S102, the processor extracts position information of various musical symbols from the target score data 102 by the position information extraction unit 122. Here, musical symbols include lines, clefs, notes, rests, time signatures, accidentals, and performance symbols. Lines include staff lines, ledger lines, bar lines, and final lines. Clefs include treble clef, bass clef, and C clef. Notes are symbols used to write sounds, and represent the relative length of a sound (note value), its time position, and its pitch (pitch). Rests are pairs of notes, and represent the absence of sound. Time signatures are located at the beginning of a score, and indicate the beat of the entire piece or the beat until the beat changes. Accidentals include key signatures and accidentals such as sharp, flat, double sharp, double flat, and natural. Performance symbols include dynamic symbols (forte, piano, mezzo forte, mezzo piano, fortissimo, pianissimo, crescendo, decrescendo, etc.), tempo symbols, expression symbols (animato, delce, etc.), repeat symbols, and modifier symbols, and more specifically, slurs, octave shifts, pedal indications, staccato, accents, tenuto, formata, fingering, ornaments, tempo indications (Allegro, vivace, etc.), and refer to symbols that are important but not essential elements of notation, among the various symbols necessary for performance in musical scores based on staff notation, excluding the above-mentioned staff lines, clefs, time signatures, notes, rests, key signatures, accidentals, etc.
[0041] FIG. 4 shows a data structure used in the musical score creation process according to an embodiment of the present invention. FIG. 4(A) shows the data structure of target musical score data 200 as an example, and the target musical score data 200 shown in FIG. 4(A) is musical score data described in the MusicicXML format. Regarding the contents of the target musical score data 200, a balloon 401 in FIG. 4(A) describes the layout of a system (a group of staves that are read and played simultaneously), and describes the distance between the top line of the system and the bottom line of the previous system. A balloon 402 in FIG. 4(A) describes the layout of the staves, and describes the distance between the bottom line of the previous staff and the top line of the specified staff. An arrow 203 in FIG. 4(A) describes that there are two staves in one voice, and a portion shown by a balloon 204 describes note information. Looking closely at the part shown in the speech bubble 204, the note information shows that it is a note on the top staff of a voice, the X-axis is specified as coordinate "113", the Y-axis is key "D4", it is an eighth note type (eighth) with a length of "6", the stem is pointing upward, the default vertical position of the stem end is 10, and it is the start point of an eighth note beam. The part shown in the speech bubble 202 also shows that a slur is added at the top with a start type. If the musical score data is in the MusicicXML format, in step S102, the position of the musical symbol (including the staff and the note) to be drawn is specified based on the standard drawing logic of MusicicXML.
[0042] 3, the processor then extracts the musical idea information 114 from the target musical score data 200 in step S103, and extracts the musical pattern information 115 from the target musical score data 200 in step S104. The extraction of the musical idea information and the musical pattern information will be described in detail later.
[0043] Steps S105 to S109 are performed for each staff line or lines constituting the system. In the case of a piano staff or grand staff in which one voice part includes two staves, such as the target score data 200 in Fig. 4(A), processing is performed in units of two staff lines, and the following explanation will be given assuming that processing is performed in units of two staves.
[0044] In step S106, the processor uses the container drawing unit 141 to generate a rectangular area surrounding a portion of a musical symbol (e.g., a note head of a musical note) such as one or more staves (two in the example), a clef, a bracket bundling the staves (a brace bundling two staves in the example), multiple notes, and accidentals. More specifically, in step S106, the top, bottom, left, and right ends of each musical symbol or its portion are calculated from the position of the musical symbol, and a rectangular area passing through these points is generated. Here, the portion of the musical symbol that extends outside the area of the staves is referred to as the extra-staff portion, and in a specific embodiment, in step S106, rectangular areas are generated, in particular, for the extra-staff portion of the note, the extra-staff portion of the clef, the extra-staff portion of the accidental, and the extra-staff portion of the bracket connecting the staves.
[0045] 4(B) and 4(C) show data structures generated when extracting the outside-staff portion of a note. Fig. 4(B) shows the data structure of a note list including information on a set of notes extracted from the target score data 102, and Fig. 4(C) shows the data structure of a partial list including a subset of notes having an outside-staff portion selected from the note list. As shown in Fig. 4(B), the target score data 102 describes the pitch (key), note value (length), and stem direction (up / down) of the note, and from this information, the protruding direction and distance from the staff are calculated and added as additional information, and only notes protruding from the staff are selected based on the pitch (key), note value (length), and stem direction, and the area is specified.
[0046] FIG. 5 illustrates a staff container generation process in a score creation process according to an embodiment of the present invention. FIG. 5(A) illustrates a process for generating a rectangular area for generating a staff container, and FIG. 5(A) illustrates a rectangular area surrounding a musical symbol and a portion of the musical symbol. FIG. 5(A) illustrates the following rectangular areas: a staff area 210 combining two staves constituting the system, a brace area 211 connecting the two staves outside the staff area, an area 212 of a portion of a clef that protrudes outside the staff area, and an area of a note symbol that protrudes outside the staff area (outside staff portions of beams 213a-213f, outside staff portions of stems and tails 214a-214f, and outside staff portions of note heads 215a-215f). As shown in FIG. 5(A), the rectangular area of the outside staff portion may be set for each portion by dividing the musical symbol into several portions, but in other embodiments, a rectangular area may be set for the entire musical symbol. In the example shown in Fig. 5(A), a rectangular area is not generated for a slur, which is a performance mark, because readability is not significantly affected if the slur is not placed in a staff container. However, in other embodiments, a rectangular area may also be defined for a slur. In the example shown in Fig. 5(A), a rectangular area is set for the staff area 210 that combines the two staves that make up the system, but a rectangular area may also be set for each staff on each level.
[0047] In step S107, the processor uses the container drawing unit 141 to combine the rectangular areas of the staves and the areas outside the staves to generate a contour area (combined area) that includes the staves area 210 and the areas outside the staves 211-215. Here, the contour area may be an area obtained by combining the rectangular areas of the staves and the areas outside the staves, or may be an area that includes a predetermined amount of blank space on each of the predetermined sides. In step S108, the processor uses the container drawing unit 141 to draw a staff container based on the contour area and, if necessary, on musical idea information. The container drawing unit 141 can draw a staff container by defining a contour that surrounds the contour area, provided that the contour area does not include any line segments parallel to the staves.
[0048] FIG. 5(B) shows an example of a staff container 220 drawn using a predetermined drawing logic. Note that FIG. 5(B) shows a state in which, in addition to the staff container 220, musical symbols 222 are drawn on the staff container 220 (the musical symbols 222 are displayed superimposed). The staff container 220 shown in FIG. 5(B) is defined by a boundary line (broken line) 221 consisting of line segments arranged in a zigzag shape. Note that the drawing example in FIG. 5(B) also shows note pattern highlights 224a and 224b, but the details of note pattern highlights will be described later. In the drawing example shown in FIG. 5(B), the staff container that surrounds the outline area with the broken line 221 is generated so as to satisfy the following conditions (1) to (4): (1) The angle of the line segment of the broken line 221 is set to a random value within a predetermined range (for example, 15 degrees to 30 degrees) with respect to the staff lines, or is set to be perpendicular to the staff lines (i.e., 90 degrees). (2) The vertical distance between the line segment of the broken line 221 and the staff is set to a certain value or more and a certain value or less. (3) The length of the line segment of the broken line 221 that forms an angle with the staff within a predetermined range (for example, 15 degrees to 30 degrees) is set to a certain value or more and a certain value or less. (4) The number of line segments of the broken line 221 perpendicular to the staff should be kept to a minimum.
[0049] The above-mentioned predetermined range, the range of the vertical distance, and the range of the length of the line segment may be given as generation parameters by the user, or may be given as system values or default values, for example.
[0050] In the drawing example shown in Fig. 5(B), after the shape of the staff container is defined as described above, an image (light gray fill) is placed inside the staff container defined above, and a predetermined image (strip-like fill and symbol along the broken line) is placed on the boundary part of the staff container. Fig. 5(C) illustrates a musical score design image 230 obtained by inserting and synthesizing the staff container 220 shown in Fig. 5(B) between the image layers of the background image and the musical symbols.
[0051] When drawing a background other than white, a simple method is to specify the range of the black image area of the staff and musical symbols, expand the range with a certain margin, and outline it with white or the like. In that case, a boundary parallel to the staff (a clear line or a boundary between two areas with light and dark) is generated, which leads to misreading of the staff and ledger lines, reducing the readability of the score. In contrast, by defining the contour line on the condition that it does not include lines parallel to the staff, as shown in Figures 5(B) and (C), it is possible to visually distinguish the musical symbols placed inside from the background image while preventing the readability of the score from being reduced. In addition, it is possible to introduce a high degree of design freedom into the background image while maintaining the readability of the score as a whole. In addition, it is preferable that the staff container itself is visually distinguished from the background, and the shape of the staff container is made a highly flexible shape while being restricted to cover the contour area, thereby improving the design.
[0052] Variations of the staff container generation process will be described below with reference to Fig. 6. Fig. 6 describes a modified example of the staff container generation process in the score creation process according to an embodiment of the present invention. As described above, methods (generation logic) for drawing a staff container while preventing a decrease in the readability of the score include (1) defining a contour line surrounding the outline area under the condition that no line segments parallel to the staff are included, as well as (2) arranging multiple objects near the outline of the outline area, (3) performing a drawing operation on the contour line defined to surround the outline area, or (4) repeatedly arranging objects probabilistically so that the outline area is more likely to be covered. The generation logic of the staff containers (2) to (4) above will be described below with reference to Figs. 6(A) to 6(F).
[0053] 6(A) and (B) explain a method of arranging multiple objects near the contour of the outline region (combined region) described above (2). FIG. 6(A) shows a state in which musical symbols are drawn on the staff container of the drawing example of (2), and FIG. 6(B) is a diagram explaining the generation logic thereof. In the generation logic described in FIG. 6(A) and (B), the container drawing unit 141 generates a staff container by arranging a predetermined graphic object 242 (a disk in this example) as shown by the dotted line in FIG. 6(B) so that the area including a certain margin from the outline region (the area obtained by combining the rectangles 241) obtained in the above step S107 is covered and the boundary of the staff container is within a certain distance D from the outline region. In this example, all boundaries of the staff container are curved by arranging multiple disk objects 242 side by side around the outline region.
[0054] 6(C) and (D) explain a method of performing a drawing operation on the contour line defined to surround the outline region (3) above. FIG. 6(C) shows a state in which a musical symbol is drawn on the staff container of the drawing example (3), and FIG. 6(D) is a diagram explaining the generation logic. In the generation logic explained in FIG. 6(C) and (D), the container drawing unit 141 generates the boundary line 264 of the staff container by performing a certain drawing operation on each line segment of the outline region 261 or the region obtained by surrounding the outline region 261 with a broken line 262 (shown by a dotted line in FIG. 6(D)) at a certain angle (0 degrees, 120 degrees, 240 degrees in this example) so that the line segments protrude outward. In addition, in order to prevent the protruding portion from becoming too large, if the broken line is long, the protruding portion is drawn by dividing it into a certain length. In the examples of Figures 6(C) and (D), a fractal figure (Koch curve) is drawn by repeatedly performing a specified operation so that each line segment of each broken line protrudes outward, and a staff container is drawn without drawing parallel contour lines even in the places where the broken line 262 is parallel to the staff. Here, the Koch curve is a figure obtained by dividing a line segment into thirds and repeatedly drawing equilateral triangles with the two divided points as vertices.
[0055] 6(E) and (F) show a method of repeatedly arranging objects probabilistically so as to increase the probability of the outline region being covered (4). FIG. 6(E) shows a state in which musical symbols are drawn on the staff container of the drawing example of (4), and FIG. 6(F) is a diagram explaining the generation logic. In the generation logic explained in FIG. 6(E) and (F), the container drawing unit 141 generates a staff container by repeatedly drawing a graphic object in the outline region (particularly around the staff and notes) so as to increase the probability of the outline region being covered. In the example of FIG. 6(E) and (F), the staff container is obtained by drawing a semi-transparent expanding speech bubble object in the outline region while randomly moving, rotating, enlarging or reducing it.
[0056] The user may appropriately set which of the generation logics described with reference to Figs. 5 and 6 or other generation logics to use. In addition, detailed generation parameters in each generation logic (e.g., the type and size of the object to be placed (e.g., disk, speech bubble), the distance D, etc.) may be specified by default or may be specified by the user. In addition, the generation logic may be provided as a program incorporated in the application or added to the application as a plug-in (provided by the application vendor or a third party, or written by an individual user) that uses the data of the outline area generated in step S107 (or the data of multiple rectangular areas before combination) and the generation parameters as input, creates the outline of the staff container, places an image in the staff container, or applies image processing. In addition, the generation logic may be configured so that the user can manually adjust the path of the shape, change the image to be placed, or change the color, based on the staff container generated by the generation logic. In this case, it is possible to determine whether or not to accept the change in shape made by the user, with the constraints that the outline area must be covered and that the staff of a length equal to or longer than a reference must not include a horizontal line segment.
[0057] 5 and 6, the staff container makes it possible to visually distinguish the musical symbols placed inside from the background image while preventing the staff container from reducing the readability of the musical score, and makes it possible to introduce a high degree of design freedom into the background image while maintaining the readability of the musical score as a whole. In addition, the staff container itself is visually distinguished from the background, and the design of the staff container can be made to have a high degree of design freedom while being constrained to cover the outline area, thereby improving the design.
[0058] The staff containers shown in Figures 5 and 6 have a single outline that encompasses two staves on a piano staff. However, the present invention is not limited to such an embodiment, and may have holes between the bar lines and final lines between the two staves, and a background may be drawn there. However, even in this case, it is preferable that no holes are provided within a single staff from the viewpoint of readability.
[0059] The above drawing example illustrates the case where the staff container is drawn according to the outline area, but as described above, in step S108, the staff container may be drawn based on the musical idea information in addition to the outline area. Below, the extraction process of musical idea information in step S103 and the processing of the staff container according to the musical idea information will be explained.
[0060] As described above, the musical mood information extraction unit 124 performs statistical processing based on the target musical score data 102, and assigns musical mood attributes indicating the musical mood to a predetermined unit based on the musical mood extraction rules 111. Information for determining musical mood attributes includes statistical information based on tempo, number of notes, volume, articulation, distribution of note values, distribution of note pitches, distribution of note differences or intervals, or expression symbols. The musical mood attributes include information on major or minor key, chords, and information on chords translated into emotions (e.g., fun, sad, stylish, etc.), information on the distribution of note lengths or articulation translated into musical mood (e.g., light, heavy, strict, smooth, etc.).
[0061] Examples of the musical idea extraction rule 111 include a rule that, using the ratio of staccato notes in a predetermined unit (e.g., a bar) as statistical information, assigns a musical idea attribute of "light" to the predetermined unit (e.g., a bar) if the ratio of staccato notes is a certain value (e.g., 20%) or more, and a rule that, using the ratio of notes in a slur as statistical information, assigns a musical idea attribute of "smooth" to the predetermined unit (e.g., a bar) if the ratio of notes in a slur is a certain value (e.g., 50%) or more. The musical idea information extraction unit 124 can extract the ratio of staccato notes and the ratio of notes in a slur as statistical information to be evaluated based on such musical idea extraction rule 111, and assign the corresponding musical idea attribute to bars that meet the conditions.
[0062] In this case, in step S108, the processor can change the image to be placed in the area corresponding to the predetermined unit in the staff container or the image processing to be applied, based on the musical attribute, by the container drawing unit 141. The image to be placed or the image processing to be applied is defined in association with each musical attribute. For example, the brightness of the area corresponding to the measure in the staff container to which the musical attribute of "light" is assigned can be changed to be brighter than the brightness of the other areas. In the above example, the unit is the measure, but this is not limited to this. Attributes may be assigned to the page, staff line, or note list, and drawn as effects on the background image of the entire page, the staff container, and the vicinity of the measures. This makes it possible to visualize musical features such as the structure of a piece of music. For example, by changing the drawing method of the background and objects only for the middle part (B) of a ternary form (ABA), it becomes possible to visually convey where the middle part begins and ends. In addition to changing the drawing method of the staff container according to the musical information described above, the image (shape or font) of the notes may be changed in a predetermined unit.
[0063] 3 again, steps S110 to S112 are performed for each set of notes to which the note pattern information or the musical mood information is added. In step S111, the processor causes the highlight drawing unit 142 to draw a highlight (a note pattern highlight or a musical mood highlight).
[0064] The process of extracting note pattern information in step S104 and the process of drawing note pattern highlights according to the note pattern information in step S111 will be described below with reference to FIGS. 2, 4(D) and 7.
[0065] The note pattern matching rules 112 are defined using attributes of notes, such as pitch (including absolute value designation such as "A" near 441 Hz, and relative value designation from the starting note or the preceding and succeeding notes), note length, time position within a measure, presence or absence of slur, identity of part (instrument, right hand, left hand of piano, etc.), identity of voice (main melody, secondary melody, etc. within the same part), and other attributes of notes. More specifically, the note pattern matching rules 112 can be defined by the data structure shown in FIG. 4(D), which designates the following matching attributes for each note. The note pattern matching rules 112 shown in FIG. 4(D) are composed of multiple rows, each of which has the items of reference note (reference note), lowest key from the reference note, highest key, minimum length, and maximum length. The reference note is set to the following values: "None", which indicates that there is no pitch matching; "First", which indicates that the first note is the reference; "Last", which indicates that the last note is the reference; "Prev", which indicates that the previous note is the reference; and "Next", which indicates that the next note is the reference. The lowest key and highest key specify the relative key to the reference note in semitones, and the allowable range of pitches from the reference note. The minimum length and maximum length specify the allowable range of note lengths from the reference note, in bar multiples of the note length. For example, 0.125 means an eighth note, and 0.5 means a half note.
[0066] In step S104, the tone pattern matching unit 126 applies tone pattern extraction rules as shown in FIG. 4(D) to the note list defined in the target score data 102, and assigns tone pattern attributes to sets of notes that match each matching attribute. It is not necessarily limited to those that match all matching attributes, and the tone pattern matching rules 112 can set a certain number of notes (for example, 4) counting from the starting point of the reference note being "None" as mandatory match targets, and the rest as optional match targets. Here, the mandatory match refers to a condition that must be matched when extracting as a tone pattern, and the optional match target refers to a condition to be extracted as a tone pattern up to that range when matched. A set of matching notes is extracted by matching the tone pattern matching rules 112 with the notes for each voice part of the target score data 102. FIG. 7(A) shows an example of a matching result 300 of the tone pattern matching rules. In FIG. 7(A), a group of notes that match any of the musical patterns is marked with rectangles 302a to 302d, and the names of the matched notes and the key based on the starting point are written in the speech bubbles associated with the rectangles.
[0067] As described above, in musical expressions, a motif is generally modified slightly each time it appears. Therefore, image-based matching is insufficient, and in order to comprehensively extract motifs, matching in a manner that can absorb ambiguity regarding the length and pitch of notes is desirable. Therefore, matching of motifs is desirably based on one or both of (1) the specification of the tolerance range for the difference in pitch between multiple notes and (2) the specification of the similarity or difference based on the pitch of the specified sample motif. As an example, in the Prelude of Book 1 of Bach's Well-Tempered Clavier, the motif "So-Do-Mi-So-Do-Mi" in the right hand appears 64 times in total. However, in a method that does not consider the ambiguity of the pitch, such as the method of Patent Document 1, only 12 points match. In contrast, in the motif matching according to this embodiment, a sheet music file is used to specify the range of the relative key difference between the previous and following notes, allowing for more ambiguity, making it possible to extract all 64 points in the above example music piece.
[0068] In step S111, the processor draws a musical pattern highlight that modifies a plurality of notes included in the note set based on the position information of each note included in the note set to which the musical mood information 114 and the musical pattern information 115 are added by the highlight drawing unit 142. The musical mood attribute and the musical pattern attribute to be added are associated in advance with the corresponding highlight modification. Figs. 7(B) and (C) show a musical pattern highlight 310 as an example. The musical pattern highlight 310 shown in Figs. 7(B) and (C) is for decorating a predetermined note set (phrase), and is a concentric arrangement of objects 314 in the shape of a "L" or a shape in which two arcs are joined, starting from the head of each note 312 in the note set. Note that the concentric musical pattern highlight shown in Figs. 7(B) and (C) is an example, and is not particularly limited. In other embodiments, images may be arranged using the position information of a line segment connecting any pair of notes or a line segment connecting adjacent pairs of notes. Alternatively, decoration may be performed by changing the note image itself instead of decoration. FIG. 7(D) illustrates a score design image 320 obtained by inserting and synthesizing the musical pattern highlight 310 shown in FIG. 7(B) between a background image and musical symbols. In this way, by modifying a set of notes that match a predetermined musical pattern, a design that modifies notes customized to the musical score data is possible. As for the highlight, a generation logic that uses the position information of each note in the set of notes as input to draw a predetermined object may be built into the application or provided as a plug-in, similar to the staff container. As described above, the drawing of the highlight also affects the readability of the surrounding musical symbols, so a tool that judges the readability of a specified image or a generated image may be provided.
[0069] Referring again to FIG. 2, in a preferred embodiment, the musical score creation system 100 includes a frequent musical pattern extraction unit 130 and a similar musical pattern generation unit 132, as described above. As described above, musical patterns defined in the musical pattern matching rules 112 include (1) designated musical patterns, (2) frequent musical patterns, and (3) similar musical patterns. The designated musical patterns include musical patterns predefined in the application or user-defined musical patterns. The frequent musical pattern extraction unit 130 can extract frequent musical patterns using the target musical score data 102. The frequent musical patterns are based on the musical pattern matching rules reverse-generated from the target musical score data 102. For example, the musical pattern matching rules can be reverse-generated by counting the frequency of occurrence of a sequence of musical notes (a sequence consisting of a musical pitch (height) or a musical pitch difference (interval) from a predetermined reference (such as the beginning of a measure) and a musical value (length)) that appears in the target musical score data 102 or the general musical score data 104, and extracting those with a certain frequency or more as candidates. Here, the general musical score data 104 is a collection of a large number of musical score data in the same format as the target musical score data 102. The frequent musical pattern extraction unit 130 reverse generates musical pattern matching rules from notes that appear in the target musical score data 102, and extracts, as frequent musical patterns, musical patterns that have a large absolute number of corresponding locations in the reverse-generated musical pattern matching rules, musical patterns that are relatively frequent in terms of the number of notes in the reverse-generated musical pattern matching rules, or musical patterns that are frequent compared to their appearance rate in the general musical score data 104.
[0070] The similar sound pattern generation unit 132 generates rules that define sound patterns similar to existing sound patterns such as the designated sound patterns and frequent sound patterns of the sound pattern matching rules 112. The similar sound pattern generation unit 132 applies at least one of the following to the predefined sound pattern matching rules and the reverse-generated sound pattern matching rules: expanding the allowable range of matching items (expanding the range of keys defined by the highest key and the lowest key, expanding the range of lengths defined by the minimum length and the maximum length, or both) and omitting some of the matching items (such as reducing the number of required matching targets), to generate a similar sound pattern that is similar to the designated sound pattern or the frequent sound pattern.
[0071] Furthermore, while the above description has focused on melodic highlights, melodic highlights may also be drawn for groups of notes on a line (stage) basis, measure basis, beat basis, or note basis, based on the melodic information described above.
[0072] 3 again, in step S113, the processor draws the background using the background drawing unit 143, and generates an image layer of the background. In step S114, the processor draws musical symbols based on the position information of various musical symbols using the musical symbol drawing unit 144, and draws an image layer of musical symbols based on the musical score data. In step S115, the processor combines the image layers of the background, staff container, note figure highlight, and musical symbols using the image composition unit 145, and generates the musical score design image 106, and ends this process in step S116.
[0073] In FIG. 3, the score creation process is described in a specific order, but the order of each process in the score creation process is not limited to the order shown in FIG. 3, and some processes may be changed in order or may be performed substantially simultaneously. In addition, some of the processes shown in FIG. 3 may be omitted, and processes other than the processes shown in FIG. 3 may be performed at any timing. For example, in the above-mentioned embodiment, the generation logic and generation parameters of the staff container and the highlights are described as being specified in advance, but after the score design image 106 is generated, some of the staff containers, the note highlights, and the melody highlights may be changed, and the score design image 106 may be updated under the changed conditions, or the user may manually adjust the path of the staff container, manually adjust the coordinates of the highlights, or change the decorations.
[0074] As described above, according to the embodiments of the present invention, it is possible to provide an information processing device, a score generation method, and a program that are capable of creating score designs with a high degree of freedom while maintaining the readability of the score information with a minimal amount of manual work.
[0075] The above-mentioned functional units can be realized by a computer-executable program written in a legacy programming language such as assembler, C, C++, C#, Java (registered trademark), or an object-oriented programming language, and can be stored on a device-readable recording medium such as ROM, EEPROM, EPROM, flash memory, flexible disk, CD-ROM, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, Blu-ray disk, SD card, or MO, or distributed via a telecommunications line.
[0076] Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to the above-described embodiments, and may be modified, added, changed, deleted, and otherwise within the scope of what can be conceived by a person skilled in the art. Any embodiment is within the scope of the present invention as long as it achieves the functions and effects of the present invention. [Explanation of symbols]
[0077] 10...information processing device, 12...CPU, 14...RAM, 16...ROM, 18...storage, 20...display, 22...input device, 24...network interface, 30...OS, 32...music score creation application program, 100...music score creation system, 102...target musical score data, 104...general musical score data, 106...music score design image, 111...music idea extraction rule, 112...note pattern matching rule, 113...position information, 114...music idea information, 115...note pattern information, 120...information extraction unit, 122...position information extraction unit, 124...music idea information extraction unit, 126...note pattern information extraction unit, 130...frequent note pattern extraction unit, 132...similar note pattern generation unit, 1 40...drawing section, 141...container drawing section, 142...highlight drawing section, 143...background drawing section, 144...musical symbol drawing section, 145...image synthesis section, 2000...target music score data, 210...staff area, 211-215...rectangular area outside the staff, 220...staff container, 221...boundary, 222...musical symbol, 230...musical score design image, 241, 261, 281...rectangle, 242...disk object, 262...broken line, 264...boundary, 282...speech bubble object, 300...matching result, 302...rectangle, 310...musical pattern highlight, 312...musical note, 314...concentric circle pattern object, 320...musical score design image 320
Claims
1. an extraction unit that extracts position information of a staff area and position information of a plurality of notes based on musical score data including at least information representing pitches; a container drawing unit that draws a staff container that includes one or more extra-staff portions of the notes that extend outside the staff area and the staff area based on position information of the staff area and position information of the notes, and visually distinguishes musical symbols arranged inside the staff container from a background; an image generating unit that generates an image of the score on which the staff container and a plurality of musical symbols based on the score data are drawn; An information processing device comprising:
2. 2. The information processing device according to claim 1, wherein the extraction unit further extracts positional information of clefs, alterations, and parentheses connecting multiple staves, and the container drawing unit generates a combined area by combining the areas of the clefs, the alterations, and the parentheses extending outside the area of the staves, the area of the one or more outside-staves, and the areas of the multiple staves connected by the parentheses, and draws the staff container based on the combined area.
3. 3. The information processing device according to claim 2, wherein the container drawing unit draws the staff container by: (1) defining a contour line surrounding the combined area on the condition that the contour line does not include any line segment parallel to the staff lines; (2) placing a plurality of objects in the vicinity of the contour line of the combined area; (3) performing a drawing operation on the contour line defined to surround the combined area; or (4) repeatedly placing objects probabilistically so as to increase the probability that the combined area is covered.
4. 3. The information processing device of claim 2, wherein the image of the musical score includes a background image, and the staff container specifies at least one of placing an image in the foreground of the background image and applying an image processing to an area of the background image corresponding to the staff container.
5. a musical pattern matching unit that matches a plurality of musical patterns based on the score data, based on one or both of (1) a tolerance range for pitch differences between a plurality of musical notes and (2) a similarity or difference based on pitch with respect to a musical pattern of a designated sample, and extracts one or a plurality of note sets each including a plurality of notes that match any one of the plurality of musical patterns; a highlight drawing unit that draws, for each of the one or more note sets, a musical pattern highlight that modifies a plurality of notes included in the note set based on position information of the plurality of notes included in the note set; The information processing apparatus according to claim 2 , further comprising:
6. 6. The information processing device according to claim 5, wherein the plurality of note patterns are each either a designated note pattern based on a predefined note pattern matching rule, a frequent note pattern based on a note pattern matching rule reverse-generated from the musical score data, or a similar note pattern similar to the designated note pattern or the frequent note pattern based on a note pattern matching rule obtained by applying at least one of an expansion of a tolerance range of matching items and an omission of a part of matching items to the predefined note pattern matching rule and the reverse-generated note pattern matching rule.
7. The information processing device according to claim 5, wherein the musical pattern highlight specifies at least one of changing images of multiple notes included in the set of notes, arranging images based on the note heads, stems, or lines connecting the note heads or stems of multiple notes included in the set of notes, and applying image processing based on the note heads, stems, or lines connecting the note heads or stems of multiple notes included in the set of notes.
8. 3. The information processing device according to claim 2, further comprising a musical idea information extraction unit which performs statistical processing based on the musical score data and assigns musical idea attributes indicating the musical idea of a predetermined unit to a predetermined unit that matches one or more musical idea extraction rules, wherein the information for determining the musical idea attributes includes statistical information based on tempo, number of notes, volume, articulation, note value distribution, pitch distribution, pitch difference or interval distribution, or expression mark, and wherein the container drawing unit changes an image to be placed in an area of the staff container corresponding to the predetermined unit or an image processing to be applied, based on the musical idea attributes.
9. 6. The information processing device according to claim 5, further comprising a musical idea information extraction unit that performs statistical processing based on the musical score data and assigns a musical idea attribute indicating the musical idea of the predetermined unit to a predetermined unit that matches one or more musical idea extraction rules, the information for determining the musical idea attribute includes statistical information based on tempo, number of notes, volume, articulation, note value distribution, pitch distribution, pitch difference or interval distribution, or expression mark, the highlight drawing unit draws musical idea highlights that modify a plurality of musical symbols included in the predetermined unit based on the musical idea attribute and position information of a plurality of musical symbols included in the predetermined unit, and the musical idea highlight specifies at least one of image placement and application of image processing based on a plurality of musical symbols included in the predetermined unit.
10. One or more computers extracting position information of a staff area and position information of a plurality of notes based on musical score data including at least information representing pitches; drawing a staff container based on position information of the staff area and position information of the plurality of notes, the staff container including one or more extra-staff portions of the plurality of notes that extend outside the staff area and the staff area, and visually distinguishing musical symbols arranged inside the staff container from a background; generating an image of the score in which the staff container and a plurality of musical symbols based on the score data are rendered; A method for generating music notation.
11. The extracting step further includes a step of extracting position information of a clef, an accidental symbol, and a bracket connecting a plurality of staves, and the drawing step includes a step of generating a combined area by combining the areas of the clef, the accidental symbol, and the bracket that extend outside the area of the staves, the area of the one or more outside staves, and the areas of the plurality of staves connected by the bracket, and a step of drawing the staff container based on the combined area.
11. The method of claim 10, comprising:
12. 12. The method of claim 11, wherein the drawing step includes the steps of: (1) defining a contour line surrounding the combined region, with the contour line not including any line segment parallel to the staff lines; (2) placing a plurality of objects in the vicinity of the contour line of the combined region; (3) performing a drawing operation on the contour line defined to surround the combined region; or (4) probabilistically repeatedly placing objects to increase the probability that the combined region is covered.
13. A program readable and executable by a processor, the program, when read and executed by the processor, causing the processor to: extracting position information of the staff area and position information of a plurality of notes based on musical score data including at least information representing pitch; drawing a staff container based on position information of the staff area and position information of the plurality of notes, the staff container including one or more extra-staff portions of the plurality of notes extending outside the staff area and the staff area, and visually distinguishing musical symbols arranged therein from a background; a program that generates an image of a musical score in which the staff container and a plurality of musical symbols based on the musical score data are drawn;
14. The program further includes: Extract the position information of clefs, accidentals, and brackets connecting multiple staves, generating a combined region by combining the regions of the clef, the accidental and the brackets that extend outside the region of the staff, the region of the one or more extra-staff portions and the regions of the staves connected by the brackets; Rendering the staff container based on the bounding region; The program according to claim 13.
15. The program causes the processor to:
15. The program of claim 14, further comprising: (1) defining a contour line surrounding the combined area, with the condition that the contour line does not include any line segments parallel to the staff lines; (2) placing a plurality of objects in the vicinity of the contour line of the combined area; (3) performing a drawing operation on the contour line defined to surround the combined area; or (4) repeatedly placing objects probabilistically to increase the probability that the combined area will be covered.
16. an extracting unit that extracts position information of a plurality of notes based on musical score data including at least information representing pitches; a musical pattern matching unit that matches a plurality of musical patterns based on the score data, based on one or both of (1) a tolerance range for pitch differences between a plurality of musical notes and (2) a similarity or difference based on pitch with respect to a musical pattern of a designated sample, and extracts one or a plurality of note sets each including a plurality of notes that match any one of the plurality of musical patterns; a highlight drawing unit that draws, for each of the one or more note sets, a musical pattern highlight that modifies a plurality of notes included in the note set based on position information of the plurality of notes included in the note set; an image generating unit that generates an image of a musical score on which a plurality of musical symbols based on the musical score data are drawn together with the musical pattern highlights; An information processing device comprising:
17. an extracting unit that extracts position information of a plurality of notes based on musical score data including at least information representing pitches; a musical mood information extraction unit that performs statistical processing based on the musical score data and assigns a musical mood attribute indicating the musical mood of a predetermined unit to a predetermined unit that matches one or more musical mood extraction rules; an image drawing unit that draws a visual object based on the position information of the plurality of notes and the musical mood attribute; an image generating unit that generates an image of a musical score on which a plurality of musical symbols based on the musical score data are drawn together with the object; An information processing device comprising:
Citation Information
Patent Citations
Score processing method and image processing apparatus
JP2009153068A
