A drum spectrum recognition method and system based on image segmentation and template matching

Abstract

The present invention provides a method and system for drum score recognition based on image segmentation and template matching in the field of music score recognition technology. The method comprises the following steps: step S10, obtaining a drum score file, converting the drum score file into a drum score picture and preprocessing it; step S20, identifying the score lines in the preprocessed drum score picture; step S30, selecting each note on the score line based on column cutting to obtain a plurality of note pictures; step S40, creating a plurality of note template pictures, matching the note pictures with the note template pictures through the OpenCV template matching algorithm to recognize the note pictures. The advantage of the present invention is that the drum score can be accurately recognized under the premise of ensuring the integrity of the drum score image information.

Description

Technical Field

[0001] The present invention relates to the technical field of music score recognition, and in particular to a method and system for recognizing drum scores based on image segmentation and template matching. Background Art

[0002] Musical notation is a method of recording music using symbols. For music beginners, knowing how to read music notation is crucial. If you cannot master the music notation, you cannot play beautiful music. Therefore, there is a need to recognize music notation for auxiliary teaching, similar to how an English reading machine recognizes English words.

[0003] Traditionally, a binary method is used to identify drum music scores, that is, the drum music score image is converted into a binary image, and each area of ​​the binary image is horizontally corrected using a cross-correlation function to obtain a horizontal image. The horizontal image is then line-located and the line is deleted to obtain an image with the line deleted. The stem is located in the image with the line deleted to obtain the coordinate data of the stem primitive, and the image of the stem is deleted. The tail bridge is located in the image with the stem deleted to obtain the coordinate data of the tail bridge primitive, and the image of the tail bridge is deleted. The note head is located in the image with the tail bridge deleted to obtain the coordinate data of the note head primitive. The data of the stem, the tail bridge and the note head are matched to complete the recognition of the drum music score.

[0004] From the above content, it can be seen that the traditional method needs to continuously delete images to complete the recognition of drum sheet music, resulting in the loss of image information of drum sheet music. Therefore, how to provide a drum sheet music recognition method and system based on image segmentation and template matching to accurately recognize drum sheet music while ensuring the integrity of drum sheet music image information has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for drum score recognition based on image segmentation and template matching, so as to accurately recognize drum score while ensuring the integrity of drum score image information.

[0006] In a first aspect, the present invention provides a drum spectrum recognition method based on image segmentation and template matching, comprising the following steps:

[0007] Step S10, obtaining a drum score file, converting the drum score file into a drum score image and performing preprocessing;

[0008] Step S20, identifying the spectrum lines in the pre-processed drum spectrum picture;

[0009] Step S30, selecting each note on the spectral line based on the column cutting frame to obtain a plurality of note images;

[0010] Step S40, create a plurality of musical note template images, and match the musical note images with the musical note template images through the OpenCV template matching algorithm to identify the musical note images.

[0011] Furthermore, the step S10 is specifically as follows:

[0012] A drum score file in a PDF format is obtained, the drum score file is converted into a drum score image in a PNG format, and the drum score image is pre-processed for noise reduction.

[0013] Furthermore, the step S20 is specifically as follows:

[0014] The spectral lines in the preprocessed drum score picture are identified by machine learning technology, and the spectral lines are grouped and numbered based on the spacing regularity between the spectral lines.

[0015] Furthermore, the step S30 is specifically as follows:

[0016] Through machine learning technology, each note on each group of the spectral lines is selected based on column cutting to obtain a number of note images, and each of the note images is numbered.

[0017] Furthermore, the step S40 is specifically as follows:

[0018] Create several musical note template images, mark each musical note template image, and set the threshold of OpenCV template matching;

[0019] Through the OpenCV template matching algorithm and the threshold, each note image and the note template image are matched in order according to the number, so as to automatically identify each note image and associate the corresponding annotation.

[0020] In a second aspect, the present invention provides a drum score recognition system based on image segmentation and template matching, comprising the following modules:

[0021] A drum score file acquisition and conversion module is used to acquire a drum score file, convert the drum score file into a drum score image and perform preprocessing;

[0022] A spectrum line recognition module, used for recognizing spectrum lines in the pre-processed drum spectrum picture;

[0023] A note image segmentation module is used to select each note on the spectral line based on the column cutting frame to obtain a plurality of note images;

[0024] The template matching module is used to create several musical note template images, and match the musical note images with the musical note template images through the OpenCV template matching algorithm to identify the musical note images.

[0025] Furthermore, the drum set score file acquisition conversion module is specifically:

[0026] A drum score file in a PDF format is obtained, the drum score file is converted into a drum score image in a PNG format, and the drum score image is pre-processed for noise reduction.

[0027] Furthermore, the spectral line identification module is specifically:

[0028] The spectral lines in the preprocessed drum score picture are identified by machine learning technology, and the spectral lines are grouped and numbered based on the spacing regularity between the spectral lines.

[0029] Furthermore, the musical note image segmentation module is specifically:

[0030] Through machine learning technology, each note on each group of the spectral lines is selected based on column cutting to obtain a number of note images, and each of the note images is numbered.

[0031] Furthermore, the template matching module is specifically:

[0032] Create several musical note template images, mark each musical note template image, and set the threshold of OpenCV template matching;

[0033] Through the OpenCV template matching algorithm and the threshold, each note image and the note template image are matched in order according to the number, so as to automatically identify each note image and associate the corresponding annotation.

[0034] The advantages of the present invention are:

[0035] By identifying the staves in the drum score image, and then selecting the notes on the staves based on the column cutting frame to obtain several note images, the note images and the pre-created note template images are matched through the OpenCV template matching algorithm to identify the note images. Compared with the traditional binary recognition method, there is no need to continuously delete images to complete the recognition of the drum score, and by setting the threshold of OpenCV template matching, the recognition problem of overlapping notes is effectively overcome. Finally, under the premise of ensuring the integrity of the drum score image information, the accuracy of drum score recognition is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0037] Figure 1 The present invention is a flow chart of a drum spectrum recognition method based on image segmentation and template matching.

[0038] Figure 2 It is a structural schematic diagram of a drum spectrum recognition system based on image segmentation and template matching of the present invention.

[0039] Figure 3 It is an effect diagram of the note frame selection of the present invention. DETAILED DESCRIPTION

[0040] The technical solution in the embodiments of the present application has the following overall idea: a number of note images are obtained by selecting the notes on the score line through column cutting frames, and the note images are matched with pre-created note template images through the OpenCV template matching algorithm to identify the note images. There is no need to delete the image to complete the recognition of the drum score, and the problem of recognizing overlapping notes is effectively overcome by setting the threshold of OpenCV template matching, so as to achieve accurate recognition of the drum score while ensuring the integrity of the drum score image information.

[0041] Please refer to Figures 1 to 3 As shown, a preferred embodiment of a drum spectrum recognition method based on image segmentation and template matching of the present invention comprises the following steps:

[0042] Step S10, obtaining a drum score file, converting the drum score file into a drum score image and performing preprocessing; the drum score file can be obtained from any channel, for example, from MuseScore;

[0043] Step S20, identifying the musical score lines in the pre-processed drum score image; the purpose of identifying the musical score lines is to facilitate subsequent note cutting (box selection);

[0044] Step S30, selecting each note on the spectral line based on the column cutting frame to obtain a plurality of note images;

[0045] Step S40, create a plurality of musical note template images, and match the musical note images with the musical note template images through the OpenCV template matching algorithm to identify the musical note images.

[0046] The step S10 is specifically as follows:

[0047] A drum score file in PDF format is obtained, the drum score file is converted into a drum score image in PNG format, and the drum score image is pre-processed for noise reduction. By pre-processing the drum score image for noise reduction, the accuracy of note recognition and matching can be greatly improved.

[0048] The step S20 is specifically as follows:

[0049] The spectral lines in the preprocessed drum score picture are identified by machine learning technology, and the spectral lines are grouped and numbered based on the spacing regularity between the spectral lines.

[0050] The step S30 is specifically as follows:

[0051] Through machine learning technology, each note on each group of the spectral lines is framed based on column cutting to obtain a number of note images, and each note image is numbered. Since the distance between the notes is uneven, that is, it is not evenly spaced, the processing effect of row cutting is not very good, and column cutting can overcome this problem. Row cutting refers to using a fixed height to frame the notes for identification, which will result in obtaining a small part of the information of multiple notes at a time, making row cutting far less easy to identify notes than column cutting; column cutting is exactly the opposite of row cutting. Column cutting uses a fixed width frame to select notes, so that it can select most of the characters of one or two notes, greatly improving the recognition accuracy of notes.

[0052] By grouping and numbering the staves and numbering the note images, when the drum score is finally recognized, it is known which group of staves and which note is recognized.

[0053] The step S40 is specifically as follows:

[0054] Create several musical note template images, mark each musical note template image, and set the threshold of OpenCV template matching;

[0055] Through the OpenCV template matching algorithm and the threshold, the note images and the note template images are matched in order according to the number, so as to automatically identify the note images and associate the corresponding annotations. Since the note heads may overlap, and the notes with overlapping note heads cannot be well identified and matched traditionally, the present invention can effectively overcome the recognition problem of overlapping notes by setting the threshold of OpenCV template matching, thereby greatly improving the accuracy of drum score recognition.

[0056] A preferred embodiment of a drum spectrum recognition system based on image segmentation and template matching of the present invention comprises the following modules:

[0057] A drum score file acquisition and conversion module is used to acquire a drum score file, convert the drum score file into a drum score image and perform preprocessing; the drum score file can be acquired from any channel, for example, from MuseScore;

[0058] A line recognition module is used to recognize the line in the pre-processed drum music picture; the purpose of recognizing the line is to facilitate the subsequent note cutting (box selection);

[0059] A note image segmentation module is used to select each note on the spectral line based on the column cutting frame to obtain a plurality of note images;

[0060] The template matching module is used to create a number of musical note template images, and match the musical note images with the musical note template images through the OpenCV template matching algorithm to identify the musical note images.

[0061] The drum score file acquisition and conversion module is specifically:

[0062] A drum score file in PDF format is obtained, the drum score file is converted into a drum score image in PNG format, and the drum score image is pre-processed for noise reduction. By pre-processing the drum score image for noise reduction, the accuracy of note recognition and matching can be greatly improved.

[0063] The spectral line identification module is specifically:

[0064] The spectral lines in the preprocessed drum score picture are identified by machine learning technology, and the spectral lines are grouped and numbered based on the spacing regularity between the spectral lines.

[0065] The musical note image segmentation module is specifically:

[0066] Through machine learning technology, each note on each group of the spectral lines is framed based on column cutting to obtain a number of note images, and each note image is numbered. Since the distance between the notes is uneven, that is, it is not evenly spaced, the processing effect of row cutting is not very good, and column cutting can overcome this problem. Row cutting refers to using a fixed height to frame the notes for identification, which will result in obtaining a small part of the information of multiple notes at a time, making row cutting far less easy to identify notes than column cutting; column cutting is exactly the opposite of row cutting. Column cutting uses a fixed width frame to select notes, so that it can select most of the characters of one or two notes, greatly improving the recognition accuracy of notes.

[0067] By grouping and numbering the staves and numbering the note images, when the drum score is finally recognized, it is known which group of staves and which note is recognized.

[0068] The template matching module is specifically:

[0069] Create several musical note template images, mark each musical note template image, and set the threshold of OpenCV template matching;

[0070] Through the OpenCV template matching algorithm and the threshold, the note images and the note template images are matched in order according to the number, so as to automatically identify the note images and associate the corresponding annotations. Since the note heads may overlap, and the notes with overlapping note heads cannot be well identified and matched traditionally, the present invention can effectively overcome the recognition problem of overlapping notes by setting the threshold of OpenCV template matching, thereby greatly improving the accuracy of drum score recognition.

[0071] In summary, the advantages of the present invention are:

[0072] By identifying the staves in the drum score image, and then selecting the notes on the staves based on the column cutting frame to obtain several note images, the note images and the pre-created note template images are matched through the OpenCV template matching algorithm to identify the note images. Compared with the traditional binary recognition method, there is no need to continuously delete images to complete the recognition of the drum score, and by setting the threshold of OpenCV template matching, the recognition problem of overlapping notes is effectively overcome. Finally, under the premise of ensuring the integrity of the drum score image information, the accuracy of drum score recognition is greatly improved.

[0073] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A drum spectrum recognition method based on image segmentation and template matching, characterized in that: The steps include: Step S10, obtaining a drum score file, converting the drum score file into a drum score image and performing preprocessing; Step S20, identifying the spectrum lines in the pre-processed drum spectrum picture; Specifically: Identify the spectral lines in the preprocessed drum score image by machine learning technology, and group and number the spectral lines based on the interval regularity between the spectral lines; Step S30, selecting each note on the spectral line based on the column cutting frame to obtain a plurality of note pictures; specifically: Using machine learning technology, each note on each group of the staves is selected based on column cutting to obtain a plurality of note images, and each of the note images is numbered; Step S40, creating a plurality of musical note template images, matching the musical note images with the musical note template images by using the OpenCV template matching algorithm to identify the musical note images; specifically: Create several musical note template images, mark each musical note template image, and set the threshold of OpenCV template matching; Through the OpenCV template matching algorithm and the threshold, each note image and the note template image are matched in order according to the number, so as to automatically identify each note image and associate the corresponding annotation.

2. The drum spectrum recognition method based on image segmentation and template matching as claimed in claim 1, characterized in that: The step S10 is specifically as follows: A drum score file in a PDF format is obtained, the drum score file is converted into a drum score image in a PNG format, and the drum score image is pre-processed for noise reduction.

3. A drum music score recognition system based on image segmentation and template matching, characterized in that: Includes the following modules: A drum score file acquisition and conversion module is used to acquire a drum score file, convert the drum score file into a drum score image and perform preprocessing; A spectral line recognition module is used to recognize spectral lines in the preprocessed drum music picture; the spectral line recognition module is specifically: Identify the spectral lines in the preprocessed drum score image by machine learning technology, and group and number the spectral lines based on the interval regularity between the spectral lines; The note image segmentation module is used to select each note on the spectral line based on the column cutting frame to obtain a number of note images; the note image segmentation module is specifically: Using machine learning technology, each note on each group of the staves is selected based on column cutting to obtain a plurality of note images, and each of the note images is numbered; The template matching module is used to create several musical note template images, and match the musical note images with the musical note template images through the OpenCV template matching algorithm to identify the musical note images; the template matching module is specifically: Create several musical note template images, mark each musical note template image, and set the threshold of OpenCV template matching; Through the OpenCV template matching algorithm and the threshold, each note image and the note template image are matched in order according to the number, so as to automatically identify each note image and associate the corresponding annotation.

4. A drum music score recognition system based on image segmentation and template matching as claimed in claim 3, characterized in that: The drum score file acquisition and conversion module is specifically: A drum score file in a PDF format is obtained, the drum score file is converted into a drum score image in a PNG format, and the drum score image is pre-processed for noise reduction.

Images