A printed body pipa simple score recognition method and system

Abstract

This invention provides a method and system for recognizing printed pipa (Chinese lute) sheet music in the field of pipa sheet music recognition technology. The method includes: Step S1, acquiring symbol images of pipa sheet music and classifying and saving each symbol image; Step S2, extracting symbol features after grayscale binarization of each symbol image, and writing each symbol feature into a KD tree to obtain a symbol feature file; Step S3, acquiring a printed pipa sheet music image, performing grayscale binarization on the pipa sheet music image, selecting each symbol in the pipa sheet music image by contour detection and slicing it to obtain slice images; Step S4, matching the symbols in the slice images using the symbol feature file; Step S5, performing rhythm analysis on each matched symbol; Step S6, storing each symbol after rhythm analysis in the form of an array to complete the recognition of the pipa sheet music image. The advantage of this invention is that it greatly improves the recognition accuracy of printed pipa sheet music.

Description

Technical Field

[0001] This invention relates to the field of pipa musical notation recognition technology, and in particular to a method and system for recognizing printed pipa musical notation. Background Technology

[0002] With the development of computers, they are now widely used in the music field, covering multiple sub-fields such as music entertainment, creation, and teaching, including multimedia playback, music production software, music composition software, and music recognition, all of which rely on the digitization of sheet music.

[0003] Digitizing sheet music can reduce paper materials and improve space utilization. It can also fully utilize computer processing power to improve the speed of sheet music retrieval. The conversion of musical information into digital information can serve as a reference for the reverse conversion process of these two types of information. It can be applied to music production and can also be used to form robot bands, laying the foundation for artificial intelligence to play musical instruments.

[0004] The premise of digitizing sheet music is to recognize the sheet music. However, traditional recognition methods are mainly for staff notation, while the composition of pipa simplified notation is complex and has many differences from the structure of staff notation. As a result, the recognition accuracy of traditional recognition methods is not satisfactory when applied to pipa simplified notation.

[0005] Therefore, how to provide a method and system for recognizing printed pipa sheet music to improve the accuracy of printed pipa sheet music recognition has become an urgent technical problem to be solved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for recognizing printed pipa sheet music, thereby improving the accuracy of printed pipa sheet music recognition.

[0007] In a first aspect, the present invention provides a method for recognizing printed pipa (Chinese lute) sheet music, comprising the following steps:

[0008] Step S1: Obtain the symbol images of the pipa simplified score, and classify and save each symbol image;

[0009] Step S2: After performing grayscale binarization on each symbol image, extract symbol features from each symbol image, convert each symbol feature into binary format and write it into a KD tree to obtain a symbol feature file;

[0010] Step S3: Obtain the printed image of the pipa musical score. After performing grayscale binarization on the pipa musical score image, select each symbol in the pipa musical score image by contour detection and slice it to obtain a sliced ​​image.

[0011] Step S4: Match the symbols in the sliced ​​image using the symbol feature file;

[0012] Step S5: Perform rhythm analysis on each of the matched symbols;

[0013] Step S6: Store each of the symbols after rhythm analysis in the form of an array to complete the recognition of the pipa simplified score image.

[0014] Furthermore, step S1 specifically includes:

[0015] Obtain symbol images of pipa sheet music from music websites or pipa teaching books, and classify and save the symbol images according to the symbol types of pipa sheet music.

[0016] The symbol types include measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, and duration.

[0017] Furthermore, step S2 specifically includes:

[0018] After grayscale processing of each symbol image, binarization processing is performed on each symbol image.

[0019] Next, contour detection is performed on each of the symbol images to obtain the contour data of the symbol. Horizontal and vertical projections are then performed on each of the symbol images to obtain the horizontal projection peak interval and the vertical projection peak interval, thereby obtaining symbol features including the contour data, the horizontal projection peak interval, and the vertical projection peak interval.

[0020] The symbol features are then converted into binary format and written into a KD tree to obtain a symbol feature file.

[0021] Furthermore, step S4 specifically includes:

[0022] An error range is set, and the slice images are grouped based on the symbol shape. The symbols in one slice image of each group are matched using the symbol feature file to obtain the matching results. After verifying the matching results based on the error range, each slice image is named based on the matching results.

[0023] Furthermore, step S5 specifically includes:

[0024] The matched symbols are subjected to rhythm analysis, including measure partitioning, processing of notes and fingering information within measures, time value analysis, and recognition of multi-line musical scores.

[0025] Secondly, the present invention provides a printed pipa (Chinese lute) musical notation recognition system, comprising the following modules:

[0026] The symbol image acquisition module is used to acquire symbol images of pipa simplified musical notation and to classify and save the symbol images accordingly.

[0027] The symbol feature extraction module is used to perform grayscale binarization on each of the symbol images, extract symbol features from each of the symbol images, convert each of the symbol features into binary format and write them into a KD tree to obtain a symbol feature file.

[0028] The symbol slicing module is used to acquire the printed pipa musical score image. After grayscale binarization of the pipa musical score image, each symbol in the pipa musical score image is selected by contour detection and sliced ​​to obtain a sliced ​​image.

[0029] The symbol matching module is used to match symbols in the sliced ​​image using the symbol feature file;

[0030] The rhythm analysis module is used to perform rhythm analysis on each of the matched symbols.

[0031] The symbol storage module is used to store each symbol after rhythm analysis in the form of an array to complete the recognition of the pipa musical score image.

[0032] Furthermore, the symbol image acquisition module is specifically used for:

[0033] Obtain symbol images of pipa sheet music from music websites or pipa teaching books, and classify and save the symbol images according to the symbol types of pipa sheet music.

[0034] The symbol types include measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, and duration.

[0035] Furthermore, the symbol feature extraction module is specifically used for:

[0036] After grayscale processing of each symbol image, binarization processing is performed on each symbol image.

[0037] Next, contour detection is performed on each of the symbol images to obtain the contour data of the symbol. Horizontal and vertical projections are then performed on each of the symbol images to obtain the horizontal projection peak interval and the vertical projection peak interval, thereby obtaining symbol features including the contour data, the horizontal projection peak interval, and the vertical projection peak interval.

[0038] The symbol features are then converted into binary format and written into a KD tree to obtain a symbol feature file.

[0039] Furthermore, the symbol matching module is specifically used for:

[0040] An error range is set, and the slice images are grouped based on the symbol shape. The symbols in one slice image of each group are matched using the symbol feature file to obtain the matching results. After verifying the matching results based on the error range, each slice image is named based on the matching results.

[0041] Furthermore, the rhythm analysis module is specifically used for:

[0042] The matched symbols are subjected to rhythm analysis, including measure partitioning, processing of notes and fingering information within measures, time value analysis, and recognition of multi-line musical scores.

[0043] The advantages of this invention are:

[0044] 1. By acquiring and classifying the symbol images of the pipa (Chinese lute) sheet music, and then performing grayscale binarization on each symbol image, the symbol features are extracted from each image and converted into binary format, which is then written into a KD tree to obtain a symbol feature file. Next, the printed pipa sheet music image is acquired and grayscale binarized. Then, each symbol in the pipa sheet music image is selected using contour detection and sliced ​​to obtain slice images. The symbols in the slice images are matched using the symbol feature file, and rhythm analysis is performed on the matched symbols. Finally, the rhythm-analyzed symbols are stored in array form to complete the recognition of the pipa sheet music image. Since grayscale binarization of the pipa sheet music image before recognition reduces computation and interference, matching the slice images with symbol features extracted from the symbol images is more effective than image template matching in recognizing deformed symbols. Rhythm analysis of the matched symbols—that is, reorganizing the symbols of the determined type in an orderly manner—makes the recognition results readable, ultimately greatly improving the recognition accuracy of printed pipa sheet music.

[0045] 2. By reducing the amount of computation through grayscale binarization, grouping each slice image by symbol shape, and then matching the symbols in one slice image of each group, the matching efficiency is effectively improved, thereby greatly improving the recognition efficiency of printed pipa sheet music.

[0046] 3. The symbols described after rhythm analysis are stored in the form of arrays, which is simple, clear and easy to read later. Attached Figure Description

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] Figure 1 This is a flowchart of a method for recognizing printed pipa sheet music according to the present invention.

[0049] Figure 2This is a schematic diagram of the structure of a printed pipa (Chinese lute) musical notation recognition system according to the present invention. Detailed Implementation

[0050] The technical solution in this application embodiment has the following general idea: Before recognition, the image of the pipa sheet music is processed by grayscale binarization to reduce interference. The slice image is matched by the symbol features extracted from the symbol image. Compared with image template matching, it can better identify deformed symbols. By performing rhythm analysis on each matched symbol, that is, reorganizing the symbols with determined symbol types in an orderly manner, the recognition result becomes readable, thereby improving the recognition accuracy of printed pipa sheet music.

[0051] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the printed pipa (Chinese lute) musical notation recognition method of the present invention includes the following steps:

[0052] Step S1: Obtain the symbol images of the pipa simplified score, and classify and save each symbol image;

[0053] Step S2: After performing grayscale binarization on each symbol image, extract symbol features from each symbol image, convert each symbol feature into binary format and write it into a KD tree to obtain a symbol feature file;

[0054] Step S3: Obtain the printed image of the pipa musical score. After performing grayscale binarization on the pipa musical score image, select each symbol in the pipa musical score image by contour detection and slice it to obtain slice images. Performing grayscale binarization on the pipa musical score image can effectively reduce the amount of computer computation and improve the efficiency of computer data processing. Each slice image contains only one symbol.

[0055] Step S4: Match the symbols in the sliced ​​image using the symbol feature file;

[0056] Step S5: Perform rhythm analysis on each of the matched symbols, that is, reorganize the symbols with the determined symbol types in an orderly manner to make the recognition results readable;

[0057] Step S6: Store each of the symbols after rhythm analysis in the form of an array to complete the recognition of the pipa simplified score image.

[0058] The symbols analyzed in the rhythm are stored in the form of arrays, which is simple and clear and easy to read later. For example, they can be stored in an array structure of [measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, duration].

[0059] Step S1 specifically involves:

[0060] Obtain symbol images of pipa sheet music from music websites or pipa teaching books, and classify and save the symbol images according to the symbol types of pipa sheet music.

[0061] The symbol types include measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, and duration.

[0062] Step S2 specifically involves:

[0063] After grayscale processing of each symbol image, binarization processing is performed on each symbol image.

[0064] Next, contour detection is performed on each of the symbol images to obtain the contour data of the symbol. Horizontal and vertical projections are then performed on each of the symbol images to obtain the horizontal projection peak interval and the vertical projection peak interval, thereby obtaining symbol features including the contour data, the horizontal projection peak interval, and the vertical projection peak interval.

[0065] The symbol features are then converted into binary format and written into a KD tree to obtain a symbol feature file.

[0066] Step S4 specifically involves:

[0067] An error range is set, and the slice images are grouped based on the symbol shape. The symbols in one slice image of each group are matched using the symbol feature file to obtain the matching results. After verifying the matching results based on the error range, each slice image is named based on the matching results.

[0068] That is, the symbol features of each slice image are compared with the existing symbol feature files, and within the error range, it is determined that the symbol to be identified belongs to the same category as the existing symbols.

[0069] By reducing computational load through grayscale binarization, grouping each slice image by symbol shape, and then matching the symbols in one slice image of each group, the matching efficiency is effectively improved, thereby greatly enhancing the recognition efficiency of printed pipa sheet music.

[0070] Step S5 specifically involves:

[0071] The matched symbols are subjected to rhythm analysis, including measure partitioning, processing of notes and fingering information within measures, time value analysis, and recognition of multi-line musical scores.

[0072] The specific method of section partitioning is as follows: filter out the bar lines in the symbols, divide the pipa sheet music image into regions according to the bar lines and the vertical spacing, and treat the symbols inside the section as a group for identification, taking one section as a unit.

[0073] The processing of notes and fingering information within a measure is specifically as follows: For the symbols within a measure, a semantic analysis method is used to first scan horizontally to find the lowest number in each measure, and then scan vertically for each number to sequentially identify the sharps and flats and fingering in the upper left, the truncation line, the dot of the lower octave, the string order and position below, and the dot of the higher octave and fingering information above.

[0074] The specific time value analysis is as follows: when detecting the subtraction line below the number, the first time value is recorded; when the recognition of a single measure ends, the time value is merged and calculated a second time; finally, for adjacent measures, it is determined whether a note has a time signature extension across measures, and a third time value calculation is performed to determine the time value.

[0075] The multi-line musical score recognition is specifically as follows: When dividing the score into measures, if the height of the part containing valid information in the vertical area where a number is located exceeds twice that of other areas, it is determined that there are multiple lines of musical score in that column; First, the two parallel lines of numbers are treated separately, and the extra numbers are marked as detected; When checking the numbers on the baseline, if an extra number is detected above, the extra number is identified; Similarly, the symbols in the upper left corner are checked first, then the symbols below, and finally the symbols above; When checking the symbols below, if a number is found, the loop is exited because the numbers below have already been checked, and finally the multi-line musical score information (symbols) is integrated.

[0076] A preferred embodiment of the printed pipa (Chinese lute) musical notation recognition system of the present invention includes the following modules:

[0077] The symbol image acquisition module is used to acquire symbol images of pipa simplified musical notation and to classify and save the symbol images accordingly.

[0078] The symbol feature extraction module is used to perform grayscale binarization on each of the symbol images, extract symbol features from each of the symbol images, convert each of the symbol features into binary format and write them into a KD tree to obtain a symbol feature file.

[0079] The symbol slicing module is used to acquire printed pipa (Chinese lute) musical notation images. After grayscale binarization of the pipa musical notation images, each symbol in the pipa musical notation images is selected by contour detection and sliced ​​to obtain sliced ​​images. Grayscale binarization of the pipa musical notation images can effectively reduce the amount of computer computation and improve the efficiency of computer data processing. Each sliced ​​image contains only one symbol.

[0080] The symbol matching module is used to match symbols in the sliced ​​image using the symbol feature file;

[0081] The rhythm analysis module is used to perform rhythm analysis on each of the matched symbols, that is, to reorganize the symbols with determined symbol types in an orderly manner to make the recognition results readable;

[0082] The symbol storage module is used to store each symbol after rhythm analysis in the form of an array to complete the recognition of the pipa musical score image.

[0083] The symbols analyzed in the rhythm are stored in the form of arrays, which is simple and clear and easy to read later. For example, they can be stored in an array structure of [measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, duration].

[0084] The symbol image acquisition module is specifically used for:

[0085] Obtain symbol images of pipa sheet music from music websites or pipa teaching books, and classify and save the symbol images according to the symbol types of pipa sheet music.

[0086] The symbol types include measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, and duration.

[0087] The symbol feature extraction module is specifically used for:

[0088] After grayscale processing of each symbol image, binarization processing is performed on each symbol image.

[0089] Next, contour detection is performed on each of the symbol images to obtain the contour data of the symbol. Horizontal and vertical projections are then performed on each of the symbol images to obtain the horizontal projection peak interval and the vertical projection peak interval, thereby obtaining symbol features including the contour data, the horizontal projection peak interval, and the vertical projection peak interval.

[0090] The symbol features are then converted into binary format and written into a KD tree to obtain a symbol feature file.

[0091] The symbol matching module is specifically used for:

[0092] An error range is set, and the slice images are grouped based on the symbol shape. The symbols in one slice image of each group are matched using the symbol feature file to obtain the matching results. After verifying the matching results based on the error range, each slice image is named based on the matching results.

[0093] That is, the symbol features of each slice image are compared with the existing symbol feature files, and within the error range, it is determined that the symbol to be identified belongs to the same category as the existing symbols.

[0094] By reducing computational load through grayscale binarization, grouping each slice image by symbol shape, and then matching the symbols in one slice image of each group, the matching efficiency is effectively improved, thereby greatly enhancing the recognition efficiency of printed pipa sheet music.

[0095] The rhythm analysis module is specifically used for:

[0096] The matched symbols are subjected to rhythm analysis, including measure partitioning, processing of notes and fingering information within measures, time value analysis, and recognition of multi-line musical scores.

[0097] The specific method of section partitioning is as follows: filter out the bar lines in the symbols, divide the pipa sheet music image into regions according to the bar lines and the vertical spacing, and treat the symbols inside the section as a group for identification, taking one section as a unit.

[0098] The processing of notes and fingering information within a measure is specifically as follows: For the symbols within a measure, a semantic analysis method is used to first scan horizontally to find the lowest number in each measure, and then scan vertically for each number to sequentially identify the sharps and flats and fingering in the upper left, the truncation line, the dot of the lower octave, the string order and position below, and the dot of the higher octave and fingering information above.

[0099] The specific time value analysis is as follows: when detecting the subtraction line below the number, the first time value is recorded; when the recognition of a single measure ends, the time value is merged and calculated a second time; finally, for adjacent measures, it is determined whether a note has a time signature extension across measures, and a third time value calculation is performed to determine the time value.

[0100] The multi-line musical score recognition is specifically as follows: When dividing the score into measures, if the height of the part containing valid information in the vertical area where a number is located exceeds twice that of other areas, it is determined that there are multiple lines of musical score in that column; First, the two parallel lines of numbers are treated separately, and the extra numbers are marked as detected; When checking the numbers on the baseline, if an extra number is detected above, the extra number is identified; Similarly, the symbols in the upper left corner are checked first, then the symbols below, and finally the symbols above; When checking the symbols below, if a number is found, the loop is exited because the numbers below have already been checked, and finally the multi-line musical score information (symbols) is integrated.

[0101] In summary, the advantages of this invention are as follows:

[0102] 1. By acquiring and classifying the symbol images of the pipa (Chinese lute) sheet music, and then performing grayscale binarization on each symbol image, the symbol features are extracted from each image and converted into binary format, which is then written into a KD tree to obtain a symbol feature file. Next, the printed pipa sheet music image is acquired and grayscale binarized. Then, each symbol in the pipa sheet music image is selected using contour detection and sliced ​​to obtain slice images. The symbols in the slice images are matched using the symbol feature file, and rhythm analysis is performed on the matched symbols. Finally, the rhythm-analyzed symbols are stored in array form to complete the recognition of the pipa sheet music image. Since grayscale binarization of the pipa sheet music image before recognition reduces computation and interference, matching the slice images with symbol features extracted from the symbol images is more effective than image template matching in recognizing deformed symbols. Rhythm analysis of the matched symbols—that is, reorganizing the symbols of the determined type in an orderly manner—makes the recognition results readable, ultimately greatly improving the recognition accuracy of printed pipa sheet music.

[0103] 2. By reducing the amount of computation through grayscale binarization, grouping each slice image by symbol shape, and then matching the symbols in one slice image of each group, the matching efficiency is effectively improved, thereby greatly improving the recognition efficiency of printed pipa sheet music.

[0104] 3. The symbols described after rhythm analysis are stored in the form of arrays, which is simple, clear and easy to read later.

[0105] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A printed body of a plectrum simple score recognition method, characterized in that: Includes the following steps: Step S1: Obtain the symbol images of the pipa simplified score, and classify and save each symbol image; Step S2: After performing grayscale binarization on each symbol image, extract symbol features from each symbol image, convert each symbol feature into binary format and write it into a KD tree to obtain a symbol feature file; Specifically: After grayscale processing of each symbol image, binarization processing is performed on each symbol image. Next, contour detection is performed on each of the symbol images to obtain the contour data of the symbol. Horizontal and vertical projections are then performed on each of the symbol images to obtain the horizontal projection peak interval and the vertical projection peak interval, thereby obtaining symbol features including the contour data, the horizontal projection peak interval, and the vertical projection peak interval. The symbol features are then converted into binary format and written into a KD tree to obtain a symbol feature file; Step S3: Obtain the printed image of the pipa musical score. After performing grayscale binarization on the pipa musical score image, select each symbol in the pipa musical score image by contour detection and slice it to obtain a sliced ​​image. Step S4: Match the symbols in the sliced ​​image using the symbol feature file; specifically: A set error range is defined, and the slice images are grouped based on the symbol shape. The symbols in one slice image of each group are matched using the symbol feature file to obtain the matching results. After verifying the matching results based on the error range, each slice image is named based on the matching results. Step S5: Perform rhythm analysis on each of the matched symbols; Step S6: Store each of the symbols after rhythm analysis in the form of an array to complete the recognition of the pipa simplified score image.

2. The printed plectrum tablature recognition method of claim 1, wherein: Step S1 specifically involves: Obtain symbol images of pipa sheet music from music websites or pipa teaching books, and classify and save the symbol images according to the symbol types of pipa sheet music. The symbol types include measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, and duration.

3. The printed plectrum tablature recognition method of claim 1, wherein: Step S5 specifically involves: The matched symbols are subjected to rhythm analysis, including measure partitioning, processing of notes and fingering information within measures, time value analysis, and recognition of multi-line musical scores.

4. A printed pipa (Chinese lute) musical notation recognition system, characterized in that: Includes the following modules: The symbol image acquisition module is used to acquire symbol images of pipa simplified musical notation and to classify and save the symbol images accordingly. The symbol feature extraction module is used to perform grayscale binarization on each of the symbol images, extract symbol features from each of the symbol images, convert each of the symbol features into binary format and write them into a KD tree to obtain a symbol feature file. The symbol slicing module is used to acquire the printed pipa musical score image. After grayscale binarization of the pipa musical score image, each symbol in the pipa musical score image is selected by contour detection and sliced ​​to obtain a sliced ​​image. The symbol matching module is used to match symbols in the sliced ​​image using the symbol feature file; The rhythm analysis module is used to perform rhythm analysis on each of the matched symbols. The symbol storage module is used to store each of the symbols after rhythm analysis in the form of an array to complete the recognition of the pipa musical score image; The symbol feature extraction module is specifically used for: After grayscale processing of each symbol image, binarization processing is performed on each symbol image. Next, contour detection is performed on each of the symbol images to obtain the contour data of the symbol. Horizontal and vertical projections are then performed on each of the symbol images to obtain the horizontal projection peak interval and the vertical projection peak interval, thereby obtaining symbol features including the contour data, the horizontal projection peak interval, and the vertical projection peak interval. The symbol features are then converted into binary format and written into a KD tree to obtain a symbol feature file; The symbol matching module is specifically used for: An error range is set, and the slice images are grouped based on the symbol shape. The symbols in one slice image of each group are matched using the symbol feature file to obtain the matching results. After verifying the matching results based on the error range, each slice image is named based on the matching results.

5. The printed pipa musical notation recognition system as described in claim 4, characterized in that: The symbol image acquisition module is specifically used for: Obtain symbol images of pipa sheet music from music websites or pipa teaching books, and classify and save the symbol images according to the symbol types of pipa sheet music. The symbol types include measure number, numerical note name and pitch, sharps and flats, fingering, position, string order, fingering, fermata, and duration.

6. The printed pipa musical notation recognition system as described in claim 4, characterized in that: The rhythm analysis module is specifically used for: The matched symbols are subjected to rhythm analysis, including measure partitioning, processing of notes and fingering information within measures, time value analysis, and recognition of multi-line musical scores.

Images