YOLOv5-based drum set music score identification method and system

Abstract

The invention provides a drum set music score recognition method and system based on YOLOv5 in the technical field of drum set music score recognition, and the method comprises the steps: S1, obtaining a large number of music score images of drum set music scores, and carrying out the standardized preprocessing of each music score image; step S2, marking music symbols in each preprocessed music score image through Labe l Img to obtain a corresponding label vector; s3, generating a corresponding label vector file based on each label vector of each music score image, storing each music score image and the label vector file, and constructing a data set based on each music score image and the label vector file; s4, creating a drum music score recognition model based on the YOLOv5 module and the convolution attention module; s5, training a drum set music score recognition model through the data set; and S6, performing drum set music score identification through the trained drum set music score identification model. The drum set music score recognition method has the advantage that the drum set music score recognition precision is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of drum music score recognition, and in particular to a drum music score recognition method and system based on YOLOv5. Background Art

[0002] With the rapid development of digital music production and analysis technology, a large number of musical works can be digitized, which facilitates the storage, retrieval and analysis of musical works. As a special music symbol system, drum music has unique challenges and practical value, so there is a need to identify drum music.

[0003] Although there are relatively complete music score recognition processes on the market, mainly for piano, drums, as a relatively niche but increasingly popular percussion instrument, have their own unique musical symbols and notation rules. There is no complete method for drum score recognition. If the music score recognition process for five-line notation is directly applied, the recognition accuracy is unsatisfactory, which makes the information extraction and transcription of drum score more difficult.

[0004] Therefore, how to provide a drum music score recognition method and system based on YOLOv5 to improve the accuracy of drum music score recognition has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for drum music score recognition based on YOLOv5, so as to improve the accuracy of drum music score recognition.

[0006] In a first aspect, the present invention provides a drum music score recognition method based on YOLOv5, comprising the following steps:

[0007] Step S1, obtaining a large number of music score images of drum music scores, and performing standardized preprocessing on each of the music score images;

[0008] Step S2, labeling the music symbols in each of the preprocessed music score images through LabelImg to obtain a corresponding label vector;

[0009] Step S3, generating a corresponding label vector file based on each label vector of each of the music score images, storing each of the music score images and the label vector file, and constructing a data set based on each of the music score images and the label vector file;

[0010] Step S4, creating a drum music score recognition model based on the YOLOv5 module and the convolutional attention module;

[0011] Step S5, training a drum music score recognition model using the data set;

[0012] Step S6: deploy the trained drum music score recognition model on the server, and call the deployed drum music score recognition model online to recognize the drum music score.

[0013] Furthermore, the step S1 is specifically as follows:

[0014] A large number of music score images of drum music scores are obtained, and standardized preprocessing including at least format conversion, cropping, scaling, rotation and noise reduction is performed on each of the music score images.

[0015] Furthermore, the step S2 is specifically as follows:

[0016] The music symbols in each of the preprocessed music score images are labeled in sequence through LabelImg to obtain corresponding label vectors including note categories and coordinate information, where the note categories are drum, hi-hat, cymbal, or rest.

[0017] Furthermore, the step S3 is specifically as follows:

[0018] Generate a corresponding label vector file based on each label vector of each of the music score images, set the same file name for each of the music score images and the corresponding label vector files, store each of the music score images in a pre-created image folder, store each of the label vector files in a pre-created label folder, and construct a data set based on each of the music score images and the label vector files.

[0019] Furthermore, in step S4, the YOLOv5 module is used to identify music symbols from the music score image; the convolutional attention module is provided with a channel attention unit and a spatial attention unit, which are respectively used to extract channel features and spatial features from the music score image to assist the YOLOv5 module in music symbol recognition;

[0020] The step S5 is specifically as follows:

[0021] The data set is divided into a training set, a validation set and a test set based on a ratio of 8:1:1; after training the drum music score recognition model for 500 epochs using the training set, the drum music score recognition model is verified using the validation set, and based on the verification result, the model parameters of the drum music score recognition model are fine-tuned to optimize the crossover ratio and confidence, and then the verified drum music score recognition model is tested using the test set to complete the training of the drum music score recognition model.

[0022] In a second aspect, the present invention provides a drum music score recognition system based on YOLOv5, comprising the following modules:

[0023] A music score image acquisition module, used to acquire a large number of music score images of drum music scores and perform standardized preprocessing on each of the music score images;

[0024] A music score image labeling module, used to label the music symbols in each of the pre-processed music score images through LabelImg to obtain a corresponding label vector;

[0025] A data set construction module, used to generate a corresponding label vector file based on each label vector of each of the music score images, store each of the music score images and the label vector file, and construct a data set based on each of the music score images and the label vector file;

[0026] A drum music score recognition model creation module is used to create a drum music score recognition model based on the YOLOv5 module and the convolutional attention module;

[0027] A drum music score recognition model training module, used for training the drum music score recognition model using the data set;

[0028] The drum music score recognition module is used to deploy the trained drum music score recognition model on the server, and call the deployed drum music score recognition model online to recognize the drum music score.

[0029] Furthermore, the music score image acquisition module is specifically used for:

[0030] A large number of music score images of drum music scores are obtained, and standardized preprocessing including at least format conversion, cropping, scaling, rotation and noise reduction is performed on each of the music score images.

[0031] Furthermore, the music score image annotation module is specifically used for:

[0032] The music symbols in each of the preprocessed music score images are labeled in sequence through LabelImg to obtain corresponding label vectors including note categories and coordinate information, where the note categories are drum, hi-hat, cymbal, or rest.

[0033] Furthermore, the data set construction module is specifically used for:

[0034] Generate a corresponding label vector file based on each label vector of each of the music score images, set the same file name for each of the music score images and the corresponding label vector files, store each of the music score images in a pre-created image folder, store each of the label vector files in a pre-created label folder, and construct a data set based on each of the music score images and the label vector files.

[0035] Furthermore, in the drum set music score recognition model creation module, the YOLOv5 module is used to recognize music symbols from the music score image; the convolutional attention module is provided with a channel attention unit and a spatial attention unit, which are respectively used to extract channel features and spatial features from the music score image to assist the YOLOv5 module in music symbol recognition;

[0036] The drum music score recognition model training module is specifically used for:

[0037] The data set is divided into a training set, a validation set and a test set based on a ratio of 8:1:1; after training the drum music score recognition model for 500 epochs using the training set, the drum music score recognition model is verified using the validation set, and based on the verification result, the model parameters of the drum music score recognition model are fine-tuned to optimize the crossover ratio and confidence, and then the verified drum music score recognition model is tested using the test set to complete the training of the drum music score recognition model.

[0038] The advantages of the present invention are:

[0039] A large number of music score images of drum music scores are obtained and standardized preprocessed, and the music symbols in each preprocessed music score image are annotated by LabelImg to obtain the corresponding label vector, and then the corresponding label vector file is generated based on each label vector of each music score image, and each music score image and label vector file are stored, and a data set is constructed based on each music score image and label vector file; then a drum music score recognition model is created based on the YOLOv5 module and the convolutional attention module, and the drum music score recognition model is trained with the data set, and the trained drum music score recognition model is deployed on the server, and the online call part The deployed drum music score recognition model is used to recognize drum music scores; that is, the drum music score recognition model is trained by obtaining a label vector file by annotating music symbols and a data set constructed by music score images. The drum music score recognition model is equipped with a YOLOv5 module and a convolutional attention module. The convolutional attention module extracts channel features and spatial features from the music score image through the channel attention unit and the spatial attention unit respectively, and then assists the YOLOv5 module in recognizing music symbols through the channel features and the spatial features, so that the drum music score recognition model can focus on music symbols, thereby greatly improving the accuracy of drum music score recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0041] Figure 1 It is a flow chart of a drum music score recognition method based on YOLOv5 of the present invention.

[0042] Figure 2 It is a structural schematic diagram of a drum music score recognition system based on YOLOv5 in the present invention. DETAILED DESCRIPTION

[0043] The technical solution in the embodiment of the present application has the following overall idea: a drum music score recognition model is trained by obtaining a label vector file obtained by annotating music symbols and a data set constructed by music score images, and the drum music score recognition model is provided with a YOLOv5 module and a convolutional attention module. The convolutional attention module extracts channel features and spatial features from the music score image through a channel attention unit and a spatial attention unit respectively, and then assists the YOLOv5 module in music symbol recognition through the channel features and spatial features, so that the drum music score recognition model can focus on music symbols to improve the accuracy of drum music score recognition.

[0044] Please refer to Figure 1 to Figure 2 As shown, a preferred embodiment of a drum music score recognition method based on YOLOv5 of the present invention comprises the following steps:

[0045] Step S1, obtaining a large number of music score images of drum music scores, and performing standardized preprocessing on each of the music score images;

[0046] Step S2, labeling the music symbols in each of the preprocessed music score images through LabelImg to obtain a corresponding label vector;

[0047] Step S3, generating a corresponding label vector file based on each label vector of each of the music score images, storing each of the music score images and the label vector file, and constructing a data set based on each of the music score images and the label vector file;

[0048] Step S4: creating a drum music score recognition model based on the YOLOv5 module and the convolutional attention module; the drum music score recognition model is used to perform end-to-end music symbol recognition on the music score image;

[0049] Step S5, training a drum music score recognition model using the data set;

[0050] Step S6: deploy the trained drum music score recognition model on the server, and call the deployed drum music score recognition model online to recognize the drum music score.

[0051] The step S1 is specifically as follows:

[0052] A large number of music score images of drum music scores are obtained, and standardized preprocessing including at least format conversion, cropping, scaling, rotation and noise reduction is performed on each of the music score images.

[0053] During specific implementation, the music score image can be directly obtained from the Musescore website. The MuseScore website allows users to upload their music score images and share them with others; the format of the music score image is png or jpg.

[0054] The step S2 is specifically as follows:

[0055] The music symbols in each of the preprocessed music score images are labeled in turn through LabelImg to obtain a corresponding label vector including note category and coordinate information, wherein the note category is Drum, Hi-Hat, Crash or Rest.

[0056] Since the information contained in the score image does not include the specific coordinate information of each musical symbol in the drum score, it needs to be marked. Since there is no definite notation method for drum scores, the representation of each drum score will have different deviations according to each notation software, each set of drum instrument configuration and each musician's notation habits, but generally follows its spatial principle: instruments with higher pitches are placed at the upper position and written at the upper part of the five-line score, such as crash; instruments with lower pitches are placed at the lower position and written at the lower part of the five-line score, such as bass drum and hi-hat for foot pedal.

[0057] The step S3 is specifically as follows:

[0058] Generate a corresponding label vector file based on each label vector of each score image, set the same file name for each score image and the corresponding label vector file, store each score image in a pre-created image folder (Images), store each label vector file in a pre-created label folder (Labels), and construct a data set based on each score image and label vector file. The format of the label vector file is txt.

[0059] In step S4, the YOLOv5 module is used to identify music symbols from the music score image; the convolutional attention module is provided with a channel attention unit and a spatial attention unit, which are respectively used to extract channel features and spatial features from the music score image to assist the YOLOv5 module in music symbol recognition;

[0060] Since the music symbols that need to be recognized in the present invention are all small targets, and considering that the area ratio of background-irrelevant elements and the object to be recognized (music symbols) is quite different when the YOLOv5 module performs target detection, a convolutional attention module (CBAM) is added to make the recognition attention of the drum music score recognition model more focused on the music symbols.

[0061] CBAM is a new attention mechanism proposed on the basis of SE (Squeeze-and-Excitation). SE improves the expressiveness of the model by adaptively learning the weight of each channel. CBAM introduces the spatial attention mechanism on the basis of SE to further improve the expressiveness of the model. CBAM includes the Channel Attention Module and the Spatial Attention Module; CBAM is added to the 11th layer of the YOLOv5 module.

[0062] The step S5 is specifically as follows:

[0063] The data set is divided into a training set, a validation set and a test set based on a ratio of 8:1:1; after training the drum music score recognition model for 500 epochs using the training set, the drum music score recognition model is verified using the validation set, and based on the verification result, the model parameters of the drum music score recognition model are fine-tuned to optimize the crossover ratio and confidence, and then the verified drum music score recognition model is tested using the test set to complete the training of the drum music score recognition model.

[0064] A preferred embodiment of a drum music score recognition system based on YOLOv5 of the present invention includes the following modules:

[0065] A music score image acquisition module, used to acquire a large number of music score images of drum music scores and perform standardized preprocessing on each of the music score images;

[0066] A music score image labeling module, used to label the music symbols in each of the pre-processed music score images through LabelImg to obtain a corresponding label vector;

[0067] A data set construction module, used to generate a corresponding label vector file based on each label vector of each of the music score images, store each of the music score images and the label vector file, and construct a data set based on each of the music score images and the label vector file;

[0068] A drum music score recognition model creation module is used to create a drum music score recognition model based on a YOLOv5 module and a convolutional attention module; the drum music score recognition model is used to perform end-to-end music symbol recognition on a music score image;

[0069] A drum music score recognition model training module, used for training the drum music score recognition model using the data set;

[0070] The drum music score recognition module is used to deploy the trained drum music score recognition model on the server, and call the deployed drum music score recognition model online to recognize the drum music score.

[0071] The music score image acquisition module is specifically used for:

[0072] A large number of music score images of drum music scores are obtained, and standardized preprocessing including at least format conversion, cropping, scaling, rotation and noise reduction is performed on each of the music score images.

[0073] During specific implementation, the music score image can be directly obtained from the Musescore website. The MuseScore website allows users to upload their music score images and share them with others; the format of the music score image is png or jpg.

[0074] The music score image annotation module is specifically used for:

[0075] The music symbols in each of the preprocessed music score images are labeled in turn through LabelImg to obtain a corresponding label vector including note category and coordinate information, wherein the note category is Drum, Hi-Hat, Crash or Rest.

[0076] Since the information contained in the score image does not include the specific coordinate information of each musical symbol in the drum score, it needs to be marked. Since there is no definite notation method for drum scores, the representation of each drum score will have different deviations according to each notation software, each set of drum instrument configuration and each musician's notation habits, but generally follows its spatial principle: instruments with higher pitches are placed at the upper position and written at the upper part of the five-line score, such as crash; instruments with lower pitches are placed at the lower position and written at the lower part of the five-line score, such as bass drum and hi-hat for foot pedal.

[0077] The data set construction module is specifically used for:

[0078] Generate a corresponding label vector file based on each label vector of each score image, set the same file name for each score image and the corresponding label vector file, store each score image in a pre-created image folder (Images), store each label vector file in a pre-created label folder (Labels), and construct a data set based on each score image and label vector file. The format of the label vector file is txt.

[0079] In the drum music score recognition model creation module, the YOLOv5 module is used to recognize music symbols from the music score image; the convolutional attention module is provided with a channel attention unit and a spatial attention unit, which are respectively used to extract channel features and spatial features from the music score image to assist the YOLOv5 module in music symbol recognition;

[0080] Since the music symbols that need to be recognized in the present invention are all small targets, and considering that the area ratio of background-irrelevant elements and the object to be recognized (music symbols) is quite different when the YOLOv5 module performs target detection, a convolutional attention module (CBAM) is added to make the recognition attention of the drum music score recognition model more focused on the music symbols.

[0081] CBAM is a new attention mechanism proposed on the basis of SE (Squeeze-and-Excitation). SE improves the expressiveness of the model by adaptively learning the weight of each channel. CBAM introduces the spatial attention mechanism on the basis of SE to further improve the expressiveness of the model. CBAM includes the Channel Attention Module and the Spatial Attention Module; CBAM is added to the 11th layer of the YOLOv5 module.

[0082] The drum music score recognition model training module is specifically used for:

[0083] The data set is divided into a training set, a validation set and a test set based on a ratio of 8:1:1; after training the drum music score recognition model for 500 epochs using the training set, the drum music score recognition model is verified using the validation set, and based on the verification result, the model parameters of the drum music score recognition model are fine-tuned to optimize the crossover ratio and confidence, and then the verified drum music score recognition model is tested using the test set to complete the training of the drum music score recognition model.

[0084] In summary, the advantages of the present invention are:

[0085] A large number of music score images of drum music scores are obtained and standardized preprocessed, and the music symbols in each preprocessed music score image are annotated by LabelImg to obtain the corresponding label vector, and then the corresponding label vector file is generated based on each label vector of each music score image, and each music score image and label vector file are stored, and a data set is constructed based on each music score image and label vector file; then a drum music score recognition model is created based on the YOLOv5 module and the convolutional attention module, and the drum music score recognition model is trained with the data set, and the trained drum music score recognition model is deployed on the server, and the online call part The deployed drum music score recognition model is used to recognize drum music scores; that is, the drum music score recognition model is trained by obtaining a label vector file by annotating music symbols and a data set constructed by music score images. The drum music score recognition model is equipped with a YOLOv5 module and a convolutional attention module. The convolutional attention module extracts channel features and spatial features from the music score image through the channel attention unit and the spatial attention unit respectively, and then assists the YOLOv5 module in recognizing music symbols through the channel features and the spatial features, so that the drum music score recognition model can focus on music symbols, thereby greatly improving the accuracy of drum music score recognition.

[0086] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A drum music score recognition method based on YOLOv5, characterized by: The steps include: Step S1, obtaining a large number of music score images of drum music scores, and performing standardized preprocessing on each of the music score images; Step S2, labeling the music symbols in each of the preprocessed music score images through LabelImg to obtain a corresponding label vector; Step S3, generating a corresponding label vector file based on each label vector of each of the music score images, storing each of the music score images and the label vector file, and constructing a data set based on each of the music score images and the label vector file; Step S4, creating a drum music score recognition model based on the YOLOv5 module and the convolutional attention module; Step S5, training a drum music score recognition model using the data set; Step S6: deploy the trained drum music score recognition model on the server, and call the deployed drum music score recognition model online to recognize the drum music score.

2. The method for recognizing drum music score based on YOLOv5 as claimed in claim 1, characterized in that: The step S1 is specifically as follows: A large number of music score images of drum music scores are obtained, and standardized preprocessing including at least format conversion, cropping, scaling, rotation and noise reduction is performed on each of the music score images.

3. The method for recognizing drum music score based on YOLOv5 as claimed in claim 1, characterized in that: The step S2 is specifically as follows: The music symbols in each of the preprocessed music score images are labeled in sequence through LabelImg to obtain corresponding label vectors including note categories and coordinate information, where the note categories are drum, hi-hat, cymbal, or rest.

4. The method for recognizing drum music score based on YOLOv5 as claimed in claim 1, characterized in that: The step S3 is specifically as follows: Generate a corresponding label vector file based on each label vector of each of the music score images, set the same file name for each of the music score images and the corresponding label vector files, store each of the music score images in a pre-created image folder, store each of the label vector files in a pre-created label folder, and construct a data set based on each of the music score images and the label vector files.

5. The method for recognizing drum music score based on YOLOv5 as claimed in claim 1, characterized in that: In step S4, the YOLOv5 module is used to identify music symbols from the music score image; the convolutional attention module is provided with a channel attention unit and a spatial attention unit, which are respectively used to extract channel features and spatial features from the music score image to assist the YOLOv5 module in music symbol recognition; The step S5 is specifically as follows: The data set is divided into a training set, a validation set and a test set based on a ratio of 8:1:1; after training the drum music score recognition model for 500 epochs using the training set, the drum music score recognition model is verified using the validation set, and based on the verification result, the model parameters of the drum music score recognition model are fine-tuned to optimize the crossover ratio and confidence, and then the verified drum music score recognition model is tested using the test set to complete the training of the drum music score recognition model.

6. A drum music score recognition system based on YOLOv5, characterized by: Includes the following modules: A music score image acquisition module, used to acquire a large number of music score images of drum music scores and perform standardized preprocessing on each of the music score images; A music score image labeling module, used to label the music symbols in each of the pre-processed music score images through LabelImg to obtain a corresponding label vector; A data set construction module, used to generate a corresponding label vector file based on each label vector of each of the music score images, store each of the music score images and the label vector file, and construct a data set based on each of the music score images and the label vector file; A drum music score recognition model creation module is used to create a drum music score recognition model based on the YOLOv5 module and the convolutional attention module; A drum music score recognition model training module, used for training the drum music score recognition model using the data set; The drum music score recognition module is used to deploy the trained drum music score recognition model on the server, and call the deployed drum music score recognition model online to recognize the drum music score.

7. A drum music score recognition system based on YOLOv5 as claimed in claim 6, characterized in that: The music score image acquisition module is specifically used for: A large number of music score images of drum music scores are obtained, and standardized preprocessing including at least format conversion, cropping, scaling, rotation and noise reduction is performed on each of the music score images.

8. A drum music score recognition system based on YOLOv5 as claimed in claim 6, characterized in that: The music score image annotation module is specifically used for: The music symbols in each of the preprocessed music score images are labeled in sequence through LabelImg to obtain corresponding label vectors including note categories and coordinate information, where the note categories are drum, hi-hat, cymbal, or rest.

9. A drum music score recognition system based on YOLOv5 as claimed in claim 6, characterized in that: The data set construction module is specifically used for: Generate a corresponding label vector file based on each label vector of each of the music score images, set the same file name for each of the music score images and the corresponding label vector files, store each of the music score images in a pre-created image folder, store each of the label vector files in a pre-created label folder, and construct a data set based on each of the music score images and the label vector files.

10. The drum music score recognition system based on YOLOv5 as claimed in claim 6, characterized in that: In the drum music score recognition model creation module, the YOLOv5 module is used to recognize music symbols from the music score image; the convolutional attention module is provided with a channel attention unit and a spatial attention unit, which are respectively used to extract channel features and spatial features from the music score image to assist the YOLOv5 module in music symbol recognition; The drum music score recognition model training module is specifically used for: The data set is divided into a training set, a validation set and a test set based on a ratio of 8:1:1; after training the drum music score recognition model for 500 epochs using the training set, the drum music score recognition model is verified using the validation set, and based on the verification result, the model parameters of the drum music score recognition model are fine-tuned to optimize the crossover ratio and confidence, and then the verified drum music score recognition model is tested using the test set to complete the training of the drum music score recognition model.

Images