Music score sequence extraction method, music search method, device and equipment

Abstract

Embodiments of the present invention disclose a music score sequence extraction method, a music search method, an apparatus, and a device, relating to the field of artificial intelligence technology. These methods are capable of improving the accuracy of music score sequence extraction under varying image quality conditions, thereby enhancing the precision and efficiency of subsequent searches based on the music score sequence. The music score sequence extraction method comprises acquiring a music score image to be identified, detecting all notes in the image to be identified, extracting the note heads of all notes, inputting the note heads into a classification model to obtain their positions within the staff, and generating a music score sequence based on the notes and the positions of their corresponding note heads within the staff. The present invention is applicable to electronic devices for music score sequence extraction and / or electronic devices for music search.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a music score sequence extraction method, a music search method, a device and equipment. Background Art

[0002] Music has long been disseminated through handwritten musical notation. Over time, digitizing and preserving musical notation has become crucial to protecting and disseminating this beautiful musical heritage. Manually transcribing handwritten notation into electronic form is time-consuming, labor-intensive, and error-prone, undoubtedly presenting significant challenges for music producers and disseminators. Related technologies involve converting sheet music images into machine-readable formats and automatically recognizing them using recognition tools. However, due to the myriad of handwriting styles, the uneven handwriting, and environmental interference when capturing images with recognition tools or smart devices, effective extraction of note sequences from sheet music is difficult, impacting music search efficiency. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a music score sequence extraction method, a music search method, an apparatus, and a device to solve the problem that existing music score sequence extraction methods have low accuracy and affect music search efficiency.

[0004] In a first aspect, an embodiment of the present invention provides a music score sequence extraction method, which is applied to a music score sequence extraction electronic device, comprising:

[0005] Obtaining a music score image to be recognized;

[0006] Detecting all musical notes in the music score image to be recognized;

[0007] Extract all note heads;

[0008] Inputting the note heads into a classification model to obtain positions of the note heads in a staff;

[0009] Generate a score sequence based on the positions of the notes and their corresponding note heads in the staff.

[0010] According to a specific implementation of the embodiment of the present invention, detecting all the notes in the music score image to be identified includes:

[0011] Inputting the music score image to be identified into a detection model to obtain a staff area;

[0012] Cutting the staff area into a plurality of bar areas with bars as cutting points;

[0013] Each measure region is input into the detection model to obtain all the notes in the music score image to be identified.

[0014] According to a specific implementation of an embodiment of the present invention, extracting the note heads of all notes includes:

[0015] splicing all the notes in the music score image to be recognized according to the input image size corresponding to the detection model;

[0016] The concatenated notes are input into the detection model to obtain the note head of each note.

[0017] According to a specific implementation of the embodiment of the present invention, the method further includes:

[0018] Determine whether each note is a chord note;

[0019] If so, determining the position of each notehead in the chord;

[0020] The note heads corresponding to each position are extracted one by one, and when the note head corresponding to one position is extracted, noise is added to the note heads corresponding to other positions.

[0021] According to a specific implementation of an embodiment of the present invention, the detection model is a yolov5 detection model.

[0022] According to a specific implementation of an embodiment of the present invention, the classification model is a ResNet optimized classification model, and inputting the note heads into the classification model to obtain positions of the note heads in the staff includes:

[0023] Get the training data of the resnet optimized classification model;

[0024] Performing data enhancement processing on the training data;

[0025] Inputting the training data after the data enhancement processing into the resnet optimization classification model, the resnet optimization classification model includes a mish activation function and an AMSgrad optimizer;

[0026] The position of the note head in the staff is determined according to the classification result output by the resnet optimized classification model.

[0027] According to a specific implementation of an embodiment of the present invention, the data enhancement processing includes:

[0028] One or more of color change, hollowing out, adding noise, sharpening, blunting, and removing ghosting.

[0029] According to a specific implementation of an embodiment of the present invention, generating a music score sequence according to the positions of notes and their corresponding note heads in a staff includes:

[0030] Group all notes by measure area;

[0031] Save the position of each note head in the staff to each note group according to the preset sorting method;

[0032] Arrange each note group into a musical score sequence according to the position of the corresponding measure area in the staff area.

[0033] According to a specific implementation of an embodiment of the present invention, the sorting method includes: sorting from left to right;

[0034] and / or,

[0035] Sort the note heads of each chord from bottom to top.

[0036] In a second aspect, an embodiment of the present invention provides a music search method, which is applied to a music search electronic device, comprising:

[0037] Obtain the music score image corresponding to the music to be searched;

[0038] Extracting a music score sequence from the music score image based on the music score sequence extraction method described in any of the aforementioned implementations;

[0039] The music score sequence is searched in a music score sequence database to obtain search results.

[0040] According to a specific implementation of an embodiment of the present invention, the music score sequence database includes:

[0041] A music score sequence and music information corresponding to the music score sequence, wherein the music information includes at least one of a name, a composer, a singer, a lyricist, music, and lyrics.

[0042] In a third aspect, an embodiment of the present invention provides a music score sequence extraction device, which is applied to a music score sequence extraction electronic device, comprising:

[0043] A first acquisition module is used to acquire a music score image to be recognized;

[0044] A detection module, configured to detect all notes in the music score image to be identified;

[0045] Extraction module, used to extract the note heads of all notes;

[0046] a position recognition module, configured to input the note heads into a classification model to obtain positions of the note heads in the staff;

[0047] The sequence generation module is used to generate a music score sequence according to the positions of the notes and the corresponding note heads in the staff.

[0048] In a fourth aspect, an embodiment of the present invention provides a music search device, which is applied to a music search electronic device, comprising:

[0049] The second acquisition module is used to obtain the music score image corresponding to the music to be searched;

[0050] an extraction module, configured to extract a music score sequence from the music score image based on the music score sequence extraction method described in any of the aforementioned implementations;

[0051] The retrieval module is used to search the music score sequence in the music score sequence database to obtain search results.

[0052] In a fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, and is used to execute the music score sequence extraction method described in any of the aforementioned implementations, and / or is used to execute the music search method described in any of the aforementioned implementations.

[0053] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the music score sequence extraction method described in any of the aforementioned implementations, and / or to execute the music search method described in any of the aforementioned implementations.

[0054] The embodiments of the present invention provide a music score sequence extraction method, music search method, device and equipment. By acquiring a music score image to be identified, detecting all notes in the music score image to be identified, extracting the note heads of all notes, inputting the note heads into a classification model to obtain the positions of the note heads in the staff, and generating a music score sequence based on the notes and the positions of the note heads corresponding to the notes in the staff, the accuracy of music score sequence extraction can be improved under different image qualities, thereby improving the accuracy and search efficiency of subsequent searches based on the music score sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 This is a flowchart of a music score sequence extraction method according to embodiment 1 of the present invention;

[0057] Figure 2 This is a flowchart of a music score sequence extraction method according to embodiment 2 of the present invention;

[0058] Figure 3 Schematic diagram of the intermediate results of the music score sequence extraction method according to the second embodiment of the present invention;

[0059] Figure 4 Schematic diagram of the intermediate results of the music score sequence extraction method according to the second embodiment of the present invention;

[0060] Figure 5 This is a flowchart of a music score sequence extraction method according to embodiment 3 of the present invention;

[0061] Figure 6 Schematic diagram of the intermediate results of the music score sequence extraction method according to the third embodiment of the present invention;

[0062] Figure 7 This is a flowchart of a music score sequence extraction method according to a fourth embodiment of the present invention;

[0063] Figure 8 This is a flowchart of a music score sequence extraction method according to a fifth embodiment of the present invention;

[0064] Figure 9 Schematic diagram of the intermediate results of the music score sequence extraction method according to the fifth embodiment of the present invention;

[0065] Figure 10 This is a flowchart of a music search method according to embodiment 1 of the present invention;

[0066] Figure 11 A functional structure diagram of a music score sequence extraction device provided by one embodiment of the present invention;

[0067] Figure 12 A functional structure diagram of a music search device provided by one embodiment of the present invention;

[0068] Figure 13 The present invention provides a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following describes embodiments of the present invention in detail with reference to the accompanying drawings. It should be understood that the embodiments described are only some of the embodiments of the present invention, and not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are intended to fall within the scope of protection of the present invention.

[0070] This embodiment provides a music score sequence extraction method to accurately identify various file formats.

[0071] Figure 1This is a flowchart of the music score sequence extraction method according to the first embodiment of the present invention. Figure 1 As shown, the recognition method of this embodiment is applied to a music score sequence extraction electronic device. The method of this embodiment may include:

[0072] Step 101: Obtain a music score image to be recognized;

[0073] In this embodiment, the music score image to be recognized can be obtained by taking a photo with a camera or a smart terminal.

[0074] Step 102: Detect all notes in the music score image to be recognized;

[0075] Step 103: extract the note heads of all notes;

[0076] Step 104: input the note heads into the classification model to obtain the positions of the note heads in the staff;

[0077] Step 105: Generate a music score sequence according to the positions of the notes and the corresponding note heads in the staff.

[0078] Traditional methods for extracting musical score sequences convert sheet images into a machine-readable format, then use recognition tools to automatically identify the sheet images. However, due to the myriad and varying handwriting styles, as well as environmental interference from recognition tools or smartphones when taking photos, these methods are unable to effectively extract the note sequences within the score, hindering music search efficiency.

[0079] This embodiment obtains a music score image to be identified, detects all notes in the music score image to be identified, extracts the note heads of all notes, inputs the note heads into a classification model to obtain the positions of the note heads in the staff, and generates a music score sequence according to the notes and the positions of the note heads corresponding to the notes in the staff. This can improve the accuracy of music score sequence extraction under different music score image qualities, thereby improving the accuracy and search efficiency of subsequent searches based on the music score sequence.

[0080] Figure 2 This is a flowchart of a music score sequence extraction method according to a second embodiment of the present invention. Figure 2 As shown, the method of this embodiment may include:

[0081] Step 201: Obtain a music score image to be recognized;

[0082] In this embodiment, the process of extracting and obtaining the music score image to be recognized is similar to step 101 of the above method embodiment, and will not be repeated here.

[0083] Step 202: Input the music score image to be recognized into the detection model to obtain the staff area;

[0084] In this embodiment, the detection model is, for example, the yolov5 detection model.

[0085] Step 203: cutting the staff area into multiple bar areas using bars as cutting points;

[0086] A measure is the most basic and regular rhythmic unit, and is usually represented by a line of music in a sheet music image.

[0087] Step 204: Input each measure area into the detection model to obtain all the notes in the music score image to be identified.

[0088] The identified staff areas and notes are as follows Figure 3 As shown;

[0089] Step 205: splice all the notes in the music score image to be recognized according to the input image size corresponding to the detection model;

[0090] The image of the note after splicing is as follows Figure 4 shown.

[0091] Step 206: Input the concatenated notes into the detection model to obtain the note head of each note.

[0092] Step 207: Input the note head of each note into the ResNet optimized classification model to obtain the position of each note head in the staff;

[0093] Step 208: Generate a score sequence according to the positions of the notes and the corresponding note heads in the staff.

[0094] In this embodiment, the yolov5 detection model is combined with the resnet optimized classification model to extract the music score sequence in the music score image. All the notes in the music score image to be identified are spliced ​​according to the input image size corresponding to the detection model. This can reduce the time waste caused by the yolov5 detection model pre-processing and save the resources required for subsequent image resizing, thereby improving the extraction efficiency.

[0095] Figure 5 This is a flowchart of a music score sequence extraction method according to a third embodiment of the present invention. Figure 5 As shown, this embodiment Figure 2 Based on the method embodiment shown, step 206 may further include:

[0096] Step 501: determine whether each note is a chord note;

[0097] A chord refers to a group of sounds with a certain interval relationship, that is, three or more notes are combined vertically according to the superposition relationship of thirds or non-thirds, which are called chord notes.

[0098] Step 502: If yes, determine the position of each note head in the chord;

[0099] Step 503: extract the note heads corresponding to each position one by one, and when extracting the note head corresponding to one position, add noise to the note heads corresponding to other positions.

[0100] The schematic diagram of adding noise is as follows Figure 6 shown.

[0101] In this embodiment, noise is added so that the classification model can focus on the note heads without noise added for classification detection, thereby further improving the accuracy and efficiency of note head recognition.

[0102] Figure 7 This is a flowchart of a method for extracting a music score sequence according to a fourth embodiment of the present invention. Figure 7 As shown, this embodiment Figure 2 Based on the method embodiment shown, step 207 may further include:

[0103] Step 701: Obtain training data for the ResNet optimized classification model;

[0104] Step 702: Perform data enhancement processing on the training data;

[0105] In this embodiment, data enhancement processing includes but is not limited to:

[0106] One or more of color change, hollowing out, adding noise, sharpening, blunting, and removing ghosting.

[0107] Step 703: Input the training data after data enhancement processing into the ResNet optimization classification model, which includes the Mish activation function and the AMSgrad optimizer;

[0108] Step 704: Determine the position of the note head in the staff according to the classification result output by the ResNet optimized classification model.

[0109] In this embodiment, the activation function relu is changed to mish, the optimizer SGD is changed to AMSgrad, and data enhancement methods such as color change, image hollowing, noise addition, sharpening, blunting, and eliminating afterimages are added during training to improve the generalization ability of the classification model.

[0110] Figure 8 This is a flowchart of a music score sequence extraction method according to a fifth embodiment of the present invention. Figure 8 As shown, this embodiment Figure 2 Based on the method embodiment shown, step 208 may further include:

[0111] Step 801: group all notes according to measure areas;

[0112] Step 802: save the position of each note head in the staff into each note group according to a preset sorting method;

[0113] Step 803: Arrange each note group into a music score sequence according to the position of the corresponding measure area in the staff area.

[0114] In this embodiment, the sorting methods include: sorting from left to right;

[0115] If it is a chord note, sort each note head in the chord note from bottom to top.

[0116] The position of notes and their corresponding note heads in the staff is as follows Figure 9 As shown, the generated music score sequence is [m1, m3, m5].

[0117] In this embodiment, the note sequence in the music score can still be effectively extracted under the condition of the image quality taken by the user through the mobile phone.

[0118] A specific embodiment is used below. Figure 1 、 Figure 2 、 Figure 5 、 Figure 7 、 Figure 8 The technical solution of any of the method embodiments shown in the embodiment is described in detail. The method of this embodiment may include:

[0119] (1) Use the yolov5 detection model to detect the high and low notes of the staff area and divide the staff area into measure areas with the measure as the cutting point, and detect all the notes in each measure area;

[0120] (2) Assembling the detected notes into note images of the size required by the classification model, thereby reducing the model calculation time;

[0121] (3) The assembled note images are passed through the yolov5 detection model again for note head detection, dividing a multi-note chord into multiple separate note heads and adding noise to the other note heads so that the algorithm can focus on the note head positions without noise.

[0122] (4) The divided single note heads are classified by the ResNet model to obtain the position of the note heads relative to the staff;

[0123] (5) The notes are grouped into measures and sorted from left to right. When encountering chords, they are sorted from bottom to top to obtain a music score sequence that can be used for searching.

[0124] This embodiment combines a detection model with a classification detection model to extract musical score sequences. Noise is added to allow the classification model to focus on unnoised note heads for classification detection. Image stitching is used on Yolov5 to reduce pre-processing time and eliminate the resources required for subsequent image resizing. The activation function and optimizer are optimized on ResNet, and data augmentation is added during training to improve the model's generalization capabilities, ultimately enhancing the efficiency and accuracy of musical score sequence extraction.

[0125] Figure 10 This is a flowchart of the first embodiment of the music search method of the present invention. The applicable scenario of this embodiment can be when performing music search, such as Figure 10 As shown, the specific music search method includes:

[0126] Step 1001: Obtain a music score image corresponding to the music to be searched;

[0127] The process of obtaining the music score image corresponding to the music to be searched and obtaining the music score image to be identified is similar to step 101 of the above method embodiment, and will not be repeated here.

[0128] Step 1002: extracting a music score sequence from the music score image;

[0129] The extraction method can refer to the above examples.

[0130] Step 1003: Search the music score sequence in the music score sequence database to obtain search results.

[0131] In this embodiment, the music score sequence database includes:

[0132] The music score sequence and the music information corresponding to the music score sequence, the music information includes the name, composer, singer, lyricist, music and lyrics.

[0133] Traditional music search methods usually use music theme names or singers as keywords to search, but the search results will output a lot, which will interfere with the user's search efficiency.

[0134] In this embodiment, searching is performed based on the music score sequence, which greatly improves the search efficiency. In addition, music information can be directly searched through the music score image, which improves the user experience.

[0135] Figure 11 This is a functional structure diagram of the first embodiment of the music score sequence extraction device of the present invention, as shown in FIG. Figure 11 As shown, the device of this embodiment may include:

[0136] A first acquisition module 111 is used to acquire a music score image to be recognized;

[0137] A detection module 112 is configured to detect all notes in the music score image to be identified;

[0138] Extraction module 113, for extracting the note heads of all notes;

[0139] a position identification module 114 for inputting the note head of each note into a classification model to obtain the position of the note head of each note in the staff;

[0140] The sequence generation module 115 is used to generate a music score sequence according to the positions of the notes and the corresponding note heads in the staff.

[0141] The detection module 112 is configured to:

[0142] Input the music score image to be recognized into the detection model to obtain the staff area;

[0143] Cut the staff area into multiple measure areas based on the measure as the cutting point;

[0144] Each measure area is input into the detection model to obtain all the notes in the music score image to be identified.

[0145] Further, it is determined whether each note is a chord note;

[0146] If so, determine the position of each notehead in the chord;

[0147] The note heads corresponding to each position are extracted one by one, and when the note head corresponding to one position is extracted, noise is added to the note heads corresponding to other positions.

[0148] The extraction module 113 is configured to:

[0149] All the notes in the music score image to be recognized are spliced ​​according to the input image size corresponding to the detection model;

[0150] The concatenated notes are input into the detection model to obtain the note head of each note.

[0151] The location identification module 114 is configured to:

[0152] Get the training data of the resnet optimized classification model;

[0153] Perform data augmentation on training data;

[0154] The training data after data enhancement is input into the resnet optimization classification model, which includes the mish activation function and the AMSgrad optimizer.

[0155] Get the position of the note head in the staff based on the ResNet optimized classification model.

[0156] Further data enhancement processing includes:

[0157] One or more of color change, hollowing out, adding noise, sharpening, blunting, and removing ghosting.

[0158] The sequence generation module 115 is configured to:

[0159] Group all notes by measure area;

[0160] Save the position of each note head in the staff to each note group according to the preset sorting method;

[0161] Arrange each note group into a musical score sequence according to the position of the corresponding measure area in the staff area.

[0162] Furthermore, the sorting methods include: sorting from left to right;

[0163] and / or,

[0164] Sort the note heads of each chord from bottom to top.

[0165] The device of this embodiment can be used to perform Figures 1 to 9 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0166] Figure 12 This is a functional structure diagram of the first embodiment of the music search device of the present invention. Figure 12 As shown, the device of this embodiment may include:

[0167] The second acquisition module 121 is used to acquire the music score image corresponding to the music to be searched;

[0168] An extraction module 122 is configured to extract a music score sequence from the music score image based on the music score sequence extraction method described in the above embodiment;

[0169] The search module 123 is used to search the music score sequence in the music score sequence database to obtain search results.

[0170] The device of this embodiment can be used to perform Figure 10 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0171] Figure 13 This is a schematic diagram of the structure of an embodiment of the electronic device of the present invention, which can realize the present invention. Figure 1-9 The process of the embodiment shown, and / or, can implement the present invention Figure 10 The process of the embodiment shown is as follows: Figure 13As shown, the electronic device may include: a housing 131, a processor 132, a memory 133, a circuit board 134, and a power supply circuit 135, wherein the circuit board 134 is placed inside the space enclosed by the housing 131, and the processor 132 and the memory 133 are arranged on the circuit board 134; the power supply circuit 135 is used to supply power to various circuits or devices of the electronic device; the memory 133 is used to store executable program code; the processor 132 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 133, so as to execute the music score sequence extraction method described in any of the aforementioned embodiments.

[0172] For details on the specific execution process of the above steps by the processor 132 and the steps further performed by the processor 132 by running the executable program code, please refer to the present invention. Figure 1-9 The description of the illustrated embodiment may be referred to in the Figure 10 The description of the illustrated embodiment will not be repeated here.

[0173] This electronic device exists in many forms, including but not limited to:

[0174] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0175] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0176] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0177] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0178] (5) Other electronic devices with data interaction functions.

[0179] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the music score sequence extraction method described in any of the aforementioned embodiments, and / or the music search method described in any of the aforementioned embodiments.

[0180] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0181] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0182] In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0183] For the convenience of description, the above device is described as being divided into various units / modules based on their functions. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0184] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0185] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for extracting a music score sequence, characterized in that: Applicable to electronic equipment for extracting music score sequences; the method comprises: Obtaining a music score image to be recognized; Detecting all musical notes in the music score image to be recognized, including: inputting the music score image to be recognized into a detection model to obtain a staff region; cutting the staff region into a plurality of bar regions using bars as cutting points; and inputting each bar region into the detection model to obtain all musical notes in the music score image to be recognized; Extracting the note heads of all notes, including: splicing all notes in the music score image to be recognized according to the input image size corresponding to the detection model to shorten the pre-processing time of the detection model and save resources consumed in subsequent image resizing; inputting the spliced ​​notes into the detection model to obtain the note heads of each note; Determine whether each note is a chord note; if so, divide a multi-notehead chord into multiple individual noteheads and determine the position of each notehead in the chord note; extract the notehead corresponding to each position one by one, and when extracting the notehead corresponding to one position, add noise to the noteheads corresponding to other positions so that the classification model focuses on the noteheads without noise for classification detection; Inputting the note heads into the classification model to obtain positions of the note heads in the staff; Generate a searchable score sequence based on the positions of the notes and their corresponding note heads in the staff.

2. The music score sequence extraction method according to claim 1, characterized in that: The detection model is the yolov5 detection model.

3. The method for extracting a musical score sequence according to claim 1, wherein: The classification model is a ResNet optimized classification model, and inputting the note heads into the classification model to obtain positions of the note heads in the staff includes: Obtaining training data for the resnet optimized classification model; Performing data enhancement processing on the training data; Inputting the training data after the data enhancement processing into the resnet optimization classification model, the resnet optimization classification model includes a mish activation function and an AMSgrad optimizer; The position of the note head in the staff is determined according to the classification result output by the resnet optimized classification model.

4. The method for extracting a musical score sequence according to claim 3, wherein: The data enhancement processing includes: One or more of: color change, hollowing out, adding noise, sharpening, blunting, and removing ghosting.

5. The method for extracting a musical score sequence according to claim 1, wherein: Generating a music score sequence according to the positions of the notes and the note heads corresponding to the notes in the staff includes: Group all notes by measure area; The position of each note head in the staff is saved to each note group according to the preset sorting method; Arrange each note group into a musical score sequence according to the position of the corresponding measure area in the staff area.

6. The method for extracting a musical score sequence according to claim 5, wherein: The sorting method includes: sorting from left to right; and / or, Sort the note heads of each chord from bottom to top.

7. A music search method, characterized in that: Applicable to music search electronic equipment; the method includes: Obtain the music score image corresponding to the music to be searched; extracting a music score sequence from the music score image based on the music score sequence extraction method according to any one of claims 1 to 6; The music score sequence is searched in a music score sequence database to obtain search results.

8. The music search method according to claim 7, wherein: The music score sequence database includes: A music score sequence and music information corresponding to the music score sequence, wherein the music information includes at least one of a name, a composer, a singer, a lyricist, music, and lyrics.

9. A music score sequence extraction device, characterized in that: Applicable to electronic equipment for extracting musical score sequences, the device comprises: A first acquisition module is used to acquire a music score image to be recognized; A detection module, configured to detect all notes in the music score image to be identified; Extraction module, used to extract the note heads of all notes; a position recognition module, configured to input the note heads into a classification model to obtain positions of the note heads in the staff; A sequence generation module is used to generate a searchable music score sequence based on the positions of the notes and the corresponding note heads in the staff; The extraction module is configured to: splice all the notes in the music score image to be identified according to the input image size corresponding to the detection model to shorten the pre-processing time of the detection model and save resources consumed in subsequent image resizing; input the spliced ​​notes into the detection model to obtain the note head of each note; The detection module is configured to: input the music score image to be identified into the detection model to obtain a staff area; Cutting the staff area into a plurality of bar areas with bars as cutting points; Input each measure area into the detection model to obtain all notes in the music score image to be identified; Determine whether each note is a chord note; If so, dividing a multi-notehead chord into a plurality of individual noteheads, and determining the position of each notehead in the chord; The note heads corresponding to each position are extracted one by one, and when extracting the note head corresponding to one position, noise is added to the note heads corresponding to other positions so that the classification model focuses on the note heads without noise added for classification detection.

10. A music search device, characterized in that: Applicable to music search electronic equipment, the device includes: The second acquisition module is used to obtain the music score image corresponding to the music to be searched; an extraction module, configured to extract a music score sequence from the music score image based on the music score sequence extraction method according to any one of claims 1 to 6; The retrieval module is used to search the music score sequence in the music score sequence database to obtain search results.

11. An electronic device, characterized in that: The electronic device includes: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, and is used to execute the music score sequence extraction method described in any one of claims 1 to 6, and / or the music search method described in claim 7 or 8.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the music score sequence extraction method described in any one of claims 1 to 6, and / or to execute the music search method described in claim 7 or 8.

Images