A simple score recognition method and system

Abstract

This invention provides a method and system for recognizing simplified musical notation, relating to the field of musical notation recognition technology. The method includes the following steps: acquiring an original simplified musical notation image and preprocessing the original image; recognizing the preprocessed image to determine if it represents complete musical information; if so, correcting the image to obtain a corresponding complete simplified musical notation recognition result; otherwise, performing note recognition and auxiliary element recognition on the image to obtain corresponding note recognition and auxiliary element recognition results, and concatenating the results to obtain the corresponding simplified musical notation recognition result. This invention can simultaneously process complete standard simplified musical notation and partial simplified musical notation, covering all usage scenarios.

Description

Technical Field

[0001] This invention relates to the field of musical score recognition technology, and more specifically to a method and system for recognizing simplified musical notation. Background Technology

[0002] Currently, numbered musical notation is a simple method of musical notation that describes information such as notes, pitch, and duration. It is the most direct way for musicians to learn, share, and disseminate music. With the rapid development of computer applications and image scanning technologies, converting paper-based numbered musical notation files into electronic versions using computer technology can be widely used in various fields such as music information retrieval and music-assisted teaching.

[0003] However, electronic versions of sheet music require the use of relevant recognition algorithms. Most existing recognition algorithms use a unified recognition process to process sheet music images. Since the length of music varies, using a unified algorithm for all of them would result in a complex process and low accuracy.

[0004] Therefore, how to provide a simplified musical notation recognition method that can solve the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for recognizing simplified musical notation, which can simultaneously process complete standard simplified musical notation of songs as well as partial simplified musical notation, covering the needs of use in all scenarios.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A simplified musical notation recognition method includes the following steps: Acquire the original simplified musical notation image and preprocess the original simplified musical notation image; The preprocessed original simplified musical score image is identified to determine whether it is complete music information. If it is, the preprocessed original simplified musical score image is corrected to obtain the corresponding complete simplified musical score recognition result. Otherwise, perform note recognition and auxiliary element recognition on the preprocessed original simplified musical notation image to obtain the corresponding note recognition results and auxiliary element recognition results, and then splice and combine the note recognition results and auxiliary element recognition results to obtain the corresponding simplified musical notation recognition results.

[0007] Preferred options also include: The complete simplified musical notation recognition result or the simplified musical notation recognition result is labeled.

[0008] Preferably, the specific processing steps to obtain the corresponding complete simplified musical notation recognition result include: An image recognition model is constructed, and the preprocessed original simplified spectrum image is input into the image recognition model for processing to obtain the corresponding image recognition result; A simplified musical notation recognition rule graph engine is constructed, and the image recognition results are processed in combination with the simplified musical notation recognition rule graph engine to determine whether it is a complete simplified musical notation; If so, the complete musical notation recognition result is corrected to obtain the corresponding complete musical notation recognition result.

[0009] Preferably, the specific processing steps for correcting the complete simplified musical notation recognition result include: The image recognition results are extracted to determine the corresponding simplified musical notation name; Obtain clear simplified musical notation images corresponding to the simplified musical notation names using big data; The image recognition result is corrected by combining the clear simplified musical notation image, and the corrected image recognition result is taken as the complete simplified musical notation recognition result.

[0010] Preferably, the specific processing steps to obtain the corresponding simplified musical notation recognition results include: The preprocessed original simplified musical notation image is subjected to note recognition and auxiliary element recognition to obtain the corresponding note recognition results and auxiliary element recognition results. The note recognition results and auxiliary element recognition results are preprocessed separately; Construct a coordinate system and map the preprocessed note recognition results and auxiliary element recognition results into the coordinate system; Calculate the vertical distance between any note element in the note recognition result and any linear element in the auxiliary element recognition result, determine the positional relationship between the note element and the linear element based on the vertical line calculation result, and complete the splicing.

[0011] Preferably, the specific processing steps for preprocessing the note recognition results and auxiliary element recognition results respectively include: The auxiliary element recognition results are subjected to line filtering and interference masking, while the note recognition results are subjected to line filtering and screening.

[0012] The present invention also provides a simplified musical notation recognition system, comprising: The acquisition module is used to acquire the original simplified musical notation image and preprocess the original simplified musical notation image; The musical notation integrity determination module is used to identify the preprocessed original musical notation image and determine whether it contains complete music information; The first recognition module is used to correct the preprocessed original simplified musical score image when it is determined to be complete music information, so as to obtain the corresponding complete simplified musical score recognition result. The second recognition module is used to perform note recognition and auxiliary element recognition on the preprocessed original simplified musical score image when it is determined to be incomplete music information, to obtain the corresponding note recognition results and auxiliary element recognition results, and to splice and combine the note recognition results and auxiliary element recognition results to obtain the corresponding simplified musical score recognition results. The annotation module is used to annotate the complete simplified musical notation recognition result or the simplified musical notation recognition result.

[0013] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a simplified spectrum identification method and system, which has the following beneficial effects: 1. This invention sets up a dual-branch recognition mechanism for complete and incomplete music scores, performs preprocessing operations on the original simplified music score images in a unified manner, reduces the cost of post-processing error correction, and takes into account both complete music scores and fragmented simplified music scores in business scenarios, thus covering a wider range of applicable materials. 2. This invention introduces a simplified musical notation recognition rule graph engine, which links the image pixel recognition results with music theory and typesetting rules for verification, thereby reducing the probability of misjudgment; 3. The overall processing of this invention can be broken down into multiple steps such as preprocessing, integrity judgment, identification of musical notes and linear elements, and coordinate splicing. Any sub-process can be optimized individually, and the cost of iterative modification is low. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1 This is an overall flowchart of a simplified musical notation recognition method provided by the present invention; Figure 2 The present invention provides a structural principle block diagram of a simplified musical notation recognition system. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] See Figure 1As shown in the figure, an embodiment of the present invention discloses a simplified musical notation identification method, including the following steps: Acquire the original simplified musical notation image and preprocess the original simplified musical notation image; The preprocessed original simplified musical score image is identified to determine whether it is complete music information. If it is, the preprocessed original simplified musical score image is corrected to obtain the corresponding complete simplified musical score recognition result. Otherwise, the preprocessed original simplified musical notation image is subjected to note recognition and auxiliary element recognition respectively to obtain the corresponding note recognition results and auxiliary element recognition results. The note recognition results and auxiliary element recognition results are then combined to obtain the corresponding simplified musical notation recognition results. The note recognition results may include numbers 0-9, accidentals, flats b, character-type note elements, etc., and the auxiliary element recognition results may include time value symbols (such as dotted notes, ties, augmentation lines, diminution lines), measures and time signatures, rests, etc.

[0018] Specifically, the preprocessing of the original simplified spectrum image may include: (1) Color space conversion processing: In the HSV space, a mask is generated using saturation, and the colored areas are set to white to reduce the interference of colored marks on recognition; (2) Detail enhancement processing: Use the difference of Gaussian function to enhance the contrast of fine strokes and local areas, making weak edges more obvious; (3) Binarization: The original spectrum image processed above is converted into foreground / background using the Otsu thresholding method, which facilitates subsequent morphological and contour analysis.

[0019] The above preprocessing can be implemented using the OpenCV vision library.

[0020] In one specific embodiment, it also includes: The complete musical notation recognition result or the musical notation recognition result can be annotated, and the musical notation sequence can be output in a top-to-bottom, line-by-line order, with each line from left to right. Colored boxes and reference lines can also be overlaid to facilitate quick manual checking of the accuracy of each element detection, and the result can be saved as a file in a specific format (e.g., ...). (e.g., .txt), making it convenient for subsequent use.

[0021] In a specific embodiment, the specific processing steps to obtain the corresponding complete simplified musical notation recognition result include: An image recognition model is constructed, and the preprocessed original simplified musical notation image is input into the image recognition model for processing to obtain the corresponding image recognition result. The image recognition model can be a multi-scale convolutional neural network model, which can better extract the note features in the simplified musical notation image and improve the recognition accuracy. A simplified musical notation recognition rule graph engine is constructed, and the image recognition results are processed in combination with the simplified musical notation recognition rule graph engine to determine whether it is a complete simplified musical notation; If so, the complete musical notation recognition result is corrected to obtain the corresponding complete musical notation recognition result.

[0022] Specifically, the simplified musical notation recognition rule graph engine can be implemented using a knowledge graph, including a single-page simplified musical notation determination rule engine and a multi-page simplified musical notation determination rule engine, as follows: (1) When the preprocessed original simplified musical score image is a single-page simplified musical score, it includes: The preprocessed original simplified musical notation image outputs a character sequence with a rest at the end, complete line breaks and the first line is complete, containing basic information such as mode and time signature, and the number of valid musical phrases reaches the threshold. (2) When the preprocessed original simplified musical score image is a multi-page simplified musical score, it includes: Concatenate all the simplified musical notation character sequences in page order; Only check if there is a rest at the end of the sequence on the last page, and allow only a single bar line on intermediate pages.

[0023] In a specific embodiment, the specific processing procedure for correcting the complete simplified musical notation recognition result includes: The image recognition results are extracted to determine the corresponding simplified musical notation name; Obtain clear simplified musical notation images corresponding to the simplified musical notation names using big data; The image recognition result is corrected by combining the clear simplified musical notation image, and the corrected image recognition result is taken as the complete simplified musical notation recognition result.

[0024] In a specific embodiment, the specific processing steps to obtain the corresponding simplified musical notation recognition result include: The preprocessed original simplified musical notation image is subjected to note recognition and auxiliary element recognition to obtain the corresponding note recognition results and auxiliary element recognition results. The note recognition results and auxiliary element recognition results are preprocessed separately; Construct a coordinate system and map the preprocessed note recognition results and auxiliary element recognition results into the coordinate system; Calculate the vertical distance between any note element in the note recognition result and any linear element in the auxiliary element recognition result, determine the positional relationship between the note element and the linear element based on the vertical line calculation result, and complete the splicing.

[0025] Specifically, the processing steps to obtain the corresponding note recognition results and auxiliary element recognition results include: (1) Regarding the recognition of horizontal lines (time-increase lines / connecting lines), multi-scale morphology can be used to extract horizontal line segment masks, and then the horizontal line box can be obtained from the contour; obviously excessively long lines can be removed and the sensitivity can be controlled by threshold; (2) Regarding the identification of vertical lines (such as nodal lines), multi-scale morphology can be used to extract vertical line segment masks, and slight tilting / bending is allowed (by connecting the break with a wider structuring element and the closing operation); at the same time, the relationship rules with the digit region are used to avoid misjudging the digit "1" as a vertical line; (3) Regarding point recognition, area / shape constraints based on small contours can be used to detect points for determining high and low pitch points and attachments; (4) Regarding the recognition of rising and falling numbers (# / b), OCR detection can be performed in the local area to the left of the number, and the box range can be narrowed by combining position constraints and contour refinement; and priority can be set to shield the repeated judgment of other elements within its range to reduce false detection conflicts.

[0026] Specifically, the process of determining the positional relationship between the musical note element and the linear element based on the vertical line calculation results to complete the splicing includes: Calculate the vertical distance between any note element and a linear element. Using the vertical line from the center of the note as a reference, determine the subordinate positional relationship between the note and the horizontal and vertical lines, specifically as follows: The number of horizontal lines that a vertical line intersects determines the note value; Dots on the left and right, and up and down, distinguish dotted notes and treble / bass markings; The vertical line with the horizontal axis serves as a section separator. Sort all elements by horizontal coordinate of any musical note element from smallest to largest, complete the splicing of musical notes and linear elements, and generate the recognition result.

[0027] In a specific embodiment, the preprocessing steps for the note recognition results and auxiliary element recognition results include: The auxiliary element recognition results are subjected to line filtering and interference masking, while the note recognition results are subjected to line filtering and screening.

[0028] See Figure 2 As shown, embodiments of the present invention also provide a system utilizing the simplified musical notation identification method described in any of the above embodiments, comprising: The acquisition module is used to acquire the original simplified musical notation image and preprocess the original simplified musical notation image; The musical notation integrity determination module is used to identify the preprocessed original musical notation image and determine whether it contains complete music information; The first recognition module is used to correct the preprocessed original simplified musical score image when it is determined to be complete music information, so as to obtain the corresponding complete simplified musical score recognition result. The second recognition module is used to perform note recognition and auxiliary element recognition on the preprocessed original simplified musical score image when it is determined to be incomplete music information, to obtain the corresponding note recognition results and auxiliary element recognition results, and to splice and combine the note recognition results and auxiliary element recognition results to obtain the corresponding simplified musical score recognition results. The annotation module is used to annotate the complete simplified musical notation recognition result or the simplified musical notation recognition result.

[0029] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0030] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying simplified musical notation, characterized in that, Includes the following steps: Acquire the original simplified musical notation image and preprocess the original simplified musical notation image; The preprocessed original simplified musical score image is identified to determine whether it is complete music information. If it is, the preprocessed original simplified musical score image is corrected to obtain the corresponding complete simplified musical score recognition result. Otherwise, perform note recognition and auxiliary element recognition on the preprocessed original simplified musical notation image to obtain the corresponding note recognition results and auxiliary element recognition results, and then splice and combine the note recognition results and auxiliary element recognition results to obtain the corresponding simplified musical notation recognition results.

2. The simplified musical notation identification method according to claim 1, characterized in that, Also includes: The complete simplified musical notation recognition result or the simplified musical notation recognition result is labeled.

3. The simplified musical notation identification method according to claim 1, characterized in that, The specific processing steps to obtain the corresponding complete simplified musical notation recognition result include: An image recognition model is constructed, and the preprocessed original simplified spectrum image is input into the image recognition model for processing to obtain the corresponding image recognition result; A simplified musical notation recognition rule graph engine is constructed, and the image recognition results are processed in combination with the simplified musical notation recognition rule graph engine to determine whether it is a complete simplified musical notation; If so, the complete musical notation recognition result is corrected to obtain the corresponding complete musical notation recognition result.

4. The simplified musical notation identification method according to claim 3, characterized in that, The specific processing steps for correcting the complete simplified musical notation recognition results include: The image recognition results are extracted to determine the corresponding simplified musical notation name; Obtain clear simplified musical notation images corresponding to the simplified musical notation names using big data; The image recognition result is corrected by combining the clear simplified musical notation image, and the corrected image recognition result is taken as the complete simplified musical notation recognition result.

5. The simplified musical notation identification method according to claim 3, characterized in that, The specific processing steps to obtain the corresponding simplified musical notation recognition results include: The preprocessed original simplified musical notation image is subjected to note recognition and auxiliary element recognition to obtain the corresponding note recognition results and auxiliary element recognition results. The note recognition results and auxiliary element recognition results are preprocessed separately; Construct a coordinate system and map the preprocessed note recognition results and auxiliary element recognition results into the coordinate system; Calculate the vertical distance between any note element in the note recognition result and any linear element in the auxiliary element recognition result, determine the positional relationship between the note element and the linear element based on the vertical line calculation result, and complete the splicing.

6. The simplified musical notation identification method according to claim 5, characterized in that, The specific preprocessing steps for the note recognition results and auxiliary element recognition results include: The auxiliary element recognition results are subjected to line filtering and interference masking, while the note recognition results are subjected to line filtering and screening.

7. A system utilizing the simplified musical notation identification method according to any one of claims 1-6, characterized in that, include: The acquisition module is used to acquire the original simplified musical notation image and preprocess the original simplified musical notation image; The musical notation integrity determination module is used to identify the preprocessed original musical notation image and determine whether it contains complete music information; The first recognition module is used to correct the preprocessed original simplified musical score image when it is determined to be complete music information, so as to obtain the corresponding complete simplified musical score recognition result. The second recognition module is used to perform note recognition and auxiliary element recognition on the preprocessed original simplified musical score image when it is determined to be incomplete music information, to obtain the corresponding note recognition results and auxiliary element recognition results, and to splice and combine the note recognition results and auxiliary element recognition results to obtain the corresponding simplified musical score recognition results. The annotation module is used to annotate the complete simplified musical notation recognition result or the simplified musical notation recognition result.

Images