An environment-adaptive music score recognition method

Abstract

The present application relates to the technical field of music score recognition, and particularly relates to an environment-adaptive music score recognition method, the present application acquires a to-be-recognized music score image, divides the music score image into regions to obtain a plurality of sub-image regions, acquires a note filling rate and an impurity coverage rate, calculates a sub-music score blur representation coefficient, determines a music score blur state of the sub-image region, calculates a note appearance rate, determines a note abnormal recognition state of the sub-image region, and recognizes the sub-image region, including, for the notes of the sub-image region in the non-abnormal recognition state, performing differentiated contour feature processing on the notes to recognize the notes of the sub-image region; for the notes of the sub-image region in the abnormal recognition state, performing note prediction based on the impurity coverage rate to recognize the notes of the sub-image region, and detecting music score fluency based on the recognized notes. The present application improves music score recognition efficiency and accuracy by performing multi-class recognition on music scores.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of music score recognition, and in particular to an environment-adaptive music score recognition method. BACKGROUND

[0002] Music score recognition is the application of image processing and pattern recognition technology in the field of music, aiming to convert paper music into a computer-readable digital format, which not only facilitates storage and dissemination, but also provides a basis for subsequent music editing, playing, analysis, etc. However, traditional paper music is prone to damage, loss, etc. during use, is difficult to preserve for a long time, and is inconvenient to search, therefore, music score recognition technology has emerged as the times require, which realizes the conversion of music score through image preprocessing, music object detection, music symbol reconstruction, and encoding into machine-readable format.

[0003] Chinese Patent Publication No. CN114092946A discloses a music score recognition method, which first divides a music score image into lines, measures, and note sequences through target recognition and segmentation into a tree structure; then uses a multi-label classification method to process and recognize the note sequence; and finally combines the extracted tree structure into a digital music score file through certain logical processing and outputs it. The concept of note sequence: the collection of notes and modifiers in each beat of each voice part. This method effectively avoids the inter-class data volume gap that cannot be ignored when using the same network, thereby balancing the samples of each class and making each network specific. Compared with optical music score recognition based on digital image processing, using the target detection network in the method to perform macro block segmentation can effectively improve the situation where the information between measures is unclear and the statistical boundary is not distinct, thereby enhancing the robustness of the system.

[0004] Chinese Patent Publication No. CN112686104A discloses a multi-voice music score recognition method based on deep learning, which proposes a music score recognition model based on deep learning for printed multi-voice music score. An end-to-end music note recognition method based on convolutional neural network, data enhancement, and transfer learning aims to solve the problems of low accuracy and note omission in the multi-voice music note recognition process. The model uses multi-task learning to simultaneously learn the classification task of pitch and time value and the regression task of note coordinates. In the training of the model, the fine-tuning technology in the neural network is used to better detect the chord notes in the music score. Experimental results show that the model can accurately recognize the notes in the multi-voice music score and also performs well in the recognition of chord notes.

[0005] However, the prior art still has the following problems,

[0006] In the process of music score recognition, there may be complete notes, fuzzy notes, half-shielded notes and full-shielded notes in the music score. If the same method is used to recognize each type of note, not only the music score recognition efficiency will be reduced, but also the music score recognition accuracy will be reduced. SUMMARY

[0007] To this end, the present application provides an environment adaptive music score recognition method to solve the problem that in the process of music score recognition, there may be complete notes, fuzzy notes, half-shielded notes and full-shielded notes in the music score. If the same method is used to recognize each type of note, not only the music score recognition efficiency will be reduced, but also the music score recognition accuracy will be reduced.

[0008] To achieve the above purpose, the present application provides an environment adaptive music score recognition method, which comprises:

[0009] Obtain the music score image to be recognized, divide the music score image into regions to obtain a plurality of sub-image regions;

[0010] Obtain the note filling rate and impurity coverage rate of the music score in the sub-image region, calculate the music score blur representation coefficient of the sub-image region based on the note filling rate and impurity coverage rate, and determine the music score blur state of the sub-image region;

[0011] Based on the division result of the music score blur state, calculate the note appearance rate, determine whether the note of the sub-image region is in an abnormal recognition state, and identify the sub-image region based on the determination result, including,

[0012] For the note of the sub-image region in the non-abnormal recognition state, the note is processed by distinguishing the contour features, including distinguishing the note contour features from the non-note contour features based on the contour fluency, and extracting the note contour features to identify the note of the sub-image region;

[0013] For the note of the sub-image region in the abnormal recognition state, the note is predicted based on the impurity coverage rate to identify the note of the sub-image region;

[0014] Detect the music score fluency based on the identified note;

[0015] Wherein, the note prediction includes predicting the note trajectory of the note in the impurity frame region to identify the note of the sub-image region, or predicting the note corresponding to the sub-image region based on the note in the corresponding adjacent sub-image region of the sub-image region to identify the note of the sub-image region.

[0016] Further, the process of obtaining the note filling rate and impurity coverage rate of the music score in the sub-image region comprises,

[0017] determining a number of note pixels within a note contour in the sub-image region;

[0018] determining a note fill rate as a ratio of the number of note pixels to a total number of pixels in the sub-image region;

[0019] determining a number of note bounding regions and a number of impurity bounding regions;

[0020] determining an impurity coverage rate as a ratio of a total number of pixels in the impurity bounding regions to a total number of pixels in the note bounding regions.

[0021] Further, the process of calculating the score blur representation coefficient of the sub-image region comprises,

[0022] determining a first blur influence factor as a ratio of the preset note fill rate to the note fill rate;

[0023] determining a second blur influence factor as a ratio of the impurity coverage rate to a preset impurity coverage rate;

[0024] determining a weighted sum of the first blur influence factor and the second blur influence factor as the score blur representation coefficient.

[0025] Further, the process of determining a score blur state of the sub-image region comprises,

[0026] if the score blur representation coefficient is greater than a reference score blur representation coefficient, determining the score blur state as blurred;

[0027] if the score blur representation coefficient is less than or equal to the reference score blur representation coefficient, determining the score blur state as unblurred.

[0028] Further, the process of calculating a note appearance rate based on the division result of the score blur state comprises,

[0029] if the score blur state is blurred, calculating the note appearance rate;

[0030] if the score blur state is unblurred, directly recognizing the note in the sub-image region;

[0031] wherein the note appearance rate is a chroma difference between a chroma of a texture region in the impurity bounding region and a chroma of a remaining region.

[0032] Further, the process of determining whether the note in the sub-image region is in an abnormal recognition state comprises,

[0033] if the note appearance rate is greater than or equal to a preset note appearance rate, determining that the note is in a non-abnormal recognition state;

[0034] If the note appearance rate is less than a preset note appearance rate, it is determined that the note is in an abnormal recognition state.

[0035] Further, the process of distinguishing the note contour feature from the non-note contour feature based on the contour fluency includes,

[0036] Determine several texture points in the texture;

[0037] Determine the pixel intensity of several texture points in the texture;

[0038] Based on the difference ratio of the pixel intensity of each texture point and the adjacent texture point;

[0039] Solve the average difference ratio, and take the average difference ratio as the contour fluency;

[0040] If the contour fluency is greater than a predetermined contour fluency threshold, it is determined that the texture is a note contour feature.

[0041] Further, the process of determining the impurity coverage rate of the note and predicting the note based on the impurity coverage rate includes,

[0042] If the impurity coverage rate is less than a reference impurity coverage rate, the note trajectory of the note in the impurity bounding box region is predicted to identify the note in the sub-image region;

[0043] If the impurity coverage rate is greater than or equal to the reference impurity coverage rate, the note corresponding to the sub-image region is predicted based on the note in the corresponding adjacent sub-image region of the sub-image region to identify the note in the sub-image region.

[0044] Further, the process of predicting the note trajectory of the note in the coverage region includes,

[0045] Determine the intersection of the note and the impurity bounding box region;

[0046] Determine the note contour segment where the intersection point is located, and calculate the tangent line at the intersection point on the note contour segment;

[0047] Extend the note contour segment in the direction of the tangent line to predict the note trajectory, and take the predicted note as the note in the sub-image region.

[0048] Further, the process of predicting the note corresponding to the sub-image region based on the note in the corresponding adjacent sub-image region of the sub-image region includes,

[0049] Screen the note, combine it with the note in the adjacent sub-image region, and form a note small sentence;

[0050] Determine the appearance probability of the note small sentence in several note small sentences in the music note library;

[0051] determine the note clause corresponding to the highest probability of occurrence;

[0052] determine the note in the note clause corresponding to the random screening;

[0053] determine the note as a sub-image region note.

[0054] Compared with the prior art, the present application acquires a music score image to be recognized, divides the music score image into regions to obtain a plurality of sub-image regions, acquires the note filling rate and impurity coverage rate of the music score in the sub-image region, calculates the music score blur representation coefficient of the sub-image region, determines the music score blur state of the sub-image region, calculates the note appearance rate, determines whether the note of the sub-image region is in an abnormal recognition state, and identifies the sub-image region based on the determination result, including, for the note of the sub-image region in a non-abnormal recognition state, performing differential contour feature processing on the note to identify the sub-image region note; for the note of the sub-image region in an abnormal recognition state, determining the impurity coverage rate of the note, and performing note prediction based on the impurity coverage rate to identify the sub-image region note, and detecting the music score fluency based on the identified note, thereby improving the music score recognition efficiency and accuracy.

[0055] In particular, by calculating the music score blur representation coefficient of the sub-image region, the music score blur state of the sub-image region is determined to distinguish the music score note recognition method. In the existing music score recognition process, if the same recognition method is adopted for each note in the music score, errors may occur in the recognition of some notes. Therefore, in order to improve the accuracy of music score recognition, the present application divides the music score into regions, calculates the music score blur representation coefficient based on the note filling rate and impurity coverage rate, determines the music score blur state of each region, directly recognizes the non-blurred music score, calculates the note appearance rate of the blurred music score, provides a data basis for determining the note state of the sub-image region, and improves the music score recognition efficiency and accuracy.

[0056] In particular, by determining whether the note of the sub-image region is in an abnormal recognition state based on the note appearance rate, the music score note recognition process of different categories is determined. When recognizing the blurred music score, the note may be covered by a small amount of texture, or the note may be blocked by a large amount of stains. The appearance of the note in the impurity may be different after being covered by different impurities. Some notes may be partially visible, and some notes may be completely covered and difficult to recognize. Different recognition methods are determined for notes in different situations to effectively improve the note recognition efficiency and improve the music score recognition accuracy. Therefore, the present application divides the notes into non-abnormal recognition states and abnormal recognition states to adaptively determine the recognition method, thereby improving the music score recognition efficiency and accuracy.

[0057] Especially, for the case that the note is blocked by a large piece of stain, the note may be partially covered and completely covered, in order to ensure the accuracy of the music score recognition, the application considers determining a prediction method for the covering state of the note, preferably, after recognizing the music score, the music score fluency is detected based on the recognized note, which improves the music score recognition efficiency and recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A step schematic diagram of the environment adaptive music score recognition method of the embodiment of the application;

[0059] Figure 2 A logic block diagram of determining the music score blur state of the sub-image region of the embodiment of the application;

[0060] Figure 3 A logic block diagram of calculating the note appearance rate based on the division result of the music score blur state of the embodiment of the application;

[0061] Figure 4 A logic block diagram of determining whether the note of the sub-image region is in an abnormal recognition state of the embodiment of the application. DETAILED DESCRIPTION

[0062] In order to make the objects and advantages of the present application clearer, the present application will be further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the present application.

[0063] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application, and are not intended to limit the protection scope of the present application.

[0064] Please refer to Figure 1 , Figure 1 A step schematic diagram of the environment adaptive music score recognition method of the embodiment of the application. The environment adaptive music score recognition method of the present application comprises:

[0065] An image of a music score to be recognized is acquired, and the music score image is regionally divided to obtain a plurality of sub-image regions. It can be understood that the image of the music score to be recognized can be a paper version music score image or an electronic version music score image;

[0066] The note filling rate and the impurity coverage rate of the music score in the sub-image region are acquired, and a music score blur representation coefficient of the sub-image region is calculated based on the note filling rate and the impurity coverage rate, and the music score blur state of the sub-image region is determined;

[0067] Calculate a note appearance rate based on the division result of the score ambiguity state, determine whether a note in a sub-image region is in an abnormal recognition state, and recognize the sub-image region based on a determination result, including,

[0068] For a note in a sub-image region in the non-abnormal recognition state, perform a differentiated contour feature processing on the note, including distinguishing note contour features from non-note contour features based on contour fluency, and extracting note contour features to recognize a note in a sub-image region.

[0069] For a note in a sub-image region in the abnormal recognition state, perform note prediction based on the impurity coverage to recognize a note in a sub-image region.

[0070] Detect score fluency based on the recognized note.

[0071] The note prediction includes predicting a note trajectory of the note in an impurity bounding box region to recognize a note in a sub-image region, or predicting a note corresponding to a sub-image region based on a note in a corresponding adjacent sub-image region to recognize a note in the sub-image region.

[0072] Specifically, the specific manner and division size of the region division are not limited, as long as there is a note in each region. In this embodiment, the division is performed according to the number of notes. Of course, the number of lines and the number of score paragraphs can also be used to divide the regions, which will not be described again.

[0073] Specifically, the process of obtaining the note filling rate and the impurity coverage of the score in the sub-image region includes,

[0074] Determine the number of note pixels in the note contour in the sub-image region.

[0075] Determine the ratio of the number of note pixels to the total number of pixels in the sub-image region as the note filling rate.

[0076] Determine a plurality of note bounding box regions and a plurality of impurity bounding box regions.

[0077] Determine the ratio of the total number of pixels of the impurity bounding box regions contained in the note bounding box region to the total number of pixels of the note bounding box region as the impurity coverage.

[0078] Specifically, the determination process of the note bounding box region is as follows,

[0079] Determine the note contour in the sub-image region.

[0080] Determine the note contour as a note bounding box region.

[0081] It can be understood that the impurities can be textures or several different shaped stains.

[0082] Specifically, the identification manner of the note contour and the impurity is not limited, for example, an image segmentation algorithm can be used to identify the corresponding contour, or an image processing model or algorithm capable of identifying the corresponding contour is trained, which will not be repeated here.

[0083] Specifically, the process of calculating the score blur representation coefficient of the sub-image region comprises,

[0084] The ratio of the preset note filling rate to the note filling rate is determined as a first blur influence factor;

[0085] The ratio of the impurity coverage rate to the preset impurity coverage rate is determined as a second blur influence factor;

[0086] The weighted sum value of the first blur influence factor and the second blur influence factor is determined as the score blur representation coefficient.

[0087] Specifically, the preset note filling rate is obtained by prior calculation, and the average value of the note filling rates of the clear scores of several same melodies is determined as the preset note filling rate.

[0088] Specifically, in order to represent the identification of the influence of the impurity covering the note, the preset impurity coverage rate can be selected in the interval [0.2, 0.5].

[0089] Specifically, the sum of the weight coefficients of the first blur influence factor and the second blur influence factor is 1, the weight coefficient of the first blur influence factor is 0.46, and the weight coefficient of the second blur influence factor is 0.54.

[0090] Specifically, by calculating the score blur representation coefficient of the sub-image region, the score blur state of the sub-image region is determined to distinguish the note recognition method. In the existing score recognition process, if the same recognition method is adopted for each note in the score, errors will be caused in the recognition of part of the notes. Based on this, in order to improve the accuracy of score recognition, the present application considers dividing the score into regions, calculating the score blur representation coefficient based on the note filling rate and the impurity coverage rate, determining the score blur state of each region, directly recognizing the non-blurred score, calculating the note appearance rate of the blurred score, providing data basis for determining the note state of the sub-image region, and improving the score recognition efficiency and recognition accuracy.

[0091] Please refer to Figure 2 , Figure 2 The logic block diagram for determining the score blur state of the sub-image region of the embodiment of the present application.

[0092] Specifically, a music score blur state of the sub-image region is determined, wherein,

[0093] If the music score blur representation coefficient is greater than the reference music score blur representation coefficient, the music score blur state is determined to be blurred.

[0094] If the music score blur representation coefficient is less than or equal to the reference music score blur representation coefficient, the music score blur state is determined to be unblurred.

[0095] Specifically, the reference music score blur representation coefficient represents the clarity of the music score. In the prior art, fine textures do not affect the recognition accuracy of the music score, and music score recognition can be performed through simple texture preprocessing, so the reference music score blur representation coefficient is selected in the interval [1.12, 1.29].

[0096] Please refer to Figure 3 , Figure 3 For the division result of the music score blur state of the embodiment of the application, a logic block diagram for calculating the note appearance rate is provided. Specifically, based on the division result of the music score blur state, the note appearance rate is calculated, wherein,

[0097] If the music score blur state is blurred, the note appearance rate is calculated.

[0098] If the music score blur state is unblurred, the notes in the sub-image region are directly recognized.

[0099] The note appearance rate is the chroma difference between the chroma of the texture in the impurity framed region and the chroma of the impurity framed region.

[0100] It can be understood that the possible appearance and disappearance of the notes caused by the impurities shielding the notes is embodied in the form of texture, or the flow inside the impurities itself can also cause the texture to appear, which will not be described again.

[0101] It can be understood that the method for directly recognizing the notes in the sub-image region is not limited, for example, in the present embodiment, an optical music score recognition method is used to quickly recognize the unblurred music score, and the specific recognition process is as follows,

[0102] The image is processed, including denoising, binarization, tilt correction, etc., to improve the image quality and reduce interference.

[0103] The notes in the music score are located through statistical transformation, structure search or cross-correlation-based method.

[0104] The staff lines are deleted using pixel analysis, run-length analysis or graph segment topological relationship method, etc. for subsequent recognition.

[0105] The features of the note primitives such as notes, stems and note tails are extracted from the image using convolutional neural networks.

[0106] Identify the note primitives such as note head, note stem, rest, etc. by pattern recognition technology;

[0107] Recombine the note primitives into complete note objects by using the score knowledge rules;

[0108] Of course, the skilled person in the art can also identify the notes of the sub-image area in other ways, which will not be described again.

[0109] Please refer to Figure 4 , Figure 4 The logic block diagram for determining whether the note of the sub-image area is in an abnormal recognition state for the embodiment of the application. Specifically, it is determined whether the note of the sub-image area is in an abnormal recognition state, wherein,

[0110] If the note appearance rate is greater than or equal to the preset note appearance rate, it is determined that the note is in a non-abnormal recognition state;

[0111] If the note appearance rate is less than the preset note appearance rate, it is determined that the note is in an abnormal recognition state.

[0112] Specifically, the preset note appearance rate is calculated in advance and represents the occlusion of the note and the identifiable probability. The preset note appearance rate is selected within the interval [0.6, 0.8].

[0113] Specifically, it is determined whether the note of the sub-image area is in an abnormal recognition state by the note appearance rate to determine the note recognition process of different categories of scores. When recognizing fuzzy scores, there may be a case where a note is covered by a small amount of texture, and there may also be a case where a note is occluded by a large amount of stains. The appearance of a note in impurities after being covered by different impurities may be different. Some may be partially visible, and some may be completely covered and not easy to identify. Different recognition methods are determined for notes in different situations, which can effectively improve the note recognition efficiency and improve the score recognition accuracy. Based on this, the present application divides the notes into non-abnormal recognition state and abnormal recognition state, and adaptively determines the recognition method, thereby improving the score recognition efficiency and recognition accuracy.

[0114] Specifically, the process of distinguishing the note contour feature from the non-note contour feature based on the contour fluency includes,

[0115] Determine a number of textures in the impurity frame area;

[0116] Determine the pixel intensity of a number of texture points in the texture;

[0117] Based on the difference ratio of the pixel intensity of each texture point and the adjacent texture point;

[0118] Solve the difference ratio mean value, and take the difference ratio mean value as the contour fluency;

[0119] If the contour fluency is greater than a predetermined contour fluency threshold, it is determined that the texture is a note contour feature.

[0120] Specifically, the contour fluency of the texture is not limited in the calculation manner, and other manners such as curvature analysis method and image processing algorithm method can also be used by those skilled in the art to calculate the contour fluency of the texture, which will not be repeated here.

[0121] Specifically, the contour fluency threshold is obtained by prior calculation. The contour fluency mean of a plurality of notes is obtained in advance, and the product of the contour fluency mean and the precision coefficient is determined as the reference fluency, and the precision coefficient is selected in the interval [0.5, 0.7].

[0122] It can be understood that when the note recognition is performed, part of the area of the note will be blocked by the transparent stain. For this kind of situation, the note needs to be preprocessed, and in this embodiment, the note contour feature is considered to be identified.

[0123] Specifically, the impurity coverage of the note is determined, and the process of predicting the note based on the impurity coverage includes,

[0124] If the impurity coverage is less than the reference impurity coverage, the note trajectory of the note in the impurity framed area is predicted to identify the note in the sub-image area;

[0125] If the impurity coverage is greater than or equal to the reference impurity coverage, the note corresponding to the sub-image area is predicted based on the note in the corresponding adjacent sub-image area of the sub-image area to identify the note in the sub-image area.

[0126] Specifically, the reference impurity coverage is determined based on the preset impurity coverage and is set to 1.15 to 1.3 times of the preset impurity coverage.

[0127] Specifically, the process of predicting the note trajectory of the note in the coverage area includes,

[0128] determining the intersection of the note and the impurity framed area;

[0129] determining the note contour segment where the intersection point is located, and calculating the tangent line at the intersection point on the note contour segment;

[0130] The note contour segment is extended along the tangent line direction to predict the note trajectory, and the predicted note is taken as the note in the sub-image area.

[0131] Specifically, the process of predicting the note corresponding to the sub-image area based on the note in the corresponding adjacent sub-image area of the sub-image area includes,

[0132] Screening the musical note, combining with the musical note in the adjacent sub-image region, to form a musical note sentence;

[0133] Determining the appearance probability of the musical note sentence in a plurality of musical note sentences in the musical note library;

[0134] Determining the musical note sentence corresponding to the highest appearance probability;

[0135] Determining the musical note corresponding to the random screening in the musical note sentence;

[0136] Determining the musical note as the sub-image region musical note.

[0137] It can be understood that in this case, the most reasonable musical note that matches is found as the predicted sub-image region musical note, ensuring reliability, and if the music image is a partial image of the entire music, the information presented is still suitable in the case of less information, and has high universality.

[0138] Specifically, the establishment method of the musical note library is not limited, for example, an existing musical note library can be directly selected, or a musical note library can be established according to an existing music score, and it can be understood that for the self-established musical note library, the music score is in a dynamic increasing state, which will not be repeated here.

[0139] Specifically, for the case where the musical note is blocked by a large amount of dirt, the musical note may be partially covered or completely covered, in order to ensure the accuracy of music recognition, the present application considers determining a prediction method for the coverage state of the musical note, preferably, after recognizing the music, the fluency of the music is detected based on the recognized musical note, which improves the music recognition efficiency and recognition accuracy.

[0140] So far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without deviating from the principles of the present application, those skilled in the art can make equivalent changes or replacements to related technical features, and the technical solutions after these changes or replacements will fall within the protection scope of the present application.

[0141] The above only describes the preferred embodiments of the present application and is not intended to limit the present application; for those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. An environment-adaptive music score recognition method, characterized in that: include: Acquire a music score image to be recognized, and divide the music score image into regions to obtain a plurality of sub-image regions; Obtaining a note filling rate and an impurity coverage rate of the music score in the sub-image area, calculating a music score fuzzy representation coefficient of the sub-image area based on the note filling rate and the impurity coverage rate, and determining a music score fuzzy state of the sub-image area; Based on the division result of the fuzzy state of the music score, the note appearance rate is calculated, and whether the note in the sub-image area is in an abnormal recognition state is determined, so as to identify the sub-image area based on the determination result, including: For the musical notes in the sub-image area that are in a non-abnormal recognition state, performing contour feature distinguishing processing on the musical notes, including distinguishing the musical note contour features from the non-musical note contour features based on contour smoothness, and extracting the musical note contour features to identify the musical notes in the sub-image area; For the musical notes in the sub-image area that is in an abnormal recognition state, performing musical note prediction based on the impurity coverage rate to identify the musical notes in the sub-image area; Detecting the fluency of musical scores based on recognized notes; The note prediction includes predicting the note trajectory of the note in the impurity framed area to identify the note in the sub-image area, or predicting the note corresponding to the sub-image area based on the notes in the adjacent sub-image area corresponding to the sub-image area to identify the note in the sub-image area; The process of obtaining the note filling rate and impurity coverage rate of the music score in the sub-image area includes: determining the number of note pixels within the note outline within the sub-image area; Determine the ratio of the number of note pixels to the total number of pixels in the sub-image area as the note fill rate; Determine a number of note selection areas and a number of impurity selection areas; Determine the ratio of the total number of pixels of the impurity frame selection area contained in the note frame selection area to the total number of pixels of the note frame selection area as the impurity coverage rate; The process of calculating the fuzzy representation coefficient of the music score of the sub-image area includes: determining a ratio of a preset note filling rate to a note filling rate as a first fuzzy influencing factor; Determining a ratio of the impurity coverage rate to a preset impurity coverage rate as a second fuzzy influencing factor; Determining a weighted sum of the first fuzzy influence factor and the second fuzzy influence factor as a music score fuzzy representation coefficient; The note appearance rate is calculated based on the division result of the fuzzy state of the music score, wherein, If the fuzzy state of the music score is fuzzy, calculating the note appearance rate; If the blur state of the music score is not blurry, the musical notes in the sub-image area are directly recognized; The note appearance rate is the chromaticity difference between the chromaticity of the texture area and the chromaticity of the remaining area within the impurity frame selection area; The process of distinguishing the note contour features from the non-note contour features based on contour fluency includes: Determine several textures within the impurity selection area; Determine pixel intensities at a number of texture points in a texture; Based on the difference ratio of the pixel intensity of each texture point and its adjacent texture points; Solve the mean of the difference ratio and use it as the contour smoothness; If the contour smoothness is greater than a predetermined contour smoothness threshold, the texture is determined to be a note contour feature; The process of determining the impurity coverage of the note and performing note prediction based on the impurity coverage includes: If the impurity coverage is less than the baseline impurity coverage, predicting the note trajectory of the note in the impurity framed area to identify the note in the sub-image area; If the impurity coverage is greater than or equal to the benchmark impurity coverage, the musical note corresponding to the sub-image area is predicted based on the musical note in the adjacent sub-image area corresponding to the sub-image area to identify the musical note in the sub-image area.

2. The method for environmentally adaptive music score recognition according to claim 1, wherein: The determining of the fuzzy state of the music score of the sub-image area, wherein: If the music score fuzzy representation coefficient is greater than the reference music score fuzzy representation coefficient, determining that the music score fuzzy state is fuzzy; If the music score fuzzy representation coefficient is less than or equal to the reference music score fuzzy representation coefficient, the music score fuzzy state is determined to be unfuzzy.

3. The environment-adaptive music score recognition method according to claim 1, wherein: The step of determining whether the musical note in the sub-image area is in an abnormal recognition state comprises: If the note appearance rate is greater than or equal to a preset note appearance rate, the note is determined to be in a non-abnormal recognition state; If the note appearance rate is less than a preset note appearance rate, the note is determined to be in an abnormal recognition state.

4. The environment-adaptive music score recognition method according to claim 1, wherein: The process of predicting the note trajectory of the note in the coverage area includes: Determine the intersection of the note and the impurity selection area; Determine the note contour segment where the intersection point is located, and calculate the tangent line at the intersection point on the note contour segment; The note contour segment is extended along the tangent direction to predict the note trajectory, and the predicted note is used as the note in the sub-image area.

5. The environment-adaptive music score recognition method according to claim 1, wherein: The process of predicting the musical note corresponding to the sub-image region based on the musical note in the adjacent sub-image region corresponding to the sub-image region includes: Selecting notes and combining them with notes in adjacent sub-image areas to form a note phrase; Determining the probability of a note phrase appearing in a plurality of note phrases in a musical score note library; Determine the note phrase with the highest probability of occurrence; Determining the randomly selected notes corresponding to the note phrase; The musical note is determined as a sub-image area musical note.

Images