A method and system for identifying musical strings

Abstract

The present invention provides a method and system for identifying musical strings. The method comprises receiving an image to be detected that includes a musical instrument surface; identifying a musical instrument surface region in the image to be detected; segmenting the musical instrument surface region from the image to be detected to obtain a segmented image; performing image preprocessing on the segmented image to obtain a preprocessed image; identifying musical strings in the preprocessed image to obtain a preliminary recognition result; and performing anomaly detection processing on the preliminary recognition result to obtain a final recognition result for the musical strings in the image to be detected. The method utilizes various techniques, including filtering, morphological image processing, and pattern detection transformation algorithms, to perform musical string recognition. It also proposes anomaly detection criteria to address potential false detections and missed detections. The method can be used in a variety of environmental conditions, effectively identifying and detecting musical strings in the image to be detected, and significantly improving the accuracy of the detection results.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a string recognition method and system. Background Art

[0002] With the advancement of internet technology and the rapid expansion of informatization, traditional Chinese musical instruments, such as the guzheng, have ushered in new opportunities for inheritance and development. Traditional music-themed gaming, entertainment, and educational industries have attracted widespread attention. Considering the analysis of user interaction with musical instruments, the automatic recognition and processing of instrument entities in images or videos is undoubtedly a key focus of intelligent technology applications. This can replace the enormous workload of manual observation and extraction of individual user information. String recognition for traditional string instruments, such as the guzheng, is one of the most widely used technologies. Identifying strings in images or videos, and laying the foundation for subsequent analysis and processing of other content, has become a hot research area.

[0003] The continuous development and improvement of image processing technology has provided a possible solution to the above problems. However, it is still difficult to efficiently and accurately identify the strings in images under different environmental conditions. Factors such as ambient lighting, background debris, the texture of the instrument surface, and image pixels can interfere with the detection and recognition of the strings. The current research focuses on how to select and improve existing image detection and processing algorithms and obtain string position information through a comprehensive image processing process. Summary of the Invention

[0004] In view of the defects in the prior art, the present invention provides a string identification method and system, which improves the accuracy of string identification in Guzheng.

[0005] In a first aspect, a string identification method comprises:

[0006] Receive the image to be detected containing the piano surface;

[0007] Identify the piano surface area of ​​the image to be detected;

[0008] Segment the piano surface area from the image to be detected to obtain a segmented image;

[0009] Performing image preprocessing on the segmented image to obtain a preprocessed image;

[0010] Identify the strings in the preprocessed image to obtain preliminary recognition results;

[0011] The preliminary recognition results are processed for anomaly detection to obtain the final recognition results of the strings in the image to be detected.

[0012] Furthermore, identifying the piano surface area of ​​the image to be detected specifically includes:

[0013] Identify all identification codes in the image to be detected; the identification codes are set around the piano surface;

[0014] Determine the piano surface area based on all identification codes.

[0015] Furthermore, the identification code is set on the corner of the piano surface;

[0016] Segmenting the piano surface area from the image to be detected specifically includes:

[0017] Define a square detection box;

[0018] Calculate the coordinates of all identification codes in the plane rectangular coordinate system;

[0019] Calculate the coordinates of the corresponding corners of the detection frame in the plane rectangular coordinate system based on the coordinates of all identification codes;

[0020] Use the detection frame to segment the piano surface area from the image to be detected.

[0021] Furthermore, the image preprocessing for the segmented image specifically includes:

[0022] The segmented image is grayscaled, binarized, Gaussian filtered, edge detected and eroded.

[0023] Furthermore, identifying the strings in the pre-processed image specifically includes:

[0024] A line detection algorithm is used to identify line segments in the preprocessed image to obtain the strings;

[0025] Define the coordinates of the two endpoints of the line segment as the coordinates of the endpoints of the string in the plane rectangular coordinate system;

[0026] Define the slope of the line segment as the slope of the string;

[0027] Define the angle between the line segment and the horizontal direction as the angle between the string and the horizontal direction.

[0028] Furthermore, the abnormality detection process for the preliminary recognition results specifically includes:

[0029] Filter out abnormal strings from the preliminary recognition results based on the angle between the strings and the horizontal direction;

[0030] Screening out broken strings from the remaining strings after filtering out abnormal strings according to the endpoint coordinates of the strings;

[0031] The remaining strings after the broken strings are filtered out are supplemented.

[0032] Furthermore, the abnormal strings are screened out from the preliminary recognition results based on the angle between the strings and the horizontal direction, specifically including:

[0033] Mark the angle between the string and the horizontal direction in the fitting coordinate system. The abscissa of the fitting coordinate system is the ordinate of the right endpoint of the string in the plane rectangular coordinate system, and the ordinate of the fitting coordinate system is the angle between the string and the horizontal direction.

[0034] Fit all the strings marked in the fitting coordinate system and filter out abnormal strings.

[0035] Furthermore, screening out broken strings from the remaining strings after screening out abnormal strings according to the endpoint coordinates of the strings specifically includes:

[0036] Set the horizontal coordinate range of Qianyue Mountain;

[0037] Get the horizontal coordinates of the right endpoints of all strings in the plane rectangular coordinate system;

[0038] When the horizontal coordinate of the right end point of the string does not meet the horizontal coordinate range of the front yoke, the string is defined as broken;

[0039] Remove broken strings;

[0040] Extend the right end points of all the remaining strings after deleting the broken strings to the front nut position.

[0041] Furthermore, the missing strings remaining after the broken strings are screened out are supplemented by:

[0042] Determine the coordinates of the two endpoints of the missing string in the plane rectangular coordinate system based on the vertical coordinates of the right endpoints of the remaining strings after the broken strings are screened out in the plane rectangular coordinate system;

[0043] Draw the missing chord based on the coordinates of its two endpoints.

[0044] In a second aspect, a string identification system includes:

[0045] Input unit: used to receive the image to be detected containing the piano surface;

[0046] Piano surface detection unit: used to identify the piano surface area in the image to be detected;

[0047] Image segmentation unit: used to segment the piano surface area from the image to be detected to obtain a segmented image;

[0048] Image preprocessing unit: used for performing image preprocessing on the segmented image to obtain a preprocessed image;

[0049] Recognition unit: used to identify the strings in the pre-processed image to obtain preliminary recognition results;

[0050] Anomaly detection unit: used to perform anomaly detection processing on the preliminary recognition results to obtain the final recognition results of the strings in the image to be detected.

[0051] As can be seen from the above technical solution, the string recognition method and system provided by the present invention utilize multiple technologies such as filtering technology, morphological image processing, and graphic detection transformation algorithm to perform string recognition, and propose abnormal detection judgment for possible false detection, missed detection, etc. The string recognition method can be used under a variety of different environmental conditions, effectively identifying and detecting the strings in the image to be detected, and greatly improving the accuracy of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for the specific embodiments or the description of the prior art. Similar elements or parts are generally identified by similar reference numerals throughout the drawings. Elements or parts in the drawings are not necessarily drawn to scale.

[0053] Figure 1 This is a flow chart of the string recognition method provided in this embodiment.

[0054] Figure 2 The picture to be detected is provided in this embodiment.

[0055] Figure 3 A schematic diagram of the principle of the etching operation provided in this embodiment.

[0056] Figure 4 A schematic diagram illustrating the principle of the expansion operation provided in this embodiment.

[0057] Figure 5 This is a schematic diagram of the fitting results provided in this embodiment.

[0058] Figure 6 This is the Hough transform detection result provided by this embodiment.

[0059] Figure 7 This is the result of screening out abnormal angle segments provided in this embodiment.

[0060] Figure 8 The result is obtained after judging and filtering out broken lines and aligning them provided in this embodiment.

[0061] Figure 9 The result is obtained after the strings provided in this embodiment are padded.

[0062] Figure 10 This is a module block diagram of the string recognition system provided in this embodiment. DETAILED DESCRIPTION

[0063] The following embodiments of the technical solution of the present invention are described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore only examples and are not intended to limit the scope of protection of the present invention. It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0064] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0065] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0066] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0067] Example:

[0068] A string identification method, see Figure 1 ,include:

[0069] S1: Receive the image to be detected containing the piano surface;

[0070] S2: Identify the piano surface area of ​​the image to be detected;

[0071] S3: Segmenting the piano surface area from the image to be detected to obtain a segmented image;

[0072] S4: performing image preprocessing on the segmented image to obtain a preprocessed image;

[0073] S5: Identify the strings in the preprocessed image to obtain preliminary recognition results;

[0074] S6: Perform anomaly detection processing on the preliminary recognition result to obtain a final recognition result of the strings in the image to be detected.

[0075] In this embodiment, the image to be detected can be an image obtained by photographing the piano surface, for example Figure 2 The method first inputs the image to be detected, which includes the surface of the guzheng. Since the image to be detected also contains parts other than the surface of the guzheng, it may cause noise interference. Therefore, it is necessary to identify the surface area of ​​the image to be detected, obtain the specific location of the surface area in the image to be detected, and then cut the surface area out of the image to be detected. In order to improve the accuracy of string recognition, image processing technology is applied to preprocess the segmented image obtained by cutting, such as grayscale, binarization, Gaussian filtering, corrosion and dilation, edge detection and other preprocessing steps to obtain a preprocessed image. The strings in the preprocessed image are identified to obtain a preliminary recognition result; the preliminary recognition result is subjected to abnormality detection processing, such as abnormal detection result judgment and deletion, Y-axis coordinate point alignment, automatic rotation correction, missing string filling and other abnormality detection processing to obtain the final recognized strings. The string recognition method can also draw line segments based on the identified strings and output the drawn image.

[0076] This string recognition method uses multiple technologies such as filtering technology, morphological image processing, and graphic detection transformation algorithm to identify strings, and proposes abnormal detection judgment for possible false detection and missed detection. This string recognition method can be used in a variety of different environmental conditions, effectively identifying and detecting the strings in the image to be detected, greatly improving the accuracy of the detection results.

[0077] Furthermore, in some embodiments, identifying the piano surface area of ​​the image to be detected specifically includes:

[0078] Identify all identification codes in the image to be detected; the identification codes are set around the piano surface;

[0079] Determine the piano surface area based on all identification codes.

[0080] In this embodiment, in order to more accurately detect the surface area of ​​the guzheng and facilitate the subsequent accurate identification of the strings, the method can pre-set identification codes around the guzheng, for example, Figure 2 , vertically paste the QR code on the front of the guzheng. This method determines the surface area of ​​the guzheng based on all the identification codes.

[0081] Identification of the piano surface area can be achieved by the following method: loading the corresponding algorithm or running the program in software such as Matlab, inputting the image to be detected into the software such as Matlab, and using the algorithm or running the program to identify the position of the QR code, and then determining the piano surface area based on the position of the QR code. Since other interfering images around the identification code may be collected at the same time during the identification code recognition process, and these interfering images increase the complexity of image processing, these interfering images need to be removed. Therefore, the image to be detected can be binarized and then dilated, and the interfering images can be identified based on brightness. Then, the interfering images can be filtered through the Euler number, and the interfering area can be filled and completed after correction, so that the interference of the interfering images can be removed.

[0082] In this embodiment, the identification code may be composed of modules of fixed dark and light colors, for example, Figure 2 Observing the identification code from left to right, the light-dark color sequence is light-dark-light-dark-light-dark-light, and the width ratio of each element is 3:1:1:3:1:1:3. This ensures that even if the image being tested is rotated during capture, the color sequence and width ratio of the identification code modules remain unchanged. During identification code recognition, the binary image being tested is scanned row by row and column by column, and adjacent pixels of the same grayscale are recorded as line segments. If seven line segments exist with a length ratio of 3:1:1:3:1:1:3 and a light-dark color sequence of light-dark-light-dark-light-light-dark-light, then that line segment is recorded. After scanning, adjacent line segments are grouped together, and line segments that are not adjacent to any other line segments (possibly random interference) are removed. The same method is used to process the column segments. The groups that intersect with each other in the row segment group and the column segment group are classified, and the center points of the intersecting row and column segment groups are found, that is, the center position of the identification code is obtained.

[0083] Furthermore, in some embodiments, the identification code is provided on a corner of the piano surface;

[0084] Segmenting the piano surface area from the image to be detected specifically includes:

[0085] Define a square detection box;

[0086] Calculate the coordinates of all identification codes in the plane rectangular coordinate system;

[0087] Calculate the coordinates of the corresponding corners of the detection frame in the plane rectangular coordinate system based on the coordinates of all identification codes;

[0088] Use the detection frame to segment the piano surface area from the image to be detected.

[0089] In this embodiment, the method can set the identification code on the corner of the piano surface. Figure 2 When the image to be detected is detected, the center positions of the upper right corner identification code and the lower right corner identification code are expressed by coordinates in the plane rectangular coordinate system, the coordinates of the upper right corner of the detection frame are defined to be equal to the coordinates of the center position of the upper right corner identification code, and the coordinates of the lower right corner of the detection frame are defined to be equal to the coordinates of the center position of the lower right corner identification code. Finally, the image to be detected is segmented according to the coordinates of the upper right corner and the lower right corner of the detection frame to obtain the piano surface area.

[0090] Furthermore, in some embodiments, performing image preprocessing on the segmented image specifically includes:

[0091] The segmented image is grayscaled, binarized, Gaussian filtered, edge detected and eroded.

[0092] In this embodiment, the method first grayscales the segmented image. Grayscale conversion can first convert the image to be detected from a color RGB image to a grayscale image, making the target area more prominent, while reducing the memory occupied by the image and improving the running speed. Grayscale conversion can be implemented using the grayscale function rgb2gray() provided by Matlab. Its calculation principle is shown in the following formula, where the coefficients are derived from the sensitivity of the human eye to the three colors r, g, and b:

[0093] I(x, y) = 0.3*I R (x, y) + 0.59 * I G (x, y) + 0.11 * I B (x, y).

[0094] This method performs binarization on the grayscaled image, creating a distinct black-and-white effect. Binarization reduces the data dimension and eliminates noise interference in the image to be detected, highlighting the outline structure of the valid area. Binarization can be implemented using the Matlab function imbinarize(), where the threshold parameter set to imbinarize() is 0.8.

[0095] Since ideal noise-free conditions are unattainable in real-world environments, all images collected for inspection contain noise. This noise comes from a variety of sources, such as device noise and environmental interference. Considering that most noise is composed of salt and pepper noise and Gaussian noise, this method uses Gaussian filtering to remove noise and blurry areas from images. The specific calculation of the Gaussian kernel in Gaussian filtering is achieved through the Gaussian function, as shown in the following formula: For example, Gaussian filtering can use the functions fspecial() and imfilter() in Matlab to complete the creation and filtering of the filter. The Gaussian kernel should not be too large and can be set to 5*5, so as to avoid edge recognition errors due to the loss of edge information. Gaussian filtering has the following three advantages: First, the Gaussian kernel is a two-dimensional square with rotational symmetry, which can ensure that the edge information of the filtered image is not lost, because the four directions of the filtered image have the same smoothing trend; second, the central pixel in the Gaussian filtering process is less affected by the neighboring pixels, especially the edge pixels, than other filters, because the value of the Gaussian kernel is the largest at the center and the smallest at the edge, and the value of the Gaussian kernel decreases with increasing distance; finally, the Gaussian filter is less affected by high-frequency information and can maintain the filtering effect.

[0096] After completing the Gaussian filter, this method can use the edge() function in Matlab to extract the edge contour of the image, improving detection efficiency. For example, you can select the Prewitt operator and set the threshold to 0.2 (return all edges stronger than the threshold). With this parameter, you can obtain relatively good edge detection results.

[0097] After completing edge detection, this method performs erosion and dilation on the binary image to better eliminate image noise. Erosion and dilation mainly convolve the binary image with the kernel to smooth the outline of objects in the image, break narrow necks, eliminate thin protrusions, and optimize subsequent detection results. During the erosion operation, the minimum value of the rectangular neighborhood of each position is taken as the output grayscale value of the position. This method can use a rectangular structure as the neighborhood, that is, a 01 binary matrix. The principle is shown in Figure 3 Similarly, during the expansion operation, the maximum value of the rectangular neighborhood value of each position is taken as the output grayscale value of the position. For the principle, see Figure 4 .

[0098] Furthermore, in some embodiments, identifying the strings in the pre-processed image specifically includes:

[0099] A line detection algorithm is used to identify line segments in the preprocessed image to obtain the strings;

[0100] Define the coordinates of the two endpoints of the line segment as the coordinates of the endpoints of the string in the plane rectangular coordinate system;

[0101] Define the slope of the line segment as the slope of the string;

[0102] Define the angle between the line segment and the horizontal direction as the angle between the string and the horizontal direction.

[0103] In this embodiment, the method can use the Hough line detection algorithm to identify strings. This allows the identification of line segments and the coordinates of their two endpoints, subject to certain constraints. The slope of the segment and the angle between the segment and the horizontal can then be calculated from the coordinates of the two endpoints. This allows the slope and angle of the string to be determined, ultimately determining the complete string position. The line detection algorithm can be implemented using the Matlab functions hough(), houghpeaks(), and houghlines(). The parameters in these functions can be determined based on experimental test data. This implementation also removes line segments that deviate too far.

[0104] Furthermore, in some embodiments, performing anomaly detection processing on the preliminary recognition result specifically includes:

[0105] Filter out abnormal strings from the preliminary recognition results based on the angle between the strings and the horizontal direction;

[0106] Screening out broken strings from the remaining strings after filtering out abnormal strings according to the endpoint coordinates of the strings;

[0107] The remaining strings after the broken strings are filtered out are supplemented.

[0108] In this embodiment, the method also requires abnormal line segment identification and processing of the preliminary string recognition results (including the corresponding coordinates of the string endpoints, the string slope, the angle with the horizontal direction, etc.), deleting redundant strings (such as those caused by misjudgment of the string surface texture) and filling in the missing strings to ensure that the final output is 21 strings and that they are regularly arranged on the Hough map.

[0109] Furthermore, in some embodiments, screening out abnormal strings from the preliminary recognition results based on the angle between the strings and the horizontal direction specifically includes:

[0110] Mark the angle between the string and the horizontal direction in the fitting coordinate system. The abscissa of the fitting coordinate system is the ordinate of the right endpoint of the string in the plane rectangular coordinate system, and the ordinate of the fitting coordinate system is the angle between the string and the horizontal direction.

[0111] Fit all the strings marked in the fitting coordinate system and filter out abnormal strings.

[0112] In this embodiment, when performing abnormality detection, the method can filter out abnormal strings based on the angle between the detected strings and the horizontal direction. Since most strings should be roughly parallel, the method can fit the angle. The fitting results can be seen in Figure 5 , filter out the abnormal points, and thus obtain abnormal strings. This method can also delete abnormal strings.

[0113] For example Figure 5 Each point in the diagram represents a string. The horizontal coordinate of each point is the vertical coordinate of the right endpoint of the string in the plane rectangular coordinate system (the distribution range of the vertical coordinate of the right endpoint of the string in the plane rectangular coordinate system is between 100-700), and the vertical coordinate of each point is the angle between the string and the horizontal direction (the distribution range of the angle between the string and the horizontal direction is between 86-92). Figure 5 The curve in is the fitting curve, where Figure 5 The hollow point is an abnormal point because it deviates far from the fitting curve. The string corresponding to the abnormal point is an abnormal string. The point with an × in the middle is a normal point, and the string corresponding to the normal point is a normal string.

[0114] Furthermore, in some embodiments, screening out broken strings from the remaining strings after screening out abnormal strings based on the endpoint coordinates of the strings specifically includes:

[0115] Set the horizontal coordinate range of Qianyue Mountain;

[0116] Get the horizontal coordinates of the right endpoints of all strings in the plane rectangular coordinate system;

[0117] When the horizontal coordinate of the right end point of the string does not meet the horizontal coordinate range of the front yoke, the string is defined as broken;

[0118] Remove broken strings;

[0119] Extend the right end points of all the remaining strings after deleting the broken strings to the front nut position.

[0120] In this embodiment, after removing abnormal strings, the method also identifies and removes broken strings based on the string coordinates. Because the endpoints of all string segments near the front shank should lie roughly on the same vertical line, a shank abscissa range is defined to represent the abscissa values ​​of points near the shank. If the endpoint coordinates of a line segment deviate significantly beyond the shank abscissa range, for example, if they are less than the lower limit of the shank abscissa range, this indicates that the endpoint coordinates of the line segment are significantly offset from the front shank, making it highly likely that the string is broken. This string is then removed, and the endpoints of the remaining strings are extended to the vicinity of the front shank using the Hough parameter.

[0121] Furthermore, in some embodiments, filling in the missing strings after the broken strings are screened out specifically includes:

[0122] Determine the coordinates of the two endpoints of the missing string in the plane rectangular coordinate system based on the vertical coordinates of the right endpoints of the remaining strings after the broken strings are screened out in the plane rectangular coordinate system;

[0123] Draw the missing chord based on the coordinates of its two endpoints.

[0124] In this embodiment, after removing broken strings, the method also performs a fill-in process for missing strings. The method determines a reasonable string spacing based on the vertical coordinates of the right endpoints between adjacent strings. If the distance between adjacent strings is approximately a multiple of the string spacing, it indicates that a string is missing between the two strings. For example, if the distance between adjacent strings is twice the string spacing, then a string may be missing between the two strings. The coordinates of the missing string and its endpoint are determined. If a string may be missing between the two strings, the horizontal coordinate of the missing string endpoint can be the position of the front Yueshan, and the vertical coordinate is the average of the vertical coordinates of the two strings. The angle between the missing string and the horizontal direction is determined based on the angle between the adjacent strings and the horizontal direction. If a string may be missing between the two strings, the angle between the missing string and the horizontal direction is between the angles between the two strings and the horizontal direction. Finally, the missing string is mapped based on this endpoint information and the angle.

[0125] To further illustrate the accuracy of the string recognition method, the following experimental results are given:

[0126] Matlab 2019b was selected as the experimental platform, and 30 images of the Guzheng surface under different environmental conditions were taken as input data for the experiment. The program was run on Matlab to read the Guzheng images for string recognition. Figure 6 is the Hough transform detection result, Figure 7 Results of screening out abnormal angle segments. Figure 8 To judge and filter out broken lines and get the result after alignment, Figure 9 The result is obtained after filling the strings.

[0127] The experimental baseline solution uses a traditional image recognition algorithm without anomaly identification and processing modules. Image preprocessing involves binarization and dilation corrosion, and line determination uses the Hough transform. The experiment uses four environmental scenarios: Scenario 1 (indoor low light), Scenario 2 (indoor high light), Scenario 3 (outdoor low light), and Scenario 4 (outdoor high light). The overall experimental results are shown in Table 1:

[0128] Table 1

[0129]

[0130] It can be seen that in the four scenarios, the circular error of the string recognition method provided by the present application is only 1.9 to 4.2 pixels, indicating that the string recognition method provided by the present application can accurately identify the strings in most cases.

[0131] A string identification system, see Figure 10 ,include:

[0132] Input unit 1: used to receive the image to be detected containing the piano surface;

[0133] Piano surface detection unit 2: used to identify the piano surface area of ​​the image to be detected;

[0134] Image segmentation unit 3: used to segment the piano surface area from the image to be detected to obtain a segmented image;

[0135] Image preprocessing unit 4: used for performing image preprocessing on the segmented image to obtain a preprocessed image;

[0136] Recognition unit 5: used to identify the strings in the pre-processed image to obtain preliminary recognition results;

[0137] Anomaly detection unit 6: used to perform anomaly detection processing on the preliminary recognition result to obtain the final recognition result of the strings in the image to be detected.

[0138] Furthermore, in some embodiments, the piano surface detection unit 2 is specifically used to:

[0139] Identify all identification codes in the image to be detected; the identification codes are set around the piano surface;

[0140] Determine the piano surface area based on all identification codes.

[0141] Furthermore, in some embodiments, the identification code is provided on at least three corners of the piano surface;

[0142] The picture segmentation unit 3 is specifically used for:

[0143] Define a square detection box;

[0144] Calculate the coordinates of all identification codes in the plane rectangular coordinate system;

[0145] Calculate the coordinates of the four corners of the detection box in the plane rectangular coordinate system based on the coordinates of all identification codes;

[0146] According to the coordinates of the four corners of the detection frame, the piano surface area is segmented from the image to be detected.

[0147] Furthermore, in some embodiments, the image pre-processing unit 4 is specifically configured to:

[0148] The segmented image is grayscaled, binarized, Gaussian filtered, edge detected and eroded.

[0149] Furthermore, in some embodiments, the identification unit 5 is specifically configured to:

[0150] A line detection algorithm is used to identify line segments in the preprocessed image to obtain the strings;

[0151] Define the coordinates of the two endpoints of the line segment as the coordinates of the endpoints of the string in the plane rectangular coordinate system;

[0152] Define the slope of the line segment as the slope of the string;

[0153] Define the angle between the line segment and the horizontal direction as the angle between the string and the horizontal direction.

[0154] Furthermore, in some embodiments, the anomaly detection unit 6 is specifically configured to:

[0155] Filter out abnormal strings from the preliminary recognition results based on the angle between the strings and the horizontal direction;

[0156] Screening out broken strings from the remaining strings after filtering out abnormal strings according to the endpoint coordinates of the strings;

[0157] The remaining strings after the broken strings are filtered out are supplemented.

[0158] Furthermore, in some embodiments, the anomaly detection unit 6 is specifically configured to:

[0159] Mark the angle between the string and the horizontal direction in the fitting coordinate system. The abscissa of the fitting coordinate system is the ordinate of the right endpoint of the string in the plane rectangular coordinate system, and the ordinate of the fitting coordinate system is the angle between the string and the horizontal direction.

[0160] Fit all the strings marked in the fitting coordinate system and filter out abnormal strings.

[0161] Furthermore, in some embodiments, the anomaly detection unit 6 is specifically configured to:

[0162] Set the horizontal coordinate range of Qianyue Mountain;

[0163] Get the horizontal coordinates of the right endpoints of all strings in the plane rectangular coordinate system;

[0164] When the horizontal coordinate of the right end point of the string does not meet the horizontal coordinate range of the front yoke, the string is defined as broken;

[0165] Remove broken strings;

[0166] Extend the right end points of all the remaining strings after deleting the broken strings to the front nut position.

[0167] Furthermore, in some embodiments, the anomaly detection unit 6 is specifically configured to:

[0168] Determine the coordinates of the two endpoints of the missing string in the plane rectangular coordinate system based on the vertical coordinates of the right endpoints of the remaining strings after the broken strings are screened out in the plane rectangular coordinate system;

[0169] Draw the missing chord based on the coordinates of its two endpoints.

[0170] The system provided in the embodiment of the present invention is briefly described. For matters not mentioned in the embodiment part, reference may be made to the corresponding content in the aforementioned embodiment.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A method for identifying musical strings, characterized in that: include: Receive the image to be detected containing the piano surface; Identifying the piano surface area of ​​the image to be detected; Segmenting the piano surface area from the image to be detected to obtain a segmented image; Performing image preprocessing on the segmented image to obtain a preprocessed image; Identifying the strings in the pre-processed image to obtain a preliminary recognition result; Performing anomaly detection processing on the preliminary recognition result to obtain a final recognition result of the strings in the image to be detected; The identifying of the musical string in the pre-processed image specifically includes: using a line detection algorithm to identify a line segment in the pre-processed image to obtain the musical string; defining the coordinates of two endpoints of the line segment as the coordinates of the endpoints of the musical string in a plane rectangular coordinate system; defining the slope of the line segment as the slope of the musical string; and defining the angle between the line segment and the horizontal direction as the angle between the musical string and the horizontal direction; The abnormality detection processing of the preliminary recognition result specifically includes: screening abnormal strings from the preliminary recognition result according to the angle between the strings and the horizontal direction; screening broken strings from the strings remaining after the abnormal strings are screened out according to the endpoint coordinates of the strings; and filling in the missing strings remaining after the broken strings are screened out; The step of screening out abnormal strings from the preliminary identification results based on the angle between the string and the horizontal direction specifically includes: marking the angle between the string and the horizontal direction in a fitting coordinate system, where the abscissa of the fitting coordinate system is the ordinate of the right endpoint of the string in a plane rectangular coordinate system, and the ordinate of the fitting coordinate system is the angle between the string and the horizontal direction; and fitting all strings marked in the fitting coordinate system to screen out abnormal strings.

2. The string identification method according to claim 1, characterized in that: The identifying of the piano surface area of ​​the image to be detected specifically includes: Identify all identification codes in the image to be detected; the identification codes are set around the piano surface; The piano surface area is determined according to all the identification codes.

3. The string identification method according to claim 2, characterized in that: The identification code is set on the corner of the piano surface; Segmenting the piano surface area from the image to be detected specifically includes: Define a square detection box; Calculating the coordinates of all the identification codes in a plane rectangular coordinate system; Calculating the coordinates of the corresponding corners of the detection frame in a plane rectangular coordinate system according to the coordinates of all the identification codes; The piano surface area is segmented from the image to be detected using the detection frame.

4. The string identification method according to claim 1, characterized in that: The performing image preprocessing on the segmented image specifically includes: The segmented image is sequentially subjected to grayscale conversion, binarization, Gaussian filtering, edge detection, and erosion and dilation.

5. The string identification method according to claim 1, characterized in that: The step of selecting broken strings from the remaining strings after selecting abnormal strings according to the endpoint coordinates of the strings specifically includes: Set the horizontal coordinate range of Qianyue Mountain; Obtaining the horizontal coordinates of the right endpoints of all the strings in a rectangular coordinate system; When the horizontal coordinate of the right end point of the string does not meet the horizontal coordinate range of the front yoke, the string is defined as broken; Deleting the broken string; Extend the right end points of all the remaining strings after deleting the broken strings to the front nut position.

6. The string identification method according to claim 5, characterized in that: The method of completing the missing strings after the broken strings are screened out specifically includes: Determine the coordinates of the two endpoints of the missing string in the plane rectangular coordinate system based on the vertical coordinates of the right endpoints of the remaining strings after the broken strings are screened out in the plane rectangular coordinate system; The missing chord is drawn according to the coordinates of the two endpoints of the missing chord.

7. A string identification system, characterized in that: include: Input unit: used to receive the image to be detected containing the piano surface; A piano surface detection unit: used to identify the piano surface area of ​​the image to be detected; Image segmentation unit: used for segmenting the piano surface area from the image to be detected to obtain a segmented image; Image preprocessing unit: used for performing image preprocessing on the segmented image to obtain a preprocessed image; an identification unit configured to identify the strings in the preprocessed image to obtain a preliminary identification result, wherein the identification of the strings in the preprocessed image specifically comprises: using a line detection algorithm to identify line segments in the preprocessed image to obtain the strings; defining the coordinates of the two endpoints of the line segment as the coordinates of the endpoints of the string in a plane rectangular coordinate system; defining the slope of the line segment as the slope of the string; and defining the angle between the line segment and the horizontal direction as the angle between the string and the horizontal direction; Anomaly detection unit: used for performing anomaly detection processing on the preliminary recognition result to obtain a final recognition result of the strings in the image to be detected; The abnormality detection processing of the preliminary recognition result specifically includes: screening abnormal strings from the preliminary recognition result according to the angle between the strings and the horizontal direction; screening broken strings from the strings remaining after the abnormal strings are screened out according to the endpoint coordinates of the strings; and filling in the missing strings remaining after the broken strings are screened out; The step of screening out abnormal strings from the preliminary identification results based on the angle between the string and the horizontal direction specifically includes: marking the angle between the string and the horizontal direction in a fitting coordinate system, where the abscissa of the fitting coordinate system is the ordinate of the right endpoint of the string in a plane rectangular coordinate system, and the ordinate of the fitting coordinate system is the angle between the string and the horizontal direction; and fitting all strings marked in the fitting coordinate system to screen out abnormal strings.

Images