Mould plate quality detection method and system based on machine vision

Abstract

The invention provides a mold plate quality detection method and system based on machine vision, and the method comprises the steps: obtaining a template image and a to-be-detected plate image through the shooting of a camera, and obtaining an image frame through an optical flow method; the method comprises the following steps: preprocessing an image, and extracting an edge through Canny edge detection; matching the feature points by using an RANSAC (Random Sample Consensus) algorithm, and removing mismatched point pairs; obtaining a feature line segment; matching the feature line segments by using an SURF algorithm, and calculating a matching line segment ratio; oRB feature point matching is carried out; carrying out line lofting registration; calculating comprehensive plate quality information; and obtaining a quality inspection result according to the quality information. According to the invention, efficient and accurate detection of the quality of the mold plate can be realized, and the automation level and accuracy of detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of automated detection technology, and more specifically, to a method and system for detecting quality of mold plates based on machine vision. Background Art

[0002] In the field of industrial manufacturing, quality inspection of mold panels is a key link in ensuring product quality. Traditional inspection methods mainly rely on manual visual inspection, which is inefficient and greatly affected by human factors, making it difficult to ensure the consistency and accuracy of inspection results. With the development of machine vision technology, automated visual inspection methods have gradually been introduced into the quality inspection of mold panels. These methods usually include steps such as image acquisition, preprocessing, feature extraction, feature matching, and quality assessment, aiming to automatically identify and evaluate defects in panels, such as cracks, scratches, and dents, through computer vision technology. However, existing technologies still face challenges in dealing with complex backgrounds, changes in lighting, and differences in panel surface properties, and the accuracy and robustness of feature matching and defect recognition need to be improved.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the feature extraction and matching algorithms are not adaptable enough when facing complex or changing environments, and the performance in terms of accuracy and false detection rate is not satisfactory. These problems are particularly prominent in mold panel inspection with high precision requirements. Summary of the invention

[0004] The invention provides a method and system for detecting the quality of a mold plate based on machine vision.

[0005] In a first aspect of the present invention, a method for detecting quality of a mold plate based on machine vision is provided, comprising:

[0006] S1, using a camera to obtain a template image and an image of the mold plate to be tested, and using an optical flow method to obtain a template image frame template image and a plate image to be tested respectively;

[0007] S2, preprocessing the template image and the image of the panel to be tested;

[0008] S3, extracting the edges of the preprocessed template image and the image of the panel to be tested respectively using the Canny edge detection algorithm;

[0009] S4, matching feature points of the edge of the template image and the edge of the image of the panel to be tested by using the RANSAC algorithm;

[0010] S5, using the random sampling consensus RANSAC algorithm to remove mismatched feature point pairs for the matched feature points;

[0011] S6, obtaining all straight line segments on the image of the panel to be tested, and obtaining characteristic line segments of the template image and the image of the panel to be tested based on the extracted edge points;

[0012] S7, using the SURF algorithm to match the characteristic line segments of the template image and the image of the panel to be tested, and obtaining the ratio of the number of matching line segments between the template image and the image of the panel to be tested to the number of total line segments;

[0013] S8, using the high threshold in the Canny edge detection algorithm to extract the template edge and the edge of the panel to be tested, performing ORB feature point matching on the template edge and the edge of the panel to be tested, and obtaining an ORB feature matching pair between the template area and the panel area to be tested;

[0014] S9, select any area, and perform line layout registration between the template area and the panel area to be tested according to the matching feature points on the edge of the regional template;

[0015] S10, calculating the comprehensive quality information of the panel in the registration area according to the registration area corresponding to the template image and the panel image to be tested, and calculating the quality information of the mold panel according to the comprehensive quality information of the panel in each registration area;

[0016] S11. According to the panel quality information, the quality inspection result is obtained, that is, whether the quality of the mold panel meets the requirements.

[0017] Furthermore, the step S2 is specifically as follows:

[0018] The template image and the image of the panel to be tested are subjected to denoising using the Gaussian blur method, and then the denoised template image and the image of the panel to be tested are subjected to gamma correction and affine transformation to make the image more balanced; then the affine-corrected template image and the image of the panel to be tested are subjected to inverse whitening and sharpening processing.

[0019] Furthermore, the step S4 is specifically as follows:

[0020] The RANSAC straight line fitting method is used to extract the line segments of the outer edge of the panel, and the line segment matching method is used to obtain the matching template image feature segments and the image feature segments of the panel to be tested. At the same time, the point matching method is used to obtain the matching point set A on the template image and the matching point set B on the panel to be tested.

[0021] Furthermore, the step S5 is specifically as follows:

[0022] S51, taking the line segments obtained by intersecting the line segments A1, A2, A3 ... An on the upper edge of the template image in pairs as key feature line segments;

[0023] S52, for each template image feature line segment, divide the straight line where the feature line segment and the key feature line segment are located into two parts: the same side as the key feature line segment is the inner segment, and the opposite side of the key feature line segment is the outer segment;

[0024] S53, for each template image feature line segment, calculating the distance from each edge point on the feature line segment to the key feature line segment;

[0025] S54. Compare the minimum distance from the inner edge point to the key feature line segment on the line segment with the minimum distance from the inner edge point of the outer segment to the key feature line segment. If the distance is smaller, the minimum distance value of the inner point is dmin. 内 ;

[0026] If the distance is large, the minimum distance value of the external point is d min外 ;

[0027] Calculate the difference between the minimum distance from the inner edge point of the inner segment to the key feature line segment and the minimum distance from the inner edge point of the outer segment to the key feature line segment;

[0028] If d min外 -d min内 >0, the characteristic line segment is taken as the boundary;

[0029] If d min外 -d min内 <0, the feature line segment is regarded as a non-boundary.

[0030] Furthermore, the step S6 is specifically as follows:

[0031] S61, using SURF features to match the feature line segments of the template image and the feature line segments of the panel image to be tested on the same straight line, to obtain matching feature point pairs on the two feature line segments;

[0032] S62, using RANSAC straight line fitting to obtain the matching relationship between the template image feature line segment and the image feature line segment of the panel to be tested;

[0033] S63, matching the line segments on the edge of the template image and the image of the panel to be tested, intersecting the line segments on the edge of the template image and the image of the panel to be tested in pairs, to obtain new key feature line segments;

[0034] At the same time, the line segments on the edge of the template image and the image of the panel to be tested intersect with the key feature line segments to obtain intersection points, and the two adjacent points are used as new line segments on the edge of the template image and the image of the panel to be tested;

[0035] S64, repeat S52-S54 to obtain the line segment type attribute of the feature line segment on the new image of the board to be tested, and repeat S63 to expand the line segment on the image of the board to be tested;

[0036] S65 . Use SURF features to match the feature line segments of the template image with the feature line segments of the image of the panel to be tested.

[0037] Furthermore, the step S10 is specifically as follows:

[0038] S101, the registration area corresponding to the template image and the image of the panel to be tested includes the entire area of ​​the panel to be tested or the sub-area of ​​the panel to be tested where the template image is located;

[0039] S102, calculating the comprehensive quality information of the panel in the registration area, including cracks, scratches and dents, according to the registration area corresponding to the template image and the panel image to be tested;

[0040] S103 , calculating the quality information of the mold plate according to the areas of the registration regions corresponding to the template image and the image of the plate to be tested.

[0041] Furthermore, the step S102 is specifically as follows:

[0042] For the concave, all the concave information of the registration area corresponding to the template image and the image of the plate to be tested in the area is obtained through the grayscale change of the registration area corresponding to the template image and the image of the plate to be tested, including the depth, width, length, ratio of depth to width, ratio of depth to length, and shape of the concave, and the concave information is comprehensively analyzed to determine whether the concave is qualified;

[0043] If it is unqualified, the unqualified area will be extracted and marked with a depression.

[0044] Furthermore, the step S102 is specifically as follows:

[0045] For scratches, the scratch information of the registration area corresponding to the template image and the image of the panel to be tested is obtained through the grayscale change of the registration area corresponding to the template image and the image of the panel to be tested, including the width, length, ratio of width to length, scratch depth, and shape of the scratch. The scratch information is comprehensively analyzed to determine whether the scratch is qualified;

[0046] If it fails, the failed area will be extracted and marked with scratches.

[0047] Furthermore, the step S102 is specifically as follows:

[0048] For cracks, the template image and the image of the plate to be tested are compared to determine whether there are overlapped points on the edge points of the template image on the plate to be tested. If there are overlapped points, the overlapped points are used as the base points to find other edge points of the cracks at the edge of the plate to be tested at a preset distance to form a crack area, extract the crack area, and obtain all crack areas; determine whether the crack is within the standard range, if not, extract the unqualified area and mark the crack;

[0049] If so, determine whether there are multiple crack areas within the standard range. If so, extract the unqualified area with multiple crack areas and mark the cracks.

[0050] In a second aspect of the present invention, a machine vision-based mold plate quality inspection system is provided, comprising:

[0051] An acquisition module is used to obtain a template image and an image of the mold plate to be tested by taking pictures with a camera, and to obtain a template image frame template image and an image of the plate to be tested respectively by an optical flow method;

[0052] A preprocessing module, used for preprocessing the template image and the image of the panel to be tested;

[0053] The edge extraction module is used to extract the edges of the preprocessed template image and the image of the panel to be tested respectively using the Canny edge detection algorithm;

[0054] The matching module is used to match the feature points of the edge of the template image and the edge of the panel image to be tested by the RANSAC algorithm, obtain all the straight line segments on the panel image to be tested, and obtain the feature line segments of the template image and the panel image to be tested based on the extracted edge points;

[0055] The feature line segment matching module is used to match the feature line segments of the template image and the image of the panel to be tested by using the SURF algorithm to obtain the ratio of the number of matching line segments between the template image and the image of the panel to be tested to the number of total line segments;

[0056] The edge detection module is used to extract the template edge and the edge of the panel to be tested by using the high threshold in the Canny edge detection algorithm, perform ORB feature point matching on the template edge and the edge of the panel to be tested, and obtain an ORB feature matching pair between the template area and the panel area to be tested;

[0057] The registration module is used to select any area and perform line layout registration between the template area and the panel area to be tested according to the matching feature points on the edge of the regional template;

[0058] A comprehensive information calculation module is used to calculate the comprehensive quality information of the plate in the registration area according to the registration area corresponding to the template image and the plate image to be tested, and calculate the quality information of the mold plate according to the comprehensive quality information of the plate in each registration area;

[0059] The quality inspection module is used to obtain the quality inspection result according to the panel quality information, that is, whether the quality of the mold panel meets the requirements.

[0060] According to the above-mentioned embodiments of the present invention, there are at least the following beneficial effects: the mold plate quality detection method based on machine vision provided by the present invention can effectively improve the accuracy and efficiency of detection. By adopting the optical flow method to obtain the template image and the image of the plate to be tested, combining the Canny edge detection and RANSAC algorithm for feature point matching, and using the SURF algorithm for feature line segment matching, this method can accurately identify and locate the edges and feature line segments of the mold plate, and can improve the accuracy of feature extraction and matching. In addition, through ORB feature point matching and line layout registration technology, this method can achieve precise alignment of the template and the image of the plate to be tested, which can further enhance the reliability of the detection results.

[0061] In addition, when calculating the comprehensive information of plate quality, the method of the present invention not only considers defects such as cracks, scratches and dents, but also comprehensively analyzes the various attributes of these defects, such as depth, width, length and shape, so that the quality assessment is more comprehensive and detailed. This method can reduce misjudgment caused by human factors, while improving the degree of automation of the detection process, reducing labor intensity, saving costs, and ultimately achieving efficient and accurate detection of mold plate quality, meeting the needs of high-efficiency and high-reliability detection in modern industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0063] Figure 1 A schematic diagram of a process flow of a mold plate quality inspection method based on machine vision provided by an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of the structure of a mold plate quality inspection system based on machine vision provided by an embodiment of the present invention;

[0065] Figure 3 The schematic diagram schematically shows the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0067] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0068] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0069] Reference below Figure 1 , Figure 1 The following is a flow chart of a method for detecting mold plate quality based on machine vision according to an embodiment of the present invention. Figure 1 As shown, a method 100 for detecting quality of mold plate parts based on machine vision includes:

[0070] S1, using a camera to obtain a template image and an image of the mold plate to be tested, and using an optical flow method to obtain a template image frame template image and a plate image to be tested respectively;

[0071] S2, preprocessing the template image and the image of the panel to be tested;

[0072] S3, extracting the edges of the preprocessed template image and the image of the panel to be tested respectively using the Canny edge detection algorithm;

[0073] S4, matching feature points of the edge of the template image and the edge of the image of the panel to be tested by using the RANSAC algorithm;

[0074] S5, using the random sampling consensus RANSAC algorithm to remove mismatched feature point pairs for the matched feature points;

[0075] S6, obtaining all straight line segments on the image of the panel to be tested, and obtaining characteristic line segments of the template image and the image of the panel to be tested based on the extracted edge points;

[0076] S7, using the SURF algorithm to match the characteristic line segments of the template image and the image of the panel to be tested, and obtaining the ratio of the number of matching line segments between the template image and the image of the panel to be tested to the number of total line segments;

[0077] S8, using the high threshold in the Canny edge detection algorithm to extract the template edge and the edge of the panel to be tested, performing ORB feature point matching on the template edge and the edge of the panel to be tested, and obtaining an ORB feature matching pair between the template area and the panel area to be tested;

[0078] S9, select any area, and perform line layout registration between the template area and the panel area to be tested according to the matching feature points on the edge of the regional template;

[0079] S10, calculating the comprehensive quality information of the panel in the registration area according to the registration area corresponding to the template image and the panel image to be tested, and calculating the quality information of the mold panel according to the comprehensive quality information of the panel in each registration area;

[0080] S11. According to the panel quality information, the quality inspection result is obtained, that is, whether the quality of the mold panel meets the requirements.

[0081] It should be noted that the method first involves using a camera to capture a template image and an image of the mold plate to be tested. The template image here refers to a standard image for comparison, while the image of the mold plate to be tested is an image of a plate to be tested in actual production. The template image frame template image and the image of the plate to be tested are obtained respectively by the optical flow method. The optical flow method is an algorithm that calculates the motion information of pixels in an image sequence and is used to track feature points in the image.

[0082] Specifically, the camera shooting parameters need to be set according to the actual panel size and detection requirements, such as resolution, exposure time, and focal length. When applying the optical flow method, different algorithm variants can be selected, such as the Lucas-Kanade method, to adapt to different image sequence characteristics. When acquiring image frames, it is necessary to ensure the continuity and clarity of the image so that subsequent edge detection and feature point matching can be performed accurately.

[0083] Preferably, for the application of the optical flow method, multi-scale optical flow estimation can be used to improve the stability and accuracy of feature point tracking. In practical applications, the parameters of the optical flow algorithm, such as window size, number of iterations, and termination conditions, can be adjusted according to the complexity and surface characteristics of the panel to achieve the best tracking effect.

[0084] Furthermore, in order to improve the image quality, the panel may be properly pre-processed before shooting, such as cleaning the surface, controlling the lighting, etc., to reduce the impact of image noise and reflection on the detection results.

[0085] In some embodiments, step S2 is specifically:

[0086] The template image and the image of the panel to be tested are subjected to denoising using the Gaussian blur method, and then the denoised template image and the image of the panel to be tested are subjected to gamma correction and affine transformation to make the image more balanced; then the affine-corrected template image and the image of the panel to be tested are subjected to inverse whitening and sharpening processing.

[0087] It should be noted that this step involves preprocessing the template image and the image of the panel to be tested. The preprocessing here refers to a series of processing steps performed on the image before edge detection and feature point matching to improve the accuracy and efficiency of subsequent processing. Specifically, it includes Gaussian blur method for noise reduction, gamma correction for affine transformation, inverting and sharpening. Gaussian blur is an image smoothing technology used to reduce image noise; gamma correction is a technology for adjusting image brightness and contrast; affine transformation is a geometric transformation used to correct the shape of an image; inverting refers to inverting the color of the image; sharpening is to enhance the edges and details of the image.

[0088] Specifically, Gaussian blur can adjust the degree of blur by setting different kernel sizes and standard deviations to adapt to different image noise levels. The parameter settings of gamma correction need to be adjusted according to the specific characteristics of the image to achieve the best balance between brightness and contrast. Affine transformation can be achieved by calculating the changes in the coordinates of feature points in the image, and usually requires determining the transformation matrix and applying it to the entire image. Inverting can be achieved by inverting the value of each pixel in the image, while sharpening can be achieved by enhancing the high-frequency components of the image, such as using a Laplacian operator or a high-pass filter.

[0089] Preferably, the pre-processing step may also include histogram equalization to further enhance the contrast of the image, particularly when the illumination conditions of the image are uneven. In addition, for Gaussian blur, different kernel functions may be selected, such as Gaussian kernel, bilateral kernel, etc., to adapt to different image characteristics and noise types. For sharpening, different sharpening algorithms may be used, such as Unsharp Masking or Wiener filtering, to obtain better sharpening effects. The specific parameter settings and selections of these pre-processing steps should be determined based on the actual image quality and detection requirements to achieve the best pre-processing effect.

[0090] In some embodiments, step S4 is specifically:

[0091] The RANSAC straight line fitting method is used to extract the line segments of the outer edge of the panel, and the line segment matching method is used to obtain the matching template image feature segments and the image feature segments of the panel to be tested. At the same time, the point matching method is used to obtain the matching point set A on the template image and the matching point set B on the panel to be tested.

[0092] It should be noted that this step describes in detail the process of matching feature points using the RANSAC algorithm. Here, RANSAC stands for random sampling consensus algorithm, which is an iterative method for estimating mathematical model parameters from a set of data containing outliers. The algorithm fits the model by repeatedly randomly selecting subsets of data and evaluating the consistency of the model to identify and exclude outliers. Feature points refer to points in an image with unique properties, such as corner points, edge points, etc., which play a key role in image matching.

[0093] Specifically, the RANSAC algorithm is used in this step to extract the line segments of the outer edge of the panel and perform line segment matching to obtain the matching template image feature segments and the image feature segments of the panel to be tested. At the same time, the point matching method is used to obtain the matching point set A on the template image and the matching point set B on the panel to be tested.

[0094] More specifically, during the implementation process, it is necessary to set parameters such as the number of iterations, consistency threshold, and outlier ratio of the RANSAC algorithm. The number of iterations determines the robustness of the algorithm, the consistency threshold is used to determine the degree of match between the data point and the model, and the outlier ratio affects the algorithm's tolerance to outliers.

[0095] Preferably, in addition to the RANSAC algorithm, other robust algorithms can also be considered, such as LMedS (minimizing the robust estimation of local errors) or MSAC (random sampling consistency of M-estimator). These algorithms have their own advantages in dealing with outliers and noise, and can be selected according to the actual image quality and matching requirements.

[0096] Furthermore, for feature point extraction, you can consider using algorithms such as SIFT (Scale Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF), which can extract image feature points at different scales and rotations, and improve the accuracy and robustness of feature point matching. In practical applications, you can adjust the parameters of these algorithms according to the specific characteristics of the panel and environmental conditions to achieve the best matching effect.

[0097] In some embodiments, step S5 is specifically:

[0098] S51, taking the line segments obtained by intersecting the line segments A1, A2, A3 ... An on the upper edge of the template image in pairs as key feature line segments;

[0099] S52, for each template image feature line segment, divide the straight line where the feature line segment and the key feature line segment are located into two parts: the same side as the key feature line segment is the inner segment, and the opposite side of the key feature line segment is the outer segment;

[0100] S53, for each template image feature line segment, calculating the distance d from each edge point on the feature line segment to the key feature line segment;

[0101] S54. Compare the minimum distance from the inner edge point to the key feature line segment on the line segment with the minimum distance from the inner edge point of the outer segment to the key feature line segment. If the distance is smaller, the minimum distance value of the inner point is dmin. 内 ;

[0102] If the distance is large, the minimum distance value of the external point is d min外 ;

[0103] Calculate the difference between the minimum distance from the inner edge point of the inner segment to the key feature line segment and the minimum distance from the inner edge point of the outer segment to the key feature line segment;

[0104] If d min外 -d min内 >0, the characteristic line segment is taken as the boundary;

[0105] If d min外 -d min内 <0, the feature line segment is regarded as a non-boundary.

[0106] It should be noted that this step involves analyzing the relative position of the feature line segment of the template image and the key feature line segment to determine whether the line segment is a boundary line segment. Here, the key feature line segment refers to the line segment obtained by intersecting two by two in the template image, and they play a core role in identifying boundary features. The inner segment and the outer segment refer to the relative position of the feature line segment and the straight line where the key feature line segment is located, where the part on the same side as the key feature line segment is the inner segment, and the part on the opposite side is the outer segment.

[0107] Specifically, for each template image feature line segment, it is necessary to calculate the distance d from each edge point on the feature line segment to the key feature line segment. This distance can be obtained by geometric calculation, such as using the Euclidean distance formula. Next, it is necessary to compare the minimum distance from the inner edge point of the line segment to the key feature line segment with the minimum distance from the inner edge point of the outer segment to the key feature line segment. This comparison can be achieved by traversing all edge points on the line segment and calculating their perpendicular distances to the key feature line segment. Specific parameter settings include the accuracy of distance calculation, the threshold of the minimum distance, etc. These parameters can be adjusted according to the actual image resolution and the distribution of feature points.

[0108] Preferably, in order to improve the accuracy of boundary line segment recognition, additional geometric features, such as angle information or curvature information, can be introduced to assist in determining whether a line segment is a boundary. For example, the angle between a feature line segment and a key feature line segment can be calculated, and if the angle exceeds a preset threshold, the feature line segment is considered to be a boundary line segment.

[0109] Furthermore, machine learning methods, such as support vector machines (SVM) or neural networks, can be considered to classify whether a line segment is a boundary. These methods can improve the accuracy of recognition by learning a large amount of training data. In practical applications, the parameters and training strategies of these methods can be adjusted according to the specific shape and characteristics of the plate to achieve the best recognition effect.

[0110] In some embodiments, step S6 is specifically:

[0111] S61, using SURF features to match the feature line segments of the template image and the feature line segments of the panel image to be tested on the same straight line, to obtain matching feature point pairs on the two feature line segments;

[0112] S62, using RANSAC straight line fitting to obtain the matching relationship between the template image feature line segment and the image feature line segment of the panel to be tested;

[0113] S63, matching the line segments on the edge of the template image and the image of the panel to be tested, intersecting the line segments on the edge of the template image and the image of the panel to be tested in pairs, to obtain new key feature line segments;

[0114] At the same time, the line segments on the edge of the template image and the image of the panel to be tested intersect with the key feature line segments to obtain intersection points, and the two adjacent points are used as new line segments on the edge of the template image and the image of the panel to be tested;

[0115] S64, repeat S52-S54 to obtain the line segment type attribute of the feature line segment on the new image of the board to be tested, and repeat S63 to expand the line segment on the image of the board to be tested;

[0116] S65 . Use SURF features to match the feature line segments of the template image with the feature line segments of the image of the panel to be tested.

[0117] It should be noted that this step describes in detail how to use SURF features to match the feature line segments of the template image and the image of the panel to be tested, and process the matching results to determine the matching relationship of the line segments. Here, SURF features refer to Speeded Up Robust Features, which is an algorithm used in computer vision to detect and match feature points between different images. It can extract key points and descriptors in the image for subsequent feature matching.

[0118] Specifically, the SURF feature matching process includes the following steps: first, extract SURF feature point pairs from the feature line segments of the template image and the image of the panel to be tested; then, use the RANSAC line fitting method to determine the matching relationship between these feature point pairs; then, intersect the line segments on the edge of the template image and the image of the panel to be tested with each other through line segment matching to obtain new key feature line segments; in addition, it is also necessary to intersect the line segments on the edge of the template image and the image of the panel to be tested with the key feature line segments to obtain the intersection points, which are used as new line segments. In this process, it is necessary to set the parameters of the SURF algorithm, such as the Hessian threshold, the number of scales, the edge response threshold, etc. These parameters affect the detection of feature points and the quality of the descriptor.

[0119] Preferably, in order to improve the accuracy and robustness of feature matching, other feature matching algorithms can be considered as a supplement or alternative to SURF, such as SIFT (Scale Invariant Feature Transform) or ORB (Oriented FAST and Rotated BRIEF). These algorithms may perform better in different application scenarios, especially when facing image scale changes, rotation or illumination changes.

[0120] Furthermore, the parameters of the SURF algorithm can be adjusted, such as increasing the Hessian threshold to reduce the number of feature points and improve the matching accuracy; or adjusting the scale number to accommodate feature points of different sizes. In practical applications, these parameters can be adjusted according to the specific characteristics of the panel and the image quality to achieve the best matching effect.

[0121] In some embodiments, step S10 is specifically as follows:

[0122] S101, the registration area corresponding to the template image and the image of the panel to be tested includes the entire area of ​​the panel to be tested or the sub-area of ​​the panel to be tested where the template image is located;

[0123] S102, calculating the comprehensive quality information of the panel in the registration area, including cracks, scratches and dents, according to the registration area corresponding to the template image and the panel image to be tested;

[0124] S103 , calculating the quality information of the mold plate according to the areas of the registration regions corresponding to the template image and the image of the plate to be tested.

[0125] It should be noted that this step involves the process of calculating the comprehensive information of the panel quality based on the registration area corresponding to the template image and the image of the panel to be tested. Here, the registration area refers to the area where the template image and the image of the panel to be tested correspond after feature matching and geometric transformation, and these areas are used for further quality assessment. The comprehensive information of the panel quality includes the detection and analysis of possible defects such as cracks, scratches and dents on the panel.

[0126] Specifically, the registration area can be the entire area of ​​the panel to be tested or the sub-area where the template image is located. When calculating the comprehensive information of the panel quality, these areas need to be analyzed in detail, including but not limited to the detection of cracks, scratches and dents. This involves the analysis of the grayscale changes, edge features and shape features of the image. Specific parameter settings may include grayscale thresholds, edge detection sensitivity, shape recognition parameters, etc. These parameters need to be adjusted according to the characteristics of the panel material and the possible defect types to ensure the accuracy of the detection.

[0127] Preferably, in order to calculate the panel quality information more accurately, machine learning or deep learning technology can be introduced to assist in identifying and classifying defects. For example, a convolutional neural network (CNN) can be used to identify cracks and scratches in an image, or a support vector machine (SVM) can be used to classify different types of dents.

[0128] Furthermore, multi-scale analysis methods can be used to improve the detection capability of tiny defects, or image enhancement technology can be used to improve image quality and make defect features more obvious. In practical applications, these technologies and parameters can be adjusted according to the specific application scenarios and quality requirements of the panels to achieve the best detection effect.

[0129] In some embodiments, step S102 is specifically as follows:

[0130] For the concave, all the concave information of the registration area corresponding to the template image and the image of the plate to be tested in the area is obtained through the grayscale change of the registration area corresponding to the template image and the image of the plate to be tested, including the depth, width, length, ratio of depth to width, ratio of depth to length, and shape of the concave, and the concave information is comprehensively analyzed to determine whether the concave is qualified;

[0131] If it is unqualified, the unqualified area will be extracted and marked with a depression.

[0132] It should be noted that this step focuses on the detection and evaluation of dents, and obtains dent information by analyzing the grayscale changes in the registration area corresponding to the template image and the image of the panel to be tested. Here, dents refer to local depressions or dents on the surface of the panel, which may affect the appearance and structural integrity of the product. Grayscale changes refer to the brightness differences between different areas in the image, which can be used to identify and locate the dented area.

[0133] Specifically, the process of detecting the dents includes obtaining all the dent information of the registration area corresponding to the template image and the image of the panel to be tested, such as the depth, width, length, depth-to-width ratio, depth-to-length ratio and dent shape of the dent. This information can be obtained by comparing the grayscale distribution of the registration area.

[0134] More specifically, the parameter settings include grayscale threshold, sensitivity of dent recognition, parameters of shape recognition, etc. These parameters need to be adjusted according to the characteristics of the plate material and possible dent types to ensure the accuracy of detection.

[0135] Preferably, in order to analyze the concave information more accurately, a three-dimensional scanning technology can be introduced to obtain more detailed concave geometric information. In addition, image processing technology, such as edge enhancement or local contrast enhancement, can be used to improve the visibility of concave features.

[0136] Furthermore, in practical applications, these techniques and parameters can be adjusted according to the specific application scenarios and quality requirements of the panels to achieve the best detection effect. For example, a statistical method can be set to determine the threshold of the dent, or a machine learning algorithm can be used to automatically identify and classify different types of dents.

[0137] In some embodiments, step S102 is specifically as follows:

[0138] For scratches, the scratch information of the registration area corresponding to the template image and the image of the panel to be tested is obtained through the grayscale change of the registration area corresponding to the template image and the image of the panel to be tested, including the width, length, ratio of width to length, scratch depth, and shape of the scratch. The scratch information is comprehensively analyzed to determine whether the scratch is qualified;

[0139] If it fails, the failed area will be extracted and marked with scratches.

[0140] It should be noted that this step focuses on the detection and evaluation of scratches, and obtains scratch information by analyzing the grayscale changes in the registration areas corresponding to the template image and the image of the panel to be tested. Here, scratches refer to long and thin scratches or abrasions on the surface of the panel, which may affect the appearance and durability of the product. Grayscale changes refer to the brightness differences between different areas in the image, which can be used to identify and locate the scratched area.

[0141] Specifically, the process of detecting scratches includes obtaining scratch information of the registration area corresponding to the template image and the image of the panel to be tested, such as the width, length, ratio of width to length, scratch depth and scratch shape of the scratch. This information can be obtained by comparing the grayscale distribution and edge features of the registration area.

[0142] More specifically, parameter settings include grayscale threshold, scratch recognition sensitivity, shape recognition parameters, etc. These parameters need to be adjusted according to the characteristics of the plate material and possible scratch types to ensure the accuracy of detection.

[0143] Preferably, in order to analyze the scratch information more accurately, high-resolution imaging technology can be introduced to obtain more detailed scratch geometry information. In addition, image processing technology, such as Gaussian filtering or Laplacian edge detection, can be used to improve the visibility of scratch features.

[0144] Furthermore, in practical applications, these techniques and parameters can be adjusted according to the specific application scenarios and quality requirements of the panels to achieve the best detection effect. For example, a machine learning-based classifier can be set up to automatically identify and classify different types of scratches, or a multi-scale analysis method can be used to improve the detection capability of tiny scratches.

[0145] In some embodiments, step S102 is specifically as follows:

[0146] For cracks, the template image and the image of the plate to be tested are compared to determine whether there are overlapped points on the edge points of the template image on the plate to be tested. If there are overlapped points, the overlapped points are used as the base points to find other edge points of the cracks at the edge of the plate to be tested at a preset distance to form a crack area, extract the crack area, and obtain all crack areas; determine whether the crack is within the standard range, if not, extract the unqualified area and mark the crack;

[0147] If so, determine whether there are multiple crack areas within the standard range. If so, extract the unqualified area with multiple crack areas and mark the cracks.

[0148] It should be noted that this step describes the crack detection process in detail, identifying the crack area by comparing the template image and the image of the panel to be tested. Here, cracks refer to breaks or cracks on the surface of the panel, which may seriously affect the structural integrity and safety of the product. The coincidence point refers to the corresponding point found on the image of the panel to be tested by the edge point of the template image, which is used to determine the starting position of the crack.

[0149] Specifically, the process of crack detection includes finding edge points on the template image and finding the coincidence points of these edge points on the image of the plate to be tested. If there are coincidence points, these points are used as base points to find other edge points of the crack on the edge of the image of the plate to be tested to form the crack area. Specific parameter settings may include the threshold of edge detection, the range of searching for coincidence points, the sensitivity of crack identification, etc. These parameters need to be adjusted according to the characteristics of the plate material and the possible crack morphology to ensure the accuracy of detection.

[0150] Preferably, in order to more accurately identify crack areas, more advanced image processing techniques, such as morphological operations or deep learning methods, can be used to improve the accuracy and robustness of crack detection. In addition, a classifier based on crack features can be set, such as using a convolutional neural network (CNN) to identify crack patterns in images.

[0151] Furthermore, in practical applications, these technologies and parameters can be adjusted according to the specific application scenarios and quality requirements of the panels to achieve the best detection effect. For example, adaptive threshold technology can be used to dynamically adjust the threshold of edge detection, or multi-scale analysis methods can be used to improve the detection capability of micro cracks.

[0152] The above-mentioned embodiments of the present invention have the following beneficial effects: The mold plate quality detection method based on machine vision described in the present invention can improve the automation and accuracy of the detection process. Through the steps of acquiring image frames, preprocessing, Canny edge detection, RANSAC algorithm matching feature points, SURF algorithm matching feature line segments, etc. by optical flow method, this method can efficiently extract key information from the image and accurately match the features between the template and the plate to be tested, which can reduce the interference of human factors and improve the consistency and reliability of detection. In addition, this method can enhance the accuracy of feature matching and improve the accuracy of quality detection results by removing mismatched feature point pairs.

[0153] Furthermore, this method can also achieve a comprehensive assessment of the quality of mold panels by comprehensively analyzing the panel quality information in the registration area, including the characteristics of defects such as cracks, scratches and dents. This method can not only accurately calculate the size and shape of the defects, but also determine whether the defects are qualified according to the preset standards, thereby providing strong data support for quality control. Through this method, unqualified panels can be effectively identified and marked, ensuring that only panels that meet quality standards can enter the next production link, which can improve product quality and production efficiency.

[0154] like Figure 2 As shown, in some embodiments, a mold plate quality inspection system 200 based on machine vision includes:

[0155] The acquisition module 201 is used to obtain the template image and the image of the mold plate to be tested by using a camera, and obtain the template image frame template image and the image of the plate to be tested respectively by using an optical flow method;

[0156] A preprocessing module 202 is used to preprocess the template image and the image of the panel to be tested;

[0157] The edge extraction module 203 is used to extract the edges of the pre-processed template image and the image of the panel to be tested respectively by using the Canny edge detection algorithm;

[0158] The matching module 204 is used to match feature points of the edge of the template image and the edge of the panel image to be tested by using the RANSAC algorithm, obtain all straight line segments on the panel image to be tested, and obtain feature line segments of the template image and the panel image to be tested based on the extracted edge points;

[0159] The feature line segment matching module 205 is used to match the feature line segments of the template image and the image of the panel to be tested by using the SURF algorithm to obtain the ratio of the number of line segments matched between the template image and the image of the panel to be tested to the number of total line segments;

[0160] The edge detection module 206 is used to extract the template edge and the edge of the panel to be tested by using the high threshold value in the Canny edge detection algorithm, and perform ORB feature point matching on the template edge and the edge of the panel to be tested to obtain an ORB feature matching pair between the template area and the panel area to be tested;

[0161] The registration module 207 is used to select any area and perform line layout registration between the template area and the panel area to be tested according to the matching feature points on the edge of the regional template;

[0162] A comprehensive information calculation module is used to calculate the comprehensive quality information of the panel in the registration area according to the registration area corresponding to the template image and the panel image to be tested, and calculate the quality information of the mold panel according to the comprehensive quality information 208 of the panel in each registration area;

[0163] The quality inspection module 209 is used to obtain the quality inspection result according to the panel quality information, that is, whether the quality of the mold panel meets the requirements.

[0164] It is understandable that the modules described in the machine vision-based mold plate quality inspection system 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the mold plate quality inspection method based on machine vision are also applicable to the mold plate quality inspection system 200 based on machine vision and the modules contained therein, and will not be repeated here.

[0165] Reference below Figure 3 , which shows a schematic diagram of a structure 300 of an electronic device suitable for implementing some embodiments of the present invention. The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0166] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0167] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0168] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0169] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention to form a technical solution.

Claims

1. A method for detecting mold plate quality based on machine vision, characterized in that: The following steps are involved: S1, using a camera to obtain a template image and an image of the mold plate to be tested, and using an optical flow method to obtain a template image frame template image and a plate image to be tested respectively; S2, preprocessing the template image and the image of the panel to be tested; S3, extracting the edges of the preprocessed template image and the image of the panel to be tested respectively using the Canny edge detection algorithm; S4, matching feature points of the edge of the template image and the edge of the image of the panel to be tested by using the RANSAC algorithm; S5, using the random sampling consensus RANSAC algorithm to remove mismatched feature point pairs for the matched feature points; S6, obtaining all straight line segments on the image of the panel to be tested, and obtaining characteristic line segments of the template image and the image of the panel to be tested based on the extracted edge points; S7, using the SURF algorithm to match the characteristic line segments of the template image and the image of the panel to be tested, and obtaining the ratio of the number of matching line segments between the template image and the image of the panel to be tested to the number of total line segments; S8, using the high threshold in the Canny edge detection algorithm to extract the template edge and the edge of the panel to be tested, performing ORB feature point matching on the template edge and the edge of the panel to be tested, and obtaining an ORB feature matching pair between the template area and the panel area to be tested; S9, select any area, and perform line layout registration between the template area and the panel area to be tested according to the matching feature points on the edge of the regional template; S10, calculating the comprehensive quality information of the panel in the registration area according to the registration area corresponding to the template image and the panel image to be tested, and calculating the quality information of the mold panel according to the comprehensive quality information of the panel in each registration area; S11. According to the panel quality information, the quality inspection result is obtained, that is, whether the quality of the mold panel meets the requirements.

2. The method for detecting mold plate quality based on machine vision according to claim 1, characterized in that: The step S2 is specifically as follows: The template image and the image of the panel to be tested are subjected to denoising using the Gaussian blur method, and then the denoised template image and the image of the panel to be tested are subjected to gamma correction and affine transformation to make the image more balanced; then the affine-corrected template image and the image of the panel to be tested are subjected to inverse whitening and sharpening processing.

3. The method for detecting mold plate quality based on machine vision according to claim 1, characterized in that: The step S4 is specifically as follows: The RANSAC straight line fitting method is used to extract the line segments of the outer edge of the panel, and the line segment matching method is used to obtain the matching template image feature segments and the image feature segments of the panel to be tested. At the same time, the point matching method is used to obtain the matching point set A on the template image and the matching point set B on the panel to be tested.

4. The method for detecting mold plate quality based on machine vision according to claim 3 is characterized in that: The step S5 is specifically as follows: S51, taking the line segments obtained by intersecting the line segments A1, A2, A3 ... An on the upper edge of the template image in pairs as key feature line segments; S52, for each template image feature line segment, divide the straight line where the feature line segment and the key feature line segment are located into two parts: the same side as the key feature line segment is the inner segment, and the opposite side of the key feature line segment is the outer segment; S53, for each template image feature line segment, calculating the distance from each edge point on the feature line segment to the key feature line segment; S54, compare the minimum distance from the inner edge point of the line segment to the key feature line segment with the minimum distance from the inner edge point of the outer segment to the key feature line segment. If the distance is smaller, the minimum distance value of the inner point is d min内 ; If the distance is large, the minimum distance value of the external point is d min外 ; Calculate the difference between the minimum distance from the inner edge point of the inner segment to the key feature line segment and the minimum distance from the inner edge point of the outer segment to the key feature line segment; If d min外 -d min内 >0, the characteristic line segment is taken as the boundary; If d min外 -d min内 <0, the feature line segment is regarded as a non-boundary.

5. The method for detecting mold plate quality based on machine vision according to claim 4, characterized in that: The step S6 is specifically as follows: S61, using SURF features to match the feature line segments of the template image and the feature line segments of the panel image to be tested on the same straight line, to obtain matching feature point pairs on the two feature line segments; S62, using RANSAC straight line fitting to obtain the matching relationship between the template image feature line segment and the image feature line segment of the panel to be tested; S63, matching the line segments on the edge of the template image and the image of the panel to be tested, intersecting the line segments on the edge of the template image and the image of the panel to be tested in pairs, to obtain new key feature line segments; At the same time, the line segments on the edge of the template image and the image of the panel to be tested intersect with the key feature line segments to obtain intersection points, and the two adjacent points are used as new line segments on the edge of the template image and the image of the panel to be tested; S64, repeat S52-S54 to obtain the line segment type attribute of the feature line segment on the new image of the board to be tested, and repeat S63 to expand the line segment on the image of the board to be tested; S65 . Use SURF features to match the feature line segments of the template image with the feature line segments of the image of the panel to be tested.

6. The method for detecting mold plate quality based on machine vision according to claim 1, characterized in that: The step S10 is specifically as follows: S101, the registration area corresponding to the template image and the image of the panel to be tested includes the entire area of ​​the panel to be tested or the sub-area of ​​the panel to be tested where the template image is located; S102, calculating the comprehensive quality information of the panel in the registration area, including cracks, scratches and dents, according to the registration area corresponding to the template image and the panel image to be tested; S103 , calculating the quality information of the mold plate according to the areas of the registration regions corresponding to the template image and the image of the plate to be tested.

7. The method for detecting mold plate quality based on machine vision according to claim 6, characterized in that: The step S102 is specifically as follows: For the concave, all the concave information of the registration area corresponding to the template image and the image of the plate to be tested in the area is obtained through the grayscale change of the registration area corresponding to the template image and the image of the plate to be tested, including the depth, width, length, ratio of depth to width, ratio of depth to length, and shape of the concave, and the concave information is comprehensively analyzed to determine whether the concave is qualified; If it is unqualified, the unqualified area will be extracted and marked with a depression.

8. The method for detecting mold plate quality based on machine vision according to claim 6, characterized in that: The step S102 is specifically as follows: For scratches, the scratch information of the registration area corresponding to the template image and the image of the panel to be tested is obtained through the grayscale change of the registration area corresponding to the template image and the image of the panel to be tested, including the width, length, ratio of width to length, scratch depth, and shape of the scratch. The scratch information is comprehensively analyzed to determine whether the scratch is qualified; If it fails, the failed area will be extracted and marked with scratches.

9. The method for detecting mold plate quality based on machine vision according to claim 6, characterized in that: The step S102 is specifically as follows: For cracks, the template image and the image of the plate to be tested are compared to determine whether there are overlapped points on the edge points of the template image on the plate to be tested. If there are overlapped points, the overlapped points are used as the base points to find other edge points of the cracks at the edge of the plate to be tested at a preset distance to form a crack area, extract the crack area, and obtain all crack areas; determine whether the crack is within the standard range, if not, extract the unqualified area and mark the crack; If so, determine whether there are multiple crack areas within the standard range. If so, extract the unqualified area with multiple crack areas and mark the cracks.

10. A mold plate quality inspection system based on machine vision, characterized in that: include: An acquisition module is used to obtain a template image and an image of the mold plate to be tested by taking pictures with a camera, and to obtain a template image frame template image and an image of the plate to be tested respectively by an optical flow method; A preprocessing module, used for preprocessing the template image and the image of the panel to be tested; The edge extraction module is used to extract the edges of the preprocessed template image and the image of the panel to be tested respectively using the Canny edge detection algorithm; The matching module is used to match the feature points of the edge of the template image and the edge of the panel image to be tested by the RANSAC algorithm, obtain all the straight line segments on the panel image to be tested, and obtain the feature line segments of the template image and the panel image to be tested based on the extracted edge points; The feature line segment matching module is used to match the feature line segments of the template image and the image of the panel to be tested by using the SURF algorithm to obtain the ratio of the number of matching line segments between the template image and the image of the panel to be tested to the number of total line segments; The edge detection module is used to extract the template edge and the edge of the panel to be tested by using the high threshold in the Canny edge detection algorithm, perform ORB feature point matching on the template edge and the edge of the panel to be tested, and obtain an ORB feature matching pair between the template area and the panel area to be tested; The registration module is used to select any area and perform line layout registration between the template area and the panel area to be tested according to the matching feature points on the edge of the regional template; A comprehensive information calculation module is used to calculate the comprehensive quality information of the plate in the registration area according to the registration area corresponding to the template image and the plate image to be tested, and calculate the quality information of the mold plate according to the comprehensive quality information of the plate in each registration area; The quality inspection module is used to obtain the quality inspection result according to the panel quality information, that is, whether the quality of the mold panel meets the requirements.

Images