Galvanized sheet surface flaw classification method and system based on image recognition

Abstract

The invention discloses a galvanized sheet surface flaw classification method and system based on image recognition, and relates to the technical field of flaw classification. The galvanized sheet surface flaw classification method based on image recognition comprises the following steps: acquiring galvanized sheet surface image data and preprocessing the data to obtain preprocessed surface image data; performing feature processing on the preprocessed surface image data to obtain multi-dimensional surface feature data; performing feature fusion and dimension reduction on the multi-dimensional surface feature data to obtain a screened surface feature set; based on the screened surface feature set, related parameters of the production process and the quality grade of the galvanized sheet are determined; classifying the surface flaws of the galvanized sheet based on the screened surface feature set and related parameters of the production process to obtain a classification result; and the classification result is optimized and corrected based on the quality grade of the galvanized sheet to obtain a corrected classification result, so that the problems that the classification of the surface defects of the galvanized sheet is inaccurate and the production process and the product quality are difficult to associate are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of defect classification, and in particular to a method and system for classifying surface defects of galvanized sheets based on image recognition. Background Art

[0002] Traditional manual visual inspection is not only inefficient and inaccurate, but also tedious. Furthermore, high industrial production volumes and long working hours can lead to visual fatigue among workers, making it prone to false detections and missed inspections. Furthermore, manual inspection criteria are not quantified, and product quality cannot be guaranteed. Therefore, machine vision-based defect detection systems have emerged. Within the field of machine vision, a wide range of algorithms are available for defect detection, which can be applied to inspect surface defects on specific items (such as fabrics, blankets, galvanized steel sheets, and floor tiles).

[0003] The Chinese invention patent with announcement number CN108647706B discloses a method for object identification, classification and defect detection based on machine vision, including an object type identification and classification process based on a support vector machine; the object type identification and classification process includes a model training process and a model loading and classification process; a defect detection process based on a machine vision defect detection algorithm; the machine vision defect detection algorithm includes a graphic correction process and calculating the Hu invariant moment of the graphic for similarity comparison; and judging whether the image has defects based on the detection results.

[0004] However, the existing machine vision-based defect classification is difficult to associate with the production process and product quality when applied to the classification of galvanized sheet surface defects, resulting in inaccuracy. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a galvanized sheet surface defect classification method and system based on image recognition, which solves the problems of inaccurate classification of galvanized sheet surface defects and difficulty in associating the production process with product quality.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a galvanized sheet surface defect classification method based on image recognition, comprising the following steps: acquiring galvanized sheet surface image data and preprocessing it to obtain preprocessed surface image data; performing feature processing on the preprocessed surface image data to obtain multidimensional surface feature data; performing feature fusion and dimensionality reduction on the multidimensional surface feature data to obtain a screened surface feature set; determining production process-related parameters and the quality grade of the galvanized sheet based on the screened surface feature set; classifying the galvanized sheet surface defects based on the screened surface feature set and the production process-related parameters to obtain a classification result; optimizing and correcting the classification result based on the quality grade of the galvanized sheet to obtain a corrected classification result.

[0007] Furthermore, the multidimensional surface feature data includes geometric features, color features, frequency domain features and gradient features, and the acquisition process is as follows: based on the OpenCV library function, the regional area, perimeter, circularity and aspect ratio of the galvanized sheet surface defect area are determined, and the regional area, perimeter, circularity and aspect ratio are recorded as geometric features; the color histogram and color moment of the galvanized sheet surface defect area are extracted, and the pixel distribution data of each color channel is statistically analyzed based on the color histogram, and the R channel mean, G channel mean, B channel mean, R channel mean and color moment of the RGB color space are obtained based on the color moment. The pixel distribution data, R channel mean, G channel mean, B channel mean, R channel variance, G channel variance and B channel variance are recorded as color features. Based on Fourier transform, the grayscale processed image of the galvanized sheet surface defect area is converted from the spatial domain to the frequency domain, and the spectrum, power spectrum, phase spectrum and frequency moment are calculated and obtained. The spectrum, power spectrum, phase spectrum and frequency moment are recorded as frequency domain features. The gradient amplitude and direction of the image are calculated based on the Sobel operator, the edge gradient information is extracted, and the edge gradient information is recorded as gradient features.

[0008] Furthermore, feature fusion and dimensionality reduction are performed on the multidimensional surface feature data to obtain a screened surface feature set, including the following steps: feature fusion is performed on the multidimensional surface feature data based on a set arrangement order to obtain fused multidimensional surface feature data; kernel principal component analysis is performed on the fused multidimensional surface feature data to obtain eigenvalues of each defect feature in the fused multidimensional surface feature data, the eigenvalues are arranged in order from large to small, the defect features corresponding to the first K eigenvalues are retained, and they are combined into a defect feature set after the initial dimensionality reduction; secondary dimensionality reduction is performed on the defect feature set after the initial dimensionality reduction based on a gradient boosting decision tree and recursive feature elimination to obtain a screened surface feature set.

[0009] Furthermore, a secondary dimensionality reduction is performed on the defect feature set after the initial dimensionality reduction based on a gradient boosting decision tree and recursive feature elimination to obtain a screened surface feature set, including the following steps: the defect features after the initial dimensionality reduction and their corresponding galvanized sheet defect category labels are input as training data into the GBDT model, the GBDT model constructs a gradient boosting decision tree and fits the residuals, and after the gradient boosting decision tree is constructed, the contribution of each defect feature after the initial dimensionality reduction in the defect feature set after the initial dimensionality reduction to the defect classification result is calculated;

[0010] Obtain the number of surface feature targets after screening; use the defect feature set after the initial dimensionality reduction and its corresponding defect labels as input to the defect pre-classification model constructed based on the support vector machine, and obtain the classification result to be verified, including the classification accuracy to be verified and the F1 value to be verified; determine the defect feature after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction and eliminate it, and obtain the defect feature set after elimination to be verified; use the defect feature set after elimination to be verified and its corresponding defect labels as input to the defect pre-classification model constructed based on the support vector machine, and obtain the classification result to be determined, including the classification accuracy to be determined and the F1 value to be determined;

[0011] Calculate the degree of decrease in the determination indicators of the classification result to be verified and the classification result to be determined, including the degree of decrease in classification accuracy and the degree of decrease in F1 value: if the degree of decrease in accuracy and the degree of decrease in F1 value are both within the set acceptable range, the defect feature after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction can be deleted; if the degree of decrease in accuracy or the degree of decrease in F1 value is not within the set acceptable range, the defect feature after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction cannot be deleted; repeatedly delete the defect feature after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction and verify until a set of screened surface features with the same number as the target number of screened surface features is obtained.

[0012] Furthermore, the production process related parameters are determined based on the screened surface feature set, including the following steps: obtaining the initial data of the galvanized sheet production process; for each screened surface feature in the screened surface feature set, calculating the correlation coefficient between it and each production process parameter in the initial data of the galvanized sheet production process, and determining the correlation coefficient between the screened surface feature and each production process parameter; repeatedly obtaining the correlation coefficients between other screened surface features and each production process parameter; if all the correlation coefficients corresponding to a certain production process parameter are greater than the set correlation threshold, then retain the production process parameter, otherwise delete the production process parameter to obtain the production process related parameters.

[0013] Furthermore, the quality grade of the galvanized sheet is determined based on the screened surface feature set, including the following steps: obtaining the template surface feature set corresponding to each galvanized sheet quality grade stored in the database; performing similarity analysis on the screened surface feature set and the template surface feature set to obtain a similarity value xsP:

[0014] xsP=σ(JH,Jh)+σ(YS,Ys)+σ(PY,Py)+σ(TD,Td);

[0015] Where JH is the geometric feature, Jh is the template geometric feature in the template surface feature set, YS is the color feature, Ys is the template color feature in the template surface feature set, PY is the frequency domain feature, Py is the template frequency domain feature in the template surface feature set, TD is the gradient feature, Td is the template gradient feature in the template surface feature set, and σ(·) is the cosine similarity function;

[0016] Determine the template surface feature set corresponding to the maximum similarity value and determine the quality grade of the galvanized sheet.

[0017] Furthermore, the surface defects of the galvanized sheet are classified based on the screened surface feature set and production process related parameters to obtain a classification result, which includes the following steps: obtaining a production parameter-weight correction coefficient mapping database stored in a database, and obtaining a weight correction coefficient according to the current production parameters; obtaining initial weighted data of the surface defect features, and correcting the initial weighted data of the surface defect features based on the weight correction coefficient; performing weighted processing on the screened surface feature set based on the corrected initial weighted data of the surface defect features to obtain a weighted surface feature set; inputting the weighted surface feature set into a galvanized sheet surface defect classification model constructed based on a support vector machine, outputting a defect classification probability distribution, determining the maximum probability in the defect classification probability distribution, and recording the defect classification label corresponding to the maximum probability as the classification result.

[0018] Furthermore, the process of constructing the production parameter-weight correction coefficient mapping database is as follows: different production parameter combinations are determined, and several groups of weight correction coefficients to be verified are assigned to each production parameter combination; for one of the production parameter combinations, the screened surface feature set is weighted based on the corresponding several groups of weight correction coefficients to be verified and the initial weighted data of the surface defect features, and the weighted data are input into the defect pre-classification model constructed based on the support vector machine to obtain the pre-classification evaluation indicators corresponding to each group of weight correction coefficients to be verified, including the pre-classification accuracy Qz, the pre-classification completion time Qt and the CPU usage during pre-classification Q cpu ; Calculate the evaluation coefficient Pxx based on the pre-classification evaluation index:

[0019]

[0020] Among them, ω1 is the weight factor of Qz, ω2 is The weight factor, ω3 is The weight factor of

[0021] The weight correction coefficient to be verified corresponding to the evaluation coefficient with the largest value is associated with the production parameter combination; the evaluation coefficients corresponding to other production parameter combinations are repeatedly calculated, and the weight correction coefficients to be verified corresponding to other production parameter combinations are determined to construct a production parameter-weight correction coefficient mapping database.

[0022] Furthermore, the classification result is optimized and corrected based on the quality grade of the galvanized sheet to obtain a corrected classification result, including the following steps: obtaining a quality grade-classification probability threshold mapping data set stored in a database, and determining the classification probability threshold according to the current quality grade of the galvanized sheet; if the maximum probability corresponding to the classification result is greater than the classification probability threshold, the classification result is not optimized and corrected, and the classification result is recorded as the corrected classification result; if the maximum probability corresponding to the classification result is not greater than the classification probability threshold, the classification result is optimized and corrected: obtaining a template surface feature set stored in the database, each template surface feature set corresponds to a defect classification template result; performing a similarity analysis on the screened surface feature set and the template surface feature set to obtain a similarity value; determining the template surface feature set corresponding to the maximum similarity value, and determining the defect classification template result; if the classification result is consistent with the defect classification template result, then outputting the classification result as the corrected classification result; if the classification result is inconsistent with the defect classification template result, then the defect classification template result is the corrected classification result.

[0023] The galvanized sheet surface defect classification system based on image recognition is applied to the above-mentioned galvanized sheet surface defect classification method based on image recognition, and includes: a preprocessing module, which is used to obtain the galvanized sheet surface image data and preprocess it to obtain the preprocessed surface image data; a feature processing module, which is used to perform feature processing on the preprocessed surface image data to obtain multi-dimensional surface feature data; a dimensionality reduction module, which is used to perform feature fusion and dimensionality reduction on the multi-dimensional surface feature data to obtain a screened surface feature set; a production process parameter and quality grade determination module, which is used to determine the production process related parameters and the galvanized sheet quality grade based on the screened surface feature set; an initial classification module, which is used to classify the galvanized sheet surface defects based on the screened surface feature set and the production process related parameters to obtain the classification results; a classification result correction module, which is used to optimize and correct the classification results based on the galvanized sheet quality grade to obtain the corrected classification results.

[0024] The present invention has the following beneficial effects:

[0025] This galvanized sheet surface defect classification method and system based on image recognition obtains galvanized sheet surface image data and preprocesses it, performs feature processing, and integrates dimensionality reduction. It uses multi-dimensional feature data and production process parameters to comprehensively and accurately classify galvanized sheet surface defects. At the same time, it optimizes the classification results based on quality grades to improve the reliability and accuracy of classification. It can solve the problem of inaccurate classification of galvanized sheet surface defects and difficulty in linking the production process with product quality.

[0026] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of the galvanized sheet surface defect classification method based on image recognition of the present invention.

[0028] Figure 2 This is a flowchart of the galvanized sheet surface defect classification system based on image recognition in the present invention. DETAILED DESCRIPTION

[0029] See also Figure 1 The present invention provides a technical solution for classifying galvanized sheet surface defects based on image recognition, including the following steps: obtaining and preprocessing galvanized sheet surface image data to obtain preprocessed surface image data; and providing basic data for subsequent analysis by obtaining the galvanized sheet surface image data. Preprocessing can remove interference factors such as noise and distortion from the image, improve image quality, and make subsequent feature processing more accurate, laying the foundation for accurate defect identification and avoiding misjudgment or omission of defects due to image quality issues.

[0030] The pre-processed surface image data is subjected to feature processing to obtain multi-dimensional surface feature data.

[0031] Multidimensional surface feature data includes geometric features, color features, frequency domain features and gradient features. The acquisition process is as follows: based on the OpenCV library function, the regional area, perimeter, circularity and aspect ratio of the galvanized sheet surface defect area are determined, and the regional area, perimeter, circularity and aspect ratio are recorded as geometric features; the color histogram and color moment of the galvanized sheet surface defect area are extracted, and the pixel distribution data of each color channel is statistically analyzed based on the color histogram, and the R channel mean, G channel mean, B channel mean, R channel variance, and G channel variance and B channel variance, pixel distribution data, R channel mean, G channel mean, B channel mean, R channel variance, G channel variance and B channel variance are recorded as color features; based on Fourier transform, the grayscale processed galvanized sheet surface defect area image is converted from the spatial domain to the frequency domain, and the spectrum, power spectrum, phase spectrum and frequency moment are calculated and obtained, and the spectrum, power spectrum, phase spectrum and frequency moment are recorded as frequency domain features; based on the Sobel operator, the gradient amplitude and direction of the image are calculated, the edge gradient information is extracted, and the edge gradient information is recorded as the gradient feature.

[0032] Features are extracted from multiple dimensions, including geometry, color, frequency domain, and gradient, to comprehensively capture the characteristic information of galvanized sheet surface defects. Geometric features can reflect the shape and size of the defect, color features can reveal the color difference between the defective area and the normal area, frequency domain features can help detect periodic or regular defect patterns, and gradient features can highlight the edge information of the defect. The comprehensive application of multi-dimensional features provides a rich data basis for defect classification, improving the accuracy and reliability of classification.

[0033] Feature fusion and dimensionality reduction are performed on multidimensional surface feature data to obtain a filtered surface feature set. Feature fusion integrates features from different dimensions to avoid information omission and enable subsequent analysis to comprehensively consider multiple factors. Dimensionality reduction removes redundant and irrelevant features, reducing data volume and computational complexity, improving model training and classification efficiency, and also avoiding overfitting caused by excessive features, making the model more generalizable.

[0034] Multidimensional surface feature data is fused based on a predefined order, yielding fused multidimensional surface feature data. The geometric, color, frequency, and gradient features reflecting the surface defects of galvanized sheet metal from different perspectives are fused in a predefined order. This integrates multidimensional information and avoids the limitations of single-feature analysis. Because different types of defects manifest differently across different feature dimensions, fusion allows for a comprehensive representation of defect characteristics, providing a richer data foundation for subsequent precise classification and improving the accuracy and reliability of defect classification.

[0035] A kernel principal component analysis is performed on the fused multidimensional surface feature data to obtain the eigenvalues of each defect feature in the fused multidimensional surface feature data. The eigenvalues are arranged in descending order, and the defect features corresponding to the first K eigenvalues are retained and combined into a defect feature set after the initial dimensionality reduction. The retained main components concentrate most of the key information of the original data, which reduces the subsequent calculation amount without losing too much effective information, improves the efficiency of model training and classification, and also helps to avoid overfitting problems, making the model more generalizable and more accurately classifying galvanized sheet surface defects.

[0036] The defect feature set after the initial dimensionality reduction is subjected to a secondary dimensionality reduction based on the gradient boosting decision tree and recursive feature elimination to obtain the filtered surface feature set.

[0037] The defect features after the initial dimensionality reduction and their corresponding galvanized sheet defect category labels are input into the GBDT model as training data. The GBDT model constructs a gradient boosting decision tree and fits the residuals. After the gradient boosting decision tree is constructed, the contribution of each defect feature after the initial dimensionality reduction in the defect feature set to the defect classification result is calculated. This can clarify the importance of each feature in the classification process, clearly understand which features play a key role in classification and which play a smaller role, and provide a basis for subsequent feature screening to ensure that the retained features are more valuable for classification.

[0038] Obtain the number of surface feature targets after screening; use the defect feature set after the initial dimensionality reduction and its corresponding defect labels as input to the defect pre-classification model constructed based on the support vector machine, and obtain the classification result to be verified, including the classification accuracy to be verified and the F1 value to be verified; determine the defect feature after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction and eliminate it, and obtain the defect feature set after elimination to be verified; use the defect feature set after elimination to be verified and its corresponding defect labels as input to the defect pre-classification model constructed based on the support vector machine, and obtain the classification result to be determined, including the classification accuracy to be determined and the F1 value to be determined;

[0039] Calculate the degree of decrease in the determination indicators of the classification result to be verified and the classification result to be determined, including the degree of decrease in classification accuracy and the degree of decrease in F1 value: If the degree of decrease in accuracy and the degree of decrease in F1 value are both within the set acceptable range, the defective feature with the smallest contribution after the initial dimensionality reduction in the defective feature set after the initial dimensionality reduction can be deleted; If the degree of decrease in accuracy or the degree of decrease in F1 value is not within the set acceptable range, the defective feature with the smallest contribution after the initial dimensionality reduction in the defective feature set after the initial dimensionality reduction cannot be deleted;

[0040] After removing the feature with the lowest contribution, the model is re-entered to obtain the classification result to be determined, and the degree of decrease in the determination index for both is calculated. This method allows for a visual assessment of the impact of removing a feature on the classification result and determines whether the feature is important. If the decrease in the index is within an acceptable range, it indicates that removing the feature will not have a significant negative impact on the classification results. This allows for the gradual selection of features with less impact on the classification, thereby optimizing the feature set.

[0041] The defect features after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction are repeatedly deleted and verified until a set of filtered surface features with the same number as the target number of filtered surface features is obtained.

[0042] Determine the production process related parameters and galvanized sheet quality grade based on the screened surface feature set;

[0043] Obtain the initial data of the galvanized sheet production process; for each screened surface feature in the screened surface feature set, calculate the correlation coefficient between it and each production process parameter in the initial data of the galvanized sheet production process (through Pearson or Spearman calculation), and determine the correlation coefficient between the screened surface feature and each production process parameter; repeatedly obtain the correlation coefficients between other screened surface features and each production process parameter; if all correlation coefficients corresponding to a certain production process parameter are greater than the set correlation threshold, retain the production process parameter; otherwise, delete the production process parameter to obtain the production process-related parameters.

[0044] By acquiring initial data from the galvanized sheet production process and calculating correlation coefficients between the screened surface features and various production process parameters, we can accurately identify the production parameters most closely related to these surface features. This not only provides key clues for analyzing the causes of defects, but also allows manufacturers to focus on monitoring and adjusting key parameters, effectively optimizing production processes and reducing defects at the source.

[0045] Obtain the template surface feature set corresponding to each galvanized sheet quality grade stored in the database; perform similarity analysis on the filtered surface feature set and the template surface feature set to obtain the similarity value xsP:

[0046] xsP=σ(JH,Jh)+σ(YS,Ys)+σ(PY,Py)+σ(TD,Td);

[0047] Among them, JH is the geometric feature, Jh is the template geometric feature in the template surface feature set, YS is the color feature, Ys is the template color feature in the template surface feature set, PY is the frequency domain feature, Py is the template frequency domain feature in the template surface feature set, TD is the gradient feature, Td is the template gradient feature in the template surface feature set, and σ(·) is the cosine similarity function.

[0048] Determine the template surface feature set corresponding to the maximum similarity value and determine the quality grade of the galvanized sheet.

[0049] Using the cosine similarity function, the similarity between the filtered surface feature set and the surface feature set of each template is calculated. The actual surface features of the galvanized sheet are quantitatively compared with standard templates of different quality grades from multiple feature dimensions, including geometry, color, frequency domain, and gradient. This quantitative approach makes quality grade determination more objective and accurate, avoiding the subjectivity and ambiguity of human judgment.

[0050] The surface defects of galvanized sheets are classified based on the screened surface feature set and production process related parameters to obtain the classification results.

[0051] Obtain the production parameter-weight correction coefficient mapping database stored in the database, and obtain the weight correction coefficient according to the current production parameters; obtain the initial weighted data of the surface defect characteristics, and correct the initial weighted data of the surface defect characteristics based on the weight correction coefficient; incorporate the variables in the production process into the consideration range of defect classification. Since different production parameters may cause changes in the surface characteristics of galvanized sheets, thereby affecting the defect performance, the weight correction coefficient can make the classification process better adapt to different production conditions and improve the accuracy and adaptability of classification.

[0052] The screened surface feature set is weighted based on the corrected initial weighted data of surface defect features to obtain a weighted surface feature set; the weighted surface feature set is input into a galvanized sheet surface defect classification model constructed based on support vector machine, the defect classification probability distribution is output, the maximum probability in the defect classification probability distribution is determined, and the defect classification label corresponding to the maximum probability is recorded as the classification result.

[0053] The weighted surface feature set is input into a galvanized sheet surface defect classification model built using a support vector machine (SVM). The classification result is determined by the defect classification probability distribution output by the model. SVMs are advantageous in handling small sample sizes and nonlinear classification problems. They can fully exploit the information in the feature set and, combined with the weighted features, more accurately determine the defect category. This provides a reliable basis for subsequent product quality assessment and processing, helping companies implement effective quality control.

[0054] The process of constructing the production parameter-weight correction coefficient mapping database is as follows: different production parameter combinations are determined, and several groups of weight correction coefficients to be verified are assigned to each production parameter combination; for one of the production parameter combinations, the screened surface feature set is weighted based on the corresponding several groups of weight correction coefficients to be verified and the initial weighted data of the surface defect features, and the weighted data are input into the defect pre-classification model constructed based on the support vector machine to obtain the pre-classification evaluation indicators corresponding to each group of weight correction coefficients to be verified, including the pre-classification accuracy Qz, the pre-classification completion time Qt and the CPU usage during pre-classification Q cpu ; Calculate the evaluation coefficient Pxx based on the pre-classification evaluation index:

[0055]

[0056] Among them, ω1 is the weight factor of Qz, ω2 is The weight factor, ω3 is The weight factor of

[0057] The weight correction coefficient to be verified corresponding to the evaluation coefficient with the largest value is associated with the production parameter combination; the evaluation coefficients corresponding to other production parameter combinations are repeatedly calculated, and the weight correction coefficients to be verified corresponding to other production parameter combinations are determined to construct a production parameter-weight correction coefficient mapping database.

[0058] The most appropriate weight correction coefficient can be found for different production conditions, resulting in a more appropriate weighting of surface defect features, optimized defect classification weights based on production parameters, and improved classification accuracy. The evaluation coefficient calculation comprehensively considers pre-classification accuracy, pre-classification completion time, and CPU usage during pre-classification. This not only focuses on classification accuracy but also takes into account the model's operational efficiency and resource utilization. This ensures that in practical applications, the classification model can achieve high classification accuracy while completing the classification task within a reasonable timeframe without excessively consuming system resources, thereby improving the practicality and stability of the classification system.

[0059] The classification results are optimized and corrected based on the quality grade of galvanized sheets to obtain the corrected classification results.

[0060] The quality grade-classification probability threshold mapping data set stored in the database is obtained. The classification probability threshold is determined based on the current galvanized sheet quality grade. If the maximum probability corresponding to the classification result is greater than the classification probability threshold, the classification result is not optimized and corrected, and the classification result is recorded as the corrected classification result. If the maximum probability corresponding to the classification result is not greater than the classification probability threshold, the classification result is optimized and corrected, avoiding the errors that may arise from judging based solely on the preliminary classification results. Galvanized sheets of different quality grades have different tolerances and evaluation criteria for defects. By using the quality grade-classification probability threshold mapping data set, the reliability of the classification results can be more accurately judged, the accuracy of defect classification can be improved, and the classification results can be more consistent with actual conditions.

[0061] Obtain the template surface feature set stored in the database, each template surface feature set corresponds to a defect classification template result; perform similarity analysis on the filtered surface feature set and the template surface feature set to obtain a similarity value (calculated in the same way as xsP); determine the template surface feature set corresponding to the largest similarity value, and determine the defect classification template result; if the classification result is consistent with the defect classification template result, then the output classification result is the corrected classification result; if the classification result is inconsistent with the defect classification template result, then the defect classification template result is the corrected classification result.

[0062] The defect classification template result corresponding to the template surface feature set with the highest similarity is used as the correction basis, so that the classification result can be adjusted according to the requirements of different quality levels, enhancing the adaptability of the classification system to different quality standards.

[0063] Galvanized sheet surface defect classification system based on image recognition, such as Figure 2 As shown, it includes: a preprocessing module, which is used to obtain the surface image data of the galvanized sheet and perform preprocessing to obtain the preprocessed surface image data; a feature processing module, which is used to perform feature processing on the preprocessed surface image data to obtain multi-dimensional surface feature data; a dimensionality reduction module, which is used to perform feature fusion and dimensionality reduction on the multi-dimensional surface feature data to obtain a screened surface feature set; a production process parameter and quality grade determination module, which is used to determine the production process related parameters and the quality grade of the galvanized sheet based on the screened surface feature set; a preliminary classification module, which is used to classify the surface defects of the galvanized sheet based on the screened surface feature set and the production process related parameters to obtain a classification result; a classification result correction module, which is used to optimize and correct the classification result based on the quality grade of the galvanized sheet to obtain a corrected classification result.

[0064] An electronic device comprises: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the above-mentioned method for classifying surface defects of galvanized sheet metal based on image recognition.

[0065] A computer-readable storage medium is used to store a program, which, when executed by a processor, implements the galvanized sheet surface defect classification method based on image recognition as described above.

[0066] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0070] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0071] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for classifying surface defects of galvanized sheet metal based on image recognition, characterized in that: The following steps are involved: Acquire and preprocess the surface image data of the galvanized sheet to obtain preprocessed surface image data; Performing feature processing on the pre-processed surface image data to obtain multi-dimensional surface feature data; Perform feature fusion and dimensionality reduction on multi-dimensional surface feature data to obtain a filtered surface feature set; Determine the production process related parameters and galvanized sheet quality grade based on the screened surface feature set; Based on the screened surface feature set and production process related parameters, the surface defects of the galvanized sheet are classified to obtain the classification results; The classification results are optimized and corrected based on the quality grade of galvanized sheets to obtain the corrected classification results.

2. The galvanized sheet surface defect classification method based on image recognition according to claim 1, characterized in that: The multi-dimensional surface feature data includes geometric features, color features, frequency domain features and gradient features, and the acquisition process is as follows: Based on the OpenCV library function, the area, perimeter, circularity and aspect ratio of the defective area on the galvanized sheet are determined, and the area, perimeter, circularity and aspect ratio are recorded as geometric features; The color histogram and color moment of the defective area on the surface of the galvanized sheet are extracted. The pixel distribution data of each color channel is statistically analyzed based on the color histogram. The R channel mean, G channel mean, B channel mean, R channel variance, G channel variance, and B channel variance of the RGB color space are obtained based on the color moment. The pixel distribution data, R channel mean, G channel mean, B channel mean, R channel variance, G channel variance, and B channel variance are recorded as color features. Based on Fourier transform, the grayscale processed image of the galvanized sheet surface defect area is converted from the spatial domain to the frequency domain, and the spectrum, power spectrum, phase spectrum and frequency moment are calculated and obtained. The spectrum, power spectrum, phase spectrum and frequency moment are recorded as frequency domain features. The gradient magnitude and direction of the image are calculated based on the Sobel operator, the edge gradient information is extracted, and the edge gradient information is recorded as the gradient feature.

3. The galvanized sheet surface defect classification method based on image recognition according to claim 1, characterized in that: Performing feature fusion and dimensionality reduction on multi-dimensional surface feature data to obtain a filtered surface feature set includes the following steps: Performing feature fusion on the multi-dimensional surface feature data based on a set arrangement order to obtain fused multi-dimensional surface feature data; Perform kernel principal component analysis on the fused multidimensional surface feature data to obtain the eigenvalues of each defect feature in the fused multidimensional surface feature data. Arrange the eigenvalues in descending order, retain the defect features corresponding to the first K eigenvalues, and combine them into the defect feature set after the initial dimensionality reduction. The defect feature set after the initial dimensionality reduction is subjected to a secondary dimensionality reduction based on the gradient boosting decision tree and recursive feature elimination to obtain the filtered surface feature set.

4. The galvanized sheet surface defect classification method based on image recognition according to claim 3, characterized in that: The defect feature set after the initial dimensionality reduction is subjected to secondary dimensionality reduction based on the gradient boosting decision tree and recursive feature elimination to obtain the filtered surface feature set, including the following steps: The defect features after the initial dimensionality reduction and their corresponding galvanized sheet defect category labels are input into the GBDT model as training data. The GBDT model constructs a gradient boosting decision tree and fits the residuals. After the gradient boosting decision tree is constructed, the contribution of each defect feature after the initial dimensionality reduction in the defect feature set to the defect classification result is calculated; Obtain the number of surface feature targets after screening; The defect feature set after the initial dimensionality reduction and its corresponding defect labels are used as input to the defect pre-classification model built based on the support vector machine to obtain the classification results to be verified, including the classification accuracy to be verified and the F1 value to be verified; Determine the defect feature after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction and eliminate it, so as to obtain the defect feature set after elimination to be verified; The defect feature set after verification and elimination and its corresponding defect label are used as input to the defect pre-classification model built based on the support vector machine to obtain the classification result to be determined, including the classification accuracy to be determined and the F1 value to be determined; Calculate the decline in the determination indicators of the classification results to be verified and the classification results to be determined, including the decline in classification accuracy and the decline in F1 value: If the decrease in accuracy and F1 value are both within the set acceptable range, the defect feature with the smallest contribution in the defect feature set after the initial dimensionality reduction can be deleted; If the accuracy drop or the F1 value drop is not within the set acceptable range, the defect feature with the smallest contribution in the defect feature set after the initial dimensionality reduction cannot be deleted; The defect features after the initial dimensionality reduction with the smallest contribution in the defect feature set after the initial dimensionality reduction are repeatedly deleted and verified until a set of filtered surface features with the same number as the target number of filtered surface features is obtained.

5. The galvanized sheet surface defect classification method based on image recognition according to claim 1, characterized in that: Determining production process related parameters based on the screened surface feature set includes the following steps: Obtain initial data of galvanized sheet production process; For each screened surface feature in the screened surface feature set, a correlation coefficient is calculated between the screened surface feature and each production process parameter in the initial data of the galvanized sheet production process, to determine the correlation coefficient between the screened surface feature and each production process parameter; Repeatedly obtain the correlation coefficients between other screened surface characteristics and various production process parameters; If all the correlation coefficients corresponding to a certain production process parameter are greater than the set correlation threshold, the production process parameter is retained; otherwise, the production process parameter is deleted to obtain the production process related parameters.

6. The galvanized sheet surface defect classification method based on image recognition according to claim 2, characterized in that: Determining the quality grade of galvanized sheet based on the surface feature set after screening includes the following steps: Obtain the template surface feature set corresponding to each galvanized sheet quality grade stored in the database; Perform similarity analysis on the filtered surface feature set and the template surface feature set to obtain the similarity value xsP: xsP=σ(JH,Jh)+σ(YS,Ys)+σ(PY,Py)+σ(TD,Td); Where JH is the geometric feature, Jh is the template geometric feature in the template surface feature set, YS is the color feature, Ys is the template color feature in the template surface feature set, PY is the frequency domain feature, Py is the template frequency domain feature in the template surface feature set, TD is the gradient feature, Td is the template gradient feature in the template surface feature set, and σ(·) is the cosine similarity function; Determine the template surface feature set corresponding to the maximum similarity value and determine the quality grade of the galvanized sheet.

7. The galvanized sheet surface defect classification method based on image recognition according to claim 1, characterized in that: The surface defects of the galvanized sheet are classified based on the screened surface feature set and production process related parameters to obtain the classification results, including the following steps: Obtain the production parameter-weight correction coefficient mapping database stored in the database, and obtain the weight correction coefficient according to the current production parameters; Obtaining initial weighted data of surface defect features, and correcting the initial weighted data of surface defect features based on a weight correction coefficient; performing weighted processing on the screened surface feature set based on the corrected initial weighted data of the surface defect features to obtain a weighted surface feature set; The weighted surface feature set is input into the galvanized sheet surface defect classification model built based on support vector machine, the defect classification probability distribution is output, the maximum probability in the defect classification probability distribution is determined, and the defect classification label corresponding to the maximum probability is recorded as the classification result.

8. The method for classifying surface defects of galvanized sheet based on image recognition according to claim 7, characterized in that: The process of constructing the production parameter-weight correction coefficient mapping database is as follows: Determine different production parameter combinations and assign several groups of weight correction coefficients to be verified for each production parameter combination; For one of the production parameter combinations, the screened surface feature set is weighted based on the corresponding groups of weight correction coefficients to be verified and the initial weighted data of the surface defect features, and then input into the defect pre-classification model constructed based on the support vector machine to obtain the pre-classification evaluation indicators corresponding to each group of weight correction coefficients to be verified, including the pre-classification accuracy Qz, the pre-classification completion time Qt and the CPU usage during pre-classification Q cpu ; Calculate the evaluation coefficient Pxx based on the pre-classification evaluation index: Among them, ω1 is the weight factor of Qz, ω2 is The weight factor, ω3 is The weight factor of Associating the weight correction coefficient to be verified corresponding to the evaluation coefficient with the largest value with the production parameter combination; Repeatedly calculate the evaluation coefficients corresponding to other production parameter combinations, determine the weight correction coefficients to be verified corresponding to other production parameter combinations, and construct a production parameter-weight correction coefficient mapping database.

9. The method for classifying surface defects of galvanized sheet based on image recognition according to claim 7, characterized in that: The classification results are optimized and corrected based on the quality grade of the galvanized sheet to obtain the corrected classification results, including the following steps: Obtain the quality grade-classification probability threshold mapping data set stored in the database, and determine the classification probability threshold according to the current galvanized sheet quality grade; If the maximum probability corresponding to the classification result is greater than the classification probability threshold, the classification result will not be optimized and corrected, and the classification result will be recorded as the corrected classification result; If the maximum probability corresponding to the classification result is not greater than the classification probability threshold, the classification result is optimized and corrected: Obtaining a template surface feature set stored in a database, each template surface feature set corresponds to a defect classification template result; Performing similarity analysis on the filtered surface feature set and the template surface feature set to obtain a similarity value; Determine the template surface feature set corresponding to the maximum similarity value and determine the defect classification template result; If the classification result is consistent with the defect classification template result, the output classification result is the corrected classification result. If the classification result is inconsistent with the defect classification template result, the defect classification template result is the corrected classification result.

10. A galvanized sheet surface defect classification system based on image recognition, applied to the galvanized sheet surface defect classification method based on image recognition according to any one of claims 1 to 9, characterized in that: include: A preprocessing module is used to obtain and preprocess the surface image data of the galvanized sheet to obtain the preprocessed surface image data; A feature processing module is used to perform feature processing on the pre-processed surface image data to obtain multi-dimensional surface feature data; Dimensionality reduction module, used to perform feature fusion and dimensionality reduction on multi-dimensional surface feature data to obtain a filtered surface feature set; Production process parameter and quality grade determination module, used to determine production process related parameters and galvanized sheet quality grade based on the screened surface feature set; The initial classification module is used to classify the surface defects of the galvanized sheet based on the screened surface feature set and production process related parameters to obtain the classification results; The classification result correction module is used to optimize and correct the classification result based on the quality grade of the galvanized sheet to obtain the corrected classification result.

Images