Galvanized sheet surface defect classification method and system based on image recognition

Abstract

The application discloses a galvanized sheet surface flaw classification method and system based on image recognition, and relates to the technical field of flaw classification. The galvanized sheet surface flaw classification method based on image recognition comprises the following steps: acquiring galvanized sheet surface image data and performing pretreatment to obtain pretreated surface image data; performing feature processing on the pretreated surface image data to obtain multi-dimensional surface feature data; performing feature fusion and dimension reduction on the multi-dimensional surface feature data to obtain a screened surface feature set; determining production process related parameters and a galvanized sheet quality grade based on the screened surface feature set; classifying galvanized sheet surface flaws based on the screened surface feature set and the production process related parameters to obtain a classification result; and optimizing and correcting the classification result based on the galvanized sheet quality grade to obtain a corrected classification result, thereby solving the problems of inaccurate galvanized sheet surface flaw classification and difficulty in correlating the production process and product quality.

Description

Technical Field

[0001] This invention relates to the field of defect classification technology, specifically to a method and system for classifying defects on the surface of galvanized steel sheets based on image recognition. Background Technology

[0002] Traditional manual visual inspection is not only inefficient and inaccurate, but also monotonous and tedious. Furthermore, the high volume of industrial production and long working hours lead to worker fatigue, increasing the likelihood of false positives and missed detections. Moreover, the judgment standards for manual inspection are not quantified, making it impossible to guarantee product quality. Therefore, machine vision-based defect detection systems have emerged. In the field of machine vision, there are many algorithms for defect detection that can be applied to the surface defect detection of specific items (such as fabrics, blankets, galvanized sheets, and floor tiles).

[0003] Chinese invention patent CN108647706B discloses a machine vision-based method for object recognition, classification, and defect detection, including an object category recognition and classification process based on support vector machines; the object category recognition and classification process includes a model training process and a model loading and classification process; a defect detection process based on a machine vision defect detection algorithm; the machine vision defect detection algorithm includes an image correction process and a similarity comparison by calculating the Hu invariant moments of the images; and determining whether the image has defects based on the detection results.

[0004] However, existing machine vision-based defect classification methods are difficult to correlate with the production process and product quality when applied to the classification of defects on galvanized sheet surfaces, resulting in inaccuracies. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for classifying surface defects of galvanized steel sheets based on image recognition, which solves the problems of inaccurate classification of surface defects of galvanized steel sheets and difficulty in associating them with the production process and product quality.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for classifying surface defects of galvanized steel sheets based on image recognition, comprising the following steps: acquiring surface image data of galvanized steel sheets and preprocessing it to obtain preprocessed surface image data; performing feature processing on the preprocessed surface image data to obtain multidimensional surface feature data; performing feature fusion and dimensionality reduction on the multidimensional surface feature data to obtain a filtered surface feature set; determining relevant parameters of the production process and the quality grade of the galvanized steel sheets based on the filtered surface feature set; classifying surface defects of the galvanized steel sheets based on the filtered surface feature set and the relevant parameters of the production process to obtain classification results; and optimizing and correcting the classification results based on the quality grade of the galvanized steel sheets to obtain corrected classification results.

[0007] Furthermore, the multidimensional surface feature data includes geometric features, color features, frequency domain features, and gradient features. The acquisition process is as follows: Based on OpenCV library functions, the area, perimeter, roundness, and aspect ratio of the defective region on the galvanized sheet surface are determined, and these are recorded as geometric features. The color histogram and color moments of the defective region on the galvanized sheet surface are extracted. Based on the color histogram, the pixel distribution data of each color channel is statistically analyzed. Based on the color moments, the mean values ​​of the R channel, G channel, B channel, and R channel in the RGB color space are obtained. The variances of the G and B channels are used to define the color features, including pixel distribution data, R channel mean, G channel mean, B channel mean, R channel variance, G channel variance, and B channel variance. Based on Fourier transform, the grayscale image of the defect area on the galvanized sheet surface is transformed from the spatial domain to the frequency domain, and the spectrum, power spectrum, phase spectrum, and frequency moment are calculated and defined as frequency domain features. The gradient magnitude and direction of the image are calculated based on the Sobel operator, and edge gradient information is extracted and defined as gradient features.

[0008] Furthermore, feature fusion and dimensionality reduction are performed on the multidimensional surface feature data to obtain a filtered set of surface features. This includes the following steps: The multidimensional surface feature data is fused based on a predetermined order to obtain fused multidimensional surface feature data; Kernel principal component analysis is performed on the fused multidimensional surface feature data to obtain the feature values ​​of each defect feature in the fused multidimensional surface feature data; the feature values ​​are arranged in descending order, and the defect features corresponding to the top K feature values ​​are retained to form a set of defect features after the first dimensionality reduction; The set of defect features after the first dimensionality reduction is further dimensionality-reduced based on gradient boosting decision trees and recursive feature elimination to obtain a filtered set of surface features.

[0009] Furthermore, a second dimensionality reduction is performed on the defect feature set after the initial dimensionality reduction based on gradient boosting decision tree and recursive feature elimination to obtain a filtered surface feature set. The steps include: inputting the defect features after the initial dimensionality reduction and their corresponding galvanized sheet defect category labels as training data into the GBDT model; constructing a gradient boosting decision tree and fitting the residuals in the GBDT model; and calculating the contribution of each defect feature after the initial dimensionality reduction to the defect classification result in the defect feature set after the initial dimensionality reduction.

[0010] Obtain the number of surface feature targets after filtering; use the initial dimensionality reduction defect feature set and its corresponding defect labels as input to the defect pre-classification model based on support vector machine to obtain the classification result to be verified, including the classification accuracy and the F1 value to be verified; identify the defect feature with the smallest contribution in the initial dimensionality reduction defect feature set and remove it to obtain the defect feature set to be verified after removal; use the defect feature set to be verified after removal and its corresponding defect labels as input to the defect pre-classification model based on support vector machine to obtain the classification result to be determined, including the classification accuracy and the F1 value to be determined.

[0011] Calculate the decrease in the determination index of the classification result to be verified and the classification result to be determined, including the decrease in classification accuracy and the decrease in F1 value: If the decrease in accuracy and the decrease in F1 value are both within the set acceptable range, the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction can be deleted; if the decrease in accuracy or the decrease in F1 value are not within the set acceptable range, the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction cannot be deleted; repeatedly delete the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction and verify until a set of filtered surface features with the same number as the number of filtered surface feature targets is obtained.

[0012] Further, the relevant parameters of the production process are determined based on the set of screened surface features, including the following steps: obtaining initial data of the galvanized sheet production process; for each screened surface feature in the set of screened surface features, calculating the correlation coefficient between it and each production process parameter in the initial data of the galvanized sheet production process, and determining the correlation coefficient between the screened surface feature and each production process parameter; repeatedly obtaining the correlation coefficients between other screened surface features and each production process parameter; if all correlation coefficients corresponding to a certain production process parameter are greater than the set correlation threshold, then the production process parameter is retained; otherwise, the production process parameter is deleted, thus obtaining the relevant parameters of the production process.

[0013] Furthermore, determining the quality grade of galvanized steel sheets based on the filtered surface feature set includes the following steps: obtaining the template surface feature set corresponding to each galvanized steel sheet quality grade stored in the database; performing a similarity analysis on the filtered surface feature set and the template surface feature set to obtain the similarity value xsP:

[0014] xsP=σ(JH,Jh)+σ(YS,Ys)+σ(PY,Py)+σ(TD,Td);

[0015] Wherein, JH is the geometric feature, Jh is the template geometric feature in the template surface feature set, YS is the color feature, Ys is the template color feature in the template surface feature set, PY is the frequency domain feature, Py is the template frequency domain feature in the template surface feature set, TD is the gradient feature, Td is the template gradient feature in the template surface feature set, and σ(·) is the cosine similarity function.

[0016] Determine the set of template surface features corresponding to the highest similarity value, and then determine the quality grade of the galvanized sheet.

[0017] Furthermore, based on the filtered surface feature set and production process-related parameters, surface defects of galvanized steel sheets are classified to obtain classification results. This includes the following steps: obtaining the production parameter-weight correction coefficient mapping database stored in the database, and obtaining the weight correction coefficients according to the current production parameters; obtaining the initial weighted data of surface defect features, and correcting the initial weighted data of surface defect features based on the weight correction coefficients; weighting the filtered surface feature set based on the corrected initial weighted data of surface defect features to obtain a weighted surface feature set; inputting the weighted surface feature set into the galvanized steel sheet surface defect classification model constructed based on support vector machine, outputting the defect classification probability distribution, determining the maximum probability in the defect classification probability distribution, and recording the defect classification label corresponding to the maximum probability as the classification result.

[0018] Furthermore, the process of constructing the production parameter-weight correction coefficient mapping database is as follows: Different combinations of production parameters are determined, and several sets of weight correction coefficients to be verified are assigned to each combination. For one of the production parameter combinations, the selected surface feature set is weighted based on its corresponding sets of weight correction coefficients to be verified and the initial weighted data of surface defect features. This weighted data is then input into a defect pre-classification model built on support vector machines to obtain the pre-classification evaluation indicators corresponding to each set of weight correction coefficients to be verified, including pre-classification accuracy Qz, pre-classification completion time Qt, and CPU utilization during pre-classification Q. cpu The evaluation coefficient Pxx is calculated based on the pre-classification evaluation indicators.

[0019]

[0020] Where ω1 is the weighting factor of Qz, and ω2 is... The weighting factor, ω3 is Weighting factors;

[0021] Associate the weight correction coefficient to be verified corresponding to the evaluation coefficient with the largest value with the production parameter combination; repeatedly calculate the evaluation coefficients corresponding to other production parameter combinations, determine the weight correction coefficients to be verified corresponding to other production parameter combinations, and construct a production parameter-weight correction coefficient mapping database.

[0022] Furthermore, the classification results are optimized and corrected based on the quality grade of the galvanized sheet to obtain the corrected classification results, including the following steps: Obtain the quality grade-classification probability threshold mapping data set stored in the database; determine the classification probability threshold based on the current galvanized sheet quality grade; if the maximum probability corresponding to the classification result is greater than the classification probability threshold, then no optimization or correction is performed on the classification result, and the classification result is recorded as the corrected classification result; if the maximum probability corresponding to the classification result is not greater than the classification probability threshold, then the classification result is optimized and corrected: obtain the template surface feature set stored in the database, each template surface feature set corresponding to a defect classification template result; perform similarity analysis on the filtered surface feature set and the template surface feature set to obtain similarity values; determine the template surface feature set corresponding to the maximum similarity value to determine the defect classification template result; if the classification result is consistent with the defect classification template result, then the output classification result is the corrected classification result; if the classification result is inconsistent with the defect classification template result, then the defect classification template result is the corrected classification result.

[0023] The image recognition-based galvanized sheet surface defect classification system, applied to the aforementioned image recognition-based galvanized sheet surface defect classification method, includes: a preprocessing module for acquiring and preprocessing galvanized sheet surface image data to obtain preprocessed surface image data; a feature processing module for performing feature processing on the preprocessed surface image data to obtain multi-dimensional surface feature data; a dimensionality reduction module for performing feature fusion and dimensionality reduction on the multi-dimensional surface feature data to obtain a filtered surface feature set; a production process parameter and quality grade determination module for determining relevant production process parameters and galvanized sheet quality grades based on the filtered surface feature set; an initial classification module for classifying galvanized sheet surface defects based on the filtered surface feature set and relevant production process parameters to obtain classification results; and a classification result correction module for optimizing and correcting the classification results based on the galvanized sheet quality grade to obtain corrected classification results.

[0024] The present invention has the following beneficial effects:

[0025] This image recognition-based method and system for classifying surface defects of galvanized steel sheets acquires and preprocesses image data of the galvanized steel sheet surface, performs feature processing, and fuses and reduces dimensions. By utilizing multi-dimensional feature data and production process parameters, it comprehensively and accurately classifies surface defects of galvanized steel sheets. At the same time, it optimizes the classification results based on quality grades to improve the reliability and accuracy of classification. This method can solve the problems of inaccurate classification of surface defects of galvanized steel sheets and difficulty in associating them with the production process and product quality.

[0026] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0027] Figure 1 This is a flowchart of the image recognition-based method for classifying surface defects of galvanized steel sheets according to the present invention.

[0028] Figure 2 This is a flowchart of the image recognition-based galvanized sheet surface defect classification system of the present invention. Detailed Implementation

[0029] Please see Figure 1 This invention provides a technical solution: a method for classifying surface defects of galvanized steel sheets based on image recognition, comprising the following steps: acquiring surface image data of galvanized steel sheets and performing preprocessing to obtain preprocessed surface image data; acquiring surface image data of galvanized steel sheets provides basic information for subsequent analysis. Preprocessing can remove noise, distortion, and other interference factors in the image, improve image quality, make subsequent feature processing more accurate, lay the foundation for accurate defect identification, and avoid misjudgment or missed judgment of defects due to image quality issues.

[0030] Feature processing is performed on the preprocessed surface image data to obtain multidimensional surface feature data.

[0031] Multidimensional surface feature data includes geometric features, color features, frequency domain features, and gradient features. The acquisition process is as follows: Based on OpenCV library functions, the area, perimeter, roundness, and aspect ratio of the defective region on the galvanized sheet surface are determined, and these are recorded as geometric features. The color histogram and color moments of the defective region on the galvanized sheet surface are extracted. Based on the color histogram, the pixel distribution data of each color channel is statistically analyzed. Based on the color moments, the mean of the R channel, the mean of the G channel, the mean of the B channel, and the variance of the R channel in the RGB color space are obtained. The variances of the G and B channels are used to define the pixel distribution data, the mean of the R channel, the mean of the G channel, the mean of the B channel, the variance of the R channel, the variance of the G channel, and the variance of the B channel as color features. Based on Fourier transform, the grayscale image of the defect area on the galvanized sheet surface is transformed from the spatial domain to the frequency domain, and the spectrum, power spectrum, phase spectrum, and frequency moment are calculated and defined as frequency domain features. Based on the Sobel operator, the gradient magnitude and direction of the image are calculated, and the edge gradient information is extracted and defined as gradient features.

[0032] Features are extracted from multiple dimensions, including geometry, color, frequency domain, and gradient, to comprehensively capture the characteristic information of defects on the surface of galvanized steel sheets. Geometric features reflect the shape and size of defects, color features show the color difference between defective and normal areas, frequency domain features help to discover periodic or regular defect patterns, and gradient features highlight defect edge information. The comprehensive application of multi-dimensional features provides rich data for defect classification, improving the accuracy and reliability of classification.

[0033] Feature fusion and dimensionality reduction are performed on multidimensional surface feature data to obtain a filtered set of surface features. Feature fusion integrates features from different dimensions, avoiding information omissions and enabling subsequent analysis to comprehensively consider multiple factors. Dimensionality reduction removes redundant and irrelevant features, reduces data volume, lowers computational complexity, and improves model training and classification efficiency. It also avoids overfitting caused by too many features, making the model more generalizable.

[0034] Based on a predetermined arrangement order, multidimensional surface feature data is fused to obtain fused multidimensional surface feature data. This involves fusing geometric, color, frequency domain, and gradient features that reflect the surface defects of galvanized steel sheets from different perspectives, according to a set order. This approach integrates multidimensional information, avoiding the limitations of single-feature analysis. Because different types of defects exhibit different characteristics across different feature dimensions, the fused data comprehensively presents the defect features, providing a richer data foundation for subsequent accurate classification and improving the accuracy and reliability of defect classification.

[0035] Kernel principal component analysis was performed on the fused multidimensional surface feature data to obtain the feature values ​​of each defect feature in the fused multidimensional surface feature data. The feature values ​​were arranged in descending order, and the defect features corresponding to the first K feature values ​​were retained to form the defect feature set after the initial dimensionality reduction. The retained principal components concentrated most of the key information of the original data, which reduced the amount of subsequent calculations and improved the efficiency of model training and classification without losing too much effective information. At the same time, it also helped to avoid overfitting problems, making the model more generalizable and more accurate in classifying defects on the surface of galvanized sheets.

[0036] A second dimensionality reduction is performed on the defective feature set after the initial dimensionality reduction using gradient boosting decision trees and recursive feature elimination, resulting in a filtered surface feature set.

[0037] The defect features after the initial dimensionality reduction and their corresponding galvanized sheet defect category labels are input into the GBDT model as training data. The GBDT model constructs a gradient boosting decision tree and fits the residuals. After the gradient boosting decision tree is constructed, the contribution of each defect feature after the initial dimensionality reduction to the defect classification result is calculated. This clarifies the importance of each feature in the classification process, clearly identifies which features play a key role in classification, and which have a smaller role, providing a basis for subsequent feature selection and ensuring that the retained features have greater classification value.

[0038] Obtain the number of surface feature targets after filtering; use the initial dimensionality reduction defect feature set and its corresponding defect labels as input to the defect pre-classification model based on support vector machine to obtain the classification result to be verified, including the classification accuracy and the F1 value to be verified; identify the defect feature with the smallest contribution in the initial dimensionality reduction defect feature set and remove it to obtain the defect feature set to be verified after removal; use the defect feature set to be verified after removal and its corresponding defect labels as input to the defect pre-classification model based on support vector machine to obtain the classification result to be determined, including the classification accuracy and the F1 value to be determined.

[0039] Calculate the decrease in the determination index between the classification result to be verified and the classification result to be determined, including the decrease in classification accuracy and the decrease in F1 score: If the decrease in accuracy and the decrease in F1 score are both within the set acceptable range, the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction can be deleted; if the decrease in accuracy or the decrease in F1 score are not within the set acceptable range, the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction cannot be deleted.

[0040] After removing the feature with the least contribution, the data is input back into the model to obtain the classification result to be determined, and the decrease in the determination index for both is calculated. This method allows for a direct assessment of the impact of removing a feature on the classification result, determining whether the feature is important. If the decrease in the index is within an acceptable range, it indicates that removing the feature will not have a significant negative impact on the classification effect, thus allowing for the gradual selection of features with less impact on classification and optimization of the feature set.

[0041] Repeatedly delete the flawed features with the smallest contribution from the flawed feature set after the initial dimensionality reduction and verify them until a set of filtered surface features with the same number as the number of filtered surface feature targets is obtained.

[0042] The relevant parameters of the production process and the quality grade of the galvanized sheet are determined based on the screened surface feature set.

[0043] Obtain initial data for the galvanized sheet production process; for each screened surface feature in the set of screened surface features, calculate the correlation coefficient between it and each production process parameter in the initial data of the galvanized sheet production process (using Pearson or Spearman's method) to determine the correlation coefficient between the screened surface feature and each production process parameter; repeatedly obtain the correlation coefficients between other screened surface features and each production process parameter; if all correlation coefficients corresponding to a certain production process parameter are greater than the set correlation threshold, retain the production process parameter; otherwise, delete the production process parameter to obtain the relevant parameters of the production process.

[0044] By acquiring initial data from the galvanized sheet production process and calculating the correlation coefficients between screened surface features and various production process parameters, production parameters closely related to surface features can be accurately identified. This not only provides crucial clues for analyzing the causes of defects but also allows production companies to focus on monitoring and adjusting key parameters, effectively optimizing production processes and reducing defects at the source.

[0045] Obtain the template surface feature set corresponding to each galvanized sheet quality grade stored in the database; perform similarity analysis on the filtered surface feature set and the template surface feature set to obtain the similarity value xsP:

[0046] xsP=σ(JH,Jh)+σ(YS,Ys)+σ(PY,Py)+σ(TD,Td);

[0047] Wherein, JH is the geometric feature, Jh is the template geometric feature in the template surface feature set, YS is the color feature, Ys is the template color feature in the template surface feature set, PY is the frequency domain feature, Py is the template frequency domain feature in the template surface feature set, TD is the gradient feature, Td is the template gradient feature in the template surface feature set, and σ(·) is the cosine similarity function.

[0048] Determine the set of template surface features corresponding to the highest similarity value, and then determine the quality grade of the galvanized sheet.

[0049] The cosine similarity function is used to calculate the similarity value between the filtered surface feature set and the surface feature sets of each template. From multiple feature dimensions such as geometry, color, frequency domain, and gradient, the actual surface features of the galvanized sheet are quantitatively compared with standard templates of different quality grades. This quantification method makes the judgment of quality grade more objective and accurate, avoiding the subjectivity and ambiguity of human judgment.

[0050] Based on the filtered set of surface features and relevant parameters of the production process, surface defects of galvanized steel sheets are classified to obtain classification results.

[0051] The system retrieves the production parameter-weight correction coefficient mapping database stored in the database, and obtains the weight correction coefficients based on the current production parameters. It also retrieves the initial weighted data of surface defect features and corrects the initial weighted data of surface defect features based on the weight correction coefficients. Variables in the production process are included in the scope of defect classification. Since different production parameters may cause changes in the surface characteristics of galvanized sheets, thus affecting defect performance, the weight correction coefficients enable the classification process to better adapt to different production conditions, improving the accuracy and adaptability of classification.

[0052] The filtered surface feature set is weighted based on the initial weighted data of the corrected surface defect features to obtain the weighted surface feature set. The weighted surface feature set is then input into the galvanized sheet surface defect classification model built based on support vector machine, and the defect classification probability distribution is output. The maximum probability in the defect classification probability distribution is determined, and the defect classification label corresponding to the maximum probability is recorded as the classification result.

[0053] The weighted set of surface features is input into a galvanized sheet surface defect classification model built on a support vector machine (SVM). The classification result is determined by the defect classification probability distribution output by the model. SVM has advantages in handling small sample sizes and nonlinear classification problems. It can fully mine the information in the feature set and, combined with the weighted features, more accurately determine the defect category, providing a reliable basis for subsequent product quality assessment and processing, and helping enterprises to conduct effective quality control.

[0054] The process of constructing the production parameter-weight correction coefficient mapping database is as follows: Different combinations of production parameters are determined, and several sets of weight correction coefficients to be verified are assigned to each combination. For one of the production parameter combinations, the selected surface feature set is weighted based on its corresponding sets of weight correction coefficients to be verified and the initial weighted data of surface defect features. This weighted data is then input into a defect pre-classification model built on a support vector machine to obtain the pre-classification evaluation indicators corresponding to each set of weight correction coefficients to be verified, including pre-classification accuracy Qz, pre-classification completion time Qt, and CPU utilization during pre-classification Q. cpu The evaluation coefficient Pxx is calculated based on the pre-classification evaluation indicators.

[0055]

[0056] Where ω1 is the weighting factor of Qz, and ω2 is... The weighting factor, ω3 is Weighting factors;

[0057] Associate the weight correction coefficient to be verified corresponding to the evaluation coefficient with the largest value with the production parameter combination; repeatedly calculate the evaluation coefficients corresponding to other production parameter combinations, determine the weight correction coefficients to be verified corresponding to other production parameter combinations, and construct a production parameter-weight correction coefficient mapping database.

[0058] For different production conditions, the most suitable weight correction coefficients can be found to make the weighting of surface defect features more reasonable, optimize the defect classification weights based on production parameters, and improve classification accuracy. The evaluation coefficient calculation comprehensively considers pre-classification accuracy, pre-classification completion time, and CPU utilization during pre-classification. This not only focuses on classification accuracy but also takes into account the model's operating efficiency and resource consumption. This ensures that in practical applications, the classification model can guarantee high classification accuracy, complete the classification task within a reasonable time, and avoid excessive consumption of system resources, thus improving the practicality and stability of the classification system.

[0059] The classification results are optimized and corrected based on the quality grade of galvanized steel sheets to obtain the corrected classification results.

[0060] The system retrieves the quality grade-classification probability threshold mapping dataset stored in the database. Based on the current galvanized sheet quality grade, it determines the classification probability threshold. If the maximum probability corresponding to the classification result is greater than the classification probability threshold, the classification result is not optimized or corrected and is recorded as the corrected classification result. If the maximum probability corresponding to the classification result is not greater than the classification probability threshold, the classification result is optimized and corrected, avoiding potential errors that may arise from judging solely based on preliminary classification results. Different quality grades of galvanized sheets have different tolerances and evaluation standards for defects. By using the quality grade-classification probability threshold mapping dataset, the reliability of the classification results can be more accurately determined, improving the accuracy of defect classification and making the classification results more consistent with reality.

[0061] Retrieve the template surface feature set stored in the database. Each template surface feature set corresponds to a defect classification template result. Perform similarity analysis on the filtered surface feature set and the template surface feature set to obtain the similarity value (calculated in the same way as xsP). Determine the template surface feature set corresponding to the largest similarity value to determine the defect classification template result. If the classification result is consistent with the defect classification template result, output the corrected classification result. If the classification result is inconsistent with the defect classification template result, the defect classification template result is the corrected classification result.

[0062] By using the defect classification template result corresponding to the template surface feature set with the highest similarity as the correction basis, the classification result can be adjusted according to the requirements of different quality levels, thereby enhancing the adaptability of the classification system to different quality standards.

[0063] Image recognition-based galvanized sheet surface defect classification system, such as Figure 2 As shown, it includes: a preprocessing module for acquiring and preprocessing image data of the galvanized sheet surface to obtain preprocessed surface image data; a feature processing module for performing feature processing on the preprocessed surface image data to obtain multi-dimensional surface feature data; a dimensionality reduction module for performing feature fusion and dimensionality reduction on the multi-dimensional surface feature data to obtain a set of filtered surface features; a production process parameter and quality grade determination module for determining relevant production process parameters and the quality grade of the galvanized sheet based on the set of filtered surface features; a preliminary classification module for classifying surface defects of the galvanized sheet based on the set of filtered surface features and relevant production process parameters to obtain classification results; and a classification result correction module for optimizing and correcting the classification results based on the quality grade of the galvanized sheet to obtain corrected classification results.

[0064] An electronic device includes: a processor; and a memory storing computer program instructions that, when executed by the processor, cause the processor to perform the image recognition-based method for classifying surface defects of galvanized steel sheets as described above.

[0065] A computer-readable storage medium for storing a program that, when executed by a processor, implements the image recognition-based method for classifying surface defects of galvanized steel sheets as described above.

[0066] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of systems, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0071] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for classifying surface defects of galvanized steel sheets based on image recognition, characterized in that, Includes the following steps: Acquire surface image data of galvanized sheet and perform preprocessing to obtain preprocessed surface image data; Feature processing is performed on the preprocessed surface image data to obtain multidimensional surface feature data; Feature fusion and dimensionality reduction are performed on multidimensional surface feature data to obtain a filtered set of surface features. The relevant parameters of the production process and the quality grade of the galvanized sheet are determined based on the screened surface feature set. Based on the filtered surface feature set and production process-related parameters, surface defects of galvanized steel sheets are classified to obtain classification results. The classification results are optimized and corrected based on the quality grade of galvanized steel sheets to obtain the corrected classification results. Determining relevant parameters for the production process based on the screened set of surface features includes the following steps: Obtain initial data for the galvanized sheet production process; For each screened surface feature in the set of screened surface features, the correlation coefficient between it and each production process parameter in the initial data of the galvanized sheet production process is calculated to determine the correlation coefficient between the screened surface feature and each production process parameter. Repeatedly obtain the correlation coefficients between other screened surface features and various production process parameters; If all the correlation coefficients corresponding to a certain production process parameter are greater than the set correlation threshold, then the production process parameter is retained; otherwise, the production process parameter is deleted, and the production process related parameters are obtained. Based on the filtered surface feature set and production process-related parameters, surface defects of galvanized steel sheets are classified to obtain classification results, including the following steps: Retrieve the production process-related parameters-weight correction coefficient mapping database stored in the database, and obtain the weight correction coefficient based on the current production process-related parameters; Obtain initial weighted data of surface defect features, and correct the initial weighted data of surface defect features based on the weight correction coefficient; The filtered surface feature set is weighted based on the initial weighted data of the corrected surface defect features to obtain the weighted surface feature set. The weighted set of surface features is input into the galvanized sheet surface defect classification model built based on support vector machine. The defect classification probability distribution is output, the maximum probability in the defect classification probability distribution is determined, and the defect classification label corresponding to the maximum probability is recorded as the classification result.

2. The image recognition-based galvanized sheet surface defect classification method according to claim 1, characterized by, The multidimensional surface feature data includes geometric features, color features, frequency domain features, and gradient features, and the acquisition process is as follows: Based on OpenCV library functions, the area, perimeter, roundness, and aspect ratio of the defect area on the galvanized sheet surface are determined, and the area, perimeter, roundness, and aspect ratio are recorded as geometric features; Extract the color histogram and color moments of the defect area on the surface of the galvanized sheet. Based on the color histogram, statistically analyze the pixel distribution data of each color channel. Based on the color moments, obtain the mean of R channel, mean of G channel, mean of B channel, variance of R channel, variance of G channel, and variance of B channel in the RGB color space. Record the pixel distribution data, mean of R channel, mean of G channel, mean of B channel, variance of R channel, variance of G channel, and variance of B channel as color features. Based on Fourier transform, the grayscale image of the defect area on the surface of the galvanized sheet is transformed from the spatial domain to the frequency domain, and the spectrum, power spectrum, phase spectrum and frequency moment are calculated and obtained. The spectrum, power spectrum, phase spectrum and frequency moment are recorded as frequency domain features. The gradient magnitude and direction of the image are calculated based on the Sobel operator, and the edge gradient information is extracted and recorded as the gradient feature.

3. The image recognition-based galvanized sheet surface defect classification method according to claim 1, characterized by, The process of fusing and reducing the dimensionality of multidimensional surface feature data to obtain a filtered set of surface features includes the following steps: Based on the set arrangement order, the multidimensional surface feature data is fused to obtain the fused multidimensional surface feature data. Kernel principal component analysis was performed on the fused multidimensional surface feature data to obtain the feature values ​​of each defect feature in the fused multidimensional surface feature data. The feature values ​​were arranged in descending order, and the defect features corresponding to the first K feature values ​​were retained and combined to form the defect feature set after the first dimensionality reduction. A second dimensionality reduction is performed on the defective feature set after the initial dimensionality reduction using gradient boosting decision trees and recursive feature elimination, resulting in a filtered surface feature set.

4. The image recognition-based galvanized sheet surface defect classification method according to claim 3, characterized by, A second dimensionality reduction is performed on the defect feature set after the initial dimensionality reduction using gradient boosting decision trees and recursive feature elimination to obtain the filtered surface feature set, including the following steps: The initial dimensionality reduction defect feature set and its corresponding galvanized sheet defect labels are used as training data and input into the GBDT model. The GBDT model constructs a gradient boosting decision tree and fits the residuals. After the gradient boosting decision tree is constructed, the contribution of each initial dimensionality reduction defect feature in the initial dimensionality reduction defect feature set to the defect classification result is calculated. Obtain the number of surface feature targets after filtering; The defect feature set after the initial dimensionality reduction and its corresponding defect labels are used as input to the defect pre-classification model built based on support vector machine to obtain the classification results to be verified, including the classification accuracy and the F1 value to be verified. Identify and remove the flawed feature with the smallest contribution from the flawed feature set after the initial dimensionality reduction, and obtain the flawed feature set after removal to be verified. The set of defect features after removal and their corresponding defect labels are used as input to a defect pre-classification model built on support vector machine to obtain the classification result to be determined, including the classification accuracy and the F1 value to be determined. Calculate the decrease in certainty metrics between the classification results to be validated and those to be determined, including the decrease in classification accuracy and the decrease in F1 score: If the decrease in accuracy and the decrease in F1 value are both within the set acceptable range, then the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction will be deleted. If the decrease in accuracy or the decrease in F1 value is not within the set acceptable range, the flawed feature with the smallest contribution in the flawed feature set after the initial dimensionality reduction cannot be deleted. Repeatedly delete the flawed features with the smallest contribution from the flawed feature set after the initial dimensionality reduction and verify them until a set of filtered surface features with the same number as the number of filtered surface feature targets is obtained.

5. The image recognition-based galvanized sheet surface defect classification method according to claim 1, characterized by, Determining the quality grade of galvanized steel sheets based on the screened surface feature set includes the following steps: Obtain the set of template surface features corresponding to each quality grade of galvanized sheet stored in the database; A similarity analysis is performed on the post-screening surface feature set and the template surface feature set to obtain a similarity value : ； in, Geometric features For template geometric features in the template surface feature set, For color characteristics, The template color feature is part of the template surface feature set. For frequency domain characteristics, For template frequency domain features in the template surface feature set, Gradient features The template gradient features are part of the template surface feature set. The cosine similarity function; Determine the set of template surface features corresponding to the highest similarity value, and then determine the quality grade of the galvanized sheet.

6. The method for classifying surface defects of galvanized steel sheets based on image recognition according to claim 1, characterized in that, The process of constructing the database for mapping production process-related parameters and weight correction coefficients is as follows: Different combinations of production process-related parameters are identified, and several sets of weight correction coefficients to be verified are assigned to each combination of production process-related parameters. For one combination of production process-related parameters, the selected surface feature set is weighted based on several sets of weight correction coefficients to be verified and the initial weighted data of surface defect features. This weighted data is then input into a defect pre-classification model built on support vector machines to obtain pre-classification evaluation indicators corresponding to each set of weight correction coefficients to be verified, including pre-classification accuracy. Pre-classification completion time CPU utilization during pre-classification ; Evaluation coefficients were calculated based on pre-classification evaluation indicators. : ； in, for Weighting factors for Weighting factors for Weighting factors; The weight correction coefficient corresponding to the evaluation coefficient with the largest value is associated with the combination of relevant parameters of the production process; Repeatedly calculate the evaluation coefficients corresponding to other combinations of production process related parameters, determine the weight correction coefficients to be verified for other combinations of production process related parameters, and construct a mapping database of production process related parameters and weight correction coefficients.

7. The image recognition-based galvanized sheet surface defect classification method according to claim 1, characterized by, The classification results are optimized and corrected based on the quality grade of galvanized steel sheets to obtain the corrected classification results, including the following steps: Obtain the quality grade-classification probability threshold mapping data set stored in the database, and determine the classification probability threshold based on the current galvanized sheet quality grade; If the maximum probability corresponding to the classification result is greater than the classification probability threshold, the classification result will not be optimized or corrected, and the classification result will be recorded as the corrected classification result. If the maximum probability corresponding to the classification result is not greater than the classification probability threshold, then the classification result is optimized and corrected. Retrieve the template surface feature set stored in the database. Each template surface feature set corresponds to a defect classification template result. A similarity analysis was performed on the filtered surface feature set and the template surface feature set to obtain similarity values; Determine the set of template surface features corresponding to the largest similarity value to determine the defect classification template result; If the classification result is consistent with the defect classification template result, the output classification result is the corrected classification result; if the classification result is inconsistent with the defect classification template result, the defect classification template result is the corrected classification result.

8. A galvanized steel sheet surface defect classification system based on image recognition, applied to the galvanized steel sheet surface defect classification method based on image recognition as described in any one of claims 1-7, characterized in that, include: The preprocessing module is used to acquire and preprocess the surface image data of the galvanized sheet to obtain the preprocessed surface image data. The feature processing module is used to perform feature processing on the preprocessed surface image data to obtain multidimensional surface feature data. The dimensionality reduction module is used to perform feature fusion and dimensionality reduction on multidimensional surface feature data to obtain a filtered set of surface features. The production process parameter and quality grade determination module is used to determine relevant production process parameters and galvanized sheet quality grades based on the screened surface feature set. The initial classification module is used to classify surface defects of galvanized steel sheets based on the filtered surface feature set and production process-related parameters, and obtain the classification results; The classification result correction module is used to optimize and correct the classification results based on the quality grade of galvanized steel sheets, and obtain the corrected classification results.

Images