A method for classifying surface defects in steel plates

Abstract

This invention relates to the field of steel plate surface defect classification. A method for classifying steel plate surface defects involves acquiring images of steel plate surface defects captured in industrial photography; performing image preprocessing followed by fusion and dimensionality reduction to form a 42-dimensional feature vector; establishing a KD-Tree; inputting the 42-dimensional feature vector into a TWSVM algorithm for training to obtain an optimal TWSVM model; outputting the trained TWSVM model in the form of probabilities, including the probability value of the defect type of each test sample; setting a probability threshold; and searching for the nearest neighbor samples in the KD-Tree for test samples with probability values ​​lower than the threshold for secondary classification, with the final result serving as the defect type for these test samples with probability values ​​below the threshold. This invention utilizes a dual classifier to maximize the classification effect of steel plate surface defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of steel plate surface defect classification. BACKGROUND

[0002] Steel is one of the most important materials in the world today, commonly used in building construction, rail transportation, parts manufacturing and other aspects, playing an irreplaceable role. Surface quality and other performance are important parameters for measuring quality, especially on the surface of steel plate. At present, there are various defects on the surface of steel plate produced in the industry, which will reduce the quality of the finished product. In order to improve the quality of steel plate, we often take different technical methods to process secondary grinding for various surface defects, and at the same time, the steel plate with large defects and difficult to repair is recycled, so the accurate classification of steel plate defects is particularly important. Today, in the process of steel plate industrial automation production, automatic production of products greatly reduces the risk coefficient faced by workers in the production line, and also improves the quality and speed of product processing. Similarly, in order to judge the type of steel plate surface defect faster and more accurately, a complete algorithm needs to be established for computer learning, so as to identify more accurately. Although deep learning is a very popular field in recent years, it needs a large number of samples for training and a large number of parameter operations, etc., which causes the long training time of the model. Therefore, compared with deep learning, traditional machine learning is more suitable for small sample learning and can train the model faster.

[0003] In the traditional classification method, KNN, SVM, etc. can be used for classification of steel plate surface type, but the traditional KNN classifier is difficult to process high-dimensional data in big data, and is easy to misjudge in the part with high overlap between different types, while the advantage of SVM is to consider the minimization of structural risk, which is suitable for small sample, nonlinear and high-dimensional classification problem, can effectively avoid overfitting, and has good generalization performance. Although the above traditional algorithms can be classified, for the prediction of the defect type of new samples, these classifiers will directly output a classification result, without outputting the confidence probability value of the predicted defect category, which leads to the inability to determine the credibility of the predicted result. Moreover, the classification credibility of a single classifier is not high, and the defect category cannot be predicted more accurately. SUMMARY

[0004] The technical problem to be solved by the present application is how to solve the problem of being unable to more accurately predict the defect category of the steel plate surface mentioned in the background.

[0005] The technical solution adopted by the present application is: a classification method for steel plate surface defects, comprising the following steps:

[0006] S1: obtaining the steel plate surface defect image shot by industry and dividing it into training set and test set;

[0007] S2: image preprocessing is performed on the image obtained in S1;

[0008] S3: HOG, GLCM features and geometric features of the preprocessed image are extracted, the HOG and GLCM features are subjected to fusion and dimension reduction processing, and then are fused with the geometric features to form a 42-dimensional feature vector;

[0009] S4: a KD-Tree is established using the 42-dimensional feature vector obtained in S3;

[0010] S5: the 42-dimensional feature vector obtained in S3 is input into a TWSVM algorithm to train an optimal TWSVM model;

[0011] S6: the trained TWSVM model in S5 is output in the form of probability, and the probability value of the test sample belonging to the defect category is output;

[0012] S7: a probability threshold is set, the test sample with a probability value less than the set probability threshold in S6 is searched for the nearest neighbor sample of the test sample in the KD-Tree established in S4 for secondary determination, and the final result will be the defect type to which the test sample with a probability value less than the set probability threshold belongs.

[0013] The steel plate surface defect images taken by the industry in step S1 are divided into 6 categories of defects, which are respectively crack (Cr), inclusion (In), patch (Pa), pitting (PS), rolled scale (RS) and scratch (Sc), and 80% of each category of images is divided into a training set and 20% is divided into a test set.

[0014] In step S2, since the camera is prone to produce noise and blur in the high temperature environment on site, the image needs to be subjected to gray scale transformation and noise removal after gray scale processing, so as to improve the image quality and make the display effect of the image clearer, so as to extract more useful features in the next step. The specific steps are as follows:

[0015] S21: first, the image is grayed;

[0016] S22: gray scale transformation is adopted to adjust the contrast of the image from the spatial domain to improve the visual effect.

[0017] The gray scale value f(μ,ν) of the pixel point coordinate (μ,v) of each image ranges from [a,b], and the gray scale value g(μ,ν) of the pixel point coordinate (μ,v) of the image after gray scale transformation ranges from [a',b'];

[0018] The relationship between f(μ,ν) and g(μ,ν) is as follows:

[0019]

[0020] a, b, a', b' are values greater than or equal to 0 and less than or equal to 255;

[0021] S23: Adopting Gabor filter to carry out denoising and smoothing processing on the image after the gray scale transformation.

[0022] The step S3 specifically comprises the following steps:

[0023] S31: Calculating the gray level co-occurrence matrix (GLCM) and the histogram of oriented gradients (HOG) of the image after the image preprocessing, to obtain the features of the image texture information and the information features of the local region gradient direction respectively;

[0024] S32: Fusing the features of the image texture information and the information features of the local region gradient direction, so that each image can be converted into a row of high-dimensional feature vectors, and finally each image obtains a total of 337-dimensional feature vectors. Since the dimensionality is too high after the feature fusion, it is easy to cause overfitting, and PCA is used for dimensionality reduction to 37 dimensions;

[0025] S33: When extracting the geometric features, the image after the image preprocessing is subjected to a binarization operation, the threshold value is used to divide the gray scale image into black and white, and the iterative method is used to select the threshold value. The specific steps are as follows:

[0026] S331: First, the maximum gray scale value and the minimum gray scale value of each image are calculated, and the average gray scale value of the maximum gray scale value and the minimum gray scale value is taken as the initial threshold value t;

[0027] S332: The initial threshold value t is used to divide the image into two parts R1 and R2, wherein R1 is the part with a gray scale value less than or equal to the threshold value t, and R2 is the part with a gray scale value greater than the threshold value t. The average gray scale values of R1 and R2 are calculated as t1 and t2 respectively;

[0028] S333: The new threshold value t new = (t1 + t2) / 2 is calculated;

[0029] S334: t and t new are compared, if they are equal, t new is the iterative threshold value; if they are not equal, t = t new , and S332-S333 are repeated until t == t new ;

[0030] S335: The gray scale image is binarized according to the threshold value obtained by iteration;

[0031] S34: Use morphological erosion to remove some small particles, then use morphological dilation to fill the voids in the defect target area and merge the broken defect target areas. Use the Canny operator to perform edge detection and extract the geometric features of the defect target area, including the number, defect area, area duty cycle, maximum defect area, and maximum area perimeter, for a total of 5-dimensional features.

[0032] S35: The features extracted in S32 and S34 are concatenated, resulting in a 42-dimensional feature vector for each image. To ensure significant comparability between different features, normalization is performed, mapping the data to the range [0,1] as follows:

[0033]

[0034] Where, x i Let x be the feature value of the i-th dimension in the current sample. min x is the smallest feature value in the current sample. max x is the largest feature value in the current sample. norm For x i The normalized value.

[0035] The S4 steps for constructing a KD-Tree are as follows:

[0036] S41: KD-Tree Description (Introduction): A KD-Tree resembles a binary tree. Each non-leaf node in a KD-Tree can be divided into two subspaces by a hyperplane, and each subspace can be partitioned in the same way. All subspaces are divided into two parts: a left subspace and a right subspace (or an upper subspace and a lower subspace). That is, constructing a KD-Tree on a K-dimensional dataset represents a partition of the K-dimensional space composed of the K-dimensional dataset. In other words, each node in the tree corresponds to a K-dimensional hyperrectangular region.

[0037] S42: After processing each image of surface defects on the steel plate in the training set according to steps S21-S35 above, each image will obtain a K-dimensional feature vector, which can be represented as... in Let be the feature value of the Kth dimension. Assume the training set T: T = {(x...} i ,y i )|x i ∈R K Let y = {i = 1, 2, ..., m}, where m is the sample size and y = ... i For data labels, R K It is a K-dimensional space;

[0038] S43: First, initialize the split axis: calculate the variance of each dimension corresponding to the m samples, then sort in descending order, select the dimension k with the largest variance, and the coordinate axis corresponding to the feature with the largest variance as the split axis;

[0039] S44: Sort the values of all sample data in the training set T in ascending order of k dimensions, and take the sample corresponding to the median as the split point, and at the same time as the KD-Tree root node. The root node is divided into two sub-regions by the hyperplane perpendicular to the coordinate axis of the split point;

[0040] S45: Take the root node as the reference to construct left and right child nodes with a depth of 1. The depth refers to the maximum layer number of all nodes, and the depth of the root node is 0. All sample points with the k-th feature less than the split point are classified as left child nodes, and those greater than the split point are classified as right child nodes;

[0041] S46: Repeat the above steps S44 and S45 to continue updating the split axis according to the variance. For the next depth h node, select x l as the split coordinate axis, where l is the dimension with the largest variance among the remaining features. Take the sample corresponding to the median of the l-th feature of all sample points in the region of this node as the split point, and continue to divide the sub-region to construct left and right child nodes with a depth of h+1. All sample points with the l-th feature less than the split point are classified as left child nodes, and those greater than the split point are classified as right child nodes;

[0042] S47: Traverse the K-dimensional features in order of variance from large to small. If not all are divided, continue to traverse until there are no sample points in the two sub-regions, and the construction of the KD-Tree is completed.

[0043] S5 specific steps are as follows:

[0044] TWSVM introduction: TWSVM and traditional SVM are both classic binary classification algorithms, but there are differences. TWSVM is actually composed of two hyperplanes, which aims to make each classification hyperplane as close as possible to one class of sample points and as far away as possible from another class of sample points, thereby maximizing the degree of separation between the two classes of samples.

[0045] S51: Considering that the steel plate surface defects are actually an n-class (multi-class) problem, here we adopt the "one-versus-all" strategy, taking one class of n defects as positive samples and the remaining n-1 classes as negative samples to construct n TWSVM binary classifiers. When classifying test samples, calculate the distance from the test sample to all hyperplanes, and the class corresponding to the hyperplane with the smallest distance is the class of the test sample;

[0046] ​S52: To construct each of the above TWSVM classifiers, first define a K-dimensional feature data training sample set T = {(x i ,y i )|x i ∈R K ,i=1,2,...,m}, where x i For each defect sample, because a "one-to-many" strategy is adopted, all defect samples of all categories will be divided into two classes, called positive samples and negative samples, respectively. At this time, y i ={+1,-1} are the sample labels, and then the training sample set T is divided into a matrix A of size m1×K and a matrix B of size m2×K, where m1+m2=m;

[0047] S52: To better determine the category of a sample, a kernel function is used to construct two hyperplanes K(x) in the TWSVM from the low-dimensional space to the high-dimensional Hilbert space. T C T u1+b1=0, K(x) T C T u² + b² = 0, where C = [A T B T ];

[0048] S53: Select the radial basis kernel function, which has the following form:

[0049]

[0050] Where, x i x j For the sample data, σ is a non-zero parameter;

[0051] S54: Establish the primal optimization problem of the nonlinear TWSVM:

[0052]

[0053] st-(K(B,C T u1+e2b1)+q2≥e2,q2≥0

[0054]

[0055] st(K(A,C T u² + e₁b² + q₁ ≥ e₁, q₁ ≥ 0

[0056] Where u1 and u2 are the normal vectors of the hyperplane, b1 and b2 are the offset vectors of the corresponding hyperplane, c1 and c2 are the penalty parameters, q1 and q2 are the slack variables, e1 and e2 are 1 vectors of appropriate dimensions, and K(,) is the kernel function.

[0057] S56: By introducing Lagrange multiplier vectors α1, α2, β1, β2 into the above two equations, and then establishing the Lagrange function:

[0058]

[0059]

[0060] S57: For the above optimization problem, according to the Kuhn-Takt conditions (KKT), its dual problem is obtained, and finally the two hyperplanes in S52 are determined. Let H = [K(A,C T )e1],G=[K(B,C T If )e2], then the dual problem of the primal optimization problem can be expressed as:

[0061]

[0062] st0≤α1≤c1e2

[0063]

[0064] st0≤α2≤c2e1

[0065] S61: After feature extraction, the test sample x of the steel plate surface defect image is input into the TWSVM model trained in S5. Its category can be determined by the TWSVM classification decision function, which is as follows:

[0066]

[0067] Among them, f λ (x) is the decision function, λ is the corresponding category, and u λ Let b be the normal vector of the hyperplane. λ Let K be the hyperplane offset vector and K(,) be the kernel function. After adopting the "one-to-many" strategy, for n multi-class problems, TWSVM will construct a hyperplane for each defect category. That is, for n classification problems, n hyperplanes will be constructed.

[0068] S62: Since the discriminant formula in S61 is a hard output, it can only reflect the class label to which the sample belongs, but does not reflect the degree of distance between the test sample and the hyperplane, which is not conducive to further subsequent processing. Therefore, by converting the distance into a soft output form of probability, the confidence of the sample's class can be reflected. The specific method is as follows:

[0069] Among them, P1…P n ∈[0,1]

[0070] The above formula reflects the reliable probability values ​​of the test sample being classified into n defect categories, namely P1, P2, ..., Pn The closer a test sample is to a hyperplane, the greater the confidence probability that it is classified as the defect type of that hyperplane.

[0071] S63: Based on the confidence probability values ​​of the test samples to each hyperplane, select the hyperplane with the larger confidence probability value as the confidence hyperplane, and output the corresponding confidence probability value P. xMax :

[0072] P xMax =argmax{P1,P2,…,P n}

[0073] The specific steps for searching for the nearest neighbor of a test sample in the KD-Tree in S7 are as follows:

[0074] S71: Set a probability threshold to distinguish the trustworthy sample set S trust and the set of untrusted samples S distrust The details are as follows:

[0075]

[0076] If the confidence probability value is greater than or equal to ε (0.92 is selected in this experiment), the test sample is judged to be a trustworthy sample, and the defect type of the test sample is directly determined. Otherwise, it is judged to be an untrustworthy sample, and the next step is to continue to judge it.

[0077] S72: Sequentially process the untrusted sample set S distrust KD-Tree nearest neighbor search was performed on the test samples.

[0078] S73: First, following the method used in S41, the dimensions corresponding to the largest to smallest variances are used as the dividing axes. distrust Each untrusted sample starts from the root node and recursively visits the KD-Tree downwards until the child node is a leaf node. This leaf node is then temporarily designated as the "current nearest point".

[0079] S74: Construct a hypersphere with the distance from the untrusted sample point to the "current nearest point" as the radius;

[0080] S75: Backtrack the parent node of the "current nearest point", check whether the hyperrectangular region contained in the other child node of the parent node intersects with the hypersphere. If they intersect, move to the other child node and continue the recursive search. If a closer neighbor is found, update the "current nearest point".

[0081] S76: If they do not intersect, backtrack upwards and find a child node that is closer to the "current nearest point", then update the current node to the "current nearest point";

[0082] S77: Repeat steps S75 and S76 until returning to the root node. The "current nearest point" updated last is the nearest neighbor.

[0083] S78: To S distrust For each untrusted sample, the nearest neighbor sample is found in the KD-Tree using the method described above;

[0084] The specific steps for the secondary determination in S7 are as follows:

[0085] S81: Compare the defect type of the nearest neighbor sample with the defect type already determined in S63;

[0086] S82: If the defect types are the same, output the defect type directly;

[0087] S83: If inconsistent, output the defect type corresponding to the nearest neighbor sample with a probability value of 1, and compare it with the corresponding defect type confidence probability value P in S6. l By summing and averaging, a new credible probability value P′ is obtained. l The formula is as follows:

[0088]

[0089] S84: Reorder the confidence probability values ​​from high to low and select the highest confidence probability value;

[0090] S85: Based on the defect type corresponding to the maximum confidence probability value, a second determination is made of the defect type to which the untrusted sample belongs, and the determined type is used as the final determination.

[0091] The beneficial effects of this invention are as follows: TWSVM is a binary classification algorithm based on SVM, which can reduce training time while ensuring classification accuracy and has good generalization performance. This invention utilizes the speed advantage of the TWSVM algorithm to classify defects on the surface of steel plates and outputs the TWSVM classification results in the form of probabilities. Simultaneously, it combines KD-Tree to continue searching for nearest neighbor samples for test samples with probabilities below a set threshold. Because nearest neighbor samples have high reliability, the probabilities are weighted accordingly. Finally, by sorting the probability values, the maximum probability is selected to determine the defect type of the test samples with probabilities below the threshold. This method can maximize classification accuracy and solve the problems caused by single classifiers, providing good support for subsequent processing.

[0092] This invention provides a method for classifying surface defects of steel plates. This method overcomes the shortcomings of existing methods that use a single classifier to classify surface defects of steel plates, such as misjudgment and low accuracy, by utilizing a dual classifier to maximize the classification effect of surface defects. The steps are as follows: S1 Acquire images of surface defects of steel plates captured in industrial photography; S2 Perform image preprocessing on the images acquired in S1; S3 Extract HOG, GLCM, and geometric features from the preprocessed images, and use them as feature vectors after dimensionality reduction; S4 Build a KD-Tree using the feature vectors obtained in S3; S5 Input the feature vectors obtained in S3 into a TWSVM algorithm for training to obtain the optimal model; S6 Output the trained TWSVM model from S5 in the form of probabilities, and output the probability value of the defect type to which the test sample belongs, taking the defect type corresponding to the maximum probability value as the predicted defect type; S7 Search for the nearest neighbor samples of test samples whose maximum probability value in S6 is less than a set probability threshold in the KD-Tree built in S4. S8 compares the defect type corresponding to the nearest neighbor sample found in S7 with the defect type corresponding to the highest probability value in S6. If they match, the defect type is directly output. If they do not match, the defect type corresponding to the nearest neighbor sample is weighted and summed with the probability predicted in S6, and the defect type corresponding to the highest probability value of all defect types is taken to finally determine the defect type of these test samples with a probability below the set threshold. This improves the overall classification accuracy. Attached Figure Description

[0093] Figure 1 This is a flowchart illustrating the present invention;

[0094] Figure 2 This is a schematic diagram of TWSVM binary classification;

[0095] Figure 3 This is a schematic diagram of the KD-Tree creation process;

[0096] Figure 4 This is a schematic diagram of the KD-Tree nearest neighbor search process. Detailed Implementation

[0097] A method for classifying surface defects in steel plates includes the following steps:

[0098] S1: Acquire images of surface defects on steel plates captured in industrial photography and divide them into training and testing sets;

[0099] The acquired industrial images of steel plate surface defects were categorized into six types: cracks (Cr), inclusions (In), patches (Pa), pitting (PS), indented scale (RS), and scratches (Sc). 300 images were collected for each type, totaling 1800 sample images. 80% of these images for each type were assigned to the training set, and 20% to the test set.

[0100] S2: Perform image preprocessing on the image acquired in S1;

[0101] Because cameras are prone to noise and blurring in high-temperature environments, we need to perform grayscale transformation and noise removal on the image after grayscale conversion to improve image quality and make the image display clearer, so as to extract more useful features in the next step. The specific steps are as follows:

[0102] S21: First, convert the image to grayscale;

[0103] S22: Use grayscale transformation to adjust image contrast in the spatial domain to improve visual effects.

[0104] The grayscale value f(μ,v) of the pixel coordinates (μ,ν) of each image is in the range of [a,b], and the grayscale value g(μ,ν) of the pixel coordinates (μ,v) of the image after grayscale transformation is in the range of [a',b'].

[0105] The relationship between f(μ,v) and g(μ,ν) is as follows:

[0106]

[0107] a, b, a', b' are values ​​greater than or equal to 0 and less than or equal to 255;

[0108] S23: Gabor filtering is used to denoise and smooth the image after grayscale transformation.

[0109] S3: Extract HOG, GLCM and geometric features from the preprocessed image. After fusion and dimensionality reduction of the HOG and GLCM features, they are fused together with the geometric features to form a 42-dimensional feature vector.

[0110] S31: Calculate the gray-level co-occurrence matrix (GLCM) and histogram of oriented gradients (HOG) for the image after image preprocessing to obtain features of image texture information and information features of gradient direction in local regions, respectively;

[0111] S32: The features of image texture information and the features of local gradient direction are fused so that each image can be transformed into a row of high-dimensional feature vectors. Finally, each image obtains a total of 337-dimensional feature vectors. Since the dimension is too high after feature fusion, it is easy to cause overfitting. PCA is used to reduce the dimension to 37.

[0112] S33: During geometric feature extraction, the image after image preprocessing is binarized, and the grayscale image is segmented into black and white using a threshold. The threshold is selected using an iterative method. The specific steps are as follows:

[0113] S331 first calculates the maximum and minimum gray values ​​for each image, and then takes the average gray value of the maximum and minimum gray values ​​as the initial threshold t;

[0114] S332 divides the image into two parts R1 and R2 using an initial threshold t, where R1 is the part with gray values ​​less than or equal to the threshold t, and R2 is the part with gray values ​​greater than the threshold t. The average gray values ​​of R1 and R2 are calculated as t1 and t2, respectively.

[0115] S333 calculates the new threshold t new = (t1+t2) / 2;

[0116] S334 compares t and t new If they are equal, then t new The iteration threshold is used; if they are not equal, then t = t_{n+1}. new Repeat S332-S333 until t==t new ;

[0117] S335 binarizes the grayscale image based on the threshold obtained through iteration;

[0118] S34: Use morphological erosion to remove some small particles, then use morphological dilation to fill the voids in the defect target area and merge the broken defect target areas. Use the Canny operator to perform edge detection and extract the geometric features of the defect target area, including the number, defect area, area duty cycle, maximum defect area, and maximum area perimeter, for a total of 5-dimensional features.

[0119] S35: The features extracted in S32 and S34 are concatenated, resulting in a 42-dimensional feature vector for each image. To ensure significant comparability between different features, normalization is performed, mapping the data to the range [0,1] as follows:

[0120]

[0121] Where, x i Let x be the feature value of the i-th dimension in the current sample.min x is the smallest feature value in the current sample. max x is the largest feature value in the current sample. norm For x i The normalized value.

[0122] S4: Construct a KD-Tree using the 42-dimensional feature vectors obtained in S3;

[0123] S41: KD-Tree Description (Introduction): A KD-Tree resembles a binary tree. Each non-leaf node in a KD-Tree can be divided into two subspaces by a hyperplane, and each subspace can be partitioned in the same way. All subspaces are divided into two parts: a left subspace and a right subspace (or an upper subspace and a lower subspace). That is, constructing a KD-Tree on a K-dimensional dataset represents a partition of the K-dimensional space composed of the K-dimensional dataset. In other words, each node in the tree corresponds to a K-dimensional hyperrectangular region.

[0124] S42: After processing each image of surface defects on the steel plate in the training set according to steps S21-S35 above, each image will obtain a K-dimensional feature vector, which can be represented as... in Let be the feature value of the Kth dimension. Assume the training set T: T = {(x...} i ,y i )|x i ∈R K Let y = {i = 1, 2, ..., m}, where m is the sample size and y = ... i For data labels, R K It is a K-dimensional space;

[0125] S43: First, initialize the splitting axes: calculate the variance of each dimension for the m samples, then sort them in descending order, select the dimension k with the largest variance, and... The coordinate axis corresponding to the feature with the largest variance is used as the dividing axis;

[0126] S44: Sort all k-dimensional values ​​of the samples in the training set T in ascending order, using the sample corresponding to the median as the split point, and also using this sample as the root node of the KD-Tree. The root node is divided into two sub-regions by using a hyperplane perpendicular to the coordinate axis.

[0127] S45: Based on the root node, construct left and right child nodes with a depth of 1. The depth refers to the maximum number of levels of all nodes. The depth at the root node is 0. All sample points whose k-th dimension features are less than the split point are classified as left child nodes, and those that are greater than the split point are classified as right child nodes.

[0128] S46: Repeat steps S44 and S45 above, continuing to update the splitting axis according to the variance. For nodes at the next depth h, select x. l Let l be the coordinate axis for segmentation, where l is the dimension with the largest variance among the remaining features. The median sample of the l-th dimension of all sample points in the node region is used as the segmentation point. Sub-regions are further divided, and left and right child nodes with a depth of h+1 are constructed. All sample points whose l-th dimension of features is less than the segmentation point are classified as left child nodes, and those whose l-th dimension of features is greater than the segmentation point are classified as right child nodes.

[0129] S47: Iterate through the K-dimensional features in descending order of variance. If the features are not completely divided, continue iterating until there are no sample points in two sub-regions. This completes the construction of the KD-Tree.

[0130] S5: Input the 42-dimensional feature vector obtained in S3 into the TWSVM algorithm to train and obtain the optimal TWSVM model;

[0131] Introduction to TWSVM: Although both TWSVM and traditional SVM are classic binary classification algorithms, they differ in that TWSVM actually consists of two hyperplanes. The goal is to make each classification hyperplane as close as possible to one class of sample points and as far away as possible from the other class of sample points, thereby maximizing the separation of the two classes of samples.

[0132] S51: Considering that steel plate surface defects are actually an n-class (multi-class) problem, a "one-to-many" strategy is adopted here. One class of defects out of the n classes is used as a positive sample, and the remaining n-1 classes are used as negative samples, constructing n TWSVM binary classifiers. When the test sample is classified, the distance to all hyperplanes is calculated, and the class corresponding to the hyperplane with the smallest distance is the class of the test sample.

[0133] S52: To construct each of the above TWSVM classifiers, first define a K-dimensional feature data training sample set T = {(x i ,y i )|x i ∈R K ,i=1,2,...,m}, where x i For each defect sample, because a "one-to-many" strategy is adopted, all defect samples of all categories will be divided into two classes, called positive samples and negative samples, respectively. At this time, y i ={+1,-1} are the sample labels, and then the training sample set T is divided into a matrix A of size m1×K and a matrix B of size m2×K, where m1+m2=m;

[0134] S52: To better determine the category of a sample, a kernel function is used to construct two hyperplanes K(x) in the TWSVM from the low-dimensional space to the high-dimensional Hilbert space. T CT u1+b1=0, K(x) T C T u² + b² = 0, where C = [A T B T ];

[0135] S53: Select the radial basis kernel function, which has the following form:

[0136]

[0137] Where, x i x j For the sample data, σ is a non-zero parameter;

[0138] S54: Establish the primal optimization problem of the nonlinear TWSVM:

[0139]

[0140] st-(K(B,C T u1+e2b1)+q2≥e2,q2≥0

[0141]

[0142] st(K(A,C T u² + e₁b² + q₁ ≥ e₁, q₁ ≥ 0

[0143] Where u1 and u2 are the normal vectors of the hyperplane, b1 and b2 are the offsets of the corresponding hyperplane, c1 and c2 are the penalty parameters, q1 and q2 are the slack variables, e1 and e2 are 1 vectors of appropriate dimensions, and K(,) is the kernel function.

[0144] S56: By introducing Lagrange multiplier vectors α1, α2, β1, β2 into the above two equations, and then establishing the Lagrange function:

[0145]

[0146]

[0147] S57: For the above optimization problem, according to the Kuhn-Takt conditions (KKT), its dual problem is obtained, and finally the two hyperplanes in S52 are determined. Let H = [K(A,C T )e1],G=[K(B,C T If )e2], then the dual problem of the primal optimization problem can be expressed as:

[0148]

[0149] st0≤α1≤c1e2

[0150]

[0151] st0≤α2≤c2e1

[0152] S6: Output the TWSVM model trained in S5 in the form of probability, and output the probability value of the defect type to which the test sample belongs;

[0153] S61: After feature extraction, the test sample x of the steel plate surface defect image is input into the TWSVM model trained in S5. Its category can be determined by the TWSVM classification decision function, which is as follows:

[0154]

[0155] Where fλ(x) is the decision function, λ is the corresponding category, and u λ Let b be the normal vector of the hyperplane. λ Let K be the hyperplane offset vector and K(,) be the kernel function. After adopting the "one-to-many" strategy, for n multi-class problems, TWSVM will construct a hyperplane for each defect category. That is, for n classification problems, n hyperplanes will be constructed.

[0156] S62: Since the discriminant formula in S61 is a hard output, it can only reflect the class label to which the sample belongs, but does not reflect the degree of distance between the test sample and the hyperplane, which is not conducive to further subsequent processing. Therefore, by converting the distance into a soft output form of probability, the confidence of the sample's class can be reflected. The specific method is as follows:

[0157] Among them, P1…P n ∈[0,1]

[0158] The above formula reflects the reliable probability values ​​of the test sample being classified into n defect categories, namely P1, P2, ..., P n The closer a test sample is to a hyperplane, the greater the confidence probability that it will be classified as the defect type of that hyperplane.

[0159] S63: Based on the confidence probability values ​​of the test samples to each hyperplane, select the hyperplane with the larger confidence probability value as the confidence hyperplane, and output the corresponding confidence probability value P. xMax :

[0160] P xMax =argmax{P1,P2,…,P n}

[0161] The steel plate surface defect images collected in this invention are divided into 6 types of defects: cracks (Cr), inclusions (In), patches (Pa), pits (PS), indented oxide scale (RS), and scratches (Sc). By training the model, six hyperplanes corresponding to the defect types are finally obtained. Twelve test samples are randomly selected, and the confidence probability value of each test sample to the six hyperplanes and the corresponding defect type are output, as shown in Table 1.

[0162]

[0163] S7: Set a probability threshold. For test samples whose probability values ​​in S6 are less than the set probability threshold, search for the nearest neighbor samples in the KD-Tree established in S4 for secondary judgment. The final result will be used as the defect type of these test samples with probability values ​​below the set probability threshold.

[0164] S71: Set a probability threshold to distinguish the trustworthy sample set S trust and the set of untrusted samples S distrust The details are as follows:

[0165]

[0166] If the confidence probability value is greater than or equal to ε (0.92 is selected in this experiment), the test sample is judged to be a trustworthy sample, and the defect type of the test sample is directly determined. Otherwise, it is judged to be an untrustworthy sample, and the next step is to continue to judge it.

[0167] S72: Sequentially process the untrusted sample set S distrust KD-Tree nearest neighbor search was performed on the test samples.

[0168] S73: First, following the method used in S41, the dimensions corresponding to the largest to smallest variances are used as the dividing axes. distrust Each untrusted sample starts from the root node and recursively visits the KD-Tree downwards until the child node is a leaf node. This leaf node is then temporarily designated as the "current nearest point".

[0169] S74: Construct a hypersphere with the distance from the untrusted sample point to the "current nearest point" as the radius;

[0170] S75: Backtrack the parent node of the "current nearest point", check whether the hyperrectangular region contained in the other child node of the parent node intersects with the hypersphere. If they intersect, move to the other child node and continue the recursive search. If a closer neighbor is found, update the "current nearest point".

[0171] S76: If they do not intersect, backtrack upwards and find a child node that is closer to the "current nearest point", then update the current node to the "current nearest point";

[0172] S77: Repeat steps S75 and S76 until returning to the root node. The "current nearest point" updated last is the nearest neighbor.

[0173] S78: To S distrust Each untrusted sample finds its nearest neighbor in the KD-Tree using the method described above.

[0174] For the test samples below the probability threshold in Table 1, after searching for nearest neighbors using the KD-Tree method, the final judgment results obtained using this invention are shown in Table 2 below:

[0175]

[0176] S81: Compare the defect type of the nearest neighbor sample with the defect type already determined in S63;

[0177] S82: If the defect types are the same, output the defect type directly;

[0178] S83: If inconsistent, output the defect type corresponding to the nearest neighbor sample with a probability value of 1, and compare it with the corresponding defect type confidence probability value P in S6. l By summing and averaging, a new credible probability value P′ is obtained. l The formula is as follows:

[0179]

[0180] S84: Reorder the confidence probability values ​​from high to low and select the highest confidence probability value;

[0181] S85: Based on the defect type corresponding to the maximum confidence probability value, a second determination is made of the defect type to which the untrusted sample belongs, and the determined type is used as the final determination.

[0182] Table 3 shows the actual defect types corresponding to the test samples in Tables 1 and 2.

[0183]

[0184] This method overcomes the shortcomings of existing methods that use a single classifier to classify steel plate surface defects, such as misjudgment and low accuracy. It maximizes the classification effect of steel plate surface defects by using a dual classifier.

Claims

1. A method of classifying surface defects of a steel sheet, characterized by: The steps include the following: S1: obtaining the steel plate surface defect image shot by industry and dividing into training set and test set; S2: image preprocessing is performed on the image obtained in S1; S3: HOG, GLCM features and geometric features of the preprocessed image are extracted, the HOG and GLCM features are subjected to fusion and dimension reduction processing, and then are fused with the geometric features to form a 42-dimensional feature vector; S4: a KD-Tree is established by using the 42-dimensional feature vector obtained in S3; S5: the 42-dimensional feature vector obtained in S3 is input into the TWSVM algorithm to train an optimal TWSVM model; S6: the trained TWSVM model in S5 is output in the form of probability, and the reliable probability value of the test sample belonging to the defect type is output; S7: a probability threshold is set, the test sample with a reliable probability value less than the set probability threshold is searched for the nearest neighbor sample of the test sample in the KD-Tree established in S4 for secondary determination, and the final result is taken as the defect type to which the test sample with the set probability threshold is belonged; the specific steps of the secondary determination are as follows: S81: the defect type to which the nearest neighbor sample is belonged is compared with the defect type determined in S6; S82: if the defect types are consistent, the defect type is directly output; S83: If not consistent, the defect type corresponding to the nearest neighbor sample is output with a probability value of 1, and the corresponding defect type confidence probability value P in S6 l Sum average is performed to obtain a new confidence probability value P' l , as follows S84: the reliable probability values are reordered from high to low, and the maximum reliable probability value is selected; S85: the defect type to which the untrusted sample is belonged is secondarily determined according to the defect type to which the maximum reliable probability value corresponds, and the determined type is taken as the final determination.

2. A method of classifying surface defects of a steel sheet according to claim 1, characterized by: The steel plate surface defect image shot by industry obtained in the step S1 is divided into six defect types, which are crack, inclusion, patch, pitting, pressed oxide skin and scratch, and 80% of the images of each type are divided into training set and 20% are divided into test set.

3. The method of claim 1, wherein: The step S2 includes the following steps: S21: the image is first grayed; S22: the image contrast is adjusted from the spatial domain to improve the visual effect by using gray scale transformation; The gray scale value f(μ,v) of the pixel point coordinate (μ,v) of each image ranges from [a,b], and the gray scale value g(μ,v) of the pixel point coordinate (μ,v) of the image after gray scale transformation ranges from [a',b']; The relationship between f(μ,v) and g(μ,v) is as follows: a, b, a', b' are values greater than or equal to 0 and less than or equal to 255; S23: the image after gray scale transformation is subjected to denoising and smoothing processing by using Gabor filtering.

4. The method of claim 1, wherein: The step S3 specifically includes the following: S31: the gray level co-occurrence matrix (GLCM) and the histogram of oriented gradients (HOG) are calculated for the image after image preprocessing, and the features of the image texture information and the information features of the local region gradient direction are obtained; S32: the features of the image texture information and the information features of the local region gradient direction are fused, so that each image can be converted into a high-dimensional feature vector, and finally a total of 337-dimensional feature vectors are obtained for each image, and the dimension is reduced to 37 dimensions by using PCA due to the high dimension after feature fusion which is prone to overfitting; S33: When extracting geometric features, the image preprocessed image is binarized, the threshold is used to separate the gray image into black and white, and the threshold is selected by iteration. The specific steps are as follows: S331: First, the maximum gray value and the minimum gray value of each image are calculated, and the average gray value of the maximum gray value and the minimum gray value is taken as the initial threshold t; S332: According to the threshold t, the image is divided into two parts R1 and R2, wherein R1 is the part with gray value less than or equal to the threshold t, and R2 is the part with gray value greater than the threshold t. The average gray values of R1 and R2 are t1 and t2 respectively; S333 calculate new threshold t new = (t1 + t2) / 2; S334 compare t and t new , if equal, then t new is the iteration threshold; if not equal, then t = t new , repeat S332-S333 until t == t new ; S335: The threshold obtained by iteration is used to binarize the gray image; S34: Some small particle points are removed by using morphological erosion operation, and then morphological dilation operation is used to fill the holes in the defect target area and merge the broken defect target areas. Edge detection is performed by using Canny operator to extract the geometric features of the defect target area, including number, defect area, area ratio, maximum defect area, and maximum area perimeter, a total of 5-dimensional features; S35: The features extracted in S32 and S34 are spliced, and each image finally obtains a 42-dimensional feature vector. In order to make different features have significant comparability, normalization is performed to map the data between [0, 1] as follows: where x i is the i-th dimension feature value in the current sample, x min is the minimum feature value in the current sample, x max is the maximum feature value in the current sample, x norm is x i the normalized value.

5. The method of claim 1, wherein: S4: The specific steps of constructing KD-Tree are as follows: S41: KD-Tree is like a binary tree. Each non-leaf node of KD-Tree can be divided into two subspaces by a hyperplane, and each subspace can also be divided in the same way. All subspaces are divided into two parts, left subspace and right subspace. A KD-Tree is constructed on a K-dimensional data set to represent the partitioning of a K-dimensional space composed of a K-dimensional data set. That is, each node in the tree corresponds to a K-dimensional hyper-rectangular region; S42: Each steel plate surface defect picture in the training set will obtain a K-dimensional feature vector, which can be expressed as wherein represents the Kth eigenvalue Suppose a training set T: T = {(x i ,y i )|x i ∈R K ,i = 1, 2, …, m}, where m is the number of samples, y i is the data label, and R K is a K-dimensional space; S43: First, initialize the split axis: calculate the variance of each dimension corresponding to the m samples, then sort in descending order, select the dimension k with the largest variance, and the coordinate axis corresponding to the feature with the largest variance as the split axis; S44: Sort the values of the k-dimensional of all sample data in the training set T in ascending order, and take the sample corresponding to the median as the split point, which is also the root node of the KD-Tree. The root node is divided into two sub-regions by the hyperplane perpendicular to the coordinate axis of the split point. S44: Sort the values of the k-dimensional of all sample data in the training set T in ascending order, and take the sample corresponding to the median as the split point, which is also the root node of the KD-Tree. The root node is divided into two sub-regions by the hyperplane perpendicular to the coordinate axis of the split point. S45: Taking the root node as the reference, left and right child nodes with depth 1 are constructed. The depth refers to the maximum layer number of all nodes, and the depth of the root node is 0. All sample points with the k-th feature less than the split point are classified as left child nodes, and those greater than the split point are classified as right child nodes; S46: repeat the above steps S44, S45, continue to update the split axis according to the variance size, and select x for the node of the next depth h l Split the coordinate axis, where l is the dimension with the largest variance in the remaining features, and the sample corresponding to the median of the lthdimensional feature of all sample points in the node region is taken as the split point. Continue to divide the sub-region and construct the left and right child nodes with a depth of h+1. The lthdimensional features of all sample points less than the split point are assigned to the left child node, and the lthdimensional features greater than the split point are assigned to the right child node. S47: The K-dimensional features are traversed from large to small according to variance. If it is not divided, continue to traverse until there is no sample point in the two sub-regions, and the construction of KD-Tree is completed.

6. A method of classifying surface defects of a steel sheet according to claim 5, characterized in that: S5: The specific steps are as follows: S51: Considering that the steel plate surface defect is actually an n-classification problem, the "one-to-many" strategy is adopted, one class of n classes of defects is taken as the positive sample, and the remaining n-1 classes are taken as the negative sample, and n TWSVM binary classifiers are constructed. When the test sample is classified, the distance to all hyperplanes is calculated, and the class corresponding to the smallest distance is the class of the test sample; S52: To construct each of the above TWSVM classifiers, first define a K-dimensional feature data training sample set T = {(x i ,y i )|x i ∈R K ,i = 1, 2, …, m}, where x i is each defect sample, because the "one-versus-all" strategy is adopted, all defect samples of the current class are divided into two classes, respectively called positive samples and negative samples, at this time y i = {+1, -1} is the sample label, and then divide the training sample set T into a matrix A of size m1 x K and a matrix B of size m2 x K, where m1 + m2 = m; S52: In order to better distinguish the class to which the sample belongs, the kernel function is used to construct two hyperplanes K(x T ,C T )u1+b1=0, K(x T ,C T )u2+b2=0, where C=[A T B T ]; S53: Select the radial basis kernel function, which has the following form: where x i , x j are sample data, and σ is a non-zero parameter. S54: The original optimization problem of nonlinear TWSVM is established: Wherein, u1, u2 are normal vectors of hyperplane, b1, b2 are offset vectors of corresponding hyperplane, c1, c2 are penalty parameters, q1, q2 are relaxation variables, e1, e2 are 1 vectors of appropriate dimension, K(,) is kernel function; S56: Introducing Lagrange multiplier vectors a1, a2, b1, b2 into the above two equations, then establishing a Lagrange function: S57: For the above optimization problem, according to the Kuhn-Tucker condition (K.K.T), the dual problem is solved to finally determine the two hyperplanes in S52, let H = [K(A, C T )e1], G = [K(B, C T )e2], then the dual problem of the original optimization problem can be expressed as: 。 7. A method of classifying surface defects of a steel sheet according to claim 6, characterized in that: S6 is specifically as follows: S61: after the steel plate surface defect image test sample x is extracted, it is input to the TWSVM model trained in S5, and its category can be judged by TWSVM classification decision function, and the function is specifically as follows: where f λ (x) is the decision function, λ is the corresponding class, u λ is the hyperplane normal vector, b λ is the hyperplane offset vector, K(,) is the kernel function, and after adopting the "one-versus-all" strategy, for n multi-classification problems, TWSVM will construct a hyperplane for each defect class, that is, n hyperplanes will be constructed for n classification problems. S62: since the judgment formula in S61 is hard output, only the class label to which the sample belongs can be reflected, and the degree of distance between the test sample and the hyperplane cannot be reflected, which is not conducive to further subsequent processing, so the distance is converted into the soft output form of probability to reflect the credibility of the class to which the sample belongs, and the specific method is as follows: wherein P1...P n ∈ [0, 1]; The above formula reflects the reliable probability value of the test sample being judged as n defect categories, respectively P1, P2, …, Pn. n The closer the test sample is to which hyperplane, the greater the reliable probability value of the test sample being judged as the defect type to which the corresponding hyperplane belongs. S63: According to the trusted probability value of the test sample to each hyperplane, select the one with a large trusted probability value as the trusted hyperplane, and output the corresponding trusted probability value P xMax : P xMax = argmax {P1, P2,..., P n}.

8. A method of classifying surface defects of a steel sheet according to claim 7, characterized in that: The specific steps of searching for the nearest neighbor sample of the test sample in the KD-Tree in S7 are as follows: S71: Set a probability threshold to distinguish between the trusted sample set S trust and the untrusted sample set S distrust , as follows: If the credibility probability value is greater than or equal to ε, the test sample is judged as a reliable sample, and the defect type to which the test sample belongs is directly determined, otherwise it is judged as an unreliable sample, and the next step is continued to determine it; S72: Perform KD-Tree nearest neighbor search on test samples in the distrust sample set S distrust in sequence; S73: Firstly, according to the method corresponding to S41, the dimension corresponding to the variance from large to small is taken as the segmentation axis, and S distrust Each untrusted sample starts from the root node and recursively accesses the KD-Tree downward until the child node is a leaf node, and the leaf node is temporarily defined as the "current nearest point". S74: a hypersphere is constructed with the distance between the unreliable sample point and the "current nearest point" as the radius; S75: the parent node of the "current nearest point" is backtracked, and it is checked whether the hyperrectangular region contained by the other child node of the parent node intersects with the hypersphere, if the hyperrectangular region intersects with the hypersphere, the other child node is moved to, and the search is continued recursively, if a nearer neighbor point appears, the "current nearest point" is updated; S76: if the hyperrectangular region does not intersect with the hypersphere, the parent node is backtracked upwards, if a child node closer than the "current nearest point" appears, the current node is updated as the "current nearest point"; S77: steps S75 and S76 are repeated until the root node is returned, and the "current nearest point" updated last time is the nearest neighbor point; S78: To S distrust Each untrusted sample finds its nearest neighbor in the KD-Tree by the above method.

Images