Device surface defect identification method, system, electronic device and medium

Abstract

The present invention discloses a method, system, electronic device, and medium for identifying equipment surface defects, and relates to the field of defect identification technology. The method comprises constructing an original data set; augmenting and histogram equalizing the original data set to obtain an enhanced data set; processing the enhanced data set using the SURF algorithm to obtain a feature vector set for each image; performing K-means clustering on all feature vectors in the feature vector set to obtain clustering results; obtaining a reduced-dimensional feature vector using a feature bag model based on the clustering results; reconstructing the reduced-dimensional feature vector based on the number of feature points in each image to obtain a reconstructed feature vector; and training an SVM model using the Adaboost algorithm based on the reconstructed feature vector and the defect category to obtain a typical equipment surface defect identification model, thereby identifying equipment surface defects. The present invention can improve the accuracy and efficiency of the results of identifying small defects on the equipment surface.

Description

Technical Field

[0001] The present invention relates to the field of defect recognition technology, and in particular to a device surface defect recognition method, system, electronic equipment and medium. Background Art

[0002] For large equipment, surface defects and failures caused by collision, vibration, impact, corrosion, etc. are quite common. In addition to causing mission termination and production suspension due to equipment downtime, surface defects and failures may also pose a huge safety hazard. In addition, large and complex equipment has numerous maintenance interfaces and complex maintenance processes. Traditional maintenance methods require high skill levels from maintenance personnel, and maintenance efficiency and quality cannot be guaranteed. During target detection and maintenance, there are often problems such as pipes and other accessories blocking each other, insufficient light, personnel fatigue and human omissions. These all seriously affect the effectiveness and efficiency of maintenance operations, making it even more difficult to detect surface defects on equipment, especially small defects such as scratches and cracks.

[0003] Augmented reality-assisted maintenance, also known as augmented maintenance, uses AR technology to assist and support maintenance operations such as assembly, repair, and scheduled inspections. The application of augmented maintenance primarily relies on computer vision technologies for target recognition and positioning. Developing detection methods for common defects on metal surfaces of equipment using image processing and target recognition technologies, suitable for AR devices, can effectively assist visual inspections during scheduled inspections, improving the efficiency and effectiveness of target detection.

[0004] Currently, most computer vision-based surface defect detection methods are almost all offline detection, which has a lag. Moreover, this target recognition and positioning technology can never be integrated into the scope of augmented reality technology. Because augmented reality requires virtual information to be superimposed on the field of view in real time, a certain degree of real-time performance must be guaranteed. However, improving the real-time performance of the method will inevitably affect the accuracy.

[0005] Current object detection technologies can be divided into machine learning-based and deep learning-based methods based on network depth. Deep learning-based methods are currently the most popular. However, due to the generally high complexity of deep learning models, such as Region CNN and Fast R-CNN, their computational efficiency simply cannot meet the real-time requirements of augmented reality. One-stage deep learning methods such as YOLO and the SSD series of algorithms also fail to meet the efficiency and accuracy requirements of equipment object detection. Furthermore, regardless of the method, the accuracy and efficiency of current object detection technology for identifying small surface defects (such as cracks and scratches) on equipment still need to be improved. Summary of the Invention

[0006] The purpose of the present invention is to provide a method, system, electronic device and medium for identifying defects on the surface of equipment, which can improve the accuracy and efficiency of the results of identifying small defects on the surface of equipment.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A method for identifying surface defects of equipment, comprising:

[0009] Constructing an original data set; the original data set includes a defect image set of various equipment surfaces; the defect image set includes images of various defect categories;

[0010] Augmenting the original data set to obtain an augmented data set;

[0011] Performing histogram equalization on the augmented data set to obtain an enhanced data set;

[0012] Using the SURF algorithm to process each image in the enhanced data set to obtain a feature vector set for each image in the enhanced data set;

[0013] Performing K-means clustering on all feature vectors in the feature vector set of all images in the enhanced dataset to obtain a clustering result;

[0014] Obtaining a reduced-dimensional feature vector of each image in the enhanced dataset using a bag-of-features model according to the clustering results;

[0015] Reconstructing the feature vectors of each image in the enhanced dataset after dimensionality reduction according to the number of feature points of each image in the enhanced dataset to obtain a reconstructed feature vector of each image in the enhanced dataset;

[0016] According to the reconstructed feature vectors of all images in the enhanced data set and the defect categories of all images in the enhanced data set, the Adaboost algorithm is used to train the SVM model to obtain an equipment typical surface defect recognition model, and the equipment typical surface defect recognition model is used to identify equipment surface defects.

[0017] Optionally, obtaining the feature vector of each image in the enhanced dataset after dimensionality reduction using a bag-of-features model according to the clustering result specifically includes:

[0018] For any image in the enhanced dataset, determining, according to the clustering result, a ratio of feature points of each cluster category in the image to all feature points in the image;

[0019] A set number of ratios are selected from the ratios of all feature points in the image to form a feature vector of the image after dimensionality reduction.

[0020] Optionally, reconstructing the feature vectors of each image in the enhanced dataset after dimensionality reduction based on the number of feature points of each image in the enhanced dataset to obtain the reconstructed feature vectors of each image in the enhanced dataset specifically includes:

[0021] For any image in the enhanced data set, obtaining a reconstructed feature value of the image according to the number of feature points of the image, the maximum number of feature points of all images, and the minimum number of feature points of all images;

[0022] The reconstructed eigenvalue of the image is added to the eigenvector of the image after dimensionality reduction to obtain the reconstructed eigenvector of the image.

[0023] Optionally, the Adaboost algorithm is used to train the SVM model based on the reconstructed feature vectors of all images in the enhanced dataset and the defect categories of all images in the enhanced dataset to obtain a typical surface defect recognition model for equipment, specifically including:

[0024] Taking the reconstructed feature vectors of the images in the enhanced dataset as input and the defect categories of the images in the enhanced dataset as output, the Adaboost algorithm is used to train the SVM model to obtain multiple SVM weak classifiers;

[0025] All SVM weak classifiers are processed using linear weighting method to obtain strong classifiers;

[0026] The strong classifier is determined to be the typical surface defect recognition model of the equipment.

[0027] A device surface defect recognition system, comprising:

[0028] An original data set construction module is used to construct an original data set; the original data set includes a defect image set of various equipment surfaces; the defect image set includes images of various defect categories;

[0029] An augmentation module, configured to augment the original data set to obtain an augmented data set;

[0030] A histogram equalization processing module, configured to perform histogram equalization on the augmented data set to obtain an enhanced data set;

[0031] a feature vector determination module, configured to process each image in the enhanced dataset using a SURF algorithm to obtain a feature vector set for each image in the enhanced dataset;

[0032] A clustering module, configured to perform K-means clustering on all feature vectors in the feature vector set of all images in the enhanced dataset to obtain a clustering result;

[0033] A feature vector dimensionality reduction module, configured to obtain a feature vector of each image in the enhanced dataset after dimensionality reduction using a bag of features model according to the clustering results;

[0034] A feature vector reconstruction module, configured to reconstruct the feature vectors of each image in the enhanced dataset after dimensionality reduction according to the number of feature points of each image in the enhanced dataset, to obtain a reconstructed feature vector for each image in the enhanced dataset;

[0035] The defect recognition module is used to train the SVM model using the Adaboost algorithm based on the reconstructed feature vectors of all images in the enhanced data set and the defect categories of all images in the enhanced data set to obtain a typical surface defect recognition model for equipment. The typical surface defect recognition model for equipment is used to identify surface defects of equipment.

[0036] Optionally, the feature vector dimensionality reduction module specifically includes:

[0037] a feature point ratio calculation unit, configured to determine, for any image in the enhanced data set, a ratio of feature points of each cluster category in the image to all feature points in the image according to the clustering result;

[0038] The feature vector dimension reduction unit is used to select a set number of ratios from the ratios of all feature points in the image to form the feature vector of the image after dimension reduction.

[0039] Optionally, the feature vector reconstruction module specifically includes:

[0040] a reconstruction feature value calculation unit, configured to obtain, for any image in the enhanced data set, a reconstruction feature value of the image according to the number of feature points of the image, the maximum number of feature points of all images, and the minimum number of feature points of all images;

[0041] The feature vector reconstruction unit is used to add the reconstructed feature value of the image to the feature vector of the image after dimensionality reduction to obtain the reconstructed feature vector of the image.

[0042] Optionally, the defect identification module specifically includes:

[0043] A weak classifier training unit is configured to use the reconstructed feature vectors of the images in the enhanced dataset as input and the defect categories of the images in the enhanced dataset as output, and to train the SVM model using an Adaboost algorithm to obtain a plurality of SVM weak classifiers;

[0044] A strong classifier determination unit is used to process all SVM weak classifiers using a linear weighting method to obtain a strong classifier;

[0045] A model building unit is used to determine that the strong classifier is the typical surface defect recognition model of the equipment.

[0046] An electronic device, comprising:

[0047] A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the device surface defect identification method described above.

[0048] A computer-readable storage medium stores a computer program, which implements the above-mentioned device surface defect identification method when executed by a processor.

[0049] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention obtains multiple defect types, expands and enhances the original data set, increases the training set, makes the model obtained in the later training more accurate, improves the accuracy of the defect recognition results, adopts the SURF feature extraction algorithm, and uses the K-means + feature bag model method to process the data to improve the efficiency of defect recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flow chart of a method for identifying surface defects of equipment provided by an embodiment of the present invention;

[0052] Figure 2 This is an example diagram of typical defects on the metal surface of equipment;

[0053] Figure 3 This is an example diagram of the expansion of typical defect data on equipment metal surfaces;

[0054] Figure 4 This is an example of enhanced image of typical defects on metal surfaces;

[0055] Figure 5 Schematic diagram for comparing adjacent pixels;

[0056] Figure 6 Schematic diagram of the SURF feature vector description process;

[0057] Figure 7 Schematic diagram of feature dimensionality reduction and reconstruction process;

[0058] Figure 8 This is the relationship between the number of cluster categories K and the defect recognition accuracy;

[0059] Figure 9 This is a comparison chart of the original image and the image after histogram equalization;

[0060] Figure 10 This is a comparison chart of the original image grayscale histogram and the histogram equalized grayscale histogram. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying surface defects of a device, comprising:

[0064] Step 101: constructing an original data set; the original data set includes a defect image set of various equipment surfaces; the defect image set includes images of various defect categories.

[0065] Step 102: augment the original data set to obtain an augmented data set.

[0066] Step 103: Perform histogram equalization processing on the augmented data set to obtain an enhanced data set.

[0067] Step 104: Using the SURF algorithm to process each image in the enhanced dataset to obtain a feature vector set for each image in the enhanced dataset.

[0068] Step 105: Perform K-means clustering on all feature vectors in the feature vector sets of all images in the enhanced dataset to obtain clustering results.

[0069] Step 106: Using the bag-of-features model according to the clustering results, obtain the feature vector of each image in the enhanced dataset after dimensionality reduction.

[0070] Step 107: reconstructing the reduced-dimensional feature vectors of each image in the enhanced dataset according to the number of feature points of each image in the enhanced dataset to obtain a reconstructed feature vector of each image in the enhanced dataset.

[0071] Step 108: Based on the reconstructed feature vectors of all images in the enhanced dataset and the defect categories of all images in the enhanced dataset, the Adaboost algorithm is used to train the SVM model to obtain an equipment typical surface defect recognition model, and the equipment typical surface defect recognition model is used to identify equipment surface defects.

[0072] In practical applications, for the original data set in step 101, each image is rotated (randomly rotated 90°, 180°, or 270° clockwise), mirrored, and scaled (reduced to 0.5 times the original size and enlarged to 2 times the original size) to obtain an augmented data set.

[0073] In practical applications, the method of obtaining the reduced-dimensional feature vector of each image in the enhanced dataset using the bag-of-features model according to the clustering results specifically includes:

[0074] For any image in the enhanced data set, the ratio of feature points of each cluster category in the image to all feature points in the image is determined according to the clustering result.

[0075] A set number of ratios of the feature points of each cluster category in the image to all the feature points in the image are selected to form a feature vector of the image after dimensionality reduction.

[0076] In practical applications, the process of reconstructing the feature vectors of each image in the enhanced dataset after dimensionality reduction based on the number of feature points of each image in the enhanced dataset to obtain the reconstructed feature vectors of each image in the enhanced dataset specifically includes:

[0077] For any image in the enhanced data set, a reconstructed feature value of the image is obtained according to the number of feature points of the image, the maximum number of feature points of all images, and the minimum number of feature points of all images.

[0078] The reconstructed eigenvalue of the image is added to the eigenvector of the image after dimensionality reduction to obtain the reconstructed eigenvector of the image.

[0079] In practical applications, the Adaboost algorithm is used to train the SVM model based on the reconstructed feature vectors of all images in the enhanced dataset and the defect categories of all images in the enhanced dataset to obtain a typical surface defect recognition model for equipment, specifically including:

[0080] The reconstructed feature vectors of the images in the enhanced data set are used as input, the defect categories of the images in the enhanced data set are used as output, and the Adaboost algorithm is used to train the SVM model to obtain multiple SVM weak classifiers.

[0081] All SVM weak classifiers are processed using linear weighting method to obtain strong classifiers.

[0082] The strong classifier is determined to be the typical surface defect recognition model of the equipment.

[0083] The present invention also provides a more specific embodiment to introduce the above method in detail:

[0084] Step 1: Equipment typical surface defect analysis and image acquisition

[0085] Hot-rolled strip steel is mainly used as the main blank in the production of metal skins for various equipment in the aerospace, shipbuilding, automobile, electromechanical and chemical industries. The typical surface defects of hot-rolled strip steel during manufacturing, transportation and use can be divided into six types: cracks, inclusions, plaques, surface pitting, rolling oxidation and scratches. Figure 2 As shown, Figure 2 (a) is cracking, Figure 2 (b) is the inclusion, Figure 2 (c) is a plaque, Figure 2 (d) is surface pitting, Figure 2 (e) is rolling oxidation, Figure 2 (f) is a scratch.

[0086] The embodiment of the present invention first screened out images of the above six defect types from the steel image database of Northeastern University in the United States, and used a camera to collect image data of skin defects of equipment such as aircraft and ships, completing the collection of image data of typical surface defects of equipment, thereby obtaining the original data set D1.

[0087] Step 2: Equipment typical surface defect image data augmentation

[0088] When developing surface defect recognition methods for equipment, we often face the situation of having few training samples. Therefore, it is necessary to perform data augmentation on the original dataset D1 constructed in step 1. On the one hand, this can increase the size of the dataset and avoid overfitting. On the other hand, it can enrich the sample morphology in the training set and improve the robustness of the method.

[0089] For the original data set, each image is rotated, mirrored and scaled to achieve augmentation and obtain the augmented data set D2. The effects of some data after processing are as follows: Figure 3 As shown, Figure 3 (a) is the original image with the defect type being cracking. Figure 3(b) for Figure 3 (a) The image obtained by rotation, Figure 3 (c) for Figure 3 (a) The image obtained by mirroring, Figure 3 (d) for Figure 3 (a) The image obtained by magnification, Figure 3 (e) for Figure 3 (a) The image obtained by shrinking, Figure 3 (f) is the original image with the defect type being inclusions. Figure 3 (g) for Figure 3 (f) The image obtained by rotation, Figure 3 (h) is for Figure 3 (f) The image obtained by mirroring, Figure 3 (i) For Figure 3 (f) The image obtained by magnification, Figure 3 (j) for Figure 3 (f) The image obtained by shrinking, Figure 3 (k) is the original image with the defect type of plaque. Figure 3 (I) for Figure 3 (k) The image obtained by rotation, Figure 3 (m) is for Figure 3 (k) The image obtained by mirroring, Figure 3 (n) for Figure 3 (k) The image obtained by magnification, Figure 3 (o) is for Figure 3 (k) The image obtained by shrinking, Figure 3 (p) is the original image with the defect type being surface pitting. Figure 3 (q) is for Figure 3 (p) is the image obtained by rotation, Figure 3 (r) is for Figure 3 (p) is the image obtained by mirroring, Figure 3 (s) is for Figure 3 (p) is the image obtained by magnification, Figure 3 (t) is for Figure 3 (p) is the image obtained by shrinking, Figure 3 (w) is the original image with the defect type being rolling oxidation. Figure 3 (v) for Figure 3 (w) is the image obtained by rotating, Figure 3 (u) is for Figure 3 (w) is the image obtained by mirroring, Figure 3 (x) is the right Figure 3 (w) is the image obtained by magnification, Figure 3 (y) is the Figure 3 (w) is the image obtained by shrinking, Figure 3(z) is the original image with the defect type being scratches. Figure 3 (A) for Figure 3 (z) is the image obtained by rotating, Figure 3 (B) for Figure 3 (z) is the image obtained by mirroring, Figure 3 (D) for Figure 3 (z) is the image obtained by magnification, Figure 3 (E) is for Figure 3 (z) The image obtained by zooming out.

[0090] (1) Rotation

[0091] Each image is randomly rotated 90°, 180°, or 270° clockwise. Rotation primarily changes the orientation and position of the defect within the image, so the rotation operation also includes some translation effects. Equipment surface defects grow in an undirected manner, and equipment surface defect images are collected in random orientations. Therefore, rotation increases sample richness while ensuring image authenticity.

[0092] (2) Mirror

[0093] Each image is mirrored, i.e., it is symmetrical twice about its horizontal and vertical axes. Mirroring also changes the position and orientation of equipment defects in the image, and can be used as a supplement to the rotation operation to increase the data set capacity.

[0094] (3) Zoom

[0095] Each image is enlarged to twice its original size and reduced to 0.5 times its original size, thereby changing the scale information of the target in the image. For some surface defects of equipment, changing their scale can produce a certain degree of confusion. For example, if the inclusions are enlarged, they will be very similar to plaques, which is very helpful for improving the robustness of the defect recognition method.

[0096] Step 3: Enhance the image of typical surface defects

[0097] In order to improve the recognition effect of equipment surface defect targets, the histogram equalization method is used for image enhancement processing, that is, the image histogram is stretched and adjusted to adjust the image contrast so that the grayscale is evenly distributed in the entire grayscale range. The histogram equalization processing is performed on the augmented dataset D2 to obtain the enhanced dataset D3. Some images are as follows Figure 4 As shown, Figure 4 (a) Figure 4 (b) and Figure 4 (c) shows three examples of plaque patterns. Figure 4 (A) Figure 4 (B) and Figure 4 (C) for Figure 4 (a) Figure 4(b) and Figure 4 (c) Examples of the three patch patterns after their histograms are processed. Figure 4 (d) Figure 4 (e) and Figure 4 (f) shows three examples of crack patterns. Figure 4 (D) Figure 4 (E) and Figure 4 (F) is for Figure 4 (d) Figure 4 (e) and Figure 4 (f) Examples of three crack patterns after histogram processing. Figure 4 (g) Figure 4 (h) and Figure 4 (i) are three pictures of the contents. Figure 4 (G) Figure 4 (H) and Figure 4 (I) For Figure 4 (g) Figure 4 (h) and Figure 4 (i) Examples of three inclusion patterns after their histogram processing. Figure 4 (j) Figure 4 (k) and Figure 4 (m) shows three examples of scratch patterns. Figure 4 (J) Figure 4 (K) and Figure 4 (M) is for Figure 4 (j) Figure 4 (k) and Figure 4 (m) Examples of three scratch patterns after histogram processing. Figure 4 (n), Figure 4 (o) and Figure 4 (p) shows three examples of surface pitting. Figure 4 (N), Figure 4 (O) and Figure 4 (P) is for Figure 4 (n), Figure 4 (o) and Figure 4 (p) Examples of three surface pitting patterns after histogram processing. Figure 4 (q), Figure 4 (r) and Figure 4 (s) are three examples of rolled oxidation patterns. Figure 4 (Q), Figure 4 (R) and Figure 4 (S) is for Figure 4 (q), Figure 4 (r) and Figure 4 (s) Examples of three rolled oxidation patterns after histogram processing.

[0098] The overall contrast of defective images processed by histogram equalization is often not affected, and the contrast of the target area (the area with defects) is enhanced. The processed image uses the difference in brightness to make the layers of different objects in the image (defects and background) clearer and more intuitive.

[0099] Step 4: Preliminary extraction of typical surface defect features of equipment

[0100] For the augmented dataset D3, we extracted SURF (speedup robust features) to describe image feature vectors that can be read and processed by computers. SURF feature vectors are scale-invariant and rotation-invariant, resulting in excellent robustness. Regarding real-time performance, the SURF feature extraction process (feature detection and description) and the size of the feature vector ensure that SURF features meet real-time requirements (frames per second ≥ 30) within the processing power of current augmented reality devices. The SURF feature extraction steps are as follows.

[0101] Step 4.1: Feature Detection

[0102] Step 4.1.1: Gaussian filtering

[0103] For each image in the enhanced dataset D3, the image is first Gaussian filtered using a Gaussian filter function to lay the foundation for making the feature points scale invariant.

[0104] Step 4.1.2: Construct the Hessian matrix

[0105] For the filtered image, the Hessian matrix of the image is constructed. The purpose is to detect feature points using the Hessian matrix discriminant. The feature points detected using the Hessian matrix are more robust and also lay a good foundation for the subsequent feature extraction stage. The Hessian matrix of an image is a square matrix composed of the second-order partial derivatives of the pixel point and the adjacent pixels. SURF calculates the second-order partial derivatives through convolution between specific kernels to obtain the Hessian matrix, denoted as H, see formula (1), where D xx Represents the second-order partial derivative of the pixel in the x direction, D yy Represents the second-order partial derivative of the pixel in the y direction, D xy It represents the second-order partial derivative of the pixel in the x and y directions. A Hessian matrix can be calculated for each pixel.

[0106]

[0107] Step 4.1.3: Construct scale space

[0108] The SURF feature extraction algorithm constructs a scale space by building an image pyramid for each image, ensuring that the detected feature points are scale-invariant. The scale pyramid construction process is as follows: First, the scale space is divided into multiple levels, and box filters of varying sizes are placed in each level. These filters are used to filter the image, resulting in images of varying scales. During this process, the image size remains constant; only the size of the box filter templates varies across levels. This makes the SURF extraction algorithm highly efficient when constructing the scale space.

[0109] Step 4.1.4: Preliminary determination of feature points using non-maximum suppression

[0110] For the Hessian matrix constructed for each pixel in step 4.1.2, in the scale space, the feature point Det(H) is determined using the feature point discriminant (also called feature point responsiveness). The discriminant of the feature point is shown in formula (2). The discriminant is the determinant of the Hessian matrix. When the discriminant of a certain pixel point obtains a local maximum, it is considered that the pixel point is more prominent than other points in its neighborhood. Specifically, the local maximum judgment method is to compare each pixel point processed by the Hessian matrix with 26 points in its image space and scale space neighborhood that have also been processed by the Hessian matrix (8 pixels in the neighborhood of the pixel point in the plane of the current layer, 9 pixels in the neighborhood of the upper plane, and 9 pixels in the neighborhood of the lower plane, for a total of 26 points), as shown in the following example: Figure 5 As shown, the key points are preliminarily located.

[0111] Det(H)=D xx D yy -(D xy ) 2 (2)

[0112] Step 4.1.5: Screening stable feature points

[0113] In order to ensure that the feature points are valid and robust, for the key points initially determined in step 4.1.4, the feature point response threshold is set to filter out feature points with weak response and feature points with incorrect positioning, so as to obtain a stable feature point set for each image.

[0114] Step 4.2: Feature Description

[0115] Step 4.2.1: Assignment of main directions of feature points

[0116] For each feature point in the feature point set, the main direction of the feature point is assigned to make the extracted feature vector rotation invariant. Specifically, a 60° fan-shaped neighborhood is first constructed within the circular neighborhood of the feature point, and the sum of the horizontal and vertical Haar wavelet response values ​​of all points in the fan-shaped neighborhood is counted, and a vector is calculated, which contains the response value size m w and direction θ w The calculation formulas are shown in formula (3) and formula (4) respectively. Where dx is the horizontal Haar wavelet response value of each feature point, dy is the vertical Haar wavelet response value of each feature point, and w represents the fan-shaped neighborhood.

[0117]

[0118]

[0119] Then, the sector is rotated at a certain interval (generally the step size is 0.2 radians) and the Harr wavelet response value in the neighborhood of the sector is counted again in the above manner. Finally, the direction indicated by the sector with the largest response value is taken as the main direction of the feature point, see formula (5), so that the feature has scale invariance.

[0120] θ=θ w |max{m w} (5)

[0121] Step 4.2.2: Generate feature vectors

[0122] For a certain feature point, first rotate the coordinate axis to make it consistent with the main direction. Then select a square area with a side length of 20 pixels according to the main direction, and then divide the area into 4×4 grids. In each grid, count the sum of the horizontal and vertical Haar wavelet response values ​​of the pixels contained. The sum of the Haar wavelet response includes the sum of the horizontal direction value, the sum of the vertical direction value, the sum of the horizontal absolute value, and the sum of the vertical absolute value of the Haar wavelet, such as Figure 6 The above four values ​​are extracted as part of the feature vector of the feature point, so that each feature point can be described as a 64-dimensional (4×4×4=64-dimensional) feature vector f.

[0123] Based on the above feature detection and feature description methods, SURF features are extracted for each image in the enhanced dataset D3. The capacity of the feature vector set extracted from the j-th image in the enhanced dataset D3 is the “number of feature points” of the image (denoted as N j ), then the feature dimension of the image is N j ×64; let the data capacity (number of images) in the enhanced dataset D3 be M, let the set of feature vectors extracted from all images in the enhanced dataset D3 be called D4, then the data dimension in D4 is

[0124] Step 5: Reconstruction of typical surface defect features

[0125] Although the SURF algorithm is very good in terms of real-time performance, each feature point still corresponds to a 64-dimensional feature vector, and each image has hundreds or even thousands of feature points. Therefore, the amount of data is very large, which is not friendly to subsequent calculations. Directly inputting it into the classifier for defect recognition usually cannot meet the real-time requirements. Therefore, the SURF features are reduced in dimension and reconstructed. This step is based on a data dimensionality reduction method that can specify the data dimension. It uses K-means clustering + feature bag model to reduce the dimension of the feature vector, and introduces the dimensionless number of feature points of each image into the reduced feature vector, thereby re-describing the feature vector. The feature reconstruction process is as follows: Figure 8 The specific steps are as follows.

[0126] Step 5.1: Cluster D4 using the K-means method to obtain K class centroids and use these K class centroids as the feature dictionary.

[0127] Input: D4, number of categories K, that is, the data set is expected to be clustered into K sets; the center of the K clusters, or called the centroid (the number of cluster categories K is obtained through multiple pre-training. The F measure, a comprehensive evaluation index of the recognition accuracy of six types of metal surface defects, is calculated and the mean is calculated. The relationship between the average F measure and the number of cluster categories K is observed, as shown in Figure 2. Figure 8 As shown, it is found that the average F-measure reaches its maximum value when K = 7. In fact, when K is 4 to 10, the average F-measure remains at a high level. K can be selected within this range according to actual conditions.

[0128] Step 5.1.1: For each eigenvector in D4, calculate its distance to each centroid and compare them, and classify it into the class corresponding to the centroid closest to it.

[0129] Step 5.1.2: Recalculate the centroid of each class.

[0130] Step 5.1.3: If the position change between the recalculated center of mass and the original center of mass is not large (usually a threshold is set to judge), stop the algorithm. If the new center of mass deviates greatly from the original center of mass, return to step 5.1.1.

[0131] Output: The class label of each feature vector (feature point), the class centroid, and the distance between each feature vector and the centroid of the class to which it belongs.

[0132] Step 5.2: For each image in D3, use the feature bag idea to count the distribution of all feature points of the image in the K classes in D4, that is, calculate the ratio of the number of feature points in each class to all feature points in the entire image, and arrange the K ratio values ​​in a certain order to construct the reduced-dimensional feature vector f'=(f1',f2',...,f' K ), the set of feature vectors of all images after dimensionality reduction is recorded as D5. The elements in the feature vector f' are calculated as shown in the following formula.

[0133]

[0134] Among them, f′ i is the value of the i-th dimension in the feature vector f', N j is the number of feature points of the jth image; N ji is the number of feature points in the jth image that are classified into the i-th category, and M represents the total number of images in the enhanced dataset.

[0135] After D4 is processed by K-means+feature bag model, the feature vector of each image is replaced by N j The (number of feature points in the jth image) × 64 dimension is reduced to K dimension, and most of the feature information of the image is still retained. Figure 7 As shown, the above processing is also performed on the image to be identified, and its original feature vector is subjected to feature bag statistical processing in the clustered feature vector set to obtain the reduced K-dimensional feature vector f' of the image to be identified. Subsequently, the reduced K-dimensional feature vector f' of the image to be identified is reconstructed to obtain a reconstructed feature vector, and the reconstructed feature vector is input into the following equipment surface defect recognition model for defect recognition.

[0136] Step 5.3: Since the number of feature points of each type of defect also has a positive value in improving the accuracy of defect recognition, and the feature vector f' does not use this information, for each feature vector f' in D5 (at this time, each image only corresponds to one K-dimensional feature vector), the number of feature points of the image is introduced into f', that is, the number of feature points of the image after dimensionless addition is added after the K-th dimension data, and the reconstructed K+1-dimensional feature vector of the image is obtained. K+1 ), where f” i =f' i , i = 1, 2, ..., K. The reconstructed feature vectors of all images are called D6.

[0137] The purpose of dimensionless transformation is to eliminate the dimension of the K+1th dimension and the first K dimensions of the feature vector. Otherwise, the last dimension of the feature vector may become the decisive factor in the defect recognition result. The dimensionless transformation method for the K+1th dimension of the feature vector is shown in the following formula.

[0138]

[0139] Among them, f” K+1 is the K+1th dimension data of the reconstructed feature vector, N j is the number of feature points of the jth image in the enhanced dataset D3, N max N is the maximum number of feature points in each image in the enhanced dataset D3. min is the minimum number of feature points in each image in the enhanced dataset D3. For the feature vector of the image to be identified after dimensionality reduction, the value of the K+1th dimension is calculated according to the above formula and concatenated to the original feature vector to reconstruct the feature vector f".

[0140] Step 6: Build a typical surface defect recognition model for equipment based on ensemble learning strategy

[0141] For D6, the adaptive boosting (Adaboost) learning strategy in the ensemble learning method is used, a one-vs-one (OVO) training method is adopted, and the support vector machine (SVM) is used as the basic classifier to train the equipment typical surface defect recognition model.

[0142] This process involves repeated learning, prediction, and updating of sample weights. Changing the weight distribution of the training data is based on the principle of "paying attention to misclassified samples." Specifically, when constructing the next training set, the weights of misclassified samples should be increased, while the weights of correctly classified samples should be decreased. The reweighted dataset is used to train the next weak classifier, resulting in a series of weak classifiers. Subsequently, each weak classifier is weighted based on its misclassification rate to output a strong classifier for that class. The weak classifiers are combined using a weighted majority voting method, where their weights are determined by their misclassification rates. Weak classifiers with low misclassification rates are assigned larger weights, giving them a greater influence in the voting. Conversely, weak classifiers with high misclassification rates should have their weights reduced to minimize their misclassification rates.

[0143] Specifically, As training samples, where x i It is used to describe the reconstructed feature vector of the i-th image in the enhanced dataset D3, y iIt represents the number of the surface defect category to which the i-th image belongs. A series of SVM weak classifiers are trained using the Adaboost learning strategy, and a strong classifier is obtained using the linear weighting method, that is, the equipment typical surface defect recognition model G(x), as shown below

[0144]

[0145] Where sign represents a sign function, which is used to output the predicted label, f(x) represents a function obtained by linearly weighting a series of weak classifiers, where x represents a reconstructed feature vector, and T represents the number of training rounds or the number of weak classifiers; α t represents the weight of the tth weak classifier; G t (x) represents the tth weak classifier.

[0146] The present invention is aimed at typical defects on the metal surface of equipment (cracks, plaques, inclusions, pitting surface, scratches and rolling oxidation), and proposes a method for equipment surface defect recognition based on SURF features and adaptive boosting learning. First, according to the appearance characteristics of typical surface defects of equipment, the data set is augmented by rotation, scaling (reduction and enlargement) and mirroring image data augmentation methods to solve the problem of lack of equipment surface defect data. Subsequently, histogram equalization is used to highlight the characteristics of typical surface defects of equipment to improve image quality and highlight the appearance characteristics of defects, thereby improving the accuracy of defect recognition. Afterwards, a feature extraction and dimensionality reduction method of SURF+K-means+feature bag model is proposed, which can convert high-dimensional feature vectors into low-dimensional feature vectors in a specified dimension, thereby improving computational efficiency while ensuring the robustness of the method; on the basis of this feature vector, the dimensionless feature point number information of each image is introduced to reconstruct the feature vector to further improve the accuracy of defect recognition. Finally, support vector machines are used to train in an adaptive boosting learning framework to achieve defect recognition. Machine learning-based target recognition methods are generally efficient and easy to optimize, and can be optimized from the image processing, feature extraction, target classification and other machine learning image recognition technology stages.

[0147] The present invention also provides a device surface defect recognition system corresponding to the above method, characterized in that it includes:

[0148] The original data set construction module is used to construct an original data set; the original data set includes a defect image set of various equipment surfaces; the defect image set includes images of various defect categories.

[0149] The augmentation module is used to augment the original data set to obtain an augmented data set.

[0150] The histogram equalization processing module is used to perform histogram equalization processing on the augmented data set to obtain an enhanced data set.

[0151] The feature vector determination module is used to process each image in the enhanced data set using the SURF algorithm to obtain a feature vector set for each image in the enhanced data set.

[0152] The clustering module is used to perform K-means clustering on all feature vectors in the feature vector set of all images in the enhanced data set to obtain a clustering result.

[0153] The feature vector dimensionality reduction module is used to obtain the reduced dimensionality feature vector of each image in the enhanced data set using the feature bag model according to the clustering result.

[0154] The feature vector reconstruction module is used to reconstruct the feature vector of each image in the enhanced data set after dimensionality reduction according to the number of feature points of each image in the enhanced data set to obtain the reconstructed feature vector of each image in the enhanced data set.

[0155] The defect recognition module is used to train the SVM model using the Adaboost algorithm based on the reconstructed feature vectors of all images in the enhanced data set and the defect categories of all images in the enhanced data set to obtain a typical surface defect recognition model for equipment. The typical surface defect recognition model for equipment is used to identify surface defects of equipment.

[0156] As an optional implementation, the feature vector dimensionality reduction module specifically includes:

[0157] A feature point ratio calculation unit is used to determine, for any image in the enhanced data set, a ratio of feature points of each cluster category in the image to all feature points in the image according to the clustering result.

[0158] The feature vector dimension reduction unit is used to select a set number of ratios from the ratios of all feature points in the image to form the feature vector of the image after dimension reduction.

[0159] As an optional implementation, the feature vector reconstruction module specifically includes:

[0160] The reconstructed feature value calculation unit is used to obtain the reconstructed feature value of any image in the enhanced data set according to the number of feature points of the image, the maximum number of feature points of all images, and the minimum number of feature points of all images.

[0161] The feature vector reconstruction unit is used to add the reconstructed feature value of the image to the feature vector of the image after dimensionality reduction to obtain the reconstructed feature vector of the image.

[0162] As an optional implementation, the defect identification module specifically includes:

[0163] The weak classifier training unit is used to use the reconstructed feature vector of the image in the enhanced data set as input and the defect category of the image in the enhanced data set as output, and to train the SVM model using the Adaboost algorithm to obtain multiple SVM weak classifiers.

[0164] The strong classifier determination unit is used to process all SVM weak classifiers using a linear weighting method to obtain a strong classifier.

[0165] A model building unit is used to determine that the strong classifier is the typical surface defect recognition model of the equipment.

[0166] An embodiment of the present invention further provides an electronic device, including:

[0167] A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the device surface defect identification method according to the above embodiment.

[0168] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the device surface defect identification method as described in the above embodiment.

[0169] The present invention has the following advantages:

[0170] (1) The present invention performs image data augmentation on the original data set, combines the appearance characteristics of 6 typical surface defect types of equipment (cracks, inclusions, plaques, surface pitting, rolling oxidation and scratches) and the effects of specific image data augmentation methods, and analyzes the image data augmentation method that is most suitable for equipment surface defects. After comparative analysis, it is found that the use of rotation, mirroring and scaling methods can basically cover the possible patterns of various defects, and the data set capacity has been sufficiently expanded; histogram equalization can enhance the contrast of the target area without changing the overall contrast, and the processed image uses the difference between brightness to make the layers of different objects in the image clearer and more intuitive. Taking scratches as an example, the effects before and after image processing and the corresponding grayscale histograms are shown as follows: Figure 9 and Figure 10 As shown, Figure 9 (a) is the original image, Figure 9 (b) is the image after histogram equalization of the original image. Figure 10 (a) is the grayscale histogram of the original image, Figure 10(b) is the grayscale histogram of the original image after histogram equalization. It can be seen that the grayscale of the original scratch image is very concentrated, and the contrast between the scratch and the background is not very obvious. After histogram equalization, the grayscale is evenly dispersed throughout the entire grayscale range, and the scratch in the middle is very obvious. During the initial feature extraction, the SURF algorithm was used to extract high-quality features with rotation invariance and scale invariance, laying a data foundation for training and generating a surface defect recognition model with high recognition accuracy. The number of feature points in each image was introduced to reconstruct the feature vector. It has been observed that the number of feature points is important information reflecting the defect category, so introducing this information can improve defect recognition accuracy. The Adaboost learning strategy was used during model training, integrating a series of weak classifiers to generate a strong classifier, which significantly improved the model's recognition accuracy for typical surface defects of equipment and improved recognition accuracy.

[0171] (2) The present invention uses the SURF feature extraction algorithm to perform preliminary feature extraction. The SURF algorithm has a very fast feature point detection speed, and the dimension of the extracted feature vector has certain advantages over most feature extraction methods. The K-means + feature bag model method is used to further reduce the dimension and reconstruct the SURF feature vector, so that the feature vector of each image is reduced from the previous N j ×64 dimensions are reduced to K dimensions, which greatly shortens the computing time for equipment surface defect recognition model training and practical application, and improves recognition efficiency.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0173] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A method for identifying surface defects of equipment, characterized in that: include: Construct the original dataset; The original data set includes a defect image set of various equipment surfaces; The defect image set includes images of various defect categories; Augmenting the original data set to obtain an augmented data set; Performing histogram equalization on the augmented data set to obtain an enhanced data set; Using the SURF algorithm to process each image in the enhanced data set to obtain a feature vector set for each image in the enhanced data set; Performing K-means clustering on all feature vectors in the feature vector set of all images in the enhanced dataset to obtain a clustering result; Obtaining a reduced-dimensional feature vector of each image in the enhanced dataset using a bag-of-features model according to the clustering results; Reconstructing the feature vectors of each image in the enhanced dataset after dimensionality reduction according to the number of feature points of each image in the enhanced dataset to obtain a reconstructed feature vector of each image in the enhanced dataset; According to the reconstructed feature vectors of all images in the enhanced data set and the defect categories of all images in the enhanced data set, the SVM model is trained using the Adaboost algorithm to obtain an equipment typical surface defect recognition model, and the equipment typical surface defect recognition model is used to identify equipment surface defects; Reconstructing the feature vectors of each image in the enhanced dataset after dimensionality reduction based on the number of feature points of each image in the enhanced dataset to obtain the reconstructed feature vectors of each image in the enhanced dataset specifically includes: For any image in the enhanced data set, a reconstructed feature value of the image is obtained according to the number of feature points of the image, the maximum number of feature points of all images, and the minimum number of feature points of all images; specifically, the number of feature points of the image is dimensionlessly converted according to the maximum number of feature points of all images and the minimum number of feature points of all images to obtain the reconstructed feature value of the image; The reconstructed eigenvalue of the image is added to the eigenvector of the image after dimensionality reduction to obtain the reconstructed eigenvector of the image.

2. The device surface defect identification method according to claim 1, characterized in that: The step of obtaining the reduced-dimensional feature vector of each image in the enhanced dataset using the bag-of-features model according to the clustering result specifically includes: For any image in the enhanced dataset, determining, according to the clustering result, a ratio of feature points of each cluster category in the image to all feature points in the image; A set number of ratios are selected from the ratios of all feature points in the image to form a feature vector of the image after dimensionality reduction.

3. The device surface defect identification method according to claim 1, characterized in that: The Adaboost algorithm is used to train the SVM model based on the reconstructed feature vectors of all images in the enhanced dataset and the defect categories of all images in the enhanced dataset to obtain a typical surface defect recognition model for equipment, specifically including: Taking the reconstructed feature vectors of the images in the enhanced dataset as input and the defect categories of the images in the enhanced dataset as output, the Adaboost algorithm is used to train the SVM model to obtain multiple SVM weak classifiers; All SVM weak classifiers are processed using linear weighting method to obtain strong classifiers; The strong classifier is determined to be the typical surface defect recognition model of the equipment.

4. A device surface defect recognition system, characterized in that: include: Original dataset construction module, used to construct the original dataset; The original data set includes a defect image set of various equipment surfaces; The defect image set includes images of various defect categories; An augmentation module, configured to augment the original data set to obtain an augmented data set; A histogram equalization processing module, configured to perform histogram equalization on the augmented data set to obtain an enhanced data set; a feature vector determination module, configured to process each image in the enhanced dataset using a SURF algorithm to obtain a feature vector set for each image in the enhanced dataset; A clustering module, configured to perform K-means clustering on all feature vectors in the feature vector set of all images in the enhanced dataset to obtain a clustering result; A feature vector dimensionality reduction module, configured to obtain a feature vector of each image in the enhanced dataset after dimensionality reduction using a bag of features model according to the clustering results; A feature vector reconstruction module, configured to reconstruct the feature vectors of each image in the enhanced dataset after dimensionality reduction according to the number of feature points of each image in the enhanced dataset, to obtain a reconstructed feature vector for each image in the enhanced dataset; a defect recognition module, configured to train an SVM model using an Adaboost algorithm based on the reconstructed feature vectors of all images in the enhanced dataset and the defect categories of all images in the enhanced dataset to obtain an equipment typical surface defect recognition model, wherein the equipment typical surface defect recognition model is used to identify equipment surface defects; The feature vector reconstruction module specifically includes: a reconstruction feature value calculation unit, which is used to obtain the reconstructed feature value of any image in the enhanced data set according to the number of feature points of the image, the maximum number of feature points of all images, and the minimum number of feature points of all images; specifically: the number of feature points of the image is dimensionless according to the maximum number of feature points of all images and the minimum number of feature points of all images to obtain the reconstructed feature value of the image; The feature vector reconstruction unit is used to add the reconstructed feature value of the image to the feature vector of the image after dimensionality reduction to obtain the reconstructed feature vector of the image.

5. The equipment surface defect recognition system according to claim 4, characterized in that: The feature vector dimensionality reduction module specifically includes: a feature point ratio calculation unit, configured to determine, for any image in the enhanced data set, a ratio of feature points of each cluster category in the image to all feature points in the image based on the clustering result; The feature vector dimension reduction unit is used to select a set number of ratios from the ratios of all feature points in the image to form the feature vector of the image after dimension reduction.

6. The equipment surface defect recognition system according to claim 4, characterized in that: The defect recognition module specifically includes: a weak classifier training unit, which is used to use the reconstructed feature vector of the image in the enhanced data set as input and the defect category of the image in the enhanced data set as output, and use the Adaboost algorithm to train the SVM model to obtain multiple SVM weak classifiers; A strong classifier determination unit is used to process all SVM weak classifiers using a linear weighting method to obtain a strong classifier; A model building unit is used to determine that the strong classifier is the typical surface defect recognition model of the equipment.

7. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the device surface defect identification method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that It stores a computer program, which, when executed by a processor, implements the device surface defect identification method according to any one of claims 1 to 3.

Images