Industrial product surface flaw detection method

Abstract

The invention provides an industrial product surface flaw detection method, which comprises the following steps: constructing and training a deep learning model, carrying out image acquisition on the surface of an industrial product to be detected, and preprocessing the acquired image; the preprocessed image is input into a trained deep learning model, the deep learning model carries out feature extraction on the preprocessed image to obtain hierarchical features of flaws in the image, and the hierarchical features comprise low-level features and high-level features; the deep learning model carries out image segmentation on the preprocessed image so as to segment a defect area and a normal area in the image; according to feature extraction and an image segmentation result, predicting an area with a flaw and a flaw category; and optimizing the prediction result to obtain a flaw detection result. According to the invention, automatic and high-efficiency detection of different types of flaws on the surface of the industrial product can be realized.

Description

Technical Field

[0001] The invention belongs to the technical field of defect detection, and in particular relates to a method for detecting surface defects of industrial products. Background Art

[0002] In industrial production, product surface quality issues have a significant impact on the product's appearance and functional performance. For example, surface defects such as scratches on electronic product shells, paint defects on car bodies, and bubbles in glass products may lead to product quality problems and customer complaints. Therefore, it is crucial to quickly and accurately detect surface defects on industrial products.

[0003] Traditional surface defect detection methods mainly rely on manual visual inspection, which has problems such as high labor cost, low efficiency and strong subjectivity. At the same time, different detection schemes need to be designed for different types of defects, which lacks versatility and flexibility. Therefore, it is necessary to propose a new method for industrial product surface defect detection, which can overcome the defects of traditional methods and realize automatic and efficient detection of different types of defects. Summary of the invention

[0004] In view of the problems raised by the background technology, the present invention provides a method for detecting surface defects of industrial products.

[0005] To achieve this object, the present invention adopts the following technical solutions:

[0006] A method for detecting surface defects of industrial products, comprising step A: constructing a deep learning model, collecting images of industrial products without defects, industrial products with different defect types and different numbers of defects, and annotating the collected images, including annotating whether the collected images have defects, defect types, and defect locations;

[0007] Step B: Using the collected images to train a deep learning model;

[0008] Step C: collecting images of the surface of the industrial product to be inspected, and preprocessing the collected images;

[0009] Step D: Input the preprocessed image into the trained deep learning model, and the deep learning model extracts features from the preprocessed image to obtain hierarchical features of defects in the image, where the hierarchical features include low-level features and high-level features;

[0010] The deep learning model performs image segmentation on the preprocessed image to separate the defect area and the normal area in the image;

[0011] Step E: predicting the defect area and defect category based on the feature extraction and image segmentation results;

[0012] Step F: Optimize the prediction results to obtain defect detection results.

[0013] Preferably, in step D, feature extraction includes extracting texture features of defects:

[0014] Convert the preprocessed image into a grayscale image;

[0015] Select a direction and a step size in the grayscale image, calculate the grayscale value of each pair of pixels and the spatial relationship between each pair of pixels to construct a grayscale co-occurrence matrix;

[0016] Extract texture features based on the gray-level co-occurrence matrix, including extracting:

[0017] Contrast: describes the roughness or variation of image texture;

[0018] Homogeneity: reflects the uniformity of image texture;

[0019] Energy: describes the repeatability of image texture;

[0020] Correlation: represents the dependency between the gray levels of image pixels.

[0021] Preferably, in step D, feature extraction includes extracting contour features of the defect:

[0022] The image is convolved by the Sobel operator to calculate the horizontal G of each pixel. x and vertical direction G y The gradient of

[0023] The gradient magnitude G of each pixel is calculated by the following formula:

[0024]

[0025] Set the edge threshold, and regard the area with gradient amplitude greater than the edge threshold as the edge area, and other areas as non-edge areas;

[0026] Determine the contour characteristics of the defect based on the edge area.

[0027] Preferably, in step D, feature extraction includes extracting color features of the defect:

[0028] The preprocessed image is converted from RGB image to HSV space using the following formula:

[0029]

[0030] S=1 / (R+G+B)×max(R,G,B)-min(R,G,B);

[0031] V = (R + G + B) / 3;

[0032] Among them: H represents hue, S represents saturation, V represents brightness, R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component;

[0033] Identify areas of color anomalies in an image based on hue, saturation, and brightness to obtain color signatures of defects.

[0034] Preferably, in step D, image segmentation includes:

[0035] Set regional thresholds;

[0036] Given a seed pixel in the image, for an adjacent pixel, if the similarity measure between the adjacent pixel and the seed pixel is less than a preset region threshold, the pixel and the seed pixel are merged into a seed region;

[0037] Find another adjacent pixel in the seed area, and continue to determine whether the similarity measure between it and a pixel in the seed area is less than the preset area threshold. If so, add the pixel to the current seed area, and continue to find the next adjacent pixel in the current seed area. Repeat this process until there are no adjacent pixels that can be added to the seed area.

[0038] The final seed area is taken as the defect area, and the other areas in the image except the seed area are taken as the normal area.

[0039] Preferably, in step D, image segmentation includes:

[0040] Get the gradient magnitude G of each pixel in the image;

[0041] Get the gradient direction of each pixel in the image θ = arctan (G y / G x );

[0042] Perform the following operation on each pixel in the image to obtain the gradient magnitude image:

[0043] For a pixel point, obtain the other two pixel points adjacent to the pixel point in the gradient direction, compare the gradient amplitude of the pixel point with the gradient amplitude of the two pixel points, and if the gradient amplitude of the pixel point is smaller than the gradient amplitude of the two pixel points, set the gradient amplitude of the pixel point to zero;

[0044] Set the high threshold T h and low threshold T l ;

[0045] If the gradient magnitude of a pixel in the gradient magnitude image is greater than the high threshold, that is, G>T h, then the pixel is defined as a strong edge;

[0046] If the gradient amplitude of the pixel in the gradient amplitude image is between the high threshold and the low threshold, that is, T l <G<T h , then the pixel is defined as a weak edge;

[0047] If the gradient magnitude of a pixel in the gradient magnitude image is less than the low threshold, G <T l , then the pixel is defined as a non-edge;

[0048] Mark all strong edges, traverse each weak edge pixel in the gradient magnitude image, if a weak edge pixel has a strong edge pixel in its area, then mark the weak edge pixel as a strong edge, otherwise discard the weak edge pixel;

[0049] The refined edge between the defect area and the normal area is obtained based on the pixel points of the strong edge.

[0050] Preferably, in said step E, it includes:

[0051] Obtain the probability that each pixel in the defective area belongs to different specific defect types, and obtain the probability distribution of different specific defect types in the area;

[0052] The specific defect type with the highest probability in the area is taken as the prediction result.

[0053] Preferably, in said step E, it also includes:

[0054] For each pixel in the defect area, among the probabilities that the pixel belongs to different specific defect types, determine the maximum probability of the pixel and the specific defect type to which the maximum probability belongs;

[0055] When the maximum probability is zero, the defect category of the pixel is defined as non-defect;

[0056] When the maximum probability is not zero, determine whether the maximum probability is lower than the matching degree of the specific defect type to which it belongs. If so, define the defect type of the pixel as an unclear defect; if not, define the defect type of the pixel as a clear defect.

[0057] Preferably, in step F, the prediction result is optimized to obtain a defect detection result, including:

[0058] For a pixel point whose defect category is a clear defect, determine the specific defect type A with the highest probability and the specific defect type B with the second highest probability for the pixel point;

[0059] Determine the defect area where the pixel is located, and find the number of specific defect types A and specific defect types B in all pixels in the defect area. If the number of specific defect type A is less than N times the number of specific defect type B, then the pixel is determined to be unreliable, otherwise the pixel is determined to be reliable.

[0060] Preferably, in step F, the prediction result is optimized to obtain a defect detection result, including:

[0061] Setting credibility thresholds;

[0062] Obtain the number of credible pixels in a defect area and the total number of pixels in the defect area;

[0063] The number of credible pixels in the defect area is divided by the total number of pixels in the defect area to obtain the credibility of the defect area;

[0064] Determine whether the credibility of the defective area is greater than the set credibility threshold. If so, the prediction result of the defective area is determined to pass the screening and the prediction result of the defective area is output as the defect detection result. Otherwise, it does not pass the screening.

[0065] For defective areas that fail the screening, a secondary identification test is performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of the method for detecting surface defects of industrial products of the present invention;

[0067] Figure 2 is a comparison diagram of defect detection effects of an embodiment of the present invention;

[0068] Figure 3 It is a comparison diagram of defect detection effects according to another embodiment of the present invention. DETAILED DESCRIPTION

[0069] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0070] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0072] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0073] Traditional surface defect detection methods mainly rely on manual visual inspection, which has problems such as high labor cost, low efficiency and strong subjectivity. At the same time, different detection schemes need to be designed for different types of defects, which lack versatility and flexibility. In order to solve the above problems, the present application proposes a method for detecting surface defects of industrial products. In this embodiment, the method described in the present application can be used in the electronic manufacturing industry: in the field of electronic product manufacturing, such as mobile phones, tablet computers, televisions and other electronic equipment production processes, it is applied to the rapid detection of surface defects of electronic products to improve product quality and production efficiency. Automobile manufacturing industry: In the automobile manufacturing process, this method can be used to detect surface defects of automobile bodies, such as scratches, dents, etc., to ensure that the appearance quality of automobiles meets the requirements. Glass manufacturing industry: In the production process of glass products, surface defects such as bubbles, scratches, cracks, etc. will affect the transparency and aesthetics of glass products. This method can be used to quickly detect surface defects of glass products to ensure product quality. Plastic product manufacturing industry: In the production of plastic products, surface defects will reduce the appearance quality and service life of the product. This method can be applied to the detection of surface defects of plastic products, such as bubbles, defects, dents, etc., to ensure that the product quality meets the standards. Pharmaceutical manufacturing industry: In the manufacturing process of medical devices and pharmaceutical packaging, surface defects may affect the hygiene and safety of the product. This method can be used to detect surface defects in medical device and pharmaceutical packaging to ensure that product quality meets medical industry standards. Food processing industry: In the food processing process, surface defects in packaging such as bottled beverages and canned foods may affect the hygiene and quality of the product. This method can be applied to the detection of surface defects in food packaging to ensure that the product meets food safety standards. Textile manufacturing industry: In textile production, surface defects on fabrics such as clothing and household items can affect the appearance and comfort of the product. This method can be used to detect surface defects in textiles to improve product quality and market competitiveness.

[0074] Specifically, Figure 1 As shown, the method includes step A: constructing a deep learning model, collecting images of industrial products without defects, industrial products with different defect types and different numbers of defects, and annotating the collected images, including annotating whether the collected images have defects, defect types, and defect locations;

[0075] Step B: Using the collected images to train a deep learning model;

[0076] Step C: collecting images of the surface of the industrial product to be inspected, and preprocessing the collected images;

[0077] Step D: Input the preprocessed image into the trained deep learning model, and the deep learning model extracts features from the preprocessed image to obtain hierarchical features of defects in the image, where the hierarchical features include low-level features and high-level features;

[0078] The deep learning model performs image segmentation on the preprocessed image to separate the defect area and the normal area in the image;

[0079] Step E: predicting the defect area and defect category based on the feature extraction and image segmentation results;

[0080] Step F: Optimize the prediction results to obtain defect detection results.

[0081] In this embodiment, building and training a deep learning model includes the following specific methods:

[0082] Collect image data, including products without defects and products with different types and numbers of defects, and annotate these images to indicate whether they have defects and the type and location of the defects. These image data are used to train deep learning models;

[0083] Select a deep learning model architecture suitable for defect detection tasks, such as convolutional neural network (CNN), recurrent neural network (RNN), etc., define the neural network layers included in the architecture, and select different model architectures according to different industrial products and actual detection computing power; in this embodiment, a convolutional neural network is used for explanation;

[0084] Define the network layers included in the convolutional neural network architecture: input layer, convolution layer, activation function, pooling layer, fully connected layer, output layer; configure the loss function and optimizer. The details are as follows:

[0085] Input layer: receives raw data and passes it to the next layer. In image classification tasks, the input is usually image data with a shape of H×W×C. For example, the input layer can be defined as: Input(H,W,C), where H is the height of the image, W is the width of the image, and C is the number of channels (for example, the number of channels for RGB images is 3);

[0086] Convolutional layer: The convolutional layer is the most important layer in CNN, which extracts local features of the input data through convolution operations. Each convolution operation involves a filter (kernel) that slides on the input image to calculate the convolution output. For a single convolution operation, the output feature map is calculated as:

[0087]

[0088] Among them, X is the input feature map, W is the filter, b is the bias term, k is the size of the filter, and Y(i,j) is the output feature map;

[0089] Activation function: An activation function, such as ReLU (Rectified Linear Unit), is usually applied after the convolution layer to increase the nonlinear characteristics of the network. The ReLU activation function formula is:

[0090] f(x)=max(0,x);

[0091] Pooling layer: The pooling layer usually follows the convolution layer to reduce the spatial dimension of the feature map, reduce the amount of calculation, and reduce overfitting. Max pooling is the pooling operation used in this embodiment, which takes the maximum value in the local area. The calculation formula of max pooling is:

[0092]

[0093] Among them, X(i,j) is the input feature map and Y(i,j) is the output after pooling.

[0094] Fully connected layer: After the convolutional layer and the pooling layer, there are usually one or more fully connected layers to flatten the feature map and map it to the final output space. Each fully connected layer is a linear transformation, calculated as:

[0095] y=W·x+b;

[0096] Among them, x is the input vector, W is the weight matrix, b is the bias term, and y is the output.

[0097] Output layer: Depending on the task, the output layer is usually a classification problem or a regression problem. For example, in this embodiment, it is the classification of defect types. In this embodiment, the Softmax activation function is used to output the probability distribution for the classification of defect types. The Softmax function formula is:

[0098]

[0099] Among them, z k is the raw score of each category in the output layer, P(y=c k |x) is the predicted probability of that category.

[0100] Loss function: The choice of loss function depends on the task type. Since this embodiment is a defect classification task for defect detection, the loss function used is cross-entropy loss:

[0101]

[0102] Among them, y is the true label, is the predicted probability.

[0103] Optimizer: The optimizer is used to adjust the weights and biases in the network to minimize the loss function. Commonly used optimizers include gradient descent (SGD), Adam optimizer, etc. In this embodiment, the Adam optimizer is selected for processing, and the update formula of the Adam optimizer is as follows:

[0104] m t =β1m t-1 +(1-β1)g t ;

[0105]

[0106] Among them, m t and v t are the first-order and second-order moment estimates of the gradient, η is the learning rate, β1, β2 are the decay coefficients, and ε is a small constant to prevent division by zero errors.

[0107] The model is trained using a large dataset of annotated images, enabling it to learn and accurately identify different types of surface defects.

[0108] like Figure 2 and Figure 3 As shown, Figure 2 The left side shows the input image of an embodiment of the model, and the right side shows the recognition effect of the model on the embodiment. Figure 3 The left side is the input image of another embodiment of the present model, and the right side is the recognition effect of the present model on the embodiment.

[0109] Further, in step C: the surface of the industrial product to be inspected is imaged and the image is preprocessed; specifically, the image is captured by using a high-resolution camera or other image acquisition device to capture the surface of the industrial product. Since the image acquisition may be affected by factors such as lighting and angle, preprocessing is required to improve the image quality and the accuracy of subsequent processing; the preprocessing includes steps such as removing noise, eliminating image distortion, enhancing contrast, adjusting brightness and color balance, etc., to ensure that the captured image is clear and accurate. Since the preprocessing operation is a conventional technology, it will not be elaborated in detail in this embodiment.

[0110] Preferably, before performing feature extraction and image segmentation, the preprocessed image needs to be converted to generate a corresponding grayscale image or color image, and subsequent operations need to be performed on the basis of these two images. The characteristics of defects include size, texture, shape, color, etc. The feature extraction method includes analyzing the texture, detecting the edges in the image, and converting the color space of the image.

[0111] When the model performs defect detection, it first uses the feature extraction method to extract the hierarchical features of the image through multiple convolutional layers, from low-level features (edges, textures, etc.) to high-level features (shapes, patterns, etc.);

[0112] Therefore, in the step D, feature extraction includes extracting texture features of defects;

[0113] Texture is a pattern in an image that reflects surface details or structural features. Texture features are extracted by statistically describing pixel grayscale changes. This embodiment uses a grayscale co-occurrence matrix for texture analysis, which describes the spatial relationship between pixel pairs in an image. The texture features of an image are reflected by calculating the change in grayscale levels. In this embodiment, texture features include contrast, correlation, energy, and homogeneity. The specific operations are as follows:

[0114] If the image to be processed is a color image, the preprocessed image needs to be converted into a grayscale image;

[0115] Select a direction and a step size in the grayscale image (for example, 0°, 45°, 90°, 135° directions, with a step size of 1 pixel), calculate the grayscale value of each pair of pixels and the spatial relationship between each pair of pixels to construct a grayscale co-occurrence matrix;

[0116] In one step, due to the large amount of calculation of the grayscale co-occurrence matrix, it is usually necessary to quantize the grayscale of the image to reduce the amount of calculation, for example, quantize the 256-level grayscale image to 8 or 16 levels. Then determine the direction and step size, which determine the distance and direction between pixel pairs. In actual operation, arbitrarily take a pixel point in the grayscale image, find another pixel point according to the step size and direction, and obtain the grayscale value (g1, g2) of this pair of pixels. Traverse all the pixels in the grayscale image, find the corresponding pixel pair according to the step size and direction, and count the number of occurrences of the grayscale value of each pixel pair. According to the statistical results, construct an N x N matrix, N is the number of grayscale quantized levels, the rows and columns of the matrix represent the grayscale levels in the grayscale value, and the element values ​​in the matrix represent the number of occurrences of the grayscale levels represented by the rows and columns in all pixel pairs with specific directions and specific step sizes. The element values ​​of the grayscale co-occurrence matrix are normalized to obtain the probability of occurrence of the grayscale value of each pixel pair.

[0117] Extract texture features based on the gray-level co-occurrence matrix, including extracting:

[0118] Contrast: describes the roughness or variation of image texture;

[0119] Homogeneity: reflects the uniformity of image texture;

[0120] Energy: describes the repeatability of image texture;

[0121] Correlation: represents the dependency between the gray levels of image pixels.

[0122] In this embodiment, the contrast is extracted by the gray level co-occurrence matrix by substituting the value in the gray level co-occurrence matrix into the contrast formula for calculation to obtain the contrast characteristic value of the image, the contrast formula is: Con = ∑i, j(ij)2P(i, j), where P(i, j) represents the probability of occurrence of a pixel pair with gray values ​​g1 and g2 in the gray level co-occurrence matrix, and (ij) represents the difference between the two gray values;

[0123] Homogeneity refers to the similarity between each element in the gray level co-occurrence matrix and its adjacent elements. It reflects the local uniformity and consistency of the image texture. In the gray level co-occurrence matrix, if the element value on the diagonal is large, it means that the gray values ​​of adjacent pixels are similar, that is, the image texture has a high homogeneity. The calculation formula is as follows: Hom = ∑i∑jp(i, j) / 1+(ij) 2 , in this formula, P(i,j) represents the probability of occurrence of adjacent pixel pairs with gray levels i and j in the gray level co-occurrence matrix;

[0124] The size of the energy value reflects the uniformity and repeatability of the image texture. When the element values ​​in the gray-level co-occurrence matrix are distributed more evenly, the image texture is more repeated, that is, the values ​​of each element are not much different, and the energy value will be relatively small, indicating that the image texture is more delicate. On the contrary, when some element values ​​in the gray-level co-occurrence matrix are significantly larger than other element values, the energy value will be larger, indicating that the image texture is relatively rough. The energy value is calculated as the sum of the squares of all element values ​​in the gray-level co-occurrence matrix.

[0125] The calculation formula of correlation can be Cor = ∑i∑j(i-μ i )(j-μ j )p(i,j) / σ i σ j , where P(i,j) represents the probability of occurrence of adjacent pixel pairs with gray levels i and j in the gray level co-occurrence matrix; μ i and μ j are the gray level means of the rows and columns of the gray level co-occurrence matrix respectively; σ i , σ j are the gray level standard deviations of the rows and columns of the gray level co-occurrence matrix respectively.

[0126] Preferably, in step D, feature extraction includes extracting contour features of defects;

[0127] Edge detection is the process of extracting the boundaries of objects in an image. It is usually used to identify the shape, structure and detail features in an image, namely the contour features. The edge is the area in the image where the brightness changes dramatically. It can effectively help identify the contour of the defect. The image is convolved by the Sobel operator to calculate the horizontal G of each pixel. x and vertical direction G y The gradient of

[0128] The gradient magnitude G of each pixel is calculated by the following formula:

[0129]

[0130] Set the edge threshold, and regard the area with gradient amplitude greater than the edge threshold as the edge area, and other areas as non-edge areas;

[0131] Determine the contour characteristics of the defect based on the edge area.

[0132] Preferably, in step D, feature extraction includes extracting color features of defects;

[0133] Color space conversion can extract meaningful color information by converting an image from one color space to another. For defect detection, certain color channels may show defect characteristics more prominently, such as the brightness channel in the RGB space or the hue and saturation in the HSV space. In this embodiment, such as RGB to HSV conversion: HSV (hue, saturation, brightness) is a way to express the color of an image. Compared with RGB, HSV is more in line with the human eye's perception of color. Through the HSV space, color features can be better distinguished, especially for defects of different colors. The details are as follows:

[0134] The preprocessed image is converted from RGB image to HSV space using the following formula:

[0135]

[0136] S=1 / (R+G+B)×max(R,G,B)-min(R,G,B);

[0137] V = (R + G + B) / 3;

[0138] Among them: H represents hue, S represents saturation, V represents brightness, R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component;

[0139] Identify areas of color anomalies in an image based on hue, saturation, and brightness to obtain color signatures of defects.

[0140] Preferably, in step D, image segmentation technology is used to segment the defective area from the normal area to provide an accurate data basis for subsequent processing;

[0141] Specifically include: setting regional thresholds;

[0142] Given a seed pixel in the image, for an adjacent pixel, if the similarity measure between the adjacent pixel and the seed pixel is less than a preset region threshold, the pixel and the seed pixel are merged into a seed region;

[0143] Find another adjacent pixel in the seed area, and continue to determine whether the similarity measure between it and a pixel in the seed area is less than the preset area threshold. If so, add the pixel to the current seed area, and continue to find the next adjacent pixel in the current seed area. Repeat this process until there are no adjacent pixels that can be added to the seed area.

[0144] The final seed area is taken as the defect area, and the other areas in the image except the seed area are taken as the normal area.

[0145] Preferably, in the foregoing, it is mentioned that edge detection is used to extract contour features of defects. On this basis, the edges can be refined to further segment defect areas and normal areas;

[0146] Including obtaining the gradient amplitude G of each pixel in the image;

[0147] Get the gradient direction of each pixel in the image θ = arctan (G y / G x );

[0148] Perform the following operation on each pixel in the image to obtain the gradient magnitude image:

[0149] For a pixel point, obtain the other two pixel points adjacent to the pixel point in the gradient direction, compare the gradient amplitude of the pixel point with the gradient amplitude of the two pixel points, and if the gradient amplitude of the pixel point is smaller than the gradient amplitude of the two pixel points, set the gradient amplitude of the pixel point to zero;

[0150] Set the high threshold T h and low threshold T l ;

[0151] If the gradient magnitude of a pixel in the gradient magnitude image is greater than the high threshold, that is, G>t h , then the pixel is defined as a strong edge;

[0152] If the gradient amplitude of the pixel in the gradient amplitude image is between the high threshold and the low threshold, that is, t l <G<Th , then the pixel is defined as a weak edge;

[0153] If the gradient magnitude of a pixel in the gradient magnitude image is less than the low threshold, G <T l , then the pixel is defined as a non-edge;

[0154] Mark all strong edges, traverse each weak edge pixel in the gradient magnitude image, if a weak edge pixel has a strong edge pixel in its area, then mark the weak edge pixel as a strong edge, otherwise discard the weak edge pixel;

[0155] The refined edge between the defect area and the normal area is obtained based on the pixel points of the strong edge.

[0156] Preferably, in said step E, it includes:

[0157] Obtain the probability that each pixel in the defective area belongs to different specific defect types, and obtain the probability distribution of different specific defect types in the area;

[0158] The specific defect type with the highest probability in the area is taken as the prediction result.

[0159] In this embodiment, the last layer of the model is usually a Softmax layer, which is used to output the probability distribution of different categories. The model will select the category with the highest probability as the prediction result. For example, in a defect area, the probability that pixel A belongs to defect type a is 20%, the probability that it belongs to defect type b is 80%, and the probability that it belongs to defect type c is 50%. The probability that pixel B belongs to defect type a is 70%, the probability that it belongs to defect type b is 60%, and the probability that it belongs to defect type c is 40%. By injecting this category and statistically analyzing the probability distribution of different specific defect types of all pixels in the defect area, it can be obtained that the defect type with the highest probability is b. Then the defect area is defined as defect type b, and defect type b is used as the prediction result.

[0160] Preferably, in said step E, it also includes:

[0161] For each pixel in the defect area, among the probabilities that the pixel belongs to different specific defect types, determine the maximum probability of the pixel and the specific defect type to which the maximum probability belongs;

[0162] When the maximum probability is zero, the defect category of the pixel is defined as non-defect;

[0163] When the maximum probability is not zero, determine whether the maximum probability is lower than the matching degree of the specific defect type to which it belongs. If so, define the defect type of the pixel as an unclear defect; if not, define the defect type of the pixel as a clear defect.

[0164] In this embodiment, the defects of each pixel can be divided into three categories: non-defects, unclear defects and clear defects, where clear defects can be many specific defect types; specific defect types are distinguished according to classification rules, and if distinguished by shape, they can be divided into circular defects, triangular defects, etc.; unclear defects usually indicate a type that has a low matching degree with a specific defect type due to factors such as small size or unclear shape;

[0165] Based on the classification of each pixel into three categories, the prediction results can be optimized later.

[0166] For example, in a defect area, the probability that pixel A belongs to defect type a is 0%, the probability that it belongs to defect type b is 0%, and the probability that it belongs to defect type c is 0%. Then the maximum probability that pixel A belongs to a specific defect type is 0%, and pixel A is defined as a non-defect category.

[0167] In a defect area, the probability that pixel B belongs to defect type a is 70%, the probability that it belongs to defect type b is 60%, and the probability that it belongs to defect type c is 40%. Then the maximum probability of the specific defect type to which pixel B belongs is 70%, and the specific defect type to which pixel B belongs is a. At this time, if the matching degree of the defect type is set to 50%, the maximum probability of 70% that pixel B belongs to the specific defect type a exceeds the matching degree of 50% of defect type a, and pixel B is defined as a clear defect type; similarly, if the probability that pixel B belongs to defect type a is 10%, the probability that it belongs to defect type b is 10%, and the probability that it belongs to defect type c is 90%, then the specific defect type to which pixel B belongs is c, and the maximum probability is 90%. If the matching degree of defect type c is set to 50%, the maximum probability of 90% that pixel B belongs to the specific defect type c exceeds the matching degree of 50% of defect type c, and pixel B is also defined as a clear defect category;

[0168] Furthermore, in a defect area, the probability that pixel C belongs to defect type a is 15%, the probability that it belongs to defect type b is 5%, and the probability that it belongs to defect type c is 30%. The maximum probability that pixel C belongs to a specific defect type is 30%. Corresponding to defect type c, if the set matching degree of defect type c is 50%, then the maximum probability that pixel C belongs to the specific defect type c is 30%, which is lower than the matching degree of defect type c of 50%. Then pixel C is also defined as an unclear defect category.

[0169] Preferably, in step F, the prediction result is optimized to obtain a defect detection result, including:

[0170] For a pixel point whose defect category is a clear defect, determine the specific defect type A with the highest probability and the specific defect type B with the second highest probability for the pixel point;

[0171] Determine the defect area where the pixel is located, and find the number of specific defect types A and specific defect types B in all pixels in the defect area. If the number of specific defect type A is less than N times the number of specific defect type B, then the pixel is determined to be unreliable, otherwise the pixel is determined to be reliable.

[0172] In this embodiment, the prediction result includes the specific defect type of the defect area, but in order to distinguish the confidence of the prediction result and screen out the defect area with high confidence, the prediction result needs to be optimized; in step E, the specific defect type with the highest probability in the defect area is taken as the prediction result, so the majority of pixels in the defect area belong to the clear defect category, but in the same defect area, there will also be pixels of unclear defect category or non-defect category, so in a defect area, each pixel has more or less probability for one or more specific defect types; on this basis, we first find the pixel classified as a clear defect, which may have a or multiple specific defect types, for example, the probability of pixel O belonging to specific defect type A is 70%, the probability of belonging to defect type B is 50%, and the probability of belonging to defect type C is 30%. Then the defect type with the highest probability for pixel O is A, and the defect type with the second highest probability is B. Then determine the number of pixels with defect type A in the defect area where pixel O is located, such as 100, and determine the number of pixels with defect type B in the defect area where pixel O is located, such as 70. Then the number of pixels with the highest probability of defect type A, 100, is less than N times the number of pixels with the second highest probability of defect type B, 70. At this time, N is 2, and the pixel O is determined to be unreliable. In this way, we can get whether each pixel is reliable and the number of reliable pixels in the defect area.

[0173] Preferably, in step F, the prediction result is optimized to obtain a defect detection result, including:

[0174] Setting credibility thresholds;

[0175] Obtain the number of credible pixels in a defect area and the total number of pixels in the defect area;

[0176] The number of credible pixels in the defect area is divided by the total number of pixels in the defect area to obtain the credibility of the defect area;

[0177] Determine whether the credibility of the defective area is greater than the set credibility threshold. If so, the prediction result of the defective area is determined to pass the screening and the prediction result of the defective area is output as the defect detection result. Otherwise, it does not pass the screening.

[0178] In this embodiment, when the number of credible pixels in a defect area is known, the ratio of the number of credible pixels to the total number of pixels in the defect area is the credibility of the defect area. Through the early training of the model, we can set a credibility threshold. When the credibility of the defect area is greater than the set credibility threshold, we determine that the prediction result of this defect area is credible, otherwise the prediction result of this defect area is unreliable.

[0179] For defective areas that fail the screening, a secondary identification test is performed.

[0180] Specifically, if the credibility of the defective area does not meet the standard, it is necessary to process the defective area, mainly including morphological operations (such as corrosion and expansion, opening and closing operations), connected component analysis (for independent identification of defective areas), and filtering processing (including Gaussian filtering and median filtering) to reduce the influence of noise before testing. The above-mentioned operations are conventional techniques and will not be elaborated in this embodiment.

[0181] In summary, this method has the following advantages:

[0182] 1. High-precision detection: This method is based on deep learning technology. After a large amount of data training, it can achieve high-precision detection of surface defects of industrial products. By learning the characteristics of a large number of surface defects, the deep learning model can accurately identify and locate defects of various shapes, sizes, and colors, ensuring the accuracy and reliability of the detection results.

[0183] 2. Multiple defect recognition: This method can recognize multiple types of surface defects, including but not limited to scratches, dents, bubbles, stains, etc. The deep learning model has good generalization ability and can cope with various complex defect forms, which is suitable for the detection needs of different industrial products.

[0184] 3. Real-time detection: This method can be used to detect surface defects in real time. The deep learning model runs efficiently on acceleration devices such as GPUs. Combined with optimized algorithms and processing procedures, it can quickly process image data and output detection results in real time, meeting the real-time requirements in industrial production.

[0185] 4. Large-scale production application: This method is suitable for application scenarios in large-scale industrial production, and can realize rapid detection of surface defects of a large number of products. Through the deployment of an automated detection system, continuous and efficient defect detection of products on the entire production line can be achieved, improving production efficiency and quality control level.

[0186] 5. Automated processing and feedback control: This method realizes automated processing and feedback control of surface defects of industrial products. For detected defects, the system can automatically mark, record and alarm, and promptly notify operators to handle them, preventing defective products from entering the market or the hands of customers, reducing the risk of product recalls and complaints.

[0187] 6. Reduce costs and improve efficiency: Compared with traditional manual visual inspection, this method can greatly reduce labor costs and improve the efficiency and accuracy of detection. Without a large amount of manpower investment, rapid detection of large-scale products can be achieved, saving human resources and time costs, while improving the intelligence level of the production line.

[0188] 7. Strong adaptability and high flexibility: This method can be customized and optimized according to the characteristics and requirements of different industrial products. By adjusting the structure and parameters of the deep learning model, it can adapt to the surface defect detection needs of different industrial products, and has strong adaptability and flexibility.

[0189] The technical principle of the present invention is described above in conjunction with specific embodiments. These descriptions are only for explaining the principle of the present invention and cannot be interpreted as limiting the scope of protection of the present invention in any way. Based on the explanations herein, those skilled in the art can associate other specific implementations of the present invention without paying creative labor, and these methods will fall within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects of industrial products, characterized in that: Step A: Build a deep learning model, collect images of industrial products without defects, industrial products with different defect types and different numbers of defects, and annotate the collected images, including annotating whether the collected images have defects, defect types, and defect locations; Step B: Using the collected images to train a deep learning model; Step C: collecting images of the surface of the industrial product to be inspected, and preprocessing the collected images; Step D: Input the preprocessed image into the trained deep learning model, and the deep learning model extracts features from the preprocessed image to obtain hierarchical features of defects in the image, where the hierarchical features include low-level features and high-level features; The deep learning model performs image segmentation on the preprocessed image to separate the defect area and the normal area in the image; Step E: predicting the defect area and defect category based on the feature extraction and image segmentation results; Step F: Optimize the prediction results to obtain defect detection results.

2. The method for detecting surface defects of industrial products according to claim 1, characterized in that: In step D, feature extraction includes extracting texture features of defects: Convert the preprocessed image into a grayscale image; Select a direction and a step size in the grayscale image, calculate the grayscale value of each pair of pixels and the spatial relationship between each pair of pixels to construct a grayscale co-occurrence matrix; Extract texture features based on the gray-level co-occurrence matrix, including extracting: Contrast: describes the roughness or variation of image texture; Homogeneity: reflects the uniformity of image texture; Energy: describes the repeatability of image texture; Correlation: represents the dependency between the gray levels of image pixels.

3. The method for detecting surface defects of industrial products according to claim 1, characterized in that: In step D, feature extraction includes extracting contour features of defects: The image is convolved by the Sobel operator to calculate the horizontal G of each pixel. x and vertical direction G y The gradient of The gradient magnitude G of each pixel is calculated by the following formula: Set the edge threshold, and regard the area with gradient amplitude greater than the edge threshold as the edge area, and other areas as non-edge areas; Determine the contour characteristics of the defect based on the edge area.

4. The method for detecting surface defects of industrial products according to claim 1, characterized in that: In step D, feature extraction includes extracting color features of defects: The preprocessed image is converted from RGB image to HSV space using the following formula: S=1 / (R+G+B)×max(R,G,B)-min(R,G,B); V = (R + G + B) / 3; Among them: H represents hue, S represents saturation, V represents brightness, R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component; Identify areas of color anomalies in an image based on hue, saturation, and brightness to obtain color signatures of defects.

5. The method for detecting surface defects of industrial products according to claim 3, characterized in that: In the step D, image segmentation includes: Set regional thresholds; Given a seed pixel in the image, for an adjacent pixel, if the similarity measure between the adjacent pixel and the seed pixel is less than a preset region threshold, the pixel and the seed pixel are merged into a seed region; Find another adjacent pixel in the seed area, and continue to determine whether the similarity measure between it and a pixel in the seed area is less than the preset area threshold. If so, add the pixel to the current seed area, and continue to find the next adjacent pixel in the current seed area. Repeat this process until there are no adjacent pixels that can be added to the seed area. The final seed area is taken as the defect area, and the other areas in the image except the seed area are taken as the normal area.

6. The method for detecting surface defects of industrial products according to claim 5, characterized in that: In the step D, image segmentation includes: Get the gradient magnitude G of each pixel in the image; Get the gradient direction of each pixel in the image θ = arctan (G y / G x ); Perform the following operation on each pixel in the image to obtain the gradient magnitude image: For a pixel point, obtain the other two pixel points adjacent to the pixel point in the gradient direction, compare the gradient amplitude of the pixel point with the gradient amplitude of the two pixel points, and if the gradient amplitude of the pixel point is smaller than the gradient amplitude of the two pixel points, set the gradient amplitude of the pixel point to zero; Set the high threshold T h and low threshold T l ; If the gradient magnitude of a pixel in the gradient magnitude image is greater than the high threshold, that is, G>T h , then the pixel is defined as a strong edge; If the gradient amplitude of the pixel in the gradient amplitude image is between the high threshold and the low threshold, that is, T l <G<T h , then the pixel is defined as a weak edge; If the gradient magnitude of a pixel in the gradient magnitude image is less than the low threshold, G <T l , then the pixel is defined as a non-edge; Mark all strong edges, traverse each weak edge pixel in the gradient magnitude image, if a weak edge pixel has a strong edge pixel in its area, then mark the weak edge pixel as a strong edge, otherwise discard the weak edge pixel; The refined edge between the defect area and the normal area is obtained based on the pixel points of the strong edge.

7. The method for detecting surface defects of industrial products according to claim 1, characterized in that: In the step E, it includes: Obtain the probability that each pixel in the defective area belongs to different specific defect types, and obtain the probability distribution of different specific defect types in the area; The specific defect type with the highest probability in the area is taken as the prediction result.

8. The method for detecting surface defects of industrial products according to claim 7, characterized in that: In the step E, it also includes: For each pixel in the defect area, among the probabilities that the pixel belongs to different specific defect types, determine the maximum probability of the pixel and the specific defect type to which the maximum probability belongs; When the maximum probability is zero, the defect category of the pixel is defined as non-defect; When the maximum probability is not zero, determine whether the maximum probability is lower than the matching degree of the specific defect type to which it belongs. If so, define the defect type of the pixel as an unclear defect; if not, define the defect type of the pixel as a clear defect.

9. The method for detecting surface defects of industrial products according to claim 8, characterized in that: In step F, the prediction result is optimized to obtain a defect detection result, including: For a pixel point whose defect category is a clear defect, determine the specific defect type A with the highest probability and the specific defect type B with the second highest probability for the pixel point; Determine the defect area where the pixel is located, and find the number of specific defect types A and specific defect types B in all pixels in the defect area. If the number of specific defect type A is less than N times the number of specific defect type B, then the pixel is determined to be unreliable, otherwise the pixel is determined to be reliable.

10. The method for detecting surface defects of industrial products according to claim 9, characterized in that: In step F, the prediction result is optimized to obtain a defect detection result, including: Setting credibility thresholds; Obtain the number of credible pixels in a defect area and the total number of pixels in the defect area; The number of credible pixels in the defect area is divided by the total number of pixels in the defect area to obtain the credibility of the defect area; Determine whether the credibility of the defective area is greater than the set credibility threshold. If so, the prediction result of the defective area is determined to pass the screening and the prediction result of the defective area is output as the defect detection result. Otherwise, it does not pass the screening. For defective areas that fail the screening, a secondary identification test is performed.

Images