Industrial product quality detection method

Abstract

The invention provides an industrial product quality detection method. The method comprises the following steps: constructing and training a quality detection model; collecting an image of the industrial product to be detected and carrying out image preprocessing; inputting the preprocessed image into a trained quality detection model for image segmentation so as to segment the industrial product and a background region, and then dividing different modules of the industrial product; according to the type of the industrial product to be detected and the plurality of clearly detected defect types, calling a plurality of corresponding detection algorithms to perform defect detection; outputting the defect grades of the plurality of defect types of the detected industrial product, and judging whether the quality of the industrial product is qualified or not based on the defect grades of the plurality of defect types; and making a plurality of quality decision strategies, and executing the corresponding quality decision strategies according to the quality qualification degree of the industrial product. The method has the advantages of high precision, efficient detection and real-time feedback and adjustment in the product quality detection process.

Description

Technical Field

[0001] The invention belongs to the technical field of industrial product quality detection, and in particular relates to an industrial product quality detection method. Background Art

[0002] In modern industrial production, the stability and consistency of product quality are crucial to the competitiveness and market reputation of enterprises. Therefore, manufacturing enterprises have increasingly higher requirements for product quality management and need reliable quality testing methods to ensure product quality.

[0003] Traditional product quality inspection methods usually rely on manual visual inspection or simple machine vision systems. These methods have problems such as low efficiency, low accuracy and susceptibility to subjective factors. In addition, traditional methods often cannot meet the high-speed production requirements of modern industrial production lines. Therefore, an efficient quality inspection method is urgently needed to solve the above problems. Summary of the invention

[0004] In view of the problems raised by the background technology, the present invention proposes an industrial product quality detection method.

[0005] To achieve this object, the present invention adopts the following technical solutions:

[0006] A method for detecting quality of industrial products, comprising:

[0007] Step A: Build a quality inspection model, collect sample images of normal industrial products and industrial products with different types of defects to form a data set, annotate the sample images, including marking whether the sample images have defects, defect locations and defect types, and use the data set to train the quality inspection model;

[0008] Step B: Collect images of the industrial products to be inspected and perform image preprocessing;

[0009] Step C: Input the preprocessed image into the trained quality inspection model. The quality inspection model performs image segmentation on the preprocessed image to separate the industrial product from the background area, and then divides the different modules of the industrial product itself to obtain the image area that needs to be analyzed;

[0010] Step D: According to the type of industrial product to be inspected and the multiple defect types to be clearly inspected, the quality inspection model calls the corresponding multiple inspection algorithms to perform defect inspection on the image area processed by step C;

[0011] Step E: the quality inspection model outputs defect levels of multiple defect types of the inspected industrial product, and determines whether the quality of the industrial product is qualified based on the defect levels of the multiple defect types;

[0012] Step F: Develop a variety of quality decision strategies and implement corresponding quality decision strategies according to the quality qualification level of industrial products.

[0013] Preferably, in step A, it also includes:

[0014] In a data set, if multiple sample images are simultaneously labeled with multiple defect types, then sample images with a single appearance of each defect type are added.

[0015] Preferably, in step C, segmenting the industrial product from the background area comprises:

[0016] Step C1: Obtain the gradient amplitude G of each pixel in the preprocessed image;

[0017] Get the gradient direction of each pixel in the preprocessed image θ = arctan (G y / G x );

[0018] Step C2: Perform the following operation on each pixel in the image to obtain a gradient magnitude image:

[0019] For a pixel point, obtain the other two pixel points adjacent to the pixel point in the gradient direction, compare the gradient amplitude of the pixel point with the gradient amplitude of the two pixel points, and if the gradient amplitude of the pixel point is smaller than the gradient amplitude of the two pixel points, set the gradient amplitude of the pixel point to zero;

[0020] Set the high threshold T h and low threshold T l ;

[0021] If the gradient magnitude of a pixel in the gradient magnitude image is greater than the high threshold, that is, G>T h , then the pixel is defined as a strong edge;

[0022] If the gradient amplitude of the pixel in the gradient amplitude image is between the high threshold and the low threshold, that is, T l <G<T h , then the pixel is defined as a weak edge;

[0023] If the gradient magnitude of a pixel in the gradient magnitude image is less than the low threshold, G <T l , then the pixel is defined as a non-edge;

[0024] Step C3: Mark all strong edges, traverse each weak edge pixel in the gradient magnitude image, and if a weak edge pixel has a strong edge pixel in its area, mark the weak edge pixel as a strong edge, otherwise discard the weak edge pixel;

[0025] The refined edge between the industrial product and the background area in the preprocessed image is obtained according to the pixel points of the strong edge, and the industrial product and the background area are segmented based on the refined edge.

[0026] Preferably, in step C, different modules of the industrial product itself are divided to obtain image areas to be analyzed, including:

[0027] Step C4: determining an image of only the industrial product after removing the background area after processing from step C1 to step C3;

[0028] Step C5: setting the area threshold;

[0029] Given a seed pixel in an image containing only industrial products, for an adjacent pixel, if the similarity measure between the adjacent pixel and the seed pixel is less than a preset region threshold, the pixel and the seed pixel are merged into a seed region;

[0030] Find another adjacent pixel in the seed area, and continue to determine whether the similarity measure between it and a pixel in the seed area is less than the preset area threshold. If so, add the pixel to the current seed area, and continue to find the next adjacent pixel in the current seed area. Repeat this process until there are no adjacent pixels that can be added to the seed area.

[0031] The final seed area is used as a separate module area of ​​the industrial product, and other areas in the image except the seed area are used as other module areas.

[0032] Step C6: Repeat step C5 for other module areas until the other module areas are also divided into multiple independent module areas;

[0033] Step C7: Record each individual module area as an image area that needs to be analyzed and output.

[0034] Preferably, in said step D, it includes:

[0035] When there are multiple defect types to be detected for an industrial product, all defect types to be detected for the industrial product are identified, and the corresponding detection algorithm is called for each defect type for detection.

[0036] Preferably, in said step E, it includes:

[0037] Each defect type is set with the same number of defect levels, and the defect level of each defect of each defect type is determined according to the defect range;

[0038] When an industrial product has at least one defect type, the defect levels of multiple defects of each defect type are the same or different;

[0039] Preferably, in said step E, it also includes:

[0040] Determine whether the maximum defect level of the current industrial product exceeds the maximum defect level allowed for the industrial product. If so, the quality of the current industrial product is deemed unqualified. If not, determine whether the sum of all defect levels of the current industrial product exceeds the preset qualified value. If so, the quality of the current industrial product is deemed unqualified. If not, the quality of the current industrial product is deemed qualified.

[0041] Preferably, in the step F, it includes:

[0042] For the industrial products that are found to be unqualified in step E, determine whether their maximum defect level exceeds the first repair value. If so, implement a discard strategy for the unqualified industrial products. If not, determine whether the number of all defects exceeds the second repair value. If so, implement a discard strategy for the unqualified industrial products. If not, implement a repair strategy for the unqualified industrial products.

[0043] Preferably, in step A, using the data set to train the quality detection model also includes using an L2 regularization method or an early stopping strategy to optimize the training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the industrial product quality detection method of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention is further described below with reference to the accompanying drawings and through specific implementation methods.

[0046] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or ends.

[0048] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0049] The present application proposes a method for detecting the quality of industrial products. In this embodiment, the method described in the present application can be used in the electronic manufacturing industry: in the field of electronic product manufacturing, such as in the production process of electronic devices such as mobile phones, tablet computers, and televisions, it is applied to the rapid detection of surface defects of electronic products to improve product quality and production efficiency. Automobile manufacturing industry: in the process of automobile manufacturing, the present method can be used to detect surface defects of automobile bodies, such as scratches, dents, etc., to ensure that the appearance quality of automobiles meets the requirements. Glass manufacturing industry: in the production process of glass products, surface defects such as bubbles, scratches, cracks, etc. will affect the transparency and aesthetics of glass products. The present method can be used to quickly detect surface defects of glass products to ensure product quality. Plastic product manufacturing industry: in the production of plastic products, surface defects will reduce the appearance quality and service life of the product. The present method can be applied to the detection of surface defects of plastic products, such as bubbles, defects, dents, etc., to ensure that the product quality meets the standards. Pharmaceutical manufacturing industry: in the manufacturing process of medical devices and pharmaceutical packaging, surface defects may affect the hygiene and safety of the product. The present method can be used to detect surface defects of medical devices and pharmaceutical packaging to ensure that the product quality meets the medical industry standards. Food processing industry: In the process of food processing, surface defects of packaging such as bottled beverages and canned foods may affect the hygiene and quality of the product. This method can be applied to the detection of surface defects in food packaging to ensure that the product meets food safety standards. Textile manufacturing industry: In textile production, surface defects of fabrics such as clothing and household items can affect the appearance and comfort of the product. This method can be used to detect surface defects of textiles to improve product quality and market competitiveness.

[0050] like Figure 1As shown, the method includes: Step A: constructing a quality inspection model, collecting sample images of normal industrial products and industrial products containing different types of defects to form a data set, annotating the sample images, including marking whether the sample images have defects, the defect location and the defect type, and using the data set to train the quality inspection model;

[0051] Preferably, in step A, it also includes:

[0052] In a data set, if multiple sample images are simultaneously labeled with multiple defect types, then sample images with a single appearance of each defect type are added.

[0053] Preferably, in step A, using the data set to train the quality detection model also includes using an L2 regularization method or an early stopping strategy to optimize the training process.

[0054] In this embodiment, this method uses deep learning technology to train a quality inspection model for detecting different types of product defects. The quality inspection model is mainly based on a neural network model. The specific steps include data preparation, model selection and construction, model training, model evaluation, and model deployment;

[0055] The data preparation stage includes collecting image data of normal products and different types of defective products to ensure that the data set covers multiple defect types to improve the generalization ability of the model; annotating the collected images, including whether the sample has defects, defect types, defect locations and other information; it should be noted that it is necessary to adjust the characteristics of the product images, increase the diversity of samples, and prevent overfitting. For example, there may be two different types of defects in the data set, but these two types of defects always appear together. After training with such a data set, the quality inspection model obtained may only recognize one type of defect when dealing with the test set of these two different defect types. Therefore, it is necessary to increase the diversity of samples, that is, to increase the number of cases where these two defect types appear alone, and to increase the number of sample images where each defect type appears alone; then normalize the images (such as scaling the pixel values ​​to the range of [0,1]) to improve the training effect; divide the data set into training set, validation set and test set, usually 70% training, 15% validation, and 15% test.

[0056] In the model selection and construction stage, a suitable neural network architecture is selected according to actual needs. In this embodiment, a convolutional neural network is selected. The convolutional neural network architecture is constructed using a deep learning framework, and the network layers included in the architecture are defined: input layer, convolution layer, activation function, pooling layer, fully connected layer, output layer, configuration loss function and optimizer.

[0057] Input layer: receives raw data and passes it to the next layer. In image classification tasks, the input is usually image data with a shape of H×W×C. For example, the input layer can be defined as: Input(H, W, C), where H is the height of the image, W is the width of the image, and C is the number of channels (for example, the number of channels for RGB images is 3);

[0058] Convolutional layer: The convolutional layer is the most important layer in CNN, which extracts local features of the input data through convolution operations. Each convolution operation involves a filter (kernel) that slides on the input image to calculate the convolution output. For a single convolution operation, the output feature map is calculated as:

[0059]

[0060] Where X is the input feature map, W is the filter, b is the bias term, k is the size of the filter, and Y(i, j) is the output feature map;

[0061] Activation function: An activation function, such as ReLU (Rectified Linear Unit), is usually applied after the convolution layer to increase the nonlinear characteristics of the network. The ReLU activation function formula is: f(x) = max(0, x);

[0062] Pooling layer: Pooling layer usually follows convolutional layer to reduce the spatial dimension of feature map, reduce computational effort, and reduce overfitting. Max pooling is the most common pooling operation, which takes the maximum value in a local area. The calculation formula of max pooling is:

[0063] Y(i, j) = max m，n X(i+m,j+n);

[0064] Among them, X(i+m, j+n) is the input feature map and Y(i, j) is the output after pooling.

[0065] Fully connected layer: After the convolutional layer and the pooling layer, there are usually one or more fully connected layers to flatten the feature map and map it to the final output space. Each fully connected layer is a linear transformation, calculated as follows: y = W × x + b; where x is the input vector, W is the weight matrix, b is the bias term, and y is the output;

[0066] Output layer: The output layer is usually a classification problem or a regression problem, depending on the task. In this embodiment, it is a classification problem, that is, the splitting of defect types, and the Softmax activation function is usually used to output the probability distribution. The Softmax function formula is:

[0067]

[0068] Among them, Z k is the original score of each category in the output layer, and P(y = C k |x) is the predicted probability of this category.

[0069] Loss function: In this embodiment, the choice of the loss function is a classification task, and the commonly used loss function is the cross-entropy loss:

[0070] Among them, y is the true label, is the predicted probability.

[0071] Optimizer: The optimizer is used to adjust the weights and biases in the network to minimize the loss function. Commonly used optimizers include the stochastic gradient descent (SGD), Adam optimizer, etc. The update formula of the Adam optimizer is as follows:

[0072] m t = β 1 m t-1 +(1 - β 1 )g t ;

[0073]

[0074]

[0075] Among them, m t and V t are the first-order moment and second-order moment estimates of the gradient respectively, η is the learning rate, β 1 , β 2 is the decay coefficient, and ε is a small constant to prevent division by zero error.

[0076] Training the model: Use the training set data for training, and monitor the model performance through the validation set. The validation set is used to evaluate whether there is overfitting or underfitting in the training process of the model. After each epoch, use the validation set data to calculate the validation loss and validation accuracy to monitor the performance of the model on unseen data. If the performance of the validation set does not meet the standard, relevant parameters can be adjusted to optimize the training process. The adjustment strategies in this application include the L2 regularization method and the early stopping strategy;

[0077] The mathematical formula of the L2 regularization method is:

[0078]

[0079] L(θ) is the original loss function, which measures the fitting degree of the model to the training data.

[0080] is the sum of the squares of all model weights and n is the number of weights.

[0081] is a regularization strength hyperparameter that controls the size of the penalty. A larger value of λ will make the regularization stronger, resulting in smaller weights.

[0082] The next step is to use the gradient descent method for optimization. We need to calculate the gradient of the loss function for each parameter (weight). For L2 regularization, the gradient of the loss function is the gradient of the original loss function plus the gradient of the regularization term.

[0083] The original loss function gradient is

[0084] The gradient of the L2 regularization term is 2λθ i ;

[0085] Therefore, the total gradient after adding the regularization term is:

[0086] Finally, we update the weights using gradient descent (or its variants). In each iteration, we update the model weights θ′ as:

[0087] Among them, η is the learning rate, which determines the step size of each update.

[0088] Since the gradient of L2 regularization is 2λθ i , the weight update formula becomes:

[0089]

[0090] This term 2λθ i The role of is to weight θ i The regularization term “punishes” larger weights, forcing the model to choose smaller weight values, making the model more concise and reducing the risk of overfitting.

[0091] You can also use the early stopping strategy, the steps are as follows:

[0092] Set a "patience value" or "tolerance value" (e.g. 5 epochs). This means that if the validation set performance does not improve in consecutive n iterations (epochs), stop training. This can avoid unnecessary early stopping due to single validation fluctuations.

[0093] Initialize the model's parameters and start training the model using the training data. Typically, the model is trained for multiple epochs, with each epoch representing a complete pass through the dataset.

[0094] At the end of each epoch, the performance of the model on the validation set (usually loss or accuracy) is calculated. This is to determine the difference in performance between the model on the training set and the validation set, and to infer whether the model has begun to overfit.

[0095] Record the validation loss of each epoch and compare it with the previous best validation loss. If the validation loss of the current epoch is lower than the best historical validation loss, update the "best loss" and "best model parameters". If the validation loss of the current epoch does not improve, record the number of failed epochs.

[0096] If the validation loss does not improve for multiple consecutive epochs and exceeds the preset "patience value" (for example, 5 epochs), the model is considered to have begun to overfit and training should be stopped. At this time, restore the optimal model parameters (that is, the parameters with the smallest validation loss during training). This step is to avoid performance degradation caused by continuing training after overfitting.

[0097] Once the early stopping condition is met (i.e. the validation loss has not improved for n consecutive epochs), training is stopped. The model parameters with the smallest validation loss during training are returned to ensure optimal generalization performance.

[0098] During the model evaluation phase, the performance of the model is evaluated on the test set, and indicators such as accuracy and recall are calculated. A confusion matrix is ​​generated to analyze the classification effect; the model's misclassification cases are analyzed to understand the model's weaknesses and improve them.

[0099] Accuracy: Accuracy is the most intuitive performance metric in classification problems, indicating the proportion of samples correctly predicted by the model to the total number of samples. The basic formula is:

[0100] Accuracy = number of correctly classified samples / total number of samples;

[0101] In the binary classification problem, the accuracy is calculated as:

[0102] Accuracy=(TP+TN) / (TP+TN+FP+FN);

[0103] in:

[0104] TP (True Positive): True positive, the number of samples correctly predicted by the model as positive.

[0105] TN (True Negative): True negative, the number of samples correctly predicted by the model as negative.

[0106] FP (False Positive): False positive, the number of samples that the model incorrectly predicts as positive.

[0107] FN (False Negative): False negative, the number of samples that the model incorrectly predicts as negative.

[0108] Recall rate: Recall rate (also called true positive rate, TPR) indicates the proportion of samples that are correctly predicted as positive among all samples that are actually positive. Recall rate emphasizes the samples that are missed as positive. Formula:

[0109] Recall = TP / (TP+FN);

[0110] in:

[0111] TP: True Positive (the number of samples predicted by the model to be positive and actually positive);

[0112] FN: False Negative (the number of samples predicted by the model as negative but actually positive);

[0113] Confusion Matrix: The confusion matrix is ​​a visualization tool used to summarize the prediction results of a classification model. It shows the distribution of actual categories and model predicted categories, and is often used to evaluate the classification effect of a classification model. For a binary classification problem, the confusion matrix is ​​usually a 2x2 matrix, and the confusion matrix is ​​in the following form:

[0114]

[0115] Among them: The first row indicates the number of actual negative classes, predicted negative classes (TN), and predicted positive classes (FP). The second row indicates the number of actual positive classes, predicted negative classes (FN), and predicted positive classes (TP).

[0116] Deploy model: Export and configure the trained model in a suitable deployment format and environment, provide interfaces for real-time product detection, regularly monitor model performance, and maintain the normal operation of the model.

[0117] Step B: Collect images of the industrial products to be inspected and perform image preprocessing;

[0118] In this embodiment, a high-resolution industrial camera is used to collect images of industrial products to be inspected, and image correction is performed on the collected images. Available methods include geometric correction (such as distortion removal and perspective correction), color correction (such as white balance correction and color space conversion), and image registration. Image denoising is performed. Available methods include spatial domain denoising technology (such as mean filtering, median filtering, Gaussian filtering and other filtering technologies), frequency domain denoising technology (such as Fourier transform and wavelet transform), and statistical methods (such as Wiener filtering, corrosion and expansion). Image enhancement is performed. Available methods include contrast enhancement (such as histogram equalization and adaptive histogram equalization AHE), brightness adjustment (such as gamma correction and linear stretching), and edge enhancement (such as Laplace filtering, Sobel filtering, and high-pass filtering).

[0119] Step C: Input the preprocessed image into the trained quality inspection model. The quality inspection model performs image segmentation on the preprocessed image to separate the industrial product from the background area, and then divides the different modules of the industrial product itself to obtain the image area that needs to be analyzed;

[0120] Preferably, in step C, segmenting the industrial product from the background area comprises:

[0121] Step C1: Obtain the gradient amplitude G of each pixel in the preprocessed image;

[0122] Get the gradient direction of each pixel in the preprocessed image θ = arctan (G y / G x );

[0123] Step C2: Perform the following operation on each pixel in the image to obtain a gradient magnitude image:

[0124] For a pixel point, obtain the other two pixel points adjacent to the pixel point in the gradient direction, compare the gradient amplitude of the pixel point with the gradient amplitude of the two pixel points, and if the gradient amplitude of the pixel point is smaller than the gradient amplitude of the two pixel points, set the gradient amplitude of the pixel point to zero;

[0125] Set the high threshold T h and low threshold T l ;

[0126] If the gradient magnitude of a pixel in the gradient magnitude image is greater than the high threshold, that is, G>T h , then the pixel is defined as a strong edge;

[0127] If the gradient amplitude of a pixel in the gradient amplitude image is between the high threshold and the low threshold, that is,

[0128] , then the pixel is defined as a weak edge;

[0129] If the gradient magnitude of a pixel in the gradient magnitude image is less than the low threshold, G <T l , then the pixel is defined as a non-edge;

[0130] Step C3: Mark all strong edges, traverse each weak edge pixel in the gradient magnitude image, and if a weak edge pixel has a strong edge pixel in its area, mark the weak edge pixel as a strong edge, otherwise discard the weak edge pixel;

[0131] The refined edge between the industrial product and the background area in the preprocessed image is obtained according to the pixel points of the strong edge, and the industrial product and the background area are segmented based on the refined edge.

[0132] Preferably, in step C, different modules of the industrial product itself are divided to obtain image areas to be analyzed, including:

[0133] Step C4: determining an image of only the industrial product after removing the background area after processing from step C1 to step C3;

[0134] Step C5: setting the area threshold;

[0135] Given a seed pixel in an image containing only industrial products, for an adjacent pixel, if the similarity measure between the adjacent pixel and the seed pixel is less than a preset region threshold, the pixel and the seed pixel are merged into a seed region;

[0136] Find another adjacent pixel in the seed area, and continue to determine whether the similarity measure between it and a pixel in the seed area is less than the preset area threshold. If so, add the pixel to the current seed area, and continue to find the next adjacent pixel in the current seed area. Repeat this process until there are no adjacent pixels that can be added to the seed area.

[0137] The final seed area is used as a separate module area of ​​the industrial product, and other areas in the image except the seed area are used as other module areas.

[0138] Step C6: Repeat step C5 for other module areas until the other module areas are also divided into multiple independent module areas;

[0139] Step C7: Record each individual module area as an image area that needs to be analyzed and output.

[0140] Step D: According to the type of industrial product to be inspected and the multiple defect types to be clearly inspected, the quality inspection model calls the corresponding multiple inspection algorithms to perform defect inspection on the image area processed by step C;

[0141] Preferably, in said step D, it includes:

[0142] When there are multiple defect types to be detected for an industrial product, all defect types to be detected for the industrial product are identified, and the corresponding detection algorithm is called for each defect type for detection.

[0143] In this embodiment, the quality inspection of industrial products includes many types of defect detection, such as surface flaws, foreign matter, cracks and other defect types. For different defect types, we need to call different detection algorithms, but it should be noted that the execution of these algorithms is based on the processing of step C mentioned above. Taking defect detection as an example, we can determine whether there are defects by extracting the features of the image area to be analyzed; specifically, the features of the defects include size, texture, shape, color, etc. The feature extraction method includes analyzing the texture, detecting the edges in the image, and converting the color space of the image.

[0144] When the model performs defect detection, it first uses the feature extraction method to extract the hierarchical features of the image through multiple convolutional layers, from low-level features (edges, textures, etc.) to high-level features (shapes, patterns, etc.);

[0145] Including extracting texture features of defects, texture is a pattern in an image that reflects surface details or structural features, and extracting texture features by statistically describing pixel grayscale changes. This embodiment uses a grayscale co-occurrence matrix for texture analysis, which describes the spatial relationship between pixel pairs in an image. The texture features of the image are reflected by calculating the change in grayscale levels. In this embodiment, texture features include contrast, correlation, energy, and homogeneity. The specific operations are as follows:

[0146] If the image to be processed is a color image, the preprocessed image needs to be converted into a grayscale image;

[0147] Select a direction and a step size in the grayscale image (for example, 0°, 45°, 90°, 135° directions, with a step size of 1 pixel), calculate the grayscale value of each pair of pixels and the spatial relationship between each pair of pixels to construct a grayscale co-occurrence matrix;

[0148] In one step, due to the large amount of calculation of the grayscale co-occurrence matrix, it is usually necessary to quantize the grayscale of the image to reduce the amount of calculation, for example, quantize the 256-level grayscale image to 8 or 16 levels. Then determine the direction and step size, which determine the distance and direction between pixel pairs. In actual operation, arbitrarily take a pixel point in the grayscale image, find another pixel point according to the step size and direction, and obtain the grayscale value (g1, g2) of this pair of pixels. Traverse all the pixels in the grayscale image, find the corresponding pixel pair according to the step size and direction, and count the number of occurrences of the grayscale value of each pixel pair. According to the statistical results, construct an N x N matrix, N is the number of grayscale quantized levels, the rows and columns of the matrix represent the grayscale levels in the grayscale value, and the element values ​​in the matrix represent the number of occurrences of the grayscale levels represented by the rows and columns in all pixel pairs with specific directions and specific step sizes. The element values ​​of the grayscale co-occurrence matrix are normalized to obtain the probability of occurrence of the grayscale value of each pixel pair.

[0149] Extract texture features based on the gray-level co-occurrence matrix, including extracting:

[0150] Contrast: describes the roughness or variation of image texture;

[0151] Homogeneity: reflects the uniformity of image texture;

[0152] Energy: describes the repeatability of image texture;

[0153] Correlation: represents the dependency between the gray levels of image pixels.

[0154] In this embodiment, the contrast is extracted through the gray level co-occurrence matrix by substituting the value in the gray level co-occurrence matrix into the contrast formula for calculation to obtain the contrast characteristic value of the image. The contrast formula is:

[0155] =∑i,j(ij)2P(i,j), where P(i,j) represents the probability of occurrence of a pixel pair with grayscale values ​​g1 and g2 in the grayscale co-occurrence matrix, and (ij) represents the difference between the two grayscale values;

[0156] Homogeneity refers to the similarity between each element in the gray level co-occurrence matrix and its adjacent elements. It reflects the local uniformity and consistency of the image texture. In the gray level co-occurrence matrix, if the element value on the diagonal is large, it means that the gray values ​​of adjacent pixels are similar, that is, the image texture has a high homogeneity. The calculation formula is as follows: Hom = ∑i∑jp(i, j) / 1+(ij) 2 , in this formula, P(i,j) represents the probability of occurrence of adjacent pixel pairs with gray levels i and j in the gray level co-occurrence matrix;

[0157] The size of the energy value reflects the uniformity and repeatability of the image texture. When the element values ​​in the gray-level co-occurrence matrix are distributed more evenly, the image texture is more repeated, that is, the values ​​of each element are not much different, and the energy value will be relatively small, indicating that the image texture is more delicate. On the contrary, when some element values ​​in the gray-level co-occurrence matrix are significantly larger than other element values, the energy value will be larger, indicating that the image texture is relatively rough. The energy value is calculated as the sum of the squares of all element values ​​in the gray-level co-occurrence matrix.

[0158] The calculation formula of correlation can be Cor = ∑i∑j(i-μ i )(j-μ j )p(i, j) / σ, where P(i, j) represents the probability of occurrence of adjacent pixel pairs with gray levels i and j in the gray level co-occurrence matrix; μ i and μ j are the gray level means of the rows and columns of the gray level co-occurrence matrix respectively; σ i , σ j are the gray level standard deviations of the rows and columns of the gray level co-occurrence matrix respectively.

[0159] It also includes extracting the contour features of defects; the method used is the edge detection algorithm. Edge detection is the process of extracting the boundaries of objects in an image. It is usually used to identify the shape, structure and detail features in an image, namely the contour features. The edge is the area in the image where the brightness changes dramatically. It can effectively help identify the contour of the defect. The image is convolved by the Sobel operator to calculate the horizontal direction G of each pixel. x and vertical direction G y The gradient of

[0160] The gradient magnitude G of each pixel is calculated by the following formula:

[0161]

[0162] Set the edge threshold, and regard the area with gradient amplitude greater than the edge threshold as the edge area, and other areas as non-edge areas;

[0163] Determine the contour characteristics of the defect based on the edge area.

[0164] It also includes extracting color features of defects; color space conversion can extract meaningful color information by converting an image from one color space to another. For detecting defects, certain color channels may more significantly show defect features, such as the brightness channel in the RGB space or the hue and saturation in the HSV space. In this embodiment, such as RGB to HSV conversion: HSV (hue, saturation, brightness) is a way to express the color of an image. Compared with RGB, HSV is more in line with the human eye's perception of color. Through the HSV space, color features can be better distinguished, especially for defects of different colors. Specifically as follows:

[0165] The preprocessed image is converted from RGB image to HSV space using the following formula:

[0166]

[0167] 1 / (RB)×max(R,G,B)-min(R,G,B);

[0168] (RB) / 3;

[0169] Among them: H represents hue, S represents saturation, V represents brightness, R represents the intensity of the red component, G represents the intensity of the green component, and B represents the intensity of the blue component;

[0170] Identify areas of color anomalies in an image based on hue, saturation, and brightness to obtain color signatures of defects.

[0171] Through the above embodiments, we can know that for different defect types, we can call corresponding detection algorithms to perform detection.

[0172] Step E: the quality inspection model outputs defect levels of multiple defect types of the inspected industrial product, and determines whether the quality of the industrial product is qualified based on the defect levels of the multiple defect types;

[0173] Preferably, in said step E, it includes:

[0174] Each defect type is set with the same number of defect levels, and the defect level of each defect of each defect type is determined according to the defect range;

[0175] When an industrial product has at least one defect type, the defect levels of multiple defects of each defect type are the same or different;

[0176] In this embodiment, this method sets each defect type to a different defect level, but the number of defect levels for each defect type is the same. For example, defect types A and B are both set to 5 defect levels, where each defect level is set according to the defect range. For example, a high defect level indicates that the defect range of this defect type is larger.

[0177] Further, according to the test results, there may be the following types:

[0178] 1. The industrial product only detects one defect type A, but there are multiple defect types A, which also means that there are multiple defect levels of defect type A, and the levels of these defect levels may be the same or different;

[0179] 2. The industrial product is detected with multiple defect types, such as defect types A and B. There are multiple defect types A and B, which means there are multiple defect levels of A and multiple defect levels of B. Then the defect levels of A or B may be the same or different, or the defect level of A may be the same or different from the defect level of B.

[0180] Preferably, in said step E, it also includes:

[0181] Determine whether the maximum defect level of the current industrial product exceeds the maximum defect level allowed for the industrial product. If so, the quality of the current industrial product is deemed unqualified. If not, determine whether the sum of all defect levels of the current industrial product exceeds the preset qualified value. If so, the quality of the current industrial product is deemed unqualified. If not, the quality of the current industrial product is deemed qualified.

[0182] In this embodiment, we first determine whether the maximum defect level of the product has exceeded the maximum defect level allowed for qualified products. If it has exceeded, it is unqualified. If not, all defect levels of the industrial product are added to obtain a value to determine whether the value exceeds the preset qualified value. If so, we also determine that it is unqualified, otherwise it is qualified. Because the defect level is formulated based on the defect range, when the maximum defect level exceeds the maximum defect level allowed for qualified products, it means that the defect range of the defect type is too large and is obviously unqualified. When there is no defect with an obviously large defect range, we consider the sum of all defect levels, that is, the sum of the defect ranges of all defects. If the sum of the defect ranges is too large, it means that the industrial product may have too many defects, resulting in a large sum of defect ranges.

[0183] Step F: Develop a variety of quality decision strategies and implement corresponding quality decision strategies according to the quality qualification level of industrial products.

[0184] Preferably, in the step F, it includes:

[0185] For the industrial products that are found to be unqualified in step E, determine whether their maximum defect level exceeds the first repair value. If so, implement a discard strategy for the unqualified industrial products. If not, determine whether the number of all defects exceeds the second repair value. If so, implement a discard strategy for the unqualified industrial products. If not, implement a repair strategy for the unqualified industrial products.

[0186] In this embodiment, by formulating different decision-making strategies, it is possible to further determine whether these products that are determined to be of unqualified quality can still be repaired. When the maximum defect level of an industrial product exceeds the first repair value, it is determined that the defect range is too large and has exceeded the repair range. At this time, an abandonment strategy is adopted to abandon the industrial product. If the maximum defect level does not exceed the first repair value, it is also necessary to determine whether the number of defects exceeds the second repair value. If it exceeds, because when the number of defects is too large, it means that there are too many places that need to be repaired and the overall repair range is too large, such products also need to be abandoned. Otherwise, the repair strategy can be executed to repair the product and then re-execute defect detection.

[0187] In summary, this method has the following advantages:

[0188] 1. High-precision defect detection: This technology can accurately detect tiny defects or foreign matter, even imperceptible surface flaws, to ensure that product quality meets standard requirements.

[0189] 2. Fast detection speed: The use of efficient algorithms and hardware optimization enables product quality inspection to be completed in a very short time, thereby improving the throughput and production efficiency of the production line.

[0190] 3. Real-time feedback and adjustment: The system can monitor the performance indicators of the product testing process in real time. Once an abnormality is found, feedback and adjustment can be made immediately to ensure the stability and consistency of product quality testing.

[0191] 4. Low false alarm rate and missed alarm rate: After training and optimization with a large amount of sample data, the system can reduce the false alarm rate and missed alarm rate during the detection process, avoiding the misjudgment and missed detection of qualified products and improving the reliability of detection.

[0192] 5. Automated production line integration: Seamless integration with industrial production lines realizes automated product quality inspection, reduces manual intervention and operating errors, and improves the stability and controllability of the production line.

[0193] 6. Data-driven quality improvement: By recording and analyzing the test results and related information of each product, it can provide important data support for production management and quality improvement, helping enterprises to continuously improve product quality.

[0194] 7. Applicable to a variety of product types: This technology is not only suitable for specific types of product quality inspection, but also has a certain degree of versatility. It can be applied to a variety of industrial products of different shapes, materials and sizes, and has strong flexibility and applicability.

[0195] The technical principle of the present invention is described above in conjunction with specific embodiments. These descriptions are only for explaining the principle of the present invention and cannot be interpreted as limiting the scope of protection of the present invention in any way. Based on the explanations herein, those skilled in the art can associate other specific implementations of the present invention without paying creative labor, and these methods will fall within the scope of protection of the present invention.

Claims

1. A method for detecting the quality of industrial products, characterized in that: Step A: Build a quality inspection model, collect sample images of normal industrial products and industrial products with different types of defects to form a data set, annotate the sample images, including marking whether the sample images have defects, defect locations and defect types, and use the data set to train the quality inspection model; Step B: Collect images of the industrial products to be inspected and perform image preprocessing; Step C: Input the preprocessed image into the trained quality inspection model. The quality inspection model performs image segmentation on the preprocessed image to separate the industrial product from the background area, and then divides the different modules of the industrial product itself to obtain the image area that needs to be analyzed; Step D: According to the type of industrial product to be inspected and the multiple defect types to be clearly inspected, the quality inspection model calls the corresponding multiple inspection algorithms to perform defect inspection on the image area processed by step C; Step E: the quality inspection model outputs defect levels of multiple defect types of the inspected industrial product, and determines whether the quality of the industrial product is qualified based on the defect levels of the multiple defect types; Step F: Develop a variety of quality decision strategies and implement corresponding quality decision strategies according to the quality qualification level of industrial products.

2. The industrial product quality detection method according to claim 1, characterized in that: In the step A, it also includes: In a data set, if multiple sample images are simultaneously labeled with multiple defect types, then sample images with a single appearance of each defect type are added.

3. The industrial product quality detection method according to claim 1, characterized in that: In the step C, the industrial product and the background area are segmented, including: Step C1: Obtain the gradient amplitude G of each pixel in the preprocessed image; Get the gradient direction of each pixel in the preprocessed image θ = arctan (G y / G x ); Step C2: Perform the following operation on each pixel in the image to obtain a gradient magnitude image: For a pixel point, obtain the other two pixel points adjacent to the pixel point in the gradient direction, compare the gradient amplitude of the pixel point with the gradient amplitude of the two pixel points, and if the gradient amplitude of the pixel point is smaller than the gradient amplitude of the two pixel points, set the gradient amplitude of the pixel point to zero; Set the high threshold T h and low threshold T l ; If the gradient magnitude of a pixel in the gradient magnitude image is greater than the high threshold, that is, G>T h , then the pixel is defined as a strong edge; If the gradient amplitude of the pixel in the gradient amplitude image is between the high threshold and the low threshold, that is, T l <G<T h , then the pixel is defined as a weak edge; If the gradient magnitude of a pixel in the gradient magnitude image is less than the low threshold, G <T l , then the pixel is defined as a non-edge; Step C3: Mark all strong edges, traverse each weak edge pixel in the gradient magnitude image, and if a weak edge pixel has a strong edge pixel in its area, mark the weak edge pixel as a strong edge, otherwise discard the weak edge pixel; The refined edge between the industrial product and the background area in the preprocessed image is obtained according to the pixel points of the strong edge, and the industrial product and the background area are segmented based on the refined edge.

4. The industrial product quality detection method according to claim 3, characterized in that: In step C, different modules of the industrial product itself are divided to obtain image areas that need to be analyzed, including: Step C4: determining an image of only the industrial product after removing the background area after processing from step C1 to step C3; Step C5: setting the area threshold; Given a seed pixel in an image containing only industrial products, for an adjacent pixel, if the similarity measure between the adjacent pixel and the seed pixel is less than a preset region threshold, the pixel and the seed pixel are merged into a seed region; Find another adjacent pixel in the seed area, and continue to determine whether the similarity measure between it and a pixel in the seed area is less than the preset area threshold. If so, add the pixel to the current seed area, and continue to find the next adjacent pixel in the current seed area. Repeat this process until there are no adjacent pixels that can be added to the seed area. The final seed area is used as a separate module area of ​​the industrial product, and other areas in the image except the seed area are used as other module areas. Step C6: Repeat step C5 for other module areas until the other module areas are also divided into multiple independent module areas; Step C7: Record each individual module area as an image area that needs to be analyzed and output.

5. The industrial product quality detection method according to claim 1, characterized in that: In the step D, it includes: When there are multiple defect types to be detected for an industrial product, all defect types to be detected for the industrial product are identified, and the corresponding detection algorithm is called for each defect type for detection.

6. The industrial product quality testing method according to claim 1, characterized in that: In the step E, it includes: Each defect type is set with the same number of defect levels, and the defect level of each defect of each defect type is determined according to the defect range; When an industrial product has at least one defect type, the defect levels of multiple defects of each defect type are the same or different.

7. An industrial product quality inspection method according to claim 6, characterized in that: In the step E, it also includes: Determine whether the maximum defect level of the current industrial product exceeds the maximum defect level allowed for the industrial product. If so, the quality of the current industrial product is deemed unqualified. If not, determine whether the sum of all defect levels of the current industrial product exceeds the preset qualified value. If so, the quality of the current industrial product is deemed unqualified. If not, the quality of the current industrial product is deemed qualified.

8. An industrial product quality inspection method according to claim 7, characterized in that: In the step F, it includes: For the industrial products that are found to be unqualified in step E, determine whether their maximum defect level exceeds the first repair value. If so, implement a discard strategy for the unqualified industrial products. If not, determine whether the number of all defects exceeds the second repair value. If so, implement a discard strategy for the unqualified industrial products. If not, implement a repair strategy for the unqualified industrial products.

9. The industrial product quality testing method according to claim 1, characterized in that: In step A, the quality detection model is trained using the data set, and the training process is also optimized using an L2 regularization method or an early stopping strategy.

Images