Stainless steel surface defect detection method based on machine vision

Abstract

The invention discloses a stainless steel surface defect detection method based on machine vision, and relates to the field of material defect detection.The stainless steel surface defect detection method comprises the steps that stainless steel surface images are collected, the collected multispectral surface images are subjected to image fusion, and a multispectral composite image is generated; performing detail enhancement on the low-resolution image through a super-resolution reconstruction algorithm to obtain a high-resolution image set; positioning a defect area position from the high-resolution image set through a defect detection algorithm, and obtaining a defect position data set; extracting a historical image defect feature set and a real-time image defect feature set from the defect position data set, and constructing a defect classification model according to the historical image defect feature set; inputting the real-time image defect feature set into a defect classification model, and performing defect classification and evaluation to obtain a defect classification data set; and generating and distributing a defect detection report. By combining the multispectral imaging algorithm, the super-resolution reconstruction algorithm and the defect classification model, the automation, precision and efficiency of stainless steel surface defect detection are improved.

Description

Technical Field

[0001] The invention relates to the field of material defect detection, in particular to a stainless steel surface defect detection method based on machine vision. Background Art

[0002] In recent years, with the development of industrial automation and intelligent manufacturing, stainless steel surface defect detection, as a key link in quality control, has received widespread attention. Traditional defect detection methods usually rely on manual visual inspection or simple mechanical detection systems. These methods are not only inefficient, but also greatly affected by human factors, and it is difficult to ensure consistency and accuracy. With the continuous advancement of machine vision algorithms, defect detection methods based on image processing have gradually become mainstream. The machine vision system collects stainless steel surface images through camera equipment, and uses image processing and analysis algorithms to automatically identify and locate defects. With the development of multispectral imaging algorithms, super-resolution reconstruction algorithms and deep learning technologies, the accuracy and reliability of image detection have been significantly improved. Especially in the fusion of multispectral images, information from multiple bands can provide more details than a single spectrum, thereby improving the accuracy of defect detection.

[0003] Although there are many defect detection methods based on machine vision, such as defect identification using traditional image processing algorithms or deep learning methods, these methods still have some shortcomings. First, defect details in low-resolution images are easily lost, and traditional image enhancement methods are difficult to effectively restore the tiny details of the defects, resulting in limited accuracy of the detection results. Secondly, although the defect classification model based on deep learning can provide better performance, it still faces great challenges in the feature extraction stage, especially when there are many and complex defect types. The classification model is easily affected by noise, resulting in unstable classification results. In addition, the existing technology usually relies on fixed historical defect data sets for training, lacking sufficient real-time and dynamic adaptability. Due to these problems, the promotion of existing technologies in practical applications is subject to certain restrictions, especially in terms of refined detection and real-time feedback. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a stainless steel surface defect detection method based on machine vision to solve the problem of insufficient efficiency and accuracy of stainless steel surface defect detection in traditional solutions.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for detecting surface defects of stainless steel based on machine vision, which includes acquiring a surface image of stainless steel, performing image fusion on the acquired multispectral surface images, and generating a multispectral composite image; based on the multispectral composite image, performing detail enhancement on the low-resolution image through a super-resolution reconstruction algorithm, and obtaining a high-resolution image set; locating the position of the defect area from the high-resolution image set through a defect detection algorithm, and obtaining a defect position data set; using a feature extraction algorithm, extracting historical and real-time image defect feature sets from the defect position data set, and constructing a defect classification model based on the historical image defect feature set; inputting the real-time image defect feature set into the defect classification model, performing defect classification and evaluation, and obtaining a defect classification data set; and generating and distributing a defect detection report based on the defect classification data set.

[0008] As a preferred solution of the stainless steel surface defect detection method based on machine vision described in the present invention, the multispectral surface image includes stainless steel surface images in the visible light band, near infrared band and short-wave infrared band.

[0009] As a preferred solution of the stainless steel surface defect detection method based on machine vision of the present invention, the steps of collecting the stainless steel surface image, fusing the collected multi-spectral surface image, and generating a multi-spectral composite image are as follows:

[0010] Use a multispectral imaging sensor to collect images of the stainless steel surface to obtain a multispectral surface image;

[0011] The multispectral surface image is denoised and de-artifacted by using the median filtering algorithm;

[0012] The processed multispectral surface images are registered by feature point matching and mutual information maximization methods;

[0013] The multispectral image fusion algorithm based on guided filtering is used to synthesize the registered multispectral surface images into a multispectral composite image, which is expressed as:

[0014]

[0015] Where I(x, y) is the pixel value of the fused multispectral composite image at position (x, y), N is the number of spectra involved in the fusion, i is the index coefficient of the number of spectra involved in the fusion, e is the natural base, λ is the image influence weighting parameter, and I i (x, y) is the pixel value of the i-th spectral image at position (x, y), μ i is the local mean of the i-th spectral image, Z(x,y) is the normalization coefficient, x is the spatial horizontal coordinate of the image, and y is the spatial vertical coordinate of the image.

[0016] As a preferred solution of the stainless steel surface defect detection method based on machine vision described in the present invention, wherein: based on the multi-spectral composite image, the low-resolution image is enhanced in detail by a super-resolution reconstruction algorithm to obtain a high-resolution image set, and the specific steps are as follows:

[0017] Based on the nonlinear sparse tensor decomposition method, the multispectral composite image is decomposed into a low-resolution base layer tensor and a low-resolution texture layer tensor. The expression is:

[0018]

[0019] Among them, T is the tensor representation of the multispectral composite image, B is the low-resolution base layer tensor, S is the low-resolution texture layer tensor, ∈ is the noise term, and ||B|| TV is the total variation regularization of the base layer tensor, ||S|| 1,2 is the mixed sparse constraint of the texture layer tensor, is the Frobenius norm of the noise term, α is the balance parameter of the low-resolution texture layer, β is the balance parameter of the noise term, and st is the constrained symbol;

[0020] Based on the low-resolution texture layer tensor, a multi-scale residual dense network model is constructed to perform super-resolution reconstruction on the low-resolution texture layer to obtain the high-resolution texture layer. The expression is:

[0021] S HR =f1(f2(f3(S)));

[0022] Among them, S HR is the high-resolution texture layer after super-resolution reconstruction, f3 is the multi-scale convolution operation, f2 is the residual dense network operation, and f1 is the sub-pixel convolution upsampling operation;

[0023] The low-resolution base layer tensor is interpolated with details based on non-parametric kernel regression to obtain a high-resolution base layer. The expression is:

[0024]

[0025] Among them, B HR (x,y) is the pixel value of the high-resolution base layer, is the neighborhood range of the target pixel, x′ is the horizontal coordinate of the neighborhood pixel, y′ is the vertical coordinate of the neighborhood pixel, K(x―x′,y―y′) is the Gaussian kernel function, and B(x′,y′) is the pixel value of the low-resolution base layer of the neighborhood pixel;

[0026] The high-resolution base layer and the high-resolution texture layer are weightedly fused to generate a high-resolution image set.

[0027] As a preferred solution of the stainless steel surface defect detection method based on machine vision of the present invention, wherein: the defect detection algorithm is used to locate the defect area position from the high-resolution image set to obtain the defect position data set, and the specific steps are as follows:

[0028] Enhance the image contrast of high-resolution image sets through histogram equalization method;

[0029] The edges in the high-resolution image set are detected using the Canny edge detection algorithm and segmented using the Otsu threshold selection algorithm to highlight the defective areas;

[0030] Use the dilation and erosion methods in morphological operations to optimize edge detection results, and smooth defect areas through opening and closing operations;

[0031] The defect area of ​​the processed image is contour extracted to obtain a defect location data set.

[0032] As a preferred solution of the stainless steel surface defect detection method based on machine vision described in the present invention, wherein: the feature extraction algorithm is used to extract historical and real-time image defect feature sets from the defect location data set, and a defect classification model is constructed based on the historical image defect feature set. The specific steps are:

[0033] Based on the defect location data set, the geometric features of the defect area are calculated;

[0034] The surface texture features of the defective area are extracted using gray-level co-occurrence matrix and local binary pattern.

[0035] Obtain the color features of the defect area by using the color histogram and calculating the color mean variance;

[0036] The frequency domain features of the defect area are extracted through discrete Fourier transform and discrete wavelet transform;

[0037] Based on the geometric features, surface texture features, color features and frequency domain features of the defect area, the principal component analysis algorithm is used to perform feature dimension reduction to obtain the image defect feature set;

[0038] By time series cutting, the image defect feature set is divided into a historical image defect feature set and a real-time image defect feature set;

[0039] Select the RBF kernel function and optimize the hyperparameters through grid search to build a defect classification model based on support vector machine;

[0040] The defect feature set of historical images is divided into a training set and a test set. The training set is used to train the support vector machine classifier, and the cross-validation algorithm is used to verify the performance of the defect classification model.

[0041] As a preferred solution of the stainless steel surface defect detection method based on machine vision described in the present invention, wherein: the real-time image defect feature set is input into the defect classification model, defect classification and evaluation are performed, and a defect classification data set is obtained. The specific steps are:

[0042] Load the trained defect classification model and verify it using the test set;

[0043] Input the real-time image defect feature set into the verified defect classification model to perform defect classification prediction and output the category label and confidence score;

[0044] Based on the category labels and confidence scores, the defect area classification results are integrated to obtain the defect classification dataset.

[0045] As a preferred solution of the stainless steel surface defect detection method based on machine vision of the present invention, wherein: the defect detection report is generated and distributed based on the defect classification data set, and the specific steps are:

[0046] Collect defect data information from the defect classification data set and organize it into a structured data set;

[0047] Define the defect detection report structure, combine the structured data set, fill in the report template through automated tools, and generate the defect detection report;

[0048] Store defect inspection reports on a local server and distribute them via email.

[0049] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the stainless steel surface defect detection method based on machine vision as described in the first aspect of the present invention is implemented.

[0050] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the stainless steel surface defect detection method based on machine vision as described in the first aspect of the present invention is implemented.

[0051] The beneficial effects of the present invention are as follows: the present invention provides an efficient and accurate method for detecting stainless steel surface defects by combining a multispectral imaging algorithm, a super-resolution reconstruction algorithm, a defect detection algorithm, a feature extraction algorithm, and a defect classification model. The multispectral image fusion algorithm is used to improve the image quality, providing high-quality input for subsequent detail enhancement; the super-resolution reconstruction algorithm is used to recover more details from low-resolution images, improving image clarity; the defect detection algorithm is used to accurately locate the defect area, providing a reliable basis for subsequent feature extraction and classification; the feature extraction algorithm is used to conduct in-depth analysis of defects, construct a more accurate defect classification model, and continuously optimize it in combination with historical and real-time data; through defect classification and evaluation, a detailed defect detection report is generated, providing real-time and accurate data support for production and quality control. The present invention improves the automation, accuracy and efficiency of stainless steel surface defect detection, and provides further technical support for quality control in industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0053] Figure 1 This is a flow chart of the stainless steel surface defect detection method based on machine vision in Example 1.

[0054] Figure 2 This is a flow chart for locating the defect area position in Example 1. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0058] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a stainless steel surface defect detection method based on machine vision, comprising:

[0059] S1: Collect stainless steel surface images, perform image fusion on the collected multispectral surface images, and generate a multispectral composite image.

[0060] Specifically, the following steps are included:

[0061] S1.1: Use a multispectral imaging sensor to collect images of the stainless steel surface to obtain a multispectral surface image.

[0062] It should be understood that a multispectral imaging sensor is a high-resolution near infrared / visible light camera and a short-wave infrared camera combination.

[0063] S1.1.1: Multispectral surface images include stainless steel surface images in the visible light band, near infrared band, and short-wave infrared band.

[0064] Specifically, the multispectral imaging sensor needs to simultaneously capture images of the stainless steel surface in different spectral ranges, mainly including the visible light band (400-700nm), near infrared band (700-1500nm) and short-wave infrared band (1500-2500nm).

[0065] S1.2: De-noise and remove artifacts from the multispectral surface image using a median filtering algorithm.

[0066] Specifically, during the multispectral imaging process, noise or artifacts may occur due to factors such as uneven illumination and local defects of the sensor. The median filter algorithm is a nonlinear filtering algorithm that can effectively remove salt and pepper noise and other types of random noise while maintaining the clarity of the image edges. The filtering operation will select the median value in the neighborhood of the image pixel by pixel and replace the pixel value in the original image with this value, thereby effectively smoothing the image and reducing noise.

[0067] S1.3: The processed multispectral surface images are registered by feature point matching and mutual information maximization method.

[0068] Specifically, the feature point matching method detects significant feature points (such as corners, edges, etc.) in the image and uses the geometric transformation relationship between these feature points to find the alignment between the images. Mutual information maximization is a registration method based on image grayscale distribution statistics. It can automatically calculate the similarity between two images and then maximize the mutual information through optimization algorithms to achieve the best registration effect. Through these methods, multispectral images can be accurately aligned to eliminate image deviations caused by different imaging device angles or surface texture changes.

[0069] S1.4: A multispectral image fusion algorithm based on guided filtering is used to synthesize the registered multispectral surface images into a multispectral composite image, which is expressed as:

[0070]

[0071] Where I(x, y) is the pixel value of the fused multispectral composite image at position (x, y), N is the number of spectra involved in the fusion, i is the index coefficient of the number of spectra involved in the fusion, e is the natural base, λ is the image influence weighting parameter, and I i (x, y) is the pixel value of the i-th spectral image at position (x, y), μ i is the local mean of the i-th spectral image, Z(x,y) is the normalization coefficient, x is the spatial horizontal coordinate of the image, and f is the spatial vertical coordinate of the image.

[0072] Preferably, guided filtering is an efficient image fusion algorithm that can effectively suppress inter-spectral interference and is suitable for multispectral image fusion. Compared with traditional image fusion methods, it can better maintain the edge information and details of the image, and has low computational complexity and can meet real-time requirements. The weighted items can dynamically adjust the weights according to the quality of different spectral images, thereby ensuring the quality of the fused image and improving the comprehensive performance of multispectral images, especially the recognition of tiny defects.

[0073] Preferably, the images of the stainless steel surface in the visible light, near infrared and short-wave infrared bands are obtained through multispectral imaging algorithms, and the median filter algorithm is used to remove noise and artifacts to ensure image quality. Image registration is performed through feature point matching and mutual information maximization methods to ensure the precise alignment of different spectral images and provide high-quality input for subsequent processing. A multispectral image fusion algorithm based on guided filtering is used to synthesize high-quality multispectral composite images, effectively retain image details and edge information, and improve the overall performance of the image by dynamically adjusting the weighting items.

[0074] S2: Based on the multispectral composite image, the super-resolution reconstruction algorithm is used to enhance the details of the low-resolution image to obtain a high-resolution image set.

[0075] Specifically, the following steps are included:

[0076] S2.1: Based on the nonlinear sparse tensor decomposition method, the multispectral composite image is decomposed into a low-resolution base layer tensor and a low-resolution texture layer tensor, which is expressed as:

[0077]

[0078] Among them, T is the tensor representation of the multispectral composite image, B is the low-resolution base layer tensor, S is the low-resolution texture layer tensor, ∈ is the noise term, and ||B|| TV is the total variation regularization of the base layer tensor, ||S|| 1,2 is the mixed sparse constraint of the texture layer tensor, is the Frobenius norm of the noise term, α is the balance parameter of the low-resolution texture layer, β is the balance parameter of the noise term, and st is the constrained symbol.

[0079] Preferably, the multispectral composite image T is decomposed by a tensor decomposition method to obtain a low-resolution base layer tensor B and a texture layer tensor S, as well as a noise term ∈. ||B|| TV The smoothness of the image is preserved and the noise is reduced, ||S|| 1,2 The purpose is to extract the detailed information of the image. By minimizing the above loss function, the base layer and texture layer of the low-resolution image are optimized, thereby achieving image decomposition and obtaining low-resolution image components that are more suitable for super-resolution reconstruction.

[0080] S2.2: Based on the low-resolution texture layer tensor, a multi-scale residual dense network model is constructed to perform super-resolution reconstruction on the low-resolution texture layer to obtain a high-resolution texture layer. The expression is:

[0081] S HR =f1(f2(f3(S)));

[0082] Among them, S HR is the high-resolution texture layer after super-resolution reconstruction, f3 is the multi-scale convolution operation, f2 is the residual dense network operation, and f1 is the sub-pixel convolution upsampling operation.

[0083] Preferably, a low-resolution texture layer tensor S is used as input to construct a multi-scale residual dense network (MDRN) for super-resolution reconstruction. The network contains multiple convolutional layers and residual modules, which can effectively learn the multi-scale features of the texture layer and enhance the details. Among them, f3 is a multi-scale convolution operation, which uses convolution kernels of different scales to capture details of different levels in the image; f2 is a residual dense network operation, which extracts deeper features through dense connections and accelerates convergence; f1 is a sub-pixel convolution upsampling operation, which converts the low-resolution image into a high-resolution image through convolution upsampling. The texture layer is super-resolved through the above network to obtain a high-resolution texture layer S HR , providing high-resolution texture information for subsequent fusion.

[0084] Furthermore, the specific steps of the f3 multi-scale convolution operation are:

[0085] The input low-resolution texture layer S is processed through a series of convolutional layers (each layer uses convolution kernels of different sizes).

[0086] Each convolutional layer passes the low-resolution texture layer tensor through the convolution operation while preserving its multi-scale features.

[0087] For each scale of convolution kernel, different strides and kernel sizes are used to accommodate details of different sizes. For example:

[0088] Small-sized convolution kernels (such as 3x3, 5x5) capture local details and texture features;

[0089] Large-sized convolution kernels (such as 7x7, 9x9) capture global information, such as textures and shapes over a larger range.

[0090] The output of each scale convolution is processed with an activation function (such as ReLU) to enhance feature expression.

[0091] The specific steps of f2 residual dense network operation are:

[0092] In each residual module, the input information is directly passed to the subsequent layers through skip connections, which alleviates the gradient vanishing problem and accelerates convergence. Each residual module contains convolution operations, activation functions (such as ReLU), and batch normalization operations.

[0093] The output of each layer is not only passed to the next layer, but also concatenated with the output of all previous layers to strengthen the flow of information. In this way, the network is able to extract more features at each layer.

[0094] Using multiple layers of residual dense modules, each layer is densely connected with other layers to build a deep network. The output of each residual module is passed to the subsequent layers, and finally the final feature map is output through the activation function (ReLU or Leaky ReLU).

[0095] The specific steps of f1 sub-pixel convolution upsampling operation are:

[0096] In the convolution operation, multiple convolution kernels are used to convolve the low-resolution image to obtain feature maps of multiple channels.

[0097] The pixel shuffle algorithm is used to convert the multi-channel feature map obtained by convolution into a high-resolution image.

[0098] Rearrange multiple channels of information for each pixel in an image into a new high-resolution output.

[0099] S2.3: Perform detail interpolation based on non-parametric kernel regression on the low-resolution base layer tensor to obtain a high-resolution base layer, which is expressed as:

[0100]

[0101] Among them, B HR (x,y) is the pixel value of the high-resolution base layer, is the neighborhood range of the target pixel, x′ is the horizontal coordinate of the neighborhood pixel, y′ is the vertical coordinate of the neighborhood pixel, K(x―x′,y―y′) is the Gaussian kernel function, and B(x′,y′) is the pixel value of the low-resolution base layer of the neighborhood pixel.

[0102] S2.4: Weightedly fuse the high-resolution base layer and the high-resolution texture layer to generate a high-resolution image set.

[0103] Specifically, the high-resolution base layer and the high-resolution texture layer are weightedly fused based on the local texture complexity, and the expression is:

[0104]

[0105] in, is the gradient modulus of the high-resolution texture layer, is the gradient modulus of the high-resolution base layer, I HR (x, y) is a high-resolution image, which is integrated into a high-resolution image set I HR .

[0106] Preferably, a method based on nonlinear sparse tensor decomposition is used to decompose the low-resolution image into a base layer and a texture layer to optimize the global structure and local details of the image. A multi-scale residual dense network is used to super-resolve the texture layer, and deep features are extracted through multi-scale convolution and dense connection to enhance details. The base layer is interpolated for details through non-parametric kernel regression to improve its resolution. A weighted fusion method based on local texture complexity is used to dynamically balance the contributions of the base layer and the texture layer, effectively enhancing details and maintaining smoothness.

[0107] S3: Using the defect detection algorithm, the defect area is located from the high-resolution image set to obtain a defect location data set.

[0108] Specifically, the following steps are included:

[0109] S3.1: Enhance the image contrast of high-resolution image sets by histogram equalization method.

[0110] Specifically, the grayscale histogram of the high-resolution image set is first calculated to represent the frequency of occurrence of different grayscale values. By accumulating the histogram, the mapping relationship of the grayscale levels is calculated to make the grayscale distribution of the image tend to be uniform and increase the visibility of low-contrast areas. The equalized grayscale value is applied to the original image to generate an image with enhanced contrast.

[0111] S3.2: The edges in the high-resolution image set are detected using the Canny edge detection algorithm and segmented using the Otsu threshold selection algorithm to highlight the defective areas.

[0112] Specifically, the Canny edge detection algorithm is used to calculate the gradient value of the image and detect the edges in the image. The Canny algorithm identifies the edges in the image through steps such as Gaussian filtering, gradient calculation, non-maximum suppression, and double threshold segmentation. Based on the edge detection results, the Otsu threshold segmentation method is applied to automatically calculate the optimal threshold and segment the image into foreground and background. The Otsu method automatically selects the threshold by maximizing the inter-class variance to effectively distinguish the defective areas. The defective areas are highlighted through the segmented binary image, and these areas are potential defect locations.

[0113] S3.3: Use the dilation and erosion methods in morphological operations to optimize the edge detection results, and smooth the defect area through opening and closing operations.

[0114] Specifically, dilation operation: dilation operation is used to expand the edge area, fill small gaps on the edge, and enhance the visibility of the defect area. The dilation operation uses a structural element (such as a 3x3 matrix) to expand the pixel values ​​near the edge to the surrounding area of ​​the structural element.

[0115] Erosion operation: Through the erosion operation, the edge area is shrunk, small noise and irregular edges are removed, and the defect area is refined. The pixel values ​​around the edge are shrunk using the structural element.

[0116] Opening operation: Combine the erosion and dilation operations, first erosion and then dilation, to remove small noise points and smooth the edges of the defective area.

[0117] Closing operation: Dilation is performed first and then erosion is performed to fill small gaps in the defect area and further smooth the defect area.

[0118] S3.4: Extract the contour of the defect area of ​​the processed image to obtain a defect position data set.

[0119] Specifically, use a contour extraction algorithm (such as OpenCV's findContours function) to extract the contour of the defect area in the binary image. The contour extraction algorithm divides the connected areas in the image into contours based on the continuity of the edges. Filter out the defect contours based on the size, shape and other features of the contours. Set the minimum and maximum area thresholds to remove areas that are too small or too large. Extract the coordinate information of each defect contour, including its location (such as center coordinates, bounding box location) and other related features (such as area, perimeter, shape, etc.). Store the extracted defect location data in JSON format to facilitate subsequent defect analysis, statistics or report generation.

[0120] Preferably, the image contrast of the high-resolution image set is first enhanced by histogram equalization to make the low-contrast area more obvious; Canny edge detection and Otsu threshold segmentation are used to automatically identify and highlight the defect area; morphological operations (dilation, erosion, opening and closing operations) are used to optimize the edge detection results and refine the shape of the defect area; and the contour extraction algorithm is used to obtain an accurate defect location data set to provide reliable data for subsequent analysis and repair. This process is automated, accurate and efficient, providing strong technical support for industrial inspection and quality control.

[0121] S4: Use feature extraction algorithm to extract historical and real-time image defect feature sets from the defect location dataset, and build a defect classification model based on the historical image defect feature sets.

[0122] Specifically, the following steps are included:

[0123] S4.1: Based on the defect location data set, calculate the geometric characteristics of the defect area.

[0124] Specifically, based on the acquired defect location data set, the geometric features of each defect area are calculated, including area, perimeter, aspect ratio, shape factor, circularity, etc.

[0125] Area: Calculates the total number of pixels in the defect area, reflecting the size of the defect.

[0126] Perimeter: The boundary length of the defect area is calculated through the contour extraction algorithm, reflecting the complexity of the defect shape.

[0127] Aspect ratio: The aspect ratio of the defect area helps to identify the shape of the defect, such as whether it is a bar, circle, etc.

[0128] Shape factor: The shape regularity of a defect is calculated based on the ratio of perimeter to area, and is often used to distinguish different types of defects.

[0129] Circularity: Measures whether the shape of the defect area is close to a circle. Defects with high circularity may represent different defect types.

[0130] S4.2: Use gray level co-occurrence matrix and local binary pattern to extract surface texture features of defect area.

[0131] Specifically, Gray Level Co-occurrence Matrix (GLCM): The spatial relationship of the grayscale values ​​of the image is described by calculating the gray level co-occurrence matrix of the defect area, and common texture features such as contrast, correlation, energy and uniformity are extracted.

[0132] Local Binary Pattern (LBP): Encodes the binary pattern of the local area of ​​each pixel to extract the texture information of the area. LBP can effectively capture texture features and is especially suitable for images with complex contrast and texture changes.

[0133] S4.3: Obtain the color features of the defect area by using the color histogram and calculating the color mean variance.

[0134] Specifically, color histogram: Calculate the color histogram of the defect area to describe the distribution of different colors in the image. Use RGB color space or other color space (such as HSV) for color distribution analysis.

[0135] Color mean and variance: Calculate the color mean and variance of the defect area to quantify the color uniformity and color distribution changes in the defect area. It is suitable for defect identification with high color contrast.

[0136] S4.4: Extract the frequency domain features of the defect area through discrete Fourier transform and discrete wavelet transform.

[0137] Specifically, Discrete Fourier Transform (DFT): converts the image from the spatial domain to the frequency domain, analyzes the frequency components in the image, and captures the high-frequency and low-frequency characteristics of the image.

[0138] Discrete Wavelet Transform (DWT): Use a multi-scale approach to perform wavelet transform on the defect area and extract frequency domain features at different scales. DWT can effectively capture the local frequency information of the image and has good adaptability to different types of defect recognition.

[0139] S4.5: Based on the geometric features, surface texture features, color features and frequency domain features of the defect area, the principal component analysis algorithm is used to perform feature dimensionality reduction to obtain the image defect feature set.

[0140] Specifically, the geometric features, surface texture features, color features, and frequency domain features of the extracted defect area are combined into a high-dimensional feature vector through feature addition, and the principal component analysis (PCA) algorithm is used to reduce the feature dimension, extract the main features, reduce the data dimension, and retain the core information of the data. PCA maps the original feature space to the new space through linear transformation, retains the principal component with the largest data variance, and removes redundant information, thereby improving the efficiency and accuracy of the defect classification model.

[0141] S4.6: Divide the image defect feature set into a historical image defect feature set and a real-time image defect feature set through time series cutting.

[0142] Specifically, the defect feature set is divided into historical data and real-time data according to the time sequence. The historical image defect feature set contains the defect feature data within a certain period of time in the past, which is used for training and defect classification model construction; the real-time image defect feature set contains the data of the latest image, which is used for real-time prediction of the defect classification model.

[0143] S4.7: Select the RBF kernel function and optimize the hyperparameters through grid search to build a defect classification model based on support vector machine.

[0144] Specifically, the support vector machine (SVM) algorithm is used for defect classification, and the radial basis function (RBF) kernel function is selected because the RBF kernel can effectively process nonlinear data. The grid search method is used to optimize the hyperparameters of the defect classification model, such as the kernel function parameters C (penalty parameter) and γ (kernel function width). Through grid search, multiple hyperparameter combinations are traversed to select the parameter combination that achieves the best classification effect.

[0145] Furthermore, the penalty parameter C controls the classifier's tolerance to misclassification. A smaller C value means a higher tolerance to misclassification, and the defect classification model may be simpler (high bias); a larger C value means a lower tolerance to misclassification, and the defect classification model is more complex (low bias, high variance).

[0146] The width γ of the kernel function determines the "crowding" of the high-dimensional feature space. If γ is too large, the defect classification model may be overfitted (low bias, high variance); if γ is too small, the defect classification model may be underfitted.

[0147] S4.8: Divide the historical image defect feature set into a training set and a test set, use the training set to train the support vector machine classifier, and use the cross-validation algorithm to verify the performance of the defect classification model.

[0148] Specifically, the historical image defect feature set is divided into a training set and a test set, with 80% of the data used for training and 20% for testing. The performance of the defect classification model is evaluated using indicators such as accuracy, precision, recall, and F1 value, and the stability and generalization ability of the defect classification model are evaluated using k-fold cross validation. By repeating the training and validation process, the risk of overfitting of the defect classification model can be reduced.

[0149] Preferably, by calculating the geometric features, texture features, color features and frequency domain features of the defects, the shape, surface texture, color distribution and frequency information of the defects are fully described, providing multi-dimensional feature support for the defect classification model. Then, principal component analysis is used to reduce the dimension of features, reduce redundant information and improve the efficiency of the defect classification model. The time series cutting method divides historical data and real-time data, making the system dynamically adaptable. Support vector machines are used for defect classification, and hyperparameters are optimized through grid search to ensure the stability and high performance of the defect classification model. The generalization ability of the defect classification model is verified by cross-validation, which further improves the detection accuracy and reliability.

[0150] S5: Input the real-time image defect feature set into the defect classification model, perform defect classification and evaluation, and obtain a defect classification data set.

[0151] Specifically, the following steps are included:

[0152] S5.1: Load the trained defect classification model and verify it using the test set.

[0153] Specifically, the trained defect classification model is loaded through PyTorch, and the defect classification model is quickly verified using the pre-prepared test set data to evaluate its performance on the test data, thereby ensuring the effectiveness and stability of the defect classification model in real scenarios.

[0154] S5.2: Input the real-time image defect feature set into the verified defect classification model, perform defect classification prediction, and output the category label and confidence score.

[0155] Specifically, the defect classification model will predict the defect category based on the input features and generate the corresponding category label. Category labels include "crack", "stain", "corrosion", etc. In addition, the defect classification model will also output a confidence score, which indicates the confidence level of the classification result. The higher the confidence score, the more confident the defect classification model is in the prediction result.

[0156] S5.3: Based on the category labels and confidence scores, integrate the defect area classification results to obtain the defect classification dataset.

[0157] Specifically, the category label and confidence score are associated with the ID or unique identifier of the image itself in the defect area classification result, combined with the defect area classification result, and stored in a JSON format file.

[0158] Preferably, through the above steps, the defect classification model is classified and evaluated based on the real-time image defect feature set, ensuring the effectiveness and stability of the defect classification model on the test set. Real-time defect classification prediction can output category labels and confidence scores to provide fast feedback for practical applications. By associating the classification results with the image ID and storing them in JSON format, the data is standardized and systematized, which is convenient for subsequent analysis, report generation and fault tracking.

[0159] S6: Generate and distribute defect detection reports based on the defect classification dataset.

[0160] Specifically, the following steps are included:

[0161] S6.1: Collect defect data information from the defect classification dataset and organize it into a structured dataset.

[0162] Specifically, the defect data information is extracted from the generated defect classification data set, including the defect category, location, confidence score, detection time, etc. The record of each defect should include the defect ID, category label, defect location area, confidence score, image ID or file name, and detection timestamp.

[0163] S6.2: Define the defect detection report structure, combine the structured data set, fill in the report template through automated tools, and generate the defect detection report.

[0164] Specifically, the defect detection report template is defined according to business requirements. The report should include the following core parts:

[0165] Report title: such as "Defect Inspection Report" and the relevant project name or image set number.

[0166] Inspection Summary: Briefly describe the inspection task, purpose, images involved, defect types, etc.

[0167] Defect Analysis: Summarizes the inspection results by category and lists detailed information for each defect, including defect category, location, confidence score, etc.

[0168] Defect statistics: summarizes the total number of defects detected, the number distributed by category, severity, and other data.

[0169] Defect example image: Embed defect images in the report and mark the defect areas as needed.

[0170] Suggestions and handling: Provide handling suggestions or follow-up work guidelines.

[0171] The automation tool refers to the Jinja2 template engine or ReportLab library in Python. The scripts include: automatically reading structured data sets; automatically filling in various parts of the report according to the classification and statistical data of defects; dynamically generating charts, tables and example graphs to enhance the visualization of the report.

[0172] S6.3: Store the defect inspection report in a local server and distribute it via email.

[0173] Specifically, the defect detection report is stored in the database, and emails are sent to relevant personnel or teams through Python's smtplib library.

[0174] Preferably, by extracting defect data information from defect classification data sets and organizing them into structured data sets, the traceability and integrity of defect data are ensured. Defining and filling report templates and generating defect detection reports through automated tools eliminates errors and time delays in manual processing and improves the efficiency and consistency of report generation. Automatic filling of report templates and dynamic generation of charts and tables improve the visualization of reports and facilitate users to understand and analyze detection results. By storing reports on local servers and distributing them via email, timely sharing and efficient distribution of reports are ensured.

[0175] This embodiment also provides a computer device, which is suitable for the case of a stainless steel surface defect detection method based on machine vision, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the stainless steel surface defect detection method based on machine vision proposed in the above embodiment.

[0176] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0177] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for detecting stainless steel surface defects based on machine vision as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0178] In summary, the present invention provides an efficient and accurate method for detecting stainless steel surface defects by combining multispectral imaging algorithms, super-resolution reconstruction algorithms, defect detection algorithms, feature extraction algorithms, and defect classification models. The multispectral image fusion algorithm is used to improve the quality of the image, providing high-quality input for subsequent detail enhancement; the super-resolution reconstruction algorithm is used to restore more details from low-resolution images, improving image clarity; the defect detection algorithm is used to accurately locate the defect area, providing a reliable basis for subsequent feature extraction and classification; the feature extraction algorithm is used to conduct in-depth analysis of defects, construct a more accurate defect classification model, and continuously optimize it in combination with historical and real-time data; through defect classification and evaluation, a detailed defect detection report is generated, providing real-time and accurate data support for production and quality control. The present invention improves the automation, accuracy, and efficiency of stainless steel surface defect detection, and provides further technical support for quality control in industrial production.

[0179] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting stainless steel surface defects based on machine vision, characterized in that: include, Collect stainless steel surface images, perform image fusion on the collected multispectral surface images, and generate multispectral composite images; Based on multispectral composite images, the low-resolution images are enhanced in detail through super-resolution reconstruction algorithms to obtain high-resolution image sets; Through the defect detection algorithm, the defect area is located from the high-resolution image set to obtain the defect location data set; Use feature extraction algorithms to extract historical and real-time image defect feature sets from defect location data sets, and build a defect classification model based on the historical image defect feature sets; Input the real-time image defect feature set into the defect classification model to perform defect classification and evaluation to obtain a defect classification data set; Generate and distribute defect detection reports based on defect classification datasets.

2. The method for detecting stainless steel surface defects based on machine vision according to claim 1, characterized in that: The multi-spectral surface image includes stainless steel surface images in a visible light band, a near infrared band and a short-wave infrared band.

3. The method for detecting stainless steel surface defects based on machine vision according to claim 2, characterized in that: The method of collecting the stainless steel surface image and fusing the collected multi-spectral surface image to generate a multi-spectral composite image comprises the following specific steps: Use a multispectral imaging sensor to collect images of the stainless steel surface to obtain a multispectral surface image; The multispectral surface image is denoised and de-artifacted by using the median filtering algorithm; The processed multispectral surface images are registered by feature point matching and mutual information maximization methods; The multispectral image fusion algorithm based on guided filtering is used to synthesize the registered multispectral surface images into a multispectral composite image, which is expressed as: Where I(x, y) is the pixel value of the fused multispectral composite image at position (x, y), N is the number of spectra involved in the fusion, i is the index coefficient of the number of spectra involved in the fusion, e is the natural base, λ is the image influence weighting parameter, and I i (x, y) is the pixel value of the i-th spectral image at position (x, y), μ i is the local mean of the i-th spectral image, Z(x,y) is the normalization coefficient, x is the spatial horizontal coordinate of the image, and y is the spatial vertical coordinate of the image.

4. The method for detecting stainless steel surface defects based on machine vision according to claim 3, characterized in that: Based on the multispectral composite image, the low-resolution image is enhanced in detail by a super-resolution reconstruction algorithm to obtain a high-resolution image set. The specific steps are: Based on the nonlinear sparse tensor decomposition method, the multispectral composite image is decomposed into a low-resolution base layer tensor and a low-resolution texture layer tensor. The expression is: Among them, T is the tensor representation of the multispectral composite image, B is the low-resolution base layer tensor, S is the low-resolution texture layer tensor, ∈ is the noise term, and ||B|| TV is the total variation regularization of the base layer tensor, ||S|| 1,2 is the mixed sparse constraint of the texture layer tensor, is the Frobenius norm of the noise term, α is the balance parameter of the low-resolution texture layer, β is the balance parameter of the noise term, and st is the constrained symbol; Based on the low-resolution texture layer tensor, a multi-scale residual dense network model is constructed to perform super-resolution reconstruction on the low-resolution texture layer to obtain the high-resolution texture layer. The expression is: S HR =f1(f2(f3(S))); Among them, S HR is the high-resolution texture layer after super-resolution reconstruction, f3 is the multi-scale convolution operation, f2 is the residual dense network operation, and f1 is the sub-pixel convolution upsampling operation; The low-resolution base layer tensor is interpolated with details based on non-parametric kernel regression to obtain a high-resolution base layer. The expression is: Among them, B HR (x,y) is the pixel value of the high-resolution base layer, is the neighborhood range of the target pixel, x′ is the horizontal coordinate of the neighborhood pixel, y′ is the vertical coordinate of the neighborhood pixel, K(x―x′,y―y′) is the Gaussian kernel function, and B(x′,y′) is the pixel value of the low-resolution base layer of the neighborhood pixel; The high-resolution base layer and the high-resolution texture layer are weightedly fused to generate a high-resolution image set.

5. The method for detecting stainless steel surface defects based on machine vision according to claim 4, characterized in that: The defect detection algorithm is used to locate the defect area from the high-resolution image set to obtain the defect location data set. The specific steps are: Enhance the image contrast of high-resolution image sets through histogram equalization method; The edges in the high-resolution image set are detected using the Canny edge detection algorithm and segmented using the Otsu threshold selection algorithm to highlight the defective areas; Use the dilation and erosion methods in morphological operations to optimize edge detection results, and smooth defect areas through opening and closing operations; The defect area of ​​the processed image is contour extracted to obtain a defect location data set.

6. The method for detecting stainless steel surface defects based on machine vision according to claim 5, characterized in that: The feature extraction algorithm is used to extract the historical and real-time image defect feature sets from the defect location data set, and a defect classification model is constructed based on the historical image defect feature sets. The specific steps are: Based on the defect location data set, the geometric features of the defect area are calculated; The surface texture features of the defective area are extracted using gray-level co-occurrence matrix and local binary pattern. Obtain the color features of the defect area by using the color histogram and calculating the color mean variance; The frequency domain features of the defect area are extracted through discrete Fourier transform and discrete wavelet transform; Based on the geometric features, surface texture features, color features and frequency domain features of the defect area, the principal component analysis algorithm is used to perform feature dimension reduction to obtain the image defect feature set; By time series cutting, the image defect feature set is divided into a historical image defect feature set and a real-time image defect feature set; Select the RBF kernel function and optimize the hyperparameters through grid search to build a defect classification model based on support vector machine; The defect feature set of historical images is divided into a training set and a test set. The training set is used to train the support vector machine classifier, and the cross-validation algorithm is used to verify the performance of the defect classification model.

7. The method for detecting stainless steel surface defects based on machine vision according to claim 6, characterized in that: The real-time image defect feature set is input into the defect classification model to perform defect classification and evaluation to obtain a defect classification data set. The specific steps are: Load the trained defect classification model and verify it using the test set; Input the real-time image defect feature set into the verified defect classification model to perform defect classification prediction and output the category label and confidence score; Based on the category labels and confidence scores, the defect area classification results are integrated to obtain the defect classification dataset.

8. The method for detecting stainless steel surface defects based on machine vision according to claim 7, characterized in that: The defect detection report is generated and distributed based on the defect classification data set. The specific steps are: Collect defect data information from the defect classification data set and organize it into a structured data set; Define the defect detection report structure, combine the structured data set, fill in the report template through automated tools, and generate the defect detection report; Store defect inspection reports on a local server and distribute them via email.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the stainless steel surface defect detection method based on machine vision according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the stainless steel surface defect detection method based on machine vision according to any one of claims 1 to 8 are implemented.

Images