A metal mesh defect detection method based on structural contrast information stacking

Abstract

The application discloses a metal mesh defect detection method based on structural contrast information stacking, and steps are as follows: metal mesh images are shot by a microscope; input images are blocked, neighborhood structural contrast calculation is carried out on each sub-block image, and a difference matrix of the input image is obtained; the position of the sub-block is displaced in different sizes, neighborhood structural contrast calculation is carried out, a plurality of difference matrixes are obtained, and the results of each layer are superimposed to obtain a priori graph; a robust principal component analysis method is used in combination with the priori graph to decompose the input image, and a low-rank, sparse and noise image is obtained; a binary mask is constructed by using the priori graph, the sparse image is filtered to obtain a saliency map; and threshold segmentation is carried out on the saliency map to obtain a binary detection result. The application does not need a training process, can detect a plurality of defects of the metal mesh, can be generalized to defect detection under other periodic texture patterns, and is simple in parameter setting and high in detection efficiency.

Description

Technical Field

[0001] A method for detecting defects in metal mesh based on superimposed structural contrast information belongs to the field of compressed sensing and computer vision, specifically referring to a metal mesh image processing algorithm based on computer vision and surface texture structure information. Background Technology

[0002] With the widespread application of high-frequency electronic equipment and the increasing complexity of the electromagnetic environment, electromagnetic interference can reduce the stability of electronic equipment. Metal mesh grids, with their high light transmittance and strong electromagnetic shielding performance, are widely used in electromagnetic shielding applications in observable environments.

[0003] However, during the actual fabrication of metal mesh grids, various factors such as scratches, external corrosion, and human error can lead to defects on the grid surface, including impurities, metal adhesion, cracks, and broken wires. These defects are difficult to detect with the naked eye due to their minute size, but the alterations they cause to the grid structure affect its shielding performance. Furthermore, these defects expand during use, threatening the stability of practical engineering applications.

[0004] Currently, in the fabrication stage of metal mesh, defect detection mainly relies on observation and identification by fabrication personnel under a microscope. However, due to the small field of view under a microscope, the detection time for large-area submicron-sized metal mesh is very long and extremely labor-intensive. This method is entirely dependent on the experience and effort of the fabrication personnel and has a significant subjective element. Furthermore, in the practical engineering application stage of metal mesh, there is still no defect detection method available.

[0005] There are relatively few patents related to defect detection in metal mesh grids. Patent 201410131635.5 discloses a method for detecting and identifying defects in metal mesh grids. This patent classifies defects in metal mesh grids into three types: broken lines, scratches, and closed areas. It simulates the length and size of these three types of defects, constructs a defect database, and trains a support vector machine to facilitate real-time identification and classification of defects in metal mesh grids. However, the defect types in this invention are insufficient to describe all defects in metal mesh grids, and the size of the defect database can affect the detection accuracy.

[0006] Defect detection in metal mesh is a type of surface defect detection problem with certain texture features. Patent 202111243941.4, "A Computer Vision-Based Method for Wood Board Defect Detection," discloses a defect detection approach based on wood grain texture. It defines three descriptors: density, texture spacing ratio, and texture tilt. The probability of corresponding knot regions within the tile is obtained, leading to a probability image. This method, considering the texture features of wood and employing edge extraction, is suitable for detecting wormholes on wood board surfaces. However, it is difficult to generalize to surface defect detection scenarios with other texture features, and it is also unsuitable for defect detection in metal mesh.

[0007] Patent 202111083642.9, "A Method for Detecting Surface Defects in Mechanical Parts Based on Image Processing," describes the calculation of the gray-level co-occurrence matrix in the neighborhood of a pixel, the calculation of the information entropy of the gray-level co-occurrence matrix, and the construction of a loss function for a semantic segmentation network based on the information entropy, thereby achieving threshold calculation and segmentation to obtain the defect image. Patent 202111064818.6, "A Device and Method for Detecting Surface Defects in Textiles," proposes an adaptive threshold segmentation algorithm. These two threshold segmentation algorithms can solve the problem of inapplicable gray-level ranges caused by uneven illumination or other factors in actual light sources. However, their performance in defect recognition and detection for actual images is relatively poor.

[0008] Patent 202210503931.8, "A Method for Detecting Defects on Complex Textured Tile Surfaces," discloses a method for detecting defects on tile surfaces using a Cascade R-CNN convolutional neural network and traditional image processing algorithms. The multi-sensor self-attention mechanism, along with the structure of variable convolutional kernels and cascaded heads, enhances the algorithm's ability to detect small defects with varying shapes. Patents 202110852729.1, 202011462376.6, 202111062913.2, and 202111023082.8 are also surface defect detection methods based on other deep learning algorithms, such as BP neural networks, ART neural networks, and YOLOv5 networks. The objects to be detected include glass, wafers, and the appearance of cigarette packs, etc. However, the drawbacks of these methods are as described in the literature "Wang J , Xu G, Li C, et al. Surface Defects Detection using Non-convex TotalVariation Regularized RPCA with Kernelization[J]. IEEE Transactions on Instrumentation and Measurement, 2021, PP(99):1-1." Although these deep learning methods can achieve significant detection results in specific scenarios, they cannot be regarded as general and stable surface defect detection methods because their performance is highly dependent on the specific selection of parameters and extreme or limited assumptions, which will limit them to specific domains and lack adaptability. Furthermore, defect image datasets are difficult to collect in many scenarios.

[0009] Patent 202111047362.2, "A Class Imbalanced Surface Defect Based on Transfer Learning," describes a method to address the imbalance problem in surface defect images, where common and rare defects exist. This effectively avoids overfitting caused by insufficient rare defect samples and achieves transfer from detecting common defects to detecting rare defects. Patent 201711298006.1, "A Deep Learning-Based Method for Detecting Magnetic Tile Surface Defects," introduces a method that performs data augmentation transformation on each original magnetic tile surface defect image in both the training and detection datasets to obtain expanded training and detection datasets. This solves the overfitting problem caused by insufficient defect datasets in deep learning methods. For example, the literature "Liu J, Wang C, Su H, et al. Multistage GAN for Fabric DefectDetection[J]. IEEE Transactions on Image Processing, 2019, 29:1-1." extracts and fuses multiple defects into other background textures, achieving better fusion results through multi-stage GANs. However, as the literature points out, its drawbacks include poor performance in detecting large-area defects due to limitations in training memory, and the fact that multi-scale detection requires more computing power, which contradicts the efficiency requirements in practice.

[0010] In summary, in order to meet the requirements of the actual processing and manufacturing stages and the engineering application stages of metal mesh, it is necessary to develop a set of efficient and high-precision methods for detecting defects in metal mesh. Summary of the Invention

[0011] To address the need for metal mesh defect detection under the aforementioned background, while considering the task indicators of detection speed and recognition accuracy, this invention is based on robust principal component analysis in the fields of data mining and compressed sensing, and introduces image enhancement and denoising algorithms and threshold segmentation algorithms from the field of image processing. It proposes a metal mesh defect detection method based on the superposition of structural contrast information, realizing a high-precision detection method for metal mesh defects.

[0012] The technical solution of the present invention is as follows:

[0013] 1. A method for detecting defects in metal mesh based on structural comparison information stacking, characterized by comprising the following steps:

[0014] Step 1: Use a microscope system to acquire an image of a metal mesh grid as the input image. The input image contains x×x grid periods, x∈[10,20]. Perform homomorphic filtering on the input image to obtain an image with uniform brightness.

[0015] Step 2: Taking a pixel in the homomorphically filtered input image as the center, iterate through the differences in feature similarity of its four diagonal neighbors. The length and width of the neighbors are greater than 1.5 times the design period of the grid. The length and width values ​​corresponding to the minimum feature similarity difference of the four neighbors are the period of the grid pattern in the length and width directions. According to the calculated period, the grid image is divided into periodic blocks. The differences of each periodic block with its four neighbors in the horizontal and vertical directions are compared. The feature similarity difference value calculated for each block is counted to obtain the difference matrix. After sliding the blocks with each point in the period as the starting point, a multi-layer difference matrix is ​​obtained. The results of each layer are superimposed to calculate the prior image P.

[0016] Step 3: Substitute the weight matrix W corresponding to the input image D and the prior image P into the optimized robust principal component analysis model:

[0017]

[0018] The weight matrix corresponding to the prior map is: E represents the defect image satisfying the sparse feature, A represents the defect-free background image, D represents the input image, and G represents the noise in the image caused by deformation and illumination changes. Parameters λ and β are used to balance the defect and noise components. The matrix has a 1-norm. Let F be the norm of the matrix. These are the singular values ​​of the matrix. For matrix Norm:

[0019]

[0020] The model decomposes the input image D into a low-rank image A, a sparse image E, and a noisy image G;

[0021] Step 4: Stretch the prior image to grayscale, and then perform thresholding on the stretched prior image to obtain a binary mask. Use the binary mask to filter the sparse image, i.e., perform Hadamard product operation, to remove noise from non-defect areas and obtain a saliency map.

[0022] Step 5: For significant Figure 4 The edge pixels of the surrounding area are set to zero to eliminate false high-pixel values ​​at the edges of the saliency map. Finally, a threshold segmentation operation is performed to obtain the defect detection result.

[0023] 2. In the above-mentioned method for detecting defects in metal mesh based on structural comparison information stacking, step 2, the step of calculating the feature similarity difference of the four diagonal neighborhoods, includes: calculating the structural similarity index of a certain pixel and its four neighboring pixels (upper left, lower left, upper right, and lower right); summing the structural similarity differences between the upper left and lower left neighbors, the lower left and lower right neighbors, the lower right and upper right neighbors, and the upper right and upper left neighbors, and then averaging them; finally, subtracting the calculated result from 1.

[0024] The steps for comparing the differences between a periodic block and its four neighboring blocks in the horizontal and vertical directions include: calculating the structural similarity index between a periodic block and its four neighboring periodic blocks above, below, left, and right; subtracting the periodic block from its four neighboring blocks above, below, left, and right and calculating the root mean square value; summing the four root mean square values ​​and averaging them to obtain the grayscale difference value; calculating the structural similarity difference between the periodic block and its four neighboring blocks above, below, left, and right; summing the four structural similarity differences and averaging them to obtain the structural similarity difference value; summing the grayscale difference value and the structural similarity difference value; and finally subtracting the summation result from 2.

[0025] 3. In the above-mentioned method for detecting defects in metal mesh based on structural contrast information stacking, step 4, the grayscale stretching step of the prior image is as follows: each pixel of the prior image is stretched using the e-index, so that high pixel values ​​are mapped to higher values, and the pixel region is located more accurately.

[0026] This invention has the following advantages and outstanding effects:

[0027] 1. The starting point of this invention is that, in high-dimensional space, when the number of image samples is much smaller than its dimensionality, image data exhibits a sparse distribution and data redundancy, making high-dimensional space analysis difficult and resulting in high time and space complexity. Therefore, a more effective solution is to reduce the dimensionality to a low-dimensional subspace for processing. This invention selects the low-rank decomposition algorithm, which has broad research significance and value in the field of data mining, and performs low-dimensional feature mapping on high-dimensional data based on the low-rank and sparsity characteristics satisfied by the image. This invention optimizes the low-rank decomposition algorithm, addressing the problem that traditional low-rank decomposition algorithms struggle to identify and detect large defects in images, and can effectively identify and detect large-area defects, defects of multiple types coexisting, and defects with a large number in a single image.

[0028] 2. This invention proposes a prior feature extraction algorithm based on structural similarity (SSIM). Traditional low-rank decomposition models struggle to detect defects in complex periodic texture samples. Furthermore, the detection capabilities of other algorithms are further limited when images exhibit uneven illumination and low signal-to-noise ratios. However, SSIM, starting from the image structure, evaluates image differences from a higher visual perspective, making it an efficient image quality assessment method. This invention can enhance defect feature information.

[0029] 3. In the mask construction process, this invention utilizes feature data from prior information as an aid for threshold segmentation. The saliency map is obtained by performing a Hadamard product operation between the mask and the sparse map. This process is equivalent to filtering the sparse information for defective parts, thus eliminating pixels in non-defective areas for subsequent accurate thresholding, resulting in better defect segmentation.

[0030] 4. This invention requires no training process and possesses strong generalization ability. Currently, many methods exist for this research field, such as support vector machines and deep learning. It's important to note that the key to the success of deep learning lies in designing a well-designed convolutional neural network and a large number of labeled samples. However, in actual production processes, due to environmental and equipment factors, it is difficult to collect large-scale, high-quality defect images, and labeling these defect images can be very costly. Furthermore, the learned model is tightly coupled to a specific dataset with specific regulatory information. When samples are lacking in the corresponding scenario, it is difficult to transfer them to other detection scenarios; therefore, their generalization ability is limited.

[0031] In summary, this invention extracts accurate prior information through layered structural comparison information, optimizes the guided low-rank decomposition process, and constructs a mask matrix using prior information to assist in defect extraction, thus eliminating the interference of stray noise on the defect separation process. This invention requires no training process, has excellent generalization ability, and can effectively identify and detect large-area defects, defects of multiple types coexisting, and defects with a large number in a single image. Attached Figure Description

[0032] Figure 1 This is a flowchart of the defect detection method in this invention.

[0033] Figure 2 This is a schematic diagram illustrating the calculation of the pattern period size in this invention.

[0034] Figure 3 This is a schematic diagram of prior image extraction in this invention.

[0035] Figure 4 This is a schematic diagram of the low-rank decomposition module in this invention.

[0036] Figure 5 This is a schematic diagram of the mask structure in this invention.

[0037] Figure 6 This is a schematic diagram of saliency extraction in this invention.

[0038] Figure 7 The results are obtained by using the method of the present invention to detect metal mesh pattern 1 (single-cycle broken wire defect).

[0039] Figure 8 The results show the detection of metal mesh pattern 2 (multi-period broken wire defect) using the method of the present invention.

[0040] Figure 9 The results show the detection of metal mesh pattern 3 (debris attachment) using the method of the present invention.

[0041] Figure 10 The results show the detection of metal mesh pattern 4 (linear foreign matter attachment) using the method of the present invention. Detailed Implementation

[0042] This invention proposes a method for detecting defects in metal mesh based on the stacking of structural comparison information. The method of this invention will be described in detail below with reference to embodiments and accompanying drawings.

[0043] This embodiment presents a defect detection method for metal mesh grids based on structural comparison information stacking, and its flowchart is as follows. Figure 1 As shown. The method consists of the following steps:

[0044] Step 1: Use a microscope system to acquire an image of a metal mesh grid as the input image. The input image contains x×x grid periods, x∈[10,20]. Perform homomorphic filtering on the input image to solve the problem of uneven brightness distribution.

[0045] Step 2: Extract prior maps from the input image based on structural similarity (SSIM). This invention proposes a period calculation method based on diagonal neighborhood difference to assist in dividing the image into periodic blocks for prior map acquisition.

[0046] Step 3: Substitute the weight matrix W corresponding to the input image D and the prior image P into the optimized robust principal component analysis model:

[0047]

[0048] The weight matrix corresponding to the prior map is: E represents the defect image satisfying the sparse feature, A represents the defect-free background image, D represents the input image, and G represents the noise in the image caused by deformation and illumination changes. Parameters λ and β are used to balance the defect and noise components. The matrix has a 1-norm. Let F be the norm of the matrix. These are the singular values ​​of the matrix. For matrix Norm:

[0049]

[0050] The model decomposes the input image D into a low-rank image A, a sparse image E, and a noisy image G;

[0051] Step 4: Construct a mask matrix from the prior image, perform filtering on the sparse image, and obtain a saliency map;

[0052] Step 5: Perform threshold segmentation on the saliency map to obtain the defect detection results.

[0053] Specifically, the periodicity calculation method based on diagonal neighborhood dissimilarity is as follows: Figure 2 As shown. Reference Figure 2 The left image shows a star-patterned fabric image, where M and N are the pixel sizes along the length and width of a single image. The specific steps involve calculating the difference between the four diagonally opposite neighbors of a given pixel, iterating through the length and width (m, n) of each neighborhood, and finding the m and n values ​​corresponding to the minimum difference among the four neighbors. The right image shows the result of iterating through the length and width of the neighborhoods. The experimental results demonstrate that the method can accurately identify the period size.

[0054] Specifically, the periodicity calculation method based on diagonal neighborhood dissimilarity is as follows: Figure 3 As shown. Reference Figure 3 The calculated period is obtained based on the aforementioned period calculation method based on diagonal neighborhood dissimilarity, and this period is used to assist in image segmentation. Furthermore, to improve the accuracy of prior information, this invention employs a multi-dissimilarity overlay method. For example... Figure 3 As shown in the first row, multiple sets of starting point auxiliary image blocks are performed. Then, the differences between each block and its four neighboring regions in the horizontal and vertical directions are compared, and the calculated difference value for each block is counted. Figure 3 The second row shows the result. The difference matrix is ​​then upsampled to the original pattern size and superimposed. The final result and the ground truth image are shown below. Figure 3 As shown in the third row, the approximate location of the defect point is well identified.

[0055] Therefore, the specific operations of step 2 can be summarized as follows:

[0056] Step 2.1: Taking a pixel in the input image as the center, calculate the structural similarity difference of its four diagonal neighbors, traverse the length and width of the neighbors, and the length and width values ​​corresponding to the minimum difference of the four neighbors are the period size of the pattern.

[0057] Step 2.2: Based on the calculated period, divide the image into periodic blocks, compare the differences between each block and its four neighboring regions in the horizontal and vertical directions, and calculate the structural similarity difference value of each block to obtain the difference matrix.

[0058] Step 2.3: Obtain multi-layer difference matrices by sliding blocks, and calculate the prior map by superimposing the results of each layer.

[0059] Specifically, in step 3, the flowchart illustrating the image decomposition using the optimized robust principal component analysis method is as follows: Figure 4 As shown. Reference Figure 4 The left image is the system input image obtained by the microscopic imaging system in step 1. The image above the arrow is the prior image of the auxiliary decomposition obtained in step 2. The three images obtained by decomposition on the right are, from top to bottom: low-rank image, sparse image, and noisy image.

[0060] Specifically, in step 4, the flowchart for constructing the mask matrix from the prior graph is as follows: Figure 5 As shown. Reference Figure 5 In sparse maps, some non-defective regions may sometimes exhibit the same high grayscale values ​​as defective regions. Therefore, it is necessary to filter out the high grayscale noise generated by non-defective regions. Since prior maps have excellent defect localization performance, this invention utilizes prior maps, stretching their overall grayscale range to construct a mask matrix. This mask matrix is ​​then used to perform a Hadamard product operation with the sparse map to obtain a saliency map. This process is equivalent to filtering the sparse information specifically for defective parts, thus eliminating pixels in non-defective regions for subsequent accurate thresholding. The process is as follows: Figure 5 As shown.

[0061] Specifically, in step 5, the flowchart for thresholding the saliency map is as follows: Figure 6 As shown. Reference Figure 6 The process proceeds clockwise, starting from the top left image. The top left image is the defective input image obtained in step 1. The second step is the sparse image obtained from the low-rank decomposition in step 3. The third step is the mask image constructed from the prior image in step 4. The fourth step is image filtering. The fifth step is thresholding to obtain the detection result, which is a binary image where a pixel value of 1 represents a defect and a pixel value of 0 represents a non-defect. The left side of the final result image is the ground truth image, showing that the detection effect is good.

[0062] The final defect detection results of the metal mesh are shown in the figure below. Figure 7 , 8, 9, 10. Figure 7 The results are obtained by using the method of the present invention to detect metal mesh pattern 1 (single-cycle broken wire defect). Figure 8 The results show the detection of metal mesh pattern 2 (multi-period broken wire defect) using the method of the present invention. Figure 9The results are obtained by using the method of the present invention to detect metal mesh pattern 3 (debris foreign matter attachment). Figure 10 This is the result of detecting metal mesh pattern 4 (linear foreign matter attachment) using the method of the present invention. The method of the present invention requires no training process; it can detect various types of defects in metal mesh; it can also be generalized to defect detection under other periodic texture patterns; its parameter settings are simple; and its detection efficiency is high.

[0063] The above description is merely a specific example of the present invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and detail without departing from the concept of the present invention, or directly / indirectly apply it to other related technical fields, which are still within the scope of protection of the claims of the present invention.

Claims

1. A metal grid defect detection method based on structural contrast information stack, characterized in that, Includes the following steps: Step 1: Use a microscope system to acquire an image of a metal mesh grid as the input image. The input image contains x×x grid periods, x∈[10, 20]. Perform homomorphic filtering on the input image to obtain an image with uniform brightness. Step 2: Taking a pixel in the homomorphically filtered input image as the center, iterate through the differences in feature similarity of its four diagonal neighbors. The length and width of the neighbors are greater than 1.5 times the design period of the grid. The length and width values ​​corresponding to the minimum feature similarity difference of the four neighbors are the period of the grid pattern in the length and width directions. According to the calculated period, the grid image is divided into periodic blocks. The differences of each periodic block with its four neighbors in the horizontal and vertical directions are compared. The feature similarity difference value calculated for each block is counted to obtain the difference matrix. After sliding the blocks with each point in the period as the starting point, a multi-layer difference matrix is ​​obtained. The results of each layer are superimposed to calculate the prior image P. Step 3: Substitute the weight matrix W corresponding to the input image D and the prior image P into the optimized robust principal component analysis model: where the weight matrix corresponding to the prior map is , E is the defect image satisfying the sparse feature, A is the background image without defects, D is the input image, G represents the noise in the image due to deformation and illumination changes, and parameters λ and β are used to balance the defect part and the noise part, is the matrix 1 norm, is the matrix F norm, is the matrix singular value, is the matrix norm: The model decomposes the input image D into a low-rank image A, a sparse image E, and a noisy image G; Step 4: Stretch the prior image in grayscale, perform threshold segmentation on the stretched prior image to obtain a binary mask image, and use the binary mask image to perform filtering operations on the sparse image, that is, perform Hadamard product operation to filter out noise in non-defect areas and obtain a saliency image. Step 5: Set the edge pixels around the saliency map to zero to eliminate false high-pixel values ​​at the edges of the saliency map. Finally, perform threshold segmentation to obtain the defect detection results.

2. The metal grid defect detection method based on structural contrast information stack according to claim 1, characterized in that, In step 2: The steps for calculating the feature similarity difference of four diagonal neighborhoods include: Calculate the structural similarity index between a pixel and its four neighboring pixels (top left, bottom left, top right, and bottom right). Sum the structural similarity differences between the top left and bottom left neighbors, the bottom left and bottom right neighbors, the bottom right and top right neighbors, and the top right and top left neighbors, and then take the average. Finally, subtract 1 from the calculated result. The step of comparing the difference between the periodic block and its four neighboring regions in the horizontal and vertical directions includes: Calculate the structural similarity index between a given periodic block and its four neighboring periodic blocks (upper, lower, left, and right). Sum the four structural similarity differences and average them to obtain the structural similarity difference value. Subtract the grayscale values ​​of this periodic block from those of its four neighboring periodic blocks and calculate the root mean square value. Sum the four root mean square values ​​and average them to obtain the grayscale difference value. Sum the grayscale difference value with the structural similarity difference value. Finally, subtract the summation result from 2.

3. The metal grid defect detection method based on structural contrast information stack of claim 1, wherein, In step 4: The grayscale stretching step for the prior image is as follows: The e-index is used to stretch each pixel of the prior image, so that high pixel values ​​are mapped to even higher values, thus locating pixel regions more accurately.

Images