Product appearance defect detection method based on machine vision technology
Through the multi-scale structure similarity calculation method based on machine vision, the problem of traditional methods relying on manual thresholds and deep learning methods is solved, and high-precision and real-time product appearance defect detection is achieved, which is suitable for industrial production.
Patent Information
- Application Number
- CN202510535894.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-08-12
AI Technical Summary
In the prior art, traditional product appearance defect detection methods rely on manual setting of thresholds, resulting in mis-detection or missed detection problems. Deep learning-based methods require a large amount of labeled data and high-performance hardware, which limits its popularity in industrial sites.
The multi-scale structure similarity calculation method based on machine vision is used to calculate the structure similarity of the image blocks by greyscale processing and Gaussian pyramid downsampling, and the structural similarity of the image blocks is calculated, and the weighted average method is used to determine whether the product has defects.
Real-time and high-precision defect detection is realized, avoiding the unreliability of manually set parameters, reducing dependence on large-scale data sets and high-performance hardware, and is suitable for low-configured industrial control machines, improving the applicability and real-timeness of detection.
Smart Images
Figure CN120471844A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of detection technology, and in particular relates to a method for detecting product appearance defects based on machine vision technology. Background Art
[0002] Product appearance defect detection is one of the important applications of machine vision technology in industrial automation production. Through visual inspection technology, product appearance defects can be efficiently detected and identified, thereby improving product quality and ensuring production efficiency.
[0003] Traditional product appearance defect detection methods typically use image acquisition components to capture product appearance images and then detect and identify defects based on features such as grayscale and gradient. These methods rely on manually set thresholds to adapt to different industrial scenarios. However, in real-world production environments, if the characteristics of the defect and the background image are similar, the detection results of traditional algorithms are often significantly affected, resulting in false or missed detections.
[0004] In contrast, deep learning-based appearance defect detection methods do not rely on manually set thresholds and offer significant advantages in detection accuracy and robustness. However, these methods require large, accurately labeled defect datasets and high hardware computing power, which to some extent limits their widespread adoption and application in industrial settings. Summary of the Invention
[0005] In response to the above technical problems existing in the prior art, the present invention proposes a product appearance defect detection method based on machine vision technology, which has a reasonable design, overcomes the shortcomings of the prior art, and has good results.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] A method for detecting product appearance defects based on machine vision technology comprises the following steps:
[0008] Step 1: Collect multiple images of the appearance of normal products, average the pixel values of the multiple normal images point by point, and generate a reference image;
[0009] Step 2: Grayscale the reference image and the image to be tested to obtain a reference grayscale image and a grayscale image to be tested, and then subdivide the two into multiple image blocks according to the set resolution;
[0010] Step 3: Compare the image blocks at corresponding positions in the reference grayscale image and the grayscale image to be tested;
[0011] Step 4: Based on the calculation results in step 3, calculate the structural similarity of the image blocks at the corresponding positions;
[0012] Step 5: Perform Gaussian pyramid downsampling on the image blocks to generate multi-level low-resolution image blocks;
[0013] Step 6: Repeat steps 3-5 until the structural similarities of all images at the set number of layers are calculated, and the multi-scale structural similarity is obtained by weighted average;
[0014] Step 7: Compare the multi-scale structural similarity with the set threshold. The image blocks whose multi-scale structural similarity is greater than or equal to the set threshold are marked as similar, otherwise they are marked as dissimilar.
[0015] Step 8: Count the image blocks marked as similar. If the number of similar image blocks is greater than the set number, the image to be tested is judged to be similar to the reference image, that is, it passes without defects. Otherwise, it is judged that the product to be tested has appearance defects.
[0016] Preferably, in step 1, the generation of the reference image is specifically as follows:
[0017] The reference image is generated by performing point-by-point arithmetic averaging on pixel values at the same position of multiple defect-free product images.
[0018] Preferably, in step 2, the grayscale processing adopts a full-channel or single-channel grayscale method, and the segmentation size of the image block is determined by the resolution parameter; as shown in formula (1):
[0019]
[0020] Among them, CellWidth and CellHeight are the length and width of the image block respectively, and Num is the set resolution, which means that the grayscale image is subdivided into Num×Num image blocks. 1_i,j ,I 2_i,j They respectively represent the image blocks with position index (i, j) after segmentation of the reference grayscale image I1 and the grayscale image to be tested I2.
[0021] Preferably, step 3 includes the following steps:
[0022] 3.1: Calculate the mean and variance of two grayscale image patches;
[0023] 3.2: Calculate the covariance of two grayscale image blocks; as shown in formula (2):
[0024]
[0025] Where N is the total number of pixels in the image block, x k ,y k The image block I with position index (i, j) in the reference image I1 and the image to be tested I2 is represented by 1_i,j ,I 2_i,j The gray value of the kth pixel in , They are image blocks I 1_i,j ,I 2_i,j The mean of is the image block I 1_i,j ,I 2_i,j The variance of is the covariance of two grayscale image patches.
[0026] Preferably, in step 4, the structural similarity of the image blocks at corresponding positions is calculated according to formula (3):
[0027]
[0028] Where C1 = (k1L) 2 ,C2=(k2L) 2 is a stability constant, with the default values of k1=0.01, k2=0.03, and L=255. The SSIM value is between [-1, 1], and the closer it is to 1, the more similar the two images are.
[0029] Preferably, in step 5, Gaussian pyramid downsampling is performed according to formula (4):
[0030]
[0031] Among them, PyrDown(I) is a Gaussian pyramid downsampling operation. After Gaussian filtering, the original image is sampled at intervals of 1 pixel, and the resolution is reduced to 1 / 2 of the original image. I 1_i,j ,I 2_i,j The result of downsampling the Gaussian pyramid once is the second layer image.
[0032] Preferably, in step 6, the weighted average formula of the multi-scale structural similarity is as shown in formula (5):
[0033]
[0034] Among them, M is the set number of layers, ω m =0.5 m is the weight of the mth layer, is the m-th layer image obtained by downsampling the Gaussian pyramid, where
[0035] Preferably, in step 7, image blocks whose multi-scale structural similarity is greater than or equal to a set threshold are marked as similar, otherwise they are marked as dissimilar; as shown in formula (6):
[0036]
[0037] Among them, Cell i,jis the label value of the image block with position index (i, j) in the image to be tested I2, Threshold is the set multi-scale structural similarity threshold, if the multi-scale structural similarity of the image block with position index (i, j) is greater than or equal to the set threshold, then Cell i,j The value is 1, indicating that the image block is a similar image block, otherwise Cell i,j A value of 0 indicates that the image block is a dissimilar image block.
[0038] Preferably, in step 8, if the number of similar image blocks is greater than a set number, the image to be tested is determined to be similar to the reference image, that is, it passes without defects; otherwise, it is determined to be defective; as shown in formula (7):
[0039]
[0040] Among them, Result is the judgment result of the image to be tested, sum(Cell i,j =1) represents the number of similar image blocks in step 8, SetNum is the set number of similar image blocks, if the number of similar image blocks in step 8 is greater than or equal to the set number of similar image blocks, then Result is 1, indicating that the image to be tested is similar to the reference image, that is, there is no defect; otherwise, Result is 0, that is, there is a defect.
[0041] The beneficial technical effects brought about by the present invention are:
[0042] This invention provides a method for detecting product appearance defects based on machine vision. By calculating multi-scale structural similarity between the image to be detected and a normal reference image in blocks, this method achieves real-time, high-precision defect detection and identification. Compared with traditional threshold-based detection methods, this method eliminates the need for manual setting of complex parameters, thus avoiding the unreliability of threshold settings. Furthermore, compared with deep learning-based detection algorithms, this method is less dependent on large-scale datasets and high-performance hardware, and exhibits greater real-time performance and applicability. Therefore, this method has broad application prospects in the field of industrial product appearance defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 Schematic diagram of the overall process of the method of the present invention.
[0044] Figure 2 The figure shows the process of generating the reference image of the method of the present invention, where (a) is 100 collected images of defect-free products, and (b) is the reference image after point-by-point averaging of the 100 images.
[0045] Figure 3Schematic diagram of image segmentation during defect detection according to the present invention, wherein (a) is an appearance image of the product to be detected, (b) is a schematic diagram of the reference image after segmentation processing, and (c) is a schematic diagram of the image to be detected after segmentation processing.
[0046] Figure 4 This is a schematic diagram of the results of product appearance defect detection according to the present invention. The image blocks selected by the highlighted boxes in the figure are the defect locations detected by the method. DETAILED DESCRIPTION
[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:
[0048] The present invention provides a method for detecting product appearance defects based on machine vision. The method divides the reference image and the image to be inspected into multiple image blocks and performs multi-scale structural similarity calculation. The method determines whether the overall product appearance has defects based on the number of image blocks whose similarity does not reach a set value. The detection process is as follows: Figure 1 shown.
[0049] The following takes the end face defect detection of a box on a high-speed production line as an example to illustrate the specific detection process. The steps are as follows:
[0050] 1. Collect 100 images of the appearance of normal products and average the pixel values of the images point by point to generate a representative reference image, such as Figure 2 shown.
[0051] 2: Grayscale the reference image and the image to be inspected according to all channels or a single channel, and subdivide them into multiple image blocks according to the set resolution:
[0052]
[0053] Among them, CellWidth and CellHeight are the length and width of the image block, which is (720, 480) in the example; Num is the set resolution, which means that the grayscale image is subdivided into Num×Num image blocks, which is 10 in the example; I 1_i,j ,I 2_i,j They respectively represent the image blocks with position index (i, j) after segmentation of the reference grayscale image I1 and the grayscale image to be tested I2.
[0054] Figure 3 (a) is the image to be inspected, and there is a white defect in the lower right corner of the end face image. Figure 3 (b) is a schematic diagram of the reference image and the image to be tested after grayscale conversion and subdivision of the image blocks. The resolution is 10×10. The two images with a resolution of (720,480) are divided into 100 image blocks, and the size of each image block is (72,48). Figure 3 (c) is a schematic diagram of the image to be detected after block processing.
[0055] 3. Compare the image blocks at the corresponding positions of the reference image and the image to be tested. Calculate the structural similarity of the image blocks at the corresponding positions:
[0056]
[0057] in, They are image blocks I 1_i,j ,I 2_i,j The mean of is the image block I 1_i,j ,I 2_i,j The variance of is the covariance of the two image patches.
[0058] Then, two low-resolution image blocks are obtained by downsampling the Gaussian pyramid:
[0059]
[0060] Among them, PyrDown(I) is the Gaussian pyramid downsampling operation, I 1_i,j ,I 2_i,j The result of one downsampling of the Gaussian pyramid is the second-level image. Pyramid downsampling works by Gaussian smoothing the image (kernel size 5×5, σ=1.0) and then downsampling the smoothed image according to a certain ratio. The input image is Gaussian filtered using a standard Gaussian kernel. The filtered image is then downsampled by taking every two rows and every two columns. The resulting image resolution is half that of the original, resulting in resolutions of 36×24 (second level) and 18×12 (third level), etc.
[0061] Calculate the structural similarity of the downsampled images until all images of the set number of layers are calculated, and perform weighted average of the structural similarities of the multi-layer images to obtain the multi-scale structural similarity:
[0062]
[0063] Among them, M is the set number of layers 5, ω m =0.5 m is the weight of the mth layer, is the m-th layer image obtained by downsampling the Gaussian pyramid, where
[0064] 4. Compare the calculated structure with the set threshold. The image blocks with multi-scale structural similarity greater than or equal to the set threshold are marked as similar and framed in green. Otherwise, they are marked as dissimilar and framed in a highlighted rectangle, such as Figure 4 shown.
[0065]
[0066] In this example, the threshold is set to 0.7, and all the white defects in the lower right corner of the end face are marked, and the corresponding image blocks are highlighted. The average multi-scale structural similarity of the highlighted image blocks is 0.479, and the average similarity of the remaining normal image blocks is 0.951.
[0067] 5. Count the image blocks marked as similar. If the number of similar image blocks is greater than the set number, the image to be tested is judged to be similar to the reference image, that is, it passes without defects (Result=1); otherwise, it is defective (Result=0).
[0068]
[0069] In this example, the set number is 100, that is, the image to be tested has no defective image blocks and is considered passed. The instance result is 95 similar image blocks, which is less than the set number, and the instance is judged to be defective.
[0070] The present invention proposes a method for detecting product appearance defects based on machine vision, effectively overcoming the limitations of existing detection algorithms. It not only avoids the subjective nature of manually setting a large number of parameters, but also achieves high-precision detection even when the defect and background image features are similar, significantly reducing false detections or missed detections caused by background interference. Furthermore, compared to big data algorithms that rely on deep learning, the present invention is less dependent on defect sample data, does not require high-performance hardware support, and can run efficiently on low-configuration industrial computers, thereby achieving a more reasonable detection principle and a wider range of applications.
[0071] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the scope of protection of the present invention.
Claims
1. A method for detecting product appearance defects based on machine vision technology, characterized in that: The steps include: Step 1: Collect multiple images of the appearance of normal products, average the pixel values of the multiple normal images point by point, and generate a reference image; Step 2: Grayscale the reference image and the image to be tested to obtain a reference grayscale image and a grayscale image to be tested, and then subdivide the two into multiple image blocks according to the set resolution; Step 3: Compare the image blocks at corresponding positions in the reference grayscale image and the grayscale image to be tested; Step 4: Based on the calculation results in step 3, calculate the structural similarity of the image blocks at the corresponding positions; Step 5: Perform Gaussian pyramid downsampling on the image blocks to generate multi-level low-resolution image blocks; Step 6: Repeat steps 3-5 until the structural similarities of all images at the set number of layers are calculated, and the multi-scale structural similarity is obtained by weighted average; Step 7: Compare the multi-scale structural similarity with the set threshold. The image blocks whose multi-scale structural similarity is greater than or equal to the set threshold are marked as similar, otherwise they are marked as dissimilar. Step 8: Count the image blocks marked as similar. If the number of similar image blocks is greater than the set number, the image to be tested is judged to be similar to the reference image, that is, it passes without defects. Otherwise, it is judged that the product to be tested has appearance defects.
2. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 1, the generation of the reference image is specifically as follows: The reference image is generated by performing point-by-point arithmetic averaging on pixel values at the same position of multiple defect-free product images.
3. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 2, the grayscale processing adopts the full-channel or single-channel grayscale method, and the segmentation size of the image block is determined by the resolution parameter; as shown in formula (1): Among them, CellWidth and CellHeight are the length and width of the image block respectively, and Num is the set resolution, which means that the grayscale image is subdivided into Num×Num image blocks. 1_i,j ,I 2_i,j They respectively represent the image blocks with position index (i, j) after segmentation of the reference grayscale image I1 and the grayscale image to be tested I2.
4. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: Step 3 includes the following steps: 3.1: Calculate the mean and variance of two grayscale image patches; 3.2: Calculate the covariance of two grayscale image blocks; as shown in formula (2): Where N is the total number of pixels in the image block, x k ,y k The image block I with position index (i, j) in the reference image I1 and the image to be tested I2 is represented by 1_i,j ,I 2_i,j The gray value of the kth pixel in , They are image blocks I 1_i,j ,I 2_i,j The mean of is the image block I 1_i,j ,I 2_i,j The variance of is the covariance of two grayscale image patches.
5. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 4, the structural similarity of the image blocks at the corresponding positions is calculated according to formula (3): Where C1 = (k1L) 2 ,C2=(k2L) 2 is a stability constant, k1=0.01, k2=0.03, L=255; SSIM is the structural similarity, the value is between [-1,1], the closer it is to 1, the more similar the two images are.
6. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 5, Gaussian pyramid downsampling is performed according to formula (4): Among them, PyrDown(I) is a Gaussian pyramid downsampling operation. After Gaussian filtering, the original image is sampled at intervals of 1 pixel, and the resolution is reduced to 1 / 2 of the original image. I 1_i,j ,I 2_i,j The result of downsampling the Gaussian pyramid once is the second layer image.
7. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 6, the weighted average formula of multi-scale structural similarity is shown in formula (5): Among them, M is the set number of layers, ω m =0.5 m is the weight of the mth layer, is the m-th layer image obtained by downsampling the Gaussian pyramid, where 8. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 7, the image blocks whose multi-scale structural similarity is greater than or equal to the set threshold are marked as similar, otherwise they are marked as dissimilar; as shown in formula (6): Among them, Cell i,j is the label value of the image block with position index (i, j) in the image to be tested I2, Threshold is the set multi-scale structural similarity threshold, if the multi-scale structural similarity of the image block with position index (i, j) is greater than or equal to the set threshold, then Cell i,j The value is 1, indicating that the image block is a similar image block, otherwise Cell i,j A value of 0 indicates that the image block is a dissimilar image block.
9. The method for detecting product appearance defects based on machine vision technology according to claim 1, characterized in that: In step 8, if the number of similar image blocks is greater than the set number, the image to be tested is judged to be similar to the reference image, that is, it passes without defects; otherwise, it is considered to have defects; as shown in formula (7): Among them, Result is the judgment result of the image to be tested, sum(Cell i,j =1) represents the number of similar image blocks in step 8, SetNum is the set number of similar image blocks, if the number of similar image blocks in step 8 is greater than or equal to the set number of similar image blocks, then Result is 1, indicating that the image to be tested is similar to the reference image, that is, there is no defect; otherwise, Result is 0, that is, there is a defect.