An image processing method for appearance defect detection

Abstract

The application discloses an image processing method for appearance defect detection, and belongs to the technical field of electronic information technology. The method comprises a first beam splitter, a second beam splitter, a first filter camera, a second filter camera and a third filter camera. The second filter camera and the third filter camera are respectively located below the second beam splitter and the first beam splitter. The first filter camera is located on the right side of the second beam splitter. The output ends of the first filter camera, the second filter camera and the third filter camera are connected with the input end of a PC host computer through a USB hub. The method solves the registration problem of multiple cameras, and the pictures of the same object image can be completely overlapped. The three filters can be flexibly replaced with filters of different wave bands. The image processing technology is added, so that the characteristics of defects are more obvious, and the detection accuracy is improved. The MobileNetV2 convolutional neural network structure is modified and migration learning is performed, so that the network structure is simplified and the detection speed is accelerated.

Description

Technical Field

[0001] This invention belongs to the field of electronic information technology, and in particular relates to an image processing method for detecting appearance defects. Background Technology

[0002] Imaging the same object with multiple cameras carrying different spectra has always been a global challenge. For example, if fruit producers want to check color and look for bruises, they might set up cameras with different wavelengths to take pictures. However, fusing the spectral data from two images is a very challenging and error-prone process. Even when two cameras are placed close together, there is still a significant optical parallax, making it almost impossible to align pixels in the two images. Furthermore, the flexibility of changing filters in multispectral cameras and the fusion of spectral data are contradictory global problems.

[0003] In industrial production, various factors inevitably lead to defects on the surface of products, making product inspection a crucial step in ensuring product quality. Given the wide variety of products and the diverse nature of their defects, traditional manual inspection is insufficient to meet industrial demands. To ensure both production speed and quality, manufacturers have begun to introduce machine vision technology, which offers advantages such as high inspection speed, standardized processes, and stable operation.

[0004] With the widespread adoption of machine vision in industry, its shortcomings in application have become increasingly apparent. Traditional machine vision inspection mainly involves studying the features of the target object and then selecting specific feature extraction methods for algorithm design. However, due to the significant differences between different detection targets in industrial applications, designers need to invest a great deal of effort and time in feature analysis and algorithm design. At the same time, manual feature extraction is extremely demanding on the designer's design experience and professional design capabilities. Often, the feature representation capabilities extracted by algorithms designed by different designers vary greatly, resulting in significant differences in the final detection performance of the algorithms.

[0005] With the development of machine learning algorithms, especially deep learning, more and more deep learning models are being used to solve problems in the field of computer vision, and they are performing exceptionally well. Many problems that were difficult to solve with traditional computer vision algorithms have been effectively addressed. However, deep learning is not a panacea. For some defects and flaws that are not obvious in appearance, although training can be performed to identify them, the results are not always satisfactory. Therefore, it is often necessary to perform relevant image processing on the video frames or photos of the defects and flaws to be identified to meet the required accuracy. Thus, utilizing convolutional neural network technology and image processing technology for defect detection is of great significance for product inspection in industrial production. Summary of the Invention

[0006] The purpose of this invention is to address the problem that deep learning is not a panacea, and that although it can be trained to recognize some defects that are not obvious in appearance, the recognition results are not satisfactory. Therefore, this invention proposes an image processing method for detecting appearance defects.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] An image processing method for detecting appearance defects includes a first beam splitter, a second beam splitter, a first filter camera, a second filter camera, and a third filter camera. The second and third filter cameras are located below the second and first beam splitters, respectively. The first filter camera is located to the right of the second beam splitter. The outputs of the first, second, and third filter cameras are connected to the input of a PC via a USB hub. The output of the PC is connected to the input of an image processing module. The output of the image processing module is connected to the inputs of an image segmentation module, a normalization module, and an image quality enhancement module, respectively. The outputs of the image segmentation module, the normalization module, and the image quality enhancement module are all connected to the input of a defect detection module. The output of the defect detection module is connected to the input of a prediction result module.

[0009] As a further description of the above technical solution:

[0010] The second beam splitter and the first beam splitter are arranged horizontally and parallel to each other, and at a 45-degree angle, splitting the input beam of light into three beams.

[0011] As a further description of the above technical solution:

[0012] The image segmentation module uses a region growing algorithm to retain the necessary parts and set the rest to black to reduce interference.

[0013] As a further description of the above technical solution:

[0014] The region growing mainly adopts two methods. One is to determine a growth point, i.e. a seed region, for the object to be segmented in the image, and expand outward from the seed region according to the set conditions to finally achieve the purpose of locating the entire target object. The other is to divide the entire image into several small regions according to similarity, and then integrate them to achieve the purpose of segmentation.

[0015] As a further description of the above technical solution:

[0016] The normalization processing module can improve the accuracy of the model by standardizing the image. For example, in the K nearest neighbor algorithm, it is necessary to calculate the distance between the point to be classified and all instance points.

[0017] As a further description of the above technical solution:

[0018] The image enhancement module uses histogram equalization, Laplacian-based image enhancement, logarithmic transform-based enhancement, and gamma transform-based image enhancement to widen the gap between defects and the background, making it easier to detect related defects in neural network detection and improving the accuracy of target detection.

[0019] As a further description of the above technical solution:

[0020] Histogram equalization adjusts the grayscale distribution of an image to make the distribution more even across grayscale levels 0 to 255, thereby improving image contrast and enhancing the subjective visual effect of the image. Images with low contrast are suitable for using histogram equalization to enhance image details.

[0021] As a further description of the above technical solution:

[0022] The Laplacian operator can enhance local image contrast, making it suitable for enhancing images with indistinct overall contrast and highlighting local features. Convolving the image with an 8-neighborhood Laplacian operator centered at 4 can achieve the purpose of sharpening and enhancing the image.

[0023] As a further description of the above technical solution:

[0024] The logarithmic transformation can expand the low grayscale portion of an image, revealing more detail in the low grayscale portion, while compressing the high grayscale portion, reducing the detail in the high grayscale portion. This achieves the purpose of emphasizing the low grayscale portion of the image. For different bases, the larger the base, the stronger the expansion of the low grayscale portion and the stronger the compression of the high grayscale portion.

[0025] As a further description of the above technical solution:

[0026] The method involves setting a γ value based on gamma transform, with 1 as the dividing line. A smaller γ value enhances the low-grayscale components of the image, while a larger γ value enhances the high-grayscale components. By using different γ values, details in both low- and high-grayscale areas can be enhanced. Gamma transform is particularly effective for images with low contrast and high overall brightness.

[0027] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0028] This invention solves the registration problem of multiple cameras, allowing images of the same object to completely overlap. The three filters can be flexibly replaced with filters of different wavelengths. Image processing technology is added to make the features of defects more obvious, which is conducive to improving detection accuracy. The MobileNetV2 convolutional neural network structure is modified and transfer learning is performed to simplify the network structure and speed up the detection process. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the structure of an image processing method for detecting appearance defects proposed in this invention;

[0030] Figure 2 This is an overall flowchart of an image processing method for detecting appearance defects proposed in this invention;

[0031] Figure 3 This is a schematic diagram of region growing in an image processing method for detecting appearance defects proposed in this invention.

[0032] Figure 4 This is a schematic diagram of channel shuffling in an image processing method for detecting appearance defects proposed in this invention.

[0033] Figure 5 This is a schematic diagram of the neural network training process for an image processing method for detecting appearance defects proposed in this invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1-5 The present invention provides a technical solution: an image processing method for detecting appearance defects, including a first beam splitter, a second beam splitter, a first filter camera, a second filter camera, and a third filter camera. The second filter camera and the third filter camera are respectively located below the second beam splitter and the first beam splitter. The first filter camera is located to the right of the second beam splitter. The second beam splitter and the first beam splitter are arranged horizontally and parallel to each other, and are arranged at an angle of 45 degrees to split an input beam of light into three beams.

[0036] The outputs of the first, second, and third filter cameras are connected to the input of a PC via a USB hub. The output of the PC is connected to the input of the image processing module. The output of the image processing module is connected to the inputs of the image segmentation module, the normalization module, and the image quality enhancement module, respectively. The outputs of the image segmentation module, the normalization module, and the image quality enhancement module are all connected to the input of the defect detection module. The image segmentation module uses a region growing algorithm to retain the required parts and set the remaining parts to black to reduce interference. The region growing mainly adopts two methods: one is to determine a growing point for the object to be segmented in the image. One approach is to use a seed region as the starting point and expand outwards according to set conditions to eventually locate the entire target object. Another approach is to divide the entire image into several smaller regions based on similarity and then integrate them to achieve segmentation. Region-based image segmentation methods can extract and segment regions within the image that are similar to the pixel features of the seed point. A pixel at a specified location is selected as the seed point, and the image is expanded to the right to find similar points. If the distance d of the expansion in this direction exceeds the set threshold d0, the image returns to the center point 0 and expands to the other side. If the distance d0 is exceeded again, the image is determined to be a pixel that does not need to be retained. By processing the pixels, the background interference can be processed, and the required content can be retained.

[0037] The normalization module can improve the accuracy of the model by standardizing the image. For example, in the K-nearest neighbor algorithm, it is necessary to calculate the distance between the point to be classified and all instance points. Assuming that each instance point consists of n features and the distance metric used is Euclidean distance, if the data has not been standardized or normalized beforehand, then those features with large absolute values ​​will play a decisive role in the Euclidean distance calculation, which will cause the final result to be biased. Standardization can make the features between different dimensions comparable in value. After the data is standardized, the optimization process of finding the optimal solution will obviously become smoother and the convergence speed will be accelerated.

[0038] The image enhancement module uses histogram equalization, Laplacian operator-based image enhancement, logarithmic transform-based enhancement, and gamma transform-based image enhancement to widen the gap between defects and the background. This makes it easier to detect related defects in neural network detection, improving target detection accuracy. Histogram equalization adjusts the grayscale distribution of the image, making the distribution more even across grayscale levels 0-255, thus improving image contrast and enhancing the subjective visual effect. Images with low contrast are suitable for using histogram equalization to enhance image details. The Laplacian operator can enhance local image contrast, making it suitable for enhancing images with low overall contrast, highlighting local areas. Using an 8-neighborhood Laplacian operator with a center of 4 for image convolution can achieve image sharpening and enhancement. The logarithmic transform expands the low-gray-level portion of the image, revealing more detail in this area, while compressing the high-gray-level portion, reducing detail in this area. This emphasizes the low-gray-level portion of the image. For different bases, a larger base results in stronger expansion of the low-gray-level portion and stronger compression of the high-gray-level portion. The gamma transform sets the γ value, with 1 as the dividing line. Smaller γ values ​​expand the low-gray-level portion more effectively, while larger values ​​expand the high-gray-level portion more effectively. By using different γ values, the effect of enhancing detail in either the low-gray-level or high-gray-level portion can be achieved. The gamma transform is particularly effective for images with low contrast and high overall brightness.

[0039] In this embodiment: After overall image processing, if local defects are still not obvious and unfavorable for detection by the neural network model, a local grayscale stretching image preprocessing method can be used. The core idea of ​​local grayscale stretching is to stretch the grayscale values ​​of the target area in the image, increasing the difference between the defect and the background, thereby highlighting the defective parts of the detected object's appearance. In actual operation results, the training model using local grayscale stretching significantly improved the recognition accuracy. The image contrast of the target area is enhanced after grayscale stretching, and the feature points of the defective parts are strengthened, achieving the effect of defect enhancement.

[0040] The output of the defect detection module is connected to the input of the prediction result module. When building the convolutional neural network model, the defect detection module uses a deep learning library to complete the model building and training. This solution uses the TensorFlow deep learning framework to build and train the button battery appearance defect detection network. The collected image data samples are divided into three subsets: training dataset, validation dataset, and test set. The resulting dataset is divided in a 7:1:2 ratio, with 70% randomly selected as the training dataset for the convolutional neural network, 10% as the validation set, and 20% as the test set. Sample images in the original dataset are not reused to ensure sample diversity. This tests the performance and generalization ability of the network model. After classification, data augmentation is performed to expand the number of samples. The labeled dataset is input into the MobileNetV2 network for training. After a certain number of iterations, the model with the minimum loss rate can be obtained. Images with defects are fed into the network model for prediction to obtain the defect detection results.

[0041] In this embodiment, for the detection of the same image at three different wavelengths, the dataset consists of images collected from the production line. The images are relatively stable, and the types of appearance defects are not numerous. Therefore, the network structure does not need to be too complex. Appropriately simplifying the structure can reduce network parameters and shorten training and inference time. Therefore, this solution will improve and train the MobileNetV2 convolutional neural network based on the application of appearance defect detection. After modifying MobileNetV2, the network retains the three core characteristics of MobileNetV2: depthwise separable convolution, inverse residual structure, and linear bottleneck structure. Furthermore, it adds group convolution and channel shuffling structures to the original structure. Figure 5 This improves the expressive power of the network while reducing the number of network model parameters.

[0042] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An image processing method for detecting appearance defects, comprising a first beam splitter, a second beam splitter, a first filter camera, a second filter camera, and a third filter camera, characterized in that, The second and third filter cameras are located below the second and first beam splitters, respectively. The first filter camera is located to the right of the second beam splitter. The outputs of the first, second, and third filter cameras are connected to the input of a PC via a USB hub. The output of the PC is connected to the input of the image processing module. The second and first beam splitters are horizontally parallel and arranged at a 45° angle to split the input beam of light into three beams, thereby eliminating optical parallax between multiple cameras and achieving pixel-level alignment of the three-channel image of the same object. The output of the image processing module is connected to the inputs of the image segmentation module, the normalization module, and the image quality enhancement module, respectively. The image segmentation module adopts a region-growing-based image segmentation method, specifically: a pixel at a specified location in the target region is selected as a seed point, and the region is expanded in a single direction with the seed point as the center. If the expansion distance d in this direction exceeds a preset threshold d0, the region retreats back to the seed point and expands to the other side. When the expansion distance to the other side exceeds the threshold d0 again, the region is determined to be a non-retained pixel and is blacked out to reduce interference. The image enhancement module performs local grayscale stretching on the target area obtained by the image segmentation module to increase the grayscale contrast between the target and the background. The outputs of the image segmentation module, the normalization processing module, and the image quality enhancement module are connected to the input of the defect detection module. The defect detection module uses an improved MobileNetV2 convolutional neural network. The improved MobileNetV2 convolutional neural network retains depthwise separable convolution, inverse residual structure and linear bottleneck structure, and adds group convolution and channel shuffling structure on the original basis. The output of the defect detection module is connected to the input of the prediction result module.

2. The image processing method for detecting appearance defects according to claim 1, characterized in that, The region growing mainly adopts two methods. One is to determine a growth point, i.e. a seed region, for the object to be segmented in the image, and expand outward from the seed region according to the set conditions to finally achieve the purpose of locating the entire target object. The other is to divide the entire image into several small regions according to similarity, and then integrate them to achieve the purpose of segmentation.

3. The image processing method for detecting appearance defects according to claim 1, characterized in that, The normalization processing module can improve the accuracy of the model by standardizing the image. For example, in the K nearest neighbor algorithm, it is necessary to calculate the distance between the point to be classified and all instance points.

4. The image processing method for detecting appearance defects according to claim 1, characterized in that, The image enhancement module uses histogram equalization, Laplacian-based image enhancement, logarithmic transform-based enhancement, and gamma transform-based image enhancement to widen the gap between defects and the background, making it easier to detect related defects in neural network detection and improving the accuracy of target detection.

5. The image processing method for detecting appearance defects according to claim 4, characterized in that, Histogram equalization adjusts the grayscale distribution of an image to make the distribution more even across grayscale levels 0-255, thereby improving image contrast and enhancing the subjective visual effect of the image. Images with low contrast are suitable for using histogram equalization to enhance image details.

6. The image processing method for detecting appearance defects according to claim 4, characterized in that, The Laplacian operator can enhance local image contrast, making it suitable for enhancing images with indistinct overall contrast and highlighting local features. Convolving the image with an 8-neighborhood Laplacian operator centered at 4 can achieve the purpose of sharpening and enhancing the image.

7. The image processing method for detecting appearance defects according to claim 4, characterized in that, The logarithmic transformation can expand the low grayscale portion of an image, revealing more detail in the low grayscale portion, while compressing the high grayscale portion, reducing the detail in the high grayscale portion. This achieves the purpose of emphasizing the low grayscale portion of the image. For different bases, the larger the base, the stronger the expansion of the low grayscale portion and the stronger the compression of the high grayscale portion.

8. The image processing method for detecting appearance defects according to claim 4, characterized in that, The method involves setting a γ value based on gamma transform. The γ value is divided by 1. The smaller the value, the stronger the expansion effect on the low grayscale part of the image. The larger the value, the stronger the expansion effect on the high grayscale part of the image. By using different γ values, the details of the low grayscale or high grayscale parts can be enhanced. Gamma transform has a significant image enhancement effect when the image contrast is low and the overall brightness value is high.

Images