Defect detection method and system based on multi-threshold segmentation and density clustering
Through the multi-threshold segmentation and density clustering methods, the problem of difficult to take into account both detection rate and efficiency in industrial defect detection is solved, and efficient and accurate defect detection is achieved to adapt to multiple categories and irregular defects.
Patent Information
- Application Number
- CN202510462573.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-14
- Publication Date
- 2025-07-25
AI Technical Summary
The prior art is difficult to meet both high detection rate and high detection efficiency in industrial defect detection. Traditional methods are sensitive to light changes and have poor generalization. Deep learning methods are difficult to cover irregular defects and have large calculations, making it difficult to meet real-time requirements.
Multi-threshold segmentation and density clustering are used to threshold segmentation and pixel clustering of images by constructing a semantic segmentation model, defect profiles are generated and detection results are synthesized, and parameters are dynamically adjusted to adapt to different scenarios.
It improves the accuracy and efficiency of defect detection, suppresses noise, enhances the detection rate of small defects, adapts to multiple categories of defects, and reduces positioning errors, and meets the adaptability of the entire scene.
Smart Images
Figure CN120374556A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a defect detection method and system based on multi-threshold segmentation and density clustering. Background Art
[0002] In the rapidly developing manufacturing and publishing industries, the quality of printed products is directly related to the company's brand image and market competitiveness. Limited by the production environment and equipment technology, some defects are inevitable on the printed surface, making the importance of industrial defect detection increasingly prominent.
[0003] At present, traditional industrial defect detection methods include the following types:
[0004] The first is traditional image processing technology, such as edge detection, threshold segmentation, and morphological operations; the second is deep learning-based target detection, such as Faster R-CNN, YOLO and other deep learning methods; the third is deep learning-based semantic segmentation, such as U-Net, DeepLab, etc. However, traditional image processing technology relies on artificial features, is sensitive to lighting changes and noise, requires repeated parameter adjustments, has poor generalization, and has problems such as high false detection rate and low efficiency; and deep learning-based target detection is difficult to cover irregular defects, and is prone to missed detection for small targets, and is easily affected by background interference, resulting in a low detection rate; the semantic segmentation decision process based on deep learning is difficult to understand, the segmentation model has many weight parameters, and when processing high-resolution images, the amount of calculation is large, making it difficult to meet real-time requirements.
[0005] Therefore, there is a need for an industrial defect detection method that can meet both defect detection rate and detection efficiency. Summary of the invention
[0006] To solve the above problems, the present invention provides a defect detection method and system based on multi-threshold segmentation and density clustering. By constructing a semantic segmentation model, the collected images are semantically segmented according to the defect type, and clustered according to different defect categories to obtain the outline of each defect, and the defect outline is synthesized with the original image to obtain the detection result. Threshold segmentation is performed according to different categories of defects, which improves the detection efficiency while ensuring the detection accuracy, and solves the problem that the defect detection rate and detection efficiency cannot be achieved simultaneously in the prior art.
[0007] To achieve the above object, the present invention provides the following technical solutions:
[0008] A defect detection method based on multi-threshold segmentation and density clustering includes the following steps:
[0009] S1. Collect the original image, pre-process the original image, and obtain the product image;
[0010] S2. Construct a semantic segmentation model, input the product image into the semantic segmentation model. The defects include several categories. The semantic segmentation model performs threshold segmentation on the product image according to the defect categories to obtain a mask image;
[0011] S4. According to the defect categories, perform pixel clustering on the mask image to obtain a clustering result;
[0012] S4. Generate several defect contours according to the clustering result, and synthesize the defect contours with the original image to obtain a defect detection result.
[0013] Further, in step S1, the preprocessing of the original image is specifically implemented as follows:
[0014] Scale the original image to a fixed size through affine transformation to obtain a product image, an affine transformation matrix, and an inverse transformation matrix.
[0015] Further, in step S2, the semantic segmentation model is an ultra-fast semantic segmentation model.
[0016] Further, after constructing the semantic segmentation model in step S2, it also includes training the semantic segmentation model. The specific implementation method is as follows:
[0017] Obtain a defect image dataset. The defect image dataset includes several product defect images with annotations. Each time during training, input a product defect image into the semantic segmentation model. The semantic segmentation model performs semantic segmentation on the defects in the product defect image to obtain a semantic segmentation result. Calculate the prediction loss between the semantic segmentation result and the annotation of the product defect image. Optimize the semantic segmentation model according to the prediction loss until the number of training times reaches a preset number or the prediction loss is less than a preset threshold, and end the training to obtain a trained semantic segmentation model.
[0018] Further, the semantic segmentation model performs threshold segmentation on the product image according to the defect categories. The specific implementation method is as follows:
[0019] The semantic segmentation model includes several channels. The number of channels is the same as the number of defect categories. Each channel corresponds to a defect category. Different defect thresholds are set for each defect category. Input the product image into each channel of the semantic segmentation model respectively. Each channel calculates the classification probability for each pixel in the product image. The classification probability is the probability that the pixel belongs to the defect category corresponding to the channel. According to the defect threshold of the defect category, when the classification probability of the pixel is less than the defect threshold, set the classification probability of the pixel to zero;
[0020] After all channels are calculated, for each pixel in the product image, the defect category with the highest classification probability is taken as the defect classification result of the pixel. If the classification probability of the pixel for any defect category is 0, the defect classification result of the pixel is set as the background. After classifying all pixels in the product image, a mask image is obtained.
[0021] Further, in step S3, the pixel clustering of the mask image is specifically implemented as follows:
[0022] For the pixels of each defect category, the neighborhood radius and the minimum number of samples are calculated according to the mask image. According to the neighborhood radius and the minimum number of samples, the pixels belonging to the defect category in the mask image are subjected to adaptive density clustering, and the pixels belonging to the same defect are divided into the same defect cluster until the adaptive density clustering of all categories of defects is completed, and the pixel clustering is completed.
[0023] Further, the calculation formula of the neighborhood radius eps is:
[0024]
[0025] Where α is an empirical coefficient, which is adjusted according to the defect category, c is the reference physical size, which is the diameter of the smallest defect in the physical object, S is the physical size ratio, that is, the size ratio between the image and the reality, Rbase is the reference resolution, and R is the product image resolution;
[0026] The minimum number of samples N min The calculation formula of is:
[0027]
[0028] Where β is the basic minimum number of samples, D is the defect density, Dbase is the reference defect density, A is the image area, and the calculation formula of the image area A is:
[0029] A = h × w × S 2
[0030] Where h is the image height of the product image, w is the image width of the product image, and S is the physical size ratio.
[0031] Further, in step S4, the generation of several defect contours according to the clustering results is specifically implemented as follows:
[0032] For each defect cluster, the center point of the defect cluster is calculated according to the average coordinates of the defect cluster pixels, and a positioning frame is generated according to the pixel farthest from the center point of the defect cluster in the defect cluster. The positioning frame does not exceed the edge of the product image, and the positioning frames of all defect clusters are the defect contours.
[0033] Further, in step S4, the synthesis of the defect contour and the original image to obtain the defect detection result is specifically implemented as follows: The defect contour is restored to the size of the original image through the inverse transformation matrix to obtain the positioning contour. After non-maximum suppression is performed on the positioning contour, it is synthesized with the original image to obtain the defect detection result.
[0034] Through the above technical solution, the present invention has the following beneficial effects:
[0035] Multi-threshold detection is set for different channels, which suppresses noise and improves the detection rate of small defects, increases the adaptability of multi-category defect detection, and uses adaptive clustering to generate accurate contours. Parameters are dynamically calculated according to the resolution and defect density, enabling full-scene adaptability to cover irregular defects. At the same time, dynamic calibration through the inverse matrix reduces the pixel error of defect positioning. Description of the Drawings
[0036] Figure 1 It is a schematic diagram of the overall process of a defect detection method based on multi-threshold segmentation and density clustering according to the present invention.
[0037] Figure 2 It is a schematic diagram of the structure of a defect detection system based on multi-threshold segmentation and density clustering in an embodiment of the present invention. Detailed Embodiments
[0038] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0039] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the drawings and specific embodiments.
[0040] Example 1
[0041] See Figure 1 , a defect detection method based on multi-threshold segmentation and density clustering, including the following steps:
[0042] S1. Collect the original image, preprocess the original image to obtain the product image;
[0043] S2. Construct a semantic segmentation model, input the product image into the semantic segmentation model. The defects include several categories, and the semantic segmentation model performs threshold segmentation on the product image according to the defect categories to obtain the mask image;
[0044] S3. Cluster the pixels of the mask image according to the defect category to obtain a clustering result;
[0045] S4. Generate a number of defect contours according to the clustering result, and synthesize the defect contours with the original image to obtain a defect detection result.
[0046] In an alternative embodiment, in step S1, the preprocessing of the original image is specifically implemented as follows:
[0047] Scale the original image to a fixed size through affine transformation to obtain a product image, an affine transformation matrix, and an inverse transformation matrix.
[0048] Since the neural network model has size requirements for the input image, the image needs to be scaled to a unified size for the neural network to perform calculations. Different from simple cropping or stretching, the scaling of affine transformation maintains the geometric structure of the image through linear interpolation. It can standardize the input data to adapt to the model structure while keeping the geometric structure stable, avoiding feature distortion caused by non-linear deformation, so as to ensure the robustness of image processing.
[0049] In an alternative embodiment, in step S2, the semantic segmentation model is an ultra-fast semantic segmentation model.
[0050] The ultra-fast semantic segmentation model, namely PP-LiteSeg, includes an encoder optimized by depthwise separable convolution, a multi-branch feature fusion module (UAFM), and a spatial enhancement module (SPD). It can extract multi-level features through the encoder, capture multi-scale context information at the same time, and refine the boundary features through a decoder with an attention mechanism. It has a certain sensitivity to small targets and strong anti-interference ability, and is suitable for the detection of small defects.
[0051] In an alternative embodiment, after building the semantic segmentation model in step S2, it further includes training the semantic segmentation model. The specific implementation method is as follows:
[0052] Obtain a defect image dataset, where the defect image dataset includes a number of product defect images with annotations. Each time during training, input a product defect image into the semantic segmentation model. The semantic segmentation model performs semantic segmentation on the defects in the product defect image to obtain a semantic segmentation result. Calculate the prediction loss between the semantic segmentation result and the annotation of the product defect image, and optimize the semantic segmentation model according to the prediction loss until the number of training times reaches a preset number or the prediction loss is less than a preset threshold, then end the training to obtain a trained semantic segmentation model.
[0053] Specifically, quantize the model into an onnx model and use the ONNX Runtime framework to accelerate the model inference.
[0054] In an optional embodiment, the semantic segmentation model performs threshold segmentation on the product image according to the defect category, and the specific implementation method is as follows:
[0055] The semantic segmentation model includes a number of channels, and the number of channels is the same as the number of defect categories. Each channel corresponds to a defect category, and different defect thresholds are set for each defect category. The product image is input into each channel of the semantic segmentation model respectively. Each channel calculates the classification probability for each pixel in the product image. The classification probability is the probability that the pixel belongs to the defect category corresponding to the channel. According to the defect threshold of the defect category, when the classification probability of the pixel is less than the defect threshold, the classification probability of the pixel is set to zero;
[0056] After all channels are calculated, for each pixel in the product image, the defect category with the largest classification probability is taken as the defect classification result of the pixel. If the classification probability of the pixel for any defect category is 0, the defect classification result of the pixel is the background. After classifying all pixels in the product image, a mask image is obtained.
[0057] The image is calculated and processed using different channels. First, softmax calculation is performed on the logits value output by the model. The dimension of the model output result is N×2500×2500, where N is the number of defect categories and the background. The probability of each pixel for different defect categories can be obtained. Independent thresholds are set for different category channels. The threshold settings for different defect categories are appropriate values obtained from actual experiments. By setting multiple thresholds, the pixels below the threshold are set to zero to suppress noise and avoid misjudgment in subsequent defect classification.
[0058] After the pixel channels of each category are suppressed by the threshold, they are input into the argmax function to take the channel with the largest probability as the final classification result, generating a single-channel mask image. Different values of each pixel in the mask image represent different categories.
[0059] In an optional embodiment, in step S3, the pixel clustering of the mask image is specifically implemented as follows:
[0060] For the pixels of each defect category, the neighborhood radius and the minimum number of samples are calculated according to the mask image. According to the neighborhood radius and the minimum number of samples, the pixels belonging to the defect category in the mask image are adaptively density-clustered, and the pixels belonging to the same defect are divided into the same defect cluster until the adaptive density clustering of all categories of defects is completed, and the pixel clustering is completed.
[0061] Dynamic density clustering for different defect categories is achieved through adaptive parameter calculation, which not only overcomes the limitations of traditional fixed-parameter clustering for sparse and irregular defects, but also avoids the interference of multi-defect overlap through step-by-step category priority processing, improving the clustering accuracy. At the same time, the algorithm can automatically adjust the density threshold according to the defect pixel distribution characteristics, taking into account adaptability and robustness, and providing a more accurate and structured pixel-level representation for defect localization, quantitative statistics, and morphological analysis.
[0062] In an optional embodiment, the calculation formula for the neighborhood radius eps is:
[0063]
[0064] where α is an empirical coefficient adjusted according to the defect category, c is the reference physical size, which is the diameter of the smallest defect in the physical object, S is the physical size ratio, that is, the size ratio of the image to the reality, Rbase is the reference resolution, and R is the product image resolution;
[0065] The minimum sample number N min The calculation formula is:
[0066]
[0067] where β is the basic minimum sample number, D is the defect density, Dbase is the reference defect density, A is the image area, and the calculation formula for the image area A is:
[0068] A = h × w × S 2
[0069] where h is the image height of the product image, w is the image width of the product image, and S is the physical size ratio.
[0070] In an optional embodiment, in step S4, generating several defect contours according to the clustering result, and its specific implementation method includes:
[0071] For each defect cluster, calculate the center point of the defect cluster according to the average coordinates of the defect cluster pixels, generate a positioning box according to the pixel farthest from the center point of the defect cluster in the defect cluster, the positioning box does not exceed the edge of the product image, and the positioning boxes of all defect clusters are the defect contours.
[0072] In an optional embodiment, connect the pixels farthest from the outermost pixels of the defect cluster in the defect cluster to form a defect contour.
[0073] In an optional embodiment, the coordinates closest to the center point in each cluster can also be selected, and a positioning box is formed by expanding a preset pixel value according to the coordinates, and the purpose of the positioning box is to cover the defect edge.
[0074] In an optional embodiment, in step S4, the synthesis of the defect contour and the original image to obtain the defect detection result is specifically implemented as follows: the defect contour is restored to the size of the original image through an inverse transformation matrix to obtain a positioning contour, and after non-maximum suppression is performed on the positioning contour, it is synthesized with the original image to obtain the defect detection result.
[0075] The defect contour is processed through an inverse transformation matrix to ensure that the defect contour can correspond to the original image. The methods of setting affine transformation and obtaining the inverse transformation matrix are such that while preprocessing and detecting the original image, the detection result can be restored to the original size without affecting the reading of the detection result.
[0076] Example 2
[0077] See Figure 2 , a defect detection system based on multi-threshold segmentation and density clustering, including:
[0078] An image processing module, configured to collect an original image, preprocess the original image, and obtain a product image;
[0079] A semantic segmentation module, configured to build a semantic segmentation model, input the product image into the semantic segmentation model. The defects include several categories, and the semantic segmentation model performs threshold segmentation on the product image according to the defect categories to obtain a mask image;
[0080] A pixel clustering module, configured to perform pixel clustering on the mask image according to the defect categories to obtain a clustering result;
[0081] A contour synthesis module, generates several defect contours according to the clustering result, and synthesizes the defect contours with the product image to obtain the defect detection result.
[0082] The embodiments disclosed in this specification are only an illustration of the unilateral features of the present invention. The protection scope of the present invention is not limited to this embodiment, and any other functionally equivalent embodiments fall within the protection scope of the present invention. For those skilled in the art, various corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all these changes and deformations should fall within the protection scope of the claims of the present invention.
Claims
1. A defect detection method based on multi-threshold segmentation and density clustering, characterized in that, Including the following steps: S1. Collect the original image, preprocess the original image to obtain the product image; S2. Construct a semantic segmentation model, input the product image into the semantic segmentation model. The defects include several categories. The semantic segmentation model performs threshold segmentation on the product image according to the defect categories to obtain a mask image; S3. According to the defect categories, perform pixel clustering on the mask image to obtain a clustering result; S4. Generate several defect contours according to the clustering result, and synthesize the defect contours with the original image to obtain the defect detection result.
2. The defect detection method based on multi-threshold segmentation and density clustering according to claim 1, wherein, In step S1, the specific implementation of the preprocessing of the original image is as follows: Scale the original image to a fixed size through affine transformation to obtain the product image, the affine transformation matrix and the inverse transformation matrix.
3. A defect detection method based on multi-threshold segmentation and density clustering according to claim 1, characterized in that, In step S2, the semantic segmentation model is an ultra-fast semantic segmentation model.
4. A defect detection method based on multi-threshold segmentation and density clustering according to claim 1, characterized in that, In step S2, after constructing the semantic segmentation model, it also includes training the semantic segmentation model. The specific implementation is as follows: Obtain a defect image dataset. The defect image dataset includes several product defect images with annotations. Each time during training, input a product defect image into the semantic segmentation model. The semantic segmentation model performs semantic segmentation on the defects in the product defect image to obtain a semantic segmentation result. Calculate the prediction loss between the semantic segmentation result and the annotation of the product defect image. Optimize the semantic segmentation model according to the prediction loss until the number of training times reaches the preset number of times or the prediction loss is less than the preset threshold, end the training, and obtain the trained semantic segmentation model.
5. A defect detection method based on multi-threshold segmentation and density clustering according to claim 1, characterized in that, The specific implementation of the semantic segmentation model performing threshold segmentation on the product image according to the defect categories is as follows: The semantic segmentation model includes several channels. The number of channels is the same as the number of defect categories. Each channel corresponds to a defect category. Different defect thresholds are set for each defect category. Input the product image into each channel of the semantic segmentation model respectively. Each channel calculates the classification probability for each pixel in the product image. The classification probability is the probability that the pixel belongs to the defect category corresponding to the channel. According to the defect threshold of the defect category, when the classification probability of the pixel is less than the defect threshold, set the classification probability of the pixel to zero; After all channels are calculated, for each pixel in the product image, take the defect category with the largest classification probability as the defect classification result of the pixel. If the classification probability of the pixel for any defect category is 0, then the defect classification result of the pixel is the background. After classifying all pixels in the product image, obtain the mask image.
6. The defect detection method based on multi-threshold segmentation and density clustering according to claim 5, wherein In step S3, the specific implementation of the pixel clustering on the mask image is as follows: For the pixels of each defect category, calculate the neighborhood radius and the minimum number of samples according to the mask image. According to the neighborhood radius and the minimum number of samples, perform adaptive density clustering on the pixels belonging to the defect category in the mask image, and divide the pixels belonging to the same defect into the same defect cluster until the adaptive density clustering of all categories of defects is completed, and the pixel clustering is completed.
7. A defect detection method based on multi-threshold segmentation and density clustering according to claim 6, characterized in that The calculation formula for the neighborhood radius eps is: Among them, α is an empirical coefficient, which is adjusted according to the defect category, c is the reference physical size, which is the diameter of the smallest defect in the physical object, S is the physical size ratio, that is, the size ratio between the image and the reality, Rbase is the reference resolution, and R is the product image resolution; Minimum sample number N min The calculation formula is as follows: Among them, β is the basic minimum number of samples, D is the defect density, Dbase is the reference defect density, A is the image area, and the calculation formula for the image area A is: A = h × w × S 2 Among them, h is the image height of the product image, w is the image width of the product image, and S is the physical size ratio.
8. A defect detection method based on multi-threshold segmentation and density clustering according to claim 6, characterized in that, In step S4, the generation of several defect contours according to the clustering result, the specific implementation method includes: For each defect cluster, calculate the center point of the defect cluster according to the average pixel coordinates of the defect cluster, generate a positioning frame according to the pixel farthest from the center point of the defect cluster in the defect cluster, the positioning frame does not exceed the edge of the product image, and the positioning frames of all defect clusters are the defect contours.
9. A defect detection method based on multi-threshold segmentation and density clustering according to claim 2, characterized in that In step S4, the synthesis of the defect contour and the original image to obtain the defect detection result, the specific implementation method includes: restoring the defect contour to the size of the original image through the inverse transformation matrix to obtain the positioning contour, performing non-maximum suppression on the positioning contour and then synthesizing it with the original image to obtain the defect detection result.
10. A defect detection system based on multi-threshold segmentation and density clustering, characterized in that, Including: An image processing module, configured to collect the original image, preprocess the original image to obtain the product image; A semantic segmentation module, configured to build a semantic segmentation model, input the product image into the semantic segmentation model, the defects include several categories, and the semantic segmentation model performs threshold segmentation on the product image according to the defect category to obtain a mask image; A pixel clustering module, configured to perform pixel clustering on the mask image according to the defect category to obtain a clustering result; A contour synthesis module, which generates several defect contours according to the clustering result, and synthesizes the defect contours with the product image to obtain the defect detection result.