Glass defect detection method and system, computer equipment and storage medium

Abstract

The glass defect detection method comprises the following steps: shooting glass to be detected to obtain an original image; processing the original image by using a pre-trained deep learning model to obtain a basic defect area and a defect type; generating a key image based on the basic defect area, and preprocessing the key image; generating a self-adaptive segmentation threshold value through a histogram smoothing algorithm and a double-peak average value algorithm, and segmenting the preprocessed key image based on the self-adaptive segmentation threshold value to obtain at least one accurate defect area; performing moment operation on the accurate defect area to obtain an image moment, and generating defect characteristics based on the image moment; and fusing the defect type and the defect characteristic to obtain a defect detection result. According to the method, firstly, the defect type in the original image can be accurately detected by using the deep learning model, the basic defect area containing the defect is extracted, and then the key image is processed, so that an accurate defect detection result can be obtained.

Description

Technical Field

[0001] This invention relates to the field of glass defect detection technology, specifically to a glass defect detection method, system, computer equipment, and storage medium. Background Technology

[0002] Glass, as an important industrial material, is widely used in construction, automobiles, electronic displays, and home furnishings. Due to its transparency, weather resistance, and aesthetic appeal, glass products require extremely high quality. However, during the glass production process, various defects inevitably occur on the glass surface or inside, such as bubbles, inclusions, impurities, scratches, cracks, and optical distortions, influenced by factors such as raw materials, production processes, and the environment. These defects not only affect the appearance and performance of the glass but may also reduce its mechanical strength and safety performance, leading to product defects or even safety accidents. Therefore, achieving efficient and accurate defect detection during glass production is crucial.

[0003] Traditional glass defect inspection mainly relies on manual visual inspection or simple optical instruments. Manual inspection is usually carried out by experienced workers using strong light or light sources at specific angles. However, this method is inefficient, labor-intensive, and susceptible to subjective factors, making it difficult to meet the high-speed and high-precision requirements of modern production lines. While optical or laser-based instrument inspection has improved inspection efficiency to some extent, it still has limitations in detecting minute defects, complex backgrounds, or high-transmittance glass, and the equipment is expensive and lacks adaptability.

[0004] In recent years, with the rapid development of computer vision and deep learning technologies, defect detection methods based on machine learning have gradually become a research hotspot. Deep learning models can automatically detect input glass images, significantly improving detection efficiency. However, in existing technologies, most glass detection methods rely entirely on deep learning models, that is, they depend entirely on deep learning models to detect defect categories and characteristics. Given the many types of glass defects and the complexity of actual defect conditions, it is difficult to train a deep learning model whose detection accuracy fully meets the needs of industrial production. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a glass defect detection method, system, computer device, and storage medium. First, a deep learning model is used to accurately detect the defect type in the original image and extract the basic defect region containing the defect. Then, after processing the key image, accurate defect detection results can be obtained.

[0006] To achieve the above objectives, the specific solution adopted by the present invention is as follows: a glass defect detection method, comprising the following steps: The original image of the glass to be tested is obtained by taking a picture; The original image is processed using a pre-trained deep learning model to obtain the basic defect region and defect type. The deep learning model includes convolutional layers built based on deformable convolution and / or dynamic serpentine convolution. Key images are generated based on the basic defect areas, and the key images are preprocessed. An adaptive segmentation threshold is generated by using a histogram smoothing algorithm and a bimodal average algorithm. Based on the adaptive segmentation threshold, the preprocessed key image is segmented to obtain at least one accurate defect region. The image moments are obtained by performing moment operations on the precise defect region, and the defect characteristics are generated based on the image moments; By combining defect type and defect characteristics, defect detection results are obtained.

[0007] As a further optimization of the aforementioned glass defect detection method, in deformable convolution and dynamic serpentine convolution, an offset is added and accumulated using a bilinear interpolation algorithm.

[0008] As a further optimization of the above-mentioned glass defect detection method, the method for preprocessing key images includes: A difference image is obtained by calculating the difference between the key image and a pre-set standard image based on grayscale values. The method for calculating the difference image is as follows: ; or ; in, The pixel grayscale values ​​of the key image. The pixel grayscale value of a standard image. The result is the difference in image calculation.

[0009] As a further optimization of the above-mentioned glass defect detection method, the method for preprocessing key images includes: Nonlinear filtering is applied to the key images after difference calculation; Smoothing and noise reduction are performed on key images that have undergone nonlinear filtering.

[0010] As a further optimization of the aforementioned glass defect detection method, a method for generating an adaptive segmentation threshold using a histogram smoothing algorithm and a bimodal averaging algorithm includes: Calculate the average pixel grayscale value and standard deviation of the preprocessed key image; The boundary point is determined based on the average pixel gray value, and the first position of the pixel gray value maximum and the second position of the pixel gray value minimum are determined based on the boundary point. Calculate the distance between the first position and the second position. If the distance is less than the preset standard deviation, use the pixel gray value corresponding to the midpoint between the first position and the second position as the adaptive segmentation threshold. If the distance is greater than the preset standard deviation, continue. Calculate the median pixel grayscale value of the preprocessed key image; If the median gray value of a pixel is greater than the standard deviation, the boundary point is adjusted, and the first and second positions are updated based on the adjusted boundary point. If the distance between the updated first and second positions is less than the standard deviation, the pixel grayscale value corresponding to the midpoint between the first and second positions is used as the adaptive segmentation threshold.

[0011] As a further optimization of the above-mentioned glass defect detection method, the method for obtaining image moments by performing moment operations on the precise defect region is as follows: ; in, , (x, y) are pixel coordinates. This represents the pixel grayscale value.

[0012] As a further optimization of the above-mentioned glass defect detection method: the defect characteristics include tilt and eccentricity; The method for calculating the tilt is as follows: ; The method for calculating eccentricity is as follows: .

[0013] A glass defect detection system for implementing the above-described glass defect detection method, the system comprising: The image acquisition module is used to acquire the original image; The model execution module is used to process the original image using a pre-trained deep learning model to obtain the basic defect regions and defect types; The data processing module is used to generate key images based on basic defect regions, preprocess the key images, generate adaptive segmentation thresholds using histogram smoothing algorithm and bimodal average algorithm, segment the preprocessed key images based on the adaptive segmentation thresholds to obtain at least one precise defect region, perform moment operations on the precise defect regions to obtain image moments, and generate defect characteristics based on the image moments. The results generation module is used to fuse defect types and defect characteristics to obtain and output defect detection results.

[0014] Computer equipment, including: Memory, used to store computer programs; A processor is used to read and execute the computer program to implement the glass defect detection method described above.

[0015] A storage medium for storing a computer program that, when executed, implements the aforementioned method for detecting glass defects.

[0016] Beneficial effects: This invention first utilizes a deep learning model to accurately detect the type of defects in the original image and extract the basic defect region containing the defects. After processing the key images, accurate defect detection results can be obtained. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 These are example images of the basic defect areas and defect types detected by the deep learning model in the experiment. Figure 3 This is a diagram showing the defect detection results during the experiment. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, the present invention first provides a glass defect detection method, including S1 to S6.

[0020] S1. The glass to be inspected is photographed to obtain the original image. Specifically, the image acquisition equipment used to photograph the glass includes an industrial camera and a light source positioned on one or both sides of the glass to be inspected. Both the industrial camera and the light source are conventional technologies in the field and will not be described in detail here. Furthermore, the original image needs to be processed into an 8-bit grayscale image. The specific processing method is also conventional technology in the field and will not be described in detail here.

[0021] S2. The original image is processed using a pre-trained deep learning model to obtain the basic defect region and defect type. The deep learning model includes convolutional layers built based on deformable convolution and / or dynamic serpentine convolution.

[0022] Common defects in glass include coating defects, bubbles, scratches, stones, water stains, and dirt. Among these, scratches are characterized by their elongated and complex shape, while other defects often exhibit similar characteristics. To enable deep learning models to better identify these defects in the original image, this invention utilizes convolutional layers constructed with deformable convolutions and / or dynamic serpentine convolutions. This enhances the deep learning model's adaptability to defect morphologies and improves segmentation accuracy through adaptive focusing on local features, thereby enhancing the ability to identify elongated defects.

[0023] The deep learning model of this invention can be obtained by improving existing models through modifications to convolutional layers. For example, it can be based on the YOLOv5 model, which includes a backbone network, a neck network, and an output module. The backbone network extracts features from the input image, extracting defective parts from the image through convolutional modules for subsequent object detection. The neck network mixes and combines the features extracted by the backbone network, and the fused features are then passed to the output module for prediction. Based on this, the deep learning model can be obtained by replacing the convolutional modules in the backbone network with convolutional layers constructed using deformable convolutions or dynamic serpentine convolutions. The specific working principle and implementation of the YOLOv5 model are mature existing technologies and will not be elaborated further here.

[0024] Furthermore, a Focus layer is included at the input of the backbone network. This Focus layer slices and reassembles the original input image. Specifically, it divides the original image into 2×2 pixel blocks, then stitches the sub-blocks together along the channel dimension, reducing the image spatial resolution to 1 / 4 while increasing the number of channels. This processing method can efficiently capture detailed features in the original image without losing feature information, laying the foundation for subsequent defect extraction. The core of the neck network consists of a spatial pyramid pooling structure and a path aggregation network. The spatial pyramid pooling structure effectively expands the receptive field of features by applying max pooling of different sizes on the same feature map, concatenating the results of multi-scale pooling, and then using convolution for dimensionality reduction. This allows deep learning models to capture feature information from both small and large defects simultaneously. The path aggregation network employs a bidirectional feature transfer path combining top-down and bottom-up approaches. The top-down path upsamples high-semantic features from the deep layers of the backbone network to the shallow layers, while the bottom-up path downsamples high-positional-precision features from the shallow layers to the deep layers. The fusion of these two approaches forms a comprehensive feature that combines semantic and positional information. This fused feature is then passed to the output module for prediction. The output module uses a multi-scale prediction mechanism to predict different scale feature maps output by the neck network, thereby covering defect targets of different sizes and avoiding missed or false detections.

[0025] Furthermore, the kernel size of the convolutional layer is set to 3×3, and is specifically represented as follows: ; Here, x and y are the pixel coordinates in the original image. This approach gives the convolutional kernel greater flexibility, allowing it to better focus on the complex set of features of defects. Furthermore, it enables an iterative strategy, selecting the next position of each target to be processed for observation, thus ensuring the deep learning model focuses on the continuity of the target. On the other hand, by setting the offset to 1, the position of the convolutional kernel can be constrained, preventing deviation from the target due to excessive deformation.

[0026] Furthermore, to further improve the performance of deep learning models, bilinear interpolation is used to add offsets and accumulate them in deformable convolution and dynamic serpentine convolution. This, combined with structured loss and continuity constraints, further improves the segmentation accuracy of defective parts in the original image. The specific method is as follows: , where K' is the input feature map, K is the feature point to be interpolated in the output feature map, and B(K',K) is the bilinear weight function.

[0027] To guide the deep learning model to supplement the attention to defects from multiple perspectives, for each K, two feature maps are extracted from the output of the l-th convolutional layer, one from the x-axis and the other from the y-axis. and Then, they are integrated, and the specific method is as follows: ; in, For position Weight at each location.

[0028] Considering that glass defects are rare in modern manufacturing processes, most areas in the original image are defect-free, meaning that most areas are not the target regions that deep learning models need to focus on. To balance the negative impact of the large area difference between target and non-target regions, this invention uses DiceLoss as the loss function in the deep learning model, causing the neural network to focus more on learning the target region during training. Specifically, this is expressed as follows: ; Where X is the true target mask, Y is the predicted target mask, and Smooth is the smoothing factor. The larger the intersection of X and Y, the better the performance of the deep learning model. However, the loss function needs to gradually decrease to converge, so a negative sign is added before the ratio.

[0029] To further improve the performance of deep learning models, a dynamic label classification strategy, TAL (Task Alignment Learning), is introduced during the training process. This strategy describes the quality of the predicted bounding boxes, guiding the deep model to strengthen its performance on the weaker task between the presence and type of defects. The implementation principle of TAL can be represented as follows: ; Where s and u are the classification score and IoU value, respectively, and α and β are weight hyperparameters. T can achieve task alignment by optimizing both the classification score and IoU, guiding the deep learning model to dynamically focus on high-quality defect prediction boxes.

[0030] S3. Generate key images based on the basic defect areas and preprocess the key images.

[0031] More specifically, the methods for preprocessing key images include calculating the difference between the key image and a pre-set standard image based on grayscale values ​​to obtain a difference image, performing nonlinear filtering on the key image after difference calculation, and smoothing and denoising the key image after nonlinear filtering.

[0032] The method for calculating the difference in shadow is as follows: ; or .

[0033] in, The pixel grayscale values ​​of the key image. The pixel grayscale value of the standard image. The result is the difference in image calculation.

[0034] Difference image calculation can reflect the differences between the key image and the standard image, and enhance the defective parts in the key image, so as to facilitate more accurate detection of defect characteristics in subsequent processing. More specifically, the key image and the original image are both 8-bit grayscale images, so the grayscale range of a single pixel is [0, 255]. After difference image calculation with the standard image, the numerical range of the calculated difference image is [-255, 255]. Using 0 as the dividing point, difference image calculation results within the range of [-255, 0] are called negative differences, and those within [0, 255] are called positive differences. Generally, defective areas such as bubbles and stones appear darker in the original image, and the corresponding difference image calculation result is positive; defective areas with poor coating appear brighter in the original image, and the corresponding difference image calculation result is negative.

[0035] It should also be noted that the specific methods of nonlinear filtering and smoothing noise reduction mentioned above are all conventional techniques in this field, and will not be elaborated here.

[0036] S4. An adaptive segmentation threshold is generated using a histogram smoothing algorithm and a bimodal averaging algorithm. Based on this adaptive segmentation threshold, the preprocessed key image is segmented to obtain at least one precise defect region. More specifically, the method for generating the adaptive segmentation threshold using the histogram smoothing algorithm and the bimodal averaging algorithm includes steps S41 to S46.

[0037] S41. Calculate the average pixel grayscale value and standard deviation of the preprocessed key image.

[0038] S42. Determine the dividing point based on the average pixel gray value, and determine the first position of the pixel gray value maximum and the second position of the pixel gray value minimum based on the dividing point.

[0039] S43. Calculate the distance between the first position and the second position. If the distance is less than the preset standard deviation, use the pixel gray value corresponding to the midpoint between the first position and the second position as the adaptive segmentation threshold. If the distance is greater than the preset standard deviation, continue.

[0040] S44. Calculate the median grayscale value of the pixels in the preprocessed key image.

[0041] S45. If the median gray value of a pixel is greater than the standard deviation, the boundary point is adjusted, and the first and second positions are updated based on the adjusted boundary point.

[0042] S46. When the distance between the updated first position and the second position is less than the standard deviation, the pixel gray value corresponding to the midpoint between the first position and the second position is used as the adaptive segmentation threshold.

[0043] Using the above method, a more appropriate adaptive segmentation threshold can be generated based on the actual situation of the key image. Based on this adaptive segmentation threshold, the key image can be segmented better, the precise defect region can be extracted, and the defect characteristics can be detected more accurately in subsequent processes.

[0044] S5. Perform moment operations on the precise defect region to obtain image moments, and generate defect characteristics based on the image moments. More specifically, the method for performing moment operations on the precise defect region to obtain image moments is as follows: .

[0045] in, , (x, y) are pixel coordinates. This represents the pixel grayscale value.

[0046] In image moments, different values ​​of p and q have different physical meanings, specifically: Indicates the area of ​​the precise defect region. This indicates the precise distribution of defect areas in the horizontal direction. This indicates the precise distribution of defect areas in the vertical direction. Indicates the tilt of the precise defect area. This indicates the exact deviation of the center of gravity of the defect area in the horizontal direction. This indicates the exact deviation of the center of gravity of the defect area in the vertical direction. This indicates the horizontal stretching uniformity of the precise defect area. This indicates the vertical stretching uniformity of the precise defect area.

[0047] Based on the calculated image moments, the defect area information and morphological features can be accurately calculated, thus forming the defect characteristics. Among the morphological features, tilt and eccentricity are the key points. More specifically, the method for calculating tilt is as follows: ; The method for calculating eccentricity is as follows: .

[0048] S6. Combine defect type and defect characteristics to obtain defect detection results.

[0049] To verify the effectiveness of the present invention, some defective glass was obtained for testing, and the test results are as follows: Figure 2 and Figure 3 As shown. Figure 2 In the image, the left side shows the predicted bounding box output by the deep learning model, and the right side shows the extracted basic defect area. From... Figure 2 and Figure 3 As can be seen, this invention first utilizes a deep learning model to accurately detect the type of defects in the original image and extract the basic defect region containing the defects. After processing the key images, accurate defect detection results can be obtained.

[0050] The present invention further provides a glass defect detection system for implementing the above-mentioned glass defect detection method. The system includes an image acquisition module, a model running module, a data processing module, and a result generation module.

[0051] The image acquisition module is used to acquire the original image.

[0052] The model execution module is used to process the original image using a pre-trained deep learning model to obtain the basic defect regions and defect types.

[0053] The data processing module is used to generate key images based on basic defect regions, preprocess the key images, generate adaptive segmentation thresholds using histogram smoothing algorithm and bimodal average algorithm, segment the preprocessed key images based on the adaptive segmentation thresholds to obtain at least one precise defect region, perform moment operations on the precise defect regions to obtain image moments, and generate defect characteristics based on the image moments.

[0054] The results generation module is used to fuse defect types and defect characteristics to obtain and output defect detection results.

[0055] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or modules may be electrical, mechanical, or other forms.

[0056] The present invention also provides a computer device, including a memory and a processor.

[0057] Memory is used to store computer programs.

[0058] A processor is used to read and execute the computer program to implement the glass defect detection method described above.

[0059] Finally, the present invention provides a storage medium for storing a computer program that, when executed, implements the glass defect detection method described above.

[0060] The memory, as a carrier of resources, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it can include the operating system, computer programs, etc., and the storage method can be temporary or permanent storage. The operating system is used to manage and control the various hardware devices and computer programs on the electronic device, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including computer programs capable of performing the adaptive emotion regulation method based on personalized reconfigurable music disclosed in any of the foregoing embodiments, the computer programs may further include computer programs capable of performing other specific tasks. The processor can be a general-purpose processor product based on architectures such as x86, IA64, RISC, MIPS, and ARM.

[0061] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of glass defect detection, characterized by, The method comprises the following steps: taking a glass to be detected to obtain an original image; processing the original image by using a pre-trained deep learning model to obtain a basic defect area and a defect type, the deep learning model comprising a convolution layer constructed based on deformable convolution and / or dynamic snake convolution; generating a key image based on the basic defect area and pre-processing the key image; generating an adaptive segmentation threshold value by using a histogram smoothing algorithm and a bimodal average value algorithm, and segmenting the pre-processed key image based on the adaptive segmentation threshold value to obtain at least one accurate defect area; performing moment operation on the accurate defect area to obtain an image moment, and generating a defect characteristic based on the image moment; fusing the defect type and the defect characteristic to obtain a defect detection result.

2. A method of glass defect detection as claimed in claim 1, wherein, In the deformable convolution and the dynamic snake convolution, a bilinear difference algorithm is used to add an offset and perform accumulation.

3. The glass defect detection method as described in claim 1, characterized in that, The method for pre-processing the key image comprises: performing difference calculation on the key image and a pre-set standard image based on a gray value to obtain a difference image, the difference calculation method being: ; or ; wherein, is the pixel gray value of the key image, is the pixel gray value of the standard image, is the difference calculation result.

4. A method of glass defect detection as claimed in claim 3, wherein, The method for pre-processing the key image comprises: performing nonlinear filtering on the key image after the difference calculation; performing smoothing and noise reduction on the key image after the nonlinear filtering.

5. The method of glass defect detection of claim 1, wherein, The method for generating the adaptive segmentation threshold value by using the histogram smoothing algorithm and the bimodal average value algorithm comprises: calculating the average value and the standard deviation of the pixel gray value of the pre-processed key image; determining a demarcation point based on the average value of the pixel gray value, and determining a first position of a maximum pixel gray value and a second position of a minimum pixel gray value based on the demarcation point; calculating the distance value between the first position and the second position, if the distance value is less than a pre-set standard deviation, taking the pixel gray value corresponding to the middle position of the first position and the second position as the adaptive segmentation threshold value, if the distance value is greater than the pre-set standard deviation, continuing; calculating the median value of the pixel gray value of the pre-processed key image; if the median value of the pixel gray value is greater than the standard deviation, adjusting the demarcation point, and updating the first position and the second position based on the adjusted demarcation point; when the distance value between the updated first position and the second position is less than the standard deviation, taking the pixel gray value corresponding to the middle position of the first position and the second position as the adaptive segmentation threshold value.

6. The method of glass defect detection of claim 1, wherein, The method for performing moment operation on the accurate defect area to obtain the image moment comprises: ； wherein , , (x, y) is a pixel coordinate value, is a pixel gray value.

7. A method of glass defect detection as claimed in claim 6, wherein, The defect characteristic comprises an inclination and an eccentricity; The calculation method of the inclination comprises: ； The calculation method of the eccentricity comprises: 。 8. A glass defect detection system, characterized by, The system for implementing the glass defect detection method according to any one of claims 1-7 comprises: an image acquisition module for acquiring an original image; a model running module for processing the original image by using a pre-trained deep learning model to obtain a basic defect area and a defect type; a data processing module for generating a key image based on the basic defect area, pre-processing the key image, generating an adaptive segmentation threshold value by using a histogram smoothing algorithm and a bimodal average value algorithm, segmenting the pre-processed key image based on the adaptive segmentation threshold value to obtain at least one accurate defect area, performing moment operation on the accurate defect area to obtain an image moment, and generating a defect characteristic based on the image moment; A result generation module is configured to fuse the defect type and the defect characteristic to obtain a defect detection result and output the defect detection result.

9. Computer device, characterized in that The method comprises the following steps: a memory for storing a computer program; a processor for reading and executing the computer program to implement the glass defect detection method according to any one of claims 1-7.

10. Storage medium, characterized in that a computer program for storing, which is executed to implement the glass defect detection method according to any one of claims 1-7.

Images