Ceramic tile multi-scale surface defect detection method based on improved YOLOv8

Abstract

The invention discloses a ceramic tile multi-scale surface defect detection method based on improved YOLOv8, and relates to the field of target detection. The invention provides a ceramic tile surface defect detection method. The method comprises the following steps: S1, acquiring a ceramic tile surface defect image; s2, image cutting and data enhancement are carried out, and a tile surface defect image data set is constructed; s3, constructing and initializing an improved YOLOv8 convolutional neural network detection model, and inputting the constructed defect image data set into the improved YOLOv8 convolutional neural network detection model for training to obtain a corresponding ceramic tile surface defect detection neural network model; the improved YOL Ov8 convolutional neural network detection model comprises using an ASF architecture to reconstruct a neck structure of an original model and adding a tiny target detection head; designing a C2fDD module for replacing a C2f module of the original backbone network; a shared parameter detection head (SpaD) is designed. And S4, the improved YOLOv8 convolutional neural network detection model is tested and evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and target detection, and in particular to a multi-scale surface defect detection method for ceramic tiles based on improved YOLOv8. Background Art

[0002] The production process of ceramic tiles generally includes raw material mixing and grinding, dehydration, embryo pressing, inkjet printing, glazing, firing, and polishing. In such a complex production process, internal and surface defects will occur, among which common surface defects include powder balls, corner cracks, glaze dripping, ink breakage, ink dripping, etc. As a building and decorative material, the surface quality of ceramic tiles directly affects the beauty and safety of buildings. With the rapid development of artificial intelligence technology, intelligent and automated technologies have been widely used in industrial production, and the production process of ceramic tiles has basically been unmanned. However, although the production process has been highly automated, quality inspection still relies heavily on manual work. According to research, a production line usually needs to be equipped with 2-6 quality inspectors, who need to observe the surface of ceramic tiles under strong light for a long time to find defects. However, manual inspection is affected by subjective factors such as personal mood, physical condition, and quality inspection experience, which may lead to missed inspections and false inspections. Therefore, intelligent detection of ceramic tile surface defects has become an important topic in industrial inspection.

[0003] Traditional image processing methods and machine learning-based methods are often only tested on a small data set or the defect types collected are limited and do not cover all common defect types. In addition, most of the algorithms used need to rely on manually designed feature extractors to extract image features, and then use trained classifiers to identify and locate defects. They do not have adaptive learning capabilities and need to manually set parameters and adjust algorithms, which makes it difficult to meet the needs of real-time detection in industry. Deep learning models have set off a boom in recent years and have made important achievements in defect detection. Deep learning models can be adjusted and optimized according to different scenarios and data, so they have strong adaptability when facing different types of defect detection tasks. Mainstream target detection algorithms have been widely used in tile surface defect detection, such as two-stage algorithms R-CNN, Fast-RCNN, Faster-RCNN, etc., and one-stage target detection algorithms SSD, YOLO, etc. However, existing methods still face challenges in detecting tiny defects on the surface of tiles, defects similar to the background, and defects with complex defect features. Summary of the invention

[0004] In view of the above problems, the present invention provides a multi-scale surface defect detection method for ceramic tiles based on improved YOLOv8, which solves the problems existing in the current technology. By optimizing the YOLOv8 convolutional neural network detection model, the defect detection accuracy is improved while balancing the model complexity. The improved model can meet the requirements of industrial environments for ceramic tile surface defect detection.

[0005] The present invention integrates artificial intelligence technology with tile surface defect detection technology to achieve rapid and accurate identification and classification of tile surface defects. The ASF framework is used to reconstruct the neck structure of YOLOv8, and a small target detection head is added on this basis to enhance the multi-scale information extraction and detail fusion capabilities. The method focuses on the characteristics of small targets and significantly improves the detection performance of small targets. Then, the DRB modules with different kernel sizes are used to replace the expansion depth convolutions with different expansion rates in the DWR module, and the Bottleneck module in the C2f module of the original YOLOv8 model is replaced with DWR-DRB. A new C2f-DD module is designed to replace the C2f modules of the sixth and eighth layers in the original YOLOv8 backbone network. This module can enhance the ability to extract features from the scalable receptive field of the high-level network and efficiently obtain multi-scale context information, so that the network pays attention to more defect detail information during feature extraction. Finally, in order to improve the parameter utilization of the YOLOv8 detection head and reduce the model complexity, the present invention designs a shared parameter detection head (SPaD).

[0006] A multi-scale surface defect detection method for tiles based on improved YOLOv8 includes the following steps: S1, obtaining a defect image of the tile surface; S2, cutting images and data enhancement, constructing a tile surface defect image dataset; S3. Construct and initialize an improved YOLOv8 convolutional neural network detection model, input the constructed defect image dataset into the improved YOLOv8 convolutional neural network detection model for training, and obtain the corresponding tile surface defect detection neural network model; the improved YOLOv8 convolutional neural network detection model includes using the ASF architecture to reconstruct the neck structure of the original model and adding a small target detection head; designing a C2f_DD module to replace the C2f module of the original backbone network; designing a shared parameter detection head (SPaD). S4. Test and evaluate the improved YOLOv8 model;

[0007] Preferably, the ASF framework reconstructs the neck structure of YOLOv8, and the ASF architecture includes a scale sequence feature fusion module (SSFF) module and a triple feature encoding module (TFE) module. The 1-5 level feature extraction branches {P1, P2, P3, P4, P5} in the trunk of the YOLOv8 neural network detection model are then fused through the feature pyramid network to output a set of {S2, S3, S4, S5}.

[0008] Preferably, the SSFF module fuses the P2, P3, and P4 feature maps output from the backbone network.

[0009] Preferably, the TFE module combines the features of large, medium and small sizes.

[0010] Preferably, the tiny target detection head is added, information of the high-resolution P2 feature map is integrated, and output to the head for detecting tiny targets larger than 4×4px.

[0011] Preferably, the expansion depths of different expansion rates in the replacement DWR module are convolved.

[0012] Preferably, the expansion rates of the expansion depth convolution are 3 and 5 respectively.

[0013] Preferably, the DRB modules have different core sizes.

[0014] Preferably, the core sizes of the DRB modules are 5 and 7 respectively.

[0015] Preferably, the shared parameter detection head (SPaD) enables the S2, S3, S4, and S5 layer feature maps output by the neck to share the 3×3GN_Conv for feature extraction.

[0016] Beneficial technical effects of the present invention: (1) By cutting the original defect data, the problem that the original defect image has too large a resolution and directly inputting it into the model for training will occupy a large amount of video memory and small-size target information will be lost is solved. (2) Data enhancement is performed on images of edge anomalies, light block defects, and aperture defects after cutting. The generated tile surface defect images not only make the dataset diverse, but also make the sample ratios of various defects more balanced, which helps the model better learn different types of defects. (3) In view of the problems existing in the existing methods for detecting defects on tile surfaces, the YOLOv8 convolutional neural network detection model is improved, including integrating the ASF framework to reconstruct the neck structure of the original YOLOv8 detection model and adding a small target detection head based on it. By increasing the multi-scale fusion and detail information acquisition capabilities, the detection capability of small targets is significantly improved. A C2f-DD module is designed to replace the original YOLOv8 convolutional neural network detection model, so that the replaced model can obtain context information more efficiently and flexibly. This can not only understand the subtle differences in context at different levels to better distinguish objects and their backgrounds, but also prioritize relevant information while ignoring non-important information. A shared parameter detection head (SPaD) is designed to improve parameter utilization while balancing detection accuracy. The improved YOLOv8 convolutional neural network detection model not only improves the ability to detect defects, but also balances the model complexity, meeting the requirements of industrial real-time detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the multi-scale surface defect detection method of tiles based on improved YOLOv8 of the present invention Figure 2 The six defect diagrams of the present invention are Figure 3 This is the structure diagram of the scale sequence feature fusion module (SSFF) used in the present invention Figure 4 This is the structure diagram of the triple feature encoding module (TFE) used in the present invention Figure 5 This is the DWR-DRB module structure diagram used in the present invention Figure 6 This is the structure diagram of the shared parameter detection head (SPaD) designed by the present invention Figure 7 This is a comparison chart of the accuracy of the YOLOv8s model of the present invention and the improved model. Figure 8 This is a comparison chart of the test results of the YOLOv8s model of the present invention and the improved model DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is described in detail below in conjunction with specific embodiments and drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Figure 1FIG. 1 is a flow chart of a method for detecting surface defects of ceramic tiles based on an improved YOLOv8 model according to an exemplary embodiment of the present invention. Figure 1 As shown, the present invention provides a multi-scale surface defect detection method for tiles based on improved YOLOv8, and the specific steps include:

[0020] S1. Obtain a tile surface defect image, where the defect image is a public dataset of tile surface defects.

[0021] like Figure 2 As shown, the types of surface defects of tiles can be divided into six types: edge anomalies, corner anomalies, white point defects, light block defects, dark block defects and aperture defects.

[0022] S2, cutting images and data enhancement, to build a tile surface defect image dataset.

[0023] The image cutting comprises the following steps:

[0024] (1) Set parameters and paths. Enter the image path and label path, define the window length and width of the cut image as 800px, the overlap rate as 0.2, and the zero pixel ratio as 0.2.

[0025] (2) Read and process the image. According to the input image path and label file path, traverse each image, read and record the image size, coordinates of the annotation box and category information. If the image size is smaller than the cut size, fill it.

[0026] (3) Loop slicing. Traverse the image and start slicing according to the set window size and overlap ratio. When slicing, check whether the target object exists in the current slicing area. If it exists, continue processing; if not, skip saving.

[0027] (4) Process the slices. Convert the color slice image to a grayscale image, perform binarization on the grayscale image, and obtain a binary image, thereby calculating the zero pixel ratio. If it is less than the set zero pixel ratio, save the slice image. Finally, output the slice image and annotation file.

[0028] It should be noted that tile images have high resolution and contain a large number of small target defects. Directly inputting them into the YOLO model for training not only requires high hardware equipment, but also small target defects contain fewer pixels. During the training process, multiple convolutions and pooling will cause a large amount of small target information to be lost and cannot be effectively represented in the feature map. This will make it difficult for the model to accurately locate these small-sized defects.

[0029] The image enhancement comprises the following steps:

[0030] (1) The cut images are divided into training set, validation set and test set in the ratio of 7.5:1.5:1.5.

[0031] (2) The images containing edge anomalies, light block defects, and aperture defects in the training set are divided into three different folders.

[0032] (3) The three defect images are rotated 90 degrees, partially cropped, scaled 50%, and noise added.

[0033] The construction obtains a tile surface defect image dataset, and the data-enhanced images are added back into the training set to complete the construction of the defect dataset.

[0034] S3. Construct and initialize an improved YOLOv8 convolutional neural network detection model, input the constructed defect image dataset into the improved YOLOv8 convolutional neural network detection model for training, and obtain the corresponding tile surface defect detection neural network model; the improved YOLOv8 neural network detection model includes using the ASF architecture to reconstruct the neck structure of the original model and adding a small target detection head; designing a C2f_DD module to replace the C2f module of the original backbone network; and designing a shared parameter detection head (SPaD).

[0035] In a feasible embodiment, the method for initializing the YOLOv8 convolutional neural network detection model is as follows:

[0036] Configure YOLOv8 environment: Download the official code of the YOLOv8 model from GitHub, and then use Anaconda to create a virtual environment. The specific environment configuration is: matplotlib>=3.3.0, numpy>=1.22.2, opencv-python>=4.6.0, pillow>=7.1.2, pyyaml>=5.3.1, requests>=2.23.0, scipy>=1.4.1, torch>=1.8.0, torchvision>=0.9.0, tqdm>=4.64.0, tensorboard>=2.13.0, pandas>=1.1.4, seaborn>=0.11.0, psutil, py-cpuinfo, thop>=0.1.1.

[0037] In a preferred embodiment, the reconstruction of the YOLOv8 convolutional neural network detection model using the ASF architecture includes using a scale sequence feature fusion module (SSFF) and a triple feature encoding module (TFE).

[0038] In the embodiment of the present invention, the 1-5 level feature extraction branches {P1, P2, P3, P4, P5} in the trunk of the YOLOv8 convolutional neural network detection model are used. Then, after P2, P3, P4, and P5 are fused through the feature pyramid of the neck, a set of {S2, S3, S4, S5} is output.

[0039] In the embodiment of the present invention, the SSFF module structure is as follows Figure 3 As shown in the figure, first, the feature maps of different scales (i.e., P3, P4, and P5) output by the backbone network are taken as input to the SSFF module, and a 1×1 convolution is applied to the input feature map to adjust the number of channels of all feature maps to be consistent. The nearest neighbor interpolation method (Interpolat) is applied to the P4_1 and P5_1 level feature maps to adjust their sizes to be consistent with the size of the P3_1 feature map. Then, through the unsqueeze and cat expansion and splicing operations, P3_1, P4_2, and P5_2 are expanded by one dimension and then spliced ​​to obtain a fused feature map. After that, 3D convolution, 3D batch normalization, and SiLU activation function are applied to the spliced ​​feature map to complete feature extraction. Finally, downsampling is performed through the 3D pooling layer, and redundant dimensions are removed to obtain the final output.

[0040] In the embodiment of the present invention, the TFE module structure is as follows Figure 4 , the TFE module first separates the feature maps of three sizes: large, medium and small. The large-size feature map (0.5C, 2S) is subjected to convolution, pooling and SiLu activation function to output the feature map (1C, 1S), and then downsampled by maximum pooling and average pooling, and finally outputs the feature map (1C, 1S). The small-size feature map (2C, 0.5S) is subjected to convolution, pooling and SiLu activation function to output the feature map (1C, 0.5S), and then upsampled by the nearest neighbor to output the feature map (1C, 1S), and finally convolution, pooling and SiLu activation function to output the feature map (1C, 1S). The large-size feature map, medium-size feature map and small-size feature map are spliced ​​in the channel dimension to output the final feature map (3C, 1S).

[0041] In the embodiment of the present invention, the small target detection head is designed based on the ASF framework. First, the P2 layer of the small target at the neck is upsampled once, and then spliced ​​with the P2 feature map of the backbone network, which is input to the fusion layer as input1. The P2 feature map of the backbone network and the P2 and P3 feature maps after feature pyramid processing are fused by the SSFF module and output to obtain input2, which is input to the fusion layer. Finally, it is output to the Head for detecting small targets larger than 4×4px.

[0042] It should be noted that in the traditional feature pyramid network of the neck structure, the feature maps of each scale are fused through a top-down path, but the information interaction between different scales is not fully considered, resulting in incomplete information fusion. In addition, when fusing feature maps of different scales, the traditional feature pyramid network first upsamples the small-scale feature map and then fuses it with the large-scale feature map, ignoring the rich detail information in the large-scale feature map.

[0043] In a preferred embodiment, the C2f_DD module is composed of a C2f module and a DWR-DRB module formed by DRB transformation. DRBs with kernel sizes of 5 and 7 are used to replace the dilated depth convolutions with expansion rates of 3 and 5 in the DWR module to form a DWR-DRB module, and then the DWR-DRB module is used to replace the Bottleneck of the C2f module in the original YOLOv8. A C2f-DD module is designed to replace the C2f modules of the sixth and eighth layers in the original YOLOv8 backbone network.

[0044] In the embodiment of the present invention, the DWR-DRB module is as follows: Figure 5 As shown in the figure. The DWR-DRB module generates feature maps of different sizes in the form of regions after the input features are processed by a 3×3 convolution mixed with a batch normalization (BN) layer and an activation function (ReLU). Then, feature maps of different sizes are processed by a dilated depth convolution with a dilation rate of 1 and a dilated reparameter block (DRB) with a kernel size of 5 and 7 for morphological filtering. After the processing of regional residualization, semantic residualization and deep convolution, the complex semantic information acquisition is transformed into morphological filtering of concise feature maps, thereby efficiently obtaining multi-scale contextual information. Then, the features are merged through a batch normalization (BN) layer and a 1×1 convolution.

[0045] In a preferred embodiment, the structure of the shared parameter detection head (SPaD) is as follows: Figure 6 As shown in the figure, the S2, S3, S4, and S5 feature layers of the neck output are convolved with 1×1GN_Conv to adjust the number of channels, and the number of channels of all feature layers is adjusted to be consistent. Then, all feature layers are passed through a 3×3GN_Conv shared convolution layer for feature extraction.

[0046] It should be noted that the detection head of YOLOv8 adopts a decoupled head design, which is divided into two branches, classification and regression, and removes the calculation branch of positioning loss. The advantage of this is that it can handle regression and classification tasks separately, improving the detection efficiency and accuracy of the model. However, YOLOv8 uses independent detection heads at different feature layers, which not only leads to low utilization of model parameters, but also increases model complexity.

[0047] In an optional embodiment, the step of inputting the defect data set into the improved YOLOv8 neural network detection model for training comprises the following steps:

[0048] (1) Configure the training environment: The training configuration used in the present invention is GPU: NVIDIA GeForce RTX3090; video memory: 24G; CPU: Intel Corei9-10900X; CPU main frequency: 3.7GHz; memory: 64G; deep learning framework is PyTorch, and the development environment is Pytoch=2.2.2, Python=3.9.11, Cuda=12.6;

[0049] (2) Configure the data YAML file and add the processed dataset address and defect type to the data YAML file.

[0050] (3) Configure the YOLOv8 network structure yaml file.

[0051] (4) Configure the corresponding parameters: set the optimizer to SGD, set the learning rate lr0 to 0.01, the number of batch samples Batch-size = 32, the number of iterations Epoch = 800, and the random seed seed = 0.

[0052] (5) Input the constructed data set into the built improved YOLOv8 convolutional neural network model for training. The training steps are as follows: 1) Use the K-means clustering method to automatically generate a priori boxes of corresponding sizes. 2) Classify the bounding boxes through bounding box regression and classifier, and output the target category probability corresponding to each bounding box. 3) Use non-maximum suppression (NMS) to remove overlapping bounding boxes and only keep the boxes with the highest confidence, thereby reducing redundant detections. 4) During the training process, the loss between the predicted value and the true value is calculated through the loss function, and back propagation is performed according to the loss value until the preset number of iterations is reached to obtain the corresponding tile surface defect detection neural network model.

[0053] S4. Test and evaluate the improved YOLOv8 model, including the following steps:

[0054] (1) Model testing: 1) Load the trained best weight file best.pt into the test file for model testing, get the test results on the test set, and then use the best weight of the original model YOLOv8s to test on the test set. The PR curve comparison after the test is as follows: Figure 7 As shown in Figure 2, the average accuracy of the improved model is 3.7% higher than that of the original model. 2) The improved optimal weight file and the original YOLOv8s optimal weight file are then used to test six different defect images in the test set, including defect category, confidence, and defect bounding box coordinates. The comparison chart is as follows: Figure 8 As shown, Figure 8 The first and third rows are the detection results of the YOLOv8s model, and the second and fourth rows are the detection results of the improved YOLOv8 model. (a) is the test result of edge anomaly. The confidence of the original model is 0.90, and the confidence of the improved model is 0.91. (b) is the test result of corner anomaly. The confidence of the original model is 0.79, and the confidence of the improved model is 0.87. (c) is the test result of white point defect. The original model detected a defect with a confidence of 0.6 6. The improved model detected two white point defects with confidence levels of 0.69 and 0.60 respectively. (d) shows the test results of light-colored block defects. The confidence levels of the original model are 0.82 and 0.71 respectively, and the confidence levels of the improved model are 0.77 and 0.70 respectively. (e) shows the test results of black point defects with a confidence level of 0.78 and a confidence level of 0.84 for the improved model. (f) shows the test results of aperture defects. The confidence level of the original model is 0.80 and that of the improved model is 0.81.

[0055] (2) Model evaluation: Average accuracy, number of parameters, and amount of computation were selected as evaluation indicators. The average accuracy was improved by 3.7%, the number of parameters was reduced by 33.5%, and the amount of computation was reduced by 0.1. Experimental verification shows that the improved model not only improves the detection accuracy, but also effectively reduces the model complexity.

Claims

1. A multi-scale surface defect detection method for tiles based on improved YOLOv8, characterized in that: The method comprises: S1, obtaining a tile surface defect image; S2, cutting images and data enhancement, constructing a tile surface defect image dataset; S3. Build and initialize an improved YOLOv8 convolutional neural network detection model, input the constructed defect image data set into the improved YOLOv8 convolutional neural network detection model for training, and obtain the corresponding tile surface defect detection neural network model; the improved YOLOv8 convolutional neural network detection model includes using the ASF architecture to reconstruct the neck structure of the original model and adding a small target detection head; design a C2f_DD module to replace the C2f module of the original backbone network; design a shared parameter detection head (SPaD); S4. Test and evaluate the improved YOLOv8 convolutional neural network detection model.

2. A ceramic tile multi-scale surface defect detection method based on improved YOLOv8 as claimed in claim 1, characterized in that: Image segmentation and data augmentation; The image cutting method cuts the original image into small images of 800×800 pixels by using an offline sliding window method; The data enhancement is performed on the cut image by means of rotation, cropping, scaling and adding noise.

3. A ceramic tile multi-scale surface defect detection method based on improved YOLOv8 as claimed in claim 1, characterized in that: Use the ASF architecture to reconstruct the neck structure of the YOLOv8 model and add a small target detection head; The neck structure of the reconstructed YOLOv8 model, the ASF architecture includes a scale sequence feature fusion module (SSFF) module and a triple feature encoding module (TFE) module; the 1-5 level feature extraction branches {P1, P2, P3, P4, P5} in the trunk of the YOLOv8 convolutional neural network detection model; then P2, P3, P4, P5 are fused through the neck feature pyramid network to output a group of {S2, S3, S4, S5}; the scale sequence feature fusion module (SSFF) module fuses the different scale feature maps of P3, P4, and P5, and the triple feature encoding module (TFE) module splices the features of large, medium, and small sizes in the spatial dimension.

4. A ceramic tile multi-scale surface defect detection method based on improved YOLOv8 as claimed in claim 3, characterized in that: The added tiny target detection head; Input1 and input2 are fused through the fusion layer and then output to the head for detecting objects larger than 4×4 pixels; The input1 includes the P2 feature map in the backbone network and the P2 feature map of the 25th layer in the upsampling neck; The input2 includes the SSFF module that fuses the trunk P2 feature map and the neck P3 and P4 feature maps.

5. A ceramic tile multi-scale surface defect detection method based on improved YOLOv8 as claimed in claim 4, characterized in that: The C2f_DD module is designed to replace the C2f module of the original backbone network; The C2f_DD module is composed of a C2f module and a DWR-DRB module after DRB transformation; The DRB module may apply dilated convolutions with different convolution kernel sizes and dilation rates to the input tensor according to the set kernel size; The DWR module is used to apply a 3×3 convolution mixed with batch normalization and ReLU activation function to the input feature map (2c), output feature maps (c) in the form of regions of different sizes, use expansion depth convolution with different expansion rates, merge features through batch normalization and 1×1 convolution, and finally output a feature map with 2c channels and add it to the input feature map; The DWR-DRB module replaces the dilated depth convolution with dilation rates of 3 and 5 in the DWR module with the DRB module with kernel sizes set to 5 and 7; The C2f_DD module replaces the Bottleneck module in the C2f module with the DWR-DRB module.

6. A ceramic tile multi-scale surface defect detection method based on improved YOLOv8 as claimed in claim 5, characterized in that: A shared parameter detection head (SPaD) is designed; The shared parameter detection head (SPaD) applies 1×1GN_Conv convolution to the four different features S2, S3, S4, and S5 output by the neck. The feature layer is extracted through a 3×3 GN shared convolution layer. Finally, the regression and classification tasks are separated, and the coordinates and category probabilities of the bounding box are output.

Images