A high-resolution method for detecting ceramic surface defects by weighted multi-scale feature fusion

Abstract

This invention provides a high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion, comprising the construction of a defect detection model, model training, and defect detection. The defect detection model includes three feature pyramid structures and two weight calculation modules. Feature information output from convolutional layers of different depths in the previous feature pyramid structure is aggregated. In the weight calculation modules, weight coefficients are calculated for each layer of the aggregated features, and finally, the aggregated features are weighted and fused using these weight coefficients. Model training: By labeling sanitary ware ceramic data, a dataset is obtained, which is then divided into training and testing sets. The established defect detection model is trained to obtain the final defect detection model. Defect detection: The sanitary ware ceramic images are first scaled to 1920×1920 pixels and then input into the defect detection model for detection, obtaining the final detection result. This method improves the detection performance of small defect targets.

Description

Technical Field

[0001] This invention relates to the field of surface defect detection in sanitary ceramics, and in particular to a high-resolution method for detecting surface defects in ceramics using weighted multi-scale feature fusion. Background Technology

[0002] Surface defect detection in sanitary ware ceramic products is a crucial step in the production process, as it directly impacts product quality. Small defects frequently appear on the surface of sanitary ware ceramic products. Because they are difficult to detect and occupy a relatively small area within the entire workpiece, detection presents a significant challenge. Existing deep learning-based surface defect detection methods for sanitary ware ceramics utilize constructed neural networks for feature extraction and perform defect detection on the outputs of a few layers of the neural network. However, due to the gradual loss of feature information about small defects during the continuous convolution process, their detection performance for small defects is not ideal.

[0003] Existing methods for detecting surface defects in sanitary ware ceramics mainly fall into two categories: those based on traditional image processing and those based on deep learning. Traditional image processing methods process acquired ceramic images, extract features, and combine this with machine learning techniques for defect classification and identification. However, because they employ manually designed feature operators, they are significantly affected by external environmental interference, resulting in limited adaptability. Deep learning-based methods train models using labeled ceramic data. Existing methods often utilize neural networks for feature extraction and perform defect detection at certain layers. Some methods employ feature fusion to enhance the feature information of small defects; however, these methods do not adequately consider the importance of different features for the detection task, causing the feature information of small defects to be overwhelmed by less important features in the neural network's extracted information. Summary of the Invention

[0004] To address the problem of small defect detection in high-resolution images of sanitary ware ceramic products, this invention provides a weighted multi-scale feature fusion method for high-resolution ceramic surface defect detection. First, the input image data is scaled to 1920x1920 with equal width and height. Then, features are extracted from the input image using multi-layer convolution. The feature information extracted from different convolutional layers is then aggregated to obtain a feature representation containing multi-scale information. Finally, a designed weight calculation module is used to calculate the weight coefficients of the feature representation and perform feature weighting. This weighting method enhances the feature information of small defect targets, improving the detection performance of small defect targets. The method mainly includes: building a defect detection model, model training, and defect detection.

[0005] The defect detection model includes three feature pyramid structures and two weight calculation modules. By aggregating the feature information output by convolutional layers of different depths in the previous feature pyramid structure, the resulting features contain multi-scale information. Then, in the weight calculation module, weight coefficients are calculated for each layer in the aggregated features. Finally, the calculated weight coefficients are used to perform weighted fusion of the previously aggregated features.

[0006] Model training: By labeling the sanitary ceramic data, a dataset is obtained. This dataset is then divided into a training set and a test set. The established defect detection model is trained and tested to obtain the final defect detection model.

[0007] The final defect detection model is used to inspect actual bathroom ceramics to obtain the final inspection results.

[0008] Furthermore, the weight calculation module includes: feature aggregation, feature weight calculation, feature weight addition, Sigmoid activation, feature fusion, and de-aggregation operations.

[0009] Furthermore, in feature aggregation, the features obtained through concatenation are processed by a 1×1 convolution operation, and the calculation formula is as follows:

[0010]

[0011] In the formula, y represents the feature information of each dimension in the feature information obtained by convolution, w0 is the parameter of a convolution kernel, * is the convolution operation operator, and f i Let be the feature information of the i-th dimension of the feature to be convolved, and n be the sum of the dimensions of the feature information to be convolved.

[0012] Furthermore, the formula for calculating the Sigmoid function is:

[0013]

[0014] Where σ is the output of the Sigmoid function, that is, the feature weight value normalized to between 0 and 1, and x is the input result of the function, that is, the feature weight value calculated by the feature addition step of the weight calculation module.

[0015] Furthermore, the defect detection model was trained using the Adam optimization method, with a maximum number of iterations set to 40 and a learning rate set to 0.001.

[0016] Furthermore, the process of training the defect detection model is as follows:

[0017] S1: Use the training set to train and test the defect detection model;

[0018] S2: Determine whether the training process has reached the maximum number of iterations. If yes, the final defect detection model is obtained. If not, perform Mosaic data augmentation, forward model prediction, loss function calculation, error backpropagation, and optimization of model parameters.

[0019] S3: Use the test set to determine if the model performance is optimal. If yes, save the current model and obtain the final defect detection model. If not, return to step S2.

[0020] Furthermore, the final defect detection model takes as input an image scaled to 1920×1920 with equal width and height, and outputs the detection results.

[0021] The beneficial effects of the technical solution provided by this invention are as follows: Compared with traditional image processing-based methods, this invention uses convolutional neural networks to train the detection model, which can effectively reduce the interference of the detection environment on the detection results and has better detection performance; it adopts a weighted multi-scale feature fusion method, which can effectively enhance the feature information of small defect targets by fusing feature information of different scales and weight calculation modules; this invention is a lightweight network that can achieve a detection time of 0.06s for a single image with a high resolution of 1920x1920, and has excellent detection speed performance. Attached Figure Description

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0023] Figure 1 This is a diagram of the defect detection model in an embodiment of the present invention.

[0024] Figure 2 This is a diagram of the weight calculation module in an embodiment of the present invention.

[0025] Figure 3 This is a flowchart of the model training process in an embodiment of the present invention. Detailed Implementation

[0026] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0027] This invention provides a high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion. By constructing a sanitary ware ceramic surface defect detection model, this invention achieves a detection speed of 0.06s for a single 1920×1920 image on an RTX5000 GPU hardware platform, meeting the real-time requirements of industrial environments. Furthermore, by designing a weighted multi-scale feature fusion module, the method aggregates feature information extracted from different levels of convolution and weights the importance of the detection task based on the feature information extracted from different levels. This effectively enhances the feature information of small targets while suppressing unimportant feature information.

[0028] This invention provides a high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion, comprising three parts: building a defect detection model, model training, and defect detection. The main components and functions of each step are as follows:

[0029] I. Building a defect detection model:

[0030] This embodiment uses Yolov5s6 as the base network to build as follows: Figure 1 The defect detection model shown consists of three existing feature pyramid structures from Yolov5s6 (Yolov5s6 only look once, v5s6 refers to the 5th version of the S6 model): a backbone network, a feature pyramid network (FPN), and a path aggregation network (PAN), along with two weight calculation modules. The two weight calculation modules have similar structures; they aggregate feature information output from convolutional layers of different depths in the previous feature pyramid structure. Since convolutional layers of different depths have different receptive fields, the aggregated features contain multi-scale information. Then, the weight calculation modules calculate weight coefficients for each layer in the aggregated features, and finally use the calculated weight coefficients to perform weighted fusion of the previously aggregated features. By weighted fusion of features extracted from different levels, the feature information of small defect targets can be effectively enhanced. The variables in the figure represent the feature maps to be output by each module. The feature map is the information extracted by the convolution kernel from the input to represent the features of the image. It is represented as a three-dimensional matrix, which is usually represented by w, h and c. These three parameters represent the width, height and feature dimension of the feature map, respectively.

[0031] Taking the weight calculation module in Backbone and FPN as an example, B1 to B4 correspond to the output feature maps of the top four Cross Stage Partial Blocks (CSPBlocks) in the Backbone feature pyramid of YOLOv5s6, with widths and heights of 240×240, 120×120, 60×60, and 30×30, respectively. Because their widths and heights are different, the size information described by the features they extract is also different. By aggregating information from different scales, richer feature information can be obtained. The weight calculation module designed in this invention is as follows: Figure 2 As shown, it can be divided into 6 steps: feature aggregation, feature weight calculation, feature weight addition, Sigmoid activation, feature fusion, and de-aggregation operation, as detailed below:

[0032] 1. Feature Convergence: Since the feature map sizes of B1, B2, and B3 are inconsistent, the weight calculation module uses the width and height of B2 as a benchmark. The width and height of the B1 feature map are doubled using an upsampling layer while maintaining the feature map's dimension. The width and height of the B3 feature map are halved using a convolutional layer with a kernel size of 3×3, padding of 1, and stride of 2, while maintaining the feature map's dimension. Then, the three feature maps are concatenated to obtain a feature dimension that is the sum of the dimensions of the three feature maps. The concatenated feature map is then processed by a 1×1 convolutional operation. The calculation formula is as follows:

[0033]

[0034] In the formula, y represents the feature information of each dimension in the feature information obtained by convolution, w0 is the parameter of a convolution kernel, * is the convolution operation operator, and f i Let B be the feature information of the i-th dimension of the feature to be convolved, and n be the sum of the dimensions of the feature information to be convolved. Each dimension of the feature information obtained after a 1×1 convolution operation contains information from all dimensions of the previous features, thus providing rich features for subsequent feature extraction. The feature information obtained from the 1×1 convolution is denoted as B. p .

[0035] 2. Feature Weight Calculation: The weight calculation module calculates B using max pooling and average pooling layers. p The importance of each dimension's feature information to the detection task is determined by the following: The max pooling layer extracts the maximum value of each dimension's feature information through max pooling to represent the importance of that dimension's feature to the detection task. The average pooling layer, on the other hand, represents the importance by calculating the average value of each dimension's feature information. The final result obtained from the max pooling and average pooling layers is a 1×1 feature map with dimensions equal to B. p Same feature weights.

[0036] 3. Weight Coefficient Addition: When adding the feature weights obtained from two different pooling layers, this invention allows each feature weight to undergo a group convolution operation. Unlike general convolution operations that use a single convolution kernel to extract information from all dimensions of the previous feature, the group convolution operation uses a single convolution kernel to extract feature information corresponding to one dimension of the previous feature. Therefore, it is equivalent to assigning a weight coefficient to each dimension of the feature weight, thereby measuring the effectiveness of different pooling operations in extracting weights.

[0037] 4. Sigmoid Activation: The summed feature weights are in the range [0, ∞]. Therefore, the weight calculation module normalizes the values ​​to [0, 1] using a Sigmoid function. The formula for the Sigmoid function is:

[0038]

[0039] After activation by the Sigmoid function, the weights corresponding to important feature dimensions are close to 1, while the weights corresponding to unimportant feature dimensions are close to 0.

[0040] 5. Feature Fusion: The feature weights activated by the Sigmoid function are combined with B... p The features of the corresponding dimensions are multiplied together to obtain the fused feature B. s Since the feature weights are updated using the backpropagation algorithm, they can effectively measure B. p The importance of different dimensions of feature information for the detection task is determined, thereby enhancing the features of small defect targets and suppressing irrelevant feature information.

[0041] 6. Anti-convergence operation: Utilizing the feature B obtained from the fusion s When enhancing the feature information of feature maps F3, F2, and F1 in the FPN feature pyramid, because the size of the feature maps is not consistent, it is necessary to adjust B... s Perform convolution operations to make the feature map size consistent with that in F3; for B s Perform an upsampling operation to match the feature map size in F1. s It has the same size as the F2 feature map, so it can be directly used to enhance the feature information of F2.

[0042] II. Model Training

[0043] The model training flowchart of this invention is as follows: Figure 3 As shown, the detailed steps for each part are as follows:

[0044] 1. Divide the data into training and test sets: Use labaelimg to label the sanitary ceramic data and obtain the corresponding label files as the dataset. Then, randomly divide the dataset into training and test sets in an 8:2 ratio. The training set is used to train the model, while the test set is used to test the model performance.

[0045] 2. Iterative training: The model is trained using the Adaptive Moment Estimation (Adam) optimization method, with a maximum number of iterations set to 40 and a learning rate set to 0.001.

[0046] 3. Mosaic Data Augmentation: For a network model with an input image size of 1920×1920, a blank template image of 3840×3840 is first generated. Then, four images are randomly selected from the dataset and independently and randomly rotated, reduced, and cropped. These images then occupy the upper left, upper right, lower left, and lower right regions of the blank template image, respectively. Since some images may have been reduced and cannot completely occupy the 1920×1920 region, the missing parts are padded with 0s. The mosaic-enhanced image is then input into the network model for forward prediction.

[0047] 5. Loss Function Calculation: This invention defines classification loss, confidence loss, and intersection-over-union (GIOU) loss from three aspects: the accuracy of defect category detection, the confidence of defect detection results, and the deviation between the bounding boxes of defect detection results and the true bounding boxes. The classification loss and confidence loss are cross-entropy loss functions, while the GIOU loss is the standard cross-entropy loss function. The loss value is obtained by calculating the loss function between the model's forward prediction results and the true label results, and the model parameters are optimized using the backpropagation algorithm.

[0048] 6. Model Saving: If the current model performs better than the historical best model, then the current model replaces the historical best model; otherwise, iterative training continues. If the preset number of iterations is reached, model training is complete.

[0049] III. Defect Detection

[0050] After training to obtain the final defect detection model, it is used for defect detection. First, the input image is scaled to 1920×1920 with equal aspect ratio. To avoid deformation of the defect target affecting the detection results during image scaling, this invention employs a constant aspect ratio scaling method. Assuming the image size to be scaled is h×w, and the maximum value between the two is max(h,w), if max(h,w) < 1920, it means that the longer side of the image (width, height) is also less than 1920, so areas in the image less than 1920 are directly padded with zeros; if max(h,w) >= 1920, the scaling ratio of the longer side is... The shorter side is scaled to the same scale as the longer side. The remaining area with less than 1920 is filled with 0.

[0051] The image, after being scaled to the same height, is then input into the trained final defect detection model for defect detection to obtain the final detection result. For a 1920×1920 input image, the detection speed is 0.06s with the acceleration of RTX5000 GPU.

[0052] The weight calculation module designed in this invention aggregates the outputs of three convolutional layers. This number can be four or more, as long as the size of the aggregated feature maps remains consistent. During model training, this invention uses the Adam trainer, but the SGD trainer can also be used instead, which will not have a significant impact on the final result.

[0053] The beneficial effects of this invention are as follows: Compared with traditional image processing-based methods, this invention uses convolutional neural networks to train the detection model, which can effectively reduce the interference of the detection environment on the detection results and has better detection performance; it adopts a weighted multi-scale feature fusion method, which can effectively enhance the feature information of small defect targets by fusing feature information of different scales and weight calculation modules; this invention is a lightweight network that can achieve a detection time of 0.06s for a single image with a high resolution of 1920x1920, and has excellent detection speed performance.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion, characterized in that: include: Build a defect detection model, train the model, and perform defect detection; The defect detection model includes three feature pyramid structures and two weight calculation modules. One weight calculation module is connected to the first and second feature pyramid structures respectively, and the other weight calculation module is connected to the second and third feature pyramid structures respectively. By aggregating the feature information output by convolutional layers of different depths in the previous feature pyramid structure, the resulting features contain multi-scale information. Then, the weight calculation module calculates the weight coefficients for each layer in the aggregated features, and finally uses the calculated weight coefficients to perform weighted fusion of the previously aggregated features. Model training: By labeling the sanitary ceramic data, a dataset is obtained. This dataset is then divided into a training set and a test set. The established defect detection model is trained and tested to obtain the final defect detection model. The final defect detection model is used to inspect actual bathroom ceramics to obtain the final inspection results; The weight calculation module includes: feature aggregation, feature weight calculation, feature weight addition, Sigmoid activation, feature fusion, and anti-aggregation operations. In feature aggregation, the features obtained through concatenation are processed through a 1 The convolution operation of 1 is calculated using the following formula: In the formula, For each dimension of the feature information obtained from convolution, For the parameters of a convolution kernel, This refers to the convolution operation operator. For the features to be convolved Dimensional feature information, Let the sum of the dimensions of the feature information to be convolved be denoted as . In the anti-convergence operation, when using the feature information of each layer of the feature pyramid obtained by feature fusion to enhance the feature information, a convolution operation is performed on the feature obtained by feature fusion.

2. The high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion as described in claim 1, characterized in that: The formula for calculating the Sigmoid function is: in, The output of the Sigmoid function is the feature weights normalized to a value between 0 and 1. x The input result of the function is the feature weight value calculated by the feature addition step of the weight calculation module.

3. The high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion as described in claim 1, characterized in that: The defect detection model was trained using the Adam optimization method, with a maximum number of iterations set to 40 and a learning rate set to 0.

001.

4. The high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion as described in claim 1, characterized in that: The process of training a defect detection model is as follows: S1: Use the training set to train and test the defect detection model; S2: Determine whether the training process has reached the maximum number of iterations. If yes, the final defect detection model is obtained. If not, perform Mosaic data augmentation, forward model prediction, loss function calculation, error backpropagation, and optimization of model parameters. S3: Use the test set to determine if the model performance is optimal. If yes, save the current model and obtain the final defect detection model. If not, return to step S2.

5. The high-resolution ceramic surface defect detection method based on weighted multi-scale feature fusion as described in claim 1, characterized in that: The final defect detection model takes as input a scaled-to-1920 pixels with equal width and height. The image from 1920 is output as the detection results.

Images