Ceramic tile surface defect detecting and positioning method based on feature adapter

Abstract

The invention discloses a ceramic tile surface defect detecting and positioning method based on a feature adapter. The method comprises the following steps: acquiring an image data set; randomly selecting one image as input, and extracting image feature representation through a pre-training encoder; carrying out image coordinate position embedding on the extracted feature representation through a feature adapter to obtain position-enhanced learnable features; the learnable features are subjected to large-scale data clustering through MiniBatchKMeans, so that memory bank initialization is carried out; performing supervised training through the loss function to optimize the feature adapter, and updating the memory bank by applying the output of the optimized feature adapter; a distance between the latest representation of the image and the fused feature is calculated to obtain an anomaly score. According to the method, the feature adapter and the memory bank are updated at the same time, so that the current most discriminative feature is obtained, and real-time and efficient defect detection and positioning are realized.

Description

Technical Field

[0001] The invention relates to the technical field of ceramic tile manufacturing and quality control, and in particular to a ceramic tile surface defect detection and positioning method based on a feature adapter. Background Art

[0002] Under the background of intelligent manufacturing, the degree of automation and intelligence is constantly improving, which puts forward higher standards for product quality control. As a product widely used in building decoration materials, the surface quality of ceramic tiles directly affects the final decoration effect and user experience. The defects that may exist on the surface of ceramic tiles include but are not limited to cracks, wear, spots, edge damage, foreign matter, etc. These defects not only affect the appearance, but also may affect the structural integrity and service life of the product.

[0003] In the detection of surface defects in ceramic tiles, traditional methods mainly rely on two methods: manual visual inspection and supervised learning methods. Manual visual inspection faces high labor costs, is slow and easy to fatigue, and is difficult to meet the needs of large-scale production. Although supervised learning can automate detection, it requires a large amount of labeled data for training, which is often difficult to obtain for ceramic tile defect detection. In addition, supervised models may not correctly identify new or unknown types of defects, and are prone to predicting normal samples, reducing the recognition rate of defective samples. In view of these limitations, unsupervised learning becomes an attractive option. However, uncertain factors in the production environment, such as noise disturbances and equipment performance fluctuations, can cause image features to drift over time, thereby reducing the effectiveness of models that rely on static training data. Therefore, it is urgent to develop online detection technologies that do not rely on large amounts of labeled data, can learn autonomously, and adapt to changes. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a method for detecting and locating ceramic tile surface defects based on a feature adapter. The method simultaneously updates the feature adapter and the memory library to obtain the most discriminative features at the moment, thereby achieving real-time and efficient defect detection and positioning.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A method for detecting and locating surface defects of ceramic tiles based on a feature adapter comprises the following steps:

[0007] Step 1: Obtain image dataset;

[0008] Step 2: randomly select an image as input and extract image feature representation through a pre-trained encoder;

[0009] Step 3: embed the extracted feature representation into the coordinate position of the image through a feature adapter to obtain a position-enhanced learnable feature;

[0010] Step 4: The learnable features are clustered on a large scale using MiniBatchKMeans to initialize the memory bank;

[0011] Step 5: Through the loss function Perform supervised training to optimize the feature adapter, and use the output of the optimized feature adapter to update the memory bank;

[0012] Step 6: Obtain the anomaly score (z) by calculating the distance between the test sample feature and the fused feature of the closest feature center in the memory library. ij ), the formula is as follows:

[0013]

[0014] Among them, z ij represents the test sample z ij , Represents retrieving the test sample z from the memory bank ij The closest L feature centers The fusion features with better generalization ability are

[0015] Preferably, the data set includes a training set and a test set, the test set includes normal samples and abnormal samples, and the test set also includes abnormal true label masks of corresponding abnormal samples.

[0016] Preferably, each sample in the dataset is resized to 256x256.

[0017] Preferably, in step 2, the pre-trained encoder selects a wrn50_2 model pre-trained on ImageNet as a pre-trained encoder for feature extraction.

[0018] Preferably, the method for extracting features through the pre-trained wrn50_2 model is:

[0019] The pre-trained wrn50_2 model is used to sample feature maps of different depths, and feature maps of the same resolution are obtained by interpolation. Then all feature maps of the same resolution are connected to generate multi-scale features. Among them, H and W are the height and width of the maximum feature map, D is the sum of the channels of the adopted feature map, and finally according to the multi-scale features The semantic information at each pixel position is used to represent the image features. where i∈[1,...,H] and j∈[1,...,W].

[0020] Preferably, the feature adapter comprises a CoordConv layer and a standard convolutional layer.

[0021] Preferably, in step 3, the image feature representation Input into the feature adapter, and embed the coordinate information of the input image into the image feature representation through the CoordConv layer to obtain a learnable feature with position enhancement. in and

[0022] Preferably, in step 4, a normal sample image is randomly selected and subjected to steps 2 and 3 to obtain a learnable feature z ij Clustering is performed using the MiniBatchKMeans algorithm to obtain K centers C k and fill it into the memory bank.

[0023] Preferably, in step 5, the feature adapter optimization method is:

[0024] First, find z through NN search ij The first M nearest neighbor centers of pass Perform supervised training so that z ij Embed in Nearby, including Described as:

[0025]

[0026] in, Indicates z ij Its mth nearest neighbor The Euclidean distance of .

[0027] Preferably, in step 5, During supervised training, the M nearest centers are assigned different weights through a Gaussian function.

[0028] Preferably, in step 5, z ij Embed in The nearby methods are: ij Embed In a hypersphere with radius r as the center.

[0029] Preferably, in step 5, the memory bank is updated by:

[0030] To T i＞0 At this moment, the feature representation center set of all learnable features obtained by the optimized feature adapter for the current image is Then the center set With T i-1 The center set Perform moving average to get the latest center set.

[0031] Preferably, when the memory bank is updated, in the moving average, the entire deviation to the original center ratio increases with the increase of data, and the corresponding description is as follows:

[0032]

[0033] Where t represents T i＞0 time.

[0034] The present invention has the following characteristics and beneficial effects:

[0035] Using the above technical solution, compared with the traditional offline learning method, the online detection solution proposed in this study has online adaptive capabilities, and can dynamically adjust and optimize the model according to the samples detected in real time, effectively overcoming the negative impact of feature drift and ensuring the stability and improvement of detection accuracy.

[0036] This method cleverly combines the adaptive ability of unsupervised learning algorithms and the powerful representation power of deep learning. It can monitor the abnormalities of the tile surface in real time in a continuous production flow, and realize the rapid identification and precise positioning of defects. What is particularly critical is that this process is accompanied by model optimization. That is, in the process of continuous monitoring and response to production data streams, the system self-learns, iterates and upgrades, and continuously refines its detection algorithm, and its accuracy and adaptability continue to improve. This provides a dynamically upgraded and highly intelligent technical solution for quality control in the tile manufacturing industry. In short, this online detection algorithm, with its dynamic adaptation and continuous optimization capabilities, provides a more flexible, efficient and robust solution for abnormality detection at the tile production site. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 This is a schematic diagram of an embodiment of a method for detecting and locating ceramic tile surface defects based on a feature adapter according to the present invention.

[0039] Figure 2 for Figure 1 Schematic diagram of the network structure of the feature adapter.

[0040] Figure 3 Schematic diagram of abnormality detection results on tile surface. DETAILED DESCRIPTION

[0041] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0042] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings.

[0043] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0044] The present invention provides a method for detecting and locating surface defects of ceramic tiles based on a feature adapter, such as Figure 1 As shown, the following steps are included:

[0045] Step 1: Obtain image dataset.

[0046] Specifically, in this embodiment, the method is performed on the tile subclass of the MVTec AD benchmark dataset, which is one of the most well-known public datasets in the field of anomaly localization. The tile subclass dataset contains training data and test sets of normal samples. The test set contains both normal and abnormal samples, and the abnormal samples contain various defect types. The test set also provides corresponding abnormal true label masks. In the preprocessing stage, each sample of the dataset is resized to 256x256.

[0047] Step 2: Select any image as input and extract image feature representation through a pre-trained encoder.

[0048] Understandably, given the scarcity of tile defect examples, unsupervised defect detection and localization is the only choice. Currently, the mainstream unsupervised detection and localization methods are mainly divided into three categories: reconstruction-based, self-supervised, and embedding vector-based methods. Reconstruction-based methods believe that abnormal image areas should not be able to be reconstructed correctly because they do not exist in the training samples. However, such methods may cause abnormal information to be reconstructed due to over-generalization of the model. Self-supervised learning gives the model spatial understanding capabilities by mining the inherent consistency of samples, such as predicting the relative positions between image blocks, but such networks often have limited feature extraction capabilities, which affects detection accuracy.

[0049] In contrast, the method based on embedded vectors performed the best, among which the memory-based strategy was particularly concise and efficient. This method imitates the human cognitive process - even if you have never seen a certain type of abnormality, as long as you are familiar with normal samples, you can recognize the abnormality. The "memory" here is equivalent to a feature vector database that stores the extracted features of normal images, which is called a memory bank. The feature vectors of abnormal images are compared with the normal features in the memory bank for similarity to determine the degree of abnormality. In order to achieve this goal, the model needs to have a strong feature extraction capability to convert images into low-dimensional feature vectors, and deep learning provides a solid foundation for this. Among them, the model pre-trained by ImageNet is an ideal choice as a feature extractor. Therefore, in this embodiment, a Wide ResidualNetwork 50-2 (wrn50_2) model pre-trained on ImageNet is selected as a pre-trained encoder for feature extraction.

[0050] It should be noted that, in this embodiment, the pre-trained encoder is only used for preliminary feature extraction and does not participate in the update.

[0051] Furthermore, the method of applying the pre-trained encoder to extract image feature representation is:

[0052] A random normal image is passed through a pre-trained encoder to extract multi-scale feature maps. The feature maps sampled at each depth have different spatial resolutions, which are interpolated to have the same resolution and then connected to generate multi-scale features. Here H and W are the height and width of the largest feature map, and D is the sum of the channels of the adopted feature map. Each pixel position of has a predetermined receptive field, so the feature is regarded as semantic information at the pixel position, where i∈[1,...,H] and j∈[1,...,W]. In this way, the image feature representation p is obtained ij .

[0053] Step 3: The extracted feature representation is embedded into the image coordinate position through a feature adapter to obtain a position-enhanced learnable feature.

[0054] In this embodiment, considering that the memory-based embedding vector method has become the core part of the defect detection algorithm of the present invention due to its excellent feature extraction ability and easy scalability, however, directly using the features of the ImageNet model faces cross-domain challenges. This is because the ImageNet model is trained based on a large number of pictures that are very different in style from the target data set, and there is a matching problem with the target data set. In addition, although ImageNet enhances the receptive field through the convolutional layer, it is still insufficient in capturing spatial position information, which causes the extracted features to lose position information, and position information is crucial for defect location. Therefore, the present invention performs adaptive adjustments on the basis of ImageNet and adds feature adapters to solve the above problems and ensure that feature extraction is both powerful and accurate to meet the stringent requirements of tile defect detection and positioning.

[0055] Specifically, in this embodiment, the feature adapter consists of a CoordConv layer and a standard convolution layer, such as Figure 2 As shown in Figure 2. The CoordConv layer can embed the coordinate information of the input image into the feature representation, so that the feature adapter can perceive the position in the feature map, which is very important for defect positioning.

[0056] It should be noted that the feature adapter is composed of CoordConv and standard convolution layers. If there is only a CoordConv layer, the feature adapter cannot be trained through back propagation. The implementation of the standard convolution layer here is a 1x1 convolution with half the number of channels.

[0057] Step 4: randomly select a normal sample image, obtain the learnable features through steps 2 and 3, and perform large-scale data clustering through MiniBatchKMeans to initialize the memory bank.

[0058] Specifically, T i＞0 At this moment, a normal image is randomly selected. The image first undergoes step (2) to obtain the feature representation where i∈[1,...,H] and j∈[1,...,W]. Then, these features p ij Through the feature adapter in step (3), position-enhanced learnable features are generated in and Finally, z ij Clustered by k-means clustering method, the K centers C obtained kTo fill the memory bank. By randomly selecting normal samples for initialization, the characteristic information of normal samples is fully utilized, which establishes a favorable starting point for the model. In addition, in order to improve efficiency, MiniBatchKMeans is used in this embodiment to replace the traditional K-means algorithm for large-scale data clustering.

[0059] It can be understood that during online learning, in this embodiment, only the parameters of the feature adapter and the memory library need to be fine-tuned to efficiently adapt to new data, greatly reducing training time and computing resource consumption while maintaining good generalization performance.

[0060] Step 5: Through the loss function Perform supervised training to optimize the feature adapter, and use the output of the optimized feature adapter to update the memory bank.

[0061] Specifically, the optimization of feature adapter,T i＞0 At this moment, the input stream image data X i As in step (4), we obtain the learnable feature z ij , which then participates in the loss function is being optimized.

[0062] It can be understood that in order to obtain more compact and discriminative features in the feature space so as to be able to clearly detect the feature space of abnormal features, this embodiment considers a method based on SVDD so that the feature vector embedded in the feature adapter has a high density.

[0063] Specifically, first, find z through the nearest neighbor search ij The first M nearest neighbor centers of Next, through Perform supervised training so that z ij Embed in Specifically, the feature adapter is supervised by z ij Embed In a hypersphere with radius r as the center, it is possible to form a high concentration between normal features. Furthermore, the M nearest centers should be given different weights. If these centers are all given the same weight, it may lead to Prefer lower similarity and larger Euclidean distance In fact, in this embodiment, the weights should be adjusted so that Favors centers with higher similarity and smaller Euclidean distance Intuitively, we can use the inverse of the distance as the corresponding weight, but the potential problem with this method is that it assigns a large weight to the nearest neighbors, and the weight of the slightly farther ones will decay quickly. But sometimes it will also make the algorithm more sensitive to noisy data. Therefore, the Gaussian function is selected as the corresponding weight, which is described as:

[0064]

[0065] Where c is an adjustable parameter.

[0066] therefore, Described as:

[0067]

[0068] in, Indicates z ij Its mth nearest neighbor The Euclidean distance of .

[0069] In the above technical solution, through the optimization of the feature adapter module, the image feature representation generated by it is also optimized to adapt to the target data set, making the features more compact, thereby effectively alleviating the cross-domain problem.

[0070] Furthermore, for T i＞0 At this moment, after the feature adapter is optimized by the current image, all feature vector representations of the current image are clustered to obtain the feature representation center set of the image in the current feature adapter state. Then with T i-1 The center set Perform moving average to get the latest center set. In the moving average, the proportion of the entire deviation to the original center increases with the increase of data, and the corresponding description is as follows:

[0071]

[0072] Where t represents T i＞0 time.

[0073] The whole method effectively saves space consumption. The whole process only requires the feature vector space complexity of a single image, which can be expressed as:

[0074]

[0075] Among them, γ represents the degree of clustering.

[0076] Step 6: Obtain the anomaly score (z) by calculating the distance between the test sample feature and the fused feature of the closest feature center in the memory library. ij ), the formula is as follows:

[0077]

[0078] Among them, z ij represents the test sample z ij , Represents retrieving the test sample z from the memory bank ij The closest L feature centers The fusion features with better generalization ability are

[0079] It should be noted that in traditional anomaly detection methods, the anomaly score is usually calculated based on the nearest feature center extracted from the memory bank. However, this method has limitations, especially in the scenario of continuous learning, because the distribution of the model's training data may drift over time, resulting in the nearest feature center may no longer be the most representative. On the other hand, simply taking the average of multiple feature centers may also cause the fused feature center to be not close enough to any single center due to significant differences between the centers, thus affecting the accuracy of anomaly detection.

[0080] Therefore, in this embodiment, a fusion prediction method based on affinity is introduced. First, the memory library is retrieved to find the ij The closest L feature centers And the Euclidean distances between them are d1, d2, ..., d L (in ). The affinity is defined as follows. In this embodiment, the Gaussian function is also used to ensure that the most relevant feature center contributes the most to the fusion result:

[0081]

[0082] Where c and L are hyper parameters.

[0083] Fuse L feature centers to obtain a more generalizable fusion feature

[0084]

[0085] go through After continuous optimization, the feature adapter has learned the ability to map normal samples to several high-dimensional spheres, which correspond to different basic microstructures of normal samples. When calculating the anomaly score, the anomaly score score (z ij ), the formula is as follows:

[0086]

[0087] This process creates a more robust representation of normal samples by weighted fusion of the nearest feature center representations, thereby improving the accuracy of anomaly detection.

[0088] Finally, this embodiment evaluates the performance of the embodiment of the present invention through two dimensions: anomaly detection (I-AUROC) and anomaly localization (P-AUROC). The former tests the ability of the model to identify whether the entire image is abnormal, while the latter examines the accuracy of the model in accurately locating the abnormal part of the image.

[0089] In the comparative analysis section, the joint optimization method of the present invention is compared with the patchcore model, which is a memory-based benchmark model. In order to comprehensively illustrate the advantages of the method of the present invention from multiple perspectives, this embodiment conducts a wide range of comparative analysis with a series of existing technologies based on different principles. This embodiment compares reconstruction-based methods, such as RIAD, and self-supervised learning-based methods, such as PatchSVDD, which cover a richer range of evaluation dimensions. The experimental comparison results are shown in Table 1.

[0090]

[0091] Table 1: Performance comparison of various models on the tile sub-dataset of MVTec AD (%).

[0092] Through such multi-faceted comparison, the superiority of the present method in image-level and pixel-level anomaly detection is verified. Under the I-AUROC indicator, the joint optimization method provided by the present invention obtained a full score of 100, which is significantly higher than PatchCore's 98.7, RIAD's 98.7, and OCSVM's 87.6. This shows that in image-level anomaly detection, the method of this embodiment can effectively determine whether the entire image contains anomalies. In terms of the P-AUROC indicator, the method of this embodiment scored 96.6, which is also better than PatchCore's 95.6, RIAD's 89.1, and CutPaste's 90.5. This shows that in pixel-level anomaly positioning, the method of this embodiment can very accurately identify which pixels belong to the abnormal area. In order to more intuitively demonstrate the effect of the present invention, the abnormality detection results of some ceramic tile surfaces are shown as follows. Figure 3 shown.

[0093] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A method for detecting and locating surface defects of ceramic tiles based on a feature adapter, characterized in that: The steps include: Step 1: Obtain image dataset; Step 2: randomly select an image as input and extract image feature representation through a pre-trained encoder; Step 3: embed the extracted feature representation into the coordinate position of the image through a feature adapter to obtain a position-enhanced learnable feature; Step 4: All learnable features are clustered on a large scale using MiniBatchKMeans to initialize the memory bank. Step 5: Loss function Perform supervised training to optimize the feature adapter, and use the output of the optimized feature adapter to update the memory bank; Step 6: Obtain the anomaly score (z) by calculating the distance between the test sample feature and the fused feature of the closest feature center in the memory library. ij ), the formula is as follows: Among them, z ij represents the test sample, Represents retrieving the test sample z from the memory bank ij The closest L feature centers The fusion features with better generalization ability are 2. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 1, characterized in that: The data set includes a training set and a test set, the test set includes normal samples and abnormal samples, and the test set also includes abnormal true label masks of corresponding abnormal samples.

3. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 2, characterized in that: Each sample in the dataset is resized to 256x256.

4. The method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 1, characterized in that: In step 2, the pre-trained encoder selects a wrn50_2 model pre-trained on ImageNet as a pre-trained encoder for feature extraction.

5. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 4, characterized in that: The method for extracting features through the pre-trained wrn50_2 model is: The pre-trained wrn50_2 model is used to sample feature maps of different depths, and feature maps of the same resolution are obtained by interpolation. Then all feature maps of the same resolution are connected to generate multi-scale features. Among them, H and W are the height and width of the maximum feature map, D is the sum of the channels of the adopted feature map, and finally according to the multi-scale features The semantic information at each pixel position is used to represent the image features. where i∈[1,...,H] and j∈[1,...,W].

6. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 5, characterized in that: The feature adapter includes a CoordConv layer and a standard convolutional layer.

7. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 6, characterized in that: In step 3, the image feature representation Input into the feature adapter, and embed the coordinate information of the input image into the image feature representation through the CoordConv layer to obtain a learnable feature with position enhancement. in and 8. The method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 7, characterized in that: In step 4, the memory bank is initialized by randomly selecting a normal sample image and performing steps 2 and 3 to obtain the learnable feature z. ij Clustering is performed using the MiniBatchKMeans algorithm to obtain K centers C k and fill it into the memory bank.

9. The method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 7, characterized in that: In step 5, the optimization method of the feature adapter is: First, find z by nearest neighbor search ij The first M nearest neighbor centers of pass Perform supervised training so that z ij Embed in Nearby, including Described as: in, Indicates z ij Its mth nearest neighbor The Euclidean distance of .

10. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 9, characterized in that: In step 5, by During supervised training, the M nearest centers are assigned different weights through a Gaussian function.

11. The method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 9, characterized in that: In step 5, z ij Embed in The nearby methods are: ij Embed In a hypersphere with radius r as the center.

12. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 11, characterized in that: In step 5, the memory bank is updated by: To T i＞0 At this moment, the feature representation center set of all learnable features obtained by the optimized feature adapter for the current image is Then the center set With T i-1 The center set Perform moving average to get the latest center set.

13. A method for detecting and locating ceramic tile surface defects based on a feature adapter according to claim 12, characterized in that: When the memory is updated, in the moving average, the entire deviation to the original center ratio increases with the increase of data, and the corresponding description is as follows: Where t represents T i＞0 time.

Images