A ceramic tile surface color difference detection method and device, electronic equipment and storage medium

Abstract

The application relates to a ceramic tile surface color difference detection method and device, electronic equipment and a storage medium. The ceramic tile surface color difference detection method comprises the following steps: acquiring a ceramic tile image to be detected; inputting the ceramic tile image into a multi-task model combined with a trained texture recognition network and a color classification network to obtain a texture descriptor with color information; extracting color features of the ceramic tile image and label features of a template ceramic tile image; fusing the texture descriptor with color information, the color features and the label features to obtain a feature vector corresponding to the ceramic tile image; and inputting the feature vector into a trained classification model to obtain a judgment result of whether the ceramic tile image to be detected has color difference. The ceramic tile surface color difference detection method has powerful feature extraction capacity of a deep convolutional neural network, can extract texture features of a ceramic tile surface, and can more efficiently and accurately describe the texture of an image.

Description

Technical Field

[0001] This invention relates to the field of ceramic tile testing technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting color difference on the surface of ceramic tiles. Background Technology

[0002] Tiles are an indispensable building material in modern architecture. With social progress and development, and the continuous improvement of people's living standards, the requirements for tile quality are also gradually increasing. Due to the complexity of tile production processes, some defects inevitably appear on the tile surface during production, among which color difference is a persistent and prominent problem. Color difference can occur in stages such as glazing, inkjet printing, kiln firing, and polishing. Currently, industrial color difference detection for tiles is mainly based on manual visual inspection, which has many problems. First, human judgment is affected by external environmental factors such as light and temperature, easily leading to misjudgments and missed detections. Second, workers experience visual fatigue after prolonged work, resulting in a decrease in the accuracy of color difference detection. Third, everyone's sensitivity to color is different, so this detection method is highly subjective; different workers may obtain different results when inspecting the same tile. Finally, the size of tiles is increasing, and the colors and textures are becoming more complex, further reducing the efficiency of manual visual inspection for color difference detection. Furthermore, in recent years, the impact of the pandemic has led to problems such as reduced economic benefits, high costs, and labor shortages for enterprises.

[0003] Existing automated methods for detecting color differences on ceramic tile surfaces mainly fall into two categories. The first category is based on unsupervised traditional image processing techniques. However, with the increasing variety of ceramic tiles and their intricate patterns, and the shift in market demand from small-format tiles to large-format tiles, color difference detection faces significant challenges. Traditional color difference detection methods are only suitable for tiles with relatively simple colors and textures, and their detection speed cannot meet production requirements. The second category is supervised color difference detection. Extracting rich and effective features as input to the neural network is crucial for improving model accuracy, and some methods are prone to overfitting. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for detecting color difference on the surface of ceramic tiles, which utilizes the powerful feature extraction capability of deep convolutional neural networks to extract the texture features of the ceramic tile surface, thereby describing the texture of the image more efficiently and accurately.

[0005] In a first aspect, the present invention provides a method for detecting color difference on the surface of ceramic tiles, comprising the following steps:

[0006] Obtain the image of the tile to be inspected;

[0007] The tile image is input into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information.

[0008] Extract the color histogram of the tile image, and use the bin centers and the statistics corresponding to each bin in the color histogram as features describing the image color to obtain the color features corresponding to the tile image;

[0009] The mean and variance of the RGB three channels of the pre-calculated template tile image are used as labeling features;

[0010] The texture descriptor with color information, the color features, and the marker features are fused to obtain the feature vector corresponding to the tile image;

[0011] The feature vector is input into the trained classification model to obtain the judgment result of whether there is color difference in the tile image to be detected.

[0012] Furthermore, it also includes the following steps:

[0013] The multi-task model includes a ResNet50 backbone network, a texture enhancement module, a multi-directional pyramid texture feature extraction module, a histogram layer, a compact bilinear pooling layer, a fully connected layer, and a global attention layer.

[0014] Furthermore, the tile image is input into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information, including the following steps:

[0015] The ResNet50 backbone network is used to extract features from the tile image, resulting in a first output feature, a second output feature, a third output feature, and a fourth output feature.

[0016] The first output feature and the second output feature are input into the texture enhancement module and the multi-directional pyramid texture feature extraction module to obtain the first texture statistics;

[0017] The fourth output feature is input into the introduced histogram layer to obtain the second texture statistics;

[0018] The first texture statistics are combined with the second texture statistics to obtain the third texture statistics;

[0019] The third texture statistics and the fourth output feature are input into the compact bilinear pooling layer, and the third texture statistics and the fourth output feature are fused using the compact bilinear pooling operation to obtain the bilinear pooling feature;

[0020] The bilinear pooling features are input into the fully connected layer, and a full connection operation is performed to reduce dimensionality, resulting in a texture descriptor.

[0021] The third output feature and the second output feature are input into the global attention layer to obtain the weighted low-level features;

[0022] The low-level features are fused with the third and fourth output features to obtain a color descriptor;

[0023] The color descriptor is combined with the texture descriptor to obtain a texture descriptor with color information.

[0024] Further, the color histogram of the tile image is extracted, and the statistics corresponding to the bin centers in the color histogram are used as features describing the image color to obtain the color features corresponding to the tile image, including the following steps:

[0025] The tile image I is converted from the RGB color space to the CIELab color space to obtain the converted image I′∈R. 3 ×H×W ;

[0026] Extract the maximum and minimum values ​​of the pixels in channel i (i = L, a, b) of the transformed image I′, and divide it into 20 equal points. Among them, V i This represents a vector consisting of the bin centers of the i-channel color histogram, where the value of each bin center is... Calculate using the following formula:

[0027]

[0028] All pixels in channel i of the transformed image I′ are assigned to various bins, and the results are statistically analyzed. Let be the total number of pixels contained in the nth bin; where is the number of pixels in the i-th channel of image I′. It belongs to the nth bin to satisfy the following conditions:

[0029]

[0030] Use the following formula to convert V i With S i Perform the following normalization operation:

[0031] V L =V L / 100.0

[0032] V a =(V a+128) / 255.0

[0033] V b =(V b +128) / 255.0

[0034] S i =S i / (H×W),i=L,a,b

[0035] V i With S i Perform the concatenation and fusion operation in the following formula order to obtain the color features corresponding to the tile image, i.e., the manually extracted image color descriptor final_color_feature∈R. 120 :

[0036] final_color_feature=Concat(V L ,S L V a ,S a V b ,S b ).

[0037] Furthermore, the classification model is a multilayer perceptron network.

[0038] Furthermore, the input of the multilayer perceptron network is a 382-dimensional feature vector, the hidden layer in the middle has 30 neurons, the output layer has 1 neuron, the connection in the middle is fully connected, and the final output will pass through the sigmoid function to obtain the probability Q∈(0,1) that the tile to be tested has a color difference.

[0039] Furthermore, if Q is greater than or equal to 0.5, it is determined that the tile image to be detected has a color difference;

[0040] If Q is less than 0.5, the tile image to be detected is determined to have no color difference.

[0041] Thirdly, the present invention also provides an electronic device, comprising:

[0042] At least one memory and at least one processor;

[0043] The memory is used to store one or more programs;

[0044] When the one or more programs are executed by the at least one processor, the at least one processor performs the steps of a method for detecting color difference on a ceramic tile surface as described in any of the first aspects of the present invention.

[0045] Fourthly, the present invention also provides a computer-readable storage medium.

[0046] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a method for detecting color difference on the surface of a ceramic tile as described in any of the first aspects of the present invention.

[0047] This invention provides a method, device, electronic device, and storage medium for detecting color difference on the surface of ceramic tiles. It utilizes the powerful feature extraction capabilities of deep convolutional neural networks to extract texture features from the surface of ceramic tiles, providing a more efficient and accurate description of image texture. The multi-task deep learning model used in this invention has the following advantages over traditional models that only extract texture features: (1) The multi-task network architecture, combining texture recognition and color classification, can obtain not only texture descriptors but also color descriptors. These color descriptors can complement the color features extracted during color difference detection, providing a more comprehensive and accurate description of the image's color information. (2) Once trained, the deep learning model has strong generalization ability. New ceramic tiles, as long as they are not a completely new texture type, can be used directly without fine-tuning the model.

[0048] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0049] Figure 1 This is a schematic diagram illustrating the steps of a method for detecting color difference on the surface of ceramic tiles provided by the present invention;

[0050] Figure 2 This is a flowchart of a method for detecting color difference on the surface of a ceramic tile, used in a preferred embodiment of the present invention;

[0051] Figure 3 This is a model architecture diagram of a multi-task model combining a texture recognition network and a color classification network used in one embodiment of the present invention.

[0052] Figure 4 This is a schematic diagram of the structure of a ceramic tile surface color difference detection device provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0054] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0055] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0056] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0057] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0058] To address the problems in the background art, embodiments of this application provide a method for detecting color difference on the surface of ceramic tiles, such as... Figure 1 and Figure 2 As shown, the method includes the following steps:

[0059] S01: Obtain the image of the tile to be detected.

[0060] In one specific embodiment, the tile image was captured using an industrial camera on the production line.

[0061] S02: Input the tile image into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information.

[0062] Preferred, such as Figure 3 As shown, the multi-task model includes a ResNet50 backbone network, a texture enhancement module, a multi-directional pyramid texture feature extraction module, a histogram layer, a compact bilinear pooling layer, a fully connected layer, and a global attention layer.

[0063] Based on this multi-task model, extracting texture descriptors with color information includes the following sub-steps:

[0064] S021: Use the ResNet50 backbone network to extract features from the tile image, and obtain the first output feature, the second output feature, the third output feature and the fourth output feature respectively;

[0065] S022: Input the first output feature and the second output feature into the texture enhancement module and the multi-directional pyramid texture feature extraction module to obtain the first texture statistics;

[0066] S023: Input the fourth output feature into the introduced histogram layer to obtain the second texture statistics;

[0067] S024: Merge the first texture statistics with the second texture statistics to obtain the third texture statistics;

[0068] S025: Input the third texture statistics and the fourth output feature into the compact bilinear pooling layer, and fuse the third texture statistics and the fourth output feature using the compact bilinear pooling operation to obtain the bilinear pooling feature;

[0069] S026: Input the bilinear pooling features into the fully connected layer, perform a fully connected operation to reduce dimensionality, and obtain a texture descriptor;

[0070] S027: Input the third output feature and the second output feature into the global attention layer to obtain the weighted low-level features;

[0071] S028: The low-level features are fused with the third output features and the fourth output features to obtain a color descriptor;

[0072] S029: Combine the color descriptor with the texture descriptor to obtain a texture descriptor with color information.

[0073] Specifically, such as Figure 3 As shown, the image data I∈R of the tile to be tested 3×H×W After being input into the multi-task network, I passes through the backbone network to obtain four layers of output feature maps, B1∈R. 256×H / 4×W / 4 B2∈R 512×H / 8×W / 8 B3∈R 1024×H / 16×W / 16 and B4∈R 2048×H / 32×W / 32 .

[0074] Inputting B1, B2, and B4 into the texture recognition branch yields the texture descriptor texture_feature∈R. 128 The specific steps are as follows;

[0075] 3.1 B1 and B2 are downsampled to the size of B4 using average pooling, and then concatenated and fused to obtain feature map B5∈R. 768×H / 32×W / 32 Dimensionality reduction is achieved by sequentially performing 1x1 convolution, batch normalization, and ReLU activation operations on B5, resulting in B6 ∈ R. 256 ×H / 32×W / 32

[0076] 3.2 Input B6 into the Texture Enhancement Module (TEM) to obtain the distribution of all spatial location features TEM∈R 256 ×H / 32×W / 32 The specific steps are as follows.

[0077] 3.2.1 First, perform global average pooling on B6 to obtain the global average feature g∈R. C×1×1 Calculate the cosine similarity between each spatial location in B6 and g to obtain S∈R. 1×H / 32×W / 32 Then readjust the dimensions of S to R. 1×(HW / 32×32) Quantize S into N levels to obtain the set of quantized levels L = {L1, L2, ..., L...} N The N levels of values ​​are obtained by dividing S into N equal points between the minimum and maximum values. Then, each spatial location S in S is calculated. i (i∈[1,HW / 32×32]) belongs to which level, and ultimately S i This will result in a quantization encoding vector H similar to one-hot encoding. i ∈R N (i∈[1,HW / 32×32]), in H i Among them, only S i The dimension of the corresponding level must be non-zero. Statistics are performed on each quantization level based on H, and normalization is applied. The result is then concatenated with the quantization level to obtain C∈R. 2×N The first dimension represents each quantization level, and the second dimension represents the corresponding statistic. Since C is the relative statistical information of B5, to obtain further absolute statistical information, we upsample g to R. C×N And it is concatenated with the C output processed by the MLP to obtain the final output. In other words, the statistical characteristics of each quantification level can be determined by one... It is represented by vectors. The MLP is used to increase the dimension of C; it consists of a convolutional layer and a Leaky ReLU layer.

[0078] 3.2.2 Quantization levels are reconstructed based on histogram equalization to enhance texture details. First, D is treated as a histogram. Then, a graph is constructed to propagate information from all levels, where the statistical features of each quantization level represent a node. A learnable adjacency matrix is ​​calculated using D, and this matrix is ​​used to update each node in the graph, thus reconstructing the quantization levels of D. Finally, the reconstructed quantization levels are assigned to their spatial locations in H to obtain the final output TEM∈R. 256×(HW / 32×32) and readjust the dimensions to R 256×H / 32×W / 32 , because H i It reflects the most primitive quantization level at position i.

[0079] 3.3 B6 and TEM are concatenated and fused to obtain feature map B7∈R 512×H / 32×W / 32 Then, a 1x1 convolution operation is performed to reduce the dimensionality, resulting in B8∈R. 256×H / 32×W / 32 To further obtain the relationships between all spatial locations, B8 is input into the Multi-Directional Pyramid Texture Feature Extraction (MD-PTFEM) module, yielding the output PTFEM∈R. 256×H / 32×W / 32 The specific steps are as follows.

[0080] 3.3.1 A pyramid structure is used to extract texture information from multi-scale features. The overall process is shown in the figure above. First, the feature map B8 is input into four different scale region partitioning branches, where the feature map will be divided into different numbers of sub-regions.

[0081] 3.3.2 Perform a multi-directional collinear feature extraction module (MD-CFEM) on each sub-region. Specifically, assume the size of the sub-region feature map is R. 256×H×W Perform the operations in 3.3.1 on each sub-region to obtain H and L. Readjust the dimensions of H to R. N×1×H×W Calculate H at each spatial location i,j ∈R N×1 With the surrounding 8 spatial locations H i+s,j+t The relationship between (s,t∈[-1,1]) yields 8 quantized collinearity feature statistical graphs, which are then summed as shown in the following formula:

[0082]

[0083] middle The value is not 0. Here, m is the quantization level of the spatial location (i,j), and n is a set containing the quantization levels of each spatial location (i+s,j+t). The final output is... A detailed and comprehensive statistical analysis was conducted on the quantization correlation between adjacent pixels. Then, based on... For each possible pair of quantization collinearities, statistics are performed, and a normalization operation is applied. The result is consistent with all quantization collinearities. By concatenating the components, we obtain C∈R 3 ×N×N The first two dimensions represent collinearity for each pair of quantizations, and the third dimension represents the corresponding statistics. Finally, similar to the texture enhancement module, the average features are upsampled to R0. C×N×N Furthermore, it is concatenated with C, which has undergone dimensionality upscaling via MLP, to obtain the absolute quantized collinear statistical features.

[0084] 3.3.3 Based on the idea of ​​gray-level co-occurrence matrix: modeling texture using statistics. Specifically, ... Input the data into a multilayer perceptron (MLP) network, and then perform layer-by-layer averaging to obtain texture features. That is, the features of each layer are similar to the statistics describing texture features such as contrast and inverse variance in the gray-level co-occurrence matrix. Among them, In the MLP, dimensionality reduction is performed sequentially by performing 1x1 convolution, batch normalization, and ReLU activation, followed by another 1x1 convolution and batch normalization operation.

[0085] 3.3.4 Finally, the texture features of all sub-regions extracted by each of the four branches are concatenated and fused to obtain four texture feature maps output by the branches. These four texture feature maps are then upsampled to the size of the input feature map using bilinear interpolation, and concatenated to obtain the final output PTFEM∈R. 256×H / 32×W / 32 ;

[0086] 3.4 B7 and PTFEM are concatenated and fused to obtain texture statistics extracted from the features of B1 and B2. Global average pooling and flattening operations are then performed to obtain O1∈R. 768 .

[0087] 3.5 Input B4 into the histogram layer (HL), use the histogram layer to describe the local spatial distribution of the deep feature B4, and finally obtain the output O2∈R. 2048 The histogram is modeled using radial basis functions. The average value (μ) of the radial basis functions is then calculated. bk As the bin center, bandwidth (γ) bk Using the bin width as the normalized frequency count Y, the normalized frequency count Y is calculated in the S×T local space. rcbk The binning operation for the histogram value (feature) of the k-th channel of input x is defined as follows:

[0088]

[0089] Where b represents the b-th bin of the histogram, k represents the k-th channel of the input, and r and c represent the spatial positions on the histogram feature map. The choice of the number of bins in the histogram, bin_number, needs to be determined based on the actual application scenario. The specific implementation steps are as follows.

[0090] 3.5.1 Reduce the dimension of B4 by performing a 1×1×K convolution to obtain B4′∈R K×H32×W32 K represents the number of channels in the new feature map.

[0091] 3.5.2 Apply a 1×1×bin_number convolution to each feature map and calculate the distance between each feature and the center of each bin (subtract μ in the formula). bk The bias in the convolution kernel serves as a learnable bin center, with a fixed weight of 1.

[0092] 3.5.3 Apply a 1×1×bin_number convolution to each feature map, multiplying the central feature by the bin width (multiplied by γ in the formula). bk The feature distributions of bins are merged. The weights of the convolution kernel serve as the learnable bin widths, and the bias is fixed at 0.

[0093] 3.5.4 Perform squaring and inversion, apply exponential operations, and calculate the contribution of each feature to each bin. The value of this contribution is restricted to between 0 and 1.

[0094] 3.5.5 Average pooling (sliding window of size S×T) is used to calculate the contribution of features in the local spatial region, thereby obtaining the normalized frequency count of features in each bin.

[0095] 3.5.6 Finally, flatten the result to obtain the output O2∈R 2048

[0096] 3.6 Concatenate and fuse O1 and O2 to obtain the final statistical information of texture details O3∈R 2816 By performing compact bilinear fusion (CBP) on O3 and spatial structure information B4 to maximize the advantages of these two complementary pieces of information, we obtain O4∈R. 8192 .

[0097] 3.7 Perform a fully connected (FC) operation on O4 to obtain the final texture descriptor texture_feature∈R 128 .

[0098] 4. Input B2, B3, and B4 into the color classification branch to obtain color descriptors, then perform a fully connected operation to obtain the final color feature color_feature∈R. 128 The specific steps are as follows:

[0099] 4.1 Input B2 and B3 into the global attention layer to obtain the output selected_feature∈R 256 The specific steps are as follows.

[0100] 4.1.1 Global average pooling, 1x1 convolution, batch normalization, and ReLU activation are performed sequentially on B3 to obtain the global context.

[0101] 4.1.2 Perform 1x1 convolution, batch normalization and ReLU activation sequentially on B2 to reduce its dimensionality, and then perform 3x3 convolution and batch normalization to obtain the dimensionality-reduced B2.

[0102] 4.1.3 Multiply the dimensionality-reduced B2 with the global context to obtain B′2, thereby achieving the purpose of accurately selecting color information in low-level features by weighting low-level features with high-level features.

[0103] 4.1.4 Perform average pooling on B′2 to the same size as B3, then add it to B3 which has undergone 1x1 convolution dimensionality reduction. Next, perform batch normalization, ReLU activation, global average pooling, and flattening operations sequentially to obtain the output selected_feature∈R. 256 .

[0104] 4.2 Perform global average pooling, flattening, and fully connected operations sequentially on B4 to obtain B′4∈R 256 .

[0105] 4.3 Concatenate and fuse B′4 with selected_feature to obtain the result O5∈R 512 .

[0106] 4.4 Dimensionality reduction of O5 is achieved by performing a fully connected layer operation, resulting in color_feature∈R 128 .

[0107] 5. Fuse the texture descriptor `texture_feature` and color feature `color_feature` obtained from the multi-task model to obtain the final colored texture descriptor `final_texture_feature∈R`. 256 .

[0108] S03: Extract the color histogram of the tile image, and use the bin centers and the statistics corresponding to each bin in the color histogram as features describing the image color to obtain the color features corresponding to the tile image.

[0109] In a preferred embodiment, the following sub-steps are included:

[0110] S031: Convert the tile image I from the RGB color space to the CIELab color space to obtain the converted image I′∈R 3×H×W .

[0111] S032: Extract the maximum and minimum values ​​of the pixels in channel i (i = L, a, b) of the transformed image I′, and divide it into 20 equal points. Among them, V i This represents a vector consisting of the bin centers of the i-channel color histogram, where the value of each bin center is... Calculate using the following formula:

[0112]

[0113] S033: Assign all pixels of channel i in the transformed image I′ to each bin, and statistically obtain... Let be the total number of pixels contained in the nth bin; where is the number of pixels in the i-th channel of image I′. It belongs to the nth bin to satisfy the following conditions:

[0114] S034: Use the following formula to convert V i With S i Perform the following normalization operation:

[0115] V L =V L / 100.0

[0116] V a =(V a +128) / 255.0

[0117] V b =(V b +128) / 255.0

[0118] S i =S i / (H×W),i=L,a,b

[0119] S035: V i With S i Perform the concatenation and fusion operation in the following formula order to obtain the color features corresponding to the tile image, i.e., the manually extracted image color descriptor final_color_feature∈R. 120 :

[0120] final_color_feature=Concat(V L ,S L Va ,S a V b ,S b ).

[0121] S04: Read the mean and variance of the RGB three channels of the pre-calculated template tile image as the label feature.

[0122] This feature was used to handle situations where two types of tiles with the same texture but different colors might be misjudged during color difference detection.

[0123] S05: The texture descriptor with color information, the color feature, and the marker feature are fused to obtain the feature vector corresponding to the tile image.

[0124] S06: Input the feature vector into the trained classification model to obtain the judgment result of whether there is color difference in the tile image to be detected.

[0125] In a preferred embodiment, the classification model is a multilayer perceptron network.

[0126] The input to the multilayer perceptron network is a 382-dimensional feature vector. The hidden layer has 30 neurons, the output layer has 1 neuron, and the connections in between are fully connected. The final output passes through a sigmoid function to obtain the probability Q∈(0,1) that the tile to be tested has a color difference.

[0127] Preferably, if Q is greater than or equal to 0.5, the tile image to be detected is determined to have a color difference;

[0128] If Q is less than 0.5, the tile image to be detected is determined to have no color difference.

[0129] This application also provides a device for detecting color difference on the surface of ceramic tiles, such as... Figure 4 As shown, the tile surface color difference detection device 400 includes:

[0130] Tile image acquisition module 401 is used to acquire the tile image to be detected;

[0131] The texture descriptor extraction module 402 is used to input the tile image into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information.

[0132] The color feature extraction module 403 is used to extract the color histogram of the tile image, and use the bin centers and the statistics corresponding to each bin in the color histogram as features describing the image color to obtain the color features corresponding to the tile image.

[0133] The marker feature extraction module 404 is used to read the mean and variance of the RGB three channels of the pre-calculated template tile image as marker features;

[0134] The feature vector fusion module 405 is used to fuse the texture descriptor with color information, the color features and the marker features to obtain the feature vector corresponding to the tile image;

[0135] The color difference judgment module 406 is used to input the feature vector into the trained classification model to obtain the judgment result of whether there is a color difference in the tile image to be detected.

[0136] Preferably, the multi-task model includes a ResNet50 backbone network, a texture enhancement module, a multi-directional pyramid texture feature extraction module, a histogram layer, a compact bilinear pooling layer, a fully connected layer, and a global attention layer.

[0137] Preferably, the texture descriptor extraction module includes:

[0138] The feature extraction unit is used to extract features from the tile image using the ResNet50 backbone network, and obtains a first output feature, a second output feature, a third output feature, and a fourth output feature, respectively.

[0139] The first texture statistics extraction unit is used to input the first output feature and the second output feature into the texture enhancement module and the multi-directional pyramid texture feature extraction module to obtain the first texture statistics.

[0140] The second texture statistics extraction unit is used to input the fourth output feature into the introduced histogram layer to obtain the second texture statistics.

[0141] The third texture statistics extraction unit is used to merge the first texture statistics with the second texture statistics to obtain the third texture statistics.

[0142] A bilinear pooling feature extraction unit is used to input the third texture statistics and the fourth output feature into the compact bilinear pooling layer, and use the compact bilinear pooling operation to fuse the third texture statistics and the fourth output feature to obtain bilinear pooling features;

[0143] The texture descriptor extraction unit is used to input the bilinear pooling features into the fully connected layer, perform a fully connected operation to reduce dimensionality, and obtain the texture descriptor.

[0144] A low-level feature extraction unit is used to input the third output feature and the second output feature into the global attention layer to obtain weighted low-level features;

[0145] The color descriptor extraction unit is used to fuse the low-level features with the third output feature and the fourth output feature to obtain a color descriptor;

[0146] A texture descriptor extraction unit with color information is used to combine the color descriptor with the texture descriptor to obtain a texture descriptor with color information.

[0147] Preferably, the color feature extraction module includes:

[0148] The image conversion unit is used to convert the tile image I from the RGB color space to the CIELab color space to obtain the converted image I′∈R. 3×H×W ;

[0149] The center value calculation unit is used to extract the maximum and minimum values ​​of the pixels in the i (i = L, a, b) channels of the transformed image I′, and divide it into 20 equal points. Among them, V i This represents a vector consisting of the bin centers of the i-channel color histogram, where the value of each bin center is... Calculate using the following formula:

[0150]

[0151] The pixel count unit is used to allocate all pixels in channel i of the transformed image I′ to various bins and count them. Let be the total number of pixels contained in the nth bin; where is the number of pixels in the i-th channel of image I′. It belongs to the nth bin to satisfy the following conditions:

[0152]

[0153] Normalization unit, used to convert V using the following formula i With S i Perform the following normalization operation:

[0154] V L =V L / 100.0

[0155] V a =(V a +128) / 255.0

[0156] V b =(V b +128) / 255.0

[0157] S i =S i / (H×W),i=L,a,b

[0158] The color feature extraction unit is used to extract V i With S i Perform the concatenation and fusion operation in the following formula order to obtain the color features corresponding to the tile image, i.e., the manually extracted image color descriptor final_color_feature∈R. 120 :

[0159] final_color_feature=Concat(V L ,S L V a ,S a V b ,S b ).

[0160] Preferably, the classification model is a multilayer perceptron network.

[0161] Preferably, the input of the multilayer perceptron network is a 382-dimensional feature vector, the hidden layer has 30 neurons, the output layer has 1 neuron, the connection in the middle is fully connected, and the final output will pass through the sigmoid function to obtain the probability Q∈(0,1) that the tile to be tested has a color difference.

[0162] Preferably, if Q is greater than or equal to 0.5, the tile image to be detected is determined to have a color difference;

[0163] If Q is less than 0.5, the tile image to be detected is determined to have no color difference.

[0164] For the apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The apparatus embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0165] This application also provides an electronic device, including:

[0166] At least one memory and at least one processor;

[0167] The memory is used to store one or more programs;

[0168] When the one or more programs are executed by the at least one processor, the at least one processor performs the steps of the tile surface color difference detection method as described above.

[0169] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0170] This application also provides a computer-readable storage medium.

[0171] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method for detecting color difference on the surface of a ceramic tile as described above.

[0172] Computer storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by computing devices.

[0173] This invention provides a method, device, electronic device, and storage medium for detecting color difference on the surface of ceramic tiles. It utilizes the powerful feature extraction capabilities of deep convolutional neural networks to extract texture features from the surface of ceramic tiles, providing a more efficient and accurate description of image texture. The multi-task deep learning model used in this invention has the following advantages over traditional models that only extract texture features: (1) The multi-task network architecture, combining texture recognition and color classification, can obtain not only texture descriptors but also color descriptors. These color descriptors can complement the color features extracted during color difference detection, providing a more comprehensive and accurate description of the image's color information. (2) Once trained, the deep learning model has strong generalization ability. New ceramic tiles, as long as they are not a completely new texture type, can be used directly without fine-tuning the model.

[0174] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for detecting color difference on the surface of ceramic tiles, characterized in that, Includes the following steps: Obtain the image of the tile to be inspected; The tile image is input into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information. Extract the color histogram of the tile image, and use the bin centers and the statistics corresponding to each bin in the color histogram as features describing the image color to obtain the color features corresponding to the tile image; The mean and variance of the RGB three channels of the pre-calculated template tile image are used as labeling features; The texture descriptor with color information, the color features, and the marker features are fused to obtain the feature vector corresponding to the tile image; The feature vector is input into the trained classification model to obtain the judgment result of whether the tile image to be detected has color difference; The multi-task model includes a ResNet50 backbone network, a texture enhancement module, a multi-directional pyramid texture feature extraction module, a histogram layer, a compact bilinear pooling layer, a fully connected layer, and a global attention layer. The tile image is input into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information, including the following steps: The ResNet50 backbone network is used to extract features from the tile image, resulting in a first output feature, a second output feature, a third output feature, and a fourth output feature. The first output feature and the second output feature are input into the texture enhancement module and the multi-directional pyramid texture feature extraction module to obtain the first texture statistics; The fourth output feature is input into the histogram layer to obtain the second texture statistics; The first texture statistics are combined with the second texture statistics to obtain the third texture statistics; The third texture statistics and the fourth output feature are input into the compact bilinear pooling layer, and the third texture statistics and the fourth output feature are fused using the compact bilinear pooling operation to obtain the bilinear pooling feature; The bilinear pooling features are input into the fully connected layer, and a full connection operation is performed to reduce dimensionality, resulting in a texture descriptor. The third output feature and the second output feature are input into the global attention layer to obtain the weighted low-level features; The low-level features are fused with the third and fourth output features to obtain a color descriptor; The color descriptor is combined with the texture descriptor to obtain a texture descriptor with color information.

2. The method for detecting color difference on the surface of ceramic tiles according to claim 1, characterized in that, Extracting the color histogram of the tile image, and using the bin centers and corresponding statistics of each bin in the color histogram as features describing the image color, yields the color features corresponding to the tile image, including the following steps: The tile image Convert from RGB color space to CIELab color space to obtain the converted image. ; Extract the converted images respectively middle The maximum and minimum values ​​of the channel pixels are divided into 20 equal points. ,in, express A vector formed by the bin centers of the channel color histogram, where the value of each bin center is... Calculate using the following formula: The converted image middle All pixels of the channel are assigned to various bins, and the results are statistically analyzed. , For the first The total number of pixels contained in each bin; where, the image middle Each pixel of the channel It belongs to the first Each bin must meet the following conditions: Use the following formula to and Perform the following normalization operation: Will and Perform the concatenation and fusion operation in the following formula order to obtain the color features corresponding to the tile image, i.e., the manually extracted image color descriptor. : 。 3. The method for detecting color difference on the surface of ceramic tiles according to claim 1, characterized in that: The classification model is a multilayer perceptron network.

4. The method for detecting color difference on the surface of ceramic tiles according to claim 3, characterized in that: The input to the multilayer perceptron network is a 382-dimensional feature vector. The hidden layers have 30 neurons, the output layer has 1 neuron, and the connections in between are fully connected. The final output passes through a sigmoid function to obtain the probability that the tested tile has a color difference. .

5. The method for detecting color difference on the surface of ceramic tiles according to claim 4, characterized in that: if If the value is greater than or equal to 0.5, the tile image to be detected is determined to have a color difference; if If the value is less than 0.5, the tile image to be detected is determined to have no color difference.

6. A tile surface color difference detection device for performing the tile surface color difference detection method as described in claim 1, characterized in that, include: The tile image acquisition module is used to acquire images of the tiles to be detected. The texture descriptor extraction module is used to input the tile image into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information. The color feature extraction module is used to extract the color histogram of the tile image, and use the bin centers and the statistics corresponding to each bin in the color histogram as features describing the image color to obtain the color features corresponding to the tile image. The label feature extraction module is used to read the mean and variance of the RGB three channels of the pre-calculated template tile image as label features; The feature vector fusion module is used to fuse the texture descriptor with color information, the color features, and the marker features to obtain the feature vector corresponding to the tile image; The color difference judgment module is used to input the feature vector into the trained classification model to obtain the judgment result of whether there is a color difference in the tile image to be detected; The multi-task model includes a ResNet50 backbone network, a texture enhancement module, a multi-directional pyramid texture feature extraction module, a histogram layer, a compact bilinear pooling layer, a fully connected layer, and a global attention layer. The tile image is input into a multi-task model that combines a trained texture recognition network and a color classification network to obtain a texture descriptor with color information, including the following steps: The ResNet50 backbone network is used to extract features from the tile image, resulting in a first output feature, a second output feature, a third output feature, and a fourth output feature. The first output feature and the second output feature are input into the texture enhancement module and the multi-directional pyramid texture feature extraction module to obtain the first texture statistics; The fourth output feature is input into the histogram layer to obtain the second texture statistics; The first texture statistics are combined with the second texture statistics to obtain the third texture statistics; The third texture statistics and the fourth output feature are input into the compact bilinear pooling layer, and the third texture statistics and the fourth output feature are fused using the compact bilinear pooling operation to obtain the bilinear pooling feature; The bilinear pooling features are input into the fully connected layer, and a full connection operation is performed to reduce dimensionality, resulting in a texture descriptor. The third output feature and the second output feature are input into the global attention layer to obtain the weighted low-level features; The low-level features are fused with the third and fourth output features to obtain a color descriptor; The color descriptor is combined with the texture descriptor to obtain a texture descriptor with color information.

7. An electronic device, characterized in that, include: At least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a method for detecting color difference on a tile surface as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a method for detecting color difference on a ceramic tile surface as described in any one of claims 1-5.

Images