A defect detection method and system based on distribution representation

Abstract

The application discloses a kind of defect detection method and system based on distribution representation, belong to industrial product defect detection technical field.The application is directed to the problem that simple random sampling is prone to distribution deviation on the industrial dataset with small defect sample size, the application designs distribution representation network to obtain data distribution as the theoretical basis for dataset splitting;Meanwhile, distance factor and density factor are considered comprehensively, distance factor can effectively control sampling in each region on the entire data distribution, ensuring the comprehensiveness of sample region sampling, and density factor can adaptively adjust the number of sample sampling according to the aggregation degree of samples, ensuring the uniformity of extracted samples.The application can effectively ensure the consistency of split subset data distribution, and help to promote more stable performance of model and lower classification error.

Description

Technical Field

[0001] This invention belongs to the field of industrial product defect detection technology, and more specifically, relates to a defect detection method and system based on distribution characterization. Background Technology

[0002] Defect detection plays a crucial role in monitoring production processes and ensuring product quality. With the rapid development of deep learning in computer science, many researchers have applied techniques such as image classification, semantic segmentation, and object detection to detect surface defects in industrial products, achieving significant progress compared to traditional manual feature extraction methods. Data splitting is a fundamental and essential step in deep learning model development, involving sampling the dataset into training, test, and validation sets. It is important to note that introducing additional bias during data splitting can significantly impact model performance. Therefore, it is essential to ensure that the training and test sets are as independent and identically distributed as possible. This allows the model's performance metrics on the test set to accurately reflect its generalization error, resulting in a truly reliable performance evaluation.

[0003] Currently, the most widely used data splitting method remains simple random sampling. For large public datasets with abundant sample numbers and rich image diversity, random splitting can ensure consistency in data distribution between the training and test sets, based on the central limit theorem of probability theory. However, in industrial scenarios, the product qualification rate on production lines typically reaches over 99%, with minimal outlier data. Random splitting can easily lead to significant differences in data distribution between the training and test sets, preventing them from equally representing the problem domain. This makes it difficult to maintain stable model performance across multiple splits and to evaluate the model's true performance. Therefore, random splitting is insufficient for splitting small industrial image datasets, making it essential to research a new method for splitting industrial datasets. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a defect detection method and system based on distribution representation. Its purpose is to solve the technical problem that random splitting can easily lead to large differences in the data distribution of the training set and the test set, which cannot equally represent the problem domain, and the model performance cannot be kept stable in multiple data splits.

[0005] To achieve the above objectives, according to one aspect of the present invention, a defect detection method based on distribution characterization is provided, comprising:

[0006] S1. Label normal images and defect-category images in the industrial image dataset;

[0007] S2. Perform multiple data augmentations on the labeled industrial image dataset images, using two augmented views from the same image as positive sample pairs and augmented views from different images as negative sample pairs;

[0008] S3. Construct a distributed representation network; the distributed representation network includes an encoder and a projection head; the encoder is used to extract image features and map the enhanced view to a latent space; the projection head is used to further map the image features extracted by the encoder to a unit hypersphere;

[0009] S4. Input positive and negative sample pairs into the distribution representation network, and introduce prior label information into the contrastive loss as a loss function to iteratively train the distribution representation network;

[0010] S5. Input the industrial image dataset into the trained distribution representation model to obtain the data distribution; the representation vector in the data distribution corresponds one-to-one with each image in the dataset.

[0011] S6. Divide all representation vectors into multiple neighborhoods: quantify the feature differences between vectors, and divide vectors with feature differences less than a set threshold into the same neighborhood;

[0012] S7. Sample the center point of each neighborhood, and determine the number of samples α in each neighborhood based on the degree of clustering of samples within the neighborhood. Sample α samples again in each neighborhood. The images corresponding to the center point of each neighborhood and the α sample points constitute the test set.

[0013] S8. Use the remaining images as the training set; and repeat steps S6-S7 on the remaining images to obtain the validation set.

[0014] S9. Defect detection is performed using the divided training, test, and validation sets.

[0015] Furthermore, the specific process of each data augmentation is as follows:

[0016] Multiple image transformation methods are selected with equal probability from the data transformation space to perform preliminary data augmentation on the dataset images; the data transformation space includes pixel value inversion, histogram equalization, random smearing, rotation, Gaussian blur and affine transformation.

[0017] Perform routine data transformations on the pre-enhanced dataset images; routine data transformations include random cropping and normalization in a fixed order, random horizontal flipping, changing image attributes, and random conversion to grayscale.

[0018] Furthermore, the loss function for iteratively training the distributed representation network is:

[0019]

[0020] C(i) is the vector v iThe set corresponding to all positive samples, i∈B={1,2,3...2N}, is the index of the 2N augmented views in batch B, v i =g(f(x) i )) is the representation vector output by the projection head, (i, j) is the index of a set of positive sample pairs, 1 (k≠i) The indicator function is 1 if and only if k ≠ i, τ is the temperature parameter, and sim(v i v j ) = v i ·v j / ||v i ||||v j || represents the cosine similarity function.

[0021] Furthermore, during the training of the distributed representation network, the silhouette coefficient is used as an evaluation index for the quality of the distributed representation. When it reaches a set threshold, the iterative training is terminated.

[0022] Furthermore, to characterize vector x * The neighborhood N centered on ∈ (x * )satisfy:

[0023] N ∈ (x * )={x i ∈D|dist(x i x * )≤∈}

[0024] Where dist(·) is the distance metric function used to numerically represent the feature differences between vectors, D={x1, x2, ..., x n} represents the set of representation vectors, and ∈ represents the set distance threshold.

[0025] Furthermore, the calculation method for determining the number α of samples in each neighborhood based on the degree of clustering of samples within the neighborhood is as follows:

[0026]

[0027] Where ρ: 0 < ρ < 1 represents the density variable, adjusting ρ controls the sampling density within each neighborhood, round is the rounding function, MinVecs is the set threshold, |N ∈ (x)| reflects the degree of clustering of samples in the neighborhood.

[0028] The present invention also provides a defect detection system based on distribution characterization, comprising:

[0029] The image labeling module is used to label normal images and defect category images in industrial image datasets;

[0030] The data augmentation module performs multiple data augmentations on the labeled industrial image dataset images, treating two augmented views from the same image as positive sample pairs, while augmented views from different images as negative sample pairs.

[0031] A distributed representation network construction module is used to construct a distributed representation network. The distributed representation network includes an encoder and a projection head. The encoder is used to extract image features and map the enhanced view to a latent space. The projection head is used to further map the image features extracted by the encoder to a unit hypersphere.

[0032] The distribution representation network training module is used to input positive sample pairs and negative sample pairs into the distribution representation network, and to introduce label prior information into the contrastive loss as a loss function to iteratively train the distribution representation network.

[0033] The data distribution acquisition module is used to input the industrial image dataset into a trained distribution representation model to obtain the data distribution; the representation vector in the data distribution corresponds one-to-one with each image in the dataset.

[0034] The neighborhood partitioning module is used to divide all representation vectors into multiple neighborhoods: it quantifies the feature differences between vectors and divides vectors with feature differences less than a set threshold into the same neighborhood.

[0035] The first dataset splitting module is used to sample the center points of each neighborhood and determine the number of samples α in each neighborhood based on the degree of clustering of samples within the neighborhood. In each neighborhood, α samples are sampled again. The images corresponding to the center points of each neighborhood and the α sample points constitute the test set.

[0036] The second dataset splitting module is used to use the remaining images as the training set; and performs the processing of the neighborhood partitioning module and the sampling module on the remaining images to obtain the validation set;

[0037] The defect detection module uses a divided training set, test set, and validation set to perform defect detection.

[0038] Overall, the above-described technical solutions conceived by this invention can achieve the following beneficial effects compared with the prior art.

[0039] (1) This invention addresses the problem that simple random sampling is prone to distribution bias in industrial datasets with a small number of defective samples. This invention designs a distribution representation network to obtain the data distribution as the theoretical basis for dataset splitting. At the same time, it comprehensively considers distance and density factors. The distance factor can effectively control the sampling in various regions of the entire data distribution to ensure the comprehensiveness of sample region sampling. The density factor can adaptively adjust the number of sample samples according to the degree of sample aggregation to ensure the uniformity of the extracted samples. This can effectively ensure the consistency of the data distribution of the training set and the test set, overcome the potential bias introduced in the data splitting process, and obtain a more stable model and a more reliable model performance evaluation.

[0040] (2) This invention uses a variety of data transformation methods to perform stronger data transformation on images, which can generate more challenging negative examples and enable the model to extract features more comprehensively.

[0041] (3) This invention uses contrastive loss and introduces label category information as prior knowledge into the loss function to guide and constrain the extraction of features of the distribution representation network, which can reduce the number of training samples of the model and strengthen the connection with subsequent defect detection tasks.

[0042] (4) This invention has good generalization and transferability, and can be applied to different industrial image datasets. There is no need to perform tedious hyperparameter optimization. It is only necessary to train the distribution representation model on the new dataset. The hyperparameters involved can be easily determined according to the split ratio of the dataset. Attached Figure Description

[0043] Figure 1 A flowchart of a novel two-stage industrial dataset splitting method based on distributed representation provided by the present invention;

[0044] Figure 2 This is a schematic diagram of part of the image data in the industrial image dataset constructed for an embodiment of the present invention.

[0045] Figure 3 A schematic diagram of the data augmentation module and distributed representation network constructed for this invention;

[0046] Figure 4 Figures (a)-(c) are schematic diagrams of the splitting strategy designed in this invention;

[0047] Figure 5 The box plots for the model classification error rate in 20 splitting experiments are shown in this embodiment of the invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] The defect detection method based on distribution representation provided by this invention includes the following steps, the core process of which is described in reference [reference needed]. Figure 1 :

[0050] (1) Dataset Construction

[0051] Since the primary objective of this invention is to split the dataset, and the entire dataset is needed for training the distributed representation model, it is not necessary to split it into training and test sets before training. All images are normalized to a pixel size of 512x512, and image-level annotations are performed on all images, labeling them as normal images and subdivided defect category images. Some dataset images can be found... Figure 2 The sample size and number of categories for the three datasets are shown in Table 1.

[0052] Table 1. Dataset sample size and number of categories

[0053] Dataset Sample size Number of classes NEU 1800 6 OCC 3300 4 MTD 1344 6

[0054] (2) Data Augmentation

[0055] The purpose of this data augmentation invention is to generate two different augmented views of the same image, which includes a conventional data transformation. A stronger data transformation See details Figure 3 (a) includes the following steps:

[0056] (2-1) Conventional data transformation methods It consists of random cropping in a fixed order, horizontal flipping, changing image attributes (brightness, contrast, saturation and hue), and conversion to grayscale. Conventional transformations mainly change the position and size of the image.

[0057] (2-2) Seven data transformation methods, including pixel value inversion, histogram equalization, random smearing, rotation (not included in conventional transformations), and Gaussian blur, are selected to construct a data transformation space. Two data transformation methods are selected from these methods with equal probability each time. According to permutations and combinations, we can know have Various combination methods.

[0058] (2-3) will Insert into This allows for more powerful data transformations on the image, increasing transformation diversity and generating more challenging negative examples, thus prompting the model to extract features more comprehensively, such as... Figure 3 As shown in (a), the input image x passes through... and Create two enhanced views x i x j .

[0059] (3) Construct a distributed representation network. The distributed representation network is used to map input data to a low-dimensional space. It consists of an encoder and a projection head.

[0060] (3-1) Construct an encoder f(·), which projects the view passed through the data augmentation module into a latent space rich in image feature information. The encoder can choose from various mainstream feature extraction models. Considering the number of samples in the industrial dataset and the feature extraction capability of the model, the encoder of this invention adopts ResNet-50 (excluding the final fully connected layer used for classification).

[0061] (3-2) Construct a projection head g(·), which is a multilayer perceptron. As an optional embodiment, this invention sets it to have a single hidden layer of size 2048, further mapping the image features extracted by the encoder to the contrast loss space. The output space is a 128-dimensional unit hypersphere.

[0062] (4) Iterative training of the model; including contrastive loss and the introduction of prior knowledge of the labels.

[0063] (4-1) Construct contrastive loss, the goal of which is to reduce the difference between positive sample pairs and increase the distance between negative sample pairs to learn the distribution representation. The calculation formula is as follows.

[0064]

[0065] Where i∈B={1,2,3...2N} is the sequence number of the 2N enhanced views in batch B, v i =g(f(x) i )) is the representation vector output by the projection head, (i, j) is the index of a set of positive sample pairs, 1 (k≠i) The indicator function is 1 if and only if k ≠ i, τ is the temperature parameter, and sim(v i v j ) = v i ·v j / ||v i ||||v j || represents the cosine similarity function.

[0066] (4-2) Label prior information is introduced into the contrastive loss to guide and constrain feature extraction, reduce the number of training samples for the model, and strengthen the connection with subsequent defect detection tasks. Samples belonging to the same category as the current image in the batch are obtained through label information, and their enhanced views are added to the positive examples. The final loss function is as follows:

[0067]

[0068] Where C(i) is the vector v i The set corresponding to all positive samples.

[0069] (4-3) Randomly sort the images in the dataset with a sample size of N and distribute them evenly, with each batch containing 256 images.

[0070] (4-4) The input data is entered into the data augmentation module in batches. The data augmentation module performs augmentation twice on each image to generate two related augmented views. Therefore, the total number of training images is 2N.

[0071] (4-5) Input the training images into the model in batches. After the gradient descent values ​​of all samples in a batch have been calculated, update the network parameter weights. Repeat the above process until all batches of samples have been updated, and then proceed to the next iteration.

[0072] This invention uses the silhouette coefficient (SC) as an evaluation index for the representation quality of the model, and the calculation formula is as follows:

[0073]

[0074] Where a(i) represents the average distance from vector i to all other samples in its cluster, b(i) represents the average distance from vector i to the sample in its nearest cluster, s(i)∈[-1,1] is the silhouette coefficient of the sample, and the silhouette coefficient of the set is the average of the silhouette coefficients of all samples. The silhouette coefficient uses the similarity measure between samples to evaluate the density and dispersion of clusters, which fits well with the requirements of the representation quality evaluation of distributed representation networks. Calculating the SC value can effectively monitor the training process of the distributed representation network to ensure that the data distribution quality acquired by the distributed representation network reaches the set threshold.

[0075] (4-6) Every 20 iterations, the current training model is evaluated using the SC evaluation index. When the SC value reaches the threshold of 0.3, training is stopped.

[0076] (5) Figure 3 In (b), the entire dataset is directly fed into the trained representation model without going through the data augmentation module to obtain the distribution representation vector. The representation vector corresponds one-to-one with the sample, thus obtaining the data distribution of the entire dataset.

[0077] (6) Based on the data distribution representation, the dataset is split, including two sampling processes: sampling based on distance factors and sampling based on density factors.

[0078] (6-1) Sampling based on distance factors (e.g.) Figure 4 As shown in (a)-(b), assume Figure 4 In diagram (a), the data distribution is obtained. A vector point is randomly selected, and its neighborhood is calculated. This process is repeated for the remaining representation vectors until all representation vector points are contained within a certain neighborhood. When ∈ takes a small value, the resulting neighborhood range is small, and the features of the samples in the neighborhood can be considered similar. By controlling the distance variable ∈ and selecting the center point of each neighborhood, it is ensured that the extracted samples are evenly distributed across the various regions of the hypersphere.

[0079] (6-2) Sampling based on density factors (e.g.) Figure 4 (b) Figure 4 As shown in (c), traverse each neighborhood in (6-1). When the number of vector points in the neighborhood exceeds the set threshold MinVecs, randomly select α samples from the neighborhood.

[0080] (6-3) Select the samples corresponding to the vector points sampled in steps (6-1) and (6-2) to form the test set. The remaining samples in the dataset are the training set. This completes the splitting of the dataset.

[0081] Dataset splitting evaluation.

[0082] First, the differences in data distribution between the training set and the test set obtained from the split are evaluated using the metrics MMD (maximum mean discrepancy) and EMD (earth mover's distance).

[0083] The present invention is applied to further split the training set into a validation set. The final ratio of the sample sizes of the training set, validation set, and test set is 6:2:2. When the model performs best on the validation set, the error rate on the test set is taken as the final result. For further comparison, the training set and test set are swapped while keeping the validation set unchanged, and another set of experiments is conducted.

[0084] A defect classification model was built on the dataset splitting results for testing. Since this invention focuses on the effectiveness of the data splitting method rather than the network structure, the most commonly used ResNet was selected and general settings were followed in order to more clearly observe the impact of data splitting. The classification error and standard deviation were used to evaluate the impact of the splitting method on the model performance.

[0085] To fully verify the effectiveness of the method and ensure the accuracy of the experiment, the above complete experimental steps were independently repeated 20 times and the experimental data were recorded. Tables 2 and 3 show the mean and variance of the MMD and EMD distances of the training and test set data distributions obtained by the proposed method and the Simple Random Sampling (SRS) method, respectively.

[0086] Table 2. Quantitative comparison of the mean distances of MMD and EMD

[0087]

[0088] Table 3. Quantitative comparison of the standard deviations of MMD and EMD distances.

[0089]

[0090] Based on the results in Tables 2 and 3, the mean and standard deviation of the distances for MMD and EMD in this invention are significantly lower than those for the SRS method. The lower mean distance indicates that the data distributions of the training and test sets are basically consistent, and the lower standard deviation indicates that the proposed partitioning method exhibits low variability in data distribution differences between partitions and good stability. Conversely, the mean and standard deviation of the random partitioning method are significantly higher than those of the proposed method, while the SRS partitioning results show large differences in data distribution and high variability and poor stability between partitions.

[0091] Secondly, the defect classification results are compared. Tables 4 and 5 show the mean and standard deviation of the test error of the model trained on 20 classification results, respectively. Figure 5 The box plot in the middle records the results of 20 test errors. The defect classification model using the proposed method exhibits low performance variability and good stability on the dataset partitioning results, leading to more accurate performance evaluations. In contrast, on random partitioning results, the model performance is highly variable and extremely unstable, resulting in erroneous performance assessments.

[0092] Table 4. Quantitative Comparison of Mean Classification Errors of Models

[0093]

[0094] Table 5. Quantitative comparison of standard deviations of model classification errors

[0095]

[0096] In summary, this invention mainly includes the collection of industrial image datasets, labeling image categories, designing a data augmentation module and data augmentation space, designing a distributed representation network (DRN), constructing model optimization objectives and introducing prior label knowledge, selecting the optimal model, and designing a splitting strategy to complete the splitting of the industrial dataset. First, this invention accurately obtains the data distribution of the dataset to be split by designing a distributed representation network (DRN) and introducing prior label knowledge. Then, the proposed splitting strategy (DDS) is used to split the obtained representation vectors to obtain the final data splitting result. The results show that this invention has low MMD and EMD distances, indicating that the proposed method can effectively ensure the consistency of the data distribution between the training and test sets, while also having lower classification errors and standard deviations, indicating that a more stable model and more accurate performance evaluation can be obtained.

[0097] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A defect detection method based on distribution representation, characterized in that, include: S1. Label normal images and defect-category images in the industrial image dataset; S2. Perform multiple data augmentations on the labeled industrial image dataset images, using two augmented views from the same image as positive sample pairs and augmented views from different images as negative sample pairs; S3. Construct a distributed representation network; the distributed representation network includes an encoder and a projection head; the encoder is used to extract image features and map the enhanced view to a latent space; the projection head is used to further map the image features extracted by the encoder to a unit hypersphere; S4. Input positive and negative sample pairs into the distribution representation network, and introduce label prior information into the contrastive loss as a loss function to iteratively train the distribution representation network; S5. Input the industrial image dataset into the trained distribution representation model to obtain the data distribution; the representation vector in the data distribution corresponds one-to-one with each image in the dataset. S6. Divide all representation vectors into multiple neighborhoods: quantify the feature differences between vectors, and divide vectors with feature differences less than a set threshold into the same neighborhood; S7. Sample the center point of each neighborhood, and determine the number of samples α in each neighborhood based on the degree of clustering of samples within the neighborhood. Sample α samples again in each neighborhood. The images corresponding to the center point of each neighborhood and the α sample points constitute the test set. S8. Use the remaining images as the training set; and repeat steps S6-S7 on the remaining images to obtain the validation set; S9. Defect detection is performed using the partitioned training, test, and validation sets.

2. The defect detection method based on distribution representation according to claim 1, characterized in that, The specific process for each data augmentation is as follows: Multiple image transformation methods are selected with equal probability from the data transformation space to perform preliminary data augmentation on the dataset images; the data transformation space includes pixel value inversion, histogram equalization, random smearing, rotation, Gaussian blur and affine transformation. Perform routine data transformations on the pre-enhanced dataset images; routine data transformations include random cropping and normalization in a fixed order, random horizontal flipping, changing image attributes, and random conversion to grayscale.

3. The defect detection method based on distribution representation according to claim 2, characterized in that, The loss function for iteratively training the distributed representation network is: C(i) is a vector v i The set corresponding to all positive samples, i∈B={1,2,3…2N}, is the index of the 2N augmented views in batch B, v i =g(f(x) i ()) is the representation vector output by the projection head, (i,j) is the index of a set of positive sample pairs, 1 (k≠i) The indicator function is 1 if and only if k ≠ i, τ is the temperature parameter, and sim(v i ,v j ) = v i ·v j / ||v i ||||v j || represents the cosine similarity function.

4. The defect detection method based on distribution representation according to claim 3, characterized in that, During the training of the distributed representation network, the silhouette coefficient is used as the quality evaluation index of the distributed representation. When it reaches a set threshold, the iterative training is terminated.

5. The defect detection method based on distribution representation according to claim 1, characterized in that, To represent vector x * The neighborhood N centered on ∈ (x * )satisfy: N ∈ (x * )={x i ∈D|dist(x i ,x * )≤∈} Where dist(·) is the distance metric function used to numerically represent the feature differences between vectors, D={x1,x2,…,x n } represents the set of representation vectors, and ∈ represents the set distance threshold.

6. The defect detection method based on distribution representation according to claim 1, characterized in that, The method for calculating the number α of samples in each neighborhood based on the degree of clustering of samples within the neighborhood is as follows: Where ρ: 0 < ρ < 1 represents the density variable, adjusting ρ controls the sampling density within each neighborhood, round is the rounding function, MinVecs is the set threshold, |N ∈ (x)| reflects the degree of clustering of samples in the neighborhood.

7. A defect detection system based on distribution representation, characterized in that, include: The image labeling module is used to label normal images and defect category images in industrial image datasets; The data augmentation module performs multiple data augmentations on the labeled industrial image dataset images, treating two augmented views from the same image as positive sample pairs, while augmented views from different images as negative sample pairs. Distributed representation network building blocks; This is used to construct a distributed representation network; the distributed representation network includes an encoder and a projection head; the encoder is used to extract image features and map the enhanced view to a latent space; the projection head is used to further map the image features extracted by the encoder to a unit hypersphere; The distribution representation network training module is used to input positive sample pairs and negative sample pairs into the distribution representation network, and to introduce label prior information into the contrastive loss as a loss function to iteratively train the distribution representation network. The data distribution acquisition module is used to input the industrial image dataset into a trained distribution representation model to obtain the data distribution; the representation vector in the data distribution corresponds one-to-one with each image in the dataset. The neighborhood partitioning module is used to divide all representation vectors into multiple neighborhoods: it quantifies the feature differences between vectors and divides vectors with feature differences less than a set threshold into the same neighborhood. The first dataset splitting module is used to sample the center points of each neighborhood and determine the number of samples α in each neighborhood based on the degree of clustering of samples within the neighborhood. In each neighborhood, α samples are sampled again. The images corresponding to the center points of each neighborhood and the α sample points constitute the test set. The second dataset splitting module is used to split the remaining images into a training set. The validation set is obtained by performing neighborhood partitioning and sampling on the remaining images. The defect detection module uses a divided training set, test set, and validation set to perform defect detection.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to perform a defect detection method based on distribution representation as described in any one of claims 1 to 5.

Images