A data labeling method for surface defect detection of industrial products

Abstract

The application discloses a data labeling method for surface defect detection of industrial products, comprising the following steps: training a feature extraction model and a labeling classification model using a defect labeled data set; extracting a feature vector of a defect unlabeled data set by using the trained feature extraction model; inputting the feature vector into the trained labeling classification model; selecting sample images that need to be manually labeled and giving them to manual labeling; expanding the manually labeled sample images to the labeled data set and removing them from the defect unlabeled data set; judging whether the feature extraction model and the labeling classification model meet the requirements, ending the defect classification labeling if the requirements are met, or repeating the above training otherwise. The method not only reduces the labeling cost, but also improves the labeling efficiency and the labeling quality.

Description

Technical Field

[0001] This invention relates to the field of surface defect detection of industrial products, and in particular to a data annotation method for surface defect detection of industrial products. Background Technology

[0002] Surface defect detection in industrial products is a crucial step in industrial production and plays a significant role in improving product quality. Because surface defect detection in industrial products requires high precision, most methods employ supervised learning. Supervised learning, however, requires a large amount of labeled data to train the model; therefore, the quality and quantity of labeled data are critical to the results of surface defect detection.

[0003] Currently, defect labeling of industrial products mainly relies on manual labeling, which consumes a lot of manpower and resources, and the consistency of the labeled data is poor. Because industrial product surface defect data suffers from problems such as few labeled samples, imbalanced samples, large differences in defect area, and the presence of outliers during the construction of intelligent defect labeling models, there is an urgent need to explore intelligent defect labeling models to reduce labeling costs and improve labeling efficiency and quality. Summary of the Invention

[0004] The main technical problem solved by this invention is to provide a data annotation method for detecting surface defects in industrial products, which addresses the issues of insufficient annotation samples, uneven sample distribution, large differences in defect area, and the presence of outliers in surface defect data of industrial products.

[0005] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a data annotation method for detecting surface defects in industrial products, comprising the following steps:

[0006] The first step is to obtain a defect-annotated dataset for labeling surface defects of industrial products. The defect-annotated dataset includes the first batch of defect sample images with labeled defects.

[0007] The second step is to train the feature extraction model using the defect-labeled dataset to extract feature vectors from the first batch of defect sample images.

[0008] The third step is to input the feature vectors of the first batch of defective sample images as training data into the labeled classification model and train it.

[0009] The fourth step involves classifying the first batch of unlabeled sample images of unlabeled defects in the unlabeled defect dataset using the trained feature extraction model and labeling classification model to obtain the second batch of unlabeled sample images of unlabeled defects.

[0010] Fifth step: If the ratio of the number of unlabeled sample images in the second batch to the number of unlabeled sample images in the defect unlabeled dataset is greater than a preset threshold, then the second batch of unlabeled sample images is manually labeled to obtain the second batch of defect sample images.

[0011] The sixth step, following the fifth step, merges the second batch of defect sample images with the first batch of defect sample images to form the labeled defect dataset required for the next round of iteration; removes the second batch of defect sample images from the unlabeled defect dataset to form the unlabeled defect dataset required for the next round of iteration; then returns to the first step to perform generational training on the feature extraction model and the labeling classification model.

[0012] Step 7, following step 4, if the ratio of the number of unlabeled sample images in the second batch of the defect-unlabeled dataset to the number of unlabeled sample images in the first batch is less than or equal to the preset threshold, it indicates that both the feature extraction model and the labeling classification model have completed training, and the defect classification labeling ends.

[0013] Preferably, the feature extraction model includes a downsampling unit, an upsampling unit, and a standardization unit. The downsampling unit includes multiple concatenated convolutional blocks. The downsampling unit performs continuous downsampling on the first batch of defect sample images and outputs multiple features with different semantics to the upsampling unit. The upsampling unit includes multiple concatenated fusion convolutional blocks. The upsampling unit performs continuous upsampling on multiple features with different semantics and laterally connects the multiple features with different semantics output from the downsampling unit. After fusion, multiple fused features are formed. The multiple fused features are then subjected to 3x3 convolution operations to form multiple feature maps. The multiple feature maps are laterally output to the standardization unit. The standardization unit performs normalization and activation processing on the multiple feature maps to prevent overfitting of the feature extraction model and performs nonlinear transformation on the multiple feature maps. Finally, it outputs multiple feature vectors of the first batch of defect sample images.

[0014] Preferably, the downsampling processing unit includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a fifth convolutional block connected in series. The first convolutional block extracts image features from the first batch of defect sample images to form a first semantic feature and outputs the first semantic feature to the second convolutional block. The second convolutional block downsamples the input first semantic feature to form a second semantic feature and outputs the second semantic feature to the third convolutional block. Simultaneously, the second semantic feature is output to the first input of the upsampling processing unit through the first output terminal of the downsampling processing unit. The third convolutional block downsamples the second semantic feature extracted by the second convolutional block to form a third semantic feature and outputs the third semantic feature. The third semantic feature is extracted from the third convolutional block and then downsampled to form the fourth semantic feature. The fourth convolutional block outputs the third semantic feature extracted from the third convolutional block to the second input of the upsampling processing unit. The fourth convolutional block downsamples the third semantic feature extracted from the third convolutional block to form the fourth semantic feature and outputs the fourth semantic feature to the fifth convolutional block. The fifth convolutional block downsamples the fourth semantic feature extracted from the fourth convolutional block to form the fifth semantic feature and outputs the fifth semantic feature. The fifth semantic feature outputs the fifth semantic feature to the fourth input of the upsampling processing unit. Preferably, the upsampling processing unit includes a fourth fusion convolutional block, a third fusion convolutional block, a second fusion convolutional block, and a first fusion convolutional block connected in series. The input of the fourth fusion convolutional block is connected to the fourth input of the upsampling processing unit. The fifth semantic feature from the downsampling processing unit is output as a fourth fusion feature to the third fusion convolutional block through the output of the fourth fusion convolutional block. Simultaneously, the fourth fusion feature is output laterally to the fourth input of the normalization processing unit through the fourth output of the upsampling processing unit. The third fusion convolutional block upsamples the fourth fusion feature and fuses it with the fourth semantic feature input from the third input of the upsampling processing unit to form a third fusion feature. The third fusion feature is then output to the second fusion convolutional block. Simultaneously, the upsampling... The third output of the processing unit horizontally outputs the third fused feature to the third input of the standardization processing unit; the second fused convolutional block upsamples the third fused feature and fuses it with the third semantic feature input from the second input of the upsampling processing unit to form the second fused feature, and outputs the second fused feature to the first fused convolutional block. At the same time, the second fused feature is horizontally output to the second input of the standardization processing unit through the second output of the upsampling processing unit; the first fused convolutional block upsamples the second fused feature and fuses it with the second semantic feature input from the first input of the upsampling processing unit to form the first fused feature, and horizontally outputs the first fused feature to the first input of the standardization processing unit through the first output of the upsampling processing unit.

[0015] The first fusion feature output from the first output terminal of the upsampling processing unit, the second fusion feature output from the second output terminal, the third fusion feature output from the third output terminal, and the fourth fusion feature output from the fourth output terminal are respectively processed by 3x3 convolution operation to form four different feature maps, which are respectively input to the first input terminal, the second input terminal, the third input terminal, and the fourth input terminal of the standardization processing unit.

[0016] Preferably, the standardization processing unit performs normalization and activation processing on the four feature maps input from its four input terminals, and outputs four 1x100 feature vectors through the first, second, third and fourth output terminals of the standardization processing unit.

[0017] Preferably, the feature extraction model is trained using the Focal Loss class-balanced loss function, which is defined as follows:

[0018] Where z represents the prediction of all sample categories, z = [z1, z2, ..., z C ] T C represents the total number of categories. N is the sample volume, n y p represents the number of samples in the true class y, γ is an adjustable factor, and p i This represents the probability of the predicted category by the feature extraction model.

[0019] Preferably, the labeling classification model fits a hypersphere to each class of data in the defect-labeled dataset based on robust least squares support vector data description. The fitted hypersphere is only related to the defect-labeled dataset of that class and is not related to the defect-labeled datasets of other classes.

[0020] Preferably, the loss function used for robust least squares support vector data description is the smooth exponential least squares loss function.

[0021] Preferably, the training defect-labeled dataset is denoted as T = {(x1,y1),(x2,y2),...,(x...}. m ,y m )},in For training samples, y i ∈{1,...,k} represents the category label of the defect sample, i∈{1,...,m}; the labeling classification model constructs a minimum hypersphere that encloses all or nearly all training defect labeled datasets for each class of data in the training defect labeled dataset. The k hyperspheres are solved by solving k quadratic programming problems.

[0022] Furthermore, the general expression for the smooth exponential least squares loss function is: L σ(ξ)=σ 2 (1-exp(-ξ 2 / σ 2 )), where ξ represents the error and σ is the loss function parameter.

[0023] Based on the above technical solutions, this invention provides a data annotation method for detecting surface defects in industrial products. It trains a feature extraction model and an annotation classification model using a labeled defect dataset. The trained feature extraction model extracts feature vectors from the unlabeled defect dataset. These feature vectors are then input into the trained annotation classification model, which selects sample images requiring manual annotation. The manually annotated sample images are then expanded into the labeled dataset and removed from the unlabeled defect dataset. The system is then evaluated to determine if the feature extraction and annotation models meet the requirements. If they do, the defect classification and annotation process ends; otherwise, the training is repeated. This invention employs a Focal Loss class-balanced loss function to train the feature extraction model, which better extracts features from imbalanced datasets. Furthermore, this invention proposes an annotation classification model based on a smooth exponential least squares loss function, effectively addressing the issue of outliers in industrial product surface defect datasets. Through active learning, it solves the problem of insufficient labeled defect samples in the defect dataset, fully utilizing the information from both labeled and unlabeled samples to achieve accurate annotation of surface defects in industrial products. This method not only reduces annotation costs but also improves annotation efficiency and quality. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the data annotation method for detecting surface defects in industrial products according to the present invention;

[0025] Figure 2 This is a schematic diagram of the structure of a feature extraction model according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the first positional relationship between a sample point and a hypersphere according to an embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram illustrating a second positional relationship between a sample point and a hypersphere according to an embodiment of the present invention;

[0028] Figure 5 This is a schematic diagram of the third positional relationship between a sample point and a hypersphere according to an embodiment of the present invention;

[0029] Figure 6 The images shown in the experiment are defective images, and all feature maps extracted using the feature extraction model in the embodiments of the present invention are also included.

[0030] Figure 7These are the defect-free images from the experiment and all feature maps extracted using the feature extraction model in this embodiment of the invention. Detailed Implementation

[0031] To facilitate understanding of the present invention, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. Preferred embodiments of the invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the present invention.

[0032] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0033] Figure 1 This is a schematic diagram of a data annotation method for detecting surface defects in industrial products according to the present invention, including the following steps:

[0034] SI: The first step is to obtain a defect-annotated dataset for labeling surface defects of industrial products. The defect-annotated dataset includes the first batch of defect sample images with labeled defects.

[0035] S2: The second step is to train the feature extraction model using the defect-labeled dataset to extract the feature vectors of the first batch of defect sample images.

[0036] S3: The third step is to input the feature vectors of the first batch of defective sample images as training data into the labeled classification model and train it.

[0037] S4: The fourth step is to classify the first batch of unlabeled sample images of unlabeled defects in the unlabeled defect dataset using the trained feature extraction model and labeling classification model to obtain the second batch of unlabeled sample images of unlabeled defects.

[0038] S5: Fifth step, if the ratio of the number of unlabeled sample images in the second batch to the number of unlabeled sample images in the defect unlabeled dataset is greater than the preset threshold, then manually label the second batch of unlabeled sample images to obtain the second batch of defect sample images.

[0039] S6: The sixth step, following the fifth step, merges the second batch of defect sample images with the first batch of defect sample images as the defect-labeled dataset required for the next round of replacement calculation; removes the second batch of defect sample images from the defect-unlabeled dataset as the defect-unlabeled dataset required for the next round of iteration calculation; then returns to the first step to perform generational training on the feature extraction model and the labeling classification model.

[0040] S7: The seventh step, following the fourth step, if the ratio of the number of unlabeled sample images in the second batch of the defect unlabeled dataset to the number of unlabeled sample images in the first batch is less than or equal to the preset threshold, it indicates that both the feature extraction model and the labeling classification model have completed training, and the defect classification labeling ends.

[0041] Furthermore, in combination Figure 2 In the second step, one embodiment of the feature extraction model is a deep learning neural network for image processing, including a downsampling processing unit 11, an upsampling processing unit 12, and a normalization processing unit 13. The downsampling processing unit 11 includes multiple concatenated convolutional blocks. The downsampling processing unit 11 performs continuous downsampling processing on the first batch of defect sample images and outputs multiple features with different semantics to the upsampling processing unit 12. The upsampling processing unit 12 includes multiple concatenated fusion convolutional blocks. The upsampling processing unit 12 performs continuous upsampling on multiple features with different semantics and laterally connects multiple features with different semantics output from the downsampling processing unit 11. After fusion, multiple fused features are formed. The multiple fused features are then subjected to 3x3 convolution operations to form multiple feature maps. The multiple feature maps are laterally output to the normalization processing unit 13. The normalization processing unit 13 performs normalization and rule activation processing on the multiple feature maps to prevent overfitting of the feature extraction model and performs nonlinear transformation on the multiple feature maps. Finally, it outputs multiple feature vectors of the first batch of defect sample images.

[0042] Furthermore, in combination Figure 2The downsampling processing unit 11 includes a first convolutional block C1, a second convolutional block C2, a third convolutional block C3, a fourth convolutional block C4, and a fifth convolutional block C5 connected in series. The first convolutional block C1 extracts image features from the first batch of defect sample images to form a first semantic feature, and outputs the first semantic feature to the second convolutional block C2. The second convolutional block C2 downsamples the input first semantic feature to form a second semantic feature, and outputs the second semantic feature to the third convolutional block C3. Simultaneously, the second semantic feature is output to the first input of the upsampling processing unit 12 through the first output terminal of the downsampling processing unit 11. The third convolutional block C3 downsamples the second semantic feature extracted by the second convolutional block C2 to form a third semantic feature, and outputs the third semantic feature. The third semantic feature is extracted from the third convolutional block C3 and then downsampled to form the fourth semantic feature. The fourth semantic feature is then output to the second input of the upsampling processing unit 12 through the second output of the downsampling processing unit 11. The fourth convolutional block C4 downsamples the third semantic feature extracted from the third convolutional block C3 to form the fourth semantic feature and outputs the fourth semantic feature to the fifth convolutional block C5. The fourth semantic feature is then output to the third input of the upsampling processing unit 12 through the third output of the downsampling processing unit 11. The fifth convolutional block C5 downsamples the fourth semantic feature extracted from the fourth convolutional block C4 to form the fifth semantic feature and outputs the fifth semantic feature. The fifth semantic feature is then output to the fourth input of the upsampling processing unit 12 through the fourth output of the downsampling processing unit 11.

[0043] Preferably, the multiple features with different semantics output by the downsampling processing unit 11 are further processed by 1x1 convolution operations and then input to the first input, second input, third input and fourth input of the upsampling processing unit 12, respectively.

[0044] Furthermore, the upsampling processing unit 12 includes a fourth fusion convolutional block M4, a third fusion convolutional block M3, a second fusion convolutional block M2, and a first fusion convolutional block M1 connected in series. The input of the fourth fusion convolutional block M4 is connected to the fourth input of the upsampling processing unit 12. The fifth semantic feature from the downsampling processing unit 11 is output as a fourth fusion feature through the output of the fourth fusion convolutional block M4 to the third fusion convolutional block M3. Simultaneously, the fourth fusion feature is output laterally through the fourth output of the upsampling processing unit 12 to the fourth input of the normalization processing unit 13. The third fusion convolutional block M3 upsamples the fourth fusion feature and fuses it with the fourth semantic feature input from the third input of the upsampling processing unit 12 to form a third fusion feature, which is then output to the second fusion convolutional block M2. The third fused feature is horizontally output from the third output of the upsampling processing unit 12 to the third input of the standardization processing unit 13; the second fused convolutional block M2 upsamples the third fused feature and fuses it with the third semantic feature input from the second input of the upsampling processing unit 12 to form a second fused feature, and outputs the second fused feature to the first fused convolutional block M1. At the same time, the second fused feature is horizontally output from the second output of the upsampling processing unit 12 to the second input of the standardization processing unit 13; the first fused convolutional block M1 upsamples the second fused feature and fuses it with the second semantic feature input from the first input of the upsampling processing unit 12 to form a first fused feature, and horizontally outputs the first fused feature from the first output of the upsampling processing unit 12 to the first input of the standardization processing unit 13.

[0045] Furthermore, in combination Figure 2 The first fusion feature output from the first output terminal, the second fusion feature output from the second output terminal, the third fusion feature output from the third output terminal, and the fourth fusion feature output from the fourth output terminal of the upsampling processing unit 12 are respectively processed by 3x3 convolution operation to form four different feature maps, namely P1, P2, P3 and P4, which are respectively input to the first input terminal, the second input terminal, the third input terminal and the fourth input terminal of the standardization processing unit 13.

[0046] Furthermore, the standardization processing unit 13 performs normalization and rule activation processing on the four feature maps (P1, P2, P3 and P4) input by the first input terminal, the second input terminal, the third input terminal and the fourth input terminal of the standardization processing unit 13, respectively, and outputs four 1x100 feature vectors through the first output terminal, the second output terminal, the third output terminal and the fourth output terminal of the standardization processing unit 13, respectively.

[0047] Furthermore, the feature extraction model utilizes the Focal Loss class-balanced loss function during training.

[0048] Furthermore, the class-balanced loss function of Focal Loss is defined as follows: Where z represents the prediction of all sample categories, z = [z1, z2, ..., z C ] T C is the number of categories. N is the sample volume, n y p represents the number of samples in the true class y, γ is an adjustable factor, and p i This represents the probability of the predicted class by the feature extraction model.

[0049] Furthermore, in combination Figure 1 In the third step S3, the labeling classification model fits a hypersphere to each class of data in the defect-labeled dataset based on the robust least squares support vector data description. The fitted hypersphere is only related to the defect-labeled dataset of that class and is not related to the defect-labeled datasets of other classes.

[0050] Furthermore, the loss function used in robust least squares support vector data description is the smooth exponential least squares loss function.

[0051] Furthermore, let the labeled training defect dataset be T={(x1,y1),(x2,y2),...,(x m ,y m )},in For training samples, y i Let ∈{1,...,k} be the class label of the defect sample, i∈{1,...,m}. For each class of data in the trained defect-labeled dataset, the labeling classification model constructs a minimal hypersphere that encloses all or nearly all trained defect-labeled datasets for that class. The k hyperspheres can be solved by addressing the following k quadratic programming problems. The labeling classification model is as follows:

[0052] Where, ξ i C represents the error of a selected class of data in a labeled dataset. j φ(x) is the penalty factor, j∈(1,...,k). i ) is x i The mapping projected onto a high-dimensional feature space, a j For the center of each hypersphere, R j For the radius of each hypersphere, m j The number of defect data for each type. L represents the matrix consisting of samples labeled j. σ (ξ i L represents the smooth exponential least squares loss function, and its general expression is: L σ (ξ)=σ2 (1-exp(-ξ 2 / σ 2 ), where ξ represents the error and σ is the parameter of the loss function.

[0053] Preferably, L σ (ξ) is a non-convex loss function; therefore, the labeling and classification model is a non-convex optimization problem. The non-convex loss function L... σ (ξ) can be decomposed into the form of the difference between two convex functions, i.e., L σ (ξ)=L sq (ξ)-L1(ξ), where L sq (ξ)=ξ 2 L1(ξ)=ξ 2 -σ 2 (1-exp(-ξ 2 / σ 2 The above labeling and classification model can be expressed as the difference between two convex functions, and the labeling and classification model can be represented as follows:

[0054]

[0055] st.||φ(x i )-a j || 2 =R j 2 +ξ i i = 1, 2, 3, ..., m j

[0056] in, The optimal solution to the following convex quadratic programming problem can be obtained by iteratively solving it until convergence:

[0057]

[0058] in, Therefore, we can conclude that:

[0059] Where, ξ i (t) =|||φ(x) i )-a j (t) || 2 -R j 2(t) , K(x i ,x j )=φ(x i ) T φ(x j Let ) be the kernel function; therefore, the labeling classification model can be rewritten as:

[0060]

[0061] st.||φ(x i )-a j || 2 =R j 2 +ξ i i = 1, 2, 3, ..., m j ,

[0062] The constructed Lagrangian function is:

[0063]

[0064] Taking the derivative of the constructed Lagrange function with respect to the variables and setting the partial derivatives of the variables to zero, we can obtain:

[0065]

[0066]

[0067] Where α is the Lagrange multiplier, α ji Let be the Lagrange multiplier for the i-th sample in defect category j. Substituting the above formula into the Lagrange function yields:

[0068]

[0069] Solving the model yields a system of linear equations:

[0070]

[0071] in, It is of dimension m j ×m j The identity matrix, from which we can obtain:

[0072]

[0073]

[0074] Among them, I S ={x i ||α ji |≤t}, where t is the threshold and S is the value of I. S The number of elements in the middle.

[0075] K ss =[K(x1,x1),K(x2,x2),...,K(x s ,x s )]T ,

[0076] The radius R of the hypersphere obtained by solving the above formula j The center a of the hypersphere j We can obtain k hyperspheres, denoted as S. j =(R j ,a j ),j∈(1,...,k).

[0077] Furthermore, in combination Figure 3 , Figure 4 and Figure 5 For each category of hyperspheres, ideally, each hypersphere is independent of the others (e.g., ...). Figure 3 and Figure 5 As shown), however, in reality, hyperspheres may overlap. For a sample point x with an unknown label, there are three relationships between the sample point and the hypersphere. The first is as follows: Figure 3 As shown, the sample point x lies within a certain hypersphere; the second type is as follows... Figure 4 As shown, the sample point x is located in the overlapping part of multiple hyperspheres; the third type is as follows Figure 5 As shown, the sample point x lies outside all hyperspheres.

[0078] In the first case, the sample point x belongs to the class determined by which hypersphere it lies within. In the second case, a radius normalization method is used to resolve class overlap. This method normalizes the relative distance by dividing the distance from the sample to the center of each hypersphere by the radius of that hypersphere. Its discriminant function is as follows: Where D(x,S) j ),j∈(1,...,l) is the distance from the sample point x to the center of each hypersphere, and l is the hypersphere containing the sample point; for the third case, the sample point x is an outlier. For such outliers, the labeling classification model labels them as unknown categories and leaves them to be labeled manually.

[0079] Specifically, for the third case of the above classification, where sample point x is an outlier, it is labeled as an unknown category. That is, when sample point x is outside all hyperspheres, it does not belong to any hypersphere. This means that when constructing the hyperspheres for each dataset, this sample point is always outside the boundaries of the hyperspheres, making it an outlier and unable to be assigned a definite category. Therefore, for such sample points, they are treated as samples requiring manual labeling. These sample points are displayed on the interface, and then labeled manually using ground truth values.

[0080] Specifically, in combination Figure 1In step S6, the manually annotated sample images are expanded into the defect-annotated dataset, and the feature extraction model and annotation classification model are iteratively trained to solve the problem of insufficient samples in the defect-annotated dataset of industrial products.

[0081] Furthermore, to verify the annotation capability of the present invention, two-quarters of the sample points in the KolektorSDD metal surface defect dataset (which comes from the Visual Perception Systems Laboratory of the University of Ljubljana and Kolektor Corporation) were randomly selected as the training dataset with labeled defects to train the feature extraction model and the annotation classification model. Then, one-quarter of the sample points were selected as the dataset with unlabeled defects, and the last quarter was used as the test dataset. In the dataset with labeled defects, 5% of the sample points were randomly selected and their labels were changed to simulate outliers.

[0082] Comparative experiment: The effectiveness of the labeling classification model is verified by comparing the experimental results of the supervised algorithm using only the defect-labeled dataset with the experimental results of the method of this invention.

[0083] For experiments on the KolektorSDD metal surface defect dataset, F1 and accuracy were used to evaluate the classification performance of the annotation classification model. To further explain, TP is a true negative example, meaning that positive sample data is correctly labeled as positive sample data; TN is a true negative example, meaning that negative sample data is correctly labeled as negative sample data; FN is a false negative example, meaning that positive sample data is incorrectly labeled as negative sample data; and FP is a false positive example, meaning that negative sample data is incorrectly labeled as positive sample data.

[0084] Furthermore, the experimental process of the supervised algorithm using only the defect-labeled dataset is as follows: the defect-labeled training dataset is used as input to train the feature extraction model, and then the image features extracted by the feature extraction model are used as input to train the labeling classification model; the test dataset is input to the trained feature extraction model to obtain the features of the input image, and then the extracted image features are used as input to the labeling classification model to obtain the experimental results of the supervised algorithm using only the labeled dataset.

[0085] The annotation process based on this invention is as follows: Figure 1 As shown, Figure 6 and Figure 7 To utilize all feature maps P1, P2, P3, and P4 obtained from the feature extraction model, Figure 6 The defective image A1 and all feature maps P11, P21, P31 and P41 obtained by the feature extraction model are given. Figure 7For the defect-free image A2 and all feature maps P12, P22, P32, and P42 obtained from the feature extraction model, these are the feature maps obtained from the defect-free image A2. Figure 6 and Figure 7 As can be seen from the data, for a defective image A1 and an undefective image A2, the feature extraction model can extract the feature maps of the defective part of the defective image and the feature maps of the undefective image. This shows that the feature extraction model proposed in this invention can extract features of imbalanced datasets better.

[0086] Table 1 Comparison of experimental results between the present invention and existing technologies

[0087]

[0088] Table 1 shows the experimental results of the two different methods. The results show that after expanding the manually labeled unlabeled defect dataset with the labeled defect dataset using the present invention, the accuracy (0.9800) and F1 (0.9886) of the feature extraction model and the labeling classification model are significantly improved compared with the accuracy (0.9300 and 0.9595) before the data expansion. This proves that the model can fully utilize the information on the unlabeled defect dataset, has good labeling performance, and achieves accurate labeling of surface defects of industrial products.

[0089] In summary, this invention provides a data annotation method for detecting surface defects in industrial products. It trains a feature extraction model and an annotation classification model using a pre-labeled defect dataset. The trained feature extraction model extracts feature vectors from the unlabeled defect dataset. These feature vectors are then input into the trained annotation classification model to select sample images requiring manual annotation. The manually annotated sample images are then expanded into the pre-labeled defect dataset and removed from the unlabeled dataset. The method is then evaluated to determine if the feature extraction and annotation models meet the requirements. If they do, the defect classification and annotation process ends; otherwise, the training is repeated. This invention employs a Focal Loss class-balanced loss function to train the feature extraction model, which better extracts features from imbalanced datasets. Furthermore, this invention proposes an annotation classification model based on a smooth exponential least squares loss function, effectively addressing the issue of outliers in industrial product surface defect datasets. Through active learning, it solves the problem of insufficient labeled defect samples in the defect dataset, fully utilizing the information from both labeled and unlabeled samples to achieve accurate annotation of surface defects in industrial products. This method not only reduces annotation costs but also improves annotation efficiency and quality.

[0090] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A data annotation method for detecting surface defects in industrial products, characterized in that, Including the following steps: The first step is to obtain a defect-annotated dataset for labeling surface defects of industrial products, which includes the first batch of defect sample images with labeled defects. The second step is to train the feature extraction model using the labeled defect dataset to extract the feature vectors of the first batch of defect sample images. The third step is to use the feature vectors of the first batch of defective sample images as training data to input into the labeled classification model and train it. The fourth step is to classify the first batch of unlabeled sample images of unlabeled defects in the unlabeled defect dataset using the trained feature extraction model and the labeling classification model to obtain the second batch of unlabeled sample images of unlabeled defects. Fifth step: If the ratio of the number of unlabeled sample images in the second batch of the defect unlabeled dataset to the number of unlabeled sample images in the first batch is greater than a preset threshold, then the second batch of unlabeled sample images is manually labeled to obtain the second batch of defect sample images. The sixth step, following the fifth step, is to merge the second batch of defect sample images with the first batch of defect sample images to form the defect-labeled dataset required for the next round of replacement calculation; Remove the second batch of defect sample images from the unlabeled defect dataset and use it as the unlabeled defect dataset for the next round of iteration; then return to the first step and perform generational training on the feature extraction model and the labeling classification model. Step 7, following step 4, if the ratio of the number of unlabeled sample images in the second batch of the defect unlabeled dataset to the number of unlabeled sample images in the first batch is less than or equal to a preset threshold, it indicates that both the feature extraction model and the labeling classification model have completed training, and the defect classification labeling ends. The feature extraction model includes a downsampling processing unit, an upsampling processing unit, and a normalization processing unit. The downsampling unit includes multiple concatenated convolutional blocks. The downsampling processing unit performs continuous downsampling processing on the first batch of defect sample images and outputs multiple features with different semantics to the upsampling processing unit. The labeled classification model is based on robust least squares support vector data description. It fits a hypersphere to each class of data in the labeled defect dataset. The fitted hypersphere is only related to the labeled defect dataset of that class and is not related to the labeled defect datasets of other classes.

2. The data annotation method for surface defect detection of industrial products according to claim 1, characterized in that, The upsampling processing unit includes multiple sequentially connected fusion convolutional blocks. The upsampling processing unit continuously upsamples multiple features with different semantics and laterally connects multiple features with different semantics output from the downsampling processing unit. After fusion, multiple fusion features are formed. The multiple fusion features are then subjected to 3x3 convolution operations to form multiple feature maps. The multiple feature maps are laterally output to the standardization processing unit. The standardization processing unit normalizes and activates the multiple feature maps to prevent the feature extraction model from overfitting, performs nonlinear transformations on the multiple feature maps, and finally outputs multiple feature vectors of the first batch of defect sample images.

3. The data annotation method for detecting surface defects in industrial products according to claim 2, characterized in that, The downsampling processing unit includes a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, and a fifth convolutional block connected in series. The first convolutional block extracts image features from the first batch of defect sample images to form a first semantic feature and outputs the first semantic feature to the second convolutional block. The second convolutional block downsamples the input first semantic feature to form a second semantic feature and outputs the second semantic feature to the third convolutional block. Simultaneously, the second semantic feature is output to the first input of the upsampling processing unit through the first output terminal of the downsampling processing unit. The third convolutional block downsamples the second semantic feature extracted by the second convolutional block to form a third semantic feature and outputs the third semantic feature. The third semantic feature is extracted from the third convolutional block and then downsampled to form the fourth semantic feature. The fourth convolutional block outputs the third semantic feature extracted from the third convolutional block to the second input of the upsampling processing unit. The fourth convolutional block downsamples the third semantic feature extracted from the third convolutional block to form the fourth semantic feature and outputs the fourth semantic feature to the fifth convolutional block. The fifth convolutional block downsamples the fourth semantic feature extracted from the fourth convolutional block to form the fifth semantic feature and outputs the fifth semantic feature. The fifth semantic feature outputs the fifth semantic feature to the fourth input of the upsampling processing unit.

4. The data annotation method for surface defect detection of industrial products according to claim 3, characterized in that, The upsampling processing unit includes a fourth fusion convolutional block, a third fusion convolutional block, a second fusion convolutional block, and a first fusion convolutional block connected in series. The input of the fourth fusion convolutional block is connected to the fourth input of the upsampling processing unit. The fifth semantic feature from the downsampling processing unit is output as a fourth fusion feature to the third fusion convolutional block through the output of the fourth fusion convolutional block. At the same time, the fourth fusion feature is output laterally to the fourth input of the normalization processing unit through the fourth output of the upsampling processing unit. The third fusion convolutional block upsamples the fourth fusion feature and fuses the fourth semantic feature input from the third input of the upsampling processing unit to form the third fusion feature. The third fusion feature is then output to the second fusion convolutional block. Simultaneously, the third fusion feature is output laterally to the third input of the standardization processing unit through the third output of the upsampling processing unit. The second fusion convolutional block upsamples the third fusion feature and fuses the third semantic feature input from the second input of the upsampling processing unit to form the second fusion feature. The second fusion feature is then output to the first fusion convolutional block. Simultaneously, the second fusion feature is output laterally to the second input of the standardization processing unit through the second output of the upsampling processing unit. The first fusion convolutional block upsamples the second fusion feature and fuses the second semantic feature input from the first input of the upsampling processing unit to form the first fusion feature. The first fusion feature is then horizontally output to the first input of the standardization processing unit through the first output of the upsampling processing unit. The four fused features output by the upsampling processing unit are respectively processed by 3x3 convolution to form four different feature maps, which are respectively input to the first input terminal, the second input terminal, the third input terminal and the fourth input terminal of the normalization processing unit.

5. The data annotation method for detecting surface defects in industrial products according to claim 4, characterized in that, The standardization processing unit performs normalization and activation processing on the feature maps input from its four input terminals, and outputs four 1x100 feature vectors through the first, second, third and fourth output terminals of the standardization processing unit.

6. The data annotation method for surface defect detection of industrial products according to claim 5, characterized in that, The feature extraction model is trained using the Focal Loss class-balanced loss function, which is defined as follows: Where z represents the predictions for all categories, C is the number of categories ， , N For sample volume, This represents the number of samples in the true class y. It is an adjustable factor. This represents the probability of the predicted category by the feature extraction model.

7. The data annotation method for surface defect detection of industrial products according to claim 1, characterized in that, The loss function used in the robust least squares support vector data description is the smooth exponential least squares loss function.

8. The data annotation method for surface defect detection of industrial products according to claim 7, characterized in that, Let the labeled dataset for the aforementioned defects be denoted as . ,in These are the first batch of defect sample images. For the category label of the defective sample, ; The labeled classification model constructs a minimal hypersphere that encloses all or nearly all labeled training defect datasets for each class of data in the training defect labeled dataset. k Each hypersphere is solved by solving k quadratic programming problems.

9. The data annotation method for surface defect detection of industrial products according to claim 8, characterized in that, The general expression for the smooth exponential least squares loss function is: ,in, Indicates error. These are the parameters of the loss function.

Images