A method for detecting defects in package images based on contrastive learning method

Abstract

A method for detecting defects in cheese winder images based on the contrast learning method, which relates to the technical field of cheese winder quality detection, includes the following steps: Step 1, collection, processing and segmentation of cheese winder images; Step 2, obtaining the row / column vector feature representation of cheese winder images based on the BERT algorithm; Step 3, a row / column image feature extraction method based on contrast learning, using the relationship between each row / column image and its surrounding images for feature learning; Step 4, positioning the defects in cheese winder images based on the clustering method. In order to reduce the complexity of data and improve the accuracy of the algorithm, the present invention converts the two-dimensional image into multiple one-dimensional vectors, integrates the method based on BERT and the method of contrast learning, thereby improving the efficiency of the algorithm. At the same time, the probability of whether there are defects in the corresponding one-dimensional vector is learned and predicted through support vector machine learning, and the defect area in the image is accurately determined by improving the DBSCAN clustering algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of bobbin quality detection, and in particular to a method for detecting bobbin image defects based on a contrast learning method. Background Art

[0002] The quality of ring-spun yarn bobbins has an important impact on the working efficiency of various production processes in subsequent spinning and the quality of the final fabric. However, due to a series of reasons such as many production process links and uncontrollable production conditions in the production process of cheese yarns, the quality of the finally produced cheese yarns is uneven, and there are appearance defects such as poor forming, bruising, hairiness, hairiness, and contamination, which are important aspects for measuring the quality of cheese yarns. Relying solely on manual inspection, subject to a series of problems such as the instability of manual work and the non-uniqueness of the evaluation criteria, all affect the judgment of the final quality of the product. With the development of computer systems, detecting the quality of bobbins through computer vision is a relatively faster means, which is beneficial to detecting problematic bobbins in advance. With the development of machine learning methods in recent years, using machine learning algorithms to detect the quality of bobbins is a more novel and effective detection method.

[0003] Currently, many institutions at home and abroad have also conducted research on fabric defect detection methods based on machine learning or deep learning. Zhang et al. proposed a method for multi-window gray-scale contrast to detect normal fabrics. In this method, the gray-scale image presents alternating gray and white patterns, and thresholds are used to segment the foreground and background regions of the image, and the image is segmented into several regions, and the defects in the image are determined by analyzing the change of the gray-scale ratio. Mei et al. proposed an automated unsupervised deep network method to detect and locate fabric defects. Wei et al. proposed a Faster RCNN based on the VGG network structure. Among them, the Region Proposal Network (RPN) is used to generate bounding boxes to frame possible defects, and these regions are passed to the ROI pooling layer, and these regions are classified through the softmax layer. At the same time, vertical and horizontal flips are used to expand the fabric dataset to detect broken weft, felting, defects, foreign objects, broken ends and oil stains of the fabric. For tiny defects, Jeyaraj et al. proposed a convolutional neural network based on ResNet512 to learn features from images. Jin et al. also proposed a Unet fabric defect detection method based on CNN, and achieved pixel-level classification of defects, and is suitable for real-time defect detection.

[0004] Although the above detection methods have made certain technological progress, they still have the following disadvantages:

[0005] 1. Although deep learning methods for fabrics have been developed to a certain extent, currently, there are few methods and applications for detecting defects in cheese yarn images.

[0006] 2. The algorithm accuracy is relatively low. For a large range of winding images, it is impossible to ensure that all defect images can be detected.

[0007] 3. The algorithm precision is not high. Although traditional machine learning algorithms or deep learning algorithms can be used to locate defects, the positioning accuracy is not high, and the position of the defect cannot be accurately found. At the same time, the overall defect situation cannot be evaluated.

[0008] 4. The algorithm efficiency is relatively low. Traditional machine learning algorithms are inefficient and cannot meet the requirements of real-time detection. Summary of the Invention

[0009] The present invention provides a method for detecting defects in winding images based on a contrast learning method. In order to ensure the complete detection of the cheese images, the image data in six directions of the cheese are acquired simultaneously to ensure the integrity of information. In order to reduce the complexity of the data and improve the algorithm accuracy, the two-dimensional image is converted into multiple one-dimensional vectors, and the method based on BERT and the contrast learning method are fused, thereby improving the algorithm efficiency. At the same time, the probability of whether there are defects in the corresponding one-dimensional vectors is learned and predicted through support vector machine learning, and the improved DBSCAN clustering algorithm is used to accurately determine the defect area in the image.

[0010] To achieve the above object, the technical solution of the present invention is as follows:

[0011] A method for detecting defects in winding images based on a contrast learning method, comprising the following steps:

[0012] Step 1, collection, processing and segmentation of cheese images;

[0013] Step 2, obtaining the row / column vector feature representation of the cheese image based on the BERT algorithm;

[0014] Step 3, a row / column image feature extraction method based on contrast learning, using the relationship between each row / column image and its surrounding images for feature learning;

[0015] Step 4, locating the defects in the cheese image based on a clustering method.

[0016] Preferably, the step 1 includes the following specific steps:

[0017] (1A) Collect images of the cheese from six directions: top, bottom, left, right, front, and back of the cheese;

[0018] (2A) Process each image separately. During the processing, assume the image is represented as I(x, y), where x and y are the horizontal and vertical coordinates of the image, and the values are the values of the red, green, and blue channels;

[0019] (3A) Extract each row and column of the image: For each row of pixels in the image, it is represented by I r (y), where y is the ordinate, and I r (y) corresponds to all the pixels in the y-th row of the image. Correspondingly, for each column of pixels in the processed image, it is represented by I c (x), where x is the abscissa, and I c (x) corresponds to all the pixels in the x-th column of the image.

[0020] Preferably, step 2 includes the following specific steps:

[0021] (1B) Adopt a self-supervised learning-based method to extract the row / column vector features of the cheese image;

[0022] (2B) Adopt a method analogous to natural language processing. Process each row vector of the cheese image in sequence: First, for each pixel, its RGB value becomes the input vector E i through linear transformation. In addition, for the positions of different pixels, the positions are encoded to obtain the position encoding E p of each pixel. The sum of the two vectors gives the input vector of each pixel input to the model, that is, E(x, y) = E i (x, y) + E p (x, y);

[0023] (3B) Adopt the BERT algorithm analogous to natural language processing. For the feature vectors of all the pixels in each row, 15% of them are selected for masking processing, where 50% of the vectors are randomly replaced with other vectors, and 50% of the vectors adopt the specified color (0, 0, 128), and the encoded vectors are marked;

[0024] (4B) Adopt an encoder-decoder-based method to predict the masked content and the replaced content: Process all the row / column data through the encoder to obtain its feature vector, and input this feature vector into the decoder. The decoder reconstructs all the input vectors, and through masking processing, improves the fault tolerance of the encoder and decoder, enhances the robustness of the feature vector, and takes the intermediate output vector b x as the representation of each row.

[0025] Preferably, in step 2, the same processing method is adopted for the column vectors.

[0026] Preferably, step 3 includes the following specific steps:

[0027] (1C) Construct positive and negative samples: For each row of images, the adjacent upper and lower rows of images are paired with this image as positive sample pairs; For each row of images, its pixel points are data-augmented in a randomly scrambled form, and the augmented data and the original row are used as positive sample pairs together; Negative sample pairs are formed by randomly selecting data from other rows and combining them with the current row;

[0028] (2C) For each row of data I c (x), its upper and lower rows are respectively denoted as I c (x - 1) and I c (x + 1), the randomly scrambled row is denoted as I c ′(x), and other randomly selected rows are denoted as I c (z); A one-dimensional CNN model is used to process each row of data, and the feature representation after convolution of each row is denoted as d x =CNN(I c (x));

[0029] (3C) For all positive and negative sample pairs, a CNN loss function based on contrastive learning is adopted:

[0030]

[0031] In formula (1), τ is a normalization factor, and M(d x , d y ) is the similarity between two vectors; For a positive sample pair I c (x) and I c (y), the higher the similarity, the better; For all negative sample pairs, the lower the similarity, the better; N is the number of pictures in the training set; A more robust CNN model is trained according to the loss function, and feature extraction is performed on each row of data;

[0032] (4C) For each row of data x, the feature vectors output by the two models are concatenated [b x , d x to obtain the final vector representation; For each column of data, the same processing method is also adopted to obtain the final vector representation.

[0033] Preferably, step 4 includes the following specific steps:

[0034] (1D) After obtaining the features of each row and column, by using the support vector machine method, according to the existing training data, predict the probability that the current row and column are defective points. After the prediction is completed, obtain the probability that the horizontal and vertical coordinates of each pixel in the image are defective points;

[0035] ] (2D) Cluster the existing defective pixels based on the DBSCAN algorithm;

[0036] (3D) After completing the clustering algorithm, count the number of clusters in each image as the number of defects present in this image.

[0037] Preferably, the specific step (2D) includes the following steps:

[0038] (21D) The DBSCAN algorithm includes 2 parameters, namely the neighborhood distance threshold ∈ of a certain sample and the threshold MinPts of the number of samples in the neighborhood of a certain sample with a distance of ∈; define the number of samples in the neighborhood of ∈ as:

[0039] N ∈ (x i ) = {x j ∈D|dist(x i , x j ) ≤ ∈} (2)

[0040] In formula (2), D is the set of all data, dist(x i , x j ) is the distance between two pixel points, and ∈ is the neighborhood distance threshold;

[0041] (22D) Select an unvisited point from the dataset as the initial point x i , and judge the number N ε (x i ) of the ∈ neighborhood of this initial point. If it is greater than MinPts, the clustering starts, and the current point x i is used as the core object;

[0042] (23D) The probability that the horizontal and vertical coordinates of each pixel output by the support vector machine are defects is P x (x i ), P y (x i ). Then, for a certain coordinate (x, y), the probability that it is a defect is:

[0043]

[0044] (24D) Define the standard for judging whether a point is extended as a core object as:

[0045]

[0046] In formula (4), MinPts is the threshold of the number of samples;

[0047] Through formula (4), calculate the sum of the sample points whose predicted probabilities in the ∈ neighborhood of x i are greater than 0.5. If the sum of these points can be greater than the threshold MinPts, then the current point x iExpand;

[0048] (25D) For x i All the points within its distance become part of the same cluster. If the points within the neighborhood meet the conditions of formula (4), they become new core objects and absorb the points within their distance as part of the same cluster. This process is repeated continuously until all the points near the cluster have been visited.

[0049] Advantages of a method for detecting yarn clearing image defects based on contrast learning method of the present invention:

[0050] 1. In the image extraction of the present invention, the row and column images are processed separately, thereby reducing the complexity of the data and facilitating the accurate positioning of the defect location.

[0051] 2. The present invention uses self-supervised learning to extract features from the row and column images, deeply mines the correlation relationship between data, and integrates the method based on BERT and the contrast learning method to ensure the accurate expression of information.

[0052] 3. The present invention adapts to the clustering problem of sample points with weights by improving the DBSCAN clustering algorithm, thereby solving the problem of locating defective images.

[0053] 4. In order to ensure the complete detection of the cheese image, the present invention simultaneously acquires the image data of six directions of the cheese to ensure the integrity of information. In order to reduce the complexity of the data and improve the accuracy of the algorithm, the two-dimensional image is converted into multiple one-dimensional vectors, and the method based on BERT and the contrast learning method are integrated, thereby improving the efficiency of the algorithm. At the same time, the probability of whether there are defects in the corresponding one-dimensional vectors is learned and predicted through support vector machine, and the defective area in the image is accurately determined by improving the DBSCAN clustering algorithm. Description of the Drawings

[0054] Figure 1 、Flowchart of the algorithm of the present invention;

[0055] Figure 2 、Row vector features of cheese images based on BERT;

[0056] Figure 3 、Method for constructing positive and negative samples;

[0057] Figure 4 、Method for calculating point probability. Detailed Embodiments

[0058] As described below, the embodiments of the present invention will be described in detail in a step-by-step manner. This description is only for the preferred embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0059] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, as well as a specific orientation structure and operation. Therefore, it should not be construed as a limitation of the present invention.

[0060] In the initial embodiment, a method for detecting defects in cheese images based on the contrast learning method of the present invention, as Figures 1-4 shown, includes the following steps:

[0061] Step 1: Collection, processing, and segmentation of cheese images;

[0062] Step 2: Obtaining row / column vector feature representations of cheese images based on the BERT algorithm;

[0063] Step 3: A row / column image feature extraction method based on contrast learning, using the relationship between each row / column image and its surrounding images for feature learning;

[0064] Step 4: Locating defects in cheese images based on the clustering method.

[0065] In a further embodiment, as Figure 1 shown, the specific steps of the above-mentioned Step 1 include the following:

[0066] (1A) Collect images of the cheese from six directions: upper, lower, left, right, front, and back;

[0067] (2A) Process each image separately. During the processing, assume the image is represented as I(x, y), where x and y are the horizontal and vertical coordinates of the image, and the values are the values of the red, green, and blue channels;

[0068] (3A) Extract each row and column of the image: For each row of pixels in the image, it is represented by I r (y), where y is the vertical coordinate, and I r (y) corresponds to all the pixels in the y-th row of the image. For the corresponding processing of each column of pixels in the image, it is represented by I c (x), where x is the horizontal coordinate, and I c (x) corresponds to all the pixels in the x-th column of the image.

[0069] In a further embodiment, as Figure 1 、 2 shown, step 2 described above includes the following specific steps:

[0070] (1B) Use a self-supervised learning-based method to extract row / column vector features of the cheese image; taking each row of pixels as an example, I c (x) = {i x，1 , i x，2 ,..., i x，y ,..., i x，N}, where i x，y is the y-th pixel in the x-th row, represented by three values of red, green, and blue;

[0071] (2B) Adopt a method analogous to natural language processing, and process each row vector of the cheese image in sequence: First, for each pixel, its RGB value is linearly transformed into an input vector E i . In addition, for the positions of different pixels, the positions are encoded to obtain the position encoding E p of each pixel. The sum of the two vectors gives the input vector of each pixel input to the model, that is, E(x, y) = E i (x, y) + E p (x, y);

[0072] (3B) Adopt the BERT algorithm analogous to natural language processing. For the feature vectors of all pixels in each row, 15% of them are selected for masking processing, as Figure 2 shown, where 50% of the vectors are randomly replaced with other vectors, and 50% of the vectors adopt the specified color (0, 0, 128), and the encoded vectors are marked;

[0073] (4B) Use an encoder-decoder approach to predict the masked content and the replaced content: Process all the data row / column data through the encoder to obtain its feature vector, and input this feature vector into the decoder. The decoder reconstructs all the input vectors, and through masking processing, improves the fault tolerance of the encoder and decoder, enhances the robustness of the feature vector, and uses the intermediate output vector b x as the representation of each row.

[0074] In a further embodiment, as Figure 1 、 2 shown, in step 2 described above, the same processing method is adopted for the column vector.

[0075] In a further embodiment, as Figure 1 、 3As shown, step 3 includes the following specific steps:

[0076] (1C) Construct positive and negative samples: For each row of images, the adjacent upper and lower rows of images are paired with this image as positive samples; For each row of images, its pixel points are data-augmented in a randomly scrambled form, and the augmented data and the original row are used as positive sample pairs together; Negative sample pairs are formed by randomly selecting data from other rows and combining them with the current row; The method of constructing positive and negative samples is as Figure 3 shown;

[0077] (2C) Since the images are of fixed length and size, for each row of data I c (x), its upper and lower rows are respectively denoted as I c (x - 1) and I c (x + 1), the randomly scrambled row is denoted as I c ′(x), and other randomly selected rows are denoted as I c (z); A one-dimensional CNN model is used to process each row of data, and the feature representation after convolution for each row is denoted as d x = CNN(I c (x));

[0078] (3C) For all positive and negative sample pairs, a CNN loss function based on contrastive learning is adopted:

[0079]

[0080] In formula (1), τ is a normalization factor, M(d x , d y ) is the similarity between two vectors. For a positive sample pair I c (x) and I c (y), the higher the similarity, the better; For all negative sample pairs, the lower the similarity, the better; N is the number of pictures in the training set; A more robust CNN model is trained according to the loss function to extract features for each row of data;

[0081] (4C) For each row of data x, the feature vectors output by the two models are concatenated [b x , d x to obtain the final vector representation; For each column of data, the same processing method is also adopted to obtain the final vector representation.

[0082] In a further embodiment, as Figure 1 , 3 shown, step 4 includes the following specific steps:

[0083] (1D)After obtaining the features of each row and column, by using the support vector machine method and based on the existing training data, predict the probability that the current row and column are defective. After the prediction is completed, obtain the probability that the horizontal and vertical coordinates of each pixel in the image are defective;

[0084] (2D)However, the simple pixel-level prediction cannot accurately locate the overall position of the image defects, nor can it accurately count the number of defects in the image. Therefore, cluster the existing defective pixels based on the DBSCAN algorithm;

[0085] (3D)After completing the clustering algorithm, count the number of clusters in each image as the number of defects existing in this image.

[0086] In a further embodiment, as Figure 1 、 3 、shown in 4, since the traditional DBSCAN algorithm fails to consider the weight problem of each point to be clustered, therefore, the present invention introduces probability into the DBSCAN algorithm, which is reflected in:

[0087] The specific step (2D) described includes the following steps:

[0088] (21D)The DBSCAN algorithm includes 2 parameters, namely the neighborhood distance threshold ∈ of a certain sample and the threshold MinPts of the number of samples in the neighborhood with a distance of ∈ for a certain sample; define the number of samples in the neighborhood of ∈ as:

[0089] N ∈ (x i )=(x j ∈D|dist(x i ,x j )≤∈} (2)

[0090] In formula (2), D is the set of all data, dist(x i ,x j ) is the distance between two pixel points, and ∈ is the neighborhood distance threshold;

[0091] (22D)Select an unvisited point from the data set as the initial point x i , judge the number N ε (x i ) of the ∈ neighborhood of this initial point. If it is greater than MinPts, the clustering starts, and the current point x i is used as the core object;

[0092] (23D)The probability that the horizontal and vertical coordinates of each pixel output by the support vector machine are defective is P x (x i ), P y (xi ), then for a certain coordinate (x, y), the probability that it is a defect is:

[0093]

[0094] (24D) Define the standard for determining whether a point is a core object for expansion as:

[0095]

[0096] In formula (4), MinPts is the threshold of the number of samples;

[0097] Through formula (4), calculate the sum of the sample points whose predicted probabilities in the ∈-neighborhood of x i are greater than 0.5. If the sum of these points can be greater than the threshold MinPts, then expand the current point x i ;

[0098] (25D) For x i The points within its distance all become part of the same cluster. If the points within the neighborhood meet the conditions of formula (4), they become new core objects and absorb the points within their distance as part of the same cluster. Continuously repeat this process until all the points near the cluster have been visited.

Claims

1. A method for detecting defects in cop winding images based on the contrastive learning method, characterized by: It includes the following steps: Step 1: Collection, processing, and segmentation of cheese images; Step 2: Obtaining the row vector and column vector feature representations of cheese images based on the BERT algorithm; Step 3: A method for extracting row vector and column vector image features based on contrastive learning, using the relationship between each row vector and column vector image and its surrounding images for feature learning; Step 4: Locating cheese image defects based on a clustering method; The specific steps of Step 2 include the following: (1B) Adopt a method based on self-supervised learning to extract row / column vector features of cheese images; (2B) Using an analogy to natural language processing, each row vector of the bobbin image is processed sequentially: First, for each pixel, its RGB value is linearly transformed into the input vector ,In addition, for the positions of different pixels, the positions are encoded to obtain the position encoding of each pixel , the two vectors are added together to get the input vector of each pixel of the input model, that is, = ; (3B) Use the BERT algorithm analogous to natural language processing. For the feature vectors of all pixels in each row, 15% of them are selected for masking processing, where 50% of the vectors are randomly replaced with other vectors, and 50% of the vectors use the specified color (0, 0, 128), and the encoded vectors are marked; (4B) Adopt an encoder-decoder method to predict the masked content and the replaced content: Process all data row / column data through the encoder and obtain their feature vectors, And input the feature vector into the decoder. All input vectors are reconstructed by the decoder, and through masking processing, the fault tolerance of the encoder and decoder is improved, the robustness of the feature vector is enhanced, and the intermediate output vector is used as the representation of each row; In Step 2, the same processing method is adopted for column vectors; The specific steps of Step 3 include the following: (1C) Construct positive and negative samples: For each row of images, the adjacent upper and lower rows of images are used as positive sample pairs with this image; For each row of images, its pixel points are data-augmented in a randomly scrambled form, and the augmented data and the original row are used as positive sample pairs together; Randomly select data from other rows and combine them with the current row to form negative sample pairs; (2C)For each row of data , the upper and lower rows are respectively represented as and , the randomly scrambled row is represented as , and the other random rows are represented as ; A one-dimensional CNN model is used to process each row of data, and the feature representation after convolution of each row is: ; (3C) For all positive and negative sample pairs, adopt a CNN loss function based on contrastive learning: （1） In formula (1), is the similarity between two vectors; for a positive sample pair and , the higher the similarity, the better; for all negative sample pairs, the lower the similarity, the better; N is the number of pictures in the training set; a CNN model with strong robustness is trained according to the loss function, and feature extraction is performed on each row of data; (4C) For the data of each row x, concatenate the feature vectors output by the two models to obtain the final vector representation; for the data of each column, use the same processing method to obtain the final vector representation.

2. The yarn winding image defect detection method based on the contrast learning method according to claim 1, characterized in that: The specific steps of Step 1 include the following: (1A) Collect images of the cheese from six directions: top, bottom, left, right, front, and back; Process each image separately. During the processing, assume the image is represented as , where are the horizontal and vertical coordinates of the image respectively, and the values are the values of the red, green, and blue channels. (3A) Extract each row and column of the image: For each row of pixels in the image, use to represent, where y is the ordinate, corresponds to all pixels in the th row in the image. For the corresponding processing of each column of pixels in the image, use to represent, where x is the abscissa, corresponds to all pixels in the th column in the image.

3. The method for detecting yarn-winding image defects based on the contrastive learning method according to claim 2, characterized in that: The specific steps of Step 4 include the following: (1D) After obtaining the features of each row and column, use the support vector machine method to predict the probability that the current row and column are defects based on the existing training data. After the prediction is completed, obtain the probability that the horizontal and vertical coordinates of each pixel in the image are defects; (2D) Cluster the existing defect pixels based on the DBSCAN algorithm; (3D) After completing the clustering algorithm, count the number of clusters in each image as the number of defects in this image.

4. The yarn-winding image defect detection method based on the contrast learning method according to claim 3, characterized in that: The specific steps of (2D) include the following steps: The (21D) DBSCAN algorithm includes two parameters, namely the neighborhood distance threshold of a certain sample and the threshold MinPts of the number of samples in the neighborhood where the distance of a certain sample is ; define the number of samples in the neighborhood where is: （2） In formula (2), D is the set of all data, is the distance between two pixel points, is the neighborhood distance threshold; (22D) Select an unvisited point from the dataset as the initial point and determine the number of neighborhoods of this initial point. If it is greater than MinPts, clustering begins, and the current point is used as the core object; The probability that the horizontal and vertical coordinates of each pixel output by the (23D) support vector machine are defect points is , then for a certain coordinate , its probability of being a defect point is: （3） (24D) Define the standard for determining whether a point is a core object for expansion as: （4） In formula (4), MinPts is the threshold of the number of samples; Through formula (4), calculate for of the sum of the sample points in the neighborhood where all predicted probabilities are greater than 0.

5. If the sum of these points can be greater than the threshold , then expand the current point ; (25D) For points within its distance all become part of the same cluster. If the points within the neighborhood satisfy the conditions of formula (4), they become new core objects and absorb the points within their distance as part of the same cluster. This process is repeated continuously until all points near the cluster have been visited.

Images