A yarn allowance detection method based on an improved UNet model

Abstract

The application relates to a yarn surplus detection method based on an improved UNet model, which comprises the following steps: collecting a yarn drum image by using an image acquisition device; expanding a data set by using YOLOV8 and performing semantic labeling on a segmentation region; inputting into an improved UNet model for training to obtain a trained model weight; obtaining a segmentation result, using a traditional image processing method to obtain a contour edge point set and performing contour fitting; and finally, according to the relationship between a pixel size and an actual size, calculating the actual surplus amount of the yarn drum. The application uses the first 13 convolutional layers of VGG16 to replace the encoder part of the UNet network, VGG16 has deep feature extraction capability, can learn complex and abstract feature representation, is helpful for the model to better extract semantic information, can effectively detect the yarn surplus on the yarn drum, improves the segmentation and detection precision, and has good production practicability.

Description

[Technical Field]

[0001] This invention relates to a method for detecting yarn excess, specifically a method for detecting yarn excess based on an improved UNet model, belonging to the field of target detection and image segmentation technology in the textile industry. [Background Technology]

[0002] In the textile production process, yarn, as a key raw material, directly affects the quality of the final product. Currently, yarn weight measurement mainly relies on manual inspection and traditional vision technology. These methods often suffer from low accuracy, weak anti-interference capabilities, and limited generalization ability, making it difficult to meet the high standards required by modern textile production. With the advancement of digital image processing technology, machine vision is being applied more and more widely in the textile field.

[0003] Because yarn bobbins are placed on rows of yarn racks in the factory, with their ends facing the aisles, cameras can only take pictures from the ends of the bobbins during inspections. The photos taken only show the sides of the bobbins, making it difficult to calculate their length. This makes calculating yarn allowance quite challenging. Furthermore, the shapes and sizes of some bobbins are inconsistent, leading to different standards for allowance measurement. In recent years, object detection and semantic segmentation have received considerable attention in the field of machine vision and have been widely adopted by many manufacturers for production line inspection and recognition. Many excellent neural networks, such as CNN, RCNN, VGG, and UNet, have been used in these applications.

[0004] Therefore, in order to solve the above problems, it is indeed necessary to provide an innovative yarn allowance detection method based on an improved UNet model to overcome the defects in the prior art. [Summary of the Invention]

[0005] To address the aforementioned problems, the present invention aims to provide a yarn allowance detection method based on an improved UNet model, which can improve the accuracy and efficiency of yarn allowance detection on yarn bobbins.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a yarn allowance detection method based on an improved UNet model, comprising the following process steps:

[0007] 1) Use a camera to acquire images of a single yarn bobbin and the entire yarn bobbin frame. After augmenting the images with the YOLOv8 dataset, label them to create a dataset in JSON format, and finally convert it into a mask image.

[0008] 2) Divide the dataset into a validation set, a training set, and a test set according to the specified proportions;

[0009] 3) Set the parameters batch_size, workers, and Epoch according to the actual configuration information of the computer's memory, graphics card, and video memory;

[0010] 4) Input the dataset into the improved UNet model for training; wherein the UNet model includes an encoder, a bridging layer and a decoder; the encoder backbone network is VGG16, which is used to extract features from the input image; the bridging layer connects the encoder and the decoder; the decoder network is used to upsample the extracted features and gradually restore them to the original image size for pixel-level classification;

[0011] 5) Within the set number of training rounds, the UNet model is trained repeatedly. In each round, the network model parameters are updated as the initial weights for the next round of training. Based on the evaluation metrics obtained in each round of training, a semantic segmentation model with the best performance within the set number of training rounds is finally obtained, achieving accurate pixel-level prediction.

[0012] 6) After obtaining the segmented yarn surplus area, the edge contour points are extracted by the Canny edge detection algorithm. The extracted edge point set is then used to perform least squares circle fitting to obtain the inner and outer edge contours of the yarn surplus area. Finally, the actual yarn surplus of the yarn bobbin is calculated based on the relationship between the pixel size and the actual size.

[0013] The yarn excess detection method based on the improved UNet model of the present invention is further described as follows: In step 1), the YOLOV8 image augmentation method is as follows: the yarn bobbins on the overall yarn bobbin frame are labeled and trained to obtain training weights, and then the yarn bobbins are target detected to obtain the position information of each yarn bobbin on the yarn bobbin frame in the foreground. Using these position information, the overall yarn bobbin image is extracted into a single yarn bobbin image.

[0014] The yarn excess detection method based on the improved UNet model of the present invention is further described as follows: In step 1), the specific process of generating the mask image is as follows: the original image is labeled using ISAT annotation software, the required categories are pre-set, the model weights of segment anything are loaded for intelligent segmentation and labeling, after all images are labeled, JSON format labels are output, and then the mask image required for UNet network training is generated by combining the label information of the original image and JSON.

[0015] The yarn excess detection method based on the improved UNet model of the present invention further comprises: in step 2), the dataset is divided into a training set, a validation set and a test set in a ratio of 8:1:1.

[0016] The yarn excess detection method based on the improved UNet model of the present invention is further described as follows: In step 3), the computer memory is 1TB, the graphics card is NVIDIA GTX3060 with 12GB of video memory, and the batch_size is set to 2, workers to 2, and Epoch to 100.

[0017] The yarn excess detection method based on the improved UNet model of the present invention is further described in step 4). The improvement of the UNet model is that the encoder in the UNet model is replaced with the first 13 convolutional layers of the VGG16 network. The input yarn image is first downsampled by the encoder, generating feature maps of four scales: 512×512×64, 256×256×128, 128×128×256, and 64×64×512. In the decoder stage, these feature maps obtained by upsampling are concatenated with the feature maps of the corresponding layers in the encoder to achieve effective fusion of deep and shallow features. Finally, a 1×1 convolutional layer is used to adjust the number of channels of the feature map to the number of target segmentation categories to output accurate segmentation results.

[0018] The yarn excess detection method based on the improved UNet model of the present invention is further described in step 4), where the specific training method of the UNet model is as follows:

[0019] 4-1) The input image first passes through the VGG16 encoder to extract features step by step. The bridging layer connects the feature layers. The decoder upsamples the feature map step by step and combines it with the feature map of the corresponding layer in the encoder step. The features are further enhanced by skip connections.

[0020] 4-2), the decoder outputs the semantic segmentation result map, and calculates the Dice loss between the output predicted map and the target ground truth label map; the loss is calculated by comparing the Dice error between the predicted classification map and the ground truth segmentation map.

[0021] 4-3) The loss is propagated to the UNet model through the backpropagation algorithm, and the Adam optimizer is used to update the network weight parameters to reduce the overall loss.

[0022] The yarn balance detection method based on the improved UNet model of the present invention is further described in step 4-2), where the loss weight formula of UNet is as follows:

[0023]

[0024] In the formula: P i To predict the value of the result at the i-th pixel, G i This represents the value of the actual label at the i-th pixel.

[0025] The yarn excess detection method based on the improved UNet model of the present invention is further described below: In step 6), the specific method for extracting the edge point set and fitting is as follows:

[0026] a. The gradient intensity (G) and direction (θ) of each pixel in the image are calculated using the Sobel operator, and then non-maximum suppression is used to remove non-edge pixels to obtain refined edge points. The specific calculation formula is as follows:

[0027]

[0028] b. Edge detection obtains a series of edge point information, and the best matching circle between the two sets of points is found by least squares fitting; a residual function is defined to calculate the difference between the fitted circle and the actual data points. For each point (x... i ,y i The residual is:

[0029] res i =(x i -x c ) 2 +(y i -y c ) 2 -r 2

[0030] By using least squares fitting, we can find (x) c ,y c Find the value of r that minimizes the sum of squared residuals:

[0031]

[0032] Where, x i ,y i x represents the coordinate position of the detected i-th contour point. c ,y c Represents the position coordinates of the center of the circle, r represents the radius of the fitted circle, and res i This represents the difference between the i-th edge point and the fitted circle.

[0033] The yarn allowance detection method based on the improved UNet model of the present invention further includes: In step 6), the method for obtaining the specific size of the yarn allowance is that after fitting the inner and outer contours, the pixel radius parameters (R′, r′) of the two circles can be obtained simultaneously, and the pixel width of the yarn bobbin allowance can be expressed as (R′-r′); based on the pixel diameter of the yarn bobbin, it can be distinguished whether the detected yarn bobbin is cylindrical or conical, and the actual diameter of each type of yarn bobbin is known as D, thus establishing the relationship between the pixel size and the actual size:

[0034]

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The yarn excess detection method based on the improved UNet model of the present invention adopts the improved UNet model, that is, the first 13 convolutional layers of VGG16 are used to replace the encoder part of the UNet network. It can learn complex and abstract feature representations, which helps the model to better extract and segment semantic information. After obtaining the segmentation result, the contour edge point set is obtained by traditional image processing methods and contour fitting is performed, thereby effectively detecting the yarn excess on the yarn bobbin, improving the segmentation and detection accuracy, and having good production practicality.

[0037] 2. The yarn allowance detection method based on the improved UNet model of the present invention has a sample detection error of less than 3mm, and can still maintain high detection accuracy when the allowance is small, which meets the needs of actual production and overcomes the problem of decreased detection accuracy when the yarn allowance is small in other current detection methods.

[0038] 3. The yarn balance detection method of the present invention utilizes deep learning combined with traditional image processing methods to overcome the problems of weak anti-interference ability and low generalization of traditional visual algorithms. [Attached Image Description]

[0039] Figure 1 This is a flowchart of the yarn balance detection method based on the improved UNet model of the present invention.

[0040] Figure 2 This is a diagram showing the positional relationship between the camera and the yarn tube in step 1) of the present invention.

[0041] Figure 3 This is a schematic diagram of the structure of the improved UNet model of the present invention.

[0042] Figure 4 This is the VGG16 network backbone diagram of the present invention.

Detailed Implementation Methods

[0043] Please refer to the instruction manual appendix. Figure 1 To be continued Figure 4 As shown, this is a yarn allowance detection method based on an improved UNet model according to the present invention, which includes the following process steps:

[0044] Step 1) Use a camera to acquire images of a single yarn bobbin and the entire yarn bobbin frame. After augmenting the images with the YOLOv8 dataset, label them to create a JSON dataset, and finally convert it into a mask image.

[0045] The YOLOv8 image augmentation method is as follows: Each yarn bobbin on the overall yarn bobbin frame image is labeled to create a YOLO dataset. This dataset is then fed into the YOLOv8 model for training to obtain training weights. These weights are then used for target detection of the yarn bobbins, yielding the positional information of each bobbin on the foreground yarn bobbin frame. Using this positional information, the overall yarn bobbin image is extracted into individual yarn bobbin images. This rapid dataset augmentation yields multi-angle images of individual yarn bobbins. Combined with single images captured by the camera, a total of 500 yarn bobbin images were collected.

[0046] The specific process of generating the mask image is as follows: the original images are labeled using ISAT annotation software, the required categories are pre-defined, the model weights of segment anything are loaded for intelligent segmentation and annotation, after all images are labeled, JSON format labels are output, and then the mask image required for UNet model network training is generated by combining the label information of the original image and JSON.

[0047] Step 2): Divide the dataset into a validation set, a training set, and a test set according to the specified ratio. In this embodiment, the dataset is divided into the training set, validation set, and test set in an 8:1:1 ratio. Place the original image and the mask image into the JPEGImages and SegmentationClass folders, respectively.

[0048] Step 3) Set the parameters batch_size, workers, and Epoch according to the actual configuration information of the computer's memory, graphics card, and video memory.

[0049] This example uses a computer with 1TB of memory and an NVIDIA GTX3060 graphics card with 12GB of video memory. Based on this, batch_size=2, workers=2, Epoch=100 are set, and the corresponding file paths in step 2) are also set.

[0050] Step 4) Input the dataset into the improved UNet model for training. For example... Figure 3 As shown, the UNet model includes an encoder, a bridge layer, and a decoder. The encoder backbone network is VGG16, which is used to extract features from the input image. The bridge layer connects the encoder and the decoder. The decoder network is used to upsample the extracted features and gradually restore them to the original image size for pixel-level classification.

[0051] The improvement to the UNet model lies in replacing the encoder with the first 13 convolutional layers of the VGG16 network. The input yarn image is first downsampled by the encoder, generating feature maps of four scales: 512×512×64, 256×256×128, 128×128×256, and 64×64×512. In the decoder stage, these upsampled feature maps are concatenated with the corresponding feature maps from the encoder, achieving effective fusion of deep and shallow features. Finally, a 1×1 convolutional layer adjusts the number of channels in the feature maps to match the number of target segmentation categories, outputting accurate segmentation results. The VGG16 network structure is as follows: Figure 4 As shown.

[0052] The specific training method for the improved UNet model is as follows:

[0053] 4-1) The input image first passes through the VGG16 encoder to extract features step by step. The bridging layer connects the feature layers, and the decoder performs step-by-step upsampling of the feature map. It also combines the feature maps of the corresponding layers in the encoder steps and further enhances the features through skip connections.

[0054] 4-2), the decoder outputs the semantic segmentation result map, and calculates the Dice loss between the output predicted map and the target ground truth map; the loss is calculated by comparing the Dice error between the predicted classification map and the ground truth segmentation map.

[0055] The loss weight formula for UNet is as follows:

[0056]

[0057] In the formula: P i To predict the value of the result at the i-th pixel, G i This represents the value of the actual label at the i-th pixel.

[0058] 4-3) The loss is propagated to the UNet model through the backpropagation algorithm, and the Adam optimizer is used to update the network weight parameters to reduce the overall loss.

[0059] In summary, step 4) uses the pre-trained weights of the VGG network on a large-scale dataset for transfer learning, enabling the model to learn complex and abstract feature representations, which helps the model to better extract and segment semantic information.

[0060] Step 5) Within the set number of training rounds, the UNet model is trained repeatedly, that is, steps 4-1) to 4-3) are repeated. In each round, the network model parameters are updated as the initial weights for the next round of training. Based on the evaluation metrics obtained in each round of training, a semantic segmentation model with the best performance within the set number of training rounds is finally obtained, achieving accurate pixel-level prediction.

[0061] Step 6): After obtaining the segmented yarn surplus area, the edge contour points are extracted by the Canny edge detection algorithm. The extracted edge point set is then used to perform least squares circle fitting to obtain the inner and outer edge contours of the yarn surplus area. Finally, the actual yarn surplus of the yarn bobbin is calculated based on the relationship between the pixel size and the actual size.

[0062] The specific methods for extracting the edge point set and fitting are as follows:

[0063] a. The gradient intensity (G) and direction (θ) of each pixel in the image are calculated using the Sobel operator, and then non-maximum suppression is used to remove non-edge pixels to obtain refined edge points. The specific calculation formula is as follows:

[0064]

[0065] b. Edge detection obtains a series of edge point information, and the best matching circle between the two sets of points is found by least squares fitting; a residual function is defined to calculate the difference between the fitted circle and the actual data points. For each point (x... i ,y i The residual is:

[0066] res i =(x i -x c ) 2 +(y i -y c ) 2 -r 2

[0067] By using least squares fitting, we can find (x) c ,y c Find the value of r that minimizes the sum of squared residuals:

[0068]

[0069] Where, x i ,y i x represents the coordinate position of the detected i-th contour point. c ,y c Represents the position coordinates of the center of the circle, r represents the radius of the fitted circle, and res iThis represents the difference between the i-th edge point and the fitted circle.

[0070] Furthermore, the method for obtaining the specific dimensions of the yarn allowance involves fitting the inner and outer contours and simultaneously obtaining the pixel radius parameters (R′, r′) of the two circles. The pixel width of the yarn bobbin allowance can be expressed as (R′-r′). Based on the pixel diameter of the yarn bobbin, it can be distinguished whether the detected yarn bobbin is cylindrical or conical. The actual diameter of each type of yarn bobbin is known as D, thus establishing the relationship between the pixel size and the actual size.

[0071]

[0072] To better and more objectively evaluate the model's segmentation performance, the confusion matrix is ​​used to determine the mean precision (mPrecision), mean recall (mRecall), mean pixel accuracy (mPA), and mean intersection over union (mIoU) to measure the model's performance. The specific formulas are as follows:

[0073]

[0074]

[0075] Where k represents the number of categories, represents the total number of pixels that originally belonged to category i but were predicted to belong to category i, and represents the total number of pixels that originally belonged to category i but were predicted to belong to category j.

[0076] The results obtained from the above embodiments show that the model after pre-trained weight fine-tuning improved the average precision by 16.14 percentage points, the average recall by 13.68 percentage points, and the average intersection-over-union (IoU) by 24.63 percentage points on the yarn region segmentation task. This clearly demonstrates that, under conditions of limited sample size and few training rounds, the method of fine-tuning the model using pre-trained weights is significantly superior to the strategy of training from scratch, as shown in Table 1. To verify and evaluate the performance of the yarn excess detection method proposed in this study, the method was used to measure the excess yarn in six groups of images with different excess yarn amounts and different bobbin shapes, and the relevant data were recorded and collected. Specific results are shown in Table 2.

[0077]

[0078] Table 1 Comparison of various metrics for UNet model training before and after improvement

[0079]

[0080] Table 2 Results of Yarn Balance Inspection

[0081] As described in the above embodiments, this invention provides a yarn excess detection method based on an improved UNet model. By replacing the encoder part of the U-Net network with the first 13 convolutional layers of VGG16 and applying transfer learning, after loading pre-trained weights, the overall performance and generalization ability of the improved UNet model are greatly enhanced. The segmentation precision, recall, and mean intersection-over-union ratio are all superior to other models. After obtaining the segmentation results, the contour edge point set is obtained using traditional image processing methods, and then the detection contour is obtained by fitting using the least squares method. Finally, the measurement result is calculated by the relationship between pixel size and actual size. The error of this method in detecting yarn excess is less than 3mm, and the relative error can also be controlled within 4%, which has high practical value. This method has high accuracy while ensuring detection efficiency.

[0082] The above-described specific embodiments are merely preferred embodiments of this invention and are not intended to limit this invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A method for detecting yarn allowance based on an improved UNet model, characterized in that: The process includes the following steps: 1) Use a camera to acquire images of a single yarn bobbin and the entire yarn bobbin frame. After augmenting the images with the YOLOv8 dataset, label them to create a dataset in JSON format, and finally convert it into a mask image. 2) Divide the dataset into validation set, training set, and test set according to the proportions; 3) Set the parameters batch_size, workers, and Epoch according to the actual configuration information of the computer's memory, graphics card, and video memory; 4) Input the dataset into the improved UNet model for training; wherein the UNet model includes an encoder, a bridging layer and a decoder; the encoder backbone network is VGG16, which is used to extract features from the input image; the bridging layer connects the encoder and the decoder; the decoder network is used to upsample the extracted features and gradually restore them to the original image size for pixel-level classification; The improvement of the UNet model lies in replacing the encoder with the first 13 convolutional layers of the VGG16 network. The input yarn image is first downsampled by the encoder, generating feature maps of four scales: 512×512×64, 256×256×128, 128×128×256, and 64×64×512. In the decoder stage, these upsampled feature maps are concatenated with the feature maps of the corresponding layers in the encoder to achieve effective fusion of deep and shallow features. Finally, a 1×1 convolutional layer is used to adjust the number of channels in the feature maps to match the number of target segmentation categories, resulting in an accurate segmentation output. 5) Within the set number of training rounds, the UNet model is trained repeatedly. In each round, the network model parameters are updated as the initial weights for the next round of training. Based on the evaluation metrics obtained in each round of training, a semantic segmentation model with the best performance within the set number of training rounds is finally obtained, achieving accurate pixel-level prediction. 6) After obtaining the segmented yarn surplus area, the edge contour points are extracted by the Canny edge detection algorithm. The extracted edge point set is then used to perform least squares circle fitting to obtain the inner and outer edge contours of the yarn surplus area. Finally, the actual yarn surplus of the yarn bobbin is calculated based on the relationship between the pixel size and the actual size.

2. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 1), the YOLOV8 dataset image augmentation method is as follows: the yarn tubes on the overall yarn tube frame are labeled and trained to obtain training weights, and then the yarn tubes are detected to obtain the position information of each yarn tube on the yarn tube frame in the foreground. Using this position information, the overall yarn tube image is extracted into individual yarn tube images.

3. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 1), the specific process of generating the mask image is as follows: the original images are labeled using ISAT annotation software, the required categories are pre-defined, the model weights of segment anything are loaded for intelligent segmentation and labeling, after all images are labeled, JSON format labels are output, and then the mask image required for UNet network training is generated by combining the label information of the original image and JSON.

4. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 2), the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:

1.

5. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 3), the computer memory is 1TB, the graphics card is an NVIDIA GTX3060 with 12GB of video memory, and the batch_size is set to 2, workers to 2, and Epoch to 100.

6. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 4), the specific training method for improving the UNet model is as follows: 4-1) The input image first passes through the VGG16 encoder to extract features step by step. The bridging layer connects the feature layers. The decoder upsamples the feature map step by step and combines it with the feature map of the corresponding layer in the encoder step. The features are further enhanced by skip connections. 4-2), the decoder outputs a semantic segmentation result map, and the Dice loss between the output predicted map and the target ground truth label map is calculated; the loss is calculated by comparing the Dice error between the predicted classification map and the ground truth segmentation map. 4-3) The loss is propagated to the UNet model through the backpropagation algorithm, and the Adam optimizer is used to update the network weight parameters to reduce the overall loss.

7. The yarn allowance detection method based on the improved UNet model as described in claim 6, characterized in that: In step 4-2), the loss weight formula for UNet is as follows: DiceLoss= In the formula: The value of the prediction result at the i-th pixel. This represents the value of the actual label at the i-th pixel.

8. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 6), the specific methods for extracting the edge point set and fitting are as follows: a. Calculate the gradient intensity G and direction of each pixel in the image using the Sobel operator. Then, non-maximum suppression is used to remove non-edge pixels to obtain refined edge points. The specific calculation formula is as follows: b. Edge detection acquires a series of edge point information. The best circle matching the two sets of points is found using the least squares method. A residual function is defined to calculate the difference between the fitted circle and the actual data points. For each point... The residual is: Find by least squares fitting The value of minimizes the sum of squared residuals: ； in, This represents the coordinate position of the detected i-th contour point. The coordinates of the center of the circle are represented by r, and r represents the radius of the fitted circle. This represents the difference between the i-th edge point and the fitted circle.

9. The yarn allowance detection method based on the improved UNet model as described in claim 1, characterized in that: In step 6), the method for obtaining the specific dimensions of the yarn allowance is that after fitting the inner and outer contours, the pixel radius parameters of the two circles can be obtained simultaneously. The pixel width of the yarn bobbin allowance can be expressed as ( Based on the pixel diameter of the yarn bobbin, it can distinguish whether the detected yarn bobbin is cylindrical or conical, and the actual diameter of each type of yarn bobbin is known. This establishes the relationship between pixel size and actual size: 。

Images