Yarn allowance detection method based on improved UNet model
Through the improved UNet model and VGG16 network, combined with YOLOV8 dataset expansion and edge detection technology, the accuracy and anti-interference problems of yarn allowance detection are solved, and efficient and accurate yarn allowance detection is achieved to meet modern textiles. Production demand.
Patent Information
- Application Number
- CN202410926703.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-11
AI Technical Summary
The existing yarn allowance detection methods rely on manual inspection and traditional visual technology, with low accuracy and weak anti-interference ability, making it difficult to meet the high standards of modern textile production, especially when the shape of the yarn barrel is inconsistent and the remaining limit detection standards are different. , it is difficult to calculate the yarn margin.
The improved UNet model is adopted and the yarn margin detection is carried out in combination with the VGG16 network. Through YOLOV8 dataset expansion, Canny edge detection and least squares fitting, accurate pixel-level prediction and yarn margin calculation are achieved.
The accuracy and efficiency of yarn allowance detection are improved, and the detection errors are within 3mm, which meets the actual production needs, overcomes the problem of the reduction of accuracy when the yarn allowance is small, and combines deep learning and traditional image processing. Improves anti-interference ability and generalization.
Smart Images

Figure CN118918359A_ABST
Abstract
Description
[Technical field]
[0001] The invention relates to a yarn remainder detection method, in particular to a yarn remainder detection method based on an improved UNet model, and belongs to the technical field of target detection and image segmentation in the textile industry. [Background technology]
[0002] In the textile production process, yarn is a key raw material, and its production quality directly affects the quality of the final product. At present, yarn excess detection mainly relies on manual inspection and traditional visual technology. These methods often have problems such as low accuracy, weak anti-interference ability and limited generalization ability, and it is difficult to meet the high standards of modern textile production. With the advancement of digital image processing technology, the application of machine vision in the textile field is becoming more and more extensive.
[0003] Because the yarn tubes in the enterprise are placed on rows of yarn racks, with the end faces of the yarn tubes facing the aisles, during the inspection process, the camera can only take pictures from the end faces of the yarn tubes. The pictures taken cannot see or can only partially see the side faces of the yarn tubes, making it difficult to calculate the length of the yarn tubes. This makes it difficult to calculate the yarn remainder. In addition, some yarn tubes have inconsistent shapes and sizes, and the standards for remainder detection are also different. In recent years, in the field of machine vision, target detection and semantic segmentation have attracted much attention, and have also been used by many manufacturers for assembly line detection and recognition. Many excellent neural networks, such as CNN, RCNN, VGG, UNet and other networks are used in them.
[0004] Therefore, in order to solve the above problems, it is necessary to provide an innovative yarn remainder detection method based on an improved UNet model to overcome the above defects in the prior art. [Summary of the invention]
[0005] To solve the above problems, the object of the present invention is to provide a yarn remainder detection method based on an improved UNet model, which can improve the accuracy and efficiency of detecting the yarn remainder on a yarn bobbin.
[0006] To achieve the above object, the technical solution adopted by the present invention is: a yarn remainder detection method based on an improved UNet model, which includes the following process steps:
[0007] 1) Use a camera to obtain a single bobbin image and a bobbin frame image of the entire bobbin. After the image is expanded with the YOLOV8 dataset, it is annotated and made into a json format dataset, which is finally converted into a mask map;
[0008] 2) Divide the data set into a validation set, a training set, and a test set according to the ratio;
[0009] 3) Set the parameters batch_size, workers and Epoch according to the actual computer memory, graphics card and video memory configuration information;
[0010] 4) Input the data set into the improved UNet model for training; wherein the UNet model includes an encoder, a bridge layer and a decoder; the encoder backbone network is VGG16, which is used to extract features from the input image; the bridge layer connects the encoder and the decoder; the decoder network is used to upsample the extracted features and gradually restore them to the original image size for pixel-level classification;
[0011] 5) Repeat the training of the UNet model within the set training rounds, and update the network model parameters in each round as the initial weights for the next round of new training; based on the evaluation indicators obtained in each round of training, finally obtain a semantic segmentation model with the best effect within the set number of training rounds to achieve accurate pixel-level prediction;
[0012] 6) After obtaining the segmented yarn surplus area, the Canny edge detection algorithm is used to extract the edge contour points of the area edge points, and the least squares circle fitting is performed using the extracted edge point set to obtain the inner and outer edge contours of the yarn surplus area. Finally, based on the relationship between the pixel size and the actual size, the actual yarn surplus amount of the yarn tube is calculated.
[0013] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 1), the YOLOV8 image expansion method is: the yarn tubes on the entire yarn tube rack are marked and trained to obtain training weights, and then the target detection of the yarn tubes is performed to obtain the position information of each yarn tube on the yarn tube rack in the foreground, and the overall yarn tube image is extracted into a single yarn tube image using this position information.
[0014] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 1), the specific generation process of the mask image is: the collected original image is annotated by ISAT annotation software, the required categories are pre-set, the model weights of segment anything are loaded for intelligent segmentation and annotation, and after all images are annotated, Json format labels are output, and then the mask image required for UNet network training is generated by combining the original image and Json label information.
[0015] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 2), the data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1.
[0016] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 3), the computer memory is specifically 1TB, the graphics card is NVIDIAGTX3060, the video memory is 12GB, and batch_size=2, workers=2, and Epoch=100 are set.
[0017] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 4), the improvement of the UNet model is that the encoder in the UNet model is replaced by the first 13 convolutional layers of the VGG16 network, and the input yarn image is first downsampled by the encoder to generate feature maps of four scales of 512×512×64, 256×256×128, 128×128×256, and 64×64×512 in sequence. In the decoder stage, these feature maps obtained by upsampling are spliced with the feature maps of the corresponding levels in the encoder to achieve effective fusion of deep and shallow features; finally, the number of channels of the feature map is adjusted to the number of target segmentation categories through a 1×1 convolutional layer to output an accurate segmentation result.
[0018] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 4), the specific training method of the UNet model is:
[0019] 4-1), the input image first passes through the VGG16 encoder to gradually extract features, the bridge layer connects the feature layer, the decoder gradually upsamples the feature map, and combines the feature map of the corresponding level in the encoder step, and further enhances the features through jump connections;
[0020] 4-2), the decoder outputs the semantic segmentation result map, calculates the Dice loss between the output prediction map and the target true label map; the loss is calculated by comparing the Dice error between the predicted classification map and the true segmentation map;
[0021] 4-3), propagate the loss to the UNet model through the back-propagation algorithm, and use the Adam optimizer to update the network weight parameters to reduce the overall loss.
[0022] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: in the step 4-2), the loss weight formula of UNet is as follows:
[0023]
[0024] Where: P i is the value of the prediction result at the i-th pixel, G i Represents the value of the true label at the i-th pixel.
[0025] The yarn remainder detection method based on the improved UNet model of the present invention is further as follows: In the step 6), the specific method of extracting the edge point set and fitting is as follows:
[0026] a. The Sobel operator is used to calculate the gradient strength (G) and direction (θ) of each pixel in the image, and then non-maximum suppression is used to remove non-edge pixels to obtain refined edge points. The specific calculation formula is as follows:
[0027]
[0028] b. Edge detection obtains a series of edge point information, and finds the best circle matching the two sets of points through least squares fitting; by defining a residual function, the difference between the fitted circle and the actual data point is calculated. For each point (x i ,y i )The residual is:
[0029] res i =(x i -x c ) 2 +(y i -y c ) 2 -r 2
[0030] Through least squares fitting, find (x c ,y c ,r) value, so that the residual sum of squares is minimized:
[0031]
[0032] Among them, x i ,y i Represents the coordinate position of the i-th contour point detected, x c ,y c represents the position coordinates of the center of the circle, r represents the radius of the fitted circle, res i represents the difference between the i-th edge point and the fitted circle.
[0033] The yarn remainder detection method based on the improved UNet model of the present invention is also as follows: in the step 6), the method for obtaining the specific size of the yarn remainder is to obtain the pixel radius parameters (R′, r′) of the two circles at the same time after fitting the inner and outer contours, and the pixel width of the bobbin remainder can be expressed as (R′-r′); according to the pixel diameter of the bobbin, it is distinguished whether the bobbin to be detected is a cylindrical bobbin or a conical bobbin, and the actual diameter of each type of bobbin is known D, so that the relationship between the pixel size and the actual size can be established:
[0034]
[0035] Compared with the prior art, the present invention has the following beneficial effects:
[0036] 1. The yarn remainder detection method based on the improved UNet model of the present invention adopts the improved UNet model, that is, the first 13 convolutional layers of VGG16 are used to replace the encoder part of the UNet network, which can learn complex and abstract feature representations, which helps the model to better extract and segment semantic information. After obtaining the segmentation result, the traditional image processing method is used to obtain the contour edge point set and perform contour fitting, so that the yarn remainder on the yarn tube can be effectively detected, the segmentation and detection accuracy is improved, and it has good production practicality.
[0037] 2. The sample detection error of the yarn remainder detection method based on the improved UNet model of the present invention is within 3 mm, and when the remainder is small, it can still maintain a high detection accuracy, meeting the needs of actual production, and overcoming the problem of decreased detection accuracy when the yarn remainder is small in other current detection methods.
[0038] 3. The yarn remainder detection method of the present invention utilizes deep learning combined with traditional image processing methods to make up for the problems of weak anti-interference ability and low generalization of traditional visual algorithms.
Brief Description of the Drawings
[0039] Figure 1 It is a flow chart of the yarn remainder detection method based on the improved UNet model of the present invention.
[0040] Figure 2 1 is a diagram showing the positional relationship between the camera and the yarn bobbin in step 1) of the present invention.
[0041] Figure 3 It is a structural diagram of the improved UNet model of the present invention.
[0042] Figure 4 It is the VGG16 network backbone diagram of the present invention. [Specific implementation method]
[0043] Please refer to the instruction manual Figure 1 To Attachment Figure 4 As shown, it is a yarn remainder detection method based on the improved UNet model of the present invention, which includes the following process steps:
[0044] Step 1), use the camera to obtain a single yarn tube image and a whole yarn tube image of the yarn tube rack. After the image is expanded by the YOLOV8 dataset, it is annotated to produce a dataset in json format, and finally converted into a mask image.
[0045] The YOLOV8 image expansion method is as follows: the yarn tubes on the overall yarn tube rack image are marked one by one to create a YOLO data set; the data set is placed in the YOLOV8 model and trained to obtain training weights, and the obtained training weights are used to detect the yarn tube target, and the position information of each yarn tube on the yarn tube rack in the foreground is obtained. Using this position information, the overall yarn tube image is extracted into a single yarn tube image. Through the above operations, the data set is quickly expanded to obtain a single yarn tube image from multiple angles. Combined with a single picture taken by the camera, a total of 500 yarn tube images are collected.
[0046] The specific generation process of the mask map is as follows: the collected original image is annotated by ISAT annotation software, the required categories are pre-set, the model weights of segment anything are loaded for intelligent segmentation and annotation, and after all images are annotated, Json format labels are output, and then the mask map required for UNet model network training is generated by combining the original image and Json label information.
[0047] Step 2), divide the data set into a validation set, a training set, and a test set according to the ratio. In this embodiment, the data set is divided into a training set, a validation set, and a test set according to the ratio of 8:1:1. Put the original image and the mask image into the JPEGImages and SegmentationClass folders respectively.
[0048] Step 3), set the parameters batch_size, workers and Epoch according to the actual computer memory, graphics card and video memory configuration information.
[0049] The computer memory used in this example is 1TB, the graphics card is NVIDIA GTX3060, and the video memory is 12GB. Based on this, batch_size=2, workers=2, Epoch=100 are set, and set to the corresponding file path in step 2).
[0050] Step 4), input the data set into the improved UNet model for training. Figure 3 As shown in the figure, the UNet model includes an encoder, a bridge layer and a decoder; the encoder backbone network is VGG16, which is used to extract features from the input image; the bridge layer connects the encoder and the decoder; the decoder network is used to upsample the extracted features and gradually restore them to the original image size for pixel-level classification.
[0051] The improvement of the UNet model is that the encoder in the UNet model is replaced by the first 13 convolutional layers of the VGG16 network. The input yarn image is first downsampled by the encoder to generate feature maps of four scales: 512×512×64, 256×256×128, 128×128×256, and 64×64×512. In the decoder stage, these feature maps obtained by upsampling are spliced with the feature maps of the corresponding levels in the encoder to achieve effective fusion of deep and shallow features. Finally, a 1×1 convolutional layer is used to adjust the number of channels of the feature map to the number of target segmentation categories to output accurate segmentation results. The VGG16 network structure is as follows: Figure 4 shown.
[0052] The specific training method of the improved UNet model is:
[0053] 4-1), the input image first passes through the VGG16 encoder to gradually extract features, the bridge layer connects the feature layer, the decoder gradually upsamples the feature map, and combines the feature maps of the corresponding levels in the encoder step, and further enhances the features through skip connections.
[0054] 4-2), the decoder outputs the semantic segmentation result map, calculates the Dice loss between the output prediction map and the target true label map; and calculates the loss by comparing the Dice error between the predicted classification map and the true segmentation map.
[0055] Among them, the loss weight formula of UNet is as follows:
[0056]
[0057] Where: P i is the value of the prediction result at the i-th pixel, G i Represents the value of the true label at the i-th pixel.
[0058] 4-3), propagate the loss to the UNet model through the back-propagation algorithm, and use the Adam optimizer to update the network weight parameters to reduce the overall loss.
[0059] In summary, this step 4) performs transfer learning through the pre-trained weights of the VGG network on a large-scale dataset, so that the model can learn complex and abstract feature representations, which helps the model to better extract and segment semantic information.
[0060] Step 5), within the set training rounds, repeatedly train the UNet model, that is, repeat steps 4-1) to 4-3), and update the network model parameters in each round as the initialization weights for the next round of new training; based on the evaluation indicators obtained in each round of training, finally obtain a semantic segmentation model with the best effect within the set number of training rounds to achieve accurate pixel-level prediction.
[0061] Step 6), after obtaining the segmented yarn surplus area, the Canny edge detection algorithm is used to extract the edge contour points of the area edge points, and the least squares circle fitting is performed using the extracted edge point set to obtain the inner and outer edge contours of the yarn surplus area. Finally, based on the relationship between the pixel size and the actual size, the actual remaining yarn amount of the yarn tube is calculated.
[0062] Among them, the specific methods of extracting edge point sets and fitting are as follows:
[0063] a. The Sobel operator is used to calculate the gradient strength (G) and direction (θ) of each pixel in the image, and then non-maximum suppression is used to remove non-edge pixels to obtain refined edge points. The specific calculation formula is as follows:
[0064]
[0065] b. Edge detection obtains a series of edge point information, and finds the best circle matching the two sets of points through least squares fitting; by defining a residual function, the difference between the fitted circle and the actual data point is calculated. For each point (x i ,y i )The residual is:
[0066] res i =(x i -x c ) 2 +(y i -y c ) 2 -r 2
[0067] Through least squares fitting, find (x c ,y c ,r) value, so that the residual sum of squares is minimized:
[0068]
[0069] Among them, x i ,y i Represents the coordinate position of the i-th contour point detected, x c ,y c represents the position coordinates of the center of the circle, r represents the radius of the fitted circle, res irepresents the difference between the i-th edge point and the fitted circle.
[0070] Furthermore, the method for obtaining the specific size of the yarn excess is to obtain the pixel radius parameters (R′, r′) of the two circles after fitting the inner and outer contours. The pixel width of the yarn excess can be expressed as (R′-r′); according to the pixel diameter of the yarn cone, it is distinguished whether the detected yarn cone is a cylindrical yarn cone or a conical yarn cone, and the actual diameter of each type of yarn cone is known D, so the relationship between the pixel size and the actual size can be established:
[0071]
[0072] In order to better objectively evaluate the segmentation effect of the model, the confusion matrix is used to determine the average precision (mPrecision), average recall (mRecall), average pixel accuracy (mPA), and average intersection over union (mIoU) to measure the performance of the model. The specific formula is as follows:
[0073]
[0074]
[0075] Where k represents the number of categories, represents the total number of pixels that originally belonged to the i-th category but were predicted to be the i-th category, and represents the total number of pixels that originally belonged to the i-th category but were predicted to be the j-th category.
[0076] According to the results obtained in the above embodiments, the average precision of the model after fine-tuning the pre-training weights in the yarn region segmentation task increased by 16.14 percentage points, the average recall rate increased by 13.68 percentage points, and the average intersection-over-union ratio increased by 24.63 percentage points. This result clearly shows that under the conditions of limited sample size and fewer training rounds, the method of fine-tuning the model using pre-training weights is significantly better than the strategy of training from scratch, as shown in Table 1. In order to verify and evaluate the performance of the yarn remainder detection method proposed in this study, the method was used to measure the yarn remainder amount for 6 groups of images with different remainders and different barrel types, and the relevant data were recorded and collected. The specific results are recorded in Table 2.
[0077]
[0078] Table 1 Comparison of various indicators of UNet model training before and after improvement
[0079]
[0080] Table 2 Yarn remainder detection results
[0081] According to the above embodiments, it can be seen that the present invention provides a yarn remainder detection method based on an improved UNet model. By replacing the encoder part of the U-Net network with the first 13 convolutional layers of VGG16, and applying transfer learning, after loading the pre-trained weights, the overall performance and generalization ability of the improved UNet model are greatly improved, and the segmentation accuracy, recall rate, and average intersection-over-union ratio are better than other models. After obtaining the segmentation result, the traditional image processing method is used to obtain the contour edge point set, and then the least squares method is used to fit the detection contour, and finally the measurement result is calculated by the relationship between the pixel size and the actual size. The error of the yarn remainder detected by this method is less than 3mm, and the relative error can also be controlled within 4%, which has a high practical value. This method has a high accuracy rate while ensuring detection efficiency.
[0082] The above specific implementation methods are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A yarn remainder detection method based on an improved UNet model, characterized in that: The process steps include: 1) Use a camera to obtain a single bobbin image and a bobbin frame image of the entire bobbin. After the image is expanded with the YOLOV8 dataset, it is annotated and made into a json format dataset, which is finally converted into a mask map; 2) Divide the data set into validation set, training set and test set according to the proportion; 3) Set the parameters batch_size, workers and Epoch according to the actual computer memory, graphics card and video memory configuration information; 4) Input the data set into the improved UNet model for training; wherein the UNet model includes an encoder, a bridge layer and a decoder; the encoder backbone network is VGG16, which is used to extract features from the input image; the bridge layer connects the encoder and the decoder; the decoder network is used to upsample the extracted features and gradually restore them to the original image size for pixel-level classification; 5) Repeat the training of the UNet model within the set training rounds, and update the network model parameters in each round as the initial weights for the next round of new training; based on the evaluation indicators obtained in each round of training, finally obtain a semantic segmentation model with the best effect within the set number of training rounds to achieve accurate pixel-level prediction; 6) After obtaining the segmented yarn surplus area, the Canny edge detection algorithm is used to extract the edge contour points of the area edge points, and the least squares circle fitting is performed using the extracted edge point set to obtain the inner and outer edge contours of the yarn surplus area. Finally, based on the relationship between the pixel size and the actual size, the actual yarn surplus amount of the yarn tube is calculated.
2. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In the step 1), the image expansion method of the YOLOV8 data set is as follows: the yarn tubes on the whole yarn tube rack are marked and trained to obtain training weights, and then the target detection of the yarn tubes is performed to obtain the position information of each yarn tube on the yarn tube rack in the foreground, and the overall yarn tube image is extracted into a single yarn tube image using this position information.
3. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In the step 1), the specific generation process of the mask map is as follows: the collected original image is annotated by ISAT annotation software, the required categories are pre-set, the model weights of segment anything are loaded for intelligent segmentation and annotation, and after all images are annotated, Json format labels are output, and then the mask map required for UNet network training is generated by combining the original image and Json label information.
4. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In the step 2), the data set is divided into a training set, a validation set and a test set in a ratio of 8:1:
1.
5. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In the step 3), the computer memory is specifically 1TB, the graphics card is NVIDIA GTX3060, the video memory is 12GB, and batch_size=2, workers=2, and Epoch=100 are set.
6. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In the step 4), the improvement of the UNet model is that the encoder in the UNet model is replaced by the first 13 convolutional layers of the VGG16 network. The input yarn image is first downsampled by the encoder to generate feature maps of four scales of 512×512×64, 256×256×128, 128×128×256, and 64×64×512 in sequence. In the decoder stage, these feature maps obtained by upsampling are spliced with the feature maps of the corresponding levels in the encoder to achieve effective fusion of deep and shallow features; finally, the number of channels of the feature map is adjusted to the number of target segmentation categories through a 1×1 convolutional layer to output accurate segmentation results.
7. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In step 4), the specific training method for improving the UNet model is: 4-1), the input image first passes through the VGG16 encoder to gradually extract features, the bridge layer connects the feature layer, the decoder gradually upsamples the feature map, and combines the feature map of the corresponding level in the encoder step, and further enhances the features through jump connections; 4-2), the decoder outputs the semantic segmentation result map, calculates the Dice loss between the output prediction map and the target true label map; the loss is calculated by comparing the Dice error between the predicted classification map and the true segmentation map; 4-3), propagate the loss to the UNet model through the back-propagation algorithm, and use the Adam optimizer to update the network weight parameters to reduce the overall loss.
8. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In step 4-2), the loss weight formula of UNet is as follows: Where: P i is the value of the prediction result at the i-th pixel, G i Represents the value of the true label at the i-th pixel.
9. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In step 6), the specific method of extracting edge point sets and fitting is as follows: a. The Sobel operator is used to calculate the gradient strength (G) and direction (θ) of each pixel in the image, and then non-maximum suppression is used to remove non-edge pixels to obtain refined edge points. The specific calculation formula is as follows: b. Edge detection obtains a series of edge point information, and finds the best circle matching the two sets of points through least squares fitting; by defining a residual function, the difference between the fitted circle and the actual data point is calculated. For each point (x i ,y i )The residual is: beef i (x) i -x c ) 2 +(and i -and c ) 2 -r 2 Through least squares fitting, find (x c ,y c ,r) value, so that the residual sum of squares is minimized: Among them, x i ,y i Represents the coordinate position of the i-th contour point detected, x c ,y c represents the position coordinates of the center of the circle, r represents the radius of the fitted circle, res i represents the difference between the i-th edge point and the fitted circle.
10. The yarn remainder detection method based on the improved UNet model according to claim 1, characterized in that: In step 6), the method for obtaining the specific size of the yarn surplus is to obtain the pixel radius parameters (R′, r′) of the two circles at the same time after fitting the inner and outer contours. The pixel width of the bobbin surplus can be expressed as (R′-r′); according to the pixel diameter of the bobbin, it is distinguished whether the bobbin being detected is a cylindrical bobbin or a conical bobbin, and the actual diameter of each type of bobbin is known D, so the relationship between the pixel size and the actual size can be established:
Citation Information
Patent Citations
Textile bobbin yarn detection method based on computer vision
CN115063424A
Bobbin yarn remaining amount detection method based on inverse projection transformation
CN115511872A
Cited By
Microstructure parameter measurement method based on UNet network
CN119477881A
