A method, readable storage medium and terminal device for online detection of fabric defects based on an improved YOLOv5 model

Abstract

The present invention relates to a method, readable storage medium, and terminal device for online detection of fabric defects based on an improved YOLOv5 model. The method comprises the following steps: collecting multiple fabric defect images to form a data set and dividing the data into a training set and a test set; labeling the fabric defect images in the training set with fabric defect categories and performing data enhancement; preprocessing the fabric defect images; training the improved YOLOv5 model using the preprocessed fabric defect images; and verifying the model using fabric defect images in the test set to obtain a fabric defect category detection and analysis processing model; and detecting fabric images in real time based on the obtained fabric defect category detection and analysis processing model, and outputting analysis results. The present invention also discloses a readable storage medium and terminal device using the method. The method reduces missed detection of fabric defects and improves the accuracy of fabric defect detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method for online detection of fabric defects based on an improved YOLOv5 model, a readable storage medium, and a terminal device. Background Art

[0002] With the deepening integration of machine vision technology and industrial manufacturing, the manufacturing industry has undergone tremendous changes, gradually shifting from a focus on quantity to a pursuit of quality. In the textile sector in particular, users and companies are increasingly demanding higher quality appearance. Users are more concerned with the appearance of textiles than their functionality. Surface defects such as stains, scratches, and holes directly impact a product's aesthetics and sales value.

[0003] In fact, as far as surface defect detection is concerned, the human eye detection method has been around since the 20th century. Due to insufficient funds, small and medium-sized enterprises still choose to rely on the human eye to complete the inspection work. The general process is as follows: fabric products move at a constant speed on the assembly line in the company's workshop, and workers stand on the observation platform to check for defects in the fabric passing by. Once a defect is found, the worker stops the assembly line, marks the defective part and records the relevant information. Obviously, this detection method has many problems.

[0004] Research on intelligent fabric defect detection using image processing has a history of over 30 years, primarily focusing on image segmentation, feature extraction, and defect classification. Methods include statistical analysis of co-occurrence matrices, spectral analysis of Fourier transforms and wavelet analysis, and classification algorithms based on artificial neural networks. The research results published by international scholars in this field have also promoted the development and innovation of this field. Although considerable research has been achieved both domestically and internationally, there are still relatively few truly marketable fabric defect detection methods.

[0005] Existing fabric defect detection methods generally target all possible defects, balancing detection performance across all sizes. However, in reality, some defects, particularly large ones, occur less frequently, while medium-sized ones occur more frequently. Therefore, while maintaining a certain detection rate for all defects, it is important to improve detection performance for medium-sized defects. This present invention addresses this issue by modifying the ResNet architecture and adding contextual information to the FPN output. This improves detection performance for medium-sized defects while maintaining detection performance across all sizes. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method, a readable storage medium and a terminal device for online detection of fabric defects based on an improved YOLOv5 model, thereby improving the accuracy of fabric defect detection and reducing missed defect detections.

[0007] The present invention is implemented by providing a method for online detection of fabric defects based on an improved YOLOv5 model, the method comprising the following steps:

[0008] Step 1: Collect multiple fabric defect images to form a data set;

[0009] Step 2: Divide the data set into a training set and a test set, label the fabric defect images in the training set by category, and perform data enhancement and histogram equalization on the labeled fabric defect images;

[0010] Step 3: Build an improved YOLOv5 model, extract features from the input image through the feature extraction network backbone and feature pyramid FPN, and build a task detection head to detect the extracted features. The feature extraction network backbone consists of Focus convolution block, CBM convolution block and CSP convolution block, wherein the CBM convolution block contains convolution layer, batch normalization layer and Mish activation layer, and the CSP convolution block contains ResNet_SPP convolution block; the feature extraction network backbone e consists of five stages, the first stage contains the Focus convolution block and the CBM convolution block, the second to the fifth stages all contain the CBM convolution block and the CSP convolution block, wherein the CSP convolution block in the second and fifth stages contains one ResNet_SPP convolution block, and the CSP convolution block in the third and fourth stages contains three ResNet_SPP convolution blocks; wherein, the feature pyramid FPN consists of the CBM convolution block, the upsampling convolution block and the downsampling convolution block; the head detection head consists of the CBM convolution block and the basic convolution layer;

[0011] Step 5: Use the fabric defect images of the training set annotated and processed in step 2 to train the improved YOLOv5 model, use the fabric defect images of the test set for verification, and use the trained improved YOLOv5 model as the fabric defect category detection and analysis processing model;

[0012] Step 6: Input the fabric image into the fabric defect category detection and analysis processing model in real time, perform online detection and analysis on the fabric image, and output the analysis results.

[0013] Furthermore, in step 1, the plurality of fabric defect images refer to fabric defect images obtained by photographing a plurality of fabrics by setting corresponding camera heights, camera focal lengths and different ambient light levels.

[0014] Furthermore, in step 2, the step of classifying and labeling the fabric defect images in the training set includes: classifying and labeling all fabric defects according to the directions of warp and weft, the difference between foreground and background, and the features of the defects.

[0015] Furthermore, in step 2, the data enhancement processing includes processing the labeled image by translating, rotating, adding noise or cropping defects and fusing them into a defect-free image.

[0016] Furthermore, in step 2, the image after the histogram equalization processing needs to be normalized again.

[0017] Furthermore, in step six, the analysis results include: categories of fabric defects and the number of defects.

[0018] Furthermore, in step six, outputting the analysis results includes transmitting the analysis results to a cloud platform and displaying them.

[0019] The present invention also provides a readable storage medium, which stores multiple instructions, and the instructions are suitable for loading by a processor to execute a method for online detection of fabric defects based on an improved YOLOv5 model.

[0020] The present invention also provides a terminal device, including a processor and a memory, wherein the memory stores multiple instructions, and the processor loads the instructions to execute a method for online detection of fabric defects based on an improved YOLOv5 model.

[0021] Compared to the prior art, the present invention, based on an improved YOLOv5 model, provides a method, readable storage medium, and terminal device for online fabric defect detection. The method first collects sufficient data to construct a dataset, labels the data, and then performs data augmentation to ensure a balanced data volume. A network model is then constructed for training and verification, and finally deployed to the hardware device. The present invention selects CSPDarknet as the model's feature extraction backbone network, optimizing the ResNet_SPP and SPP layers to improve the feature extraction method. FPN is selected as the multi-scale feature extraction network, optimizing the multi-scale feature extraction method to improve the multi-scale feature extraction method. This results in an improved fabric defect detection and analysis processing model, improving the accuracy of online fabric defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 Schematic diagram of the principle flow of the method for online detection of fabric defects based on the improved YOLYOv5 model of the present invention;

[0024] Figure 2 This is the global structure diagram of the improved YOLOv5 model;

[0025] Figure 3 This is the structural block diagram of the CBM convolution block and CSP convolution block in the improved YOLOv5 model;

[0026] Figure 4 for Figure 3 The structural block diagram of the ResNet_SPP module in the CSP module;

[0027] Figure 5 This is a structural diagram of a terminal device for online detection of fabric defects based on an improved YOLYOv5 model of the present invention. DETAILED DESCRIPTION

[0028] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0029] Please refer to Figure 1 As shown, a preferred embodiment of the method for online detection of fabric defects based on the improved YOLYOv5 model of the present invention includes the following steps:

[0030] Step 1: Use an online image acquisition device to pre-collect multiple fabric images to form a data set;

[0031] Step 2: The data set is divided into a training set and a test set, the fabric images in the training set are annotated with multiple categories, and the annotated fabric images are subjected to data enhancement processing;

[0032] Step 3: preprocessing the fabric image after data enhancement processing, wherein the preprocessing includes performing histogram equalization processing on the fabric image;

[0033] Step 4: Build an improved YOLOv5 model, extract features from the input model image through the feature extraction network backbone and feature pyramid FPN, and build a task detection head to detect the extracted features. The feature extraction network backbone consists of Focus convolution block, CBM convolution block and CSP convolution block, wherein the CBM convolution block contains convolution layer, batch normalization layer and Mish activation layer, and the CSP convolution block contains ResNet_SPP convolution block; the .... one consists of five stages, the first stage contains Focus convolution block and CBM convolution block, the second and fifth stages both contain CBM convolution block and CSP convolution block, among which the CSP convolution block in the second and fifth stages contains 1 ResNet_SPP convolution block, and the CSP convolution block in the third and fourth stages contains 3 ResNet_SPP convolution blocks; among them, the feature pyramid FPN consists of CBM convolution block, upsampling and downsampling convolution blocks; the Head detection head consists of CBM convolution block and basic convolution layer.

[0034] Step 5: Use the improved YOLOv5 model obtained in step 4 to train the fabric defect images of the training set preprocessed in step 3 within the Pytorch framework, and use the fabric defect images of the test set for verification to obtain a fabric defect category detection and analysis processing model;

[0035] Step 6: Input the fabric image into the fabric defect category detection and analysis processing model in real time, perform online detection and analysis on the fabric image, and output the analysis results.

[0036] The analysis results include: fabric defect category, defect size, defect aspect ratio, and defect quantity. Outputting the analysis results includes transmitting the analysis results to a cloud platform and displaying them.

[0037] The following is a specific example to further illustrate a method for online detection of fabric defects based on an improved YOLOv5 model of the present invention.

[0038] Example 1

[0039] Please refer to Figure 1 As shown, a method for online detection of fabric defects based on an improved YOLOv5 model of the present invention comprises the following steps:

[0040] Step 1: Image acquisition and dataset construction

[0041] Fabric images are collected on multiple selected actual fabric inspection machines to form a data set. The multiple fabric images refer to fabric defect images obtained by shooting multiple fabrics at different camera heights, camera focal lengths, and ambient light levels.

[0042] Step 2: Dataset Division

[0043] The dataset is divided into a training set and a test set in a ratio of 7:3.

[0044] Step 3: Data annotation and data enhancement

[0045] The fabric defect image data in the training set are labeled by category using labelImg software.

[0046] When marking each fabric defect image, the image needs to be enlarged and marked to prevent missing small target defects or inconspicuous defects.

[0047] The category labeling refers to the detailed classification and labeling of all fabric defects based on the warp and weft directions of the fabric defects, the differences between the foreground and background, and the characteristics of the defects.

[0048] After the images are annotated, a text file of annotation boxes is generated. The annotated images are read through the text-formatted annotation boxes. The annotated images are then translated, rotated, noised, cropped, and fused with defect-free images for data enhancement.

[0049] The data augmentation makes the test data as close as possible to the real data, otherwise the performance of the model may be mistakenly overestimated.

[0050] Step 4: Data Preprocessing

[0051] The fabric images taken on site may appear bright or dark due to the influence of external light, background reflection, or reflection of the fabric itself. Therefore, the fabric images in the training set need to be processed with histogram equalization.

[0052] Histogram equalization is a method to enhance image contrast. Its main idea is to make the histogram distribution of the image more uniform, so that the resulting image will be clearer than the original image.

[0053] Histogram equalization is a transformation based on the probability density function of random variables, which widens the grayscale of the image with more pixels and reduces the grayscale of the image with fewer pixels. First, the normalized grayscale r of the original grayscale k in the image is counted. k Probability of occurrence:

[0054]

[0055] Where L is the gray level of the image, n is the total number of pixels in the image, and n k The gray level in the image is r k The number of pixels, the transformation function is:

[0056]

[0057] The gray level in the image is r k Each pixel value is mapped to a new grayscale s after being corrected by the transformation function k middle.

[0058] After performing histogram equalization on the fabric defect images in the training set, the pixel values ​​in the defect images are normalized (min-max normalization). This will achieve better training performance and faster convergence. The principle is to perform a linear transformation on the original data, mapping the values ​​to the range [0, 1]. The min-max normalization formula is as follows:

[0059] For each pixel x1, x2, ..., x n Perform the transformation:

[0060]

[0061] Then the new data y1, y2, ..., y n ∈[0, 1] and is dimensionless.

[0062] Step 5: Build the algorithm model

[0063] The overall model structure of the improved YOLOv5 is as follows Figure 2 As shown in the figure, the overall network is divided into three parts: backbone, FPN and Head.

[0064] Backbone is divided into the following five stages:

[0065] In the first stage, the input image data passes through the input layer and outputs image data of size 640*640*3. After the channel is expanded through the Focus layer, the output size is 320*320*12. Then, the number of channels is increased through the CBM module, and the image data of 320*320*12 is outputted as image data of size 320*320*64.

[0066] The second stage: The input image data is the output of the first stage. After passing through a CBM module and a CSPLayer module, the width and height are reduced by half and the depth is doubled, that is, the output image data is 160*160*128 in size.

[0067] Phase 3: The input image data of size 160*160*128 is converted to image data of size 80*80*256 through the CBM module and CSPLayer module. The output of the third phase is also recorded as P3_IN, which provides data for subsequent work.

[0068] In the fourth stage, the CBM module and CSPLayer module are used to convert the input image data of size 80*80*256 into image data of size 40*40*512. The output data of the fourth stage is recorded as P4_IN.

[0069] In the fifth stage, the input image data of size 40*40*512 is converted to image data of size 20*20*1024 through the CBM module and the CSPLayer module. The output data of the fifth stage is recorded as P5_IN.

[0070] The FPN is shaped like a sun. The inputs from the three nodes on the left undergo a series of transformations, ultimately outputting images of large, medium, and small scales at the three nodes on the right. First, the output data P5_IN from the fifth backbone stage undergoes a 1x1 convolution, converting the 20x20x1024 image data into a 20x20x512 image data. This data is denoted as P5_MD. P5_MD undergoes an upsampling operation, reducing the width and height from 20x20 to 40x40, while maintaining the number of channels. This upsampled data is concatenated with P4_IN, increasing the number of channels to 1024. This concatenated data then passes through a CSPLayer layer, converting the original 40x40x1024 image data into a 40x40x512 image data. A further 1x1 convolution reduces the number of channels by half. The resulting image data is now 40x40x256, denoted as P4_MD. P4_MD upsamples the feature map to double its size and then concatenates it with the backbone's third-stage output, P3_IN. The concatenated data passes through the CSPlayer layer, resulting in a feature map size of 80*80*256. This output is referred to as P3_OUT, the maximum-scale output. P3_OUT undergoes a downsampling step and concatenates it with P4_MD and P4_IN to a size of 40*40*1024. It then passes through the CSPlayer layer, resulting in a mid-scale output of 40*40*512. P4_OUT undergoes a downsampling step and concatenates it with P5_MD and P5_IN to a size of 20*20*2048. It then passes through the CSPlayer layer, resulting in a minimum-scale output of 20*20*1024.

[0071] The head detection consists of basic convolutional layers. The output of the FPN layer passes through a 1x1 convolutional layer, maintaining the original scale but changing only the number of channels: 80x80x256 becomes 80x80x(3x(5+class)), 40x40x512 becomes 40x40x(3x(5+class)), and 20x20x1024 becomes 20x20x(3x(5+class)). Class refers to the number of categories.

[0072] like Figure 3 As shown in the figure, the CBM convolutional block includes a convolutional layer, a batch normalization layer, and a mixed activation layer, while the CSP convolutional block includes a ResNet_SPP convolutional block. The difference is that the CSPLayer layer in the backbone does not change the channel size, while the CSPLayer layer in the FPN reduces the number of channels by half. The CSP convolutional block will include n ResNet_SPP convolutional blocks added to the original CSP structure.

[0073] like Figure 4 As shown in the figure, the ResNet_SPP convolution block adds an SPP structure to the original ResNet. After SPP, the original data is added to the result of concat and convolution.

[0074] Step 6: Model training and model validation

[0075] The improved YOLOv5 model obtained in Step 5 was trained and validated using the PyTorch framework based on the fabric defect images from the training set preprocessed in Step 4. The Adam optimizer (Adaptive Moment Estimation) was used during model training. Adam combines the Momentum and RMSProp algorithms, using momentum to accumulate gradients, resulting in faster convergence and smaller fluctuations, while also correcting for bias. The learning rate (lr) was set to 0.0003, the number of epochs to 300, and the batch size to 16. The training set was then fed into the improved YOLOv5 model for training. The trained algorithm model was validated using the test set. If it did not meet expectations, adjustments were made and training continued until it met expectations. The trained model was then used as the fabric defect classification detection and analysis model.

[0076] Step 7: Online detection and analysis result output

[0077] Real-time online fabric defect detection and analysis is performed based on a fabric defect classification detection and analysis processing model. Images captured by online image acquisition equipment are fed into the model to calculate and output classification results. The classification results are transmitted to a server, where further analysis results, such as defect size, aspect ratio, and number of defects, are calculated and uploaded to a cloud platform. The cloud platform interface displays this information in real time on a monitor, visualizing it.

[0078] Corresponding to the aforementioned embodiment of a method for online detection of fabric defects based on an improved YOLYOv5 model, the present invention also provides an embodiment of a terminal device for online detection of fabric defects based on an improved YOLYOv5 model.

[0079] See also Figure 5 An embodiment of the present invention provides a terminal device for online detection of fabric defects based on an improved YOLYOv5 model, including a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, it is used to implement a method for online detection of fabric defects based on the improved YOLYOv5 model in the above embodiment.

[0080] An embodiment of a terminal device for online detection of fabric defects based on an improved YOLYOv5 model of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The terminal device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a terminal device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 5 As shown in the figure, it is a hardware structure diagram of a terminal device for online detection of fabric defects based on the improved YOLYOv5 model of the present invention, where any device with data processing capability is located, except Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the terminal device is located in the embodiment may also include other hardware based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0081] The implementation process of the functions and effects of each unit in the above-mentioned terminal device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0082] For the terminal device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The terminal device embodiment described above is only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Ordinary technicians in this field can understand and implement it without paying any creative work.

[0083] An embodiment of the present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method for online detection of fabric defects based on the improved YOLYOv5 model in the above embodiment is implemented.

[0084] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0085] The above embodiments are used to illustrate the present invention rather than to limit the present invention. Any modifications and changes made to the present invention within the spirit of the present invention and the protection scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A method for online detection of fabric defects based on an improved YOLOv5 model, characterized in that: The method comprises the following steps: Step 1: Collect multiple fabric defect images to form a data set; Step 2: Divide the data set into a training set and a test set, label the fabric defect images in the training set by category, and perform data enhancement and histogram equalization on the labeled fabric defect images; Step 3: Build an improved YOLOv5 model, extract features from the input image through the feature extraction network backbone and feature pyramid FPN, and build a task detection head to detect the extracted features. The feature extraction network backbone consists of Focus convolution block, CBM convolution block and CSP convolution block, wherein the CBM convolution block contains convolution layer, batch normalization layer and Mish activation layer, and the CSP convolution block contains ResNet_SPP convolution block; the feature extraction network backbone It consists of five stages. The first stage contains Focus convolution blocks and CBM convolution blocks. Stages 2 to 5 all contain CBM convolution blocks and CSP convolution blocks. The CSP convolution blocks in stages 2 and 5 contain one ResNet_SPP convolution block, and the CSP convolution blocks in stages 3 and 4 contain three ResNet_SPP convolution blocks. The feature pyramid FPN consists of a CBM convolution block, an upsampling convolution block, and a downsampling convolution block. The task detection head consists of a CBM convolution block and a basic convolution layer. Step 5: Use the fabric defect images of the training set annotated and processed in step 2 to train the improved YOLOv5 model, use the fabric defect images of the test set for verification, and use the trained improved YOLOv5 model as the fabric defect category detection and analysis processing model; Step 6: Input the fabric image into the fabric defect category detection and analysis processing model in real time, perform online detection and analysis on the fabric image, and output the analysis results.

2. The method for online detection of fabric defects based on the improved YOLOv5 model according to claim 1, wherein: In step 1, the plurality of fabric defect images are fabric defect images obtained by photographing a plurality of fabrics by setting corresponding camera heights, camera focal lengths, and different ambient light levels.

3. The method for online detection of fabric defects based on the improved YOLOv5 model according to claim 1, wherein: In step 2, the step of classifying and labeling the fabric defect images in the training set includes: classifying and labeling all fabric defects according to the warp and weft directions, the difference between the foreground and background, and the defect features.

4. The method for online detection of fabric defects based on the improved YOLOv5 model according to claim 1, wherein: In step 2, the data enhancement processing includes processing the labeled image by translating, rotating, adding noise or cropping defects and fusing them into a defect-free image.

5. The method for online detection of fabric defects based on the improved YOLOv5 model according to claim 1, wherein: In step 2, the image after histogram equalization processing needs to be normalized again.

6. The method for online detection of fabric defects based on the improved YOLOv5 model according to claim 1, wherein: In step six, the analysis results include: the type of fabric defects and the number of defects.

7. The method for online detection of fabric defects based on the improved YOLOv5 model according to claim 1, wherein: In step six, outputting the analysis results includes transmitting the analysis results to a cloud platform and displaying them.

8. A readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for loading by a processor to execute any one of claims 1 to 7 of the method for online detection of fabric defects based on the improved YOLOv5 model.

9. A terminal device comprising a processor and a memory, characterized in that: The memory stores a plurality of instructions, and the processor loads the instructions to execute any one of claims 1 to 7 of the method for online detection of fabric defects based on the improved YOLOv5 model.

Images