A defect detection method and device based on FabricMosaic and improved fine-grained feature extraction network

Abstract

The present application discloses a defect detection method and device based on FabricMosaic and an improved fine-grained feature extraction network, relating to the technical field of cloth defect detection, and solving the problems of incomplete defect targets and easy missed detection of small target defects caused by the fixed cropping method in the prior art. The method comprises: collecting samples, expanding samples using a dynamic FabricMosaic data enhancement strategy based on target foreground and cloth texture background features, using images in a training set as input to a YOLOv5s network model, obtaining a global fine-grained feature map after feature extraction using a feature fine-grained enhancement extraction module based on a divergent path, training the YOLOv5s network model, using the final YOLOv5s network model to detect cloth defects, and expanding samples using a dynamic FabricMosaic data enhancement strategy. During the sample integration process, the integrity of the defect target can still be effectively retained, thereby reducing the risk of missed detection when detecting small target defects.

Description

Technical Field

[0001] The present application relates to the technical field of cloth defect detection, and in particular to a defect detection method and device based on FabricMosaic and an improved fine-grained feature extraction network. Background Art

[0002] In the textile industry, fabric quality inspection is an essential step and a crucial link in ensuring product quality. Traditional methods for fabric defect detection rely primarily on manual inspection, which is not only time-consuming and labor-intensive but also highly susceptible to subjective factors, leading to inconsistent results. With the rapid advancement of global science and technology, computer vision technology has reached new heights, and automated fabric defect detection systems have gradually become a research hotspot. Existing automated inspection methods are typically based on image processing and machine learning techniques. However, these methods often face challenges when dealing with complex backgrounds, diverse defects, and small object detection. Complex background textures can easily lead to false detections, while the diverse defect morphology and presence of small objects increase the risk of missed detections. Furthermore, traditional feature extraction networks typically use fixed convolution and pooling operations, which can result in the loss of information about subtle defects.

[0003] In order to improve the accuracy and precision of detection, researchers have proposed a variety of improvement strategies. For example, multi-scale feature extraction methods enhance the detection ability of targets by combining features of different scales. However, these methods still have shortcomings when dealing with defects with extreme aspect ratios. In addition, traditional mosaic data enhancement methods fail to fully consider the ratio of positive and negative samples during the sample integration process. This fixed cropping method usually causes the cropped image to mainly contain the texture background area of ​​the negative sample, while the defect foreground information of the positive sample is lost, thereby failing to effectively improve the diversity of the data. Traditional feature extraction networks usually use strided convolution for feature extraction and pooling operations to obtain global information. This method easily leads to the loss of information about small defects in the feature map, thereby increasing the risk of missed detection when detecting small target defects. Summary of the Invention

[0004] The purpose of this application is to overcome the problems of incomplete defect targets and easy omission of small target defects caused by the fixed cropping method used in the prior art, and to provide a defect detection method and device based on FabricMosaic and an improved fine-grained feature extraction network.

[0005] First, a defect detection method based on FabricMosaic and an improved fine-grained feature extraction network is provided, including:

[0006] S100, obtaining cloth images and establishing a sample data set;

[0007] S200, using a dynamic FabricMosaic data augmentation strategy based on target foreground and cloth texture background features to perform sample expansion to obtain an expanded data set. During the sample expansion process, the cropping center offset point is dynamically adjusted based on the prior information of the morphological characteristics of the defect target and the cloth texture background features to ensure the integrity of the defect target;

[0008] S300, dividing the expanded data set into a training set, a validation set, and a test set;

[0009] S400, using an image in the training set as input to the YOLOv5s network model, and obtaining a global fine-grained feature map after feature extraction by a divergent path-based fine-grained feature enhancement extraction module, wherein the divergent path-based fine-grained feature enhancement extraction module includes an information sending path for shunting input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the shunted features;

[0010] S500, for the global fine-grained feature map, predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0011] S600, calculating a network loss value according to the bounding box and the GT box of the corresponding image, and using a gradient descent method to update the parameters of the YOLOv5s network model;

[0012] S700, repeat steps S400-S600 until all images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set based on the YOLOv5s network model with updated parameters, and output the AP value of each category in the validation set after statistics; iterate until the statistical mAP value stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0013] S800, using the test set to test the trained YOLOv5s network model, and adjusting the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0014] S900, uses the final YOLOv5s network model to detect cloth defects.

[0015] In some possible implementations, the pruning method in the sample expansion process includes:

[0016] Calculate the defect target pixel area;

[0017] Dynamically adjust the movable range of the offset center point based on the background ratio of the defect with the largest pixel count;

[0018] Adjust the center point of the cropping area according to the target offset, then calculate the distance vector of each defect target relative to the cropping center point. Based on the distribution of defect targets in the four quadrants of the cropping area, dynamically adjust the boundaries of the cropping area to ensure that all defect targets are contained in the cropping area;

[0019] Ensure that the cropped area does not exceed the image boundary according to the constraint formula;

[0020] The width and height of the cropped area are scaled proportionally so that the cropped area matches the original area.

[0021] In some possible implementations, after calculating the pixel area of ​​the defect target, the defect target is divided into four categories according to the pixel area using the KMeans clustering algorithm. The clustering goal is to minimize the sum of squares of intra-class differences, which is calculated as follows:

[0022] A=w·W·h·H;

[0023]

[0024] Where A is the pixel area of ​​the defect target, W is the width of the given image, H is the height of the given image, the relative coordinates of the bounding box bbox=(x,y,w,h), ν k It is cluster C k The average pixel area, A i is the individual area value in the cluster.

[0025] In some possible implementations, the formula for calculating the movable range of the offset center point is:

[0026]

[0027] Where: C w C is the width of the movable range of the offset center point. h The height of the movable range of the offset center point, is the maximum pixel amount of defects after clustering, To adjust the offset, St is the detection image size, and t is the total number of classes.

[0028] In some possible implementations, the center point of the cropping area is adjusted according to the target offset. The offset of the center of the cropping area is flexibly offset according to the distribution of the defect targets:

[0029] Δx=x c -x o ;

[0030] Δy=y c -y o ;

[0031] Among them, the center point of the dynamic cropping area is (x c ,y c ), The center point of the original cropping area is (x o ,y o ), the offset of the cropping area center is flexibly offset according to the distribution of defect targets (Δx, Δy);

[0032] Calculate the distance vector of each defect target relative to the clipping center point. For each defect target, its constraint boundary reference point is (xn s ,yn s ), the reference points of each quadrant constraint boundary are (xn l ,yn l ), where n is the corresponding quadrant number, then the distance vector (d x ,d y )for:

[0033] d x =xn s -xn l ;

[0034] d y =yn s -yn l ;

[0035] According to the distribution of defect targets in the four quadrants of the cropping area, the boundaries of the cropping area are dynamically adjusted to ensure that all defect targets are included in the cropping area. The boundary adjustment of each quadrant is expanded according to the distribution of defect targets. (xb1 new :xb2 new ,yb1 new :yb2 new ) is the set of coordinate offsets after adjustment for each quadrant, then:

[0036] Upper left quadrant (d x >0andd y >0):

[0037] x1b new =min(xn s ,xn l );

[0038] y1b new =min(yn s ,yn l );

[0039] Upper right quadrant (d x <0andd y >0):

[0040] x2b new =max(xn s ,xn l );

[0041] y1b new =min(yn s ,yn l );

[0042] Lower left quadrant (d x >0andd y <0):

[0043] x1b new =min(xn s ,xn l );

[0044] y2b new =max(yn s ,yn l );

[0045] Lower right quadrant (d x <0andd y <0):

[0046] x2b new =max(xn s ,xn l );

[0047] y2b new =max(yn s ,yn l ).

[0048] In some possible implementations, the constraint formula is:

[0049] x1b new =max(0,x1b new );y1b new =max(0,y1b new );

[0050] x2b new =min(img w ,x2b new );y2b new =min(img h ,y2b new ).

[0051] Some possible implementations involve scaling the width and height of the cropped region, including:

[0052] Calculate the area A of the original clipping area originaland the adjusted new area A new :

[0053] A original =(x2b-x1b)·(y2b-y1b);

[0054] A new =(x2b new -x1b new )·(y2b new -y1b new )

[0055] If the new area after adjustment is A new Larger than the area A of the original cropped area original , scale the crop region as follows to maintain the original area:

[0056]

[0057] new_width=(x2b new -x1b new )·scale_factor;

[0058] new_height=(y2b new -y1b new )·scale_factor;

[0059]

[0060]

[0061] In a second aspect, a defect detection device based on FabricMosaic and an improved fine-grained feature extraction network is provided, using the method described in the first aspect, wherein the system includes:

[0062] Acquisition module, used to acquire cloth images and build sample data sets;

[0063] The sample expansion module is used to expand samples using a dynamic FabricMosaic data augmentation strategy based on the target foreground and cloth texture background features to obtain an expanded dataset. During the sample expansion process, the cropping center offset point is dynamically adjusted based on the morphological characteristics of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target.

[0064] A partitioning module, configured to partition the expanded data set into a training set, a validation set, and a test set;

[0065] A feature extraction module is configured to use images in the training set as input to the YOLOv5s network model and obtain a global fine-grained feature map after feature extraction by a divergent path-based fine-grained feature enhancement extraction module. The divergent path-based fine-grained feature enhancement extraction module includes an information sending path for shunting input features according to feature channels and a feature extraction unit for extracting fine-grained features from the shunted features.

[0066] The bounding box generation module is used to predict the probability that each pixel in the global fine-grained feature map belongs to the object to be detected and the bounding box information of the object, and generate the bounding box;

[0067] A parameter update module is used to calculate the network loss value based on the bounding box and the GT box of the corresponding image, and to update the parameters of the YOLOv5s network model using the gradient descent method;

[0068] The training module is used to repeatedly execute the feature extraction module, bounding box generation module, and parameter update module until all images in the training set are input into the YOLOv5s network model at least once; the YOLOv5s network model with updated parameters is used to predict each image in the validation set, and the AP value of each category in the validation set is output after statistics; the loop is iterated until the statistical mAP value stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0069] The testing module is used to test the trained YOLOv5s network model using the test set and adjust the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0070] The detection module is used to detect cloth defects using the final YOLOv5s network model.

[0071] In a third aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0072] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores program code for execution by a device, the program code including steps for executing the method in any one of the implementations of the first aspect above.

[0073] In a fifth aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements a method as in any one of the implementations in the first aspect above.

[0074] This application has the following beneficial effects:

[0075] 1. This application adopts a dynamic FabricMosaic data augmentation strategy to expand samples, which can effectively preserve the integrity of defect targets during the sample integration process. Based on the morphological characteristics of the defect targets and prior information about the background, the cropping center offset point can be dynamically adjusted. In the case of a serious imbalance in the ratio of positive and negative samples, the data diversity is effectively improved, and the integrity of the pixel information and features of these special targets with extreme aspect ratio defects is guaranteed.

[0076] 2. This application uses a divergent path feature fine-grained enhancement extraction module (DPFEM) for feature extraction, which is not easy to lose the information of tiny defects in the feature map, and can effectively reduce the risk of missed detection when detecting small target defects. Secondly, through the information divergence path, the input features are diverted according to the feature channels, which can not only retain the original feature information while improving the feature extraction efficiency, avoiding the problem of gradient disappearance during model training, but also fully retain the fine-grained features, greatly improving the model's ability to detect tiny defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The drawings that constitute a part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application.

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0079] Figure 1 This is a flow chart of the defect detection method based on FabricMosaic and an improved fine-grained feature extraction network according to Example 1 of the present application;

[0080] Figure 2 Schematic diagram of pixel ratio distribution of clustered image data in the defect detection method based on FabricMosaic and improved fine-grained feature extraction network in Example 1 of the present application;

[0081] Figure 3 Schematic diagram of calculating the coordinates of the center point and the movable range of the offset center point in the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network in Example 1 of the present application;

[0082] Figure 4 This is a schematic diagram of parsing quadrants and adjusting cropping ranges in the defect detection method based on FabricMosaic and an improved fine-grained feature extraction network in Example 1 of the present application;

[0083] Figure 5 4 is a schematic diagram of the adjusted cropping area in the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network in Example 1 of the present application;

[0084] Figure 6 1 is a structural diagram of a feature fine-grained enhancement extraction module of a divergent path in a defect detection method based on FabricMosaic and an improved fine-grained feature extraction network according to Example 1 of the present application;

[0085] Figure 7 This is a comparison chart of the mAP of the improved model of the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network and the baseline model in identifying different types of defects in Example 1 of the present application;

[0086] Figure 8 This is a structural block diagram of a defect detection device based on FabricMosaic and an improved fine-grained feature extraction network according to Example 2 of the present application;

[0087] Figure 9 This is a schematic diagram of the internal structure of the electronic device of Example 4 of the present application.

[0088] Reference numerals:

[0089] 100, acquisition module; 200, sample expansion module; 300, division module; 400, feature extraction module; 500, bounding box generation module; 600, parameter update module; 700, training module; 800, testing module; 900, detection module. DETAILED DESCRIPTION

[0090] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0091] Example 1

[0092] like Figure 1 As shown, the defect detection method based on FabricMosaic and an improved fine-grained feature extraction network involved in Example 1 of the present application includes:

[0093] S100: Acquire cloth images and establish a sample data set, wherein the acquired cloth images are captured by devices such as a camera, a still camera, or a webcam.

[0094] S200, using a dynamic FabricMosaic data augmentation strategy based on target foreground and cloth texture background features to perform sample expansion to obtain an expanded data set. During the sample expansion process, the cropping center offset point is dynamically adjusted based on the prior information of the morphological characteristics of the defect target and the cloth texture background features to ensure the integrity of the defect target;

[0095] like Figure 2-Figure 5 As shown in the figure, in the dynamic mosaic data augmentation strategy, the following steps are performed: first, the pixel area of ​​defect targets is calculated and clustered. Using the KMeans clustering algorithm, defect targets are classified into four categories based on pixel area. Second, the center offset of the clustered defects is calculated. Specifically, the movable range of the offset center point is dynamically adjusted based on the background ratio of the defect class with the largest pixel count. Third, dynamic cropping region analysis and adjustment are performed. First, the center point of the cropping region is adjusted based on the target offset. Then, the distance vector of each defect target relative to the cropping center point is calculated. Then, based on the distribution of the targets in the four quadrants of the cropping region, the boundaries of the cropping region are dynamically adjusted to ensure that all targets are contained within the cropping region. Third, the cropping region is ensured to not exceed the image boundaries according to relevant constraint formulas. Finally, the cropping region area is adjusted to maintain its original value. This operation not only ensures that the center point of the cropping region remains in the same relative position, but also scales the width and height of the cropping region to keep its area consistent with the original area, thus maintaining data augmentation consistency.

[0096] Specifically:

[0097] S201、 Figure 2 As shown in the figure, defect target pixel area calculation and cluster analysis. The pixel area A of the defect target is calculated using the width and height of the target bounding box. Given the width W and height H of the image, and the relative coordinates of the bounding box bbox = (x, y, w, h), the pixel area A of the defect target is shown as follows. Subsequently, the KMeans clustering algorithm is used to classify the defect targets into four categories based on pixel area. The clustering goal is to minimize the sum of squared differences within the class, as shown in the following formula:

[0098] A=w·W·h·H

[0099]

[0100] In the above formula: ν k It is cluster C k The average pixel area, A iis the individual area value in the cluster. Through this clustering method, defect targets of different sizes can be effectively distinguished, providing a basis for subsequent cropping and enhancement.

[0101] S202, such as Figure 3 As shown in the figure, the center offset after clustering is calculated. The movable range of the offset center point is dynamically adjusted according to the background ratio of the defect class with the largest pixel count. The formula for calculating the movable range of the offset center point is as follows:

[0102]

[0103] In the above formula: C w and C h The width and height of the movable range of the offset center point; is the maximum pixel count of defects after clustering; is the adjustment offset; St is the detection image size; t is the total number of classes.

[0104] S203, such as Figure 4 As shown in the figure, the dynamic cropping area is analyzed and adjusted. In the dynamic cropping process, the center point of the cropping area is first adjusted according to the target offset. The center point of the dynamic cropping area is set to (x c ,y c ),in The center point of the original cropping area is (x o ,y o The target's offset (Δx, Δy) is given by the following formula. With this offset, the center of the cropping area is flexibly shifted according to the target's distribution:

[0105] Δx=|x c -x o |;

[0106] Δy=|y c -y o |

[0107] Secondly, the distance vector of each defect target relative to the cropping center point is calculated to evaluate whether the target is reasonably located within the cropping area, thereby guiding the adjustment offset of the defect target position. For each defect target, its constraint boundary reference point is (xn s ,yn s ), the reference points of each quadrant constraint boundary are (xn l ,yn l ), where n is the corresponding quadrant number. Then the distance vector (d x ,d y ) is shown in the following formula:

[0108] d x =xn s-xn l ;

[0109] d y =yn s -yn l

[0110] Afterwards, the boundaries of the cropping region are dynamically adjusted based on the distribution of the targets in the four quadrants of the cropping region to ensure that all targets are included in the cropping region. The adjusted cropping region is as follows: Figure 5 As shown. The boundary adjustment of each quadrant is flexibly expanded according to the target distribution to adapt to the position and size distribution of different targets. Suppose the coordinate offset set of each quadrant after adjustment is (x1b new :x2b new ,y1b new :y2b new ),but

[0111] a. Upper left quadrant (d x >0andd y >0):

[0112] x1b new =min(xn s ,xn l );

[0113] y1b new =min(yn s ,yn l )

[0114] b. Upper right quadrant (d x <0andd y >0):

[0115] x2b new =max(xn s ,xn l );

[0116] y1b new =min(yn s ,yn l )

[0117] c. Lower left quadrant (d x >0andd y <0):

[0118] x1b new =min(xn s ,xn l );

[0119] y2b new =max(yn s ,yn l )

[0120] d. Lower right quadrant (d x <0andd y <0):

[0121] x2b new =max(xn s ,xn l );

[0122] y2b new =max(yn s ,yn l )

[0123] S204: To ensure that the cropped area does not exceed the image boundary, the cropped area is limited to the image range using the following constraint formula. This constraint ensures that the cropped image maintains consistency and integrity before and after data augmentation. The boundary constraint conditions for image cropping are as follows:

[0124] x1b new =max(0,x1b new );y1b new =max(0,y1b new );

[0125] x2b new =min(img w ,x2b new );y2b new =min(img h ,y2b new )

[0126] S205, the area of ​​the cropped area is adjusted to maintain the original value:

[0127] Calculate the area A of the original clipping area original and the adjusted new area A new :

[0128] A original =(x2b-x1b)·(y2b-y1b);

[0129] A new =(x2b new -x1b new )·(y2b new -y1b new )

[0130] If A new Greater than A original , scale the crop region as follows to maintain the original area:

[0131]

[0132] new_width=(x2b new -x1b new ) scale_factor

[0133] new_height=(y2b new -y1b new ) scale_factor

[0134]

[0135]

[0136] In this embodiment, the cropped area can be ensured to maintain its original area after adjustment, while the center point of the cropped area remains in the relative position. By scaling the width and height of the cropped area so that its area is consistent with the original area, the consistency of data enhancement is maintained.

[0137] S300, dividing the expanded data set into a training set, a validation set, and a test set, for example, dividing the image data into a training set, a validation set, and a test set in a ratio of 7:2:1;

[0138] S400, using an image in the training set as input to the YOLOv5s network model, and obtaining a global fine-grained feature map after feature extraction by a divergent path-based fine-grained feature enhancement extraction module, wherein the divergent path-based fine-grained feature enhancement extraction module includes an information sending path for shunting input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the shunted features;

[0139] like Figure 6 As shown, the feature fine-grained enhancement extraction module based on the divergent path includes: an information divergent path for diverting input features according to feature channels and a feature extraction unit for extracting fine-grained features from the features after diversion processing.

[0140] like Figure 6As shown in the figure, the divergent path-based fine-grained feature enhancement extraction module uses the information divergence path to divert input features according to feature channels to improve feature extraction efficiency. Some features are directly downsampled using DS convolution (Depthwise Separable Convolution) to preserve the original feature information and avoid the gradient vanishing problem during model training. Other features are extracted using SDFI (Spatial-Depth Fission Intensifier), ensuring that fine-grained features are fully preserved, thereby improving the model's ability to detect minor defects.

[0141] Specifically:

[0142] like Figure 6 As shown in Figure 1, combined with the idea of ​​SPDConv (Space-to-Depth Convolution), the SDFI module reduces the information loss in the process of small target feature extraction by spatial fission. Specifically, SDFI first reorganizes the input feature information. Assuming the size of the original input feature map G is (W, H, C), and the downsampling step size is St, the original feature sequence is split into St through the spatial fission operation. 2 A spatial fission feature G′, whose size is Next, the divergent feature subsequences are reorganized by channel aggregation to form the aggregated channel feature G″. This process reduces the spatial scale while increasing the channel dimension, ensuring that the total feature elements of the cloth image information are preserved. The expression for the deep sequence reorganization from space to channel is as follows:

[0143]

[0144] In the above formula: a, b are the starting index; S is the ending index; St is the sampling step; Cat[,1] is the sub-feature tensor splicing operation in the channel dimension; G″′ is the global fine-grained feature output by SDFI.

[0145] like Figure 6 As shown, after the above reorganization operation, the size of the new channel aggregation feature G″ is Transformed into On the basis of retaining the fine-grained information of the cloth image, the feature space size is compressed. Subsequently, the depthwise separable convolution (DS convolution) is used to further extract feature information. Specifically, a DW (Depthwise) extractor with multiple independent k×k convolution kernels is first used to perform a convolution operation on each input channel. Then, a PW (Pointwise) extractor is used to perform point convolution on the channel dimension to achieve cross-channel feature projection and fusion. The feature extraction process output by the SDFI module is shown in the following formula:

[0146] G″′=PWConv(DWConv(G″))

[0147] In the above formula: G″′ is the global fine-grained feature output by SDFI; PWConv is the PW extractor; DWConv is the DW extractor.

[0148] like Figure 6 As shown in the figure, the DW extractor and the PW extractor together form the DS convolution, which can effectively retain feature information while improving computational efficiency, thereby maintaining good detection performance. The proportional relationship between the computational effort of the DS convolution and that of the traditional convolution is shown in the formula:

[0149]

[0150] In the above formula: Fl db The ratio of the computational complexity of ordinary convolution to that of DW convolution; Flops NorConv is the amount of computation for ordinary convolution; Flops DWConv is the computational cost of DW convolution; k is the size of the convolution kernel. This comparison of computational costs shows the advantage of DW convolution in computational efficiency, especially when the number of channels is large.

[0151] S500, for the global fine-grained feature map, predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0152] S600: Calculate the network loss value based on the bounding box and the GT box (i.e., the Ground Truth box, which refers to the correct target detection box manually annotated) of the corresponding image, and use the gradient descent method to update the parameters of the YOLOv5s network model;

[0153] Specifically, the intersection-over-union loss is calculated based on the bounding box and the corresponding GT box, and the classification confidence loss and border confidence loss are calculated based on the classification confidence and border confidence of each predicted box contained in the network output feature map. The intersection-over-union loss, classification confidence loss, and border confidence loss are weighted and summed in a preset ratio to obtain the overall network loss, and backpropagation is performed to update the network parameters.

[0154] S700, repeat steps S400-S600 until all images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set based on the YOLOv5s network model with updated parameters, and output the AP value of each category in the validation set after statistics; iterate until the statistical mAP value stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0155] S800, using the test set to test the trained YOLOv5s network model, and adjusting the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0156] S900, uses the final YOLOv5s network model to detect cloth defects.

[0157] To verify the performance of the method proposed in this embodiment, the YOLOv5s network was also used to predict images in the test set, and a comparative analysis was performed with the YOLOv5s+fabric_mosaic and YOLOv5s+DPFEM of this application. Table 1 below shows the effectiveness analysis table of FabricMosaic provided in this embodiment, and Table 2 below shows the effectiveness analysis table of the divergent path feature enhancement module provided in this embodiment. As can be seen from Tables 1 and 2, FabricMosaic of this application effectively preserves the integrity of defective targets by dynamically adjusting the cropping area and optimizing the boundaries, improving data diversity and model detection performance. Its accuracy increased by 16.3%, and the F1 value increased to 69.1%, without increasing the complexity of the model. The dynamic cropping strategy based on target features and flexible boundary constraints solve the problem of losing positive sample information in traditional methods, demonstrating excellent detection accuracy and robustness.

[0158] Table 1:

[0159]

[0160] Table 2:

[0161]

[0162] like Figure 7The figure below compares the use of a baseline model (i.e., YOLOv5s) and the improved model of this embodiment (i.e., YOLOv5s+fabric_mosaic+DPFEM) in identifying different types of defects. It can be seen that the DPFEM of this embodiment enhances the ability to extract small target features through information diversion and spatial fission operations, significantly improving detection performance. Compared with the baseline model, the YOLOv5s+DPFEM model has achieved varying degrees of improvement in key evaluation indicators such as Precision, Recall, mAP@0.5, and F1 value. In particular, mAP@0.5 has increased by 1.8 percentage points, demonstrating a significant advantage in the small target defect detection task. It uses depthwise separable convolution (DS convolution) to optimize computational efficiency, retain fine-grained features, improve the ability to capture complex features and the detection rate of small target defects, and demonstrate excellent detection results and good robustness.

[0163] Example 2

[0164] like Figure 8 As shown, the defect detection device based on FabricMosaic and an improved fine-grained feature extraction network involved in Example 2 of the present application includes:

[0165] An acquisition module 100 is used to acquire cloth images and establish a sample data set;

[0166] The sample expansion module 200 is used to expand the sample using a dynamic FabricMosaic data augmentation strategy based on the target foreground and cloth texture background features to obtain an expanded data set. During the cropping process of the sample expansion, the cropping center offset point is dynamically adjusted based on the morphological characteristics of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target;

[0167] A partitioning module 300 is configured to partition the expanded data set into a training set, a validation set, and a test set;

[0168] A feature extraction module 400 is configured to use images in the training set as input to the YOLOv5s network model and obtain a global fine-grained feature map after feature extraction by a divergent path-based fine-grained feature enhancement extraction module. The divergent path-based fine-grained feature enhancement extraction module includes an information transmission path for shunting input features according to feature channels and a feature extraction unit for extracting fine-grained features from the shunted features.

[0169] The bounding box generation module 500 is used to predict the probability that each pixel in the global fine-grained feature map belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0170] A parameter updating module 600 is configured to calculate a network loss value based on the bounding box and the ground truth box (i.e., the manually annotated correct object detection box) of the corresponding image, and update the parameters of the YOLOv5s network model using a gradient descent method;

[0171] The training module 700 is configured to repeatedly execute the feature extraction module, the bounding box generation module, and the parameter update module until all images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set based on the parameter-updated YOLOv5s network model, and output the AP value of each category in the validation set after statistics; and iterate until the statistical mAP value stabilizes at a certain value to obtain a trained YOLOv5s network model.

[0172] The testing module 800 is used to test the trained YOLOv5s network model using the test set and adjust the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0173] The detection module 900 is used to detect cloth defects using the final YOLOv5s network model.

[0174] It should be noted that other specific implementations of the defect detection device based on FabricMosaic and the improved fine-grained feature extraction network in this embodiment can be found in the specific implementations of the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network mentioned above. To avoid redundancy, they will not be repeated here.

[0175] Example 3

[0176] A computer-readable storage medium according to embodiment 3 of the present application, wherein the computer-readable storage medium stores program code for execution by a device, the program code including steps for executing the method in any one of the implementations in embodiment 1 of the present application;

[0177] Among them, the computer-readable storage medium can be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM); the computer-readable storage medium can store program code, and when the program stored in the computer-readable storage medium is executed by the processor, the processor is used to execute the steps of the method in any one of the implementation methods in Example 1 of the present application.

[0178] Example 4

[0179] like Figure 9As shown, an electronic device involved in Example 4 of the present application includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the method in any one of the implementations in Example 1 of the present application;

[0180] Among them, the processor can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the method in any one of the implementation methods in Example 1 of the present application.

[0181] The processor may also be an integrated circuit electronic device with signal processing capabilities. In the implementation process, each step of the method in any one of the implementations in Example 1 of the present application may be completed by hardware integrated logic circuits in the processor or software instructions.

[0182] The above-mentioned processor can also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in combination with its hardware, completes the functions required to be executed by the units included in the data processing device of the embodiment of the present application, or executes the method in any one of the implementation modes in Example 1 of the present application.

[0183] The above are only preferred specific implementations of this application; however, the scope of protection of this application is not limited thereto. Any person skilled in the art who, within the technical scope disclosed in this application, makes equivalent substitutions or modifications based on the technical solutions and improved concepts of this application shall be covered by the scope of protection of this application.

Claims

1. A defect detection method based on FabricMosaic and an improved fine-grained feature extraction network, characterized in that: include: S100, obtaining cloth images and establishing a sample data set; S200, using a dynamic FabricMosaic data enhancement strategy based on target foreground and cloth texture background features to perform sample expansion to obtain an expanded data set. During the cropping process of the sample expansion, the cropping center offset point is dynamically adjusted according to the morphological characteristics of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target. The cropping method in the sample expansion process includes: calculating the pixel area of ​​the defect target; dynamically adjusting the movable range of the offset center point according to the background ratio of the defect class with the largest pixel volume; adjusting the center point of the cropping area according to the target offset, and then calculating the distance vector of each defect target relative to the cropping center point; dynamically adjusting the boundary of the cropping area according to the distribution of the defect targets in the four quadrants of the cropping area to ensure that all defect targets are included in the cropping area; ensuring that the cropping area does not exceed the image boundary according to the constraint formula; and scaling the width and height of the cropping area so that the cropped area is consistent with the original area; S300, dividing the expanded data set into a training set, a validation set, and a test set; S400, using an image in the training set as input to the YOLOv5s network model, and obtaining a global fine-grained feature map after feature extraction by a divergent path-based fine-grained feature enhancement extraction module, wherein the divergent path-based fine-grained feature enhancement extraction module includes an information sending path for shunting input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the shunted features; S500, for the global fine-grained feature map, predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and generate a bounding box; S600, calculating a network loss value according to the bounding box and the GT box of the corresponding image, and using a gradient descent method to update the parameters of the YOLOv5s network model; S700, repeat steps S400-S600 until all images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set based on the YOLOv5s network model with updated parameters, and output the AP value of each category in the validation set after statistics; iterate until the statistical mAP value stabilizes at a certain value to obtain a trained YOLOv5s network model; S800, using the test set to test the trained YOLOv5s network model, and adjusting the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model; S900, uses the final YOLOv5s network model to detect cloth defects.

2. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 1 is characterized in that: After calculating the pixel area of ​​the defect target, the KMeans clustering algorithm is used to divide the defect target into four categories according to the pixel area. The clustering goal is to minimize the sum of squares of intra-class differences. The calculation formula is: ； ； in, is the pixel area of ​​the defect target, is the width of a given image, The relative coordinates of the bounding box for a given image height , It is a cluster The mean pixel area, is the individual area value in the cluster.

3. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 2, characterized in that: The formula for calculating the movable range of the offset center point is: ； Where: The width of the movable range of the offset center point, The height of the movable range of the offset center point, is the maximum pixel amount of defects after clustering, To adjust the offset, To detect the image size, is the total number of classes.

4. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 3 is characterized in that: Adjust the center point of the cropping area according to the target offset. The offset of the cropping area center is flexibly offset according to the distribution of defect targets: ； ； Among them, the center point of the dynamic cropping area is , ; The center point of the original cropping area is The offset of the cropping area center is flexibly offset according to the distribution of defect targets. ; Calculate the distance vector of each defect target relative to the clipping center point. For each defect target, its constraint boundary reference point is , the reference points of each quadrant constraint boundary are ,in is the corresponding quadrant number, then the distance vector of the target constraint point relative to the reference point of each quadrant constraint boundary is for: ； ； According to the distribution of defect targets in the four quadrants of the cropping area, the boundaries of the cropping area are dynamically adjusted to ensure that all defect targets are included in the cropping area. The boundary adjustment of each quadrant is expanded according to the distribution of defect targets. is the adjusted coordinate offset set of each quadrant, then: Upper left quadrant : ； ； upper right quadrant : ； ； Lower left quadrant : ； ； lower right quadrant : ； 。 5. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 4 is characterized in that: The constraint formula is: ； 。 6. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 5, characterized in that: By scaling the width and height of the cropped area proportionally, including: Calculate the area of ​​the original clipping region and the new area after adjustment : ； ； If the new area after adjustment An area larger than the original clipping area , scale the crop region as follows to maintain the original area: ； ； ； ； 。 7. A defect detection system based on FabricMosaic and an improved fine-grained feature extraction network, characterized in that: According to any one of claims 1 to 6, the system comprises: Acquisition module, used to acquire cloth images and build sample data sets; The sample expansion module is used to expand the sample by adopting a dynamic FabricMosaic data enhancement strategy based on the target foreground and cloth texture background features to obtain an expanded data set. In the cropping process of the sample expansion, the cropping center offset point is dynamically adjusted according to the morphological characteristics of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target. The cropping method in the sample expansion process includes: calculating the pixel area of ​​the defect target; dynamically adjusting the movable range of the offset center point according to the background ratio of the defect class with the largest pixel volume; adjusting the center point of the cropping area according to the target offset, and then calculating the distance vector of each defect target relative to the cropping center point; dynamically adjusting the boundary of the cropping area according to the distribution of the defect target in the four quadrants of the cropping area to ensure that all defect targets are included in the cropping area; ensuring that the cropping area does not exceed the image boundary according to the constraint formula; and scaling the width and height of the cropping area to make the cropped area consistent with the original area. A partitioning module, configured to partition the expanded data set into a training set, a validation set, and a test set; A feature extraction module is configured to use images in the training set as input to the YOLOv5s network model and obtain a global fine-grained feature map after feature extraction by a divergent path-based fine-grained feature enhancement extraction module. The divergent path-based fine-grained feature enhancement extraction module includes an information sending path for shunting input features according to feature channels and a feature extraction unit for extracting fine-grained features from the shunted features. The bounding box generation module is used to predict the probability that each pixel in the global fine-grained feature map belongs to the object to be detected and the bounding box information of the object, and generate the bounding box; A parameter update module is used to calculate the network loss value based on the bounding box and the GT box of the corresponding image, and to update the parameters of the YOLOv5s network model using the gradient descent method; The training module is used to repeatedly execute the feature extraction module, bounding box generation module, and parameter update module until all images in the training set are input into the YOLOv5s network model at least once; the YOLOv5s network model with updated parameters is used to predict each image in the validation set, and the AP value of each category in the validation set is output after statistics; the loop is iterated until the statistical mAP value stabilizes at a certain value to obtain a trained YOLOv5s network model; The testing module is used to test the trained YOLOv5s network model using the test set and adjust the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model; The detection module is used to detect cloth defects using the final YOLOv5s network model.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program codes for execution by a device, wherein the program codes include steps for executing the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the method according to any one of claims 1 to 6.

Images