Flaw detection method and device based on FabricMosaic and improved fine-grained feature extraction network

Abstract

The invention discloses a flaw detection method and device based on FabricMosaic and an improved fine-grained feature extraction network, relates to the technical field of cloth flaw detection, and solves the problems that in the prior art, a flaw target is incomplete and small target flaws are prone to missing detection due to the fact that a fixed cutting mode is adopted. A dynamic FabricMosaic data enhancement strategy based on target foreground and cloth texture background features is adopted to carry out sample expansion, images in a training set serve as input of a YOLOv5s network model, feature extraction is carried out through a feature fine-grained enhancement extraction module based on a divergent path to obtain a global fine-grained feature map, the YOLOv5s network model is trained, and a target foreground and cloth texture background feature is obtained. The final YOLOv5s network model is used for cloth flaw detection, a dynamic FabricMosaic data enhancement strategy is used for sample expansion, the integrity of a flaw target can still be effectively reserved in the sample integration process, and the risk of missing detection during small target flaw detection is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of cloth defect detection, and in particular to a defect detection method and device based on FabricMosaic and an improved fine-grained feature extraction network. Background Art

[0002] In the textile industry, cloth quality inspection is an essential step and a crucial link to ensure product quality. Traditional cloth defect detection methods mainly rely on manual inspection, which not only takes a lot of time and effort, but is also very susceptible to human subjective factors, resulting in inconsistent detection results. In contemporary times, with the rapid development of global science and technology, the development of computer vision technology has also reached a new height, and automated cloth defect detection systems have gradually become a research hotspot. Existing automated detection methods are usually based on image processing and machine learning techniques. However, these methods often face many challenges when dealing with complex backgrounds, diverse defects, and small target detection. Complex background textures are prone to false detection, while the presence of diverse defect forms and small targets increases the risk of missed detection. In addition, traditional feature extraction networks usually use fixed convolution and pooling operations, which may lead to the loss of subtle defect information.

[0003] In order to improve the accuracy and precision of detection, researchers have proposed a variety of improvement strategies. For example, multi-scale feature extraction methods enhance the detection ability of targets by combining features of different scales. However, these methods still have shortcomings when dealing with defects with extreme aspect ratios. In addition, traditional mosaic data enhancement methods fail to fully consider the ratio of positive and negative samples during sample integration. This fixed cropping method usually causes the cropped image to mainly contain the texture background area of ​​the negative sample, while the defect foreground information of the positive sample is lost, thereby failing to effectively improve the diversity of the data. Traditional feature extraction networks usually use strided convolution for feature extraction and pooling operations to obtain global information. This method easily leads to the loss of information about small defects in the feature map, thereby increasing the risk of missed detection when detecting small target defects. Summary of the invention

[0004] The purpose of this application is to overcome the problems of incomplete defect targets and easy omission of small target defects caused by the fixed cropping method used in the prior art, and to provide a defect detection method and device based on FabricMosaic and an improved fine-grained feature extraction network.

[0005] In the first aspect, a defect detection method based on FabricMosaic and an improved fine-grained feature extraction network is provided, comprising:

[0006] S100, acquiring cloth images and establishing a sample data set;

[0007] S200, using a dynamic FabricMosaic data enhancement strategy based on target foreground and cloth texture background features to expand samples to obtain an expanded data set. In the cropping of the sample expansion process, according to the morphological features of the defect target and the prior information of the cloth texture background features, the cropping center offset point is dynamically adjusted to ensure the integrity of the defect target;

[0008] S300, dividing the expanded data set into a training set, a validation set and a test set;

[0009] S400, using the image in the training set as the input of the YOLOv5s network model, and obtaining a global fine-grained feature map after feature extraction by a feature fine-grained enhancement extraction module based on a divergent path, wherein the feature fine-grained enhancement extraction module based on a divergent path includes an information sending path for shunting the input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the features after the shunting process;

[0010] S500, for the global fine-grained feature map, predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0011] S600, calculating a network loss value according to the bounding box and the GT box of the corresponding image, and using a gradient descent method to update the parameters of the YOLOv5s network model;

[0012] S700, repeating steps S400-S600 until all images in the training set are input into the YOLOv5s network model at least once; predicting each image in the validation set according to the YOLOv5s network model with updated parameters, and outputting the AP value of each category in the validation set after statistics; looping and iterating until the mAP value counted stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0013] S800, using the test set to test the trained YOLOv5s network model, and adjusting the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0014] S900, use the final YOLOv5s network model to detect cloth defects.

[0015] In some possible implementations, the clipping method in the sample expansion process includes:

[0016] Calculate the defect target pixel area;

[0017] Dynamically adjust the movable range of the offset center point according to the background ratio of the defect with the largest pixel volume;

[0018] Adjust the center point of the cropping area according to the target offset, calculate the distance vector of each defect target relative to the cropping center point, and dynamically adjust the boundary of the cropping area according to the distribution of defect targets in the four quadrants of the cropping area to ensure that all defect targets are included in the cropping area;

[0019] Ensure that the cropped area does not exceed the image boundary according to the constraint formula;

[0020] The width and height of the cropped area are scaled proportionally to make the cropped area consistent with the original area.

[0021] In some possible implementations, after calculating the pixel area of ​​the defect target, the defect target is divided into four categories according to the pixel area by using the KMeans clustering algorithm. The clustering goal is to minimize the sum of squares of intra-class differences. The calculation formula is:

[0022] A=w·W·h·H;

[0023]

[0024] Where A is the pixel area of ​​the defect target, W is the width of the given image, H is the height of the given image, the relative coordinates of the bounding box bbox = (x, y, w, h), ν k It is cluster C k The average pixel area, A i is the individual area value in this cluster.

[0025] In some possible implementations, the movable range of the offset center point is calculated as follows:

[0026]

[0027] Where: C w C is the width of the movable range of the offset center point. h is the height of the movable range of the offset center point, is the maximum pixel amount of defects after clustering, To adjust the offset, St is the detection image size, and t is the total number of classes.

[0028] In some possible implementations, the center point of the cropping area is adjusted according to the target offset, and the offset of the center of the cropping area is flexibly offset according to the distribution of the defect target:

[0029] Δx=x c -x o ;

[0030] Δy=y c -y o ;

[0031] The center point of the dynamic cropping area is (x c ,y c ), The center point of the original cropping area is (x o ,y o ), the offset of the cropping area center flexibly shifted according to the distribution of defect targets is (Δx, Δy);

[0032] Calculate the distance vector of each defect target relative to the clipping center point. For each defect target, its constraint boundary reference point is (xn s ,yn s ), the reference points of each quadrant constraint boundary are (xn l ,yn l ), where n is the corresponding quadrant number, then the distance vector (d x ,d y )for:

[0033] d x =xn s -xn l ;

[0034] d y =yn s -yn l ;

[0035] According to the distribution of defect targets in the four quadrants of the cropping area, the boundaries of the cropping area are dynamically adjusted to ensure that all defect targets are included in the cropping area. The boundary adjustment of each quadrant is expanded according to the distribution of defect targets. (xb1 new :xb2 new ,yb1 new :yb2 new ) is the set of coordinate offsets after adjustment for each quadrant, then:

[0036] Upper left quadrant (d x >0andd y >0):

[0037] x1b new =min(xn s ,xn l );

[0038] y1b new =min(yn s ,yn l );

[0039] Upper right quadrant (d x <0andd y >0):

[0040] x2b new =max(xn s ,xn l );

[0041] y1b new =min(yn s ,yn l );

[0042] Lower left quadrant (d x >0andd y <0):

[0043] x1b new =min(xn s ,xn l );

[0044] y2b new =max(yn s ,yn l );

[0045] Lower right quadrant (d x <0andd y <0):

[0046] x2b new =max(xn s ,xn l );

[0047] y2b new =max(yn s ,yn l ).

[0048] In some possible implementations, the constraint formula is:

[0049] x1b new =max(0,x1b new );y1b new =max(0,y1b new );

[0050] x2b new =min(img w ,x2b new );y2b new =min(img h ,y2b new ).

[0051] In some possible implementations, the width and height of the cropped region are scaled proportionally, including:

[0052] Calculate the area A of the original clipping area originaland the adjusted new area A new :

[0053] A original =(x2b-x1b)·(y2b-y1b);

[0054] A new =(x2b new -x1b new )·(y2b new -y1b new )

[0055] If the new area after adjustment is A new A is larger than the original clipping area original , scale the crop region by the following ratios to maintain the original area:

[0056]

[0057] new_width=(x2b new -x1b new )·scale_factor;

[0058] new_height=(y2b new -y1b new )·scale_factor;

[0059]

[0060]

[0061] In a second aspect, a defect detection device based on FabricMosaic and an improved fine-grained feature extraction network is provided, using the method described in the first aspect, wherein the system comprises:

[0062] An acquisition module is used to acquire cloth images and establish a sample data set;

[0063] The sample expansion module is used to expand the sample using a dynamic FabricMosaic data enhancement strategy based on the target foreground and cloth texture background features to obtain an expanded data set. In the cropping process of the sample expansion, the cropping center offset point is dynamically adjusted according to the morphological features of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target;

[0064] A partitioning module, used to partition the expanded data set into a training set, a validation set and a test set;

[0065] A feature extraction module is used to use the image in the training set as the input of the YOLOv5s network model, and obtain a global fine-grained feature map after feature extraction by a feature fine-grained enhancement extraction module based on a divergent path, wherein the feature fine-grained enhancement extraction module based on a divergent path includes an information sending path for shunting the input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the features after shunting;

[0066] The bounding box generation module is used to predict the probability that each pixel in the global fine-grained feature map belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0067] A parameter updating module is used to calculate the network loss value according to the bounding box and the GT box of the corresponding image, and to update the parameters of the YOLOv5s network model using a gradient descent method;

[0068] The training module is used to repeatedly execute the feature extraction module, the bounding box generation module, and the parameter update module until all images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set according to the YOLOv5s network model after parameter update, and output the AP value of each category in the validation set after statistics; iterate in a loop until the mAP value counted stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0069] The test module is used to test the trained YOLOv5s network model using the test set, and adjust the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0070] The detection module is used to detect cloth defects using the final YOLOv5s network model.

[0071] In a third aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0072] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable medium stores a program code for execution by a device, wherein the program code includes steps for executing the method in any one of the implementations of the first aspect described above.

[0073] In a fifth aspect, an electronic device is provided, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements a method as in any one of the implementations in the first aspect above.

[0074] This application has the following beneficial effects:

[0075] 1. This application adopts a dynamic FabricMosaic data enhancement strategy to expand samples. It can effectively retain the integrity of defective targets during the sample integration process. According to the morphological characteristics of the defective targets and the prior information of the background, it can dynamically adjust the cropping center offset point. In the case of a serious imbalance in the ratio of positive and negative samples, it can effectively improve the diversity of data and ensure the integrity of the pixel information and features of these special targets in defect types with extreme aspect ratios.

[0076] 2. This application uses a divergent path feature fine-grained enhancement extraction module (DPFEM) for feature extraction, which is not easy to lose the information of tiny defects in the feature map, and can effectively reduce the risk of missed detection when detecting small target defects. Secondly, through the information divergence path, the input features are diverted according to the feature channels, which can not only retain the original feature information while improving the feature extraction efficiency, avoiding the problem of gradient disappearance during model training, but also fully retain the fine-grained features, greatly improving the model's ability to detect tiny defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0079] Figure 1 It is a flow chart of a defect detection method based on FabricMosaic and an improved fine-grained feature extraction network according to Example 1 of the present application;

[0080] Figure 2 It is a schematic diagram of pixel ratio distribution of clustered image data in the defect detection method based on FabricMosaic and improved fine-grained feature extraction network in Example 1 of the present application;

[0081] Figure 3 It is a schematic diagram of calculating the coordinates of the center point and the movable range of the offset center point in the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network in Example 1 of the present application;

[0082] Figure 4 It is a schematic diagram of parsing quadrants and adjusting cropping ranges in the defect detection method based on FabricMosaic and an improved fine-grained feature extraction network in Example 1 of the present application;

[0083] Figure 5 It is a schematic diagram of the adjusted cropping area in the defect detection method based on FabricMosaic and improved fine-grained feature extraction network in Example 1 of the present application;

[0084] Figure 6 It is a structural schematic diagram of a feature fine-grained enhancement extraction module of a divergent path in a defect detection method based on FabricMosaic and an improved fine-grained feature extraction network in Example 1 of the present application;

[0085] Figure 7 This is a comparison chart of mAP between the improved model of the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network and the baseline model in identifying different types of defects in Example 1 of the present application;

[0086] Figure 8 It is a structural block diagram of a defect detection device based on FabricMosaic and an improved fine-grained feature extraction network according to Embodiment 2 of the present application;

[0087] Fig. 9 It is a schematic diagram of the internal structure of the electronic device of Example 4 of the present application.

[0088] Reference numerals:

[0089] 100, acquisition module; 200, sample expansion module; 300, division module; 400, feature extraction module; 500, bounding box generation module; 600, parameter update module; 700, training module; 800, testing module; 900, detection module. DETAILED DESCRIPTION

[0090] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0091] Example 1

[0092] like Figure 1 As shown, a defect detection method based on FabricMosaic and an improved fine-grained feature extraction network according to Embodiment 1 of the present application includes:

[0093] S100, obtaining cloth images and establishing a sample data set, wherein the obtained cloth images are obtained by taking pictures with a camera, a still camera or a webcam.

[0094] S200, using a dynamic FabricMosaic data enhancement strategy based on target foreground and cloth texture background features to expand samples to obtain an expanded data set. In the cropping of the sample expansion process, according to the morphological features of the defect target and the prior information of the cloth texture background features, the cropping center offset point is dynamically adjusted to ensure the integrity of the defect target;

[0095] like Figure 2-Figure 5 As shown in the figure, in the dynamic Mosaic data enhancement strategy: first, the pixel area calculation and clustering analysis of the defect target are completed, wherein the defect target is divided into four categories according to the pixel area through the KMeans clustering algorithm. Secondly, the center offset calculation after defect clustering is completed. The specific operation is to dynamically adjust the movable range of the offset center point according to the background proportion of the defect class with the largest pixel volume. Thirdly, the dynamic cropping area is parsed and adjusted, wherein the center point of the cropping area is first adjusted according to the target offset, and then the distance vector of each defect target relative to the cropping center point is calculated. Then, according to the distribution of the target in the four quadrants of the cropping area, the boundary of the cropping area is dynamically adjusted to ensure that all targets are included in the cropping area. Secondly, according to the relevant constraint formula, it is ensured that the cropping area does not exceed the image boundary. Finally, the adjustment of the original value of the cropping area area is completed. This operation can not only ensure that the center point of the cropping area remains in a relative position, but also can make its area consistent with the original area by scaling the width and height of the cropping area, thereby maintaining the consistency of data enhancement.

[0096] Specifically:

[0097] S201, such as Figure 2 As shown in the figure, defect target pixel area calculation and cluster analysis. The pixel area A of the defect target is calculated by the width and height of the target bounding box. Given the width W and height H of the image, and the relative coordinates of the bounding box bbox = (x, y, w, h), the pixel area A of the defect target is shown in the formula. Subsequently, the defect targets are divided into four categories according to the pixel area through the KMeans clustering algorithm. The goal of clustering is to minimize the sum of squares of intra-class differences, as shown in the following formula:

[0098] A=w·W·h·H

[0099]

[0100] In the above formula: ν k It is cluster C k The average pixel area, A iis the individual area value in the cluster. Through this clustering method, defect targets of different sizes can be effectively distinguished, providing a basis for subsequent cropping and enhancement.

[0101] S202, such as Figure 3 As shown in the figure, the center offset after clustering is calculated. According to the background ratio of the defect class with the largest pixel volume, the movable range of the offset center point is dynamically adjusted. The calculation formula for the movable range of the offset center point is as follows:

[0102]

[0103] In the above formula: C w and C h The width and height of the movable range of the offset center point; is the maximum pixel amount of defects after clustering; is the adjustment offset; St is the detection image size; t is the total number of classes.

[0104] S203, such as Figure 4 As shown in the figure, the dynamic cropping area is analyzed and adjusted. In the dynamic cropping process, the center point of the cropping area is first adjusted according to the target offset. The center point of the dynamic cropping area is set to (x c ,y c ),in The center point of the original cropping area is (x o ,y o ). The target offset (Δx, Δy) is shown in the following formula. Through this offset, the center of the cropping area is flexibly offset according to the distribution of the target:

[0105] Δx=|x c -x o |;

[0106] Δy=|y c -y o |

[0107] Secondly, the distance vector of each defect target relative to the cropping center point is calculated to evaluate whether the target is reasonably located in the cropping area, thereby guiding the adjustment offset of the defect target position. For each defect target, its constraint boundary reference point is (xn s ,yn s ), the reference points of each quadrant constraint boundary are (xn l ,yn l ), where n is the corresponding quadrant number. Then the distance vector (d x ,d y ) is shown in the following formula:

[0108] d x =xn s-xn l ;

[0109] d y =yn s -yn l

[0110] Afterwards, according to the distribution of the targets in the four quadrants of the cropping area, the boundaries of the cropping area are dynamically adjusted to ensure that all targets are included in the cropping area. The adjusted cropping area is as follows: Figure 5 As shown. The boundary adjustment of each quadrant is flexibly expanded according to the target distribution to adapt to the location and size distribution of different targets. Suppose the coordinate offset set of each quadrant after adjustment is (x1b new :x2b new ,y1b new :y2b new ),but

[0111] a. Upper left quadrant (d x >0andd y >0):

[0112] x1b new =min(xn s ,xn l );

[0113] y1b new =min(yn s ,yn l )

[0114] b. Upper right quadrant (d x <0andd y >0):

[0115] x2b new =max(xn s ,xn l );

[0116] y1b new =min(yn s ,yn l )

[0117] c. Lower left quadrant (d x >0andd y <0):

[0118] x1b new =min(xn s ,xn l );

[0119] y2b new =max(yn s ,yn l )

[0120] d. Lower right quadrant (d x <0andd y <0):

[0121] x2b new =max(xn s ,xn l );

[0122] y2b new =max(yn s ,yn l )

[0123] S204: To ensure that the cropped area does not exceed the image boundary, the cropped area is limited to the image range by the following constraint formula. This constraint ensures that the cropped image maintains consistency and integrity before and after data enhancement. The boundary constraint conditions for image cropping are as follows:

[0124] x1b new =max(0,x1b new );y1b new =max(0,y1b new );

[0125] x2b new =min(img w ,x2b new );y2b new =min(img h ,y2b new )

[0126] S205, the area of ​​the cropped area is adjusted to maintain the original value:

[0127] Calculate the area A of the original clipping area original and the adjusted new area A new :

[0128] A original =(x2b-x1b)·(y2b-y1b);

[0129] A new =(x2b new -x1b new )·(y2b new -y1b new )

[0130] If A new Greater than A original , scale the crop region by the following ratios to maintain the original area:

[0131]

[0132] new_width=(x2b new -x1b new )·scale_factor

[0133] new_height=(y2b new -y1b new )·scale_factor

[0134]

[0135]

[0136] In this embodiment, it is possible to ensure that the cropped area maintains the original area after adjustment, and the center point of the cropped area remains in a relative position. By scaling the width and height of the cropped area so that its area is consistent with the original area, the consistency of data enhancement is maintained.

[0137] S300, dividing the expanded data set into a training set, a validation set, and a test set, for example, dividing the image data into a training set, a validation set, and a test set in a ratio of 7:2:1;

[0138] S400, using the image in the training set as the input of the YOLOv5s network model, and obtaining a global fine-grained feature map after feature extraction by a feature fine-grained enhancement extraction module based on a divergent path, wherein the feature fine-grained enhancement extraction module based on a divergent path includes an information sending path for shunting the input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the features after the shunting process;

[0139] like Figure 6 As shown, the feature fine-grained enhancement extraction module based on the divergent path includes: an information divergent path for diverting input features according to feature channels and a feature extraction unit for extracting fine-grained features from the features after the diversion process.

[0140] like Figure 6As shown in the figure, the feature fine-grained enhancement extraction module based on the divergent path uses the information divergent path to divert the input features according to the feature channel to improve the efficiency of feature extraction. Among them, some features are directly downsampled through DS convolution (Depthwise Separable Convolution) to retain the original feature information and avoid the problem of gradient disappearance during model training. The other features are extracted through SDFI (Spatial-Depth Fission Intensifier) ​​to ensure that fine-grained features are fully retained, thereby improving the model's ability to detect minor defects.

[0141] Specifically:

[0142] like Figure 6 As shown in Figure 1, combined with the idea of ​​SPDConv (Space-to-Depth Convolution), the SDFI module reduces information loss in the process of small target feature extraction by spatial fission. Specifically, SDFI first reorganizes the input feature information. Assuming the size of the original input feature map G is (W, H, C), and the downsampling step size is St, the original feature sequence is split into St through the spatial fission operation. 2 A spatial fission feature G′, whose size is Next, the divergent feature subsequences are reorganized by channel aggregation to form the aggregated channel feature G″. This process increases the channel dimension while reducing the spatial scale, ensuring that the total feature elements of the cloth image information are maintained. The deep sequence reorganization operation expression from space to channel is as follows:

[0143]

[0144] In the above formula: a, b are the starting index; S is the ending index; St is the sampling step; Cat[,1] is the concatenation operation of the sub-feature tensor in the channel dimension; G″′ is the global fine-grained feature output by SDFI.

[0145] like Figure 6 As shown in the figure, after the above reorganization operation, the size of the new channel aggregation feature G″ is Transformed into On the basis of retaining the fine-grained information of the cloth image, the feature space size is compressed. Subsequently, the feature information is further extracted using the depthwise separable convolution (DS convolution). Specifically, each input channel is first convolved separately through a DW (Depthwise) extractor with multiple independent k×k convolution kernels. Then, a PW (Pointwise) extractor is used to perform point convolution in the channel dimension to achieve cross-channel feature projection and fusion. The feature extraction process output by the SDFI module is shown in the following formula:

[0146] G″′=PWConv(DWConv(G″))

[0147] In the above formula: G″′ is the global fine-grained feature output by SDFI; PWConv is the PW extractor; DWConv is the DW extractor.

[0148] like Figure 6 As shown in the figure, the DW extractor and the PW extractor together form the DS convolution, which can effectively retain feature information while improving computational efficiency, thereby maintaining good detection performance. The proportional relationship between the computational amount of the DS convolution and the traditional convolution is shown in the formula:

[0149]

[0150] In the above formula: Fl db Flops is the ratio of the amount of computation of ordinary convolution to that of DW convolution. NorConv It is the amount of calculation of ordinary convolution; Flops DWConv is the amount of computation of DW convolution; k is the size of the convolution kernel. Through this comparison of computational amounts, we can see the advantage of DW convolution in computational efficiency, especially when the number of channels is large.

[0151] S500, for the global fine-grained feature map, predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0152] S600, calculating a network loss value according to the bounding box and the GT box (i.e., Ground Truth box, which refers to the correct target detection box manually annotated) of the corresponding image, and using a gradient descent method to update the parameters of the YOLOv5s network model;

[0153] Specifically, the intersection-over-union loss is calculated according to the bounding box and the corresponding GT box, the classification confidence loss and the border confidence loss are calculated according to the classification confidence and border confidence of each predicted box contained in the network output feature map, and the intersection-over-union loss, the classification confidence loss, and the border confidence loss are weighted and summed in a preset ratio to obtain the overall network loss, and back propagation is performed to update the network parameters.

[0154] S700, repeating steps S400-S600 until all images in the training set are input into the YOLOv5s network model at least once; predicting each image in the validation set according to the YOLOv5s network model with updated parameters, and outputting the AP value of each category in the validation set after statistics; looping and iterating until the mAP value counted stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0155] S800, using the test set to test the trained YOLOv5s network model, and adjusting the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0156] S900, use the final YOLOv5s network model to detect cloth defects.

[0157] In order to verify the performance of the method proposed in this embodiment, the YOLOv5s network is also used to predict the images in the test set, and a comparative analysis is made with the YOLOv5s+fabric_mosaic and YOLOv5s+DPFEM of this application, as shown in Table 1 below, which is the effectiveness analysis table of FabricMosaic provided in this embodiment, and as shown in Table 2 below, which is the effectiveness analysis table of the divergent path feature enhancement module provided in this embodiment. It can be seen from Tables 1 and 2 that FabricMosaic of this application effectively retains the integrity of defective targets and improves data diversity and model detection performance by dynamically adjusting the cropping area and boundary optimization. Its accuracy is improved by 16.3%, and the F1 value is increased to 69.1%, without increasing the complexity of the model. The dynamic cropping strategy based on target features and flexible boundary constraints solve the problem of losing positive sample information in traditional methods, showing excellent detection accuracy and robustness.

[0158] Table 1:

[0159]

[0160] Table 2:

[0161]

[0162] like Figure 7As shown in the figure, it is a comparison diagram of using the baseline model (i.e., YOLOv5s) and the improved model of this embodiment (i.e., YOLOv5s+fabric_mosaic+DPFEM) in identifying different types of defects. It can be seen that the DPFEM of this embodiment enhances the small target feature extraction capability and significantly improves the detection performance through information diversion and spatial fission operations. Compared with the baseline model, the YOLOv5s+DPFEM model has achieved different degrees of improvement in key evaluation indicators such as Precision, Recall, mAP@0.5 and F1 value, especially mAP@0.5 has increased by 1.8 percentage points, showing a significant advantage in the small target defect detection task. It uses deep separable convolution (DS convolution) to optimize computational efficiency, retain fine-grained features, improve the ability to capture complex features and the detection rate of small target defects, and show excellent detection effect and good robustness.

[0163] Example 2

[0164] like Figure 8 As shown, a defect detection device based on FabricMosaic and an improved fine-grained feature extraction network according to Embodiment 2 of the present application includes:

[0165] An acquisition module 100 is used to acquire cloth images and establish a sample data set;

[0166] The sample expansion module 200 is used to expand the sample by adopting a dynamic FabricMosaic data enhancement strategy based on the target foreground and the cloth texture background features to obtain an expanded data set. In the cropping of the sample expansion process, the cropping center offset point is dynamically adjusted according to the morphological features of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target;

[0167] A partitioning module 300 is used to partition the expanded data set into a training set, a validation set and a test set;

[0168] A feature extraction module 400 is used to use the image in the training set as the input of the YOLOv5s network model, and obtain a global fine-grained feature map after feature extraction by a feature fine-grained enhancement extraction module based on a divergent path, wherein the feature fine-grained enhancement extraction module based on a divergent path includes an information sending path for shunting the input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the features after shunting;

[0169] A bounding box generation module 500 is used to predict the probability that each pixel in the global fine-grained feature map belongs to the object to be detected and the bounding box information of the object, and generate a bounding box;

[0170] A parameter updating module 600 is used to calculate a network loss value according to the bounding box and the GT box (i.e., Ground Truth box, which refers to the correct target detection box manually annotated) of the corresponding image, and to update the parameters of the YOLOv5s network model using a gradient descent method;

[0171] The training module 700 is used to repeatedly execute the feature extraction module, the bounding box generation module and the parameter update module until all the images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set according to the YOLOv5s network model after parameter update, and output the AP value of each category in the validation set after statistics; iterate in a loop until the mAP value counted stabilizes at a certain value to obtain a trained YOLOv5s network model;

[0172] The testing module 800 is used to test the trained YOLOv5s network model using the test set, and adjust the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model;

[0173] The detection module 900 is used to detect cloth defects using the final YOLOv5s network model.

[0174] It should be noted that other specific implementations of the defect detection device based on FabricMosaic and the improved fine-grained feature extraction network in this embodiment can refer to the specific implementations of the defect detection method based on FabricMosaic and the improved fine-grained feature extraction network mentioned above. To avoid redundancy, they will not be repeated here.

[0175] Example 3

[0176] A computer-readable storage medium according to Embodiment 3 of the present application, wherein the computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing a method in any one of the implementations in Embodiment 1 of the present application;

[0177] Among them, the computer-readable storage medium can be a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM); the computer-readable storage medium can store program code, and when the program stored in the computer-readable storage medium is executed by the processor, the processor is used to execute the steps of the method in any one of the implementation methods in Example 1 of the present application.

[0178] Example 4

[0179] like Fig. 9As shown, an electronic device involved in Embodiment 4 of the present application includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, a method in any one of the implementations in Embodiment 1 of the present application is implemented;

[0180] Among them, the processor can adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the method in any one of the implementation methods in Example 1 of the present application.

[0181] The processor may also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any implementation of Embodiment 1 of the present application may be completed by an integrated logic circuit of hardware in the processor or by instructions in software form.

[0182] The above-mentioned processor may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the functions required to be performed by the unit included in the data processing device of the embodiment of the present application in combination with its hardware, or executes the method in any one of the implementation modes in the embodiment 1 of the present application.

[0183] The above are only preferred specific implementations of the present application; however, the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved ideas of the present application within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A defect detection method based on FabricMosaic and improved fine-grained feature extraction network, characterized in that: include: S100, acquiring cloth images and establishing a sample data set; S200, using a dynamic FabricMosaic data enhancement strategy based on target foreground and cloth texture background features to expand samples to obtain an expanded data set. In the cropping of the sample expansion process, according to the morphological features of the defect target and the prior information of the cloth texture background features, the cropping center offset point is dynamically adjusted to ensure the integrity of the defect target; S300, dividing the expanded data set into a training set, a validation set and a test set; S400, using the image in the training set as the input of the YOLOv5s network model, and obtaining a global fine-grained feature map after feature extraction by a feature fine-grained enhancement extraction module based on a divergent path, wherein the feature fine-grained enhancement extraction module based on a divergent path includes an information sending path for shunting the input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the features after the shunting process; S500, for the global fine-grained feature map, predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and generate a bounding box; S600, calculating a network loss value according to the bounding box and the GT box of the corresponding image, and using a gradient descent method to update the parameters of the YOLOv5s network model; S700, repeating steps S400-S600 until all images in the training set are input into the YOLOv5s network model at least once; predicting each image in the validation set according to the YOLOv5s network model with updated parameters, and outputting the AP value of each category in the validation set after statistics; looping and iterating until the mAP value counted stabilizes at a certain value to obtain a trained YOLOv5s network model; S800, using the test set to test the trained YOLOv5s network model, and adjusting the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model; S900, use the final YOLOv5s network model to detect cloth defects.

2. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 1, characterized in that: The cropping method in the sample expansion process includes: Calculate the defect target pixel area; Dynamically adjust the movable range of the offset center point according to the background ratio of the defect with the largest pixel volume; Adjust the center point of the cropping area according to the target offset, calculate the distance vector of each defect target relative to the cropping center point, and dynamically adjust the boundary of the cropping area according to the distribution of defect targets in the four quadrants of the cropping area to ensure that all defect targets are included in the cropping area; Ensure that the cropped area does not exceed the image boundary according to the constraint formula; The width and height of the cropped area are scaled proportionally to make the cropped area consistent with the original area.

3. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 2 is characterized in that: After calculating the pixel area of ​​the defect target, the defect target is divided into four categories according to the pixel area through the KMeans clustering algorithm. The clustering goal is to minimize the sum of squares of intra-class differences. The calculation formula is: A=w·W·h·H; Where A is the pixel area of ​​the defect target, W is the width of the given image, H is the height of the given image, the relative coordinates of the bounding box bbox = (x, y, w, h), ν k It is cluster C k The average pixel area, A i is the individual area value in this cluster.

4. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 3 is characterized in that: The calculation formula for the movable range of the offset center point is: Where: C w C is the width of the movable range of the offset center point. h is the height of the movable range of the offset center point, is the maximum number of defect pixels after clustering, To adjust the offset, St is the detection image size, and t is the total number of classes.

5. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 4 is characterized in that: The center point of the cropping area is adjusted according to the target offset. The offset of the cropping area center flexibly shifted according to the distribution of defect targets is: Δx=|x c -x o |; Δy=|y c -y o |; The center point of the dynamic cropping area is (x c ,y c ), The center point of the original cropping area is (x o ,y o ), the offset of the cropping area center flexibly shifted according to the distribution of defect targets is (Δx, Δy); Calculate the distance vector of each defect target relative to the clipping center point. For each defect target, its constraint boundary reference point is (xn s ,yn s ), the reference points of each quadrant constraint boundary are (xn l ,yn l ), where n is the corresponding quadrant number, then the distance vector (d x ,d y )for: d x =xn s -xn l ; d y =in s -in l ; According to the distribution of defect targets in the four quadrants of the cropping area, the boundaries of the cropping area are dynamically adjusted to ensure that all defect targets are included in the cropping area. The boundary adjustment of each quadrant is expanded according to the distribution of defect targets. (xb1 new :xb2 new ,yb1 new :yb2 new ) is the set of coordinate offsets after adjustment for each quadrant, then: Upper left quadrant (d x >0andd y >0): x1b new =min(xn s ,xn l ); y1b new =min(in s , is l ); Upper right quadrant (d x <0andd y >0): x2b new =max(xn s ,xn l ); y1b new =min(in s , is l ); Lower left quadrant (d x >0andd y <0): x1b new =min(xn s ,xn l ); y2b new =max(in s , is l ); Lower right quadrant (d x <0andd y <0): x2b new =max(xn s ,xn l ); y2b new =max(in s , is l ) 6. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 5 is characterized in that: The constraint formula is: x1b new =max(0,x1b new );y1b new =max(0,y1b new ); x2b new =min(img w ,x2b new );y2b new =min(img h ,y2b new )。 7. The defect detection method based on FabricMosaic and improved fine-grained feature extraction network according to claim 6 is characterized in that: Scaling the width and height of the cropped region by proportions, including: Calculate the area A of the original clipping area original and the adjusted new area A new : A original =(x2b-x1b)·(y2b-y1b); A new =(x2b new -x1b new )·(y2b new -y1b new ) If the new area after adjustment is A new A is larger than the original clipping area original , scale the crop region by the following ratios to maintain the original area: new_width=(x2b new -x1b new )·scale_factor; new_height=(y2b new -y1b new )·scale_factor; 8. A defect detection system based on FabricMosaic and improved fine-grained feature extraction network, characterized in that: According to the method according to any one of claims 1 to 7, the system comprises: An acquisition module is used to acquire cloth images and establish a sample data set; The sample expansion module is used to expand the sample using a dynamic FabricMosaic data enhancement strategy based on the target foreground and cloth texture background features to obtain an expanded data set. In the cropping process of the sample expansion, the cropping center offset point is dynamically adjusted according to the morphological features of the defect target and the prior information of the cloth texture background features to ensure the integrity of the defect target; A partitioning module, used to partition the expanded data set into a training set, a validation set and a test set; A feature extraction module is used to use the image in the training set as the input of the YOLOv5s network model, and obtain a global fine-grained feature map after feature extraction by a feature fine-grained enhancement extraction module based on a divergent path, wherein the feature fine-grained enhancement extraction module based on a divergent path includes an information sending path for shunting the input features according to feature channels, and a feature extraction unit for extracting fine-grained features from the features after shunting; The bounding box generation module is used to predict the probability that each pixel in the global fine-grained feature map belongs to the object to be detected and the bounding box information of the object, and generate a bounding box; A parameter updating module is used to calculate the network loss value according to the bounding box and the GT box of the corresponding image, and to update the parameters of the YOLOv5s network model using a gradient descent method; The training module is used to repeatedly execute the feature extraction module, the bounding box generation module, and the parameter update module until all images in the training set are input into the YOLOv5s network model at least once; predict each image in the validation set according to the YOLOv5s network model after parameter update, and output the AP value of each category in the validation set after statistics; iterate in a loop until the mAP value counted stabilizes at a certain value to obtain a trained YOLOv5s network model; The test module is used to test the trained YOLOv5s network model using the test set, and adjust the parameters of the YOLOv5s network model according to the test results to obtain the final YOLOv5s network model; The detection module is used to detect cloth defects using the final YOLOv5s network model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code for execution by a device, wherein the program code includes steps for executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method as claimed in any one of claims 1 to 7 when executed by the processor.

Images