Weakly supervised fabric surface defect recognition method based on VGG-9 reduced network

Abstract

The present application belongs to the technical field of fabric detection, and specifically discloses a weakly supervised fabric surface defect identification method based on VGG-9 simplified network, which comprises collecting original images of a fabric surface and dividing the original images into a group of small images; then the group of small images are input into the VGG-9 network according to the division order to obtain a sequential output small image feature map matrix and a class id; then the Grad-CAM algorithm is applied to the small image feature map matrix in sequence to obtain a small image score matrix; the small image score matrix is further subjected to zero normalization processing, threshold processing, image dilation and image erosion; then the score matrix after image erosion is subjected to an operation of obtaining a contour circumscribed rectangle of a defect to obtain an image with a defect marking box, and a visual neural network heat map defect map is obtained by using a neural network heat map visualization operation. The method is suitable for high-precision identification of defects (horizontal and vertical) of different types of fabrics under extreme sizes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of fabric detection, and particularly relates to a weakly supervised fabric surface flaw identification method based on a VGG-9 simplified network. BACKGROUND

[0002] The research on fabric flaw detection algorithms has a long history, and a large amount of achievements have been made. However, the existing methods have certain limitations in the calculation amount, recognition accuracy and reliability for fabrics of extreme sizes, and it is difficult to meet the online detection needs of industrial sites. In recent years, with the addition of big data and deep learning methods, fabric flaw detection has begun to enter the intelligent era from the automatic era, and deep learning algorithms for automatically extracting fabric features have gradually been applied to flaw detection. Deep learning has the characteristics of automatically extracting deep features from massive data, and has received extensive attention in the field of fabric flaw detection in recent years. In the field of deep learning, flaw detection can be classified as a target detection problem, and deep learning algorithms such as YOLO, SSD and U-Net have achieved good results in the field of fabric flaw detection in recent years. However, the existing deep learning flaw detection methods still have the following difficulties: (1) the workload of manual labeling is huge under a large amount of samples; (2) the recognition accuracy of different types of flaws is low under extreme sizes. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a weakly supervised fabric surface flaw identification method based on a VGG-9 simplified network, which is used for identifying and positioning the flaw types of fabric surface images under extreme sizes.

[0004] In order to solve the above technical problems, the present application provides a weakly supervised fabric surface flaw identification method based on a VGG-9 simplified network, which comprises: collecting the original image of the fabric surface and sending it to the computer for preprocessing, the preprocessing mainly divides the original image into a group of small images; then inputting the group of small images after preprocessing into the VGG-9 network according to the segmentation order to obtain the sequential output small image feature map matrix and the class_id, then applying the Grad-CAM algorithm to the small image feature map matrix in turn to obtain the small image score matrix, and further performing zero normalization processing, threshold processing, image dilation and image erosion on the small image score matrix, then obtaining the image with a flaw marking box by obtaining the contour bounding rectangle of the flaw from the score matrix after image erosion, and simultaneously obtaining the flaw image of the visualized neural network heat map by further using the neural network heat map visualization operation on the score matrix after image erosion.

[0005] As an improvement of the weakly supervised fabric surface flaw identification method based on the VGG-9 simplified network of the present application:

[0006] The VGG-9 network comprises 9 hidden layers, 1 Max pooling layer and 1 Softmax layer classification, the 9 hidden layers are 8 convolutional layers and 1 fully connected layer, the input image is output after the 9 hidden layers, and then the matrix of the small picture feature map is output after the Max pooling layer and the Softmax layer classification, the highest probability category is output as the category class_id, and the category class_id is arranged in sequence to obtain the class_id array;

[0007] After the fabric surface image is collected and the pretreatment is performed, the fabric surface image is classified and stored according to whether the fabric surface image contains defects, is randomly divided into a training set and a test set, and then the VGG-9 network is trained and tested to obtain the trained VGG-9 network.

[0008] As a further improvement of the weakly supervised fabric surface defect recognition method based on the VGG-9 simplified network of the application:

[0009] The Grad-CAM algorithm is specifically:

[0010]

[0011]

[0012] Wherein, A k is the matrix of the small picture feature map, c is the category class_id, is the gradient of c, i and j are the indexes of the gradient, is the importance weight of the neuron, is the small picture score matrix.

[0013] As a further improvement of the weakly supervised fabric surface defect recognition method based on the VGG-9 simplified network of the application:

[0014] The zero normalization processing is specifically:

[0015] Each small picture score matrix is searched for the corresponding category class_id in the class_id array according to the corresponding segmentation order, if the category class_id is a non-defect category, the small picture score matrix is set to zero, otherwise the small picture score matrix is normalized:

[0016]

[0017] Wherein, B is the small picture score matrix, x represents the original value of the small picture score matrix element, x' represents the value of the corresponding element of the small picture score matrix after operation, min B is the minimum element of the small picture score matrix, max B is the maximum element of the small picture score matrix;

[0018] Then, the obtained small image score matrices are assembled into the original image score matrix according to the segmentation order.

[0019] As a further improvement to the weakly supervised fabric surface defect identification method based on the VGG-9 simplified network of the present invention:

[0020] The threshold processing specifically involves:

[0021]

[0022] Where y represents the original value of each element in the original graph score matrix, and y' represents the value after thresholding.

[0023] As a further improvement to the weakly supervised fabric surface defect identification method based on the VGG-9 simplified network of the present invention:

[0024] The image dilation and image erosion are specifically as follows:

[0025]

[0026]

[0027] in, is the dilation operator, - is the erosion operator, C represents the original image score matrix output by the thresholding process, D is the convolution kernel for image dilation and erosion operations, and is an a×b unit 2D matrix.

[0028]

[0029] Where c is the average length of the transverse defect and d is the average length of the vertical defect.

[0030] As a further improvement to the weakly supervised fabric surface defect identification method based on the VGG-9 simplified network of the present invention:

[0031] The specific steps for obtaining the bounding rectangle of the contour are as follows:

[0032] The bilinear interpolation algorithm is used to upsample the score matrix of the eroded image to the same resolution as the original image. Each element in the upsampled matrix is ​​multiplied by 255 to obtain the contour score matrix. The contour extraction algorithm is used to approximate each contour in the contour score matrix. The bounding rectangle of the approximate contour is then obtained as the defect marking box. The defect marking box is marked on the original image to obtain an image with defect detection boxes.

[0033] As a further improvement to the weakly supervised fabric surface defect identification method based on the VGG-9 simplified network of the present invention:

[0034] The specific process of visualizing the neural network heatmap is as follows:

[0035] The contour score matrix is ​​applied to the JET mapping algorithm to obtain a heatmap matrix. This heatmap matrix is ​​then fused with the original image or the image containing the defect detection boxes to obtain a defect map that displays a visual neural network heatmap. The formula for image fusion is expressed as follows:

[0036]

[0037] Where Z is the image fusion output matrix, X is the original image or an image with defect detection boxes, and Y is the heat map matrix obtained by applying the JET mapping algorithm to the feature map mask.

[0038] The beneficial effects of this invention are mainly reflected in:

[0039] (1) This invention integrates the VGG-9 simplified classification network, the Grad-CAM algorithm and traditional image processing methods to accurately locate defects while identifying the types of defects;

[0040] (2) This invention can achieve accurate identification and accurate location of fabric defects by training the VGG-9 network model using only category information, making full use of dataset information to obtain defect features, effectively avoiding the heavy manual annotation problem in existing target detection algorithms; the VGG-9 simplified classification network used in this invention has a simple structure, small number of parameters, and excellent detection performance, making it suitable for online detection in the fabric production site.

[0041] (3) The present invention solves the problem of difficulty in identifying and low accuracy of heterogeneous distribution defects (horizontal and vertical) in fabrics under extreme sizes;

[0042] (4) This invention introduces traditional image processing methods such as contour scanning, image dilation, and image erosion, which solves the problems of local focusing and weak real-time performance in the defect localization and marking process of conventional weakly supervised target detection methods;

[0043] (5) The present invention uses a heat map, which makes it easy to determine whether the defect image detection structure is a false defect and the specific coordinates of the image features that caused the false detection. If it is not a false detection, the specific coordinates of the defect can be further determined, which is beneficial to carry out secondary development without affecting the current defect detection effect and reduce the false detection rate. Attached Figure Description

[0044] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0045] Figure 1 This is a flowchart illustrating the weakly supervised fabric surface defect identification method based on the VGG-9 simplified network of the present invention.

[0046] Figure 2 This is a schematic diagram of the structure of the VGG-9 simplified classification network of the present invention;

[0047] Figure 3 for Figure 1 Example diagram of preprocessing;

[0048] Figure 4 Image of a fabric with vertical defects;

[0049] Figure 5 Image of a fabric with lateral defects;

[0050] Figure 6 Image of fabric with vertical defects marked in a defect box;

[0051] Figure 7 Image of fabric with a defect marking box indicating a horizontal defect;

[0052] Figure 8 Example diagram of the operation process for visualizing neural network heatmaps;

[0053] Figure 9 A heatmap of a visualized neural network with vertical defects;

[0054] Figure 10 A heatmap of a visualized neural network with lateral defects;

[0055] Figure 11 Visual neural network heatmaps of fabric images with vertical defects after defect detection using VGG11 with 1FC-layers, VGG16 with 1FC-layers and VGG-9 networks respectively ((a) is the visual neural network obtained without thresholding, image dilation and image erosion operations, (b) is the visual neural network obtained after thresholding, image dilation and image erosion operations).

[0056] Figure 12 Visual neural network heatmaps are generated for defect detection of fabric images with lateral defects using VGG11 with 1FC-layers, VGG16 with 1FC-layers, and VGG-9 networks, respectively. ((a) is the visual neural network obtained without thresholding, image dilation, and image erosion operations, and (b) is the visual neural network obtained after thresholding, image dilation, and image erosion operations). Detailed Implementation

[0057] The present invention will be further described below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto:

[0058] Example 1: A weakly supervised method for identifying fabric surface defects based on a simplified VGG-9 network, such as... Figures 1-10 As shown, to address the difficulty of identifying defects of extreme sizes, this invention segments the original image. A single original image with a resolution of 3072*96 pixels is sequentially divided horizontally into 32 smaller images with a resolution of 96*96 pixels, each segmented with a step length of 96. These smaller images are then categorized and stored, eliminating the need for labeling the defect locations. Subsequently, a simplified VGG-9 classification network is constructed and trained. This network, combined with the Grad-CAM algorithm and traditional image processing methods, identifies the defect types while simultaneously obtaining the precise coordinates of the defects, thus achieving accurate defect localization. Specifically:

[0059] Step 1: Fabric surface image acquisition:

[0060] Images of the fabric surface to be inspected are acquired using a distributed image acquisition module (including industrial cameras and light sources), resulting in a raw image dataset with a resolution of 3072*96 pixels; the images are encoded using common encoding formats such as BMP, PNG, and JPEG. Figure 4 , Figure 5 Typical examples of vertical and horizontal defects are given. As can be seen from the figures, the fabric to be inspected in this invention has an aspect ratio of 32 in the fabric surface image (original image). The two types of defects have large size differences and account for less than 12.5% ​​of the original image. The aspect ratio of the bounding rectangle of the vertical defect is in the range of [5, 32], and the aspect ratio of the bounding rectangle of the horizontal defect is in the range of [10, 280]. In contrast, the aspect ratio of a normal fabric surface image is in the range of [0.2, 5].

[0061] Step 2: Flawed image segmentation and dataset partitioning:

[0062] The images in the original image dataset obtained in step 1 are segmented. Each original image with a resolution of 3072*96 pixels is sequentially divided into 32 smaller images with a resolution of 96*96 pixels in the horizontal direction. The aspect ratio of the vertical blemish bounding rectangle and the aspect ratio of the horizontal blemish bounding rectangle in the smaller images are both within the range of [5, 32].

[0063] Small images containing and without defects are selected and stored separately, retaining only image-level annotations; this serves as the classification dataset, without needing to label the location of defects. The small images are then randomly divided into training and test sets in a 9:1 ratio.

[0064] Segmenting the original image into smaller images can help the VGG-9 model in step 4 to better extract the feature information of defects and better distinguish between defective and non-defective classes.

[0065] Step 3: Build a simplified VGG-9 classification network based on the PyTorch framework:

[0066] The VGG network, proposed by the renowned Visual Geometry Group at Oxford University, is primarily used for classification tasks. It includes VGG11, VGG13, VGG16, and VGG19, among others. The different structures are not fundamentally different, only differing in network depth. This invention's simplified VGG-9 classification network (referred to as the VGG-9 network) is built upon the VGG11 network, with the following structural optimizations: It retains the same number of convolutional layers to ensure their feature extraction capabilities; it reduces the number of pooling layers from five to three, increasing the size of the feature matrix output by the last convolutional layer. This makes the attention position of the score matrix upsampled to the original image resolution more accurate in subsequent steps, further enabling the bounding rectangle of the generated contour to more precisely mark defects; and it reduces the number of fully connected layers from three to one, reducing computational cost (FLOPs) while decreasing the decision-making power of the fully connected layers, thus prompting the convolutional layers to extract more accurate feature maps.

[0067] Table 1 compares the VGG9, VGG11, and VGG16 structures of this invention. Input image: An RGB image with a size of 224×224 (96×96 in practical applications). Convolutional layers use 3×3 convolutional filters with stride=1 and padding of the same value, resulting in an unchanged image size after convolution. Pooling layers have 2×2 windows with stride=2 (halving the image size after pooling). They follow some convolutional layers, but not all convolutional layers are followed by pooling layers. Fully connected layers: If there are three fully connected (FC) layers, the first two have 4096 channels, and the last one has 1000 channels; if there is only one fully connected layer, it has 1000 channels.

[0068] Table 1: Structural Comparison

[0069]

[0070]

[0071] The VGG-9 network structure of the present invention is as follows: Figure 2 As shown, it contains 9 hidden layers. The input image passes through 8 convolutional layers and 1 fully connected layer to output an image feature matrix. This matrix is ​​then classified through a max pooling layer and a softmax layer, and the class with the highest probability is taken as the output classification result, i.e., class_id. The number of output channels in the 8 convolutional layers are 64, 64, 128, 128, 512, 512, 512, and 512, respectively, and the convolutional layers with different output channels are max pooling layers.

[0072] Step 4: Train the VGG-9 network based on the defect dataset obtained in Step 2 to obtain the optimal VGG-9 network weights for fabric defect recognition and detection.

[0073] During training, pre-trained weights are not required. The training parameters are as follows: epoch set to 400, batch size set to 128, initial learning rate set to 0.0006, minimum learning rate set to 0.000006, SGD optimizer selected, optimizer momentum parameter set to 0.9, weight_decay set to 0.0005, and cosine annealing as the learning rate decay method. The model parameters of the epoch with the minimum loss on the validation set are stored as the optimal model network weights.

[0074] Step 5: Online Detection

[0075] 5.1 A distributed image acquisition module is used to acquire images of the fabric surface in real time, obtaining original images with a resolution of 3072*96 pixels. The acquired fabric images are then transmitted to a computer, where online detection of fabric surface defects is achieved.

[0076] 5.2 Defect Detection

[0077] After preprocessing the acquired raw images, they are input into the trained VGG-9 network to obtain the feature matrix. The Grad-CAM algorithm is then applied to the feature matrix to obtain the score matrix. The score matrix is ​​then subjected to image processing operations such as zeroing and normalization, thresholding, image dilation and erosion, upsampling, contour scanning, contour approximation, and obtaining the contour bounding rectangle to achieve the goal of defect detection.

[0078] 5.2.1 Preprocessing: The original image with an input resolution of 3072*96 pixels is sequentially divided horizontally into 32 smaller images with a resolution of 96*96 pixels, with a segmentation step size of 96. Then, the 32 smaller images are sorted according to the order of segmentation, as follows: Figure 3 As shown.

[0079] 5.2.2. Input the 32 preprocessed small images into the VGG-9 network trained in step 4. After passing through the VGG-9 network, a feature map matrix A of 32 small images sorted in segmentation order is obtained. k The class_id is obtained by sequentially arranging the 32 class_ids.

[0080] 5.2.3. Apply the Grad-CAM algorithm sequentially to the 32 small image feature map matrices obtained in step 5.2.2 to obtain the small image score matrix:

[0081]

[0082]

[0083] Among them, A k is the feature map matrix of the small image obtained in step 5.2.2. c is the class_id obtained in step 5.2.2.

[0084] To obtain the score matrix of class c with height u and width v. First calculate the gradient of class c. The obtained gradients are indexed by i and j, and global average pooling is performed across the width and height dimensions to obtain the importance weights of the neurons. Then examine the feature map matrix A of the smaller image. k Importance weight A weighted combination is performed, and a linear rectified function (ReLU) is applied to it to obtain the size and small-map feature map matrix A. k Same small graph score matrix

[0085] 5.2.4 Zeroing and Normalization Process

[0086] After normalizing the 32 small image score matrices obtained in step 5.2.3 by setting them to zero, the original image score matrix is ​​assembled according to the segmentation order. The specific zero-setting and normalization process is as follows:

[0087] First, for each small image score matrix, search for the corresponding category in the class_id array obtained in step 5.2.2 according to the corresponding segmentation order. If the corresponding category (class_id) is a non-defective class, set the small image score matrix to zero; otherwise, apply the min-max normalization method to the small image score matrix to achieve normalization.

[0088]

[0089] Where B is the small-image score matrix, x represents the original value of the element in the small-image score matrix, x' represents the value of the corresponding element in the small-image score matrix after the operation, and min B is the minimum element of the small graph score matrix, and maxB is the maximum element of the small graph score matrix.

[0090] 5.2.5. After zeroing and normalizing in step 5.2.4, threshold each item in the original image score matrix obtained in the segmentation order to obtain a two-dimensional score matrix:

[0091]

[0092] Where y represents the original value of the element in the score matrix, and y' represents the value of the corresponding element in the score matrix after thresholding.

[0093] The purpose of thresholding is to make the defect location information in the submatrix more accurate, thereby making the position of the defect bounding rectangle (defect annotation box) more accurate.

[0094] 5.2.6 Apply image dilation and image erosion operations sequentially to the two-dimensional score matrix output in step 5.2.5:

[0095]

[0096]

[0097] in, is the dilation operator, - is the erosion operator, C represents the two-dimensional score matrix output from the thresholding process in step 5.2.5, and D is the convolution kernel for the image dilation and erosion operations, which is an a×b unit 2D matrix.

[0098]

[0099] Where c is the average length of horizontal defects in the original image dataset from step 1, and d is the average length of vertical defects in the original image dataset.

[0100] Image dilation and erosion operations can reduce the local focusing problem in the defect localization and labeling process of conventional weakly supervised target detection methods. They can connect the contours of the same horizontal or vertical defect in the score matrix obtained by the subsequent contour extraction algorithm, improve the IOU between the defect annotation box and the actual bounding rectangle of the defect, and solve the problem of difficulty in identifying and low accuracy of heterogeneous defects (horizontal and vertical) in fabrics under extreme sizes.

[0101] Defect type information is shown in Table 2. Among them, non-returning, weft-attached, loose weft, tight weft, and loose weft are lateral defects, while broken diameter and double diameter are vertical defects. The average length of lateral defects is 2600 pixels, and the average length of vertical defects is 96 pixels (original image resolution is 3072*96 pixels). Here, length refers to the longer side (length or width) of the circumscribed rectangle of the lateral or vertical defect. The average lengths of vertical and lateral defects are empirical values ​​obtained after multiple experiments.

[0102] The image dilation and image erosion operations are both iterated once.

[0103] 5.2.7 Apply bilinear interpolation algorithm to upsample the score matrix after the image erosion operation in step 5.2.6 to 3072×96 (consistent with the resolution of the original image acquired in step 1), and multiply each element in the upsampled score matrix by 255 to obtain the contour score matrix;

[0104] 5.2.8. Using a contour extraction algorithm, approximate each contour within the contour score matrix obtained in step 5.2.7, and then obtain the bounding rectangle of the approximate contour, which is the defect marking box. The defect marking box is shown below. Figure 6 He Ru Figure 7 As shown.

[0105] Mark defect detection boxes on the original image to obtain an image with defect detection boxes, and save the marked original image to the server.

[0106] 5.3 Neural Network Heatmap Visualization: As a method that transforms a two-dimensional matrix into a heatmap to visualize attention and determine the specific location of classification features, this technique displays the exact location of defects using near-pixel-level heatmaps, achieving the effect of semantic segmentation models. Simultaneously, visualizing neural network heatmaps helps determine whether the defect detection results are false positives, misclassifying non-defect classes as defects. By visualizing neural network heatmaps, it's possible to identify which details in the small image cause false positives, such as fabric texture or irregular light spots caused by ambient lighting.

[0107] If false detections occur, more non-defective images containing image features that lead to false detections can be added to the training set in step 2 to promote the VGG-9 network's extraction of non-defective features, achieve secondary development, and reduce the false detection rate.

[0108] The specific operation process for visualizing neural network heatmaps is as follows:

[0109] Apply the JET mapping algorithm to the contour score matrix obtained in step 5.2.7 to obtain the mapped two-dimensional matrix, namely the heatmap matrix, which shows the focus of attention of the VGG classification model.

[0110] The heatmap matrix is ​​fused with either the unprocessed original image acquired in step 5.1 or the labeled image with defect detection boxes from step 5.2.8 to obtain a defect map from the visualized neural network heatmap. An example of the process is shown below. Figure 8 As shown.

[0111] The formula for image fusion is expressed as follows:

[0112]

[0113] Where Z is the image fusion output matrix, X is the original image or an image with defect detection boxes, and Y is the heat map matrix obtained by applying the JET mapping algorithm to the feature map mask.

[0114] A defect map of a visualized neural network heatmap, the result of which is shown in the image below. Figure 9 and Figure 10 As shown.

[0115] The original images acquired in steps 1 and 5 are not required to be 3072*96. They can be adjusted to a suitable resolution according to actual needs, as long as they can be divided into several small images of the same size, with an aspect ratio within [0.5, 1.5] and appropriate size.

[0116] experiment:

[0117] The experiment used the Windows 10 operating system, the PyTorch deep learning development framework, and Python as the development language. The computer used in the experiment had an Intel(R) Core(TM) i5-12490F CPU @ 3.00GHz and an NVIDIA GeForce RTX 3060 GPU.

[0118] The experimental dataset information is as follows: The dataset used in this project was collected from a fabric production line in a factory. The original dataset contained 1200 images with a resolution of 96*3072, representing 7 defect classes and 3 non-defect classes. After segmentation and classification, the classification dataset contained 7 defect classes and 3 non-defect classes, representing 10 classes with a resolution of 96*96, totaling 76702 images. The distribution of defect types in the original dataset and the classification dataset is shown in Table 2. Referring to general object detection algorithms, most of the non-defect class samples were retained to implement the background class function.

[0119] 1. Calculation of the average length of horizontal and vertical defects

[0120] Due to the limitations of the original image size, the length of vertical blemishes is set to 96 pixels by default (i.e., the height of the original image). The average length of horizontal blemishes is determined by first counting the number of images (e) segmented from the original image, then counting the number of images (f) containing the smaller images with horizontal blemishes. The average length of the horizontal blemishes is then calculated. 96 is the step size for segmentation.

[0121] Table 2. Distribution of defect types in the original dataset and the classification dataset.

[0122]

[0123] 2. Comparative Experiment

[0124] The experiment compared VGG11 with 1FC-layers, VGG11, VGG16 with 1FC-layers, VGG16, and VGG-9 models. Pre-trained weights were not used during training. The training parameters were uniformly set as follows: training epochs of 400, batch size of 128, initial learning rate of 0.0006, minimum learning rate of 0.000006, SGD optimizer with a momentum parameter of 0.9, weight decay of 0.0005, and cosine annealing as the learning rate decay method. The model parameters of the epoch with the minimum loss on the validation set were used as the optimal model network weights.

[0125] The comparative evaluation results of VGG11 with 1FC-layers, VGG11, VGG16 with 1FC-layers, VGG16 and the model VGG-9 proposed in this invention are shown in Table 3.

[0126] Table 3 Comparison of Evaluation Results of Different Models

[0127]

[0128] Analysis of Table 3 shows that, while maintaining similar accuracy, precision, and recall, VGG-9 has significantly fewer parameters than VGG11 and VGG16. Table 1 shows that, compared to VGG11 with 1FC-layers and VGG16 with 1FC-layers, VGG-9 reduces two softmax layers and has a larger output feature map size.

[0129] The visualization results of VGG11 with 1FC-layers, VGG16 with 1FC-layers, and VGG-9 networks after defect detection on fabric images with vertical and horizontal defects are as follows: Figure 11 and Figure 12 As shown in the figure, VGG-9 retains a larger feature map size, resulting in less distortion during feature map upsampling and thus more accurate defect attention in its neural network heatmap. Furthermore, from Figure 11 , Figure 12The comparison shows that adding thresholding, image dilation, and erosion operations increases the actual defect contour and the intersection-union ratio (IU / U) of the defect detection box in the Grad-CAM algorithm (the ratio of the intersection and union areas of two images, where the union area is reduced). Therefore, its neural network heatmap shows more accurate attention to defects. The experimental results clearly demonstrate that this invention can effectively solve the common problems of heterogeneous (horizontal and vertical) defect distribution in extreme-sized images and the local focusing and weak real-time performance issues in defect localization and labeling of conventional weakly supervised object detection methods, while reducing the number of model parameters. This improves the accuracy of defect recognition and has high practical application value.

[0130] Finally, it should be noted that the above examples are merely some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A weakly supervised fabric surface defect recognition method based on VGG-9 simplified network, characterized in that : The method comprises collecting an original image of a fabric surface and sending the original image to a computer for preprocessing, the preprocessing mainly being segmentation of the original image into a set of small images; then inputting the set of small images after preprocessing into a VGG-9 network in the order of segmentation to obtain a small image feature map matrix and a class_id in sequence, then applying a Grad-CAM algorithm to the small image feature map matrix to obtain a small image score matrix, and then further performing zero normalization processing, threshold processing, image dilation and image erosion on the small image score matrix, then obtaining an image with a defect marking box by obtaining a contour circumscribed rectangle of the defect from the score matrix after image erosion, and further obtaining a visual neural network heat map defect map by using a neural network heat map visualization operation on the score matrix after image erosion; The zero normalization processing is: Finding the corresponding class_id in the class_id array in the order of segmentation for each small image score matrix, and if the class_id is a non-defect class, setting the small image score matrix to zero, otherwise normalizing the small image score matrix: (3) wherein, is the subgraph score matrix, denotes the original value of a subgraph score matrix element, denotes the value of the corresponding element of the subgraph score matrix after the operation, is the minimum element of the subgraph score matrix, is the maximum element of the subgraph score matrix; Then the obtained small image score matrix is combined into an original image score matrix in the order of segmentation. The threshold processing is: (4) wherein, denotes the original value of each element in the original graph score matrix, denotes the value after thresholding.

2. The weakly supervised fabric surface defect identification method based on a VGG-9 simplified network according to claim 1, characterized in that: The VGG-9 network comprises 9 hidden layers, 1 Max pooling layer and 1 Softmax layer classification, the 9 hidden layers being 8 convolutional layers and 1 fully connected layer, the input image being output as the small image feature map matrix after passing through the 9 hidden layers, and then passing through the Max pooling layer and the Softmax layer classification to output the class_id with the highest probability as the class_id, and the class_id is arranged in sequence to obtain the class_id array; After collecting the fabric surface image and performing the preprocessing, the image is classified and stored according to whether it contains defects, and then randomly divided into a training set and a test set, and then the VGG-9 network is trained and tested to obtain the trained VGG-9 network.

3. The weakly supervised fabric surface defect identification method based on a VGG-9 simplified network according to claim 2, characterized in that: The Grad-CAM algorithm is specifically: (1) (2) wherein, is the small graph feature matrix, c is the class_id, is the gradient of and is the index of the gradient, is the importance weight of the neuron, is the small graph score matrix.

4. The weakly supervised fabric surface defect identification method based on a VGG-9 simplified network according to claim 3, characterized in that: The image dilation and image erosion are specifically: (5) (6) wherein, is an operator of dilation, is an operator of erosion, represents a raw graph score matrix output by the thresholding process, is a convolution kernel for image dilation, image erosion operation, and is a unit 2-dimensional matrix of (7) wherein is the average length of the transverse flaws, is the average length of the vertical flaws.

5. The weakly supervised fabric surface defect identification method based on a VGG-9 simplified network according to claim 4, characterized in that: The contour circumscribed rectangle is specifically: The bilinear interpolation algorithm is applied to upsample the score matrix after image erosion to the same resolution as the original image, and each element in the upsampled matrix is multiplied by 255 to obtain a contour score matrix, the contours in the contour score matrix are obtained and approximated using a contour extraction algorithm, and then the circumscribed rectangle of the approximated contour is obtained as a defect marking box, and the defect marking box is marked on the original image to obtain an image with a defect detection box.

6. The weakly supervised fabric surface defect identification method based on the VGG-9 simplified network according to claim 5, characterized in that: The specific process of the neural network heat map visualization operation is as follows: The contour score matrix is applied to the JET mapping algorithm to obtain a heat map matrix, the heat map matrix is fused with the original image or the image with the defect detection frame, and a defect image showing the visualized neural network heat map is obtained; the formula expression of image fusion is as follows: (8) wherein, is the image fusion output matrix, is the original image or the image with the defect bounding box, is the heat map matrix obtained by applying the JET mapping algorithm to the feature map mask.

Images