A weft-knitted fabric classification method based on a lightweight convolutional neural network

Abstract

The application discloses a weft-knitted fabric classification method based on a light-weight convolutional neural network, and belongs to the technical field of knitted fabric classification. The method comprises the following steps: acquiring sample images of various weft-knitted fabrics, and establishing a sample dataset; performing standardization processing on each sample image, and establishing a standardized dataset; dividing the standardized dataset into a training set, a verification set and a test set; constructing a neural network model; adopting a weight sharing strategy to process the sample images of the training set, and fusing the obtained feature vectors to obtain a fused feature vector; classifying an input sample image group according to the obtained fused feature vector; using the loss value of the training set, using the prediction accuracy of the verification set to evaluate the training effect, adjusting the hyperparameters of the neural network model, retaining the model with the highest prediction accuracy of the verification set as an optimal neural network model, inputting the test set sample into the optimal neural network model, and determining the final neural network model according to the prediction accuracy of the test set.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of knitted fabric classification, and particularly relates to a weft-knitted fabric classification method based on a light-weight convolutional neural network. BACKGROUND

[0002] Weft-knitted fabrics occupy an important position in the textile market due to their comfortable texture and rich and varied textures. When designing a knitted product required by a user, the structure parameters of the knitted product are first obtained to guide the production of a knitting machine. At present, the identification of the structure of a knitted fabric is mainly performed by a skilled worker through visual observation and a poking method, which is low in efficiency and prone to errors due to fatigue of the worker.

[0003] With the increasing chip computing power year by year, another type of method based on deep learning has gradually become popular. By using the powerful feature extraction capability of a convolutional neural network, a researcher usually applies it to a classification task of a structure arrangement rule without manually designing a feature detector. Weft-knitted fabrics are formed by interlacing loops along the weft direction, and it is often difficult to accurately determine the structure category thereof only by the morphology of a single surface. The traditional fabric classification method needs to consume a large amount of computing resources to classify weft-knitted fabrics with complex structures, and is low in efficiency and poor in classification effect. SUMMARY

[0004] The purpose of the embodiment of the application is to provide a weft-knitted fabric classification method based on a light-weight convolutional neural network, which can solve the technical problem that the existing fabric classification method is high in computing resource consumption, low in efficiency, and poor in classification effect when classifying weft-knitted fabrics with complex structures.

[0005] In order to solve the above technical problem, the application is implemented as follows:

[0006] The embodiment of the application provides a weft-knitted fabric classification method based on a light-weight convolutional neural network, which comprises the following steps:

[0007] S101: acquiring sample images of a plurality of weft-knitted fabrics to establish a sample data set, wherein the sample data set comprises a plurality of sample image groups, and each sample image group comprises a front image and a back image of the same weft-knitted fabric;

[0008] S102: calculating the mean and standard deviation of the sample data set, and performing standardization processing on a plurality of color channels of each sample image according to the mean and standard deviation to establish a standardized data set;

[0009] S103: dividing the standardized data set into a training set, a verification set and a test set according to a preset ratio;

[0010] S104: Introduce a lightweight attention mechanism module and a plurality of ghost modules, and combine a double-branch network architecture to construct a neural network model having a first sub-network and a second sub-network;

[0011] S105: Randomly input positive images and negative images in each sample image group belonging to the training set and the validation set to the first sub-network and the second sub-network respectively, and perform convolution operation by adopting a weight sharing strategy to obtain first image features and second image features;

[0012] S106: Fuse the feature vectors of the first image features and the feature vectors of the second image features through the neural network model to obtain a fusion feature vector representing each sample data group;

[0013] S107: Classify the input sample image group according to the fusion feature vector, and output the predicted structure corresponding to each sample image group in the training set and the validation set;

[0014] S108: Determine whether the predicted structure is the same as the actual structure of the corresponding sample image group, and calculate the prediction accuracy of the training set, the loss value of the training set, and the prediction accuracy of the validation set;

[0015] S109: In the process of gradually reducing the loss value of the training set, determine whether the prediction accuracy of the validation set is stable in a first preset range, and complete the preliminary training of the neural network model in the case that the prediction accuracy of the validation set is stable in the first preset range.

[0016] S110: Adjust the hyperparameters of the neural network model according to the prediction accuracy of the validation set, retain the neural network model with the highest prediction accuracy of the validation set, and obtain the optimal neural network model in the case that the prediction accuracy of the validation set is greater than a first preset value;

[0017] S111: Input each sample image group in the test set to the optimal neural network model for classification to obtain the predicted structure of the test set, calculate the prediction accuracy of the test set, and in the case that the prediction accuracy of the test set is stable in a second preset range, take the optimal neural network model as the final neural network model.

[0018] In this embodiment of the invention, a lightweight attention mechanism module is applied during the feature extraction process of the collected sample images using a constructed neural network model. This enhances the focus on useful information and reduces the interference of useless features, thereby improving the final classification accuracy. The collected front and back images are processed simultaneously using a weight-sharing strategy through a dual-branch network architecture, reducing convolutional computation and significantly saving computational resources, thus improving classification performance. The collected sample images are divided into training, validation, and test sets. The validation set is used for initial validation and participates in optimizing the hyperparameters of the neural network model. The test set is used to finally verify the effectiveness and accuracy of the neural network model. The final established neural network model is applied to the classification of structurally complex weft-knitted fabrics, reducing computational resource consumption while improving prediction accuracy, demonstrating high efficiency and good classification results. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a weft-knitted fabric classification method based on a lightweight convolutional neural network provided in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of a weft-knitted fabric sample image provided in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of a dual-branch network architecture provided in an embodiment of the present invention.

[0022] The realization of the objective, functional characteristics and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0024] The following description, in conjunction with the accompanying drawings, details a method for classifying weft-knitted fabrics based on a lightweight convolutional neural network, through specific embodiments and application scenarios.

[0025] Reference Figure 1 The diagram shows a flowchart of a weft-knitted fabric classification method based on a lightweight convolutional neural network provided by an embodiment of the present invention.

[0026] This invention provides a method for classifying weft-knitted fabrics based on a lightweight convolutional neural network, comprising:

[0027] S101: Obtain sample images of a plurality of weft knitted fabrics, and establish a sample dataset, wherein the sample dataset comprises a plurality of sample image groups, and each sample image group comprises a front image and a back image of the same weft knitted fabric.

[0028] Referring to Figure 2 , a schematic diagram of a weft knitted fabric sample image provided by an embodiment of the present application is shown.

[0029] Optionally, the obtained weft knitted fabrics can come from different manufacturers, and the range of the gram weight can be 170g-300g. The material components of the selected weft knitted fabrics include cotton, polyester, spandex, and rayon, etc. The weft knitted fabric organizations include single pearl ground, 1x1 rib, 2x2 rib, 3x3 rib, 4x2 rib, 4x3 rib, terry, plain stitch, and double pearl ground.

[0030] In a possible implementation, S101 specifically comprises:

[0031] S1011: Collect different weft knitted fabrics;

[0032] S1012: Establish an image acquisition system by combining an industrial camera and a brightness-adjustable LED light source, and adjust the acquisition resolution of the image acquisition system to 640x640 pixels;

[0033] Optionally, the image acquisition system is composed of a Hikvision MV-CE050-30UC color industrial camera and a brightness-adjustable LED light source.

[0034] S1013: Capture the front image and the back image of each weft knitted fabric by using the image acquisition system, take the front image and the back image as sample images, and establish a sample dataset.

[0035] It can be understood that, in the image acquisition process, in order to avoid the influence of the industrial production environment on the final classification and recognition effect, the shooting conditions of the sample images are artificially changed, including four different rotation angles and three different shooting distances.

[0036] S102: Calculate the mean and standard deviation of the sample dataset, and perform standardization processing on a plurality of color channels of each sample image according to the mean and the standard deviation, and establish a standardized dataset.

[0037] It can be understood that, after the standardization processing, the sample images can accelerate the convergence speed of the neural network model, and thus reduce the training time.

[0038] In a possible implementation, S102 specifically comprises:

[0039] S1021: Calculate the mean and standard deviation of the sample data set;

[0040] S1022: According to the mean and standard deviation, the multiple color channels of the sample image are normalized by formula 1 to obtain a normalized output image:

[0041]

[0042] Wherein, input represents the sample image, output indicates the normalized output image, mean represents the mean of the sample data set, and std is the standard deviation of the sample data set;

[0043] S1023: According to the normalized output image, a normalized data set is established.

[0044] S103: The normalized data set is divided into a training set, a validation set and a test set according to a preset ratio.

[0045] Optionally, the preset ratio is 6:2:2.

[0046] In one possible implementation, after S103, it further includes:

[0047] S112: Online data augmentation is performed on each sample image in the training set to expand the number of sample images in the training set.

[0048] Optionally, the online data augmentation includes brightness enhancement, contrast enhancement, saturation change enhancement, 360° random rotation enhancement, random horizontal enhancement, vertical flip enhancement and random scaling and cropping enhancement.

[0049] It can be understood that, compared with the offline augmentation method of expanding the data set before training the neural network model, using online data augmentation can make the sample images fed into the neural network model have different enhancement effects without changing the number of sample images of the sample data set, thereby providing more diverse sample images, avoiding the neural network model from learning irrelevant features, and improving the generalization ability of the neural network model.

[0050] Referring to Figure 3 , a schematic diagram of a double-branch network architecture provided by an embodiment of the present application is shown.

[0051] From Figure 3 It can be seen that the front image and the back image of the front image of the sample image group are respectively input into the double-branch network architecture for separate processing, the double-branch network has a unified structure, adopts a weight sharing strategy, and reduces the parameters of training. The feature vectors of the output two-side images are fused in the Concat unit, and then classified to obtain a predicted structure.

[0052] S104: Introduce a lightweight attention mechanism module and a plurality of ghost modules, and combine a double-branch network architecture to construct a neural network model with a first sub-network and a second sub-network.

[0053] The lightweight attention mechanism module (Convolutional Block Attention Module, CBAM) has high flexibility as a general module and can be seamlessly integrated into an existing network architecture to participate in training with the original network. There are two ways to add the lightweight attention mechanism module to the feature extraction stage of the weft-knitted fabric structure classification network to produce better description features of weft-knitted fabric sample images. The first adding method is to place the module directly after the last ghost module in the feature extraction stage of the neural network model, and the second is to integrate the lightweight attention mechanism module into the design of the bottleneck structure and connect it before the last ghost module of the bottleneck structure.

[0054] The two ghost modules in the bottleneck structure are used to reduce and increase the number of channels of features, respectively, and can reduce the parameter amount of convolution operation while deepening the number of convolution layers of the neural network model. The bottleneck module with a step of 1 will not change the size of the sample image feature map, and the bottleneck module with a step of 2 can reduce the size of the sample image feature map to half of the original size by layer-by-layer convolution to compress the size of different feature layers. The lightweight attention mechanism module emphasizes or suppresses the original feature information of the sample image in the network through learning, enhances the attention to useful information and reduces the interference of useless features, thereby improving the performance of feature extraction of the neural network model.

[0055] It should be noted that the channel attention module and the spatial attention module of the lightweight attention mechanism module respectively strengthen the information across the channel and the space. When the input feature is F, the lightweight attention mechanism module generates a one-dimensional channel feature map M c and a two-dimensional spatial feature map M s The final refined output F" is calculated as follows:

[0056]

[0057]

[0058] Wherein, represents the multiplication of corresponding position elements.

[0059] It should be noted that the Ghost module is used instead of ordinary convolution, the correlation and redundancy between the sample image features are utilized, a small amount of inherent feature maps generated by ordinary convolution are subjected to cheap linear operation, and similar feature mapping to the original sample image features is generated in a more efficient manner, so that the total amount of parameters and the calculation complexity can be reduced without changing the number of output feature map channels.

[0060] S105: respectively input the positive and negative images in each sample image group belonging to the training set and the validation set into the first sub-network and the second sub-network, adopt a weight sharing strategy to perform convolution operation to obtain first image features and second image features;

[0061] It should be noted that the structure of the weft-knitted fabric is special, and only the information of the single sample image may misjudge the type of the structure. In order to effectively utilize the feature information of the positive and negative images of the weft-knitted fabric, a double-branch network architecture is adopted.

[0062] The feature extraction part of the double-branch network is composed of two sub-networks, each of which receives one side of the picture in a sample image group as input. The random input strategy ensures that the double-branch network is not sensitive to the order of inputting the positive and negative images into the network when classifying. Since the paired input image signals belong to the same modality, the two sub-networks are configured with a unified structure and a weight sharing strategy is adopted to reduce the required training parameters. The sample images passing through the neural network model will participate in the training process and parameter optimization of the sub-network, and the rich sample information can improve the feature selection capability of the neural network model.

[0063] S106: fuse the feature vectors of the first image features and the feature vectors of the second image features through the neural network model to obtain a fusion feature vector representing each sample data group.

[0064] In a possible implementation, S106 specifically includes:

[0065] S1061: serially splice the feature vectors of the first image features and the feature vectors of the second image features to obtain a fusion feature vector Y representing each sample data group:

[0066] Y = concat(X1, X2) formula 2

[0067] Wherein, X1 represents the feature vector of the first image features, X2 represents the feature vector of the second image features, and concat represents splicing the vectors in a serial manner.

[0068] S107: classify the input sample image group according to the fusion feature vector, and output the predicted structure corresponding to each sample image group in the training set and the validation set.

[0069] It should be noted that after classification, the probability of the input sample image group corresponding to all the trained weft-knitted fabric categories is obtained, and the output prediction structure is the weft-knitted fabric category with the maximum probability.

[0070] In a possible implementation, S107 specifically includes:

[0071] S1071: performing global average pooling and 1x1 convolution dimension reduction on the fusion feature vector, and outputting the obtained output result to a fully connected layer of the neural network model;

[0072] It should be noted that after the global average pooling and 1x1 convolution dimension reduction processing, the weft-knitted fabric category can be better matched with the feature map of the last convolution layer, the parameter amount is reduced, and overfitting is prevented.

[0073] S1072: mapping the output result using a softmax function to obtain the prediction accuracy of each sample image group corresponding to different weft-knitted fabric categories, and outputting a prediction structure with the maximum prediction accuracy.

[0074] The softmax function, also known as the normalized exponential function, is an algorithm for solving multi-class regression problems, and is a classifier widely used in the supervised learning part of the deep network in the current deep learning research.

[0075] S108: determining whether the prediction structure is the same as the actual structure of the sample image group, and calculating the prediction accuracy of the training set.

[0076] It should be noted that by determining whether the prediction structure is the same as the actual structure, the prediction accuracy of the total training set can be calculated according to different statistical results, and the obtained prediction accuracy of the training set can be used as a basic index for further adjusting the neural network model.

[0077] In a possible implementation, S108 specifically includes:

[0078] S1081: counting the case that the prediction structure is the same as the actual structure of the sample image group;

[0079] S1082: calculating the prediction accuracy of the training set, the loss value of the training set, and the prediction accuracy of the validation set.

[0080] S109: in the process of gradually reducing the loss value of the training set, determining whether the prediction accuracy of the validation set is stable in a first preset range, and completing the preliminary training of the neural network model in the case that the prediction accuracy of the validation set is stable in the first preset range.

[0081] Wherein, the person skilled in the art can select a suitable training stopping opportunity by observing the training set loss value and the prediction accuracy change of the validation set, and it can be understood that the person skilled in the art can select the size of the first preset range according to the actual situation, and the sample data in the training set is the parameter used by the neural network model for automatic learning. After inputting all the sample images of the training set into the neural network model for training, the prediction accuracy obtained is the parameter for judging whether the preliminary training is completed. The validation accuracy obtained in the preliminary training may be low, because the neural network model will be further adjusted and optimized subsequently.

[0082] S110: Adjusting the hyperparameters of the neural network model according to the prediction accuracy of the validation set, retaining the neural network model with the highest prediction accuracy of the validation set, and obtaining the optimal neural network model in the case that the prediction accuracy of the validation set is greater than the first preset value;

[0083] It can be understood that the hyperparameters of the model are adjusted manually through the performance of the validation set in the neural network model. After adjusting the hyperparameters, the neural network model with excellent performance, that is, the model with the highest prediction accuracy of the validation set, is obtained as the final optimal neural network model.

[0084] The person skilled in the art can select the size of the first preset value according to the actual situation, and it can be understood that the prediction accuracy obtained after the sample images in the validation set are classified is the only index for judging whether the optimal neural network model is qualified. If the obtained prediction accuracy cannot reach the first preset value, the hyperparameters of the optimal neural network model will be adjusted to make the final prediction accuracy of the validation set greater than the first preset value.

[0085] In one possible implementation, S110 specifically includes:

[0086] S1101: Inputting the sample image groups in the validation set into the neural network model and recording the prediction accuracy of the validation set;

[0087] S1102: Adjusting the hyperparameters of the neural network model according to the prediction accuracy of the validation set greater than the first preset value, and obtaining the optimal neural network model. It should be noted that by observing the change of the prediction accuracy of the validation set, the hyperparameters of the neural network model are constantly adjusted, so that the prediction accuracy of the adjusted and optimized neural network model constantly approaches the first preset value, and finally the optimal neural network model with the prediction accuracy greater than the first preset value is obtained.

[0088] S111: input each sample image group in the test set to the optimal neural network model for classification to obtain a prediction structure of the test set, calculate a prediction accuracy of the test set, and in a case where the prediction accuracy of the test set is stable in a second preset range, take the optimal neural network model as a final neural network model.

[0089] It can be understood that a person skilled in the art can select the size of the second preset range according to actual conditions, and in actual application, the requirement for the final neural network model can not be very high, so that the training time can be saved and the training can be ended in advance. If the prediction accuracy of the neural network model is required to be relatively high, the training time can be appropriately prolonged to increase the prediction accuracy of the neural network model.

[0090] Each sample image group in the test set is input to the optimal neural network model for classification to obtain a prediction structure of the test set, the actual generalization ability of the model is evaluated, and the optimal neural network model is taken as the final neural network model.

[0091] In the embodiment of the present application, by constructing a neural network model, a lightweight attention mechanism module is applied in the feature extraction process of the collected sample images, the attention to useful information is improved and the interference of useless features is reduced, so that the final classification accuracy is improved, the collected positive images and negative images are simultaneously processed by adopting a weight sharing strategy through a double-branch network architecture, the convolution operation amount is reduced, the calculation resources are greatly saved, and the classification performance is improved, the collected sample images are divided into a training set, a validation set and a test set, the initial validation is completed through the validation set, the validation set participates in the optimization of the hyperparameters of the neural network model, and the effectiveness and accuracy of the neural network model are finally verified through the test set. The finally established neural network model is applied to the classification of complex weft-knitted fabrics, reduces the consumption of calculation resources, improves the prediction accuracy, has high efficiency, and has good classification effect.

[0092] The above only describes the embodiments of the present application and is not used to limit the present application. The present application can have various changes and modifications for a person skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A weft-knitted fabric classification method based on a lightweight convolutional neural network, characterized by, The application relates to a method for training a neural network model for classifying weft-knitted fabrics, comprising the following steps: S101: acquiring sample images of multiple weft-knitted fabrics, and establishing a sample data set, wherein the sample data set comprises multiple sample image groups, and each sample image group comprises a front image and a back image of the same weft-knitted fabric; S102: calculating the mean value and standard deviation of the sample data set, and performing standardization processing on multiple color channels of each sample image according to the mean value and the standard deviation, and establishing a standardized data set; S103: dividing the standardized data set into a training set, a validation set and a test set according to a preset proportion; S104: introducing a lightweight attention mechanism module and multiple ghost modules, and combining a double-branch network architecture to construct a neural network model with a first sub-network and a second sub-network; S105: randomly inputting the front image and the back image in each sample image group belonging to the training set and the validation set into the first sub-network and the second sub-network respectively, and performing convolution operation by adopting a weight sharing strategy to obtain first image features and second image features; S106: fusing feature vectors of the first image features and feature vectors of the second image features through the neural network model to obtain a fusion feature vector representing each sample data group; S107: classifying the input sample image group according to the fusion feature vector, and outputting a predicted structure corresponding to each sample image group in the training set and the validation set; S108: judging whether the predicted structure is the same as the actual structure of the corresponding sample image group, and calculating the prediction accuracy of the training set, the loss value of the training set and the prediction accuracy of the validation set; S109: in the process that the loss value of the training set gradually decreases, judging whether the prediction accuracy of the validation set is stable in a first preset range, and completing the preliminary training of the neural network model in the case that the prediction accuracy of the validation set is stable in the first preset range; S110: adjusting the hyperparameters of the neural network model according to the prediction accuracy of the validation set, retaining the neural network model with the highest prediction accuracy of the validation set, and obtaining an optimal neural network model in the case that the prediction accuracy of the validation set is greater than a first preset value; S111: inputting each sample image group in the test set into the optimal neural network model for classification to obtain a predicted structure of the test set, calculating the prediction accuracy of the test set, and taking the optimal neural network model as a final neural network model in the case that the prediction accuracy of the test set is stable in a second preset range.

2. The method of classifying a weft knitted fabric according to claim 1, wherein, The S101 specifically comprises the following steps: S1011: collecting different weft-knitted fabrics; S1012: combining an industrial camera and a brightness-adjustable LED light source to establish an image acquisition system, and adjusting the acquisition resolution of the image acquisition system to 640*640 pixels; S1013: using the image acquisition system to shoot a front image and a back image of each weft-knitted fabric, taking the front image and the back image as sample images, and establishing the sample data set.

3. The method of claim 1, wherein, The S102 specifically comprises: S1021: calculating the mean and standard deviation of each sample data set; S1022: according to the mean and the standard deviation, performing normalization processing on multiple color channels of the sample image through formula 1 to obtain a normalized output image: Wherein, input represents the sample image, output represents the normalized output image, mean represents the mean of the sample data set, and std is the standard deviation of the sample data set; S1023: establishing a normalized data set according to the normalized output image.

4. The method of claim 1, wherein, After the S103, it further comprises: S112: performing online data enhancement on each sample image in the training set to expand the number of sample images in the training set.

5. The method of classifying a weft knitted fabric according to claim 4, wherein, The online data enhancement comprises brightness enhancement, contrast enhancement, saturation change enhancement, 360° random rotation enhancement, random horizontal enhancement, vertical flip enhancement and random scaling and cropping enhancement.

6. The method of classifying a weft knitted fabric according to claim 1, wherein, The S106 specifically comprises: S1061: serially splicing the feature vectors of the first image features and the feature vectors of the second image features to obtain the fusion feature vector Y representing each sample data group: Y = concat (X1, X2) formula 2 Wherein, X1 represents the feature vector of the first image feature, X2 represents the feature vector of the second image feature, and concat represents splicing vectors in a serial manner.

7. The method of classifying a weft knitted fabric according to claim 6, wherein, The S107 specifically comprises: S1071: performing global average pooling and 1×1 convolution dimension reduction on the fusion feature vector, and outputting the obtained output result to the fully connected layer of the neural network model; S1072: using a softmax function to map the output result to obtain the prediction accuracy of different types of interwoven knitted fabrics corresponding to each sample image group, and outputting the predicted structure with the maximum prediction accuracy.

8. The method of claim 7, wherein, The S108 specifically comprises: S1081: counting the case that the predicted structure is the same as the actual structure of the sample image group; S1082: calculating the prediction accuracy of the training set, the loss value of the training set and the prediction accuracy of the verification set.

9. The method of claim 1, wherein, The S110 specifically comprises: S1101: inputting the sample image group in the verification set into the neural network model and recording the prediction accuracy of the verification set; S1102: according to the prediction accuracy of the verification set greater than the first preset value, adjusting the hyperparameters of the neural network model to obtain the optimal neural network model.

10. The method of claim 1, wherein, The preset ratio is 6:2:2.

Images