A fabric weaving matching method based on contrastive learning and self-attention mechanism
By introducing a deep learning method of comparative learning and self-attention mechanism in fabric weaving matching, the problems of low accuracy and high resource consumption in the existing technology are solved, and more efficient and accurate fabrics are achieved Weave matching effect.
Patent Information
- Application Number
- CN202210645732.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-08
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-06-08
AI Technical Summary
The prior art identification accuracy in fabric weaving matching is not high, resulting in the matching fabrics that are often not the same weaving method, which reduces the matching efficiency and practicality. In addition, traditional convolutional neural networks require a large amount of label data and human resources, and There is a problem that mislabeling affects the accuracy of the model.
Deep learning method based on contrast learning and self-attention mechanism is adopted, and feature extraction and matching of fabric weaving images is performed through the autoencoder and multi-head self-attention layer. The comparison loss function is used for label-free training, which simplifies the network structure and reduces the Computing resource requirements.
It improves the accuracy and efficiency of fabric weaving matching, enhances user experience, achieves higher recognition accuracy and robustness, and reduces training costs and calculation overhead.
Smart Images

Figure CN114882253B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a fabric weaving matching method based on contrast learning and self-attention mechanism. Background Art
[0002] Fabric weaving pattern matching technology can help fabric manufacturers to conveniently manage fabric information, and can also assist end users such as fabric dealers and clothing designers to quickly match more fabrics with similar weaving patterns based on existing fabrics, and has a very wide range of applications. At present, there are very few products on the market that use deep learning technology to match fabric weaving patterns, and existing products often find it difficult to accurately judge the weaving patterns of fabrics. Due to the low recognition accuracy, the matched fabrics are often not of the same weaving pattern, which undoubtedly greatly reduces the efficiency and practicality of fabric matching.
[0003] Computer vision image recognition technology mostly uses deep learning networks for self-supervised learning or unsupervised learning. Compared with traditional image processing technologies based on pure mathematical methods, such as image filters, edge extraction, and sharpening, deep learning-based methods can extract hierarchical features, better process high-dimensional sparse data such as images, perform multiple nonlinear combinations and processing of key information, and extract deeper features in the image, thus having better robustness and recognition accuracy.
[0004] However, traditional supervised learning based on convolutional neural networks requires a large amount of labeled training data to be trained and adjusted before it can achieve relatively ideal results. However, these precisely labeled image information will undoubtedly consume a lot of manpower, material resources and resources. In addition, some mislabeled samples in the mixed sample set may also have a certain impact on the accuracy of the model, which is undoubtedly not conducive to model training. Self-supervised learning represented by contrastive learning can train models without labels on sample data, and has shown better recognition performance and accuracy than traditional convolutional neural networks; and the autoencoder trained by contrastive learning can be extracted separately and combined with other network structures. Compared with traditional convolutional neural networks, the network structure of autoencoders is simpler when used for downstream tasks, and the parameters and computer resource requirements are also lower. Applying autoencoders to fabric weaving matching problems is conducive to solving the inherent shortcomings of traditional neural networks in terms of accuracy, robustness, and training efficiency. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention proposes a fabric weaving matching method based on contrastive learning and self-attention mechanism, adopts the basic structure of deep learning to train the autoencoder, introduces the multi-head self-attention mechanism into the traditional contrastive learning architecture, achieves better matching of fabric weaving images, and improves the accuracy and efficiency of target recognition and matching.
[0006] A fabric weaving matching method based on contrastive learning and self-attention mechanism specifically includes the following steps:
[0007] Step 1: Collect a variety of fabric images with different weaving methods, and use the fabric weaving methods in the images as corresponding labels. After data enhancement and edge enhancement, scale the fabric images to the same size and store them in the pattern matching database in the form of tensors.
[0008] Step 2: Construct a contrastive learning network, including an autoencoder for feature extraction and a projection layer for nonlinear transformation. The projection layer includes two layers of residual networks cascaded in sequence and a linear rectification function.
[0009] Step 3: Input the tensors and their labels in the pattern matching database into the contrastive learning network constructed in step 2 for multiple iterations of training. The contrastive loss function is used to calculate the error and loss between positive and negative examples, and the gradient backpropagation is performed on the parameters in the contrastive learning network.
[0010] Step 4: Input the fabric image to be matched into the autoencoder trained in step 3. After obtaining the feature image, it passes through multiple layers of multi-head self-attention layers in sequence, performs dot product calculations with the query matrix, key matrix, and value matrix, and finally passes through the fully connected layer, normalization, and residual connection in sequence to output the feature vector.
[0011] Step 5: Input the feature vector obtained in step 4 into the classifier, sort the feature vector by probability projection, and output the label of the fabric image to be matched. According to the obtained label, retrieve the fabric image with the same label in the pattern matching database as the matching result.
[0012] The present invention has the following beneficial effects:
[0013] This method uses the weaving pattern matching database training set of the fabric to pre-train a contrast learning autoencoder with high recognition efficiency, extracts key feature information in the image, and accurately judges the weaving method of the fabric through pattern matching. Compared with the traditional method, it has higher accuracy, better efficiency and robustness, and greatly enhances the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 This is a flow chart of the fabric weaving matching method based on contrastive learning and self-attention mechanism;
[0015] Figure 2 The fabric images of different weaving methods collected in the embodiment;
[0016] Figure 3 Schematic diagram of the residual network structure;
[0017] Figure 4 This is the flow chart of back propagation training. DETAILED DESCRIPTION
[0018] The present invention is further explained below in conjunction with the accompanying drawings; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0019] like Figure 1 As shown, a fabric weaving matching method based on contrastive learning and self-attention mechanism specifically includes the following steps:
[0020] Step 1: Collect 500,000 images of fabrics with different weaves through microphotography. According to the different warp and weft yarn running rules of the fabrics, they are classified into plain weave, twill weave, satin weave and various combined fabric weaves of different combinations of these three original weaves. The fabric weave in the image is used as the label of the corresponding image. Figure 2 As shown in the figure, a, b, and c represent plain weave, twill weave, and satin weave, respectively. Supersampling technology and edge enhancement technology based on Sobel operator are used to enhance the edge of the fabric image to highlight the detail features. Finally, the preprocessed image is uniformly scaled to 128x128 pixels and stored in the pattern matching database in the form of a tensor.
[0021] The super sampling technology combines multiple pixels in the image into a super pixel, so that the processed image has better quality, can show more details, and increases the pixel density.
[0022] Image processing based on the Sobel operator is a traditional image processing technology. The operator is used to convolve the image and extract the edges in the image one by one. Considering that the fabric weaving information is often contained in the textile edge area, the edge enhancement technology can effectively reduce useless information, greatly improve the efficiency and accuracy of training, and reduce the computational overhead. The Sobel operator used in this embodiment includes two groups of 3x3 matrices, one for the horizontal and one for the vertical. By performing a plane convolution with the image, the horizontal and vertical brightness difference approximations can be obtained respectively. The Sobel operator is as follows:
[0023]
[0024] Where A represents the original image before processing, Gx and Gy represent the images obtained after horizontal and vertical edge detection respectively. The horizontal and vertical gradient approximation G of each pixel in the original image can be expressed as:
[0025]
[0026] The gradient direction θ is:
[0027]
[0028] Step 2: Construct a contrastive learning network, including an autoencoder for feature extraction and a projection layer for nonlinear transformation. In this embodiment, the autoencoder is composed of three fully connected layers cascaded in sequence. The projection layer includes Figure 3 The two-layer residual network and a linear rectification function are shown.
[0029] Step 3: Input the tensors and their labels in the pattern matching database into the contrastive learning network constructed in step 2 in batches for multiple iterative training. Before inputting into the contrastive learning network, randomly crop, rotate, flip or change the color of each original image collected in step 1 and convert it into multiple augmented images. The augmented images of these original images will be used as positive examples. In order to increase the learning effect, randomly select other images that are not in the pattern matching database and use them as negative examples of the original images after the same augmentation process. For any image, it is required to be closer to the positive example and less close to the negative example after passing through the neural network, so as to achieve unlabeled self-supervised learning. The model is gradually tuned through back propagation and gradient descent. The contrast loss function is used to calculate the error and loss between the positive and negative examples, requiring the positive examples to be as close as possible and the distance between the positive and negative examples to be as far as possible. Gradient back propagation is performed on the parameters in the contrastive learning network according to the training results.
[0030] like Figure 4 As shown, this embodiment implements gradient back propagation by calculating the normalized temperature-scale cross entropy loss. The specific steps are as follows:
[0031] s3.1. Calculate the cosine similarity of the feature images output by the projection layer in the contrastive learning network.
[0032] s3.2. Use the normalized exponential function and the cosine similarity of different images to calculate the probability of similarity between different images. In a batch, there are comparisons between the image's own augmented images and its other augmented images, and there are also comparisons between the image's own augmented images and the augmented images of other images. There will be multiple pairs of pairing combinations, all of which require calculating the cosine similarity one by one and using the normalized exponential function to calculate the probability of similarity.
[0033] s3.3. Calculate the loss for this set of images using the noise contrast estimation loss.
[0034] s3.4. Calculate the loss of all pairs in the entire batch and take the average to obtain the normalized temperature-scale cross entropy loss for this training.
[0035] s3.5. Back-propagate the results of the normalized temperature-scale cross entropy loss function to the contrastive learning network and dynamically adjust the parameters of the autoencoder and projection layer.
[0036] s3.6. Perform multiple iterations of training calculations based on gradient descent, and use an adaptive moment estimation optimizer to reduce the number of iterations and shorten training time.
[0037] Step 4: Input the fabric image to be matched into the autoencoder trained in step 3. After obtaining the feature image, it passes through three layers of multi-head self-attention layers in sequence, performs dot product calculations with the query matrix, key matrix, and value matrix, and finally passes through the fully connected layer, normalization, and residual connection in sequence to output the feature vector.
[0038] Step 5: Input the feature vector obtained in step 4 into the classifier, adopt probability projection sorting, perform pattern matching, and output the matching result of the fabric weave. In this embodiment, not only the result with the maximum similarity probability matching the fabric weave in the image is presented to the user, but also multiple results with a large similarity probability are presented as similar recommendations.
Claims
1. A fabric weaving matching method based on contrastive learning and self-attention mechanism, Features: The specific steps include: Step 1: Collect images of fabrics with different weaving methods, and use the weaving methods of the fabrics in the images as corresponding labels; After data enhancement and edge enhancement processing, the fabric images are scaled to the same size and stored in the pattern matching database in the form of tensors; Step 2: construct a contrastive learning network, including an autoencoder for feature extraction and a projection layer for nonlinear transformation; the projection layer includes two layers of residual networks cascaded in sequence and a linear rectification function; Step 3: Input the tensors and their labels in the pattern matching database into the contrastive learning network constructed in step 2, and perform multiple iterations of training; use the contrastive loss function to perform gradient backpropagation on the parameters in the contrastive learning network; Step 4: Input the fabric image to be matched into the autoencoder trained in step 3, obtain the feature image, and then pass through multiple layers of multi-head self-attention layers in sequence, and perform dot product calculations with the query matrix, key matrix, and value matrix; finally, pass through the fully connected layer, normalization, and residual connection in sequence to output the feature vector; Step 5: Input the feature vector obtained in step 4 into the classifier, perform probability projection sorting on the feature vector, and output the label of the fabric image to be matched; according to the obtained label, retrieve the fabric image with the same label in the pattern matching database as the matching result.
2. A fabric weaving matching method based on contrastive learning and self-attention mechanism as claimed in claim 1, Features: The weaving methods of the collected fabric images include plain weave, twill weave, satin weave and a combination of two or more.
3. A fabric weaving matching method based on contrastive learning and self-attention mechanism as claimed in claim 1, Features: The edge enhancement of fabric images is performed using supersampling technology and edge enhancement technology based on Sobel operator.
4. A fabric weaving matching method based on contrastive learning and self-attention mechanism as claimed in claim 3, Features: The Sobel operator is: Where A represents the original image before processing, Gx and Gy represent the images obtained by horizontal and vertical edge detection respectively; the horizontal and vertical gradient approximation G of each pixel in the original image is expressed as: The gradient direction θ is:
5. A fabric weaving matching method based on contrastive learning and self-attention mechanism as claimed in claim 1, Features: The autoencoder includes three fully connected layers cascaded in sequence, and the number of multi-head self-attention layers is 3.
6. A fabric weaving matching method based on contrastive learning and self-attention mechanism as claimed in claim 1, Features: The contrast loss function is the normalized temperature-scale cross entropy loss.
7. A fabric weaving matching method based on contrastive learning and self-attention mechanism as claimed in claim 1 or 6, Features: The specific steps of contrastive learning network training are as follows: s3.
1. Input the fabric images into the contrastive learning network in batches and calculate the cosine similarity of the feature images output by the projection layer; s3.
2. Based on the cosine similarity calculation results, use the normalized exponential function to compare the similarity probabilities between different images; s3.
3. Calculate the loss of a set of images using noise contrast estimation loss; s3.
4. Calculate the loss of all pairs in the entire batch and take the average to get the normalized temperature-scale cross entropy loss for this training; s3.
5. Back-propagate the results of the normalized temperature-scale cross entropy loss function to the contrastive learning network and dynamically adjust the parameters of the autoencoder and projection layer. s3.
6. Perform multiple iterations of training calculations based on gradient descent, and use an adaptive moment estimation optimizer to reduce the number of iterations and shorten training time.
8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 6.
