A tire pattern classification method and device based on attention mechanism and transfer learning

By adopting adaptive histogram equalization and attention mechanism in tire pattern image classification, combined with transfer learning and knowledge distillation, the problems of low efficiency and overfitting of tire pattern classification in the existing technology are solved, and rapid convergence and high efficiency are achieved Classification, suitable for edge devices.

CN114882278BActive Publication Date: 2024-11-19SHANGHAI YABO DIGITAL SENSE TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210497966.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-09
Publication Date
2024-11-19
Estimated Expiration
2042-05-09

AI Technical Summary

Technical Problem

The existing tire pattern image classification technology has a semantic gap, lack of visual features and high-level semantic features, resulting in low classification efficiency, and deep learning models are prone to overfitting on small data sets, high training time and computing resources are consumed, and insufficient detailed feature extraction is required. .

Method used

Using the tire pattern classification method based on attention mechanism and transfer learning, training samples are generated through adaptive histogram equalization processing and data augmentation, ResNet 50 network and channel space attention mechanism is constructed, combining the pyramid pooling layer and the fully connected layer. Use knowledge distillation methods to design lightweight student models to reduce training time and computing resources and avoid overfitting.

Benefits of technology

The detailed feature extraction capability of the tire pattern classification model is improved, training time and computing resources are reduced, overfitting is avoided, and fast convergence and efficient classification are achieved. It is suitable for edge equipment with low computing power and meets the needs of the industrial market.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114882278B_ABST
    Figure CN114882278B_ABST
Patent Text Reader

Abstract

The present invention discloses a tire pattern classification method based on attention mechanism and transfer learning, including obtaining a training sample set by processing the original tire image through histogram equalization and data enhancement, constructing a training model including a ResNet 50 network, a channel space attention mechanism, a pooling layer and a fully connected layer, and obtaining a tire pattern classification model through cross entropy loss function training of category labels. The present invention also provides a method for compressing the tire pattern classification model, using a knowledge distillation method to train the tire pattern classification model as a teacher network to obtain a student network, and a compressed tire pattern classification model trained through a cross entropy loss function. The present invention also discloses a tire pattern classification device based on attention mechanism and transfer learning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a tire pattern classification method and device based on attention mechanism and transfer learning. Background Art

[0002] Tire tread images can often provide important clues for police investigations in criminal cases. In order to improve police work efficiency, it is particularly important to achieve efficient tire tread classification technology. In addition, in the commercial field of tire manufacturing, commercial disputes often occur due to patent infringement of tire treads.

[0003] Tire pattern images have rich textures, clear edges, and the visual effects of tire patterns of different models are similar, so texture features are often used for characterization. The literature Liu Y, Li Z, Gao Z M. An Improved Texture Feature Extraction Method for Tyre Tread Patterns [C] / / International Conference on Intelligent Science and Big Data Engineering. Springer Berlin Heidelberg, 2013. discloses an improved Tamura feature characterization image using brightness histogram statistical moments. The literature Wang S, Liu Y, Li D, et al. An Improved SIFT Feature Extraction Method for Tyre Tread Patterns Retrieval [C] / / 2014 Seventh International Symposium on Computational Intelligence and Design. 0. discloses a tire image characterization algorithm that combines DWT with scale-invariant features. The texture features extracted by traditional methods are low-level representations of image content, lacking visual features and high-level semantic features, which leads to a semantic gap.

[0004] In order to solve this problem, it is necessary to intelligently classify the tire pattern classification problem. With the rapid development of deep learning technology, convolutional neural networks have been widely used in image classification tasks due to their computational characteristics. In recent years, many scholars have conducted in-depth research on convolutional neural networks and achieved research results. References Krizhevsky A, Sutskever I, Hinton G. ImageNet Classification with Deep Convolutional Neural Networks [J]. Advances in neural information processing systems, 2012, 25 (2). The AlexNet model was made public. This model introduced a new deep structure and dropout method, which greatly reduced the error rate and subverted the field of image recognition. In addition, the more classic convolutional neural networks include GoogleNet and ResNet. However, when the data set is small, once the model structure is deepened, the model is likely to be overfitted, and training the entire model from scratch also takes a lot of time and computing resources. In addition, the above networks also have defects in extracting detailed features. Summary of the invention

[0005] The present invention provides a tire pattern classification method based on attention mechanism and transfer learning, which can reduce training time and computing resources and has a faster convergence speed.

[0006] A tire pattern classification method based on attention mechanism and transfer learning, comprising:

[0007] (1) obtaining an original tire image set with real labels, performing adaptive histogram equalization processing on each original tire image to obtain a transformed tire image set, obtaining a derived image set based on the transformed tire image set using a data enhancement method, and constructing a training sample set using the derived image set and the transformed tire image set;

[0008] (2) Constructing a training model, including a ResNet 50 network, a channel space attention mechanism, a pooling layer, and a fully connected layer. The ResNet 50 network is obtained by training the ImageNet dataset, and the feature extraction layer in the ResNet 50 network is retained. The feature extraction layer in the ResNet 50 network is used to extract the feature map of the training sample. The channel space attention mechanism is used to extract the features of the tire texture classification area in the input feature map to obtain a tire texture classification feature map. The tire texture classification feature map is sequentially passed through the pooling layer and the fully connected layer to obtain multiple predicted category labels of the training sample;

[0009] (3) constructing a first cross entropy loss function through multiple predicted category labels and true labels, and training and updating the parameters of the training model through the first cross entropy loss function to obtain a tire pattern classification model;

[0010] (4) When applied, the tire image is input into the tire pattern classification model to obtain the tire pattern type of the tire image.

[0011] The tire texture classification feature map is unified in size through the pooling layer, and then the tire texture classification feature map with consistent size is input into the fully connected layer for classification. The pooling layer is obtained by replacing the parallel maximum pooling layer in the pyramid pooling layer with the serial maximum pooling layer based on the pyramid pooling layer.

[0012] Adaptive histogram equalization is performed on each original tire image to obtain a transformed tire image set, including:

[0013] Adaptive histogram equalization is used to adjust the probability distribution of the RGB chromaticity level of each pixel of the original tire image, and the histogram distribution threshold is set. The probability of the RGB chromaticity level of the pixels exceeding the histogram distribution threshold is redistributed, making the histogram probability density distribution of the original tire image more uniform.

[0014] A derived image set is obtained by using a data augmentation method based on the transformed tire image set, where the data augmentation method includes rotation, cropping or exposure.

[0015] The tire pattern classification model is used as the teacher network, and the student network is trained by the knowledge distillation method. The student network includes a convolutional layer, multiple compression layers, a pooling layer, and a fully connected layer. The compression layer includes multiple bottleneck layers and SE layers. The training samples are sequentially input into the convolutional layer, multiple compression layers, the pooling layer, and the fully connected layer to obtain multiple predicted category labels of the student network.

[0016] The predicted category labels obtained by the teacher network and the multiple predicted category labels of the student network are used to construct a second cross entropy loss function, and the multiple predicted category labels of the student network and the true labels are used to construct a third cross entropy loss function. The parameters of the student network are trained and updated through the second and third cross entropy loss functions to obtain a compressed rear tire pattern classification model;

[0017] When applied, the tire image is input into the compressed tire pattern classification model to obtain the tire pattern type of the tire image.

[0018] The parameters of the teacher network are compressed through multiple bottleneck layers. The bottleneck layer includes a 1x1 convolution kernel and a one-channel 3x3 convolution kernel, which are used to compress the parameters in the calculation process and increase the nonlinearity of the model through the Relu activation function. Finally, a linear layer is used for dimensionality increase or decrease.

[0019] A tire pattern classification device based on attention mechanism and transfer learning, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein the tire pattern classification model according to any one of claims 1 to 4 is used in the computer memory;

[0020] When the computer processor executes the computer program, the following steps are implemented:

[0021] Input the tire image into the tire pattern classification model to obtain the tire pattern type of the tire image.

[0022] A tire pattern classification device based on attention mechanism and transfer learning, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, wherein the compressed tire pattern classification model according to any one of claims 5 to 6 is used in the computer memory;

[0023] The computer processor implements the following steps when executing the computer program: inputting the tire image into the compressed tire pattern classification model to obtain the tire pattern type of the tire image.

[0024] Compared with the prior art, the present invention has the following excellent effects:

[0025] First, CLAHE image preprocessing is performed, and a tire pattern classification method based on attention mechanism and transfer learning is proposed. The attention mechanism is used to improve the network's ability to extract detail features, improve the SPP structure, and speed up the model's operation. The transfer learning method is used to greatly reduce the training time and computing resources, and speed up the network's convergence speed to avoid overfitting. Through knowledge distillation and the design of a lightweight student model, the network can be easily deployed on edge devices with lower computing power to meet the needs of the industrial market. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The original tire image provided in the specific implementation manner and the transformed tire image obtained by adaptive histogram equalization processing, wherein Figure 1 a is the original tire picture, Figure 1 b is the tire image after transformation;

[0027] Figure 2 A structural diagram of a tire pattern classification model provided for a specific implementation method;

[0028] Figure 3 A schematic diagram of the structure of a student network provided for a specific implementation method;

[0029] Figure 4 A schematic diagram of the structure of the bottleneck layer provided for a specific implementation method. DETAILED DESCRIPTION

[0030] The present invention provides a tire pattern classification method based on attention mechanism and transfer learning, and the specific steps are as follows:

[0031] (1) Obtain a training sample set and obtain a set of original tire images with real labels, such as Figure 1 As shown in a, the original tire image has a relatively single color and a relatively concentrated color level distribution, which makes the tire pattern texture characteristics easily affected by background environment and other noise. The original tire image is processed by adaptive histogram equalization to obtain the transformed tire image, as shown in Figure 1 As shown in Figure 2, histogram equalization is a transformation method that makes the color distribution of an image more uniform. According to information theory, the greater the entropy of information, the more information it contains. The entropy calculation formula is as follows:

[0032]

[0033] Among them, χ i is the RGB chromaticity level of the i-th pixel, n is the total number of pixels, p(χ i ) is the probability of the RGB chromaticity level of the i-th pixel.

[0034] So when p(χ i ) is more evenly distributed, the entropy value is larger. Corresponding to the image, when the image histogram can be more evenly distributed, the image contrast is larger, so as to achieve the effect of enhancing the tire pattern characteristics. And in order to prevent some areas from becoming noise points due to excessive contrast enhancement. By setting the threshold of the histogram distribution, the distribution exceeding the threshold is "evenly" dispersed to the probability density distribution, thereby limiting the increase of the conversion function (cumulative histogram). In addition, the interpolation algorithm is used to improve the calculation efficiency. Although this process increases the amount of linear interpolation calculation, it greatly reduces the number of calculations of the transformation function, so the calculation speed is improved.

[0035] The original dataset is collected from the public dataset and network of Kaggle. In order to prevent overfitting of the network and increase the generalization ability of the network, we use data enhancement methods such as rotation, cropping, and exposure to generate many derivative images by transforming the tire images. The derivative image set and the transformed tire image set construct a training sample set to increase the number of images in this dataset.

[0036] (2) Construct a training model, including a ResNet 50 network, a convolutional block attention module (CBAM), a pooling layer, and a fully connected layer. The feature extraction layer in the ResNet 50 network is retained. The feature extraction layer in the ResNet 50 network is used to extract the feature map of the training sample. The channel space attention mechanism is used to extract features from the tire texture classification area in the input feature map to obtain a tire texture classification feature map. The tire texture classification feature map is sequentially passed through the pooling layer and the fully connected layer to obtain multiple predicted category labels of the training sample.

[0037] The tire texture classification feature map is unified in size through the pooling layer, where the pooling layer (SPPF) is obtained by replacing the parallel maximum pooling layer in the pyramid pooling layer with the serial maximum pooling layer based on the pyramid pooling layer (SPP). And the kernel_size is the same for all. The purpose of this modification is to practice that the serial max_pool can also get the result of spp, and at the same time it can bring more performance improvements, including forward calculation and reverse calculation. After testing, after the SPP network was modified to SPPF, the number of parameters did not change, but SPPF used less GPU, and the forward calculation time was reduced by 25ms, the back propagation time was reduced by 57ms, and the overall calculation time was reduced.

[0038] Then, the tire texture classification feature map of the same size is input into the fully connected layer, and the obtained feature map is flattened as the neurons of the first fully connected layer. Two more fully connected layers are set up afterwards. Normalization and dropout operations are added between every two adjacent fully connected neuron layers to enhance the generalization ability of the model and avoid overfitting. Finally, 10 neurons and the scores of 10 types of patterns are output, and 10 predicted category label probabilities are obtained after normalization.

[0039] (2.1) Construct a ResNet 50 network through transfer learning. The specific steps are as follows:

[0040] The accuracy of texture classification is a prerequisite for improving the judgment of tire tread texture. If a deep learning method is used, a large number of tire tread image data sets need to be used as samples to train the deep learning model to solve the tire tread image recognition classification problem. When the data set is small, once the model structure is deepened, the problem of model overfitting is likely to occur. The generalization ability of the model obtained by such training is insufficient, resulting in a high accuracy not being obtained on the verification set. And if the entire model is retrained from scratch, it also takes a lot of time and computing resources, which is relatively difficult, so the present invention proposes to use transfer learning to solve the above problems. The present invention uses the model parameters trained on the huge dataset of ImageNet for initialization, rather than randomly initializing the model parameters. The feature extraction layer is retained, the other network parameters are kept unchanged, the fully connected layer and the pooling layer of the network are modified, and the fully connected layer is retrained on its own data set. In this way, the entire network can be trained with a smaller learning rate.

[0041] There are many commonly used image classification models in deep convolutional neural networks, such as AlexNet, VGG, GoogLeNet, ResNet, DesneNet

[12] , and PReLU-Net. The depth of the convolutional network model is greatly increased, which greatly improves the classification accuracy of the model. Table 1 shows the Top-1 error and Top-5 error of several representative deep learning models trained on the ImageNet dataset.

[0042] Table 1 Accuracy of commonly used classification models on ImageNet

[0043]

[0044] In Table 1, the smaller the Top-1error and Top-5error values ​​of the model, the higher the accuracy of the model in image classification. From the Top-1 and Top-5 accuracy on the ImageNet dataset in Table 1, it can be seen that ResNet 50 has a certain advantage in accuracy. Although there are larger and more complex networks with higher accuracy such as DesneNet

[12] , deeper networks are too large and require a higher hardware environment for network training. Therefore, this paper chooses ResNet50 as the basic network.

[0045] (2.2) The channel-space attention mechanism extracts features from the tire texture classification area in the input feature map to obtain a tire texture classification feature map. The tire texture classification area is the depth and shape of the texture. The specific steps are: input the feature map into CBAM to calculate the attention map in the feature map from the channel and space dimensions, and then multiply the attention map with the input feature map to obtain the tire texture classification feature map.

[0046] (3) Construct a cross entropy loss function. The first cross entropy loss function is constructed through the 10 predicted category label probabilities and the true labels. The tire pattern classification model is obtained by training and updating the parameters of the training model through the first cross entropy loss function. The model structure is as follows: Figure 2 As shown;

[0047] The present invention also provides a method for compressing a tire pattern classification model to achieve lightweighting of the model, and the specific steps are as follows:

[0048] Through the knowledge distillation method, the tire pattern classification model is used as the teacher network to train the student network. The student network includes a convolutional layer, two compression layers, a pooling layer, and a fully connected layer. The compression layer includes three bottleneck layers and an SE layer, such as Figure 3 As shown in the figure, the training samples are first convolved (Conv2d), and the bottleneck layer (bottlenck) uses a 1x1 convolution kernel and a 3x3 convolution kernel of one channel, which reduces the number of parameters in the operation process, speeds up the operation, and increases the nonlinearity of the model through the Relu activation function. Finally, a linear layer is used for dimensionality increase or decrease, as shown in the figure. Figure 4 As shown. After the same operation is performed three times, the SE structure is added for feature extraction to strengthen the focus on the recognition area. The above operation is repeated, and finally the SPPF layer is input to unify the size and the result is input as the fully connected layer to finally obtain the classification result. The probabilities of multiple predicted category labels of the student network are obtained;

[0049] The predicted category labels obtained by the teacher network and the multiple predicted category label probabilities of the student network are used to construct a second cross entropy loss function, and the multiple predicted category label probabilities of the student network and the true labels are used to construct a third cross entropy loss function. The parameters of the student network are trained and updated through the second and third cross entropy loss functions to obtain a compressed rear tire pattern classification model;

[0050] When applied, the tire image is input into the compressed tire pattern classification model to obtain the tire pattern type of the tire image.

[0051] Among them, the specific implementation method of the SE structure is: 1. Perform global average pooling on the input feature map; 2. Then perform two full connections. The number of neurons in the first full connection is small, and the number of neurons in the second full connection is the same as the input feature layer; 3. After completing two full connections, the value is fixed to between 0 and 1 through Sigmoid. At this time, we obtain the weight of each channel of the input feature layer (between 0 and 1); 4. After obtaining this weight, we multiply this weight by the original input feature map to obtain the output feature map.

[0052] Performance Analysis:

[0053] Load the tire pattern dataset to the tire pattern classification model, train the model in the built PyTorch environment, save the accuracy of each iteration in the training process to the log file, and then use tensorboard to draw the Accuracy change curve on the validation set. In order to verify the effectiveness of the method proposed in this paper, we conducted 5 control experimental groups, including no CLAHE processing and using SDG optimizer; using SDG optimizer and CLAHE processing without adding attention mechanism; using Adam optimizer and CLAHE processing without attention mechanism; using Adam optimizer without adding Attention mechanism, and the scheme proposed in this paper. The final drawing results are shown in Table 2.

[0054] Table 2 shows the best accuracy during training.

[0055]

[0056] From the figure, we can see that due to the use of transfer learning to use the model weights obtained on ImageNet as the feature extraction layer parameters, the model can obtain a good accuracy from the beginning, and as the number of training times changes, the model obtains tire pattern features through the attention mechanism, which further improves the accuracy. In addition, from the comparison of the results of each group in the above table, it can be seen that each method proposed in this paper has obtained positive feedback during the experiment, which can enhance the accuracy and convergence speed of the model for tire texture classification.

[0057] Compressed model results

[0058]

[0059] From the table above, we can see that the student model has a model recognition accuracy similar to that of the teacher model, while having much less complexity and parameters than the teacher model. It greatly reduces the time for recognition and classification, and the network is easy to deploy on the edge with weaker computing power. It has great application prospects and improves the automation level of the tire industry.

Claims

1. A tire pattern classification method based on attention mechanism and transfer learning, It is characterized in that include: (1) obtaining an original tire image set with real labels, performing adaptive histogram equalization processing on each original tire image to obtain a transformed tire image set, obtaining a derived image set based on the transformed tire image set using a data enhancement method, and constructing a training sample set using the derived image set and the transformed tire image set; (2) Constructing a training model, including a ResNet 50 network, a channel space attention mechanism, a pooling layer, and a fully connected layer. The ResNet 50 network is obtained by training the ImageNet dataset, and the feature extraction layer in the ResNet 50 network is retained. The feature extraction layer in the ResNet50 network is used to extract the feature map of the training sample. The channel space attention mechanism is used to extract the features of the tire texture classification area in the input feature map to obtain a tire texture classification feature map. The tire texture classification feature map is sequentially passed through the pooling layer and the fully connected layer to obtain multiple predicted category label probabilities of the training sample; (3) constructing a first cross entropy loss function through multiple predicted category label probabilities and true labels, and training and updating the parameters of the training model through the first cross entropy loss function to obtain a tire pattern classification model; (4) When applied, the tire image is input into the tire pattern classification model to obtain the tire pattern type of the tire image.

2. The tire pattern classification method based on attention mechanism and transfer learning according to claim 1, It is characterized in that The tire texture classification feature map is unified in size through the pooling layer, and then the tire texture classification feature map with consistent size is input into the fully connected layer for classification. The pooling layer is obtained by replacing the parallel maximum pooling layer in the pyramid pooling layer with the serial maximum pooling layer based on the pyramid pooling layer.

3. The tire pattern classification method based on attention mechanism and transfer learning according to claim 1, It is characterized in that Adaptive histogram equalization is performed on each original tire image to obtain a transformed tire image set, including: Adaptive histogram equalization is used to adjust the probability distribution of the RGB chromaticity level of each pixel of the original tire image, and the histogram distribution threshold is set. The probability of the RGB chromaticity level of the pixels exceeding the histogram distribution threshold is redistributed, making the histogram probability density distribution of the original tire image more uniform.

4. The tire pattern classification method based on attention mechanism and transfer learning according to claim 1, It is characterized in that A derived image set is obtained by using a data augmentation method based on the transformed tire image set, where the data augmentation method includes rotation, cropping or exposure.

5. The tire pattern classification method based on attention mechanism and transfer learning according to claim 1 or 2, It is characterized in that The tire pattern classification model is used as the teacher network, and the student network is trained by the knowledge distillation method. The student network includes a convolutional layer, multiple compression layers, a pooling layer, and a fully connected layer. The compression layer includes multiple bottleneck layers and SE layers. The training samples are sequentially input into the convolutional layer, multiple compression layers, pooling layers, and fully connected layers to obtain multiple predicted category label probabilities of the student network. The predicted category labels obtained by the teacher network and the multiple predicted category label probabilities of the student network are used to construct a second cross entropy loss function, and the multiple predicted category label probabilities of the student network and the true labels are used to construct a third cross entropy loss function. The parameters of the student network are trained and updated through the second and third cross entropy loss functions to obtain a compressed rear tire pattern classification model; When applied, the tire image is input into the compressed tire pattern classification model to obtain the tire pattern type of the tire image.

6. The tire pattern classification method based on attention mechanism and transfer learning according to claim 5, It is characterized in that The parameters of the teacher network are compressed through multiple bottleneck layers. The bottleneck layer includes a 1x1 convolution kernel and a one-channel 3x3 convolution kernel, which are used to compress the parameters in the calculation process and increase the nonlinearity of the model through the Relu activation function. Finally, a linear layer is used for dimensionality increase or decrease.

7. A tire pattern classification device based on attention mechanism and transfer learning, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, It is characterized in that The computer memory uses the tire pattern classification model described in any one of claims 1 to 4; When the computer processor executes the computer program, the following steps are implemented: Input the tire image into the tire pattern classification model to obtain the tire pattern type of the tire image.

8. A tire pattern classification device based on attention mechanism and transfer learning, comprising a computer memory, a computer processor, and a computer program stored in the computer memory and executable on the computer processor, It is characterized in that The computer memory adopts the compressed tire pattern classification model according to any one of claims 5 to 6; The computer processor implements the following steps when executing the computer program: inputting the tire image into the compressed tire pattern classification model to obtain the tire pattern type of the tire image.