Tire defect detection method based on anomaly detection and conditional generative adversarial network

Abstract

The invention discloses a tire defect detection method based on anomaly detection and a conditional generative adversarial network, and the method comprises the steps: inputting a to-be-detected tire image into a tire defect detection model, and obtaining a defect detection result; the tire defect detection model is obtained by taking a total training data set as a training set for training; the training data total set is obtained by expanding a real data set by applying an auxiliary classification generative adversarial network based on feature reconstruction, and attention modules with abnormal weights are embedded in a generator and a discriminator of the auxiliary classification generative adversarial network based on feature reconstruction. The attention module with the abnormal weight is a CBAM attention module combined with an abnormal weight matrix. According to the method, the quality of the training set is improved through the generative data enhancement model, and the method is suitable for a small sample unbalanced data environment; the data enhancement effect is improved through the idea of feature reconstruction and the attention module with the abnormal weight; detection precision is high and an application prospect is good.

Description

Technical Field

[0001] The present invention relates to the field of image defect detection, and in particular to a method for detecting tire defects by applying an image defect detection method, and more particularly to a tire defect detection method based on anomaly detection and conditional generative adversarial network and its application. Background Art

[0002] With the vigorous development of the automotive industry and the increasing number of automobiles, as a key component of vehicle safety, the quality and safety performance of tires have been increasingly emphasized. During the tire production process, due to multiple factors such as the external environment, material properties, and production processes, various structural defects will inevitably occur, which not only affect the service life of the tires but may also lead to serious traffic accidents, directly threatening the life and property safety of people inside and outside the vehicle. Therefore, how to quickly and accurately detect tire defects has become an important problem to be solved urgently in the tire manufacturing industry. With the rapid development of technologies such as computer vision and machine learning, data-driven intelligent tire defect detection technologies have gradually become a research hotspot due to their advantages such as accuracy, speed, and no need for manual participation.

[0003] In recent years, data-driven tire defect detection mainly relies on deep learning technologies. For example, various models designed based on convolutional neural networks (CNNs) can automatically extract and learn deep features of tire defects in complex image data by virtue of their powerful feature learning ability and efficient parallel processing ability. These features often go beyond the scope that traditional image processing models can capture. By training on large-scale labeled datasets, deep learning models can gradually optimize their internal parameters and improve the accuracy and generalization ability of defect detection. However, the performance of deep learning models is limited by expensive labeling and scarce samples. Especially in actual industrial production scenarios, the number of tire defect samples is small and the types of defects tend to be diverse, which results in a small-sample and unbalanced data environment when the model is trained.

[0004] To address the above problems, a reliable method is to augment the training data through generative models represented by generative adversarial networks (GANs) to improve the training quality of defect detection models. Traditional GANs usually achieve good performance on public datasets, but on industrial image datasets with strong domain specificity and uneven data quality, their generation effects are usually not satisfactory. On the one hand, traditional GANs use the JS divergence to measure the difference between the generated distribution and the real distribution. Once there is no overlap between the two distributions, the JS divergence will become a constant, resulting in the gradient vanishing problem. Even if the gradient does not vanish, it is still very difficult to balance the capabilities between the generator and the discriminator. If not handled well, it will lead to mode collapse, resulting in images with poor quality and serious homogenization. On the other hand, for some small-scale defects, it is difficult for GANs to capture them from the complete image.

[0005] Therefore, it is of great practical significance to develop a tire defect detection method that is applicable to small - sample unbalanced data environments, has good data generation effects, and high defect capture sensitivity. Summary of the Invention

[0006] Due to the above - mentioned defects in the prior art, the present invention provides a tire defect detection method that is applicable to small - sample unbalanced data environments, has good data generation effects, and high defect capture sensitivity, which overcomes the problems of poor applicability of existing tire defect detection methods to small - sample unbalanced data environments, poor data generation effects, and difficulty in capturing small - scale defects.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A tire defect detection method based on anomaly detection and conditional generative adversarial network, which inputs the tire image to be detected into a tire defect detection model, and the tire defect detection model outputs the defect detection result of the tire image to be detected;

[0009] The training process of the tire defect detection model is a process of taking the image data in the total training data set as input, taking the known defect detection result corresponding to the image data as the theoretical output, and continuously adjusting the parameters of the model;

[0010] The total training data set is obtained by augmenting the real data set using an auxiliary classification generative adversarial network based on feature reconstruction. In the generator and discriminator of the auxiliary classification generative adversarial network based on feature reconstruction (abbreviated as FR - ACGAN network), an attention module with anomaly weights is embedded to improve the network's learning and reconstruction ability for defects. The attention module with anomaly weights is a CBAM attention module combined with an anomaly weight matrix S, and the anomaly weight matrix S is obtained by processing the real data set using the PaDiM anomaly detection algorithm.

[0011] The present invention significantly improves the quality of the training set of the defect detection model based on deep learning through a generative data augmentation model, thereby reducing the negative impact of small - sample and unbalanced data environments on defect detection in the actual production process; in addition, through the idea of feature reconstruction and the AW - CBAM module, the effect of data augmentation is improved. Finally, the feasibility and competitiveness of this method are verified on the tire image data set collected in actual enterprises.

[0012] The tire defect detection method based on anomaly detection and conditional generative adversarial network of the present invention is reasonably designed. By transforming the auxiliary classification generative adversarial network based on feature reconstruction and embedding an attention module with anomaly weights in its generator and discriminator, the network's learning and reconstruction capabilities for defects are greatly improved. The transformed auxiliary classification generative adversarial network based on feature reconstruction is used to expand the limited dataset, and its data generation effect is good and the defect capture sensitivity is high. Then, the expanded dataset is used to train the model, and the model training accuracy is high, that is, the detection accuracy of tire defects is high. The above method is particularly suitable for the industrial data environment with small sample imbalance and has great application prospects.

[0013] As a preferred technical solution:

[0014] For the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above, the tire image to be detected is preprocessed before being input into the tire defect detection model;

[0015] The image data in the real dataset is the image data after image preprocessing.

[0016] For the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above, the operations of the image preprocessing are as follows:

[0017] (1) Perform histogram equalization on the original image to enhance the global contrast of the image and highlight important local information;

[0018] (2) Perform edge enhancement on the image processed in step (1) to highlight the contour information and enable the model to better focus on the important structures and features in the image;

[0019] (3) Sharpen the image processed in step (2) to enhance the edges and details in the image and improve the model's perception ability of subtle differences;

[0020] (4) Normalize the image processed in step (3) to convert the image into a form suitable for model input and promote the convergence of model training.

[0021] For the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above, the real dataset includes normal class image data and defect class image data;

[0022] Take 70% of the data of each category in the real dataset as real data training samples to form the real data training set, and take the other 30% of the data of each category in the real dataset as real data test samples to form the real data test set.

[0023] The tire defect detection method based on anomaly detection and conditional generative adversarial network as described above, wherein the tire defect detection model is an EfficientNet model. Those skilled in the art can select a suitable model as the tire defect detection model according to actual needs. Here, only a feasible technical solution is given.

[0024] The training of the tire defect detection model in the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above is specifically as follows:

[0025] First, pre-train the tire defect detection model using a real dataset; change the number of neurons in the last fully connected layer of the pre-trained tire defect detection model to the number of image categories in the specific task; finally, perform full-parameter fine-tuning using the total training dataset, and evaluate the performance of the tire defect detection model through the real data test set. When the performance meets the actual needs or the model converges sufficiently, stop the training.

[0026] The acquisition process of the CBAM attention module combined with the anomaly weight matrix S in the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above is as follows:

[0027] 1) In the training stage of the PaDiM anomaly detection algorithm, extract the embedding vectors of different semantic levels of each image patch in the normal images through the pre-trained ResNet18 network. The l-th layer embedding vector of the image patch at position (i, j) can be expressed as f l (i, j). Concatenate these embedding vectors along the channel dimension to obtain an embedding vector e ij =concat(f 1 (i, j), f 2 (i, j), …, f L (i, j));

[0028] 2) For the N normal images in the real data training set, the set of embedding vectors at each image position (i, j) is . Assume that is generated by a multivariate Gaussian distribution N(μ ij , Σ ij ), where the sample mean μ ij and the sample covariance matrix Σ ij can be calculated as follows:

[0029]

[0030]

[0031] where ∈ is a small regularization term used to ensure the invertibility of the covariance matrix, and I is the identity matrix;

[0032] 3) In the inference stage of the PaDiM anomaly detection algorithm, for a defective image, each image patch extracts an embedding vector containing multi-semantic level information through the same pre-trained ResNet18 network, and calculates the Mahalanobis distance between this vector and the multivariate Gaussian distribution at the corresponding position as the anomaly score of this image patch. For the embedding vector at the position (i, j) of the test image , its anomaly score s ij can be obtained by the following formula:

[0033] ;

[0034] s ij The larger the value, the greater the probability that the image patch is abnormal. Thus, for an image with height height and width width, its anomaly weight matrix S can be expressed as ;

[0035] 4) Combine the anomaly weight matrix with the CBAM attention module to construct the AW-CBAM module. The CBAM module includes a channel attention module CAM and a spatial attention module SAM. In CAM, the feature map X obtains two different feature embeddings through average pooling and max pooling, and these two feature embeddings generate channel weights M C through a shared multi-layer perceptron, which is used to weight each channel of the input feature map. In SAM, the feature map X·M C after being processed by CAM performs average pooling and max pooling in the channel dimension to obtain two feature embeddings. After splicing these two feature embeddings, a convolutional layer is used to generate spatial attention weights M S , which is weighted with the input in the spatial dimension. The final output of CBAM is (X·M C )·M S . When X is the original image, multiply the output of CBAM element-wise by S, which is the output of AW-CBAM (X·M C )·M S ·S. When X is a certain feature map obtained after the original image is processed by a deep learning network, then S needs to be processed in exactly the same way as above to obtain the anomaly feature embedding S'. Multiply the output of CBAM element-wise by S' to obtain the output of AW-CBAM (X·M C )·M S ·S'.

[0036] For the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above, specifically, in the generator and discriminator of the auxiliary classification generative adversarial network based on feature reconstruction, the attention module with anomaly weights is embedded as follows:

[0037] In the encoder of the generator of the auxiliary classification generative adversarial network based on feature reconstruction, there are five layers in total. The first layer is a convolutional layer, and the following four layers are all downsampling layers. Since the attention module with abnormal weights does not change the shape of the feature map, the attention module with abnormal weights is directly used as an additional network layer and embedded between the convolutional layer and the subsequent downsampling layer in the encoder part of the generator;

[0038] The bottom network of the discriminator of the auxiliary classification generative adversarial network based on feature reconstruction consists of four downsampling layers, and the attention module with abnormal weights is embedded between the first downsampling layer and the second downsampling layer.

[0039] The construction of the auxiliary classification generative adversarial network based on feature reconstruction and the embedding operation of the attention module with abnormal weights are as follows:

[0040] A) Construct the FR-ACGAN network (auxiliary classification generative adversarial network based on feature reconstruction). Its basic architecture includes a generator G and a discriminator D. The generator is used to reconstruct a fake image of a specified category from a real image to deceive the discriminator as much as possible, while the discriminator is used to correctly identify the authenticity and category of a given image as much as possible. The two continuously improve their respective performances through adversarial training. In FR-ACGAN, the generator is a CNN-based encoder-decoder structure, and the number of layers of the encoder and decoder is the same, and the size of the feature maps in the corresponding layers is exactly the same. The reference image x is encoded into a vector by the encoder E, multiplied element-wise by the given category vector c, and then added element-wise to the random noise vector z, and then used as the input of the decoder R. The output of the decoder is the fake image. The discriminator contains a real / fake discrimination network D src and a classification network D cls , and the two share a CNN-based bottom feature extraction network.

[0041] B) Embed the AW-CBAM module (i.e., the attention module with abnormal weights) into the generator and discriminator of FR-ACGAN;

[0042] In the generator of FR-ACGAN, there are five layers in total in the encoder. The first layer is a convolutional layer, and the following four layers are all downsampling layers. Since AW-CBAM does not change the shape of the feature map, it is directly used as an additional network layer and embedded between the convolutional layer and the subsequent downsampling layer in the encoder part of the generator.

[0043] The bottom network of the discriminator of FR-ACGAN consists of four downsampling layers, and AW-CBAM is embedded between the first downsampling layer and the second downsampling layer.

[0044] C) After constructing the FR-ACGAN network structure with AW-CBAM, adversarial training begins. During training, all the images (real images) in the real data training set and their class labels are used as the input to the generator in FR-ACGAN, and fake images are output, whose labels are considered the same as the reference real images. All the real images and fake images are used as the input to the discriminator, and the output includes the authenticity discrimination results of each image and the auxiliary classification results.

[0045] D) Based on the loss function of FR-ACGAN, the Adam optimizer is used to optimize the network parameters of the generator and the discriminator. The loss function of the generator is as follows:

[0046]

[0047] where λ adv 、λ aux and λ mrec are the weight coefficients of the adversarial loss, the auxiliary classification loss, and the multi-level reconstruction loss term respectively, P z (z) is the noise distribution, P data (x, c) is the joint distribution of the real image and the class, E k (x) is the output of the k-th layer of the generator encoder, R k (z, c, E(x)) is the output of the k-th layer of the decoder, α k is the weight coefficient of the k-th layer reconstruction loss term;

[0048] The loss function of the discriminator is as follows:

[0049]

[0050] where, x' is the generated fake image, p G is the distribution of the fake image.

[0051] E) When the loss function converges sufficiently or the adversarial training reaches the preset maximum number of iterations, the training stops. A batch of images is sampled from the real data training set as reference images, and the generator is used to generate the same number of fake images to expand the total training data set. The specific sampling method and quantity can be customized by the user.

[0052] The present invention also provides a computer device, and the computer device includes:

[0053] At least one processor; and,

[0054] A memory communicatively connected to the at least one processor; wherein,

[0055] The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above is implemented.

[0056] In addition, the present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the tire defect detection method based on anomaly detection and conditional generative adversarial network as described above is implemented.

[0057] The above technical solutions are only one feasible technical solution of the present invention, and the protection scope of the present invention is not limited thereto. Those skilled in the art can reasonably adjust the specific design according to actual needs.

[0058] The above-mentioned invention has the following advantages or beneficial effects:

[0059] (1) The tire defect detection method based on anomaly detection and conditional generative adversarial network of the present invention is reasonably designed. By transforming the auxiliary classification generative adversarial network based on feature reconstruction and embedding an attention module with anomaly weights in its generator and discriminator, the network's learning and reconstruction ability for defects is greatly improved.

[0060] (2) The tire defect detection method based on anomaly detection and conditional generative adversarial network of the present invention uses the transformed auxiliary classification generative adversarial network based on feature reconstruction to expand a limited dataset. Its data generation effect is good and the defect capture sensitivity is high. Then, the expanded dataset is used to train the model, and the model training accuracy is high, that is, the detection accuracy is high.

[0061] (3) The tire defect detection method based on anomaly detection and conditional generative adversarial network of the present invention is particularly suitable for the industrial data environment with small samples and imbalance, and has good application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, the present invention and its features, shape and advantages will become more obvious. The same reference numerals indicate the same parts in all the drawings. The drawings are not drawn to scale, and the emphasis is on showing the gist of the present invention.

[0063] Figure 1 It is a schematic flowchart of the tire defect detection method based on anomaly detection and conditional generative adversarial network of the present invention;

[0064] Figure 2 It is a network structure diagram of the generator of the auxiliary classification generative adversarial network based on feature reconstruction;

[0065] Figure 3It is the discriminator network structure diagram of the auxiliary classification generative adversarial network based on feature reconstruction;

[0066] Figure 4 It is the generation effect of the auxiliary classification generative adversarial network based on feature reconstruction on various defect images under different training times;

[0067] Figure 5 It is the confusion matrix obtained by the tire defect detection model in the multi-classification task of defects after data augmentation. Specific implementation mode

[0068] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but it is not a limitation of the present invention.

[0069] Embodiment 1

[0070] A tire defect detection method based on anomaly detection and conditional generative adversarial network is specifically as follows:

[0071] After preprocessing the tire image to be detected, it is input into the tire defect detection model, and the tire defect detection model outputs the defect detection result of the tire image to be detected.

[0072] The specific operations of image preprocessing are as follows:

[0073] (1) Perform histogram equalization processing on the original image;

[0074] (2) Perform edge enhancement on the image processed in step (1);

[0075] (3) Sharpen the image processed in step (2);

[0076] (4) Standardize the image processed in step (3), convert the image into a form suitable for model input, and promote the convergence of model training.

[0077] The acquisition process of the tire defect detection model is as follows:

[0078] I) Obtain a real dataset, the real dataset includes normal-class image data and defect-class image data, and perform the above-mentioned image preprocessing operations on the images in the real dataset. Divide the preprocessed real dataset into a real data training set and a real data test set. Specifically, take 70% of the data of each category in the real dataset as real data training samples to form the real data training set, and take the other 30% of the data of each category in the real dataset as real data test samples to form the real data test set;

[0079] II) Obtain a CBAM attention module combined with an anomaly weight matrix S, specifically:

[0080] 1) During the training phase of the PaDiM anomaly detection algorithm, the embedding vectors of different semantic levels of each image patch in the normal images are extracted through the pre-trained ResNet18 network. The l-th layer embedding vector of the image patch at position (i, j) can be represented as f l (i,j). These embedding vectors are concatenated along the channel dimension to obtain an embedding vector e ij =concat(f 1 (i,j),f 2 (i,j),…,f L (i,j));

[0081] 2) For the N normal images in the real data training set, the set of embedding vectors at each image position (i, j) is . Assume that is generated by the multivariate Gaussian distribution N(μ ij ,Σ ij ), where the sample mean μ ij and the sample covariance matrix Σ ij can be calculated as follows:

[0082]

[0083]

[0084] where ϵ is a regularization term and I is the identity matrix;

[0085] 3) During the inference phase of the PaDiM anomaly detection algorithm, for a defective image, the embedding vector containing multi-semantic level information of each image patch is extracted through the same pre-trained ResNet18 network, and the Mahalanobis distance between this vector and the multivariate Gaussian distribution at the corresponding position is calculated as the anomaly score of this image patch. For the embedding vector at the test image position (i, j), its anomaly score s ij can be obtained from the following formula:

[0086] ;

[0087] For an image with height height and width width, its anomaly weight matrix S can be represented as ;

[0088] 4) Combine the anomaly weight matrix with the CBAM attention module. The CBAM module includes a channel attention module CAM and a spatial attention module SAM. In CAM, the feature map X is subjected to average pooling and max pooling to obtain two different feature embeddings, and these two feature embeddings are passed through a shared multi-layer perceptron to generate the channel weight MC , which is used to weight each channel of the input feature map. In SAM, the feature map X·M after being processed by CAM C Performs average pooling and max pooling on the channel dimension to obtain two feature embeddings. After concatenating these two feature embeddings, a convolutional layer is used to generate the spatial attention weight M S , which is weighted with the input in the spatial dimension, and the final output of CBAM is (X·M C )·M S , when X is the original image, the output of CBAM is multiplied element-wise with S, that is, the output (X·M C )·M S ·S. When X is a certain feature map obtained after the original image is processed by a deep learning network, then S needs to undergo the exact same processing as above to obtain the abnormal feature embedding S'. The output of CBAM is multiplied element-wise with S' to obtain the output (X·M C )·M S ·S';

[0089] III) Construct an auxiliary classification generative adversarial network based on feature reconstruction and embed an attention module with abnormal weights, and then expand the dataset. The specific operations are as follows:

[0090] A) Construct an FR-ACGAN network (auxiliary classification generative adversarial network based on feature reconstruction). Its basic architecture includes a generator G and a discriminator D. The generator is used to reconstruct a fake image of a specified category from a real image to deceive the discriminator as much as possible, while the discriminator is used to correctly identify the authenticity and category of a given image as much as possible. The two continuously improve their respective performances through adversarial training. In FR-ACGAN, the generator is an encoder-decoder structure based on CNN, and the number of layers of the encoder and decoder is the same, and the sizes of the feature maps in the corresponding layers are exactly the same. The reference image x is encoded into a vector by the encoder E, multiplied element-wise with the given category vector c, and then added element-wise with the random noise vector z, and then used as the input of the decoder R. The output of the decoder is the fake image. The discriminator contains a real / fake discrimination network D src and a classification network D cls , and the two share a bottom-layer feature extraction network based on CNN;

[0091] B) Embed the AW-CBAM module (i.e., the attention module with abnormal weights) into the generator and discriminator of FR-ACGAN;

[0092] In the generator of FR-ACGAN, the encoder has five layers. The first layer is a convolutional layer, and the following four layers are all downsampling layers. Since AW-CBAM does not change the shape of the feature map, it is directly used as an additional network layer and embedded between the convolutional layer and the subsequent downsampling layer in the encoder part of the generator;

[0093] The bottom network of the discriminator of FR-ACGAN consists of four downsampling layers, and AW-CBAM is embedded between the first downsampling layer and the second downsampling layer;

[0094] C) After constructing the FR-ACGAN network structure with AW-CBAM, adversarial training begins. During training, all the images (real images) and their class labels in the real data training set are used as the input to the generator in FR-ACGAN, and fake images are output. Their labels are regarded as the same as the reference real images. All the real images and fake images are used as the input to the discriminator, and the output includes the authenticity discrimination results of each image and the auxiliary classification results;

[0095] D) Based on the loss function of FR-ACGAN, the Adam optimizer is used to optimize the network parameters of the generator and the discriminator. The loss function of the generator is as follows:

[0096]

[0097] where λ adv 、λ aux and λ mrec are the weight coefficients of the adversarial loss, the auxiliary classification loss, and the multi-level reconstruction loss term respectively, P z (z) is the noise distribution, P data (x,c) is the joint distribution of the real image and the class, E k (x) is the output of the k-th layer of the generator encoder, R k (z,c,E(x)) is the output of the k-th layer of the decoder, and α k is the weight coefficient of the k-th layer reconstruction loss term;

[0098] The loss function of the discriminator is as follows:

[0099]

[0100] where, x' is the generated fake image, and p G is the distribution of the fake image;

[0101] E) When the loss function converges sufficiently or the adversarial training reaches the preset maximum number of iterations, stop the training. Sample a batch of images from the real data training set as reference images, and use the generator to generate the same number of fake images to expand the total training data set. The specific sampling method and quantity can be customized by the user;

[0102] IV) First, pre-train the tire defect detection model (EfficientNet model) using the real data set, and change the number of neurons in the last fully connected layer of the pre-trained tire defect detection model (EfficientNet model) to the number of image categories in the specific task. Finally, perform full-parameter fine-tuning using the total training data set, and evaluate the performance of the tire defect detection model through the real data test set. When the performance meets the actual requirements or the model converges sufficiently, stop the training.

[0103] Specifically, in this embodiment, on a tire production line of an enterprise, 320 8-bit grayscale images of tires are obtained through an X-ray imaging system as the real data set, including 100 normal-class images and 220 defect-class images. There are 4 types of defect categories, namely cord misalignment (80 images), sidewall foreign object (54 images), belt splice open (46 images), and bubble (40 images). According to the specific processes described in steps (1)-(4), preprocess this batch of images. For each type of preprocessed image, take 70% of them as samples of the real data training set and 30% as samples of the real data test set.

[0104] According to step II), process all the normal images and defect images in the real data training set to obtain the pixel-level anomaly weight matrix for each defect image, as shown in the "Anomaly Detection of DefectImages" module in Figure 1 For normal-class images, since there are no defects, the values of their anomaly weight matrices are all set to 1. According to step 4), combine the anomaly weight matrix with the CBAM attention module in the manner shown in the "CBAM with Anomaly Weight" module to form AW-CBAM, as shown in the "CBAM with Anomaly Weight" module in Figure 1

[0105] According to step A), construct the network structure of FR-ACGAN for expanding the training set, including two parts: a generator and a discriminator, as shown in the "Image Generation by Improved ACGAN" module in Figure 1 According to step B), integrate the AW-CBAM module into the generator and discriminator networks of FR-ACGAN. So far, the specific network structures of the generator and discriminator are respectively as shown inFigure 2 and Figure 3 As shown. According to steps C) - E), FR-ACGAN is trained to continuously optimize its network parameters. Figure 4 It shows the effects of image generation after different numbers of iterations (from left to right are 5 types of images: "normal", "cord irregularity", "sidewall foreign object", "belt splice open", "bubble"). It can be found that a good defect reconstruction effect is achieved after 1000 iterations. Therefore, the generator after 1000 adversarial trainings is selected for image generation in this example. Since the number distributions of the 5 types of images are uneven, when expanding the training set, the images of all categories are supplemented to 80.

[0106] After training the tire defect detection model with reference to step IV), the image to be detected is input into the trained tire defect detection model to obtain the final defect detection result. The number of neurons in the last fully connected layer of EfficientNet is set to 5, and the number of training times of EfficientNet is set to 100. After training is completed, it is verified with the real data test set, and the verification accuracy reaches 93.75%. And from Figure 5 the confusion matrix shown, it can be seen that EfficientNet performs relatively evenly on various types of images, proving that the method described in this patent has good performance.

[0107] Example 2

[0108] A computer device, comprising: at least one processor and a memory communicatively connected to at least one processor;

[0109] Wherein, the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the tire defect detection method based on anomaly detection and conditional generative adversarial network as described in Example 1.

[0110] Example 3

[0111] A computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, it implements the tire defect detection method based on anomaly detection and conditional generative adversarial network as described in Example 1.

[0112] Those skilled in the art should understand that those skilled in the art can implement variations in combination with the prior art and the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention and will not be elaborated here.

[0113] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and the devices and structures not described in detail should be understood to be implemented in a common manner in the art; any person skilled in the art can, without departing from the scope of the technical solution of the present invention, make many possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above, or modify it into equivalent embodiments with equivalent changes, which does not affect the essence of the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A tire defect detection method based on anomaly detection and conditional generative adversarial network, characterized in that: Inputting the tire image to be detected into the tire defect detection model, and the tire defect detection model outputs the defect detection result of the tire image to be detected; The training process of the tire defect detection model is a process of continuously adjusting the parameters of the model by taking the image data in the total set of training data as input and the known defect detection results corresponding to the image data as theoretical output; The total set of training data is obtained by expanding the real data set by applying the auxiliary classification generative adversarial network based on feature reconstruction. The generator and discriminator of the auxiliary classification generative adversarial network based on feature reconstruction are embedded with an attention module with abnormal weights. The attention module with abnormal weights is a CBAM attention module combined with an abnormal weight matrix S. The abnormal weight matrix S is obtained by processing the real data set by applying the PaDiM anomaly detection algorithm.

2. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 1, characterized in that: Performing image preprocessing on the tire image to be detected before inputting the tire image to the tire defect detection model; The image data in the real data set is image data after image preprocessing.

3. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 2, characterized in that: The image preprocessing operation is as follows: (1) Perform histogram equalization on the original image; (2) performing edge enhancement on the image processed in step (1); (3) sharpening the image processed in step (2); (4) Standardize the image processed in step (3) and convert it into a form suitable for model input to promote the convergence of model training.

4. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 2, characterized in that: The real data set includes normal image data and defect image data; Take 70% of the data of each category in the real data set as the real data training samples, as the real data training set, and take the other 30% of the data of each category in the real data set as the real data test samples, as the real data test set.

5. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 1, characterized in that: The tire defect detection model is an EfficientNet model.

6. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 4, characterized in that: The training of the tire defect detection model is specifically as follows: Firstly, the tire defect detection model is pre-trained using a real data set. The number of neurons in the last fully connected layer of the pre-trained tire defect detection model is changed to the number of image categories in the specific task. Finally, all parameters are fine-tuned using the total set of training data, and the performance of the tire defect detection model is evaluated using a real data test set. When the performance meets actual requirements or the model is fully converged, the training is stopped.

7. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 1, characterized in that: The acquisition process of the CBAM attention module combined with the abnormal weight matrix S is as follows: 1) In the training phase of the PaDiM anomaly detection algorithm, the embedding vectors of different semantic levels of each image block in the normal image are extracted through the pre-trained ResNet18 network. The l-th layer embedding vector of the image block at position (i, j) can be expressed as f l (i, j), these embedding vectors are concatenated according to the channel dimension to obtain an embedding vector e containing multiple semantic levels ij =concat(f1(i,j),f2(i,j),…,f L (i,j)); 2) For the N normal images in the real data training set, the set of embedded vectors at each image position (i, j) is , assuming From the multivariate Gaussian distribution N(μ ij ,Σ ij ) is generated, where the sample mean μ ij and the sample covariance matrix Σ ij It can be calculated as follows: ， ， Among them, ∈ is a regularization term, I is the identity matrix; 3) In the inference stage of the PaDiM anomaly detection algorithm, for each image block of the defect image, the same pre-trained ResNet18 network is used to extract the embedding vector containing multi-semantic level information, and the Mahalanobis distance between the vector and the multivariate Gaussian distribution at the corresponding position is calculated as the anomaly score of the image block. For the embedding vector at the test image position (i, j), , its anomaly score s ij It can be obtained by the following formula: ； For an image with height and width, its abnormal weight matrix S can be expressed as ; 4) The abnormal weight matrix Combined with the CBAM attention module, the CBAM module includes a channel attention module CAM and a spatial attention module SAM. In CAM, the feature map X is embedded in two different features through average pooling and maximum pooling. These two feature embeddings are used to generate channel weights M through a shared multi-layer perceptron. C , used to weight each channel of the input feature map. In SAM, the feature map X·M after CAM processing C Average pooling and maximum pooling are performed on the channel dimension to obtain two feature embeddings. After concatenating these two feature embeddings, a convolutional layer is used to generate the spatial attention weight M. S , and the input is weighted in the spatial dimension, and the final output of CBAM is (X·M C )·M S , when X is the original image, multiply the output of CBAM by S element by element, which is the output (X·M C )·M S· S, when X is a feature map obtained after the original image is processed by the deep learning network, S needs to be processed exactly the same as above to get the abnormal feature embedded in S', and the output of CBAM is multiplied element by element with S' to get the output (X·M C )·M S ·S'.

8. The tire defect detection method based on anomaly detection and conditional generative adversarial network according to claim 1, characterized in that: The attention module with abnormal weights embedded in the generator and discriminator of the auxiliary classification generative adversarial network based on feature reconstruction is specifically: In the generator of the feature reconstruction-based auxiliary classification generative adversarial network, the encoder has five layers. The first layer is a convolutional layer, and the following four layers are downsampling layers. The attention module with abnormal weights is embedded between the convolutional layer of the encoder part of the generator and the subsequent downsampling layer. The underlying network of the discriminator of the generative adversarial network for auxiliary classification based on feature reconstruction consists of four downsampling layers, and the attention module with abnormal weights is embedded between the first downsampling layer and the second downsampling layer.

9. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the tire defect detection method based on anomaly detection and conditional generative adversarial network as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the tire defect detection method based on anomaly detection and conditional generative adversarial network as described in any one of claims 1 to 8 is implemented.

Images