Tire image defect detection method based on U-Net discriminator

Abstract

The invention provides a tire image detection method based on a network model of a U-Net discriminator, and relates to the technical field of tire detection. In the method, an attention mechanism, namely a coding stage of a generator, is added, and the attention mechanism calculates the importance score of each position in a feature map and uses the scores to weight original features, so that the accuracy of the tire image detection is improved; in the decoding stage of the generator, details of the generated image are gradually refined through an attention mechanism; meanwhile, an improved loss function is designed for the generator, so that the problem of vision inconsistency caused by the fact that only Euclidean distance is used for measuring reconstruction errors is solved, and the robustness of irregular texture defect detection is improved; therefore, the technical problem that the shape and the problem of the defect image are difficult to distinguish from those of the real image in the prior art is relieved, and the technical effect of improving the tire defect detection precision is achieved.

Description

Technical Field

[0001] The present invention relates to the related field of tire detection, and in particular to a tire image defect detection method based on a U-Net discriminator. Background Art

[0002] Radial tires are important components of motor vehicles. Public transportation accidents caused by tire damage on highways account for 55% of the total traffic accident rate. Faced with the high accident rate caused by tire failure, various companies have begun to improve and update tire defect detection equipment. In the tire production process, defect detection is a very important link. Researching advanced defect detection technology can effectively ensure product quality and maintain production stability. Most of the current defect detection equipment is based on laser shearing speckle interferometry technology, ultrasonic detection technology and X-ray scanning technology. Although ultrasonic detection technology can quickly and cheaply detect thicker materials and accurately locate and quantify defects, its application in tire defect detection is difficult due to waveform limitations. X-ray scanning technology has extremely high penetration and can produce optical density differences on tires of different thicknesses and textures. The tire scanning imaging details are highly visible and the defect targets are intuitive. With the development of deep learning technology, image defect detection based on deep learning has achieved significant results. Compared with supervised learning and semi-supervised learning, unsupervised learning does not require image annotation, saving a lot of human resources.

[0003] At present, the anomaly detection method based on generative adversarial network (GAN) has made great breakthroughs. Generative adversarial network is a classic unsupervised learning method, which is currently widely used in many fields such as manufacturing, video surveillance and video monitoring. Schleg et al. first proposed the use of AnoGAN for image defect detection. Subsequently, scholars have proposed model structures such as GANomaly, F-AnoGAN, MAGAD, MAGAN and SRGAN. In 2019, Akcay proposed Skip-GANomaly, which can reduce the number of image features lost by the generator during image reconstruction. However, these emerging generative adversarial networks have the following disadvantages. For example, AnoGAN requires repeated iterations for optimization, resulting in a time-consuming iteration process that significantly increases the model inference time. While GANomaly obtains better detection results, it requires a longer training time for the generator, and the detection accuracy needs to be improved. Skip-GANomaly enhances the detection capability, but its performance shows great differences in different types of data, and the reconstructed abnormal images carry too much defect information, making them unsuitable for actual defect detection tasks. As a result, during tire inspection, the shape and texture of the defect image are difficult to distinguish from the real image. Summary of the invention

[0004] The purpose of the embodiment of the present invention is to propose a tire image defect detection method based on a U-Net discriminator to alleviate the technical problem in the prior art that the shape and texture of the defect image are difficult to distinguish from the real image during the tire detection process.

[0005] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] A first aspect of the present invention provides a tire image defect detection method based on a U-Net discriminator, the method comprising the following steps:

[0007] S10. Introduce encoding and decoding in the discriminator. The encoder module performs image-by-image classification, and the decoder module outputs pixel-by-pixel classification decisions to provide spatially coherent feedback to the generator.

[0008] S20. Based on the Skip-GANomaly algorithm, the attention mechanism is added to the generator, and the skip connection layer is used to transfer the tire image information; the generator is divided into a structure consisting of a downsampling network and an upsampling network;

[0009] In the encoding phase of the generator, the attention mechanism calculates the importance scores of each position in the feature map and uses these scores to weight the original features to emphasize key information and suppress unimportant information;

[0010] In the decoding stage of the generator, the details of the generated image are gradually refined through the attention mechanism;

[0011] S30. Use loss function to improve the robustness of input images and generated images.

[0012] In some modified implementations of the first aspect of the present invention, the attention mechanism is SimAM, and the steps are as follows:

[0013] S21. Define neurons, the expression is as follows:

[0014]

[0015] S22. The expression in step S31 is replaced with a binary label and a regular term is added. The expression is:

[0016]

[0017] The analytical formula is:

[0018]

[0019]

[0020] S23. Calculate the mean and variance of the input features in two dimensions H and W;

[0021]

[0022] The expression of the calculation process is:

[0023]

[0024] In some modified implementations of the first aspect of the present invention, the optimization step of the loss function for the generator is:

[0025] S31. For the model input image and generated image, the expression of SSIM value is:

[0026]

[0027] where μ x and μ y denote the means of x and y, σ x and σ y denote the variance of x and y, σ xy Respectively represent the covariance of x and y, c1 and c2 represent constants;

[0028] S32. Establish a loss function, which is expressed as:

[0029] L SSIM (x,y)=1-SSIM(x,y);

[0030] Loss function L SSIM Used L con Loss and L SSIM The loss joint function is used as the image reconstruction loss, and its expression is:

[0031] L=w adv L adv +w recon L recon +w lat L lat ;

[0032] Among them, w adv 、w recon and w lat They are all weight parameters, which are used to adjust the impact of each loss on the overall optimization objective function.

[0033] In some modified implementations of the first aspect of the present invention, the image reconstruction loss L recon The expression is:

[0034]

[0035] Adversarial loss Ladv The expression is:

[0036]

[0037] Potential loss L lat The expression is:

[0038]

[0039] Among them, P x is the distribution of normal data, x is the P that obeys x Normal data, is the data reconstructed by the generator; D is the discriminator, α is the weight parameter, f(x) and are the bottleneck features obtained from the test sample and the reconstructed image input to the discriminator, respectively.

[0040] The present invention proposes a tire image detection method based on a network model of a U-Net discriminator, namely UDGANomaly. First, encoding and decoding are introduced in the discriminator. The encoder module performs image-by-image classification, and the decoder module outputs pixel-by-pixel classification decisions, providing spatial coherence feedback to the generator to achieve tire defect detection. Secondly, self-attention is introduced in the encoder and decoder of the generator to further focus on the representative information contained in multi-scale features. Finally, an improved generator loss function based on structural similarity is designed to solve visual inconsistencies, thereby improving the robustness of irregular texture detection.

[0041] In this method, from the perspective of accuracy, the present invention adds an attention mechanism, that is, in the encoding stage of the generator, the attention mechanism calculates the importance score of each position in the feature map and uses these scores to weight the original features to achieve the emphasis on key information and the suppression of unimportant information. In the decoding stage of the generator, the details of the generated image are gradually refined through the attention mechanism; at the same time, the present invention is also based on designing an improved loss function for the generator to solve the visual inconsistency problem caused by the reconstruction error using only the Euclidean distance metric, thereby improving the robustness of irregular texture defect detection.

[0042] Therefore, the present invention alleviates the technical problem in the prior art that the shape and problem of the defect image are difficult to distinguish from the real image, and achieves the technical effect of improving the accuracy of tire defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the embodiments are briefly introduced below.

[0044] Figure 1A model structure diagram of a tire image detection method based on a network model of a U-Net discriminator provided by the present invention;

[0045] Figure 2 An image acquisition diagram of a normal tire sample in a tire X-ray image training set in a tire image detection method based on a network model of a U-Net discriminator provided by the present invention;

[0046] Figure 3 An image acquisition diagram of defect samples in a tire X-ray image training set in a tire image detection method based on a network model of a U-Net discriminator provided by the present invention;

[0047] Figure 4 A comparison diagram of ROC curves for tire X-ray defect detection in a tire image detection method based on a network model of a U-Net discriminator provided by the present invention;

[0048] Figure 5 A heat map of the training set in the tire image detection method based on the network model of the U-Net discriminator provided by the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0051] The present invention is further described in detail below by way of examples. Through these descriptions, the features and advantages of the present invention will become more clear and distinct.

[0052] like Figure 1 As shown, this embodiment provides a tire image detection method based on a network model of a U-Net discriminator, the method comprising the following steps:

[0053] S10. Introduce encoding and decoding in the discriminator. The encoder module performs image-by-image classification, and the decoder module outputs pixel-by-pixel classification decisions to provide spatially coherent feedback to the generator.

[0054] S20. Based on the Skip-GANomaly algorithm, the attention mechanism is added to the generator, and the skip connection layer is used to transfer the tire image information; the generator is divided into a structure consisting of a downsampling network and an upsampling network;

[0055] In the encoding phase of the generator, the attention mechanism calculates the importance scores of each position in the feature map and uses these scores to weight the original features to emphasize key information and suppress unimportant information;

[0056] In the decoding stage of the generator, the details of the generated image are gradually refined through the attention mechanism;

[0057] S30. Use loss function to improve the robustness of input images and generated images.

[0058] In this method, encoding and decoding are first introduced in the discriminator. The encoder module performs image-by-image classification, and the decoder module outputs pixel-by-pixel classification decisions to provide spatially coherent feedback to the generator to achieve tire defect detection. Secondly, self-attention is introduced in the encoder and decoder of the generator to further focus on the representative information contained in multi-scale features. Finally, an improved generator loss function based on structural similarity is designed to address visual inconsistencies, thereby improving the robustness of irregular texture detection.

[0059] In this method, from the perspective of accuracy, the present invention adds an attention mechanism, that is, in the encoding stage of the generator, the attention mechanism calculates the importance score of each position in the feature map and uses these scores to weight the original features to achieve the emphasis on key information and the suppression of unimportant information. In the decoding stage of the generator, the details of the generated image are gradually refined through the attention mechanism; at the same time, the present invention is also based on designing an improved loss function for the generator to solve the visual inconsistency problem caused by the reconstruction error using only the Euclidean distance metric, thereby improving the robustness of irregular texture defect detection.

[0060] Therefore, the present invention alleviates the technical problem in the prior art that the shape and problem of the defect image are difficult to distinguish from the real image, and achieves the technical effect of improving the accuracy of tire defect detection.

[0061] To further explain the attention mechanism and improved loss function of this embodiment, the following research methods are described in detail in the specification:

[0062] 1. UDGANomaly

[0063] Skip-GANomaly is an anomaly detection algorithm based on unsupervised learning, which is implemented using a generative adversarial network structure. Skip-GANomaly is an extension of the traditional GAN ​​framework. Skip-GANomaly consists of an autoencoder and a discriminator. The autoencoder part can also be regarded as a generator. There is a skip-connection between the encoder and the decoder. Skip-GANomaly also adds a reconstruction loss term and a reconstruction loss term at the back end of the discriminator to enhance the performance of anomaly detection.

[0064] The Skip-GANomaly discriminator may have limitations in processing subtle structures and details, and cannot better identify some subtle defects. By changing the discriminator architecture to look like the U-Net network architecture, the difference between the generated image and the real image can be evaluated, and whether there is anomaly can be determined. The use of the U-Net architecture enables the discriminator to better capture the details and structural information in the image, thereby improving the ability to detect anomalies. The U-Net architecture allows detailed image feedback to be provided to the generator, while maintaining the global consistency of the synthesized image by providing global image feedback, effectively improving the accuracy of model anomaly detection.

[0065] The UDGANomaly model structure is as follows Figure 1 As shown. An attention mechanism is added to the generator, and the skip connection layer is still used to pass the tire image information. Now, the discriminator has evolved into a structure consisting of the original downsampling network and a brand new upsampling network. The two modules are connected by a bottleneck, as well as a skip connection that copies and connects the feature maps from the encoder and decoder modules. The original D(x) divides the input image x into true and false. The U-Net discriminator classifies on a pixel-by-pixel basis, splits the image x into true and false regions, and classifies the original image of x with the encoder to complete the anomaly detection task.

[0066] 2. SimAM Attention Mechanism

[0067] SimAM (Simplified Self-Attention Module for Visual Tasks) is a simplified self-attention mechanism designed specifically for visual tasks to improve the network model's ability to capture spatial information. Different from previous attention mechanisms such as BAM and CBAM, which combine spatial attention with channel attention in parallel or serially, SimAM's simplest way to find important neurons is to measure the linear separability between neurons. Based on these neuroscience discoveries, the following energy function is defined for each neuron:

[0068]

[0069] Minimizing the above formula is equivalent to enhancing the linear separability between neuron t and other neurons in the same channel through training. For simplicity, SimAM uses binary labels and adds regularization terms. The final energy function is as follows:

[0070]

[0071]

[0072] The above analytical formula is as follows:

[0073]

[0074]

[0075] Since all neurons on each channel follow the same distribution, we can first calculate the mean and variance of the input features in the two dimensions H and W to avoid repeated calculations, as shown in the formula:

[0076]

[0077] The calculation process of the above formula is expressed as the following formula:

[0078]

[0079] This paper adds SimAM to the encoding and decoding parts of the generator. During the encoding stage of the generator, SimAM can be applied to the feature map to help the model better understand the spatial relationship and important areas of the input data. Specifically, SimAM can emphasize key information and suppress unimportant information by calculating the importance score of each position in the feature map and using these scores to weight the original features. This helps the encoder capture richer and more discriminative feature representations, providing a better foundation for the decoder to generate high-quality images.

[0080] In the decoding stage, SimAM can help the generator reconstruct images more finely. By applying SimAM on different layers of the decoder, the generator can gradually refine the details of the generated image while maintaining attention to the global structure. The introduction of this self-attention mechanism enables the generator to better balance local details and global consistency when generating images, thereby improving the quality of the final generated image.

[0081] 3. Loss Function

[0082] In order to consider the brightness, contrast and structural information correlation of tire training images, this paper designs an improved loss function for the generator based on structural similarity (SSIM) to solve the visual inconsistency problem caused by reconstruction error using only Euclidean distance measurement and improve the robustness of irregular texture defect detection. For the model input image and generated image, the SSIM value is defined as follows:

[0083]

[0084] where μ x and μ y denote the means of x and y, σ x and σ y denote the variance of x and y, σ xy They represent the covariance of x and y respectively, and c1 and c2 represent constants.

[0085] The design based on the loss function is as follows:

[0086] L SSIM (x,y)=1-SSIM(x,y);

[0087] Loss function L SSIM The generator is optimized from three perspectives: brightness, contrast, and structure. The generator is trained to focus on the brightness changes and color deviations of the sample images, thereby retaining high-frequency information such as the edges and details of the tire images, which is suitable for regular tire texture samples. con Loss and L SSIM The loss joint function is used as the image reconstruction loss, and its expression is as follows:

[0088] L recon =αL con +(1-α)L SSIM

[0089] where α is the equilibrium L con and L SSIM Weighting factor for weight.

[0090] The final optimization function expression is as follows:

[0091] L=w adv L adv +w recon L recon +w lat L lat

[0092] where w adv 、w recon and w lat They are all weight parameters, which are used to adjust the impact of each loss on the overall optimization objective function. Image reconstruction loss L recon, against the loss L adv and potential loss L lat , the three sub-loss functions are as follows:

[0093] Image reconstruction loss L recon The expression is:

[0094]

[0095] Adversarial loss L adv The expression is:

[0096]

[0097] Potential loss L lat The expression is:

[0098]

[0099] Where P x is the distribution of normal data, x is the P that obeys x Normal data, is the data reconstructed by the generator. D is the discriminator, α is the weight parameter, f(x) and are the bottleneck features obtained from the test sample and the reconstructed image input to the discriminator, respectively.

[0100] The above is a discussion of the method of this embodiment. In order to verify the technical effect of the tire image detection method based on the network model of the U-Net discriminator provided by this embodiment, the following experimental data is provided as proof:

[0101] 1. Preparation of experimental data set

[0102] like Figure 2 and Figure 3 As shown in Figure 1, the training data set consists of 12,000 defect-free tire X-ray images. Specifically, radial tire X-ray images are selected, and after comparing X-ray images of different positions of the tire, it is finally determined to select X-ray images of the sidewall part with relatively simple structure and background.

[0103] The test data set consists of defect-free images and defective images. There are 1908 defect-free images and 1908 defective images, among which defective images include impurity images, bubble images, tire thinning images, and cord bending images. Impurity images may have irregular shapes and sizes, making them difficult to distinguish from normal images. Bubbles may appear in different sizes and shapes, and have a liquid-like appearance, which can be easily mistaken for normal parts. Tire thinning may appear irregularly in the image, blending in with the surrounding environment, making it difficult to be accurately detected. Cord bending may have different shapes and degrees of curvature, and is similar to the surrounding texture, so it is also difficult to be accurately detected. Specific types such as Figure 3 shown.

[0104] 2. Experimental related parameter settings

[0105] 2.1 Parameter settings

[0106] Adam; lr = 0.0001; b1 = 0.5; b2 = 0.999; latent variable dimension Z = 100; batchsize = 30;

[0107] epoch=100 / 50.

[0108] 2.2 Experimental platform configuration

[0109] The hardware configuration of the computer in this experiment is Intelr-i5, Nvidia GTX1050 GPU, and 12G memory. The programming environment uses the Windows operating system, Python 3.10, and Pytorch 1.7.0 deep neural network development tools.

[0110] 3. Experimental results analysis

[0111] This experiment used the receiver operating characteristic (ROC) curve as a method to evaluate model performance. . The horizontal axis on the ROC curve represents the false positive rate, that is, the proportion of data that are actually all abnormal data that are mistakenly judged as normal, and the vertical axis represents the true positive rate, that is, the proportion of data that are actually all normal data that are correctly judged as normal. The closer the ROC curve is to the upper left corner of the coordinate axis, the better the detection effect, and vice versa. The area enclosed by the ROC curve and the horizontal axis represents the AUC value. The larger this value is, the better the detection effect is, and vice versa. The verification of experimental results based solely on the ROC curve lacks uniformity. Using AUC can better evaluate the results of the experiment. Taking the tire X-ray image dataset as an example, the AUC comparison between the original Skip-GANomaly and the improved model UDGANomaly is as follows. Figure 4 shown.

[0112] 3.1 Comparative Experiment

[0113] In order to further verify the superiority of the UDGANomaly model over other models, we selected the same tire dataset to train and test the UDGANomaly, F-AnoGAN, AnoGAN, Skip-GANomaly and other models. By comparing and analyzing the experimental results, we can more clearly understand the performance differences of each model in the anomaly detection task. The specific results are shown in Table 1.

[0114] Table 1 Experimental data

[0115]

[0116] It can be seen that the improved model UDGANomaly has improved the AUC of the same tire data set. Compared with the original Skip-GANomaly model, the average AUC has increased by 0.234. The rest are compared with the other three anomaly detection

[0117] The average AUC of the test models GANormaly, AnoGAN, and f-AnoGAN increased by 0.121, 0.116, and 0.158, respectively. The experiments proved that this method is effective.

[0118] After the model training is completed, we evaluate the performance of each model by comparing the heatmaps. Since the model is only exposed to the features of normal images, when an abnormal image is input into the generator, the generated image will lack abnormal features. Therefore, we can generate an abnormal heatmap by comparing the difference between the original image and the reconstructed image, so as to accurately locate the abnormal area in the image. Figure 5 In , we show some abnormal images, their corresponding reconstructed images, and the generated heatmaps. It can be observed that UDGANomaly can accurately identify the location of abnormal features. Although GANomaly performs best in reconstructing images, it still has a certain gap compared with our proposed UDGANomaly.

[0119] 3.2 Ablation test

[0120] In order to verify the effectiveness of changing the U-net discriminator, adding the attention mechanism (SimAM) and citing the SSIM loss function in Skip-GANomaly proposed in this paper, an ablation experiment was conducted. Under the same experimental conditions, the impact of different modules on the performance of the model in this paper was evaluated. The final results are shown in Table 2.

[0121] Table 2 Experimental data

[0122]

[0123] Table 2 shows that the average AUC is increased by 0.081 when the discriminator is replaced with the U-Net architecture, and the average AUC is increased by 0.119 when the SimAM attention mechanism is added to the original network model. The average AUC is increased by 0.128 when the SSIM loss function is used on the basis of adding attention. The specific results are shown in Table 2.

[0124] In summary, the UDGANomaly model proposed in this paper changes the discriminator to the U-net architecture based on the original Skip-GANomaly model, which effectively solves the problem that the shape and texture of defective images are difficult to distinguish from real images, and the image identification of abnormal features is unstable when the discriminator performs image identification. Through relevant experimental verification, the improved model UDGANomaly can improve the accuracy of anomaly detection.

[0125] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A tire image defect detection method based on U-Net discriminator, characterized in that: The method comprises the following steps: S10. Introduce encoding and decoding in the discriminator. The encoder module performs image-by-image classification, and the decoder module outputs pixel-by-pixel classification decisions to provide spatially coherent feedback to the generator. S20. Based on the Skip-GANomaly algorithm, the attention mechanism is added to the generator, and the skip connection layer is used to transfer the tire image information; the generator is divided into a structure consisting of a downsampling network and an upsampling network; In the encoding phase of the generator, the attention mechanism calculates the importance scores of each position in the feature map and uses these scores to weight the original features to emphasize key information and suppress unimportant information; In the decoding stage of the generator, the details of the generated image are gradually refined through the attention mechanism; S30. Use loss function to improve the robustness of input images and generated images.

2. The tire image defect detection method based on U-Net discriminator according to claim 1, characterized in that: In step S20, the attention mechanism is SimAM, and its steps are as follows: S21. Define neurons, the expression is as follows: in t and x i They refer to the target neuron and other neurons of the input feature X, i refers to the index in the spatial dimension, M = H × W refers to the number of all neurons in a certain channel, and w t and b t Refers to the "weight" and "bias" of a neuron when it is transformed, and introduces binary labels instead of y t and o ; S22. The expression in step S31 is replaced with a binary label and a regular term is added. The expression is: The analytical formula is: S23. Calculate the mean and variance of the input features in two dimensions H and W; The expression of the calculation process is: where μ x and μ y denote the means of x and y, σ x and σ y denote the variance of x and y, σ xy They represent the covariance of x and y respectively, and c1 and c2 represent constants.

3. The tire image defect detection method based on U-Net discriminator according to claim 1, characterized in that: The optimization steps of the loss function for the generator are: S31. For the model input image and generated image, the expression of SSIM value is: where μ x and μ y denote the means of x and y, σ x and σ y denote the variance of x and y, σ xy Respectively represent the covariance of x and y, c1 and c2 represent constants; S32. Establish a loss function, which is expressed as: L SSIM (x,y)=1-SSIM(x,y); Loss function L SSIM Used L con Loss and L SSIM The loss joint function is used as the image reconstruction loss, and its expression is: L=w adv L adv +w recon L recon +w lat L lat ; Among them, w adv 、w recon and w lat They are all weight parameters, which are used to adjust the impact of each loss on the overall optimization objective function.

4. The tire image defect detection method based on U-Net discriminator according to claim 3 is characterized in that: Image reconstruction loss L recon The expression is: Adversarial loss L adv The expression is: Potential loss L lat The expression is: Among them, P x is the distribution of normal data, x is the P that obeys x Normal data, is the data reconstructed by the generator; D is the discriminator, α is the weight parameter, f(x) and are the bottleneck features obtained from the test sample and the reconstructed image input to the discriminator, respectively.

Images