GANOmaly tire anomaly detection method based on double auto-encoders and GAM (Gamma Amplitude Modulation)
By introducing the GANomaly model of dual autoencoder and GAM channel attention mechanism in tire defect detection, the problems of low detection accuracy of X-ray sensors and unstable reconstruction image quality are solved, and more efficient tire internal defect detection is achieved.
Patent Information
- Application Number
- CN202411435628.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-01-14
AI Technical Summary
In the existing tire defect detection technology, the accuracy of detection through X-ray sensors is not high and the quality of the reconstruction image is unstable, making it difficult to effectively detect internal tire defects.
Using the GANomaly model based on dual autoencoder and GAM, the original image is mapped to low-dimensional vectors and reconstructed through the first subnet. The second subnet compresses and reconstructs the image. The third subnet uses the discriminator to detect abnormalities and introduces the attention of the GAM channel. Force mechanism to improve the weight adjustment and noise immunity of the autoencoder.
It significantly improves the accuracy of tire defect detection and reconstructed image quality, and improves the robustness and accuracy of abnormal detection.
Smart Images
Figure CN119313640A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the related field of tire detection, and in particular to a GANomaly tire anomaly detection method based on dual autoencoders and GAM. Background Art
[0002] Industrial product defect detection is an important study on the quality of industrial products. It uses visual equipment to obtain images and determine whether there are defects in the images. As of the end of November 2022, the number of cars in China has reached 318 million, and the performance of tires is directly related to people's driving safety. Due to the complex internal structure of tires and cumbersome processes, various defects such as bubbles and tire thinning may occur during the production process. If these defects are not discovered in time, the flow of problematic tires into the market may cause serious safety accidents during driving. Therefore, it is necessary to conduct necessary production defect detection on tires before they leave the factory.
[0003] In the prior art, tire defect detection equipment is usually used to perform production defect detection at the tire production site. The detection equipment extracts tire information through an X-ray sensor and generates an X-ray image to inspect the internal structure of the tire. Defect problems can be discovered without destroying the tire structure, thus avoiding impact and damage to the tire. However, the accuracy of tire detection using X-ray sensors is not high and is not stable enough; and there are quality problems in the reconstructed image. Summary of the invention
[0004] The purpose of the embodiment of the present invention is to propose a GANomaly tire anomaly detection method based on dual autoencoders and GAM to alleviate the problems in the prior art of low accuracy and instability in tire detection by X-ray sensors; and low quality of reconstructed images.
[0005] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:
[0006] A GANomaly tire anomaly detection method based on dual autoencoders and GAM, the GANomaly model comprising a first subnetwork, a second subnetwork and a third subnetwork; the detection method comprises the following steps: S10. the encoder in the first subnetwork maps the original image to a low-dimensional vector, forms a latent vector of the original image, maps the n-dimensional vector (bottleneck1) to a 3-channel image, and reconstructs the output image through a decoder, and the first subnetwork is provided with a dual autoencoder parallel structure;
[0007] S20. In the second sub-network, compressing the output image reconstructed in step S10 into a low-dimensional vector; forming a latent vector of the reconstructed image;
[0008] S30. In the third sub-network, anomaly detection is performed by the discriminator according to the difference between the latent vector of the original image and the latent vector of the reconstructed image.
[0009] Furthermore, in the first sub-network, an autoencoder is added on the basis of the GANomaly model to form a network structure with two autoencoders in parallel to model high-dimensional data that captures the distribution of the original real data. The GAM channel attention mechanism is introduced in some convolutional layers of the encoder and decoder of the generator. The channel attention sub-module of GAM aims to combine the feature map with the weights of the model, calculate the feature map of the final convolutional layer and the weights of the global average pooling, and then superimpose it with the input image to visualize the area that the model pays attention to when making decisions, so that the autoencoder in the generator can adaptively adjust the weight parameters of the autoencoder according to the content of the original image.
[0010] Furthermore, in the first sub-network, a loss function is used to reduce the abnormal loss of the output image: adversarial loss:
[0011] L adv =||f(x)-f(G(x))||2 (1)
[0012] Reconstruction loss:
[0013] L con =||x-x'||1 (2)
[0014] Encoder loss:
[0015] L enc =||Z-Z'||2 (3)
[0016] Generator total loss:
[0017] L=w adv L adv +w enc L enc +w con L con (4)
[0018] Among them, L adv Represents the error of feature matching: Calculate the L2 distance between the input image and the reconstructed image feature representation, f(x) represents the input image, and f(G(x)) represents the reconstructed image feature. L con Represents the reconstruction error loss: continuously reducing the difference between the reconstructed image and the original image, x represents the original image, and x' represents the reconstructed image. L enc represents the difference between the low-dimensional vector of the input image and the latent vector of the reconstructed image, Z represents the low-dimensional vector of the input image, and Z' represents the latent vector of the reconstructed image. L represents the total loss of the generator, w adv、w enc 、w con is the weighting parameter.
[0019] Furthermore, in the third subnetwork, a discriminator loss function is introduced, and the discriminator loss consists of two parts: the loss of real samples and the loss of generated samples; the loss of real samples: the discriminator needs to classify real samples as real; the loss of generated samples: the loss of the discriminator that classifies the generated samples as fake can be obtained by adding the loss of real samples and the loss of generated samples.
[0020]
[0021] Among them, the optimization adversarial loss Minimize the generator loss and maximize the discriminator loss. E(*) represents the expected value of the distribution function, P data(x) represents the distribution of real samples, P Z(Z) represents a low-dimensional noise distribution, log(1-logD(G(Z)) represents minimizing the binary cross entropy function, D(x) represents the original image judged by the discriminator, and G(Z) is the false sample generated by the generator.
[0022] Furthermore, the channel attention submodule of the GAM first performs convolution and pooling operations on the input image to obtain a high-level feature representation; then obtains the weight of each channel through a global average pooling operation, which represents the importance of each channel for identifying and locating objects.
[0023] The present invention proposes a GANomaly tire anomaly detection method based on dual autoencoders and GAM. Firstly, in order to solve the problem of insufficient reconstruction ability of the autoencoder, an autoencoder is added on the basis of the GANomaly model to form a network structure with two autoencoders in parallel, thereby effectively strengthening the information representation and the reconstruction constraints of the information representation under noise interference, and improving the quality of the reconstructed image; and the GANomaly model is correspondingly formed into three sub-networks. In the first sub-network, the GAM channel attention mechanism is introduced in some convolutional layers of the encoder and decoder of the generator, so that the autoencoder in the generator can adaptively adjust the weight parameters of the autoencoder according to the content of the original image, thereby paying more attention to the extraction of detail features; in the second sub-network, the image reconstructed by the first sub-network is compressed into a low-dimensional vector, and the additional abstraction can greatly improve its ability to resist noise interference, thereby obtaining a more robust anomaly detection model; in the third sub-network, the discriminator network needs to simultaneously learn the two mapping relationships of "original image->reconstructed image" and "original image potential vector->reconstructed image potential vector", and the difference between the potential vector of the original image and the potential vector of the reconstructed image is used for inference, thereby effectively improving the overall accuracy of anomaly detection.
[0024] Therefore, the present invention alleviates the technical problem in the prior art that the shape and problem of the defect image are difficult to distinguish from the real image, and achieves the technical effect of improving the accuracy of tire defect detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required to be used in the embodiments are briefly introduced below.
[0026] Figure 1 This is the DA-GANomaly network structure diagram;
[0027] Figure 2 is the channel attention module;
[0028] Figure 3 is a normal tire image in the test dataset;
[0029] Figure 4 Defective tire images in the test dataset;
[0030] Figure 5 The ROC curve of the improved GANomaly model after testing on the tire defect dataset;
[0031] Figure 6 This is the ROC curve of the improved GANomaly model tested on the tire defect dataset;
[0032] Figure 7 To reconstruct the loss function curve graph;
[0033] Figure 8 The test result diagrams are of some defective images, reconstructed images and residual images. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0036] The present invention is further described in detail below by way of examples. Through these descriptions, the features and advantages of the present invention will become more clear and distinct.
[0037] A GANomaly tire anomaly detection method based on dual autoencoders and GAM, the GANomaly model includes a first sub-network, a second sub-network and a third sub-network; the detection method includes the following steps:
[0038] S10. The encoder in the first sub-network maps the original image to a low-dimensional vector to form a latent vector of the original image, maps the n-dimensional vector (bottleneck1) to a 3-channel image, and reconstructs the output image through the decoder, and the first sub-network is provided with a dual autoencoder parallel structure;
[0039] S20. In the second sub-network, compressing the output image reconstructed in step S10 into a low-dimensional vector; forming a latent vector of the reconstructed image;
[0040] S30. In the third sub-network, anomaly detection is performed by the discriminator according to the difference between the latent vector of the original image and the latent vector of the reconstructed image.
[0041] This embodiment firstly addresses the problem of insufficient reconstruction capability of the autoencoder by adding an autoencoder on the basis of the GANomaly model to form a network structure with two autoencoders in parallel, thereby effectively strengthening the information representation and the reconstruction constraints of the information representation under noise interference, and improving the quality of the reconstructed image; then, the GAM channel attention mechanism is introduced in some convolutional layers of the encoder and decoder of the generator, so that the autoencoder in the generator can adaptively adjust the weight parameters of the autoencoder according to the content of the original image, and thus pay more attention to the extraction of detail features; finally, relevant experiments are conducted on a self-made tire X-ray image dataset. The experimental results show that the network can effectively improve the accuracy of anomaly detection, proving the effectiveness and superiority of the model.
[0042] This embodiment uses GANomaly and DA-GANomaly as network models, which are described as follows:
[0043] (1) GANomaly
[0044] GANomaly is composed of three sub-networks, and the generation network adopts the autoencoder-encoder sub-network structure. Among them, the design of the autoencoder structure refers to DCGAN, which can model high-dimensional data that captures the distribution of original real data. The encoder in the autoencoder can map the original image to a lower-dimensional vector (bottleneck1), and the decoder reconstructs the low-dimensional vector into the output image.
[0045] The second sub-network decoder (encoder) compresses the image reconstructed by the first sub-network into a low-dimensional vector (bottleneck2). It abandons the common method of inferring anomalies by comparing the differences between the original image and the reconstructed image, which is commonly used by most tire anomaly detection methods based on autoencoders. Instead, it infers anomalies by comparing the differences between the original image and the reconstructed image in a higher level of abstract space. This additional level of abstraction can greatly improve its ability to resist noise interference and obtain a more robust anomaly detection model.
[0046] The third sub-network is a discriminant network (D-Net). The discriminator network needs to learn two mapping relationships at the same time: "original image -> reconstructed image" and "original image latent vector -> reconstructed image latent vector". Its function is to distinguish the original image from the reconstructed image, that is, the original image is judged as true and the reconstructed image is judged as false. The structure is the same as the decoder of the sub-network. Ultimately, the judgment of abnormality does not depend on the difference between the original image and the reconstructed image, but is inferred by the difference between the latent vector of the original image and the latent vector of the reconstructed image.
[0047] (2)DA-GANomaly
[0048] This paper introduces dual autoencoders based on the existing GANomaly model. The output of the dual autoencoders can well preserve the features of the original image. The encoder can better retain as much relevant information as possible in the latent space through learning and discard irrelevant parts. Relevant experiments have proved that the introduction of dual autoencoders can reduce reconstruction errors, reduce anomaly threshold scores, and effectively improve the accuracy of model anomaly detection.
[0049] The model structure of Dual AutoEncoder GANomaly (DA-GANomaly) is as follows Figure 1 As shown in the figure. GANomaly originally had three sub-networks. In the first sub-network, dual autoencoders were implemented in parallel. The convolutional layer was the same as before, and the rest of the structure was the same as the original GANomaly model. The dual autoencoders minimized the reconstruction error and learned the important features of the input data through the encoding process, so that the implicit information of the input data can be better represented, allowing the decoder to reconstruct the input data as accurately as possible.
[0050] The goal of DA-GANomaly is to minimize the adversarial loss and reconstruction loss of the generator, maximize the adversarial loss of the discriminator, optimize the encoder, and ultimately enable the generator to generate realistic data samples by continuously iteratively training the generator and discriminator.
[0051] In addition, the loss function used in this embodiment is as follows:
[0052] (1) Fighting against losses
[0053] The role of adversarial loss is to make the two images as close as possible, that is, to make the pictures generated by the generator more realistic.
[0054] L adv =||f(x)-f(G(x))||2 (5)
[0055] (2) Reconstruction loss
[0056] The reconstruction loss is measured by calculating the difference between the reconstructed image and the real image. By minimizing the reconstruction loss, the reconstructed image is made more similar to the real image, thereby improving the quality of the reconstructed image.
[0057] L con =||x-x'||1 (6)
[0058] (3) Encoder loss
[0059] The encoder loss measures the degree of distortion introduced when recovering the original data from the latent space, and it is achieved by calculating the difference between the reconstructed data and the original input data.
[0060] L enc =||Z-Z'||2 (7)
[0061] (4) Generator total loss
[0062] The total loss of the generator is the weighted sum of the above three losses. The meaning of this loss function is that we hope that the samples generated by the generator are as realistic as possible, that is, enough to deceive the discriminator.
[0063] L=w adv L adv +w enc L enc +w con L con (8)
[0064] (5) Discriminator loss function
[0065] The discriminator loss usually consists of two parts: the loss of real samples and the loss of generated samples. Loss of real samples: The discriminator needs to classify real samples as real. Loss of generated samples: The discriminator also needs to classify generated samples as fake. The discriminator loss can be obtained by adding the loss of real samples and the loss of generated samples.
[0066]
[0067] In addition, the generator has been improved as follows:
[0068] Global Attention Mechanism (GAM) improves the performance of deep neural networks by reducing information diffusion across spatial channel dimensions and amplifying "global" cross-dimensional interactions. The channel attention submodule of GAM aims to combine feature maps with model weights, calculate the class activation map by combining the feature map of the final convolutional layer with the weights of the global average pooling, and then overlay it with the input image to visualize the area that the model pays attention to when making decisions. Figure 2 As shown in the figure, it works as follows: first, convolution and pooling operations are performed on the input image to obtain a high-level feature representation. Then, the weight of each channel is obtained through a global average pooling operation, which represents the importance of each channel for identifying and locating objects.
[0069] The introduction of the Channel Attention Module mechanism can learn the correlation between multiple input channels and strengthen related channels. This paper adds it to the convolutional network of the generator to improve the model's ability to process input data. It can help the model automatically learn which input channels are more important for specific tasks and adjust the weight of each channel in the network, thereby improving the network's ability to represent input data. By introducing the channel attention mechanism, the model better learns the interactions and dependencies between input data, improving the performance and generalization ability of the model. GAM and DA-GANomaly form a new model DAG-GANomaly (Dual AutoEncoderGAM GANomaly, DAG-GANomaly).
[0070] To further illustrate the technical effect of this embodiment, the experimental data are as follows:
[0071] (1) Dataset
[0072] The dataset used in this paper is a tire dataset made in the laboratory. To verify the method proposed in this paper, 9740 normal tire images are used as training sets, and the image size is 512×512px. Figure 3 shown.
[0073] The test data set consists of three parts: 500 normal tire X-ray images, 499 tire X-ray images with bubbles, 455 tire X-ray images with thin tires, and 441 tire X-ray images with cord bending, totaling 1,895 images. The image size is 512×512px, such as Figure 4 shown.
[0074] (2) Experimental parameters
[0075] The hardware configuration of the experimental computer in this paper is: operating system Windows 10, GPU selected NVIDIA GeForceGTX 1650 graphics card, 16G memory, CPU is AMD Ryzen 5 5600H, main frequency is 3.30GHz. The programming software environment is: programming language is Python (3.10.9), deep learning framework Pytorch (1.12.0), required dependencies include: matplotlib, argparse, Numpy, os, etc. lr = 0.0001; b1 = 0.5; b2 = 0.999; hidden variable dimension Z = 100; batchsize = 132; epoch = 50.
[0076] (3) Experimental results and analysis
[0077] The model indicator used in this article is AUC (Area Under Curve), which is the area under the ROC curve. The ROC (Receiver Operating Characteristic) evaluation indicator is a common indicator for evaluating the performance of classification models. It uses the true positive rate (True Positive Rate, also known as the recall rate) and the false positive rate (False Positive Rate) to measure the performance of the model. The ROC curve is a curve graph with the true positive rate as the vertical axis and the false positive rate as the horizontal axis. The closer the ROC curve is to the upper left corner, the better the performance of the model. The larger the area under the ROC curve, that is, the AUC value, the better the performance of the model.
[0078] Depend on Figure 5 It can be seen that the AUC value of the ROC curve of the GANomaly model before improvement after testing the tire defect dataset is 0.83.
[0079] Depend on Figure 6 It can be seen that after the same training, the AUC value of the overall improved tire defect data set test ROC curve is increased to 0.91. The comparison of experimental results shows that the improved network model has significantly improved the abnormal monitoring effect of tire images.
[0080] Depend on Figure 7 From the reconstruction loss function curve, we can see that the convergence value of the reconstruction error loss of GANomaly is 0.02534, and the convergence value of the improved model DAG-GANomaly is 0.02421. The reconstruction error loss has decreased by 0.00093, which enhances the reconstruction ability of the model and effectively improves the accuracy of model anomaly detection.
[0081] Figure 8 Some defect images, reconstructed images, and residual images are shown. Figure 8It can be seen that the model obtains a reconstructed image that is almost indistinguishable from the normal image. Since the model only learns the normal image features, when the defective image passes through the generator during the test phase, the defective features of the image will be discarded. The residual image corresponding to the defective image is obtained by calculating the difference between the original image and the reconstructed image.
[0082] In order to verify the superiority of the anomaly detection algorithm DAG-GANomaly based on dual autoencoders and GAM proposed in this paper, the experimental accuracy results obtained by selecting the same data set and other comparison methods are shown in Table 1. The experimental results use AUC as the experimental evaluation indicator, and the accuracy is used as an auxiliary.
[0083] Table 1 Comparison of experimental results
[0084]
[0085] The experimental results show that under the same experimental conditions, the AUC value of the DAG-GANomaly proposed in this paper is 0.913 and the accuracy is 0.896, which is 0.078 higher than the AUC value of the original GANomaly model and 0.098 higher than the AUC value of the original GANomaly model. The AUC value of the AnoGAN model is 0.156 higher, the AUC value of the f-AnoGAN model is 0.187 higher, and the AUC value of the skip-GANomaly model is 0.308 higher. This shows that the new DAG-GANomaly model effectively improves the accuracy of defect detection. When detecting tire image data, it can not only extract the implicit information of the input data well, but also accurately capture the structural features, which has a strong reference significance for the abnormal detection of tire images.
[0086] (3) Ablation test study
[0087] In order to verify the effectiveness of introducing dual autoencoders and GAM channel attention mechanism in the GANomaly model proposed in this paper, an ablation experiment was conducted to evaluate the impact of different modules on model performance under the same experimental conditions. The ablation experiment selected GANomaly as the benchmark model and conducted experiments on the tire image dataset. The relevant experimental parameters are consistent with the above, and the experimental results still use AUC value and accuracy as evaluation indicators. The experimental results are shown in Table 2.
[0088] Table 2 Data sheet for ablation experiments
[0089]
[0090] The above experimental data show that the abnormality detection of tire X-ray images is relatively complex. The introduction of dual autoencoders improves the reconstruction ability of the encoder, and the detection accuracy increases by 1.7% and the accuracy rate increases by 2.8%. The introduction of the CAM channel attention mechanism increases the detection accuracy by 4.5%, but the accuracy rate does not increase. Based on the dual autoencoders, the introduction of the CAM channel attention mechanism increases the detection accuracy by 7.8% and the accuracy rate by 9.8%. It proves that the two are complementary and effective for model improvement.
[0091] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A GANomaly tire anomaly detection method based on dual autoencoders and GAM, characterized in that: The GANomaly model includes a first sub-network, a second sub-network and a third sub-network; and the detection method thereof includes the following steps: S10. The encoder in the first sub-network maps the original image to a low-dimensional vector to form a latent vector of the original image, maps the n-dimensional vector bottleneck1 to a 3-channel image, and reconstructs the output image through the decoder, and the first sub-network is provided with a dual autoencoder parallel structure; S20. In the second sub-network, compressing the output image reconstructed in step S10 into a low-dimensional vector; forming a latent vector of the reconstructed image; S30. In the third sub-network, anomaly detection is performed by the discriminator according to the difference between the latent vector of the original image and the latent vector of the reconstructed image.
2. The GANomaly tire anomaly detection method based on dual autoencoders and GAM according to claim 1, characterized in that: In the first subnetwork, an autoencoder is added on the basis of the GANomaly model to form a network structure with two autoencoders in parallel, which models the high-dimensional data that captures the distribution of the original real data. The GAM channel attention mechanism is introduced in some convolutional layers of the encoder and decoder of the generator. The channel attention submodule of GAM aims to combine the feature map with the weight of the model, calculate the feature map of the final convolutional layer and the weight of the global average pooling, and then superimpose it with the input image to visualize the area that the model pays attention to when making decisions, so that the autoencoder in the generator can adaptively adjust the weight parameters of the autoencoder according to the content of the original image.
3. The GANomaly tire anomaly detection method based on dual autoencoders and GAM according to claim 1, characterized in that: In the first subnetwork, a loss function is used to reduce the abnormal loss of the output image: Fighting Loss: L adv =||f(x)-f(G(x))||2 (1) Reconstruction loss: L con =||x-x'||1 (2) Encoder loss: L enc =||Z-Z'||2 (3) Generator total loss: L=w adv L adv +w enc L enc +w con L con (4) Among them, L adv Represents the error of feature matching: Calculate the L2 distance between the input image and the reconstructed image feature representation, f(x) represents the input image, f(G(x)) represents the reconstructed image feature; L con Represents the reconstruction error loss: continuously reduce the difference between the reconstructed image and the original image, x represents the original image, x' represents the reconstructed image; L enc represents the difference between the low-dimensional vector of the input image and the latent vector of the reconstructed image, Z represents the low-dimensional vector of the input image, and Z' represents the latent vector of the reconstructed image; L represents the total loss of the generator, w adv 、w enc 、w con is the weighting parameter.
4. The GANomaly tire anomaly detection method based on dual autoencoders and GAM according to claim 1, characterized in that: In the third subnetwork, the discriminator loss function is introduced. The discriminator loss consists of two parts: the loss of real samples and the loss of generated samples; the loss of real samples: the discriminator needs to classify real samples as real; the loss of generated samples: the loss of the discriminator that classifies the generated samples as fake can be obtained by adding the loss of real samples and the loss of generated samples: Among them, the optimization adversarial loss Minimize the generator loss and maximize the discriminator loss; E(*) represents the expected value of the distribution function, P data(x) represents the distribution of real samples, P Z(Z) represents a low-dimensional noise distribution, log(1-logD(G(Z)) represents minimizing the binary cross entropy function, D(x) represents the original image judged by the discriminator, and G(Z) is the false sample generated by the generator.
5. The GANomaly tire anomaly detection method based on dual autoencoders and GAM according to claim 2, characterized in that: The channel attention submodule of the GAM first performs convolution and pooling operations on the input image to obtain a high-level feature representation; then a global average pooling operation is performed to obtain the weight of each channel, which represents the importance of each channel for identifying and locating objects.