Industrial image defect intelligent detection system based on multi-domain adversarial visual transformer

Abstract

The application discloses an industrial image defect intelligent detection system based on a multi-domain adversarial visual Transformer, which comprises a data preprocessing module, an attention-based spatial conversion module, an adversarial disturbance module, a VIT module and a classifier module. The attention-based coordinate space conversion module comprises a multi-spectrum channel attention unit and a coordination attention unit. MaviT is an attention-based network model suitable for the field of computer vision, which can well perform image classification tasks, and can quickly and stably evaluate the surface morphology characteristics of industrial materials such as composite fibers under the condition that the number of industrial images with defects is insufficient. The model uses the attention mechanism, which is a technology used in deep learning models to enhance the attention and use of important information. It can be regarded as a way to simulate human attention mechanism, so that the model can focus on specific parts or features when processing input data.

Description

Technical Field

[0001] This invention relates to the field of target detection and classification algorithms in industrial machine vision, specifically an intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT). Background Technology

[0002] High-end equipment manufacturing is a pillar industry of my country's economy. Shanxi Province, as an important industrial equipment production base, needs to actively develop the intelligent transformation and upgrading of its high-end equipment manufacturing industry. During the equipment manufacturing process, some industrial defects are inevitable, affecting product use. Due to the extreme industrial production environment, traditional manual inspection methods are often insufficient to meet inspection needs. Therefore, applying non-contact intelligent measurement technology based on computer vision for industrial image defect detection is of great significance. In the industrial production process, image acquisition using high-definition cameras, electron microscopes, CT scanners, and other equipment, combined with computer image processing and artificial intelligence technology, can automatically and accurately assess whether products have defects, thus assisting the industrial production process.

[0003] In recent years, unsupervised anomaly detection algorithms have been increasingly widely used in industrial inspection tasks. Unsupervised means that the training phase only includes normal images, without using any defective samples. This avoids the difficulty of collecting abnormal or defective samples, because in industrial scenarios, the number of normal images without defects far exceeds the number of abnormal samples. Secondly, unsupervised methods eliminate the labeling cost of training samples found in supervised methods, while also avoiding the influence of label bias. Therefore, using unsupervised anomaly detection algorithms to perform binary classification of industrial images, accurately distinguishing the subtle features of normal and abnormal images, is of great significance for improving the quality and efficiency of industrial production.

[0004] The Vision-Based Transformer (ViT) is a deep learning model for the visual domain that offers better scalability and predictive capabilities than traditional Convolutional Neural Networks (CNNs). ViT networks segment images into a series of patches using a self-attention mechanism and model their dependencies, enabling global contextual awareness. However, this approach can lead to ViT models failing to effectively utilize local structure. In industrial defect images, defects are often localized, and ViT models' insufficient utilization of local information may result in low detection accuracy. Summary of the Invention

[0005] Based on the above ideas, this invention proposes an intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT). Addressing the problem that real-time non-contact measurement is not possible in the equipment manufacturing industry, this system can remotely capture the features of industrial products through a high-definition camera, detect abnormalities in the surface texture of industrial products, and ultimately determine whether the product has defects, thereby solving the problem of non-contact measurement.

[0006] To achieve the above objectives, the following technical solution was adopted:

[0007] An intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a data preprocessing module, an attention-based spatial transformation module (CSAM), an adversarial perturbation module (HAT), a VIT module, and a classifier module.

[0008] Data preprocessing module: Based on the JPEG compression standard Wallace (1992), the JPEG image of normal industrial samples is converted into a color space composed of luminance component Y, blue chrominance component Cb and red chrominance component Cr. The frequency domain coefficients of the image in spatial location are calculated by discrete cosine transform (DCT).

[0009] Attention-based Coordinate Space Transformation Module (CSAM): The system network module includes a multi-spectral channel attention unit and a coordinated attention unit, which are used to adaptively adjust the weight coefficients of all input frequency channels to capture global dependencies that are crucial to the visual task, so as to obtain full-frequency information with orientation and position sensitivity.

[0010] Multi-spectral channel attention unit: Employs an attention mechanism to automatically adjust the weight or attention level of each frequency band within multiple frequency bands of the original image.

[0011] Coordinated attention unit: Features are aggregated along two spatial directions, one to capture long-range dependencies and the other to retain precise location information, thus obtaining a pair of direction-aware feature maps.

[0012] Adversarial perturbation module: Enhances the high-frequency components of the feature map by adding adversarial perturbations, thereby improving the VIT model's ability to recognize details.

[0013] The VIT module utilizes an attention mechanism to model global image information. By segmenting the image into a series of image patches, it enables contextual awareness of the global context, further capturing key features in the image and improving the classification accuracy of subsequent classifiers.

[0014] Classifier module: The features output by the VIT module are processed by the classifier to obtain accurate classification predictions.

[0015] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a data preprocessing module that converts the original JPEG image to the YCbCr color space, dividing the luminance and chrominance components into 8×8 block components. Then, for each 8×8 block component of the three channels, a Discrete Cosine Transform (DCT) is applied to transform the spatial domain to the frequency domain. In other words, the two-dimensional DCT transform represents each 8×8 block as a set of 8×8 DCT coefficients, used to represent image features at different frequencies. The DCT function is as follows:

[0016]

[0017] In the above formula, f(h,w) are the coefficients after DCT transformation, x x,j The original input is H, where x is the input. x,j The height, W is x x,j The width of the channel is determined by combining all DCT coefficients into a single channel. This means combining all DCT coefficients in the Y channel into one channel, all DCT coefficients in the Cb channel into one channel, and all DCT coefficients in the Cr channel into one channel. Therefore, each Y, Cb, and Cr channel provides 8 × 8 = 64 channels. Assume the original RGB input image has a shape of H × W × C, where C = 3, and the image height and width are H and W, respectively. After transformation to the frequency domain, the channel feature shapes of Y, Cb, and Cr become H / 8 × W / 8 × 64. After obtaining the features of the Y, Cb, and Cr channels, CSAM is used to assign different weights to different channels to maximize the low-frequency information representing the function contour while preserving other high-frequency information that determines the detailed information.

[0018] The aforementioned intelligent defect detection system for industrial images based on Multi-Domain Adversarial Vision Transformer (MAViT) includes an attention-based Coordinate Space Transformation Module (CSAM). This module combines channel attention with coordinate attention to preserve both global relationships and precise location information. The calculation process of the CSAM module can be represented by the following formula:

[0019] F = f × f c ×z h ×z w

[0020] Where f is the input feature, f c The weights generated for the multispectral channel attention unit, z h , z wF represents the horizontal and vertical attention weights generated by the coordinate attention unit, and F is the output feature of CSAM.

[0021] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a multi-spectral channel attention unit that processes the input frequency channels f = [f0, f1, ... f...]. n-1 After passing through a fully connected (FC) layer and a sigmoid activation function, global average pooling is performed on the frequency channels, retaining only the lowest frequency channel. In other words, for an input feature f composed of different frequency channels, global average pooling is performed only on the lowest frequency channel, resulting in a scalar value as the low-frequency global pooling result, which is f. c This can be expressed as a formula:

[0022] f c =Sigmoid(FC(f))

[0023] The aforementioned intelligent defect detection system for industrial images based on Multi-Domain Adversarial Vision Transformer (MAViT) utilizes a coordinated attention unit that aggregates features in two spatial directions. One direction captures long-range dependencies, while the other retains precise positional information, resulting in a pair of direction-aware feature maps. First, for a given input feature f, two different pooling kernels are used to encode each channel in the horizontal and vertical coordinates, obtaining a pair of one-dimensional features fh and ffh. w Then, these are concatenated and fed into a shared 1×1 convolutional activation function, which then splits them into two separate tensors along the spatial dimension. Each tensor is then convolved into a tensor with the same number of channels as the input f. The activation function generates attention weights; therefore, the entire coordinate attention can be represented by the following formula:

[0024] f→f h ,f w

[0025] y = Sigmoid[W1Concat(f h ,f w )]

[0026] y→y h ,y w

[0027] z h =Sigmoid(W h y h )

[0028] z w =Sigmoid(W w y w)

[0029] Where Sigmoid is the activation function, W i (i = 1, w, h) is a 1×1 convolution kernel used to perform linear transformations on the features. `Concat(·)` is the concatenation function used to join two tensors along their spatial dimensions. `y` is the intermediate vector, representing the result of concatenating the features in the two directions. h and y w These are one-dimensional features in the horizontal and vertical directions. The purpose of these operations and functions is to help the network generate spatially selective attention maps more accurately. By introducing activation functions and linear transformations, non-linear transformations and dimensional mappings can be applied to the features, enhancing the network's expressive power. By using connection functions, features in the horizontal and vertical directions are connected, allowing the network to simultaneously consider and utilize feature information from different directions. This allows the network to generate attention maps more accurately for different locations, thus better focusing on regions of interest and targets. Unlike channel attention, which generates independent feature vectors, these operations help the network generate spatially selective attention maps more accurately, thereby improving the model's perceptual ability and performance.

[0030] After processing by the spectral channel attention unit and the coordinated attention unit described above, a feature map with redistributed weights of dimension C×H×W is obtained.

[0031] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes an adversarial perturbation module. For training samples, this module adds adversarial perturbations to alter the high-frequency components of the training images. Specifically, for each training sample, adversarial samples are generated to calculate the adversarial perturbation, which is then applied to the high-frequency components of the input image. The modified images are then used for model training to optimize the objective function. In this way, during training, the model gradually learns feature representations that are more sensitive to high-frequency components, improving its ability to process high-frequency information and thus enhancing the performance of the ViT model. The adversarial training objective function for the high-frequency perturbation is as follows:

[0032]

[0033] L(θ,x,y)=CE(f θ (x),y)

[0034]

[0035] Here, ε represents the maximum perturbation strength. CE(·) and KL(·) calculate the cross-entropy and KL divergence, respectively, and α and β are two hyperparameters. We use a high-pass filter. To limit disturbances in the high-frequency domain.

[0036] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) involves a ViT module that converts image data into a vector representation usable for classification. It segments the input image into a set of small image patches, then adds a positional encoding to each patch, inputting these patches into a stacked Transformer encoder for feature extraction and modeling. These encoders progressively map the features of the image patches to higher-level feature representations, ultimately generating a representation vector for the entire image. This generated vector can then be passed to subsequent classifier modules. The formula for adding the positional encoding is as follows:

[0037]

[0038]

[0039] Where pos represents the position of the image patch in the sequence, i represents the position index of the position encoding, and d model This represents the dimension of the hidden state in the Transformer encoder.

[0040] In addition, Transformer's self-attention mechanism is used to capture the relationships between image patches, generating three different vectors based on the input vector: Query, Key, and Value. k The scaling factor for attention weights. The process of calculating attention weights and weighted aggregation by the self-attention mechanism is as follows:

[0041]

[0042] To enhance the model's representational power, ViT employs a multi-head attention mechanism, which performs parallel computation on different projection matrices and then concatenates them in the final layer. The calculation formula for the multi-head attention mechanism is as follows:

[0043] MultiHead(Q,K,V)=Concat(head1,head2,…,head h W O

[0044] head i =Attention(QW qi ,KW ki VW vi )

[0045] Where h is the number of heads, W O It is the output transformation matrix, W qi W ki Wvi These are the query, key, and value transformation matrices for the i-th head, respectively.

[0046] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a classifier module consisting of three fully connected layers and a softmax function. The loss function is defined as cross-entropy loss, as shown in the following formula: where m is the number of images and Y is the known classification label.

[0047]

[0048] During the training process of this algorithm, the minimum value of the loss function is calculated using gradient descent, and the parameters are updated according to the calculated gradient direction. After a complete training process, a relatively ideal prediction and classification effect will be obtained.

[0049] The beneficial effects of this invention are:

[0050] This invention, by incorporating the latest research in related fields, constructs and trains a model capable of accurately predicting whether the surface morphology of carbon fiber materials is abnormal. The trained model can be used for detecting surface micro-texture defects in industrial images (such as nanofiber materials and other fiber composites), and can be integrated and adapted to different devices, comprehensively improving the impact of surface texture anomalies on the quality of industrial materials such as fiber composites. The results of this invention will be applicable to multiple fields and are implemented at a very low cost; general industrial manufacturers can complete the entire process using purchased high-definition cameras and computer equipment. This project can also cultivate a group of important technical backbones for the theoretical system of surface micro-morphology characteristics of industrial materials. These researchers will become core members supporting the sustainable development of the project team, and their research results, such as papers and monographs, will be applied to the research and application of the theoretical system of surface micro-morphology characteristics.

[0051] The model classification result of the intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) has an AUC of 96.2%, and the response time to complete a classification task is usually less than or equal to 1.5 seconds. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall process of the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of the present invention.

[0053] Figure 2 This is a schematic diagram of the operation of the preprocessing module of the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of the present invention.

[0054] Figure 3This is a schematic diagram of the operation of the coordinate space transformation module (CSAM) of the intelligent industrial image defect detection system based on multi-domain adversarial vision Transformer of the present invention.

[0055] Figure 4 The ROC curves for the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of this invention and two other methods are shown.

[0056] Figure 5 This is a graph showing the detection rate of the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of this invention, compared with two other methods.

[0057] Figure 6 This is a graph showing the false detection rate of the intelligent industrial image defect detection system based on multi-domain adversarial vision Transformer of this invention, compared with two other methods.

[0058] Figure 7 The image in question is a defective image predicted by the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer according to the present invention.

[0059] Figure 8 This is the image showing the difference between the output and the original image in the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of this invention. Detailed Implementation

[0060] The present invention will be described in detail below with reference to specific embodiments.

[0061] A multi-domain adversarial vision transformer (MAViT) intelligent industrial image defect detection system, the overall process of which is as follows: Figure 1 As shown:

[0062] For A1, which provides normal JPEG images with texture features from equipment manufacturing enterprises, the image size is standardized by adjusting the input image to 256×256. This system uses an unsupervised method and does not require labels.

[0063] A2 inputs images of uniform size into the data preprocessing module. The structure of the data preprocessing module is as follows: Figure 2 As shown, a typical industrial JPEG image is converted into a color space composed of the luminance component Y, the blue chrominance component Cb, and the red chrominance component Cr. The luminance and chrominance components are then divided into 8×8 blocks. The frequency domain coefficients of the image at its spatial location are calculated using Discrete Cosine Transform (DCT). The DCT function is shown in the following formula:

[0064]

[0065] In the above formula, f(h,w) are the frequency domain coefficients after DCT transformation, x x,j The original input is H, where x is the input. x,j The height, W is x x,j The width of the channel is determined by combining all DCT frequency domain coefficients into the same channel. This means combining all DCT frequency domain coefficients in the Y channel into one channel, all DCT frequency domain coefficients in the Cb channel into one channel, and all DCT frequency domain coefficients in the Cr channel into one channel. Therefore, each Y, Cb, and Cr provides 8 × 8 = 64 channels. Assume the original RGB input image has a shape of H × W × C, where C = 3, and the image height and width are H and W, respectively. After conversion to the frequency domain, the channel feature shapes of Y, Cb, and Cr become H / 8 × W / 8 × 64. After obtaining the features of the Y, Cb, and Cr channels, CSAM is used to assign different weights to different channels to maximize the low-frequency information representing the function contour while preserving other high-frequency information that determines the details.

[0066] A3 inputs the processed channel information into the Coordinate Space Transformation Module (CSAM), which integrates channel attention and coordinate attention, for redistribution of channel weights. The specific implementation process of the Coordinate Space Transformation Module (CSAM) is as follows: Figure 3 As shown, the calculation process can be represented by the following formula:

[0067] F = f × f c ×z h ×z w

[0068] Where f is the input feature, f c The weights generated for the multispectral channel attention unit, z h , z w F represents the horizontal and vertical attention weights generated by the coordinate attention unit, and F is the output feature of CSAM.

[0069] The specific steps of step A3 are as follows:

[0070] A31, calculate channel attention from the input feature map.

[0071] like Figure 3 As shown, in Figure 3 In the input feature map, the dimension is C×H×W, and there are n frequency channels, f=[f0,f1,…f n-1The input feature f is transformed into a C×1×1 tensor by passing it through a fully connected layer (FC). Then, it undergoes a sigmoid activation function, performing global average pooling on the frequency channels and retaining only the lowest frequency channel. For an input feature f composed of different frequency channels, global average pooling is performed only on the lowest frequency channel, resulting in a scalar value as the low-frequency global pooling result, which is f. c This can be expressed as a formula:

[0072] f c =Sigmoid(FC(f))

[0073] A32, the input feature map C×H×W is encoded in each channel on the horizontal and vertical coordinates using two different pooling kernels. By performing average pooling on each row and column of each channel while keeping other dimensions unchanged, the width or height is compressed to 1, resulting in a C×H×1 feature map f. h and f with dimension C×1×W w .

[0074] f→f h ,f w

[0075] A33, then concatenate them into a C×1×(W+H) tensor, and after 1×1 Conv activation, batch normalize the data to accelerate model training and improve model convergence and robustness.

[0076] y = Sigmoid[W1Concat(f h ,f w )]

[0077] A34 is then average-pooled along its horizontal and vertical coordinates, causing it to be split into two independent tensors along the spatial dimension. Each tensor is then convolved into a tensor with the same number of channels as the input f.

[0078] y→y h ,y w

[0079] A35, then activated by the Sigmoid activation function to obtain z. h and z w The original feature map f is compared with the calculated f. c z h and z w After multiplication, we obtain a new feature map F with redistributed weights. The dimensions remain the same as the original feature map, which is still C×H×W.

[0080] z h =Sigmoid(W h yh )

[0081] z w =Sigmoid(W w y w )

[0082] F = f × f c ×z h ×z w

[0083] Where Sigmoid is the activation function, W i (i = 1, w, h) is a 1×1 convolution kernel used to perform linear transformations on the features. `Concat(·)` is the concatenation function used to join two tensors along their spatial dimensions. `y` is the intermediate vector, representing the result of concatenating the features in the two directions. h and y w These are one-dimensional features in the horizontal and vertical directions. The purpose of these operations and functions is to help the network generate spatially selective attention maps more accurately. By introducing activation functions and linear transformations, non-linear transformations and dimensional mappings can be applied to the features, enhancing the network's expressive power. By using connection functions, features in the horizontal and vertical directions are connected, allowing the network to simultaneously consider and utilize feature information from different directions. This allows the network to generate attention maps more accurately for different locations, thus better focusing on regions of interest and targets. Unlike channel attention, which generates independent feature vectors, these operations help the network generate spatially selective attention maps more accurately, thereby improving the model's perceptual ability and performance.

[0084] The new feature map F after the weights of A4 are reassigned is input into the adversarial perturbation module.

[0085] An adversarial perturbation module is a module that adds adversarial perturbations to training samples to alter the high-frequency components of training images. This is because in the process of intelligent defect detection in industrial images, defective parts often appear in the form of high-frequency information, while the attention-based ViT network does not make sufficient use of high-frequency information. To compensate for this missing important information, an adversarial perturbation module is added.

[0086] In practice, for each training sample, adversarial perturbations are calculated by generating adversarial examples and applied to the high-frequency components of the input image. Then, the modified images are used for model training to optimize the objective function. In this way, during training, the model gradually learns feature representations that are more sensitive to high-frequency components, improving its ability to process high-frequency information and thus enhancing the performance of the ViT model. The adversarial training objective function for high-frequency perturbations is as follows:

[0087]

[0088] L(θ,x,y)=CE(f θ (x),y)

[0089]

[0090] Here, ε represents the maximum perturbation strength. CE(·) and KL(·) calculate the cross-entropy and KL divergence, respectively, and α and β are two hyperparameters. We use a high-pass filter. To limit disturbances in the high-frequency domain.

[0091] Cross-entropy is used to measure the difference between two probability distributions. In deep learning, cross-entropy is often used as a loss function to measure the difference between the model's predictions and the true labels. Assuming there are two probability distributions P and Q, the formula for cross-entropy is:

[0092] H(p,q)=-∑ i p i log2(q i )

[0093] Where i represents event i, p i q i Let p and q represent the probabilities of two distributions occurring in event i. Cross-entropy can be used to evaluate how well the model's predicted distribution p fits the true distribution q.

[0094] High-pass filter A filter is a commonly used filter in image processing to enhance or highlight high-frequency information in an image, that is, the details in an image that change dramatically. It can help extract edges, textures, and other high-frequency features from an image.

[0095] A5 inputs the frequency domain information, which has been processed by adversarial perturbation, into the ViT module to extract features.

[0096] The A51ViT module divides the input frequency domain information into small blocks, each block being a patch. Each patch is first processed by Patch Embedding, then compressed into a vector of a certain dimension via a fully connected network. It segments the input image into a set of small image patches and then adds a positional code to each patch. The formula for adding the positional code is as follows:

[0097]

[0098]

[0099] Where pos represents the position of the image patch in the sequence, i represents the position index of the position encoding, and dmodel This represents the dimension of the hidden state in the Transformer encoder.

[0100] The A52 takes small image patches with added positional encoding and inputs them into a stacked Transformer encoder to extract features and model each patch. First, there's the attention block, which generates three different vectors based on the input vector: Query, Key, and Value. An attention score is generated from the query and key vectors. This score is then calculated by multiplying q by k and dividing by the scaling factor d. k Dividing by a scaling factor is necessary because the result of the dot product is usually very large, making the softmax result unable to accurately represent the attention value. Dividing by a scaling factor can alleviate this problem to some extent. Multiplying the obtained attention score by the Value yields a new attention score, which, after passing through softmax, results in an attention vector for the currently computed image patch. The entire calculation process can be expressed by the following formula:

[0101]

[0102] To enhance the model's representational power, ViT employs a multi-head attention mechanism. This mechanism performs parallel computation on different projection matrices and then concatenates them in the final layer. Specifically, when generating q, k, and v, they are divided into n parts, each undergoing a self-attention operation. Finally, these parts are concatenated. This method ensures that the self-attention computations between different blocks are independent and do not interfere with each other. The calculation formula for the multi-head attention mechanism is as follows:

[0103] MultiHead(Q,K,V)=Concat(head1,head2,…,head h W O

[0104] head i =Attention(QW qi ,KW ki VW vi )

[0105] Where h is the number of heads, W O It is the output transformation matrix, W qi W ki W vi These are the query, key, and value transformation matrices for the i-th head, respectively.

[0106] In Transformer, A53 employs a residual network mechanism, adding the attention score from the multi-head attention output to the Transformer's input Patch Embedding, and then inputting it into the layer normalization block. Layer normalization normalizes all features of each sample.

[0107] The features from A6 after layer normalization are input into the classifier for accurate classification and prediction.

[0108] The classifier consists of three fully connected layers and a softmax function. The loss function is defined as the cross-entropy loss, as shown in the following formula, where m is the number of images and Y is the known class label:

[0109]

[0110] During the training process of this algorithm, the minimum value of the loss function is calculated using gradient descent, and the parameters are updated based on the calculated gradient direction. After a complete training process, a relatively ideal prediction and classification effect can be obtained.

[0111] An intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a data preprocessing module, an attention-based spatial transformation module (CSAM), an adversarial perturbation module (HAT), a VIT module, and a classifier module.

[0112] Data preprocessing module: Based on the JPEG compression standard Wallace (1992), the JPEG image of normal industrial samples is converted into a color space composed of luminance component Y, blue chrominance component Cb and red chrominance component Cr. The frequency domain coefficients of the image in spatial location are calculated by discrete cosine transform (DCT).

[0113] Attention-based Coordinate Space Transformation Module (CSAM): The system network module includes a multi-spectral channel attention unit and a coordinated attention unit, which are used to adaptively adjust the weight coefficients of all input frequency channels to capture global dependencies that are crucial to the visual task, so as to obtain full-frequency information with orientation and position sensitivity.

[0114] Multi-spectral channel attention unit: Employs an attention mechanism to automatically adjust the weight or attention level of each frequency band within multiple frequency bands of the original image.

[0115] Coordinated attention unit: Features are aggregated along two spatial directions, one to capture long-range dependencies and the other to retain precise location information, thus obtaining a pair of direction-aware feature maps.

[0116] Adversarial perturbation module: Enhances the high-frequency components of the feature map by adding adversarial perturbations, thereby improving the VIT model's ability to recognize details.

[0117] The VIT module utilizes an attention mechanism to model global image information. By segmenting the image into a series of image patches, it enables contextual awareness of the global context, further capturing key features in the image and improving the classification accuracy of subsequent classifiers.

[0118] Classifier module: The features output by the VIT module are processed by the classifier to obtain accurate classification predictions.

[0119] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a data preprocessing module that converts the original JPEG image to the YCbCr color space, dividing the luminance and chrominance components into 8×8 block components. Then, for each 8×8 block component of the three channels, a Discrete Cosine Transform (DCT) is applied to transform the spatial domain to the frequency domain. The DCT function is as follows:

[0120]

[0121] In the above formula, f(h,w) are the coefficients after DCT transformation, x x,j The original input is H, where x is the input. x,j The height, W is x x,j The width of the Y, Cb, and Cr channels is given. Each Y, Cb, and Cr channel provides 8×8 = 64 channels. Assume the original RGB input image has a shape of H×W×C, where C = 3, and the image height and width are H and W, respectively. After transformation to the frequency domain, the channel feature shapes of Y, Cb, and Cr become H / 8×W / 8×64. After obtaining the features of the Y, Cb, and Cr channels, CSAM is used to assign different weights to different channels to maximize the low-frequency information representing the function profile while preserving other high-frequency information that determines detailed information.

[0122] The aforementioned intelligent defect detection system for industrial images based on Multi-Domain Adversarial Vision Transformer (MAViT) includes an attention-based Coordinate Space Transformation Module (CSAM). This module combines channel attention with coordinate attention to preserve both global relationships and precise location information. The calculation process of the CSAM module can be represented by the following formula:

[0123] F = f × f c ×z h ×z w

[0124] Where f is the input feature, f cThe weights generated for the multispectral channel attention unit, z h , z w F represents the horizontal and vertical attention weights generated by the coordinate attention unit, and F is the output feature of CSAM.

[0125] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a multi-spectral channel attention unit that processes the input frequency channels f = [f0, f1, ... f...]. n-1 After passing through a fully connected layer (FC) and the sigmoid activation function, when performing global average pooling on the frequency channels, only the lowest frequency channel is retained, i.e., f. c This can be expressed as a formula:

[0126] f c =Sigmoid(FC(f))

[0127] The aforementioned intelligent defect detection system for industrial images based on Multi-Domain Adversarial Vision Transformer (MAViT) utilizes a coordinated attention unit that aggregates features in two spatial directions: one direction captures long-range dependencies, while the other retains precise location information, resulting in a pair of direction-aware feature maps. First, for a given input feature f, two different pooling kernels are used to encode each channel in both the horizontal and vertical coordinates, obtaining a pair of one-dimensional feature maps f. h and f w Then, these are concatenated and fed into a shared 1×1 convolutional activation function, which then splits them into two separate tensors along the spatial dimension. Each tensor is then convolved into a tensor with the same number of channels as the input f. The activation function generates attention weights; therefore, the entire coordinate attention can be represented by the following formula:

[0128] f→f h ,f w

[0129] y = Sigmoid[W1Concat(f h ,f w )]

[0130] y→y h ,y w

[0131] z h =Sigmoid(W h y h )

[0132] z w =Sigmoid(W w y w )

[0133] Where Sigmoid is the activation function, W i (i = 1, w, h) is a 1×1 convolution kernel used to perform linear transformations on the features. `Concat(·)` is the concatenation function used to join two tensors along their spatial dimensions. `y` is the intermediate vector, representing the result of concatenating the features in the two directions. h and y w It is a one-dimensional feature in the horizontal and vertical directions.

[0134] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes an adversarial perturbation module. For training samples, this module adds adversarial perturbations to alter the high-frequency components of the training images. Specifically, for each training sample, adversarial perturbations are calculated by generating adversarial examples, and these perturbations are applied to the high-frequency components of the input image. Then, the modified images are used for model training to optimize the objective function. The adversarial training objective function for the high-frequency perturbations is as follows:

[0135]

[0136] L(θ,x,y)=CE(f θ (x),y)

[0137]

[0138] Here, ε represents the maximum perturbation strength. CE(·) and KL(·) calculate the cross-entropy and KL divergence, respectively, and α and β are two hyperparameters. We use a high-pass filter. To limit disturbances in the high-frequency domain.

[0139] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) involves a ViT module that converts image data into a vector representation usable for classification. It segments the input image into a set of small image patches, then adds a positional encoding to each patch, inputting these patches into a stacked Transformer encoder for feature extraction and modeling. These encoders progressively map the features of the image patches to higher-level feature representations, ultimately generating a representation vector for the entire image. This generated vector can then be passed to subsequent classifier modules. The formula for adding the positional encoding is as follows:

[0140]

[0141]

[0142] Where pos represents the position of the image patch in the sequence, i represents the position index of the position encoding, and d model This represents the dimension of the hidden state in the Transformer encoder.

[0143] In addition, Transformer's self-attention mechanism is used to capture the relationships between image patches, generating three different vectors based on the input vector: Query, Key, and Value. k The scaling factor for attention weights. The process of calculating attention weights and weighted aggregation by the self-attention mechanism is as follows:

[0144]

[0145] To enhance the model's representational power, ViT employs a multi-head attention mechanism, which performs parallel computation on different projection matrices and then concatenates them in the final layer. The calculation formula for the multi-head attention mechanism is as follows:

[0146] MultiHead(Q,K,V)=Concat(head1,head2,…,head h W O

[0147] head i =Attention(QW qi ,KW ki VW vi )

[0148] Where h is the number of heads, W O It is the output transformation matrix, W qi W ki W vi These are the query, key, and value transformation matrices for the i-th head, respectively.

[0149] The aforementioned intelligent industrial image defect detection system based on Multi-Domain Adversarial Vision Transformer (MAViT) includes a classifier module consisting of three fully connected layers and a softmax function. The loss function is defined as cross-entropy loss, as shown in the following formula: where m is the number of images and Y is the known classification label.

[0150]

[0151] During the training process of this algorithm, the minimum value of the loss function is calculated using gradient descent, and the parameters are updated according to the calculated gradient direction. After a complete training process, a relatively ideal prediction and classification effect will be obtained.

[0152] 0149] Figure 4 This is a comparison of the ROC curves of the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of this invention with two other methods. Method 1 uses a traditional visual Transformer model for detection without the CSAM and HAT modules, while Method 2 uses a single CNN model for classification and detection. It can be seen that the intelligent industrial image defect detection system achieves the best average detection rate after incorporating the CSAM and HAT modules. This is because the CSAM module adaptively adjusts the weight coefficients for all input frequency channels, capturing the global dependencies crucial to the visual task. Simultaneously, the HAT module enhances high-frequency information, compensating for the Transformer model's insufficient utilization of high-frequency information. It can be seen that the AUC value of this invention reaches 0.962, and compared to the other two methods, the overall system accuracy, sensitivity, and specificity are all higher.

[0153] Figure 5 This is a comparison chart of the detection rates of the industrial image defect intelligent detection system based on multi-domain adversarial visual Transformer of this invention and two other methods. Methods 1 and 2 are described as follows: Figure 4 As can be seen, compared with other methods, the system proposed in this invention has the highest detection rate, and the average detection rate is still higher than the other two models as the number of image sequences increases.

[0154] Figure 6 This is a graph showing the false detection rate of the intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer of this invention, compared to two other methods. Methods 1 and 2 are described as follows: Figure 4 As can be seen, the false detection rate of this method is the lowest. As the number of image sequences increases, the false detection rate can still be stabilized within a relatively reliable range, and the false detection rate is still the lowest.

[0155] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An intelligent industrial image defect detection system based on multi-domain adversarial visual Transformer, characterized in that, It includes a data preprocessing module, an attention-based spatial transformation module (CSAM), a HAT adversarial perturbation module, a VIT module, and a classifier module; the data preprocessing module is used to process industrial product images, based on... Compression Standard (1992), normal samples Image, converted to luminance components Blue chromaticity component and red chromaticity component The color space is composed of discrete cosine transform ( The frequency domain coefficients of the image at its spatial location are calculated. Attention-Based Coordinate Space Transformation Module (CSAM): The coordinate space transformation module includes a multi-spectral channel attention unit and a coordinated attention unit, which are used to adaptively adjust the weight coefficients of all input frequency channels to capture global dependencies that are crucial to the visual task, so as to obtain full-frequency information with orientation and position sensitivity; the multi-spectral channel attention unit: uses an attention mechanism to automatically adjust the weight or attention of each frequency band of information in multiple frequency bands of the original image. The multi-spectral channel attention unit, as described above, will input frequency channels After the fully connected layer and after The activation function, when performing global average pooling on frequency channels, only retains the lowest frequency channel; that is, for input features composed of different frequency channels... Global average pooling is performed only on the lowest frequency channel, resulting in a scalar value as the global pooling result for the low frequency channel. This can be expressed as a formula: ; Coordinated Attention Unit: Aggregates features along two spatial directions, one to capture long-range dependencies and the other to preserve precise location information, thus obtaining a pair of direction-aware feature maps; Adversarial Perturbation Module: Enhances the high-frequency components of the feature maps by adding adversarial perturbations, thereby improving the VIT module's ability to recognize details; The VIT module converts image data into a vector representation that can be used for classification; it segments the input image into a set of small image patches, then adds a positional code to each patch before inputting it into a stack. Encoders extract and model features from each image patch; these encoders gradually map the features of the image patches to higher-level feature representations, ultimately generating a representation vector for the entire image. This generated vector is then passed to the subsequent classifier module. The formula for adding positional encoding is as follows: ; ;in, Representing an image Position in the sequence, The location index represents the location code. express The dimension of the hidden state in the encoder; in addition, The self-attention mechanism in [the image] is used to capture images. The relationship between them, based on the input vector, will generate three different vectors, namely... , , The scaling factor for attention weights; the process of calculating attention weights and weighted aggregation using the self-attention mechanism is as follows: To increase the model's representational power, The multi-head attention mechanism performs parallel computation on different projection matrices and then concatenates them in the final layer. The calculation formula for the multi-head attention mechanism is as follows: ; ;in, It refers to the number of heads. It is the output transformation matrix. , , They are the first The query, key, and value transformation matrix for each item; Classifier module: The features output by the VIT module are processed by the classifier to obtain accurate classification predictions.

2. The intelligent industrial image defect detection system according to claim 1, characterized in that, The data preprocessing module input is a multi-branch input, which involves processing the raw data... Image conversion Space, namely the luminance component Blue chromaticity component and red chromaticity component Then, the luminance and chrominance components are divided into 8×8 block components; then, a discrete cosine transform is applied to each 8×8 block component of the three channels. Transform the spatial domain to the frequency domain, i.e., two-dimensional. The transformation will represent each 8×8 block as a group of 8×8 blocks. Coefficients are used to represent image features at different frequencies; The function is as follows: In the above formula yes The transformed frequency domain coefficients, For the original input, for height, for The width of all Coefficients are merged into the same channel, each , , Provides 8×8=64 channels; assuming the original The input image has The shape, in which The height and width of the image are respectively and After converting to the frequency domain, , and The channel feature shape becomes ,get , and After characterizing the channels, CSAM is used to assign different weights to different channels to maximize the low-frequency information representing the function profile while retaining other high-frequency information that determines the details.

3. The intelligent industrial image defect detection system according to claim 1, characterized in that, The attention-based coordinate space transformation module (CSAM) combines channel attention with coordinate attention in order to preserve the global relationships of the image while retaining precise positional information. The calculation process of the Coordinate Space Transformation Module (CSAM) is represented by the following formula: ;in, These are input features. Weights generated for multispectral channel attention units, , The horizontal and vertical attention weights generated for the coordinate attention unit. It is the output feature of CSAM.

4. The intelligent industrial image defect detection system according to claim 1, characterized in that, The aforementioned coordinated attention unit aggregates features in two spatial directions: one direction captures long-range dependencies, and the other direction retains precise location information, thus obtaining a pair of direction-aware feature maps. First, given the input features... Two different pooling kernels are used to encode each channel on the horizontal and vertical coordinates respectively, to obtain a one-to-one one-dimensional feature. and Then they are concatenated and fed into a shared 1×1 convolutional activation function, and then split into two separate tensors along the spatial dimension, which are then convolved into the input tensors. Tensors with the same number of channels; the activation function generates attention weights, therefore, the entire coordinate attention representation is as follows: ; ; ; ;in It is an activation function. yes Convolution kernels are used to perform linear transformations on features; It is a join function used to join two tensors according to their spatial dimensions; It is an intermediate vector, representing the result of connecting features in two directions; and It is a one-dimensional feature in the horizontal and vertical directions; after processing by the above-mentioned spectral channel attention unit and coordinated attention unit, a one-dimensional feature is obtained. The feature map after weight redistribution.

5. The intelligent industrial image defect detection system according to claim 1, characterized in that, The adversarial perturbation module, for training samples, modifies the high-frequency components of the training images by adding adversarial perturbations. For each training sample, adversarial perturbations are calculated by generating adversarial examples and applied to the high-frequency components of the input image. Then, the modified images are used for model training to optimize the objective function. The adversarial training objective function for high-frequency perturbations is as follows: ; ; Use a high-pass filter To limit disturbances in the high-frequency domain.

6. The intelligent industrial image defect detection system according to claim 1, characterized in that, The classifier module consists of three fully connected layers and one... The function consists of a loss function defined as cross-entropy loss, as shown in the following formula, where... For the number of images, Known category labels: During training, the minimum value of the loss function is calculated using gradient descent, and the parameters are updated based on the calculated gradient direction. After a complete training process, a relatively ideal prediction and classification effect can be obtained.

Images