Feature Information Reconstruction Industrial Defect Detection Method Based on Multidimensional Feature Fusion

Abstract

The present invention discloses a feature information reconstruction industrial defect detection method based on multi-dimensional feature fusion, which relates to the technical field of defect detection. Compared with previous defect detection methods, the present invention solves the problems of traditional defect detection methods being sensitive to environmental changes, having poor generalization ability, being unable to meet the requirements of real-time performance, having poor reconstruction performance effects, and low generalization performance of the system; introducing a cropped autoencoder AE to complete the reconstruction of normal feature information of defective samples, reducing the number of network parameters and computational complexity. And using an embedding module to replace the clipping part to maintain the system performance, increase the feature information of normal images, and complete the reconstruction function of the reconstruction module. And introducing an MFF module to reuse the feature information of the encoder module in the AE reconstruction module; an information correction mechanism is additionally introduced in the MFF module to prevent the problem of the reconstruction performance decline caused by the leakage of abnormal information, and the normal feature information of the teacher network is used to guide the correct use of multi-dimensional feature information.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and particularly to an industrial defect detection method for reconstructing feature information based on multi-dimensional feature fusion. Background Art

[0002] In industrial inspection, defect detection is a very important part. By detecting defects in products, problems can be discovered and repaired in a timely manner to ensure product quality and safety. However, traditional manual inspection is time-consuming and laborious, and may also lead to the bad situations of missed detection and misdetection. Therefore, with the development of technology, machine vision has been widely introduced into the field of defect detection in industrial inspection and achieved excellent results.

[0003] Traditional industrial defect detection technologies perform detection based on texture features, shape features and deep learning methods.

[0004] Defect detection methods based on texture features can be further divided into two categories: statistical methods and signal processing methods. Statistical methods mainly regard the gray value distribution on the surface of an object as a random distribution, analyze the distribution of random variables from a statistical perspective, and describe the spatial distribution of gray values through features such as histogram features, gray level co-occurrence matrix, local binary pattern, autocorrelation function, mathematical morphology, etc. Signal processing methods mainly regard an image as a two-dimensional signal and analyze the image from the perspective of signal filter design, so they are also called spectral methods.

[0005] Methods based on shape features can effectively use the targets of interest in an image for retrieval. Among them, the method based on contour is the main method type. The method based on contour obtains the shape parameters of an image by describing the outer boundary features of an object; representative methods are the Hough transform and the Fourier shape descriptor. The Hough transform uses the global features of an image to connect edge pixels to form a closed boundary of a region, and its theoretical basis is the duality of point-to-line. It is mainly used for detecting defects on the surface of bottles; in the ROI extraction stage, the fast Hough transform is used to detect the light source boundary line.

[0006] However, traditional detection methods based on texture features and shape features often rely on simple feature description and classification methods, and are difficult to handle complex texture and shape changes. Moreover, traditional image processing algorithms are relatively sensitive to the brightness changes of the environment, and both over-brightness and over-darkness will affect the statistical information of the histogram, etc.

[0007] Deep learning-based methods can also be divided into two categories: reconstruction-based and embedding-based industrial defect detection methods. Reconstruction-based industrial defect detection technology models image feature information in the embedding space based on the autoencoder AE and reconstructs it from the embedding space. However, since these abnormal information are not evolved during the training process, they cannot be reconstructed. Therefore, the difference between the detected image and its reconstructed image represents the abnormal detection result. The core idea of the embedding-based method is to use a pre-trained model to save the target feature information and use it directly or indirectly. The former uses the feature information compressed by the strategy to compare with the sample feature information to locate the abnormality, and the latter establishes a memory bank through different means with the feature information to assist in abnormal location.

[0008] In order to improve the reconstruction performance, the reconstruction-based detection method uses a neural network model with excessive parameters and computational complexity in the reconstruction module, resulting in excessive parameter and computational complexity of the entire system and unable to meet the real-time requirements. Because the utilization of feature information is ignored, the performance effect of reconstruction cannot reach the best. The embedding-based detection method has poor generalization performance due to the introduction of additional computational modules and embedding modules.

[0009] To solve the above problems, the present invention proposes a feature information reconstruction industrial defect detection method based on multi-dimensional feature fusion. Summary of the Invention

[0010] The purpose of the present invention is to propose a feature information reconstruction industrial defect detection method based on multi-dimensional feature fusion to solve the problems raised in the background technology:

[0011] Traditional defect detection methods are sensitive to environmental changes and have poor generalization ability; they cannot meet the real-time requirements; the performance effect of reconstruction is not good; the generalization performance of the system is not high.

[0012] To achieve the above purpose, the present invention adopts the following technical solutions:

[0013] A feature information reconstruction industrial defect detection method based on multi-dimensional feature fusion includes the following steps:

[0014] S1: Build a cropped autoencoder AE;

[0015] S2: Introduce a teacher network to guide the cropped autoencoder AE to complete the reconstruction target of normal feature information;

[0016] S3: Build an embedding module and define a set of memories, use the input features to perform similarity addressing on the memory group, and obtain the reconstructed features therefrom;

[0017] S4: Build an MFF module to complete the guidance of the decoder characteristic reconstruction by the encoder multi-scale features;

[0018] S5: Build a lightweight segmentation network to complete the segmentation and localization of defects.

[0019] Preferably, in the trimmed autoencoder AE of S1, both the Encoder and the decoder are composed of the first three blocks of ResNet18 without pre-training. The Encoder is denoted as E k , k ∈ {1, Λ, N}, and the decoder is denoted as D k , k ∈ {1, Λ, N}; they are linked through an embedded memory module M in the middle; the image feature information corresponds respectively to the projection of the original data I into the embedding space; where, C k , H k , W k respectively represent the number of channels, height, and width of the activation tensor of the k-th layer.

[0020] Preferably, the teacher network of S2 uses the ResNet18 model pre-trained on the ImageNet dataset, and uses the feature information tensors of the first three block layers for information guidance; uses the cosine similarity as the KD loss; obtains the feature information tensor of the decoder through the autoencoder AE The normal image I n passes through the pre-trained teacher network to obtain the normal feature information tensor k ∈ {1, Λ, N}, calculates the vector-to-cosine similarity loss of k with the axis of the channel C and , and uses the similarity loss as the difference between the normal feature information and the abnormal feature information. By reducing the similarity loss, the autoencoder AE is enabled to have the ability to reconstruct the normal feature information of abnormal images. The similarity loss L [[ID=3 forty]] rec (I n , I ca ) is calculated as follows:

[0021]

[0022] where, d k (h, w) is the vector-to-cosine similarity loss; I ca is the abnormal image.

[0023] Preferably, the input of the embedding module in S3 is A group of memories is defined inside the embedding module for reconstructing features Use the input features to perform similarity addressing on the memory group:

[0024]

[0025] where m i is a subset of the memory group; i is the number of memories; w is the set of similarities between the input features and the memory group features; ω i represents the similarity between the input features and the memory group features;

[0026] The similarity ω i is calculated by the cosine distance:

[0027]

[0028]

[0029] where is a subset of the input features ; T0 is the transpose of the vector;

[0030] The memory group is reorganized with different feature ratios according to the magnitude of the similarity, and finally a processing process with the query feature as the input and the reorganized feature as the output is obtained. The output is defined as

[0031] The memory group M is obtained through the first-round model distillation training method; during the first-round model distillation training, the model input of the autoencoder AE is the normal sample image I n , and the data passes through the Encoder to obtain the normal feature information The memory data is saved by reducing the dimension of the normal feature information after the model parameters in the memory group are updated; the memory group obtains the reconstructed feature through similarity addressing Fitting and Finally, the memory group M of the embedding module is obtained.

[0032] Preferably, the input of the MFF module in S4 is the feature outputs of different blocks of the autoencoder AE Pooling is performed on E1, E2, and E3. After the processing, the alignment of the feature dimensions is completed, and the pooled E1, E2, and E3 are used as a feature information block; the feature information block contains semantic information of multiple scales, and the 1×1 is used to fuse the information in the channel direction of the block feature; the channel of the feature information block is adjusted, and the adjustment is sequentially performed to the corresponding channels and feature sizes, and M3, M2, M1 = {pool(D k , k ∈ 3, 2, 1)};

[0033] The MFF module also introduces a function of suppressing difference information; based on the cosine formula, the multi-scale feature information M3, M2, M1 and the feature information tensor of the teacher network are measured Differences between:

[0034]

[0035] Generate a threshold using the activation function Relu(·), and use the threshold to suppress the transmission of abnormal information:

[0036]

[0037] The multi-scale fusion information obtained by suppressing abnormal information will directly act on the corresponding feature layers of the decoder of the autoencoder AE.

[0038] Preferably, the segmentation network in S5 takes the feature information {T1, T2, T3} of the teacher network with the abnormal image as the input, and the feature information {S D1 , S D2 , S D3} of the Decoder of the AE autoencoder with the abnormal image as the input, and uses the difference in feature mapping between the feature information as the input of the segmentation network; before segmentation, the segmentation network also performs dimensionality reduction processing on the data information to be segmented;

[0039] In the segmentation network, the deeper information of the high-level semantic features is gradually passed to the shallow features through the IFC module, and the IFC module takes the output from the Encoder of the segmentation network as the input of the IFC module, performs convolution processing on each layer output, and then passes through a pooling layer; a CA attention processing block is introduced for feature blocks of different scales, and the dimensions and channels before and after the feature block remain unchanged after passing through the CA attention module; after passing through the CA Block, convolution is performed again to adjust the number of channels, and at this time The output of the IFC module is obtained through the following formula:

[0040]

[0041] Then, according to the basic structure of U-Lite, the processed multi-scale feature information is fused with the corresponding Decoder feature information block to obtain the fused feature information

[0042]

[0043] Through the above process, the IFC passes the deepest feature information layer by layer upwards, and each shallow feature contains the deeper feature information starting from the sublayer. Finally, the corresponding IFC output multi-scale feature information is transmitted to the Decoder module of U-Lite, and the segmentation function is finally completed. The segmentation module outputs a segmentation value Mask of a specified size pre .

[0044] Preferably, the segmentation network optimizes the segmentation training using focal loss and L1 loss;

[0045] Resample Mask pre and anomalyMask GT linearly to the same size, where i and j represent the xy coordinates of the image:

[0046]

[0047] where γ is a hyperparameter; p ij is the result of linear resampling; L focal is the focal loss; L l1 is the L1 loss;

[0048] The training of the segmentation network is completed through the above two loss functions of focal loss and L1 loss. The segmentation network model finally outputs a Mask mask showing the classification of the pixel points of the detected sample.

[0049] Compared with the prior art, the present invention provides a feature information reconstruction industrial defect detection method based on multi-dimensional feature fusion, having the following beneficial effects:

[0050] The present invention introduces a cropped autoencoder AE to complete the reconstruction of the normal feature information of the defective sample, cropping off the deepest convolutional layer, which greatly reduces the number of network parameters and the computational amount. And an embedding module is used to replace the clipped network convolutional layer to prevent the reduction of system performance and increase the feature information of the normal industrial sample image to complete the reconstruction function of the reconstruction module. And the MFF module is introduced to reuse the feature information of the encoder module in the AE reconstruction module; an information correction mechanism is additionally introduced in the MFF module to prevent the problem of the decline of the reconstruction performance caused by the leakage of abnormal information in the multi-dimensional feature information reconstruction, and the normal feature information of the teacher network is used to guide the correct use of the multi-dimensional feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is the flowchart of the detection method mentioned in Embodiment 1 of the present invention;

[0052] Figure 2 is the schematic diagram of the detection network mentioned in Embodiment 1 of the present invention;

[0053] Figure 3 is the schematic diagram of the IFC module mentioned in Embodiment 1 of the present invention;

[0054] Figure 4 is the schematic diagram of the feature fusion student-teacher network mentioned in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0056] The present invention introduces a cropped autoencoder AE to reconstruct the normal feature information of defective samples, and crops the deepest convolutional layer, which greatly reduces the number of network parameters and the amount of computation. Moreover, an embedding module is used to replace the clipped network convolutional layer to prevent the degradation of system performance, increase the feature information of normal industrial sample images, and complete the reconstruction function of the reconstruction module. And the MFF module is introduced to reuse the feature information of the encoder module in the AE reconstruction module; an information correction mechanism is additionally introduced in the MFF module to prevent the problem of the degradation of reconstruction performance caused by the leakage of abnormal information in the multi-dimensional feature information reconstruction, and use the normal feature information of the teacher network to guide the correct use of multi-dimensional feature information. The specific content is as follows.

[0057] Embodiment 1:

[0058] Please refer to Figures 1-4 , for the feature information reconstruction industrial defect detection method based on multi-dimensional feature fusion of the present invention, defect detection is performed on the MVTec AD dataset:

[0059] Preparation stage: For the MVTec AD dataset, adjust its mean and variance, and adjust the size of the image [256*256] using the bilinear interpolation method.

[0060] Simulate abnormal images: Generate random two-dimensional Berlin noise and binarize it according to a preset threshold to obtain an abnormal mask M. By replacing the masked area with a linear combination of the anomaly-free image I n and any image A in the external data source, an abnormal image I ca is generated.

[0061] Build a defect detection model, including the following steps:

[0062] S1: Build a cropped autoencoder AE, that is, a student network; specifically as follows:

[0063] Referring to the connection layer of the U-Net network, its connection structure can complete the transfer of feature information, thereby improving the performance of the model. Therefore, the proposed multi-scale feature fusion student-teacher network can be referred to Figure 4 , and after completing the multi-scale feature fusion, the same connection structure is also used to help the model transfer feature information.

[0064] In the autoencoder AE, both the Encoder and the decoder consist of the first three blocks of a pre-trained ResNet18. The Encoder can be denoted as E k , k ∈ {1, Λ, N}, and the decoder can be denoted as D k , k ∈ {1, Λ, N}; they are linked through an embedded memory module M in the middle; the image feature information corresponds respectively to the projection of the original data I into the embedding space; where, C k , H k , W k respectively represent the number of channels, height, and width of the activation tensor of the k-th layer.

[0065] S2: Introduce a teacher network to guide the cropped autoencoder AE to complete the reconstruction target of normal feature information; specifically as follows:

[0066] Introduce a teacher network, the purpose of which is to be able to guide the cropped autoencoder AE to complete the reconstruction target of normal feature information.

[0067] The teacher network uses a ResNet18 model pre-trained on the ImageNet dataset. For data alignment, the feature information tensors of the first three block layers are also selected for information guidance; for knowledge transfer in the T-S model, in order to accurately capture the relationship between high-dimensional and low-dimensional information, the cosine similarity is used as the KD loss; the feature information tensor of the decoder is obtained through the autoencoder AE Referring to the denoising mechanism of the student network model of the DeSTSeg model, the normal image I n obtains the normal feature information tensor through the pre-trained teacher network ResNet18 Calculate the vector-to-cosine similarity loss with the channel C k as the axis for and . Take the similarity loss as the difference between the normal feature information and the abnormal feature information, and enable the autoencoder AE to have the ability to reconstruct the normal feature information of abnormal images by reducing this similarity loss. The loss formula is as follows:

[0068]

[0069] where, d k (h, w) is the vector-to-cosine similarity loss; I ca is the abnormal image; L rec (I n , I ca ) is the similarity loss.

[0070] However, reconstructing normal features from abnormal features is a challenging task. Referring to TrustMAE, we introduced additional memory information to help the student network decoder better complete the feature reconstruction task. The introduced embedding module replaces the deepest block of the original ResNet18, namely the 2*conv3x3, C=512 convolution.

[0071] S3: Build an embedding module and define a set of memories. Use the input features to perform similarity addressing on the memory groups and obtain the reconstructed features. The details are as follows:

[0072] The input of the embedding module is A set of memories is defined inside the embedded module To reconstruct features Using input features Similarity addressing of memory groups:

[0073]

[0074] Among them, m i is a subset of the memory group; i is the number of memories; w is the similarity set between the input features and the memory group features; ω i Represents the similarity between input features and memory group features;

[0075] Similarity ω i Calculated by cosine distance:

[0076]

[0077] in, is the input feature A subset of ; T0 is the transpose of the vector;

[0078] By using the similarity, the memory group is reorganized with different size ratios of features, and finally a processing process is obtained with query features as input and reorganized features as output. The output is defined as

[0079] The memory group M is obtained through the first round of model distillation training; in the first round of model distillation training, the model of the student network inputs the normal sample image I n , the data passes through the student network Encoder to obtain normal feature information After the model parameters are updated inside the memory group, the normal feature information dimension reduction data is saved as memory data; the memory group obtains the reconstructed features through similarity addressing. Fitting and Finally, the memory group M of the embedded module is obtained.

[0080] S4: Build the MFF module to complete the guidance of the encoder's multi-scale features for the decoder's feature reconstruction. Specifically as follows:

[0081] The input of the MFF (Multi-scale feature fusion) module is the feature outputs of different blocks of the autoencoder AE To reduce the computational burden of the MFF module, the patch information of the extracted features is notified, and E1, E2, and E3 are pooled. After processing, the alignment of the feature dimensions is completed. To fuse the multi-scale feature information, the pooled E1, E2, and E3 are concatenated as a feature information block. In this way, the feature information block contains semantic information of multiple scales, and the 1×1 is used to fuse the information of the block feature in the channel direction. To match the right positions, that is, the semantic guidance of the same level guides the features of the same level, so as to avoid incorrect semantic guidance. The channel of the feature information block is adjusted, and adjusted in turn to The corresponding channels and feature sizes, and M3, M2, M1 = {pool(D k , k ∈ 3, 2, 1)} can be obtained; Since the feature information of the autoencoder AE contains abnormal information, if the information guidance is directly carried out, there may be a problem of abnormal feature information leakage during the feature information reconstruction. Therefore, the MFF module introduces a differential information suppression function; The differential information suppression function requires two input data, which are the multi-scale feature information M3, M2, M1 and the feature information tensor of the teacher network The former has abnormal feature information, and the latter only has the feature information of normal images. The difference between the feature information represents the abnormal information points. The difference between the two is measured based on the cosine formula:

[0082]

[0083] The activation function Relu(·) is used to generate a threshold, and the threshold is used to suppress the transmission of abnormal information:

[0084]

[0085] The multi-scale fusion information obtained by suppressing abnormal information will be directly applied to the corresponding feature layers of the student decoder of AE.

[0086] S5: Build a lightweight segmentation network to complete the segmentation and localization of defects. Specifically as follows:

[0087] The abnormal image is used as the feature information {T1, T2, T3} of the input teacher network and the feature information {S D1 , S D2 , SD3}, using the difference in feature mapping between feature information as the input to the segmentation network; before segmentation, considering the consumption of computing performance, the data information to be segmented is dimensionally reduced, and the channel dimension is reduced to 3 dimensions through a residual block of ResNet18.

[0088] Refer to Figure 3 , in the segmentation network, the deeper information of the high-level semantic features is gradually passed to the shallow features through the IFC module, so as to complete the multi-path refinement function and compensate for the information difference problem introduced by the reconstructed multi-scale feature information. The IFC module takes the output from the Encoder of the segmentation network as the input of the IFC module, performs convolution processing on the outputs of each layer to complete the compression of the channels, and then passes through a pooling layer to reduce the size of the features; a CA attention processing block is introduced for feature blocks of different scales, and the dimensions and channels before and after the feature block remain unchanged after passing through the CA attention module; in order to maintain the fusion of subsequent multi-scale feature information, the number of channels is adjusted again by convolution after passing through the CABlock, and at this time the output of the IFC module is obtained through the following formula:

[0089]

[0090] Then, according to the basic structure of U-Lite, the processed multi-size feature information is fused with the corresponding Decoder feature information block to obtain the fused feature information

[0091]

[0092] Through the above process, the IFC passes the deepest feature information layer by layer upwards, and each shallow feature contains deeper feature information starting from the sub-layer. Finally, the corresponding IFC output multi-scale feature information is passed to the Decoder module of U-Lite, and the segmentation function is finally completed. The segmentation module outputs a segmentation value Mask of a specified size pre .

[0093] The function of the segmentation network in the system is to segment the positions of the reconstructed features and defect features in the image. The so-called segmentation is the classification of image pixels, classified into normal pixels and abnormal pixels, and finally the defect positioning of the defective samples is completed.

[0094] The segmentation network uses focal loss and L1 loss to optimize the segmentation training; focal loss can help the model focus on minority classes and difficult samples. And L1 loss can improve the sparsity of the output and make the boundary of the segmentation mask clearer. The Mask pre and anomalyMaskGT Perform linear sampling to the same size, where i and j represent the xy coordinates of the image:

[0095]

[0096] where γ is a hyperparameter; p ij is the result of linear sampling; L focal is the focal loss; L l1 is the L1 loss;

[0097] Complete the training of the segmentation network through the above two loss functions, the focal loss and the L1 loss. The segmentation network model finally outputs a Mask mask showing the classification of the pixel points of the detected sample.

[0098] Training stage:

[0099] Adapt to the data distribution and obtain the storage module:

[0100] Traverse the normal images of industrial samples: Obtain the feature outputs of the student network and the teacher network Use the teacher network to guide the learning network to reconstruct the feature information:

[0101]

[0102] Initialize the embedding module and use the loss function L m to make the outputs of the student network and the teacher network similar and complete the construction of the embedding module:

[0103]

[0104] Use the Adam optimizer Adam(L rec +L m ) to train the model.

[0105] Reconstruct the abnormal feature information:

[0106] Use Perlin noise to simulate abnormal images through normal images; Traverse the normal image I n and the abnormal image I ca ;

[0107] Take I n and I ca as the inputs of the teacher network and the student network respectively to obtain the image feature information

[0108] Take the output of the student network encoder as the input of the MFF module to obtain the multi-dimensional feature fusion information;

[0109] Inject the multi-dimensional feature fusion information into the decoder module of the student network;

[0110] Calculate the loss function:

[0111]

[0112] Using SGD optimizer SGD(L rec ) to train the model.

[0113] Train the segmentation network:

[0114] Use the student network to reconstruct the output feature information and the teacher's output feature information and integrate the two information:

[0115]

[0116] Use ResNet18 residual module to adjust the channel:

[0117]

[0118] Input the adjusted feature information into the segmentation network to obtain the output of the segmentation network:

[0119]

[0120] Calculate the loss function and train the model, where Mask pre Represents the correct classification mask for the image:

[0121] L focal =(Mask pre ,Mask GT )

[0122] Adam(L focal )

[0123] Reasoning stage:

[0124] The defect detection process of the system: Input the image to be detected; Based on the student network: AE{E k ,D k Teacher Network: Segmentation Network: Traverse the image to be detected; output the segmented image of the image to be detected, as follows:

[0125] Preprocessing is completed by adjusting the size and mean variance of the image to be detected;

[0126] The image to be detected is sent as input to the teacher and student networks, and the student network completes the feature information reconstruction:

[0127] Use ResNet18 residual module to adjust feature information Number of channels:

[0128]

[0129] Use the segmentation network Seg to obtain the defect segmentation image output of the image to be detected:

[0130]

[0131] Finally, the system outputs the defect segmentation image output of the image to be detected.

[0132] Compare the defect detection method of this embodiment on the MVTec AD dataset with traditional methods such as Patch core, EfficientAD, RD, RD++, Dream, and Fastflow in terms of performance. Specifically, refer to Table 1:

[0133] Table 1 Computational efficiency performance

[0134]

[0135] As can be seen from the table, for the method of this embodiment in the MVTec AD public industrial defect data, the average value of AUROC [Area Under the Receiver Operating Characteristic Curve] at the image level reaches 99.40%, the average value of AUROC at the pixel level reaches 98.25%, and the average value of the PRO [Per-Region Overlap] metric reaches 95.44%. Moreover, in the MPDD public industrial defect data, the image-level defect detection performance reaches the SOTA level.

[0136] As described above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent replacements or changes, and should be covered within the protection scope of the present invention.

Claims

1. A method for industrial defect detection based on feature information reconstruction based on multi-dimensional feature fusion, characterized in that: The steps include: S1: Build a cropping autoencoder AE; S2: Introducing a teacher network to guide the cropped autoencoder AE to complete the reconstruction of normal feature information; the teacher network of S2 uses the ResNet18 model pre-trained on the ImageNet dataset, and uses the feature information tensors of the first three block layers for information guidance; using cosine similarity as the KD loss; obtaining the decoder feature information tensor through the autoencoder AE , the normal image The pre-trained teacher network obtains the normal feature information tensor , with channel Calculate for axis and The vector-to-cosine similarity loss is used as the difference between normal feature information and abnormal feature information. By reducing the similarity loss, the autoencoder AE has the ability to reconstruct the normal feature information of the abnormal image. The similarity loss The calculation is as follows: in, is the vector cosine similarity loss; For abnormal images; S3: Build an embedding module and define a set of memories. Use the input features to perform similarity addressing on the memory group and obtain the reconstruction features. S4: Build the MFF module to guide the reconstruction of decoder features using encoder multi-scale features. This includes the following: The input of the MFF module in S4 is the feature output of different blocks of the autoencoder AE ,Will Perform pooling processing, complete the alignment of feature dimensions after processing, and splice the pooled As a feature information block; the feature information block contains semantic information of multiple scales, and the block feature is fused in the channel direction through 1×1; the channel of the feature information block is adjusted and adjusted to , the corresponding channels and feature sizes, we get ; The MFF module also introduces a difference information suppression function; it measures multi-scale feature information based on the cosine formula. And the feature information tensor of the teacher network The differences between: Using the activation function Generate a threshold and use it to suppress the transmission of abnormal information: The multi-scale fusion information obtained to suppress abnormal information will directly act on the corresponding feature layer of the decoder of the autoencoder AE; S5: Build a lightweight segmentation network to complete the segmentation and positioning of industrial sample defects.

2. The method for industrial defect detection based on feature information reconstruction of multi-dimensional feature fusion according to claim 1 is characterized in that: In the cropped autoencoder AE of S1, both the encoder and decoder consist of the first three blocks of ResNet18 that have not been pre-trained. The encoder is represented as , decoder is represented as ; Through an embedded memory module Make a link; Image feature information corresponds to , Represents the original data Projection to embedding space; ;in, 、 、 Respectively represent The number of channels, height, and width of the layer activation tensor.

3. The method for industrial defect detection based on feature information reconstruction of multi-dimensional feature fusion according to claim 1 is characterized in that: The input of the embedding module in S3 is , define a set of memory inside the embedded module , used to reconstruct features , use the input features to perform similarity addressing on the memory group: in, It is a subset of the memory group; is the amount of memory; is the similarity set between input features and memory group features; Represents the similarity between input features and memory group features; Similarity Calculated by cosine distance: in, is the input feature A subset of is the transpose of the vector; By using the similarity, the memory group is reorganized with different size ratios of features, and finally a processing process is obtained with query features as input and reorganized features as output. The output is defined as ; Memory Group It is obtained through the first round of model distillation training; during the first round of model distillation training, the model of the autoencoder AE inputs a normal sample image , the data gets normal feature information through Encoder , the memory group is updated with model parameters, and the normal feature information dimension reduction data is saved as memory data; the memory group is addressed by similarity to obtain the reconstructed features , fitting and , and finally get the memory group of the embedded module .

4. The method for industrial defect detection based on feature information reconstruction of multi-dimensional feature fusion according to claim 1 is characterized in that: The segmentation network in S5 takes the abnormal image as the feature information of the teacher network as input , and the feature information of the Decoder of the AE autoencoder that takes the abnormal image as input , taking the difference in feature mapping between the feature information as the input of the segmentation network; the segmentation network also performs dimensionality reduction processing on the data information to be segmented before segmentation; In the segmentation network, the deeper information of the high-level semantic features is gradually passed to the shallow features through the IFC module. The IFC module transmits the output from the segmentation network Encoder As input to the IFC module, The output of each layer is convolved and then passed through a pooling layer. A CA attention processing block is introduced for feature blocks of different scales. The front and back dimensions and channels of the feature blocks remain unchanged after passing through the CA attention module. After the CA Block, the number of channels is adjusted again by convolution. ; The output of the IFC module is obtained through the following formula: Then, according to the basic structure of U-Lite, the processed multi-scale feature information is fused with the corresponding Decoder feature information block to obtain the fused feature information : Through the above process, IFC passes the deepest feature information up layer by layer. Each shallow feature contains the feature information of the deeper layer from the sub-layer. Finally, the corresponding IFC output multi-scale feature information is passed to the U-Lite Decoder module to complete the segmentation function. The segmentation module outputs a segmentation value of a specified size. .

5. The method for industrial defect detection based on feature information reconstruction of multi-dimensional feature fusion according to claim 4 is characterized in that: The segmentation network optimizes segmentation training using focal loss and L1 loss; Will and Linear sampling is performed to the same size, where , Represents the xy coordinates of the image: in, is a hyperparameter; is the linear sampling result; is the focal loss; is L1 loss; The segmentation network is trained using the two loss functions mentioned above: focal loss and L1 loss. The segmentation network model ultimately outputs a mask that displays the classification of the detected sample pixels.

Images