An unsupervised defect classification and segmentation method, system, and storage medium based on a two-branch flow model.

The unsupervised defect classification and segmentation method based on the bi-branch flow model solves the problems of high annotation cost, large storage space and poor generalization in unsupervised deep learning, and achieves high-precision defect detection and localization, which is suitable for defect detection of industrial products.

CN118279643BActive Publication Date: 2026-11-10HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
View PDF 22 Cites 0 Cited by

Patent Information

Application Number
CN202410241534.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-04
Publication Date
2026-11-10
Estimated Expiration
2044-03-04

AI Technical Summary

Technical Problem

Existing defect detection methods based on unsupervised deep learning suffer from problems such as high annotation costs, large storage space requirements, poor generalization, and difficulty in detecting small defects and accurately describing defect details.

Method used

An unsupervised defect classification and segmentation method based on a dual-branch flow model is adopted. The image is input into the dual-branch flow model through a feature extractor. The segmentation flow model and the classification flow model perform defect segmentation and classification respectively. Combined with the Real-NVP architecture and gated feature fusion module, a variance stability loss function is designed for training to achieve high-precision defect detection and localization.

Benefits of technology

High-precision defect classification and segmentation were achieved using only normal samples for training. It exhibits excellent generalization ability, reduces computational load and parameter count, and improves detection accuracy and stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118279643B_ABST
    Figure CN118279643B_ABST
Patent Text Reader

Abstract

This invention provides an unsupervised defect classification and segmentation method, system, and storage medium based on a dual-branch flow model. The unsupervised defect classification and segmentation method includes: Step 1: Adjusting the image and inputting it into a feature extractor to obtain a feature map x. 0 x 1 x 2 x 3 Step 2: Transfer the feature map x 0 x 1 x 2 x 3 Input a two-branch flow model, feature map x 0 x 1 Input segmentation flow model, feature map x 1 x 2 x 3 Input classification flow model, feature map x 0 x 1 A larger-scale feature map z is obtained through the coupling layer of the segmented flow model. 0 z 1 The segmented flow model utilizes a large-scale feature map z. 0 z 1 Complete the defect segmentation task, feature map x 1 x 2 x 3 The feature map z with a smaller scale is obtained by multi-scale coupling layer mapping of the classification flow model. 2 z 3 z 4 The classification flow model utilizes a small-scale feature map z. 1 z 2 z 3 This invention accomplishes the task of defect classification, achieving the goal of balancing defect detection and localization capabilities. The beneficial effects of this invention are: the unsupervised defect classification and segmentation method requires no annotation costs, has high accuracy, and good generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an unsupervised defect classification and segmentation method, system, and storage medium based on a two-branch flow model. Background Technology

[0002] Existing automated defect detection technologies are mainly divided into three categories: traditional machine vision-based methods, supervised deep learning-based methods, and unsupervised deep learning-based methods. Traditional machine vision-based methods primarily identify and segment defects in images through image processing techniques such as edge processing, thresholding, and morphological operations, or by manually designing feature extractors such as SIFT, ORB, or Canny edge extractors to extract image features for classification. Patent CN113245241A proposes a machine vision-based method for detecting defects in glass. Patent CN114486916A proposes a machine vision-based method for detecting defects in mobile phone glass covers. Patent CN114813748A discloses a machine vision-based method and system for detecting defects on steel surfaces, employing more effective image filtering and denoising techniques to make images clearer, and optimizing the image feature selection and defect classification processes to improve the method's detection accuracy. Patent CN114299020A discloses a machine vision-based method for detecting defects in scooter displays. It employs image processing to extract the display area and segment individual characters, ultimately combining the geometric and grayscale features of the font image for defect detection. Patent CN115228770A proposes a machine vision-based system for detecting and sorting surface defects in aluminum profiles. This system uses machine vision to achieve high-precision identification of surface defects in aluminum profiles and uses a robotic arm for grasping and classification. Patent CN115201212A uses the LOG edge detection algorithm for image processing. It achieves defect segmentation in plastic products by converting defective areas into normal areas and comparing them pixel-by-pixel with the original image. Patent CN114894808A discloses a machine vision-based device and method for detecting defects in heat pipe openings. Through image preprocessing, feature extraction, and defect detection, it automates the detection of burrs and missing roundness defects at the pipe opening, significantly improving product process efficiency and detection accuracy.

[0003] In supervised deep learning-based methods, defects are classified, detected, and segmented using classification, detection, and segmentation models constructed from neural networks. Based on the level of detail in the annotation information used to train the model, these methods can be further categorized into fully supervised models and weakly supervised models. Patent CN116542929A discloses an AOI defect detection method based on DenseNet. Patent CN116051502A discloses a large-tow carbon fiber defect detection method based on a combination of image processing and deep learning. Patent CN114998202A discloses a semi-supervised defect detection method based on student convolutional neural networks and teacher convolutional neural networks. Patent CN117110305A discloses a battery casing surface defect detection method based on deep learning. Patent CN117218106A discloses a chip blue film dirt and damage defect detection method based on YOLOv5. Patent CN111652883A discloses a glass surface defect detection method based on a deep learning classification model. Patent CN111833324A discloses a deep learning-based method for detecting defects in fiber optic ferrules, which improves defect detection accuracy through dual detection using deep learning algorithms and traditional feature detection algorithms. Patent CN114494142A proposes a deep learning-based method for detecting defects in the frame of a mobile terminal, which locates and predicts the category of defects by inputting the extracted target frame image into a trained target detection model.

[0004] To overcome the reliance on expensive annotations during training of supervised deep learning methods, unsupervised defect detection methods have been proposed. Patent CN117237429A discloses an unsupervised texture-based defect detection method based on salient object detection. This method extracts the defective portion from a defective image in a texture dataset as the foreground image and uses a normal image of the corresponding category as the background image. The foreground and background images are then combined to generate a new defective image for training. Patent CN117237309A discloses an unsupervised chip defect detection method that detects defects by comparing the differences before and after reconstructing the chip image using a reconstruction network. Patent CN116912173A discloses a product appearance defect detection method based on unsupervised anomaly detection. It extracts features from normal sample images using a pre-trained neural network to construct a source feature space. This source feature space is then compressed and stored via an adaptive repository. Next, a feature mapping network remaps the sample features from the source feature space to a new target feature space. Finally, in the high-dimensional feature space, the sample features in the target feature space are compared with the compressed features in the repository to generate an anomaly score map of possible abnormal regions in the test sample image, thus achieving defect detection and localization. Patent CN115375604A proposes an unsupervised defect detection method based on a quantized autoencoder. This method localizes defects by converting the input image into a normal sample and comparing it pixel-by-pixel with the original input. Furthermore, the combination of MSE and SSIM metrics enhances the defect localization capability. Patent CN115187525A discloses an unsupervised image defect detection method based on knowledge distillation. This method trains a teacher network using publicly available datasets from other fields to guide a student network in feature acquisition on defect data. Finally, it determines the presence of defects in an image by analyzing the relationship between a distance matrix and pixel-level and image-level defect thresholds. This method uses only normal samples during training, significantly reducing the training cost of the model. Patent CN111340791A proposes an unsupervised defect detection method for photovoltaic modules based on an improved GAN algorithm. It trains a generative adversarial network using the SSIM-GAN algorithm model, achieving fast and accurate detection of unknown defects even with a small number of defective images, and exhibits advantages such as environmental adaptability and robustness. Meanwhile, a number of mature deep learning-based defect detection software programs have been launched in the industry, such as VIDI software from Switzerland, SuaKIT deep learning defect detection software from South Korea, Halcon software from Germany, and commercial deep learning surface defect detection products from companies like Baidu, Alibaba, and Tencent in China, which have been successfully applied in industries such as 3C components, solar panels, and new energy batteries.

[0005] Problems and shortcomings of existing technology:

[0006] Traditional machine vision-based defect detection methods lack versatility, often requiring customized solutions, and their speed and accuracy are inferior to deep learning-based methods. Machine vision algorithms based on edge extraction or feature extraction consume significant time in image processing, and their detection accuracy is relatively low. While deep learning methods have greatly improved detection speed and accuracy, supervised deep learning methods often require collecting a large number of defect samples and performing precise pixel-level annotations to ensure the model's detection accuracy, resulting in high annotation costs. For example, patents CN116542929A, CN116051502A, CN117218106A, CN111833324A, and CN114494142A all require extensive and expensive bounding box annotations or pixel-level annotations to achieve good defect detection performance. Patents CN117237429A, CN117237309A, CN116912173A, CN115375604A, CN115187525A, and CN111340791A employ unsupervised learning methods to alleviate the problem of high annotation costs. However, methods like CN117237429A, which generate pseudo-defect images to train the model, limit the types of defects the model can detect, making it difficult to detect all types of defects. Methods like CN116912173A, which construct feature memory databases, consume significant storage space, placing high demands on hardware storage. For knowledge distillation-based methods like CN115187525A, the model's defect detection ability is greatly influenced by the selected pre-trained teacher network, easily resulting in the model performing well for some types of defects but poorly for others, indicating insufficient generalization. Algorithms based on the reconstruction idea, such as those in patents CN117237309A, CN115375604A, and CN111340791A, suffer from the problem of easily overlooking minor defects and failing to reconstruct defect details well. Therefore, there is currently a lack of an unsupervised deep learning algorithm that possesses strong generalization capabilities and excellent defect classification and segmentation abilities. Summary of the Invention

[0007] Existing supervised defect detection methods often require a large amount of expensive pixel-level annotations and samples during training, which greatly limits the large-scale application of deep learning methods in the field of defect detection. Unsupervised methods do not require defect samples and their annotation information, only normal samples as training datasets, which can fundamentally solve the problem of high annotation costs. Existing unsupervised methods also have some problems, such as excessive storage space consumption, ignoring small defects, difficulty in accurately describing defect details, and poor generalization. To address these problems, this invention proposes an unsupervised defect classification and segmentation method based on a bi-branch flow model, which can achieve high-precision and stable image-level classification and pixel-level segmentation of defects using only normal samples for training, and has excellent generalization. This invention provides an unsupervised defect classification and segmentation method based on a bi-branch flow model, including the following steps:

[0008] Step 1: After adjusting the image, input it into the feature extractor to obtain the feature map x. 0 x 1 x 2 x 3 ;

[0009] Step 2: Transfer the feature map x obtained in Step 1 to... 0 x 1 x 2 x 3 Input a two-branch flow model, where the feature map x 0 x 1 Input segmentation flow model, feature map x 1 x 2 x 3 Input classification flow model, feature map x 0 x 1 A larger-scale feature map z is obtained through the coupling layer of the segmented flow model. 0 z 1 The segmented flow model utilizes a large-scale feature map z. 0 z 1 Complete the defect segmentation task, feature map x 1 x 2 x 3 The feature map z with a smaller scale is obtained by multi-scale coupling layer mapping of the classification flow model. 2 z 3 z 4 The classification flow model utilizes a small-scale feature map z. 2 z 3 z 4 The task of defect classification was completed, achieving the goal of balancing the algorithm's defect detection and localization capabilities.

[0010] As a further improvement of the present invention, in step one, the input image is first adjusted to 1 / 2 or 1 / 4 of the original image, and the original image is denoted as I. 0 It will be adjusted to the original Figure 1 The half-size image and the quarter-size image are denoted as I. 1 and I 2 , will I 0 I 1 I 2 Input to the feature extractor, I 0 The feature map x is obtained after feature extraction. 0 x 1 I 1 The feature map x is obtained after feature extraction. 2 I 2 The feature map x is obtained after feature extraction. 3 As a further improvement of the present invention, in step two, the segmented flow model coupling layer is implemented using a Real-NVP architecture, and the specific steps are as follows:

[0011] The input sample feature map x is first shuffled through channels to obtain the feature map x. i Then, according to the channel dimension, it is evenly divided into feature maps x i1 Feature map x i2 Next, the feature map x i1 The feature map with 2C channels is obtained after processing by the convolutional neural network c1, and then it is divided into features according to the channel dimension. Figure 1 s1, after exponentiation, is combined with x i2 The feature map z is obtained by multiplying the elements together and then adding it to t1. o2 ;

[0012] z o2 The convolutional neural network c2 processes the data to obtain a feature map with 2C channels. This feature map is then divided into two equally spaced feature maps t2 and s2 according to the channel dimension. s2 is then subjected to an exponential operation and multiplied by x. i1 The feature map z is obtained by multiplying the elements together and then adding it to t2. o1 ;

[0013] Finally, z o1 With z o2 The feature map z is obtained by concatenating the data along the channel dimension; x represents x. 0 or x 1 x 0 and x 1 These are two feature maps extracted by the backbone network; corresponding to x 0 and x 1 z represents z 0 or z 2As a further improvement of the present invention, the calculation process of the affine transformation of the coupling layer of the segmented flow model can be described by formula (1):

[0014]

[0015] Here, ⊙ represents element-wise multiplication. Functions c1 and c2 are composed of convolutional neural networks, which preserve the spatial location information of the feature map. For training stability, the parameters of the convolutional neural network are initialized to 0, i.e., z = x initially. During inference, the l2 norm at position (i, j) of the feature map z is calculated. The average value across the channel dimension is used as the predicted anomaly score map to complete the defect localization task.

[0016] As a further improvement of the present invention, in step two, the multi-scale coupling layer architecture of the classification flow model is implemented using the Real-NVP architecture, and the specific steps are as follows:

[0017] Step 1: For the three input feature maps x at different scales 1 x 2 x 3 First, the coupling order is changed to shuffle the channels in order to fully mix the information of each dimension, so that the features of each dimension can be processed by the mapping of the flow model.

[0018] Step 2: The three feature maps used in Step 1 to change the coupling order and shuffle the channels are divided into upper and lower parts according to the channel dimension, resulting in a total of six feature maps. The upper part consists of three feature maps x i 1 1. x i 2 1. x i 3 1. Simultaneously, after being mapped by the first gated feature fusion module GMFF, the following is obtained: and Where k represents the three feature map numbers. It will be multiplied element-wise with the feature map of the corresponding scale in the lower half, and the result will then be multiplied by... Element-wise addition yields The three feature maps z in the lower half o 1 2, z o 2 2, z o 3 2. Simultaneously, after being mapped by the second gated feature fusion module GMFF, the following is obtained: and Where k represents the three feature map numbers. It will be multiplied element-wise with the feature map of the corresponding scale in the upper part, and the result will then be multiplied by... Element-wise addition yields Finally obtained and There are six feature maps in total. By concatenating the corresponding scale feature maps along the channel dimension, we can obtain the mapped feature map z. 2 z 3 z 4 .

[0019] As a further improvement of the present invention, the gated feature fusion module GMFF operates as follows:

[0020] Step S1: Input feature map The number of channels is doubled by using a 3×3 convolution.

[0021] Step S2: Feature Map After leaky ReLU, gated units, and bilinear interpolation downsampling, and with the feature map Element-wise addition, simultaneously, feature maps After leaky ReLU, gating unit and bilinear interpolation upsampling, and with Element-wise addition, for feature maps It undergoes leaky ReLU, gated unit, and bilinear interpolation for upsampling and downsampling, and then is compared with the feature map. Feature map By adding elements together, through this simple and effective feature fusion process, the feature map at each scale can obtain the weighted information of the feature maps at the other two scales.

[0022] Step S3: After the fusion process described in step S2 It undergoes another 3×3 convolution to eliminate the overlap effect that may be introduced by the upsampling process, followed by another gating unit operation;

[0023] Step S4: According to the structural requirements of the flow model, the outputs are as follows: and These two tensors serve as the output of the gated feature fusion module.

[0024] As a further improvement of the present invention, in the operation step of the gated feature fusion module, the calculation formula of the gated unit is as shown in Formula 3.

[0025]

[0026] Where γ is a user-defined learnable weight parameter, initialized to 0, the sigmoid function is used to limit the value range of the weight parameter to between 0 and 1, e represents the natural constant, G(x) represents the calculation function of the gating unit, and x represents the sample feature map;

[0027] During inference, the l2 norm of the feature map z is calculated. The average value is then taken across the channel dimension to form the predicted image anomaly score, thus completing the defect detection task.

[0028] As a further improvement of the present invention, the two-branch flow model employs a maximum likelihood estimation method to construct a loss function to guide the training of the two-branch flow model during training, specifically as follows:

[0029] First, we obtain z∈X that follows a pre-defined known distribution. Z (z) Calculated using formula (5):

[0030]

[0031] For the sake of simplicity, we perform logarithmic operations on both sides of Equation 5 to obtain Equation 6:

[0032]

[0033] Where logp x (x) represents the logarithm of the probability density function with respect to x, logp Z (z) denotes the logarithm of the probability density function with respect to z. This represents the logarithm of the absolute value of the Jacobian determinant. The result of this calculation is the sum of all values ​​of 's', where 's' refers to the intermediate calculation results s1 and s2 in the coupling layers of the segmented flow model branch and the multi-scale coupling layers of the classification flow model branch. Then, variance stability loss is designed to address the cosine similarity loss term. The loss function of the two-branch flow model is shown in Equation 9, which consists of two parts: the segmentation flow model loss and the classification flow model loss.

[0034]

[0035] The feature map z follows a standard Gaussian distribution, var(z) class ) is used to measure the feature map z class The variance stability loss is specifically calculated by calculating the variance between the feature maps of each positive sample in the Batch dimension.

[0036] Substituting formula (6) into formula (9), we get formula (10):

[0037]

[0038] Where λ is the balance coefficient, set to 5, and μ B Represents z class Take the average value along the Batchsize dimension. This indicates summation along the Batchsize dimension. To ensure stability during training, the L2 norm of the gradient is restricted to 1.

[0039] This invention also discloses an unsupervised defect classification and segmentation system based on a dual-branch flow model, comprising: a memory, a processor, and a computer program stored in the memory, wherein the computer program is configured to implement the steps of the unsupervised defect classification and segmentation method of this invention when invoked by the processor. This invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the unsupervised defect classification and segmentation method of this invention when invoked by a processor.

[0040] The beneficial effects of this invention are: 1. No annotation cost – Addressing the problems of high annotation costs and difficulty in obtaining pixel-level defect annotations, this invention proposes an unsupervised defect detection and localization model based on a dual-branch flow model. This model can simultaneously achieve high-precision classification and segmentation of defects even when trained only on normal samples. 2. High accuracy – Addressing the problem of existing unsupervised algorithms struggling to balance defect classification and segmentation accuracy, and the issue of misclassifying normal samples as defect samples, this invention designs a dual-branch flow model architecture that can simultaneously complete defect classification and segmentation tasks, while maintaining high accuracy in both areas. This invention also designs a gated feature fusion module to further improve the model's defect classification capability and reduce the number of parameters and computational load. To address the misclassification problem, this invention designs a variance stability loss function to further constrain the consistency of positive sample features during training, achieving end-to-end training and effectively mitigating the occurrence of misclassification. 3. Good generalization ability – Experiments were conducted on three datasets: MVTec AD, MVTec loco AD, and BTAD. The AUC index of the proposed two-branch flow model was compared with that of the classic unsupervised algorithm, the latest unsupervised algorithm, and the proposed two-branch flow model. Visual images were provided to demonstrate that the proposed two-branch flow model has good generalization ability. Attached Figure Description

[0041] Figure 1 This is a diagram of the training framework for the dual-branch flow model of this invention;

[0042] Figure 2 This is a diagram of the inference framework for the dual-branch flow model of this invention;

[0043] Figure 3 This is an architecture diagram of the dual-branch flow model of the present invention;

[0044] Figure 4 This is a diagram of the coupling layer architecture of the segmented flow model of this invention;

[0045] Figure 5This is a diagram of the multi-scale coupling layer architecture of the classification flow model of this invention;

[0046] Figure 6 This is an architecture diagram of the gating feature fusion module of the present invention;

[0047] Figure 7 This is a flowchart of the training and testing process for the flow model of this invention;

[0048] Figure 8 This is a schematic diagram illustrating the qualitative evaluation of the dual-branch flow model on the MVTec AD dataset of this invention;

[0049] Figure 9 This is a schematic diagram of the qualitative evaluation of the dual-branch flow model on the MVTec LOCO AD of this invention;

[0050] Figure 10 This is a schematic diagram of the qualitative evaluation of the present invention on BTAD;

[0051] Figure 11 This is a qualitative evaluation comparison between the dual-branch flow model of this invention and CSFlow in the bottle category;

[0052] Figure 12 This is an anomaly score histogram of the dual-branch flow model and CSFlow prediction of bottle categories in this invention; Detailed Implementation

[0053] The training and inference framework of the DualFlow model of this invention are as follows: Figure 1 , Figure 2 As shown.

[0054] This invention discloses an unsupervised defect classification and segmentation method based on a bi-branch flow model, which can simultaneously complete defect classification and defect segmentation tasks. The specific steps are as follows:

[0055] Step 1: First, adjust the input image to 1 / 2 or 1 / 4 of the original image, denoted as I. 0 I 1 I 2 I 0 The feature map x is obtained after feature extraction. 0 x 1 I 1 I 2 The feature maps x are obtained by the feature extractor respectively. 2 x 3 The probability density function of x∈X is denoted as p. X (x).

[0056] Step 2: Transfer the feature map x obtained in Step 1 0 x 1 x 2x 3 Input a two-branch flow model, where the feature map x 0 x 1 As input to the segmentation flow model branch, feature map x 1 x 2 x 3 As the input to the classification flow model branch, z is obtained after mapping processing through the coupling layer and the multi-scale coupling layer. 0 z 1 z 2 z 3 z 4 The probability density function of z∈Z is denoted as p. Z (z), feature map x 0 x 1 Input segmentation flow model, feature map x 1 x 2 x 3 Input classification flow model, feature map x 0 x 1 A larger-scale feature map z is obtained through the coupling layer of the segmented flow model. 0 z 1 The segmented flow model utilizes a large-scale feature map z. 0 z 1 Complete the defect segmentation task, feature map x 1 x 2 x 3 The feature map z with a smaller scale is obtained by multi-scale coupling layer mapping of the classification flow model. 2 z 3 z 4 The probability density function of z∈Z is denoted as p. Z (z), where Z represents a dataset, and the classification flow model utilizes a smaller-scale feature map z. 1 z 2 z 3 The task of defect classification was completed, achieving the goal of balancing the algorithm's defect detection and localization capabilities.

[0057] During training, maximum likelihood estimation is used to estimate the density of positive sample features, i.e., by calculating p X (x) is used to determine the parameters of the flow model. Since the distribution of x is unknown, p cannot be obtained directly. X The formula for calculating (x). Therefore, based on the bidirectional mapping characteristics of the flow model, it is possible to calculate p by pre-setting a known value. Z (z) is used for calculation; during inference, the flow model maps the normal part of the predicted sample features to a pre-defined standard Gaussian distribution, while the defective part is mapped to a distribution outside the pre-defined distribution.

[0058] Meanwhile, to address the misjudgment problem encountered by existing unsupervised algorithms, and to improve the algorithm's defect detection capability while reducing computational cost, this invention designs a gated feature fusion module applied in a multi-scale coupling layer. Furthermore, from the perspective of feature stability after flow model mapping, this invention designs a variance stability loss function to achieve end-to-end training and inference of the entire model. The following sections describe the segmentation flow model branch architecture and the classification flow model branch architecture in the dual-branch flow model.

[0059] 1.1 Segmented Stream Model Architecture

[0060] In step two, the split flow model coupling layer is implemented using the flow model architecture Real-NVP. The specific structure of the coupling layer is as follows: Figure 4 As shown. The input sample feature map x is first subjected to a channel shuffling operation to obtain the feature map x. i Then, according to the channel dimension, it is evenly divided into feature maps x i1 Feature map x i2 Then x i1 The feature map with 2C channels is obtained after processing by convolutional neural network c1. It is then divided into t1 and s1 according to the channel dimension. s1 is then subjected to exponential operation and then multiplied by x. i2 The z product is obtained by multiplying the elements together and then adding it to t1. o2 Calculate z o1 Process and calculation z o2 Consistent, z o2 The convolutional neural network c2 processes the data to obtain a feature map with 2C channels. This feature map is then divided into two equally spaced feature maps t2 and s2 according to the channel dimension. s2 is then subjected to an exponential operation and multiplied by x. i1 The feature map z is obtained by multiplying the elements together and then adding it to t2. o1 Finally, z o1 With z o2 The feature map z is obtained by concatenating the data along the channel dimension; x represents x. 0 or x 1 x 0 and x 1 These are two feature maps extracted by the backbone network; corresponding to x 0 and x 1 z represents z 0 or z 1 The calculation process for the above affine transformation can be described by Equation 1:

[0061]

[0062] Here, ⊙ represents element-wise multiplication. Functions c1 and c2 are composed of convolutional neural networks, preserving the spatial location information of the feature map. For training stability, the parameters of the convolutional neural network are initialized to 0, i.e., z = x initially. During inference, the l2 norm at position (i, j) of the feature map z is calculated. The average value across the channel dimension is used as the predicted anomaly score map to complete the defect localization task.

[0063] 1.2 Classification Flow Model Architecture

[0064] In step two, the multi-scale coupling layer architecture of the classification flow model also adopts the RealNVP architecture, as shown in the following structure: Figure 5 As shown. The specific steps include:

[0065] Step 1: For the three input feature maps x at different scales 1 x 2 x 3 First, the coupling order is changed to shuffle the channels in order to fully mix the information of each dimension, so that the features of each dimension can be processed by the mapping of the flow model.

[0066] Step Two: Next, the feature maps at three scales from the channel shuffling operation in Step One, which changed the coupling order, are divided into two parts according to the channel dimension, resulting in a total of six feature maps. The three feature maps in the upper part are x i 1 1. x i 2 1. x i 3 1. Simultaneously, after being mapped by the first gated feature fusion module c1, the following is obtained: and Where k represents the three feature map numbers. It will be multiplied element-wise with the feature map of the corresponding scale in the lower half, and the result will then be multiplied by... Element-wise addition yields The operations performed on the three feature maps in the lower half are the same as those performed on the three feature maps in the upper half. The three feature maps in the lower half, x... i 1 2, x i 2 2, x i 3 2 is obtained by mapping with the second gated feature fusion module c2. and Where k represents the three feature map numbers. It will be multiplied element-wise with the feature map of the corresponding scale in the upper part, and the result will then be multiplied by... Element-wise addition yields Finally obtained and There are six feature maps in total. By concatenating the corresponding scale feature maps along the channel dimension, we can obtain the mapped feature map z. 2 z 3 z4 The structure described above for feature mapping mixing is a coupling layer. The above calculation process can also be expressed by Equation 2:

[0067]

[0068] Where k represents the feature map numbers of three different scales, and functions c1 and c2 are the gated feature fusion modules (GMFF) designed in this invention.

[0069] 2. Gating Feature Fusion Module

[0070] The specific structure of the gated feature fusion module designed in this invention is as follows: Figure 6 As shown. The computational flow of the gating feature fusion module is as follows:

[0071] Step S1: First, input the feature map. The number of channels is doubled by using a 3×3 convolution.

[0072] Step S2: Then, feature map After leaky ReLU (a commonly used activation function in deep learning), gated units, and bilinear interpolation downsampling, and compared with the feature map... Add elements together, and at the same time... After leaky ReLU, gating unit and bilinear interpolation upsampling, and with Element-wise addition. For feature maps. It undergoes leaky ReLU, gated unit, and bilinear interpolation for upsampling and downsampling, and is then compared with the feature map. Feature map Element-wise addition. Through this simple yet effective feature fusion process, the feature map at each scale can obtain weighted information from the feature maps at the other two scales.

[0073] Step S3: Next, after the fusion process in step S2... The sample undergoes another 3×3 convolution to eliminate any overlap effects that might be introduced during the upsampling process. Then, it is processed by a gating unit once more.

[0074] Step S4: Finally, according to the structural requirements of the flow model, the outputs are as follows: and These two tensors serve as the output of the gated feature fusion module. The gated feature fusion module is designed to effectively combine feature information from multiple scales, providing the model with richer contextual information.

[0075]

[0076] The calculation formula for the gating unit is shown in Equation 3, where γ is a user-defined learnable weight parameter initialized to 0. The sigmoid function is used to restrict the value range of the weight parameter to between 0 and 1. The gating unit operation aims to allow the model to determine the importance of feature information at each scale during training. During inference, the l2 norm of the feature map z is calculated. The average value is then taken across the channel dimension to form the predicted image anomaly score, thus completing the defect detection task. The purpose of this design is to enable the model to adapt to data features at different scales, and to allow the model to adjust the contribution of features at each scale during the learning process through a gating mechanism, so as to better adapt to changes in different scenarios.

[0077] 3. Variance stability loss function

[0078] As a generative model, the flow model employs maximum likelihood estimation to construct a loss function during training to guide the model's development. Figure 7 It can be seen that during training, the true distribution p of the data is used... data Sample x is drawn from (x) i After G -1 Mapping to obtain z i =G -1 (x i This ensures that the generator satisfies a preset prior distribution, where G represents the forward generation process from z to x; G-1 represents the reverse generation process from x to z. During testing, the test sample z is input. i x is obtained through the forward generation computation process G of the generator. i =G(z) i This refers to the new image generated through the flow model. Its loss function is Equation 4:

[0079]

[0080] Formula 4 expresses the desired loss during training. gen The smaller the better, that is, the more we hope The larger the better, that is, given a batch of input samples x i When the log-likelihood is larger, the better; this is the method of maximum likelihood estimation, where p G (x i ) represents the feature map x obtained after passing through generator G. i The probability density function, G -1 (x i ) represents the reverse computation process z of generator G. i =G -1 (x i ), Representation function G -1The Jacobian determinant. Similar to flow models as generative models during training, two-branch flow models aim for the flow model f to maximize the likelihood of the training sample features x during training. However, since the distribution of the sample features x is unknown, it cannot be calculated directly. Therefore, it can be calculated by considering z∈p, which follows a pre-defined known distribution. Z (z) Calculated using formula (5):

[0081]

[0082] For the sake of simplicity, we perform logarithmic operations on both sides of formula (5) to obtain formula (6):

[0083]

[0084] in This represents the logarithm of the absolute value of the Jacobian determinant. The result of this calculation is the sum of all values ​​of 's', where 's' refers to the intermediate calculation results s1 and s2 in the coupling layers of the segmented flow model branch and the multi-scale coupling layers of the classification flow model branch.

[0085] Considering the misjudgment problem of existing flow model-based algorithms, the reason is that during training, the flow model fails to map the positive sample features that follow an unknown complex distribution to a standard Gaussian distribution in a relatively concentrated and compact manner. That is, the constraint effect of the loss function is not strong enough during training, making it difficult to distinguish between positive samples and defective samples during inference. Therefore, this invention considers adding a loss term to the loss function to measure the consistency of positive sample features. Unlike supervised methods, which can construct a loss function based on the difference between the label and the model prediction value, using the label value as a benchmark, the model prediction value gradually approaches the label value during training. However, the loss function based on the flow model does not have an accurate label value, so it cannot construct a loss term based on the difference between the label and the model prediction value. Therefore, it is necessary to establish an additional loss term from the positive sample features themselves after being mapped by the flow model. Since maximum likelihood estimation considers the magnitude of positive sample features, and the loss function is established from the perspective of numerical magnitude, a natural idea is to establish a loss term from the perspective of positive sample features and apply cosine similarity loss to further constrain the consistency of positive sample features. The cosine similarity calculation formula is formula (7):

[0086]

[0087] The numerator of formula (7) represents the inner product between the label value vector and the model prediction vector, i.e., the positive sample feature vector, and the denominator represents the product of the magnitudes of the label value vector and the model prediction vector. This allows us to consider the influence of the length of the two vectors to be ignored to a certain extent, thus obtaining the angular difference between the two vectors. To apply cosine similarity to the loss function of an unsupervised algorithm based on a flow model, another problem needs to be addressed: how to determine the value of the label vector z′, since unsupervised algorithms based on flow models are unlabeled. A feasible approach is to use the average value of the model prediction vector along the batch dimension. Using the label vector z′ in the original formula, we can obtain formula (8):

[0088]

[0089] Equation (8) shows the average value of the model prediction vector in the Batch dimension. While the label vector z′ in the original formula is feasible, it was found to be ineffective in actual training. This is because, during actual training, the cosine similarity loss gradually brings the model's predicted values ​​closer to the average of the model's predicted values. However, the model's predicted values ​​do not conform to a Gaussian distribution. Using cosine similarity loss to guide the model's predicted values ​​to gradually approach the average of the model's predicted values, which does not conform to a Gaussian distribution, is essentially training inaccurate predictions in the direction of inaccurate predictions, thus failing to achieve good training results.

[0090] Based on the analysis of the cosine similarity loss term above, this invention designs a variance stability loss to address this problem. The loss function of the two-branch flow model is shown in formula (9), consisting of two parts: the segmentation flow model loss and the classification flow model loss.

[0091]

[0092] The feature map z follows a standard Gaussian distribution, var(z) class ) is used to measure the feature map z class The variance stability loss is specifically calculated by calculating the variance between each positive sample feature map in the Batch dimension. The intention is to minimize the variance and increase the similarity between positive sample feature maps during training, thereby improving the consistency between positive sample features. Substituting formula (6) into (9), we obtain formula 10:

[0093]

[0094] Where λ is the balance coefficient, set to 5, and μ B Represents z class Take the average value along the Batchsize dimension. This represents summation along the Batchsize dimension. To ensure stability during training, the L2 norm of the gradient is constrained to 1.

[0095] 4. Experimental verification using an industrial product defect dataset

[0096] The dual-branch flow model proposed in this invention will be experimentally validated on three mainstream industrial product defect datasets, including the MVTec AD dataset, the MVTec LOCO AD dataset, and the BTAD dataset.

[0097] The MVTec AD dataset contains 15 categories, with 3629 images used for training and validation, and 1725 images used for testing, as shown in Table 1. The training set only includes images without defects. The test set includes two types of images: one containing images of various defect types, and the other containing defect-free images. Five texture categories include carpets, mesh, leather, tiles, and wood; ten product categories include fixed-shape products such as lenses, metal nuts, toothbrushes, capsules, and pills, and variable-shape products such as cables and hazelnuts. The test images cover a variety of anomalies, including surface defects (e.g., scratches and dents), structural defects in object parts, and defects due to missing parts of the object. Pixel-level annotations are provided for the defect images in the test set, totaling 1900 defect regions. The MVTec LOCO AD consists of five object categories from industrial inspection scenarios. A total of 1772 images are provided for training, 304 for validation, and 1568 for testing, as shown in Table 2. The BTAD dataset contains RGB images of three industrial products. Product 1 has a resolution of 1600×1600 pixels, Product 2 has a resolution of 600×600 pixels, and Product 3 has a resolution of 800×600 pixels. There are 400, 1000, and 399 training images for Products 1, 2, and 3, respectively. For each outlier image, a pixel-level label is provided.

[0098] Table 1. Overview of MVTec AD Dataset Statistics

[0099]

[0100] Table 2. Overview of statistics for the MVTec LOCO AD dataset

[0101]

[0102] The experimental parameters for the proposed dual-branch flow model are set as follows: input images at three pixel scales: 768×768, 384×384, and 192×192 pixels. Feature maps of size 48×48 extracted from layer 18 and 24×24 extracted from layer 39 of EfficientNet-B7 are used as inputs to the segmentation flow model branch, while feature maps of sizes 24×24, 12×12, and 6×6 extracted from layer 39 are used as inputs to the classification flow model branch. This allows feature maps containing more detailed information to be used for the localization task, and feature maps containing more high-level semantic information to be used for the detection task. The feature extractor is pre-trained on ImageNet, and its parameters are not changed during training and inference. The segmentation flow model branch of the dual-branch flow model uses two coupled layer modules, and the classification flow model branch uses four multi-scale coupled layer modules. The Adam optimizer is used, with an initial learning rate of 2e-4 and a weight decay of 1e-5. The batch size is set to 16 during training, and the training lasts for 450 epochs.

[0103] The image-level AUC experimental results of the proposed dual-branch flow model and existing algorithms on MVTec AD are shown in Table 3, and the pixel-level AUC experimental results are shown in Table 4. It can be seen that the dual-branch flow model achieves 99.33% AUC and 98.05% AUC in image-level classification and pixel-level segmentation, respectively, surpassing existing algorithms in both defect detection and localization tasks. The '-' in Tables 3 and 4 indicates that specific experimental results were not provided in this paper. In the image-level defect detection task, the dual-branch flow model achieved 100% AUC in 6 industrial product categories, over 99% AUC in 13 industrial product categories, and over 96% AUC in all industrial product categories. This demonstrates the powerful defect detection capability of the proposed dual-branch flow model. In the pixel-level defect localization task, the dual-branch flow model achieved over 98% AUC in 8 industrial product categories and over 95% AUC in all industrial product categories, demonstrating its powerful defect localization capability.

[0104] Table 3. Image-level AUC results on the MVTec AD dataset.

[0105]

[0106] Table 4. Pixel-level AUC results on the MVTec AD dataset.

[0107]

[0108] Visualization of the inference results for 15 types of industrial defective products, as shown below Figure 8 As shown, it includes linear defects, blocky defects, and irregularly shaped structural defects. Figure 8 The first row shows the defective product image, the second row shows the defect probability map predicted by the bi-branch flow model, the third row shows the label map, and the last row shows the superposition of the defective product image and the predicted defect probability map. It can be seen that the bi-branch flow model can accurately locate structural defects in the product. However, for the 'screw' product, when facing extremely small point defects, the defect area predicted by the bi-branch flow model is slightly larger than the actual defect area. This is because unsupervised algorithms locate defects on the extracted feature map, and locating extremely small defects has always been one of the challenges faced by unsupervised algorithms. Figure 8 This also fully demonstrates that the dual-branch flow model proposed in this invention has the ability to accurately detect defect samples and locate defect areas.

[0109] The experimental results of the two-branch flow model and existing algorithms on MVTEC LOCO AD are shown in Table 5, and the qualitative results are as follows: Figure 9 As shown, the dual-branch flow model achieved an accuracy of 86.9% in the defect detection task, surpassing existing algorithms. Since logical defects prioritize the detection task, and the metric for localization is not yet clearly defined, this invention only compares the defect detection metrics of various algorithms.

[0110] Table 5. Image-level AUC results on the MVTec LOCO AD dataset.

[0111]

[0112] Figure 9 There are five product categories in total, and each product contains two columns of images, which include both logical defects and structural defects. Figure 9 The first row shows the defective product image, the second row shows the defect probability map predicted by the dual-branch flow model, the third row shows the label map, and the last row is a superposition of the defective product image and the predicted defect probability map. A logical defect refers to an error in the quantity of certain parts in the product image. For example, in a pushpins product, each small compartment should contain only one pushpin; if a compartment contains two pushpins, this is a logical defect. The label for a logical defect is the pixel range of the two pushpins in that compartment, indicating that these two pushpins have a logical defect. Another example is a breakfast box product. The left side of the box should contain nuts and oats, and the right side should contain fruit. If the order of the items is reversed, a logical defect occurs. The label for this logical defect is the food placement area on both sides, indicating that the food placement on the sides of the box is incorrect. Figure 9As can be seen from the overall results diagram, the dual-branch flow model can effectively detect the corresponding locations of structural defects in the product, but it failed to achieve good results in detecting logical defects in the breakfast box and pushpins products, indicating the direction for further work in this invention.

[0113] The experimental results of the two-branch flow model and existing algorithms on BTAD are shown in Table 6, and the visualization results are shown in the figure below. Figure 10 As shown, the dual-branch flow model achieved an index of 96.7% in the defect detection task and 97.8% in the defect localization task, both exceeding existing methods and yielding highly competitive results. Figure 10 The first row shows the defective product image, the second row shows the defect probability map predicted by the dual-branch flow model, the third row shows the label image, and the last row shows the superposition of the defective product image and the predicted defect probability map. It can be seen that the dual-branch flow model method proposed in this invention can effectively detect structural defects in the three types of products.

[0114] Table 6. Image-level and pixel-level AUC results on BTAD.

[0115]

[0116] To further illustrate the effectiveness of the proposed dual-branch flow model architecture, gated feature fusion module, and variance stability loss function, ablation experiments were conducted. Using CSFLOW as the baseline algorithm, the results after implementing various optimization decisions were compared, and experimental verification was performed on MVTec AD. Table 7 shows a comparison of the experimental results for CSFLOW, CSFLOW+VARLoss, CSFLOW+GMFF, CSFLOW+VARLoss+GMFF, and DualFlow (CSFLOW+VARLoss+GMFF+DualBranch). It can be seen that the designed variance stability loss improves the model's defect detection capability but slightly reduces its defect localization capability. The designed GMFF module also improves the model's defect detection capability, but significantly reduces its defect localization capability. The dual-branch architecture design can solve the problem of the model's difficulty in simultaneously achieving detection and localization capabilities. The experimental results demonstrate that the dual-branch architecture effectively improves both the model's detection and localization capabilities. In order to find a more suitable feature extractor, this invention also compared the experimental results of using EfficientNet-B5 and EfficientNet-B7 as feature extractors, as shown in the last two rows of Table 7. The experimental results show that EfficientNet-B7 performs better.

[0117] Table 7 Results of each decision on MVTec AD

[0118]

[0119] To more intuitively demonstrate the superiority of the dual-branch flow model algorithm, the following uses the bottle category as an example to compare the defect detection and defect localization effects of CSFLOW and the dual-branch flow model. Figure 11 This display shows the defect probability maps predicted by two methods. The first row of images is the input image, the second row is the defect probability map predicted by CSFLOW, the third row is the defect probability map predicted by the two-branch flow model, and the fourth row is a superposition of the first and third row images. Figure 11 It is evident that in the defect probability map predicted by CSFLOW, the non-defect parts of the image have a higher probability, leading to misjudgment. In contrast, the defect probability map predicted by the dual-branch flow model effectively solves the misjudgment problem encountered by CSFLOW, achieving more accurate defect localization.

[0120] Figure 12 The chart shows the bar graphs of anomaly scores predicted by CSFLOW and the dual-branch flow model. It can be seen that the anomaly scores predicted by CSFLOW overlap with those of defective samples, indicating a misclassification problem. In contrast, the anomaly scores predicted by the dual-branch flow model do not overlap with those of defective samples, and the anomaly scores of normal samples cluster in a smaller anomaly score region, while the anomaly scores of defective samples cluster in a larger anomaly score region. This demonstrates that the dual-branch flow model can better distinguish between normal and defective samples and has a stronger defect detection capability.

[0121] This invention also compares the number of parameters and computational cost of the gated feature fusion module (GMFF) in the dual-branch flow model with the feature fusion module in CSFLOW, as shown in Table 8. It can be seen that the gated feature fusion module designed in this invention reduces the number of parameters by 47% and the computational cost by 45%, demonstrating the superior performance of the gated feature fusion module.

[0122] Table 8. Parameters and FLOPs of the Feature Fusion Module

[0123]

[0124] This invention discloses an unsupervised defect detection, classification, and segmentation method based on a dual-branch flow model. Its advantage lies in achieving high-precision defect classification and segmentation using only a training dataset composed of normal samples. Furthermore, it proposes a dual-branch flow model architecture, a gated feature fusion module, and a variance-stability loss function. The method demonstrates superior classification and segmentation performance on three publicly available datasets. Specific advantages are as follows:

[0125] 1. No annotation cost required – To address the issues of high annotation costs and difficulty in obtaining pixel-level annotations for defects, this invention proposes an unsupervised defect detection and localization model based on a dual-branch flow model. This model can simultaneously achieve high-precision classification and segmentation of defects even when trained on only normal samples.

[0126] 2. High Accuracy – Addressing the challenge of balancing defect classification and segmentation accuracy in existing unsupervised algorithms, and the problem of misclassifying normal samples as defective samples, this invention designs a dual-branch flow model architecture capable of simultaneously performing defect classification and segmentation tasks while maintaining high accuracy in both areas. Furthermore, this invention incorporates a gated feature fusion module to further enhance the model's defect classification capability and reduce the number of parameters and computational load. To address the misclassification issue, this invention designs a variance stability loss function to further constrain the consistency of positive sample features during training, achieving end-to-end training and effectively mitigating the occurrence of misclassification.

[0127] 3. Good generalization ability – Experiments were conducted on three datasets: MVTec AD, MVTec loco AD, and BTAD. The AUC index of the proposed two-branch flow model was compared with that of the classic unsupervised algorithm, the latest unsupervised algorithm, and the proposed two-branch flow model. Visual images were provided to demonstrate that the proposed two-branch flow model has good generalization ability.

[0128] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An unsupervised defect classification and segmentation method based on a bi-branch flow model, characterized in that... This includes the following steps: Step 1: After adjusting the image, input it into the feature extractor to obtain the feature map. , , , ; Step 2: Transfer the feature map obtained in Step 1 to... , , , Input a two-branch flow model, where the feature map , Input segmentation flow model, feature map , , Input classification flow model, feature map , Large-scale feature maps are obtained through the coupling layer of the segmented flow model. , The segmented flow model utilizes feature maps of a larger scale. , Complete the defect segmentation task, feature map , , The feature maps with smaller scales are obtained through multi-scale coupling layer mapping processing of the classification flow model. , , The classification flow model utilizes feature maps with a smaller scale. , , The task of defect classification was completed, achieving the goal of balancing the algorithm's defect detection and localization capabilities. In step two, the multi-scale coupling layer architecture of the classification flow model is implemented using the Real-NVP architecture, and the specific steps are as follows: Step 1: For the three different scale feature maps input... , , First, the coupling order is changed to shuffle the channels in order to fully mix the information of each dimension, so that the features of each dimension can be processed by the mapping of the flow model. Step 2: The three feature maps used in Step 1 to change the coupling order and shuffle the channels are divided into upper and lower parts according to the channel dimension, resulting in a total of six feature maps. The upper part consists of three feature maps. , , Simultaneously, after being mapped by the first gated feature fusion module GMFF, the following is obtained: and ,in This represents the three feature map numbers. It will be multiplied element-wise with the feature map of the corresponding scale in the lower half, and the result will then be multiplied by... Element-wise addition yields The three feature maps in the lower half , , Simultaneously, after being mapped by the second gated feature fusion module GMFF, the following is obtained: and ,in This represents the three feature map numbers. It will be multiplied element-wise with the feature map of the corresponding scale in the upper part, and the result will then be multiplied by... Element-wise addition yields Finally obtained and There are six feature maps in total. By concatenating the feature maps of the corresponding scale along the channel dimension, the mapped feature map can be obtained. , , ; The gated feature fusion module GMFF operates as follows: Step S1: Input feature map The number of channels is doubled by using a 3×3 convolution. Step S2: Feature Map After leaky ReLU, gated units, and bilinear interpolation downsampling, and with the feature map Element-wise addition, simultaneously, feature maps After leaky ReLU, gating unit and bilinear interpolation upsampling, and with Element-wise addition, for feature maps It undergoes leaky ReLU, gated unit, and bilinear interpolation for upsampling and downsampling, and is then compared with the feature map. Feature map By adding elements together, through this simple and effective feature fusion process, the feature map at each scale can obtain the weighted information of the feature maps at the other two scales. Step S3: After the fusion process described in step S2 It undergoes another 3×3 convolution to eliminate the overlap effect that may be introduced by the upsampling process, followed by another gating unit operation; Step S4: According to the structural requirements of the flow model, the outputs are as follows: and These two tensors serve as the output of the gated feature fusion module.

2. The unsupervised defect classification and segmentation method according to claim 1, characterized in that, In step one, the input image is first adjusted to 1 / 2 or 1 / 4 of the original image, and the original image is denoted as... The images adjusted to 1 / 2 and 1 / 4 of the original image are denoted as follows: and ,Will , , Input them into the feature extractor respectively. Feature maps are obtained after feature extraction. , , Feature maps are obtained after feature extraction. , Feature maps are obtained after feature extraction. .

3. The unsupervised defect classification and segmentation method according to claim 1, characterized in that, In step two, the split-flow model coupling layer is implemented using the Real-NVP architecture, and the specific steps are as follows: Input sample feature map First, the feature map is obtained through a channel shuffling operation. Then, divide them into feature maps according to the channel dimension. Feature map Next, the feature map After convolutional neural network The number of channels obtained after processing is The feature map is then divided into feature maps according to the channel dimension. , , After exponential operation and Multiplying by elements and then... Adding them together yields the feature map. ; Will After convolutional neural network The number of channels obtained after processing is The feature map is then divided into feature maps according to the channel dimension. , , After exponential operation and The feature map is obtained by multiplying the elements and then adding it to t2. ; Finally and The feature map is obtained by concatenating the data along the channel dimension. ; represent or , and These are two feature maps extracted by the feature extractor; corresponding to and , represent or .

4. The unsupervised defect classification and segmentation method according to claim 3, characterized in that, The calculation process of the affine transformation of the coupled layer of the segmented flow model can be described by formula (1): (1) Where ⊙ represents element-wise multiplication, the function , Composed of convolutional neural networks, it preserves the spatial location information of the feature maps. To ensure training stability, the parameters of the convolutional neural network are initialized to 0 initially. During inference, feature maps are calculated. of L2 norm at position The average value across the channel dimension is used as the predicted anomaly score map to complete the defect localization task.

5. The unsupervised defect classification and segmentation method according to claim 1, characterized in that, In the operation steps of the gated feature fusion module, the calculation formula for the gated unit is shown in Formula 3. (3), Where γ is a user-defined learnable weight parameter, initialized to 0. The sigmoid function is used to restrict the value range of the weight parameter to between 0 and 1. Represents the natural constant. The calculation function representing the gate control unit. Represents the sample feature map; During inference, feature maps are calculated. , , L2 norm The average value is then taken across the channel dimension to form the predicted image anomaly score, thus completing the defect detection task.

6. The unsupervised defect classification and segmentation method according to claim 3, characterized in that, The two-branch flow model employs maximum likelihood estimation to construct a loss function during training to guide the model's training. Specifically: First, by following a pre-defined known distribution... Calculations are performed using formula (5): (5), in, z Representing feature maps, Z Represents a data set, express The probability density function, express The probability density function; For simplicity, performing logarithmic operations on both sides of Equation 5 yields Equation 6: (6), Where logp x ( x ) represents taking the logarithm of the probability density function with respect to x, logp Z (z) denotes the logarithm of the probability density function with respect to z. This represents the logarithm of the absolute value of the Jacobian determinant; the result of this calculation is the sum of all values ​​of 𝑠. s Generally refers to the intermediate computational results s1 and s2 in the coupling layer of the segmented flow model branch and the multi-scale coupling layer of the classification flow model branch. , ; Then, variance stability loss is designed to address the cosine similarity loss term. The loss function of the two-branch flow model is shown in Equation 9, which consists of two parts: the segmentation flow model loss and the classification flow model loss. (9), The feature map 𝑧 follows a standard Gaussian distribution. To measure feature maps The variance stability loss is specifically calculated by calculating the variance between the feature maps of each positive sample in the Batch dimension. Substituting formula (6) into formula (9), we get formula (10): (10), Where 𝜆 is the balance coefficient, set to 5. represent Take the average value along the Batchsize dimension. This indicates summation along the Batchsize dimension. To ensure stability during training, the L2 norm of the gradient is restricted to 1.

7. An unsupervised defect classification and segmentation system based on a bi-branch flow model, characterized in that, include: A memory, a processor, and a computer program stored on the memory, the computer program being configured to implement the steps of the unsupervised defect classification and segmentation method of any one of claims 1-6 when invoked by the processor.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the unsupervised defect classification and segmentation method according to any one of claims 1-6 when invoked by a processor.

Citation Information

Patent Citations

  • Photovoltaic module unsupervised defect detection method based on GAN improved algorithm

    CN111340791A

  • Glass surface defect detection method based on deep learning

    CN111652883A

  • Optical fiber ferrule defect detection method based on deep learning

    CN111833324A

  • Glass slide defect detection method based on machine vision

    CN113245241A

  • Scooter display screen defect detection method and system based on machine vision

    CN114299020A