Defect Detection Method Based on Reconstruction Network

Abstract

The present invention provides a defect detection method based on a reconstruction network, by collecting defect-free pictures, and obtaining enhanced samples through data enhancement processing; then constructing a reconstruction network, and using it as a feature extractor after training, extracting features from the enhanced samples to obtain feature vectors; then constructing a detection network based on a cross-scale flow, transforming the feature vectors into different proportions, and performing density estimation on multi-scale feature maps corresponding to different proportions in parallel, outputting an abnormal density map, and obtaining defect detection results. Through the above manner, the present invention can use the reconstruction network as a feature extractor to accurately extract sample features; and introduce a cross-scale flow to jointly process image features of different scales, and then use a fully convolutional normalized flow to cross-connect between scales, perform density estimation on the features, so that the defect area on the image can be visualized, and can achieve better detection results with a small number of training samples.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a defect detection method based on a reconstruction network. Background Art

[0002] In recent years, more and more industrial technologies have focused on surface defect detection and improved quality control in the industrial field. Since surface defects have a great impact on the quality results of industrial products, it is crucial to ensure that defective products are noticed at an early stage. In the defect detection process, small defects need to be carefully and reliably detected. Traditional manual detection has been difficult to meet the needs. The detection method based on deep learning has gradually become the development direction of defect detection.

[0003] Most existing defect detection technologies are divided into supervised and unsupervised methods. The supervised method requires a large amount of manual labeling of defects before segmentation training. However, because the styles and sizes of defects vary, manual labeling is a time-consuming and labor-intensive task. In contrast, the unsupervised method is easier and requires a large number of normal samples for training. Manual labeling is not required. However, the detection effect of this method is relatively not so ideal. Different data enhancement methods will lead to different training effects. The characteristics of defects are also difficult to define. New types of defects may occur all the time. At the same time, the detection results will also be biased, such as the inability to detect small defects, etc., and the data volume of normal samples is also large. In addition, real-world product images actually contain a lot of background noise, which often makes the results of defect detection less reliable.

[0004] The patent with publication number CN114235847A provides a device and method for detecting surface defects of liquid crystal panel glass substrates. When using a deep learning algorithm to judge glass substrate defects, the patent segmented the image, extracted features from interference items, and then established an image reconstruction network to judge defects, thus completing the judgment of glass substrate defects. However, the conventional method of feature extraction and the use of the reconstruction network for defect judgment in this patent not only make it difficult to accurately extract defect features, but also the defect judgment results are not accurate enough, and require a large number of training samples. It is not suitable for defect detection scenarios in the industry where samples are scarce, and has certain limitations.

[0005] In view of this, it is necessary to design an improved defect detection method based on reconstruction network to solve the above problems. Summary of the invention

[0006] In view of the defects of the above-mentioned prior art, the purpose of the present invention is to provide a defect detection method based on a reconstruction network. By adopting a reconstruction method, the reconstruction network is used as a feature extractor to accurately extract sample features; and a cross-scale flow is introduced to jointly process image features of different scales, and then a fully convolutional normalized flow is used to cross-connect between scales to estimate the density of features, so that the defect area on the image can be visualized, and better detection results can be achieved with a small number of training samples.

[0007] To achieve the above object, the present invention provides a defect detection method based on a reconstruction network, comprising the following steps:

[0008] S1, collect defect-free pictures, and obtain enhanced samples through data enhancement processing;

[0009] S2, constructing a reconstruction network, using the trained reconstruction network as a feature extractor, performing feature extraction on the enhanced sample obtained in step S1, and obtaining a feature vector;

[0010] S3. Construct a detection network based on cross-scale flow, transform the feature vector obtained in step S2 into different proportions, and perform density estimation on multi-scale feature maps corresponding to different proportions in parallel, output an abnormal density map, and obtain a defect detection result.

[0011] As a further improvement of the present invention, in step S3, the defect detection process based on the cross-scale flow detection network includes the following steps:

[0012] Obtain multiple feature maps of different scales;

[0013] Jointly process multiple feature maps of different scales;

[0014] A fully convolutional normalization flow is used to perform cross-connections between scales, density estimation is performed on feature maps of different scales, and likelihood is assigned to input samples based on the density estimation, and the likelihood is used as an indicator of the defect to locate and mark the defect.

[0015] As a further improvement of the present invention, in step S1, the data enhancement processing includes enhancing normal samples without defects and generating defective pictures, and the defective pictures are generated based on a defect generation network of GAN and histogram matching; the way of enhancing the normal samples includes one or more of rotation, flipping, random cropping, pasting, and erasing.

[0016] As a further improvement of the present invention, in step S1, the defect generation network based on GAN and histogram matching includes a generator and a discriminator; the generator randomly mixes the generated area into the defect-free sample by adopting local Gaussian blur to simulate the real defect sample; the discriminator adds the reference defect sample generated by histogram matching, and increases the calculation of the local histogram loss.

[0017] As a further improvement of the present invention, in step S2, the reconstruction network includes an encoder and a decoder, which are used to reconstruct the defective image generated in step S1 into a simulated defect-free sample; the loss of the reconstruction network includes an SSIM loss function and an L2 loss function.

[0018] As a further improvement of the present invention, in step S2, a trainable bottleneck module is also included between the encoder and the decoder; the bottleneck module includes a trainable MFF module and an OCE module, the MFF module is used to process multi-scale features from the encoder, and the OCE module is used to condense the obtained features into a bottleneck code.

[0019] As a further improvement of the present invention, in step S2, the encoder includes five convolution blocks, each of which contains two convolution layers, two BN layers and two ReLU layers; the decoder includes four feature extraction blocks, each of which includes an upsampling block, a contact block and a deconvolution block; the upsampling block adopts a bilinear upsampling method, and the deconvolution block includes two convolution layers, two BN layers and two ReLU layers.

[0020] As a further improvement of the present invention, in step S2, when the trained reconstructed network is used for feature extraction, the following two methods are included:

[0021] The first feature extraction method is to extract only the features extracted from the last layer of the last convolution block of the encoder when the normal sample is input;

[0022] The second feature extraction method is to simultaneously extract the features extracted by the last layer of the encoder and the decoder, and establish a connection through the contact block.

[0023] As a further improvement of the present invention, in step S3, the feature vector is subjected to bijective and parallel transformation, and the dimension of the feature vector is retained; the NF framework is used to perform density estimation on multi-scale feature maps corresponding to different scales in parallel, so that fine-grained position and context information can be maintained; the density estimation is obtained by bidirectional mapping learning from an unknown distribution in the feature space to a latent space with a Gaussian distribution.

[0024] As a further improvement of the present invention, in step S3, bilinear upsampling or layered convolution is used to adjust the size of single feature maps of different scales, and then aggregation is performed by summing.

[0025] The beneficial effects of the present invention are:

[0026] (1) The defect detection method based on the reconstruction network provided by the present invention adopts a reconstruction method, uses the reconstruction network as a feature extractor, and accurately extracts sample features; and introduces a cross-scale flow to jointly process image features of different scales, and then uses a fully convolutional normalization flow to cross-connect between scales, and performs density estimation on the features, so that the defect area on the image can be visualized, and can achieve better detection results when the number of training samples is small. Based on this, the defect detection method based on the reconstruction network provided by the present invention is a semi-supervised learning method, which not only effectively solves some of the shortcomings of existing supervised and unsupervised methods, but is also very suitable for defect detection scenarios with fewer samples in industry, and has a wide range of applications.

[0027] (2) The defect detection method based on reconstruction network provided by the present invention innovatively introduces cross-scale flow on the basis of normalized flow, forming a new type of full convolution cross-scale normalized flow, which can simultaneously process image features of different scales, and propagate in parallel through normalized flow, while influencing each other, and perform density estimation on features, so as to detect defects by jointly estimating the likelihood on multi-scale feature maps. Considering that the discriminability of defects is unknown during training, the method provided by the present invention utilizes the full potential of information and associations in local and global contexts to accurately learn distributions to identify defective samples. In addition, because the method provided by the present invention adopts a fully convolutional structure, the spatial arrangement of feature vectors is retained, so that the defective area on the image can be visualized. Based on this, the method provided by the present invention can achieve good performance even when the number of training samples is small, which makes the method have a wider application scenario and more accurate detection effect, effectively overcomes the shortcomings of the prior art, and has strong robustness in defect detection.

[0028] (3) The defect detection method based on reconstruction network provided by the present invention further adds a trainable bottleneck module between the encoder and decoder of the reconstruction network, which helps to prohibit the propagation of some abnormal features and retain effective features. Compared with the residual network feature extraction method commonly used in the prior art, it can significantly improve the feature extraction effect and thus improve the accuracy of the defect detection results.

[0029] (4) The defect detection method based on reconstruction network provided by the present invention can expand the limited data set in the real scene by further introducing the defect generation network based on GAN and histogram matching in the process of data enhancement. Combined with histogram matching, it can not only avoid the overfitting problem in training as much as possible, but also simulate and generate a certain amount of defective images and normal images to meet the data volume required for accurate training, which can effectively solve the problem of insufficient samples and poor training effect in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic flow chart of a defect detection method based on a reconstructed network provided by the present invention.

[0031] Figure 2 Schematic diagram of the structure of Block in GAN in some embodiments of the present invention.

[0032] Figure 3 Schematic diagram of defects generated according to mask simulation in some embodiments of the present invention.

[0033] Figure 4 The figure is a schematic diagram of the network structure of the reconstructed network in some embodiments of the present invention.

[0034] Figure 5 It is a schematic diagram of the structure of the Block of the encoder in the reconstruction network in some embodiments of the present invention.

[0035] Figure 6 Schematic diagram of an image reconstructed by a reconstruction network in some embodiments of the present invention.

[0036] Figure 7 Schematic diagram of the network structure of cross-scale flow in some embodiments of the present invention.

[0037] Figure 8 1 is an abnormal density map obtained in some embodiments of the present invention.

[0038] Fig. 9 Schematic diagram of defect location obtained in some embodiments of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the scheme of the present invention are shown in the drawings, while other details that are not closely related to the present invention are omitted.

[0041] In addition, it should be noted that the terms "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or apparatus.

[0042] The present invention provides a defect detection method based on a reconstruction network, and its flow chart is as follows: Figure 1 As shown, the following steps are included:

[0043] S1, collect defect-free pictures, and obtain enhanced samples through data enhancement processing;

[0044] S2, constructing a reconstruction network, using the trained reconstruction network as a feature extractor, performing feature extraction on the enhanced sample obtained in step S1, and obtaining a feature vector;

[0045] S3. Construct a detection network based on cross-scale flow, transform the feature vector obtained in step S2 into different proportions, and perform density estimation on multi-scale feature maps corresponding to different proportions in parallel, output an abnormal density map, and obtain a defect detection result.

[0046] Specifically, since the traditional methods for defect detection have some limitations and are only effective for specific texture backgrounds, and there are few defective samples on industrial production lines, directly using them for training is likely to lead to overfitting. Therefore, in some embodiments of the present invention, the data enhancement processing described in step S1 includes not only enhancing the normal samples without defects, but also generating defective pictures. Among them, the methods for enhancing the normal samples of the normal samples without defects include rotation, flipping, random cropping, pasting, erasing, etc.

[0047] In some embodiments of the present invention, a defect generation network based on GAN and histogram matching is specifically used to generate defective images, and a regional unpaired training strategy is used on the conventional GAN ​​to achieve forward generation of defects within any specified shape. In this process, local Gaussian blur is used to randomly mix the generated area into the defect-free sample to simulate the real defect sample, while ensuring the diversity of the enhanced training samples to avoid overfitting, and ensuring the effectiveness of image enhancement under complex backgrounds, thereby ensuring that the use of enhanced samples in subsequent training networks can effectively converge.

[0048] Specifically, the defect generation network based on GAN and histogram matching includes a generator G and a discriminator D. The structure of the Block in GAN is as follows: Figure 2 shown.

[0049] Among them, the generator G uses local Gaussian blur to randomly mix the generated area into the defect-free sample to simulate the real defect sample. During the training process of the generator G, the random perspective method and the random resize cropping method are used for preprocessing to obtain enough image blocks h from a single reference object H. At the same time, the defect mask is added to the original defect-free sample O and the image block h, that is, the mask image O' and h'; then, the mask image is input into the generator G; after being fused with the background, a Gaussian blur is used to obtain the output and In this process, considering that the goal of the defect generation network based on GAN and histogram matching is to generate defects, in the local specified area, the generator G only uses the local loss in the defect area for training. For the real reference image block h, the generator G reconstructs the original defect based on the local L1 loss. The calculation formula is as follows:

[0050]

[0051] Among them, h, Represent the input image block and the output image block respectively, represents element-wise multiplication, m is the mask added to the input image block h, and N is the number of points in m.

[0052] The role of the discriminator D is to determine the authenticity of the output, focusing on distinguishing generated defects from real defects, while the focus of the generator G is to generate a more realistic defect to deceive the discriminator D. Based on this, the adversarial loss function is defined as follows:

[0053]

[0054] Among them, x is the output of the discriminator D, and y is the recognized label.

[0055] Just need to output h and The defective area is input into the discriminator D to obtain the local loss.

[0056] When training with a masked image O, there is no corresponding defect reference sample, resulting in the above loss being invalid. Since this method only uses a single reference object H as a reference sample to train the generator G, in order to achieve a better simulation generation effect, another local histogram loss is used to avoid overfitting. By adding the mask M of the reference defect sample generated by histogram matching and increasing the calculation of the local histogram loss, The defect area is histogram matched with the reference defect to obtain a matching area, and then an L2 loss is used between the two defects before and after the matching.

[0057] Specifically, the above local histogram loss is defined as follows:

[0058]

[0059] Among them, R'[] is the local histogram matching operator.

[0060] In the above way, the added local histogram loss can ensure the uniform structure in the simulated defect area. At the same time, in the above process, when training the generator G, the weight of the discriminator D remains unchanged. After local Gaussian blurring, the final simulated defect map is obtained. Figure 3 shown.

[0061] After completing the above data enhancement processing steps, the obtained normal sample images without defects and the generated defective images are input into the reconstruction network together for feature extraction.

[0062] In some embodiments of the present invention, the reconstruction network is trained to reconstruct the original image I from the defective image Ia generated in step S1. The closer the reconstructed simulated original image is to the real image, the more effective and accurate the features extracted by the network are. Using the reconstruction network as a feature extractor can achieve more accurate features than most networks using the Resnet series, and can capture more effective features for defect detection.

[0063] Specifically, the reconstruction network consists of an encoder E, a decoder D and a bottleneck module, and its network structure diagram is shown in the following figure: Figure 4 As shown. Figure 4 It can be seen that the reconstruction network is a U-net-like structure, consisting of an encoder E and a decoder D. It converts the local defects of the input image into a distribution closer to the normal sample, that is, it cascades the previously simulated defect image Ia and the defect-free original image I into a simulated defect-free sample Ir.

[0064] Specifically, the encoder E consists of five convolution blocks, each of which contains two 3×3 convolutional layers, two BN layers, and two ReLU layers. There is a maximum pooling layer between the convolution blocks. The schematic diagram of its Block structure is shown in the following figure. Figure 5 shown.

[0065] The bottleneck module includes a trainable MFF module and an OCE module, wherein the MFF module is used to process the multi-scale features from the encoder, and the OCE module is used to condense the obtained rich features into a compact bottleneck code Since the encoder E usually has a higher capacity. Although the high-capacity model helps to extract rich features, the high-dimensional descriptors contained may have conceivable redundancy, and the high degree of freedom and redundancy are harmful to the inherently abnormal characteristics of the decoder D. Based on this, the present invention uses compact embedding as an information bottleneck to help prevent the propagation of abnormal perturbations to the decoder D, thereby increasing the representative difference of the encoder E in terms of abnormalities. In a specific embodiment of the present invention, the fourth residual block of ResNet is used as a type of embedding block, and the bottleneck module uses a 3×3 convolution layer, a ReLU activation layer, and a fully connected layer.

[0066] The decoder D is mainly composed of four feature extraction blocks, each of which includes an upsampling block, a contact block and a deconvolution block; the upsampling block adopts a bilinear upsampling method, and the deconvolution block includes two 3×3 convolution layers, two BN layers and two ReLU layers.

[0067] In existing reconstruction-based methods, L2 loss is often used, which is calculated as follows:

[0068]

[0069] However, this assumes the independence between adjacent pixels. Based on this, the present invention adds an additional SSIM loss, whose loss function is defined as:

[0070]

[0071] Where H and W are the height and width of image I. Np is equal to the number of pixels in I. Ir is the reconstructed image output by the network. SSIM(I, Ir)(i, j) is the SSIM value of the blocks I and Ir centered at image coordinates (i, j).

[0072] Therefore, the loss of the reconstruction network is:

[0073] Lrec(I,Ir)=λLSSIM(I,Ir)+L2(I,Ir)

[0074] Among them, λ is a loss balancing hyperparameter.

[0075] After training, the reconstructed network can be used as a feature extractor. There are two feature extraction methods to suit different data sets. The specific methods are as follows:

[0076] The first feature extraction method is to extract only the features extracted from the last layer of the last convolutional block of the encoder when the normal samples are input. This method makes the extracted features free of abnormal interference and is suitable for data sets with no defective samples at all.

[0077] The second feature extraction method is to extract the features extracted by the last layer of the encoder and the decoder at the same time, and establish a connection through the contact block. This method can integrate the data lost by downsampling in the previous encoder E, and obtain richer semantic features.

[0078] Through the above method, the image reconstructed by the reconstruction network is as follows Figure 6 As shown, the better the reconstruction effect is, the better the feature extraction effect is.

[0079] After completing the above feature extraction, defect detection is performed using a detection network based on cross-scale flow, which includes the following steps:

[0080] Obtain multiple feature maps of different scales;

[0081] Jointly process multiple feature maps of different scales;

[0082] A fully convolutional normalization flow is used to perform cross-connections between scales, density estimation is performed on feature maps of different scales, and likelihood is assigned to input samples based on the density estimation, and the likelihood is used as an indicator of the defect to locate and mark the defect.

[0083] In the above process, the detection network based on cross-scale flow can model the distribution of the obtained defect-free image data, detect and locate defects based on the density estimation of feature maps in images of different sizes, and jointly process multi-scale feature maps, use a fully convolutional normalization flow, cross-connect between scales, and assign meaningful likelihoods to input samples, so that effective defect detection can be performed at the image level. Compared with the current methods that use strong statistical priors or overly simplified data representations, this method includes fine-grained representations of global and local image backgrounds while flexibly estimating density. In addition, the fully convolutional approach retains spatial arrangement, which allows the defective areas in the image to be accurately located.

[0084] Specifically, when performing defect detection, the detection network based on cross-scale flow directly inputs the features y∈Y of the defect-free image x∈X extracted by the reconstruction network into the detection network. During the inference process, a probability is assigned to the input image x by using the density estimate of the image feature u, and a low probability is assumed to be an indicator of a defect. The density estimate is learned by bidirectional mapping from the unknown distribution py in the feature space Y to the Gaussian distribution pz in the latent space Z.

[0085] Specifically, in one embodiment of the present invention, the cross-scale flow fcsf is used to perform bijective and parallel transformation on the feature vector y obtained in step S2, so that it can be transformed into different scales and retain the dimension of the feature vector. The transformation method is as follows:

[0086] fcsf(y (1) , ..., y (s) )=[z (1) , ..., z (s) ]=z∈Z

[0087] Among them, y (i) Defined as the input image x (i) 3D feature vector of , i∈{1,...,s}.

[0088] At the same time, the above method uses the NF framework to perform density estimation on multi-scale feature maps corresponding to different scales in parallel, so that fine-grained position and context information can be maintained.

[0089] In the above process, the possibility pz is measured according to the target distribution. In the example based on this method, the target distribution is a multivariate standard normal distribution. According to pz and the set threshold θ, it can be judged whether the image x is abnormal, which is specifically expressed as follows:

[0090]

[0091] In some embodiments of the present invention, the network structure diagram of the cross-scale flow is as follows: Figure 7 As shown. Figure 7 It can be seen that the cross-scale flow is a chain of so-called coupling blocks, each of which performs an affine transformation as the basis of the framework structure of the coupling block. In a specific embodiment of the present invention, Real-NVP is selected. Figure 7 It can also be seen that the convolutions are performed on two levels, with cross connections of different scales on the second level.

[0092] Specifically, in the process of defect detection using a detection network based on cross-scale flow, bilinear upsampling or layered convolution can be used to adjust the size of individual feature maps of different scales, and then aggregated by summing. At the same time, the output is split in the channel dimension to obtain scale and displacement parameters, which are estimated by coupling blocks, and the output is split into [s1, t1] and [s2, t2] through two separate sub-networks and then used in the following way:

[0093] yout,2=yin,2⊙γ1s1(yin,1)+γ1t1(yin,1)

[0094] yout,1=yin,1⊙γ1 s2(yout,2)+γ2 t2(yout,2)

[0095] In the above process, in order to stabilize the initialization model, some embodiments of the present invention introduce trainable block scaling coefficients γ1 and γ2, which are initialized to 0, thus resulting in yout = yin. Among them, the two internal sub-networks do not need to be reversible and can be any differentiable functions. In the design of this method, they are fully convolutional networks, and the two modules are regressed by segmenting the output. When processing images, each scale has a hidden layer, and the number of channels will increase.

[0096] During training, the cross-scale flow maximizes the likelihood of the input feature vectors y by mapping them into the latent space z, where an explicit density likelihood can be defined:

[0097]

[0098] Therefore, the loss objective is formulated as the minimization of the negative log-likelihood as follows:

[0099]

[0100]

[0101] In the above manner, the detection network based on cross-scale flow adopted by the present invention adopts full convolution, which can retain all position features of the feature map, so the position formation is retained, which enables the present invention to interpret the output with the possibility of a single image area and directly locate the defect. Similar to the definition of the anomaly score of the entire image, the present invention defines anomaly scores for each local position (i, j) of the feature map, and then aggregates these values, finds image regions with high paradigms along the channel size and mark to determine the location, and marks the defects. The detected anomaly density map and defect location schematic diagram are shown as follows: Figure 8 , Fig. 9 Based on this, the method provided by the present invention can make the defect area on the image visible, achieve good performance even when the number of training samples is small, and has strong robustness in defect detection.

[0102] In summary, the present invention provides a defect detection method based on a reconstruction network, by collecting defect-free pictures, and obtaining enhanced samples through data enhancement processing; then constructing a reconstruction network, and using it as a feature extractor after training, to extract features from the enhanced samples to obtain feature vectors; then constructing a detection network based on a cross-scale flow, transforming the feature vectors into different proportions, and performing density estimation on multi-scale feature maps corresponding to different proportions in parallel, outputting an abnormal density map, and obtaining defect detection results. Through the above manner, the present invention can use the reconstruction network as a feature extractor to accurately extract sample features; and introduce a cross-scale flow to jointly process image features of different scales, and then use a fully convolutional normalized flow to cross-connect between scales, perform density estimation on the features, so that the defect area on the image can be visualized, and can achieve better detection results with a small number of training samples.

[0103] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A defect detection method based on a reconstructed network, characterized in that: The steps include: S1, collect defect-free pictures, and obtain enhanced samples through data enhancement processing; S2, constructing a reconstruction network, using the trained reconstruction network as a feature extractor, performing feature extraction on the enhanced sample obtained in step S1, and obtaining a feature vector; S3, constructing a detection network based on cross-scale flow, transforming the feature vector obtained in step S2 into different proportions, and performing density estimation on multi-scale feature maps corresponding to different proportions in parallel, outputting an abnormal density map, and obtaining a defect detection result; In step S1, the data enhancement process includes enhancing a normal sample without defects and generating a defective picture, wherein the defective picture is generated based on a defect generation network of GAN and histogram matching; The manner of enhancing the normal sample includes one or more of rotation, flipping, random cutting, pasting, and erasing; The defect generation network based on GAN and histogram matching includes a generator and a discriminator. The generator randomly mixes the generated area into the defect-free sample by using local Gaussian blur to simulate the real defect sample. The discriminator adds the reference defect sample generated by histogram matching and increases the calculation of the local histogram loss. The generator G randomly mixes the generated area into the defect-free sample by using local Gaussian blur to simulate the real defect sample. During the training process of the generator G, the random perspective method and the random resize cropping method are used for preprocessing to obtain enough image blocks h from a single reference object H. At the same time, the defect mask is added to the original defect-free sample O and the image block h, that is, the mask image O ′ and h ′ ; Then, the mask image is input into the generator G; after being fused with the background, a Gaussian blur is performed to obtain the output and In this process, considering that the goal of the defect generation network based on GAN and histogram matching is to generate defects, in the local specified area, the generator G only uses the local loss in the defect area for training. For the real reference image block h, the generator G reconstructs the original defect based on the local L1 loss. The calculation formula is as follows: Among them, h, Represent the input image block and the output image block respectively, represents element-wise multiplication, m is the mask added to the input image block h, and N is the number of points in m; The adversarial loss function is defined as follows: Among them, x is the output of the discriminator D, and y is the recognized label; The h and The defective area is input into the discriminator D to obtain the local loss; When training with masked images O, another local histogram loss is used to avoid overfitting by adding the mask M of the reference defect sample generated by histogram matching and increasing the calculation of the local histogram loss. The defect area is matched with the reference defect by histogram matching to obtain a matching area, and then an L2 loss is used between the two defects before and after matching; The above local histogram loss is defined as follows: Among them, R ′ [] is the local histogram matching operator; In step S2, the reconstruction network includes an encoder and a decoder, which are used to reconstruct the defective picture generated in step S1 into a simulated defect-free sample; the loss of the reconstruction network includes an SSIM loss function and an L2 loss function; Among them, the L2 loss function is: The SSIM loss function is: Where H and W are the height and width of image I, Np is equal to the number of pixels in I, Ir is the reconstructed image output by the network, and SSIM(I,Ir)(i,j) is the SSIM value of the I and Ir blocks centered at the image coordinate (i,j); The loss of the reconstruction network is: Lrec(I, Ir) = λLSSIM(I, Ir) + L2(I, Ir) Among them, λ is a loss balancing hyperparameter.

2. The defect detection method based on reconstruction network according to claim 1 is characterized in that: In step S3, the defect detection process based on the cross-scale flow detection network includes the following steps: obtaining a plurality of feature maps of different scales; Jointly process multiple feature maps of different scales; A fully convolutional normalization flow is used to perform cross-connections between scales, density estimation is performed on feature maps of different scales, and likelihood is assigned to input samples based on the density estimation, and the likelihood is used as an indicator of the defect to locate and mark the defect.

3. The defect detection method based on reconstruction network according to claim 1, characterized in that: In step S2, a trainable bottleneck module is further included between the encoder and the decoder; the bottleneck module includes a trainable MFF module and an OCE module, the MFF module is used to process multi-scale features from the encoder, and the OCE module is used to condense the obtained features into a bottleneck code.

4. The defect detection method based on reconstruction network according to claim 1, characterized in that: In step S2, the encoder includes five convolution blocks, each of which contains two convolution layers, two BN layers and two ReLU layers; the decoder includes four feature extraction blocks, each of which includes an upsampling block, a contact block and a deconvolution block; the upsampling block adopts a bilinear upsampling method, and the deconvolution block includes two convolution layers, two BN layers and two ReLU layers.

5. The defect detection method based on reconstruction network according to claim 4 is characterized in that: In step S2, when the trained reconstructed network is used for feature extraction, the following two methods are included: The first feature extraction method is to extract only the features extracted from the last layer of the last convolution block of the encoder when the normal sample is input; The second feature extraction method is to simultaneously extract the features extracted by the last layer of the encoder and the decoder, and establish a connection through the contact block.

6. The defect detection method based on reconstruction network according to claim 1 is characterized in that: In step S3, the feature vector is bijectively and parallelly transformed, and the dimension of the feature vector is retained; the NF framework is used to perform density estimation in parallel on multi-scale feature maps corresponding to different scales, so that fine-grained position and context information can be maintained; the density estimation is obtained by bidirectional mapping learning from an unknown distribution in the feature space to a latent space with a Gaussian distribution.

7. The defect detection method based on reconstruction network according to claim 1 is characterized in that: In step S3, bilinear upsampling or layered convolution is used to resize individual feature maps of different scales, and then they are aggregated by summing.

Images