Surface anomaly detection method and device based on cascade reconstruction-discrimination structure

Abstract

The present disclosure relates to a surface anomaly detection method and device based on a cascade reconstruction-discrimination structure. The method comprises: reconstructing a to-be-detected image to obtain a first reconstructed image; reconstructing the first reconstructed image to obtain a second reconstructed image; concatenating the to-be-detected image, the first reconstructed image and the second reconstructed image from the channel dimension to obtain a first anomaly score map and a second anomaly score map; and obtaining an anomaly detection result of the to-be-detected image by performing pixel-by-pixel average processing on the first anomaly score map and the second anomaly score map. The present disclosure improves the performance of surface anomaly detection.

Description

Technical Field

[0001] This invention belongs to the field of anomaly detection in machine vision, specifically relating to a surface anomaly detection method and apparatus based on a cascaded reconstruction-discrimination structure. Background Technology

[0002] Surface anomaly detection (SAD) aims to detect anomalous patterns that differ from normal samples. Anomaly detection has been widely applied in fields such as intelligent industrial manufacturing, medical image analysis, and video surveillance, yielding significant socio-economic benefits. For example, automated visual defect detection in industrial production processes can improve enterprise production efficiency and save costs. In many applications, normal sample data is abundant and easily obtained, but anomalous data is generally scarce or very costly to acquire. Therefore, surface anomaly detection requires unsupervised methods, that is, using only normal data to detect whether test samples contain anomalies. This type of task is also known as a one-class classification (OCC) problem.

[0003] Anomaly detection comprises two subtasks: image-level anomaly detection (image binary classification) and pixel-level anomaly localization (image segmentation). Anomaly localization is more challenging because it requires precise localization of unknown anomalies. These anomalies vary greatly in type and scale, with some anomalous regions appearing very similar to normal samples. Most anomalous regions, such as thin cracks and scratches, occupy only a small fraction of the image pixels.

[0004] Current methods primarily focus on the anomaly classification subtask, neglecting accurate localization of anomaly regions. Existing reconstruction-based and representation-based methods often predict anomaly score maps containing erroneous responses. Representation-based methods, such as PatchCore, produce relatively coarse anomaly segmentation results. These methods utilize pixel or image patch comparisons in feature space to obtain anomaly scores. Due to the lack of true anomalies, they inevitably predict inaccurate responses from blurry pixels in the anomaly map. Reconstruction-based methods, which define anomaly scores as pixel-level reconstruction errors, also face similar drawbacks. The method jointly trained with a discriminative network (DRAEM) only uses texture-type anomaly generation and is not suitable for detecting anomalies other than texture types. The DRAEM method is referenced in "Draem-adiscriminatively trained reconstruction embedding for surface anomaly detection," Zavrtanik V, Kristan M. D.in Proc.ICCV 2021. The PatchCore method refers to "Towards total recall in industrial anomaly detection", Roth K, Pemula L, ZepedaJ, et al. in Proc.CVPR 2022.

[0005] The invention content is passed

[0006] This invention proposes a surface anomaly detection method and apparatus based on a cascaded reconstruction-discrimination structure. The method is based on two cascaded reconstruction subnetworks and their respective discrimination subnetworks, which effectively improves the performance of surface anomaly detection.

[0007] The technical content of this invention includes:

[0008] A surface anomaly detection method based on a cascaded reconstruction-discrimination structure, the method comprising:

[0009] The image to be detected is reconstructed to obtain the first reconstructed image;

[0010] The first reconstructed image is reconstructed to obtain the second reconstructed image;

[0011] The image to be detected is concatenated with the first reconstructed image and the second reconstructed image from the channel dimension to obtain the first anomaly score map and the second anomaly score map.

[0012] By averaging the first and second anomaly score maps pixel by pixel, the anomaly detection result of the image to be detected is obtained.

[0013] Further, the reconstruction of the image to be detected to obtain a first reconstructed image includes:

[0014] Construct a first reconstructed sub-network, which adopts the encoder-decoder architecture of U-Net;

[0015] Generate texture anomaly images of the original image;

[0016] The original image is subjected to a nonlinear transformation to obtain the squeezed structure image, the magnified structure image, the rotated structure image, and the wave structure image;

[0017] Three polygons and one circular region are randomly generated to segment label M. a ;

[0018] Based on the segmentation label M aThe original image is segmented, and a squeezed structure image, a magnified structure image, a rotated structure image, and a wave structure image are applied to the four segmented regions respectively, while the other regions remain unchanged, to obtain a structural anomaly image.

[0019] A training dataset was constructed based on texture anomaly images and structural anomaly images;

[0020] The first reconstructed sub-network is trained based on the training dataset;

[0021] The image to be detected is input into the trained first reconstruction network to obtain the first reconstructed image.

[0022] Furthermore, the nonlinear transformation of the original image to obtain the extruded structure image includes:

[0023] Place the original image in a polar coordinate system and set the squeezing center at the origin of the polar coordinate system;

[0024] Obtain the parameter ratio for adjusting the extrusion ratio. p ;

[0025] For any point P(r,θ) in the original image, keep the angle θ constant and calculate the polar radius after compression. To obtain the polar coordinates (r′, θ) of point P after compression;

[0026] After performing extrusion changes on each point of the original image, an extruded structure image is obtained.

[0027] Furthermore, the nonlinear transformation of the original image to obtain the magnified structural image includes:

[0028] Place the original image in a polar coordinate system and set the magnification center at the origin of the polar coordinate system;

[0029] Obtain the parameter ratio for adjusting the amplification ratio. m ;

[0030] For any point P(r,θ) in the original image, keep the angle θ constant and calculate the polar radius after magnification. To obtain the magnified polar coordinates (r″, θ) of point P; where R represents the radius of the magnified circular region;

[0031] After magnifying and transforming each point of the original image, an magnified structural image is obtained.

[0032] Further, the nonlinear transformation of the original image to obtain the rotated structure image includes:

[0033] Place the original image in a polar coordinate system and set the center of rotation at the origin of the polar coordinate system;

[0034] Obtain the parameter ratio that adjusts the rotation rate. v ;

[0035] For any point P(r,θ) in the original image, keep the polar radius r constant and calculate the polar angle after rotation. To obtain the polar coordinates (r, θ′) of point P after rotation; where R represents the radius of the enlarged circular region;

[0036] After rotating each point of the original image, a rotated structure image is obtained.

[0037] Furthermore, the nonlinear transformation of the original image to obtain the wave structure image includes:

[0038] Place the image in a two-dimensional Cartesian coordinate system;

[0039] Obtain the amplitude A and frequency parameter ω of the trigonometric function;

[0040] For any point P(x,y) in the original image, keep the x-coordinate unchanged and calculate the changed y-coordinate y′=A·sin(ωx) to obtain the changed coordinates P(x,y′) of point P.

[0041] Furthermore, when training the first reconstructed sub-network based on the training dataset, the loss... Where I represents the original image in the training dataset, I r Let λ represent the image output by the first reconstructed subnetwork, and λ represent the weights. Represents pixel-level L2 loss, based on structural similarity loss of image patches. N p =H·W represents the number of pixels in the original image, and i and j are the pixel positions in the image.

[0042] Further, the step of concatenating the image to be detected with the first reconstructed image along the channel dimension to obtain the first anomaly score map includes:

[0043] Construct a first discriminative subnetwork based on a bidirectional channel attention module; the first discriminative subnetwork adopts a U-Net structure, which consists of n downsampling and upsampling stages, generating n feature layers of different sizes, and each layer in the first n-1 stages uses a bidirectional channel attention module;

[0044] Generate texture anomaly images and structural anomaly images to construct the training dataset;

[0045] The first discriminant sub-network is trained based on the training dataset;

[0046] The image to be detected and the first reconstructed image are input into the first reconstructed network after training to obtain the first anomaly score map.

[0047] Further, the step of obtaining the anomaly detection result of the image to be detected by pixel-by-pixel averaging of the first anomaly score map and the second anomaly score map includes:

[0048] The first and second anomaly score maps are averaged pixel by pixel to obtain the anomaly score map.

[0049] The noise in the anomaly score map is removed by applying average pooling to obtain a smooth anomaly score map;

[0050] Obtain the maximum value in the smoothed anomaly score map to obtain the image-level anomaly score;

[0051] The image-level anomaly score is compared with a threshold to obtain the apparent anomaly detection result of the image to be detected.

[0052] A surface anomaly detection device based on a cascaded reconstruction-discrimination structure, characterized in that the device comprises:

[0053] The first reconstruction module is used to reconstruct the image to be detected to obtain the first reconstructed image.

[0054] The second reconstruction module is used to reconstruct the first reconstructed image to obtain the second reconstructed image;

[0055] The first discrimination module is used to concatenate the image to be detected with the first reconstructed image from the channel dimension to obtain a first anomaly score image;

[0056] The second discrimination module is used to concatenate the image to be detected and the second reconstructed image along the channel dimension to obtain a second anomaly score image.

[0057] The result generation module is used to obtain the apparent anomaly detection result of the image to be detected by averaging the first anomaly score map and the second anomaly score map pixel by pixel.

[0058] Compared with existing technical solutions, the present invention has at least the following technical effects:

[0059] This invention achieves state-of-the-art anomaly localization performance on the general surface anomaly detection dataset MVTec AD, exceeding the current best method by 9.3% AP. It also achieves state-of-the-art anomaly classification and localization performance on the industrial anomaly detection dataset BTAD. The anomaly classification accuracy reaches 96.5% AUROC, and the anomaly localization performance exceeds the current best method by 8.4% PRO and 7.8% AP. Attached Figure Description

[0060] Figure 1 This is a diagram of the cascaded reconstruction-discrimination network framework of the present invention.

[0061] Figure 2 This is a flowchart of the method of the present invention.

[0062] Figure 3 This invention relates to a bidirectional channel attention module.

[0063] Figure 4 This is a schematic diagram of the structural anomaly generation process proposed in this invention.

[0064] Figure 5 This is a sample of experimental results from the BTAD dataset used in this invention.

[0065] Figure 6 This is an example of the experimental results of this invention on the MVTec AD dataset. Detailed implementation method:

[0066] The exemplary embodiments will now be described in detail with reference to the accompanying drawings.

[0067] It should be noted that the relevant embodiments and accompanying drawings are only for describing and illustrating exemplary embodiments provided by this disclosure, and not all embodiments of this disclosure, nor should this disclosure be understood to be limited to the relevant exemplary embodiments.

[0068] It should be noted that the terms "first," "second," etc., used in this disclosure are only used to distinguish different steps, devices, or modules. These terms do not represent any specific technical meaning, nor do they indicate any order or interdependence between them.

[0069] It should be noted that the term "at least one" as used in this disclosure is illustrative rather than restrictive. Unless otherwise expressly indicated in the context, it should be understood as "one or more".

[0070] It should be noted that the term "and / or" used in this disclosure is used to describe the relationship between related objects, and generally indicates that there are at least three relationships. For example, A and / or B can at least indicate: the existence of A alone, the existence of both A and B, and the existence of B alone.

[0071] It should be noted that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of this disclosure is not limited by the order in which the steps are described in the relevant embodiments.

[0072] The surface anomaly detection method of this invention uses a cascaded reconstruction-discrimination network (CRDN) for unsupervised surface anomaly detection. For example... Figure 1 As shown, the network consists of two reconstruction subnetworks and their respective discriminant subnetworks. The two reconstruction subnetworks are cascaded, enabling better reconstruction of the input image across different anomaly types and image regions. Compared to a single reconstruction network, the cascaded paradigm enhances the network's reconstruction capability. Then, the two anomaly segmentation masks obtained from the two discriminant subnetworks are averaged pixel-wise to obtain an accurate anomaly segmentation mask. In this network, a bidirectional channel attention module (BiCA) is proposed. In this module, the two discriminant subnetworks mutually guide and enhance detection performance. Considering that surface anomalies include both texture and structural anomalies, a general structural anomaly generation process (SAG) is designed to simulate real structural anomalies, enabling the network to learn the reconstruction of structural anomalies. SAG uses nonlinear transformations to implement four representative structural variation types. The proposed structural variations are general and can be applied to any category containing structural anomalies.

[0073] 1. Cascaded Reconstruction-Discriminative Network

[0074] The Cascade Reconstruction-Discriminant Network (CRDN) consists of two reconstruction subnetworks and two discriminative subnetworks. The first reconstruction subnetwork is trained to detect and reconstruct anomalous regions while preserving normal regions. To enhance the network's ability to more accurately reconstruct the original image, another reconstruction subnetwork is further cascaded. For the outputs of the two reconstruction subnetworks, the discriminative subnetwork generates an anomaly score map from the channel-dimensionally stitched reconstructed image and the original image. The two anomaly score maps are averaged pixel-wise to produce a more accurate final result. Anomaly training samples are created on anomaly-free images using the proposed SAG and existing Texture Anomaly Generation (TAG) processes. The end-to-end CRDN network is jointly trained without real anomaly samples. The TAG process is based on the DRAEM method paper.

[0075] Both reconstruction subnetworks employ the same U-Net encoder-decoder architecture to transform the local patterns of the input image into patterns closer to a normal sample distribution. The first reconstruction subnetwork (R1) reconstructs anomalous regions in the input image and generates an image with a normal appearance. However, the reconstructed image is not accurate enough, so a second cascaded reconstruction subnetwork (R2) is added to further reconstruct the output image. The second reconstruction subnetwork can reconstruct inaccurate regions in the image. Furthermore, the two reconstruction networks can learn to focus on reconstructing different types of anomalous features, thus reconstructing the image... The results are relatively more accurate across different regions. Therefore, pixel-wise averaging of the anomaly score maps M1 and M2 generated by the two networks is performed to obtain more accurate results. The reconstruction loss uses pixel-level L2 loss and patch-based SSIM (Wang et al. 2004) loss.

[0076] For reconstructing image I r The channel dimensions of the input image I are concatenated (two reconstructed images). (Operations are the same), a U-Net-based network is used to identify subnetworks for anomalous segmentation. I and I r The stitching results show significant differences in the anomalous region, providing sufficient information for anomalous segmentation. Based on a joint learning strategy, the reconstruction sub-network learns discriminative reconstructed images, while the discriminative sub-network learns appropriate distance metrics. The discriminative sub-networks D1 and D2 output anomalous score maps M1 and M2 with the same size as I. Through the BiCA module, the effective features of the two sub-networks are integrated to learn more effective discriminative features. Focal Loss (Lin et al, 2017) (L...) dis It is used as a loss function for anomaly segmentation to improve the detection accuracy of difficult samples.

[0077] For the anomaly classification subtask, average pooling is applied to remove noise from M. The image-level anomaly score η is obtained by maximizing the smoothed anomaly score map. For normal samples, there is no significant response in the anomaly map.

[0078] When training the cascaded reconstruction-discrimination network, the reconstruction sub-network of this invention uses pixel-level Euclidean distance. Loss and image patch-based Loss, total loss This is the weighted sum of the two parts. The formula is as follows:

[0079]

[0080]

[0081] Where I and I r These represent the input image and the reconstructed image, respectively. Np =H·W represents the number of pixels in the image, and i and j are the pixel positions in the image. λ is a hyperparameter used to balance the two parts of the loss.

[0082] The discriminant subnetwork uses Focal Loss as its loss function. Therefore, the total network loss can be written as:

[0083]

[0084] in M represents the image reconstructed by the two reconstruction sub-networks. a The labels represent the segmentation maps; M1 and M2 are the segmentation maps output by the two discriminant subnetworks.

[0085] 2. Bidirectional Attention Module

[0086] To enable the two discriminative subnetworks to learn from each other and acquire more discriminative features, based on conventional channel attention (Hu, Shen and Sun, 2018), this invention proposes a bidirectional channel attention module (BiCA). Figure 2 The segmentation network U-Net consists of five stages of downsampling and upsampling, producing five feature layers of different sizes. For each layer in the first four stages, a BiCA module is used.

[0087] Specifically, for each upsampling stage i of the discriminant sub-network U-Net, Let D1 represent the feature maps of the encoder and decoder stages of the discriminant subnetwork D2, respectively. The process of using a bidirectional channel attention module for the discriminant subnetwork D1 is formally represented as follows:

[0088]

[0089]

[0090]

[0091]

[0092] Where [·,·] represents the concatenation of channel dimensions, UP represents bilinear interpolation, Conv represents 3×3 convolution, batch normalization (BN), and activation function ReLU, and l and h represent the low-level features of the encoder and the high-level features of the decoder, respectively.

[0093] The characteristics of the discriminant subnetwork D1 are obtained through a process similar to that in formula (4). AvgPool represents global average pooling. δ represents the ReLU function. and This is a learnable parameter matrix. r is the decay rate, and σ is the sigmoid activation function. The output feature map... It is obtained by pixel-level matrix multiplication and residual connections. Feature map of discriminant subnetwork D2. The operation process is symmetrical.

[0094] 3. Structural Anomaly Generation Process

[0095] Structural anomaly generation does not require simulating the true appearance of anomalies in the target domain. Instead, it generates appearances that are just out of distribution, allowing the discriminant subnetwork to learn an appropriate distance function to identify anomalies. Texture anomaly generation (TAG) randomly generates texture anomalies of different shapes, colors, and appearances on normal samples. However, surface anomalies include not only texture anomalies but also structural anomalies such as deformation and damage. TAG only focuses on texture anomalies, generating anomalous samples only in terms of texture. Therefore, for real samples with structural anomalies, DRAEM struggles to reconstruct anomalous regions and predict accurate anomaly segmentation. To address these issues, this invention proposes a novel structural anomaly generation (SAG), which helps the network reconstruct the appearance of structurally anomalous regions as normal samples. The entire generation process is as follows: Figure 3 As shown, SAG utilizes four representative nonlinear transformation structural variations: squeezing, amplification, vortex, and wave. These structural anomalies may not necessarily resemble real-world anomalies, but they effectively enhance the network's ability to learn structural variations.

[0096] The generation of structural anomalies employs four nonlinear transformations: compression, magnification, rotation, and wave. (See below) Figure 3 .

[0097] Extrusion creates the effect of compressing the surface of an object. Let the center of the extrusion be O. For any point P(r,θ) in the polar coordinate system image, the angle θ remains unchanged after transformation, and the polar radius r is transformed into r′:

[0098]

[0099] Where ratio p It is a parameter that adjusts the extrusion ratio.

[0100] Similarly, the formula for magnification is as follows:

[0101]

[0102] Where R is the radius of the magnified circular region, and ratio m Adjust the amplification ratio.

[0103] Under the vortex effect transformation, the polar radius r of a point remains unchanged, and the conversion formula for the polar angle is as follows:

[0104]

[0105] Where ratio v Control the amplitude of rotation. As can be seen from the formula, the farther a point is from the center point O, the greater the amplitude of the angle change.

[0106] For the wave effect, this invention uses a trigonometric sine function.

[0107] Finally, in the anomaly generation process, three polygons (achieved by calculating the convex hull of random points) and a circular region are first randomly generated as the segmentation label M. a Then, four nonlinear transformations SA are applied to the four regions respectively, while the other regions remain unchanged. The entire process can be formally represented as:

[0108]

[0109] in Indicates element-wise multiplication, I n This represents the original input image.

[0110] Based on the above, an example of a surface anomaly detection method based on a cascaded reconstruction-discrimination structure is presented, such as... Figure 4 As shown, it includes the following steps.

[0111] Step 1: Reconstruct the image to be detected to obtain the first reconstructed image.

[0112] Step 1.1: Construct the first reconstructed subnetwork.

[0113] The first reconstructed subnetwork adopts the encoder-decoder architecture of U-Net.

[0114] Step 1.2: Generate a texture anomaly image of the original image.

[0115] Step 1.3: Perform a nonlinear transformation on the original image to obtain the squeezed structure image, the magnified structure image, the rotated structure image, and the wave structure image.

[0116] Step 1.4: Randomly generate 3 polygons and 1 circular region to segment label M. a .

[0117] Step 1.5: Based on the segmented label M aThe original image is segmented, and a squeezed structure image, a magnified structure image, a rotated structure image, and a wave structure image are applied to the last four regions respectively, while the other regions remain unchanged, to obtain a structure anomaly image.

[0118] Step 1.6: Construct a training dataset based on texture anomaly images and structural anomaly images.

[0119] Step 1.7: Train the first reconstructed subnetwork based on the training dataset.

[0120] Step 1.8: Input the image to be detected into the trained first reconstruction network to obtain the first reconstructed image.

[0121] Step 2: Reconstruct the first reconstructed image to obtain the second reconstructed image.

[0122] Since the reconstruction process of the first reconstructed image is basically the same as the reconstruction process of the image to be detected in step 1, it will not be described again in this paper.

[0123] Step 3: Concatenate the image to be detected with the first reconstructed image and the second reconstructed image from the channel dimension to obtain the first anomaly score map and the second anomaly score map.

[0124] Since the stitching process between the image to be detected and the first reconstructed image, or the second reconstructed image, is the same, this paper only discusses the stitching process between the image to be detected and the first reconstructed image. The specific details are as follows:

[0125] Step 3.1: Construct the first discriminative subnetwork based on the bidirectional channel attention module; the first discriminative subnetwork adopts the U-Net structure, which consists of n downsampling and upsampling stages, generating n feature layers of different sizes, and each layer in the first n-1 stages uses the bidirectional channel attention module;

[0126] Step 3.2: Generate texture anomaly images and structural anomaly images to construct the training dataset;

[0127] Step 3.3: Train the first discriminative sub-network based on the training dataset;

[0128] Step 3.4: Input the image to be detected and the first reconstructed image into the trained first reconstruction network to obtain the first anomaly score map.

[0129] Step 4: By averaging the first and second anomaly score maps pixel by pixel, the anomaly detection result of the image to be detected is obtained.

[0130] Step 4.1: Average the first and second anomaly score maps pixel by pixel to obtain the anomaly score map;

[0131] Step 4.2: Apply average pooling to remove noise from the anomaly score map to obtain a smoothed anomaly score map;

[0132] Step 4.3: Obtain the maximum value in the smoothed anomaly score map to obtain the image-level anomaly score;

[0133] Step 4.4: Compare the image-level anomaly score with a threshold to obtain the apparent anomaly detection result of the image to be detected.

[0134] To verify the effectiveness of the present invention, experiments were conducted on the publicly available unsupervised surface anomaly detection datasets MVTecAD and BTAD.

[0135] The MVTec AD dataset is referenced from "MVTec AD: A comprehensive real-world dataset for unsupervised anomaly detection," Bergmann P, Fauser M, Sattlegger D, et al., in Proc. CVPR 2019. The BTAD dataset is referenced from "VT-ADL: A vision transformer network for image anomaly detection and localization," Mishra P, Verk R, Fornasier D, et al., in ISIE 2021.

[0136] (1) Experimental setup

[0137] During training and testing, all images were resized to 256×256. Similar to the DRAEM method, the network was trained for 700 epochs with a batch size of 8. The learning rate was set to 10⁻⁴ and multiplied by 0.1 after 400 and 600 epochs. Image rotation (-45, 45) was used as data augmentation to mitigate overfitting. TAG and SAG were used on the MVTec AD dataset during training, while only SAG was used on the BTAD dataset.

[0138] (2) Experimental Results

[0139] The quantitative results of pixel-level anomaly localization on the BTAD dataset are shown in Table 1 below. CRDN significantly outperforms all recent methods (including VT-ADL (Mishra et al. 2021) and (Tsai et al. 2022)), achieving the best results on all evaluation metrics: AUROC, PRO, and AP. It surpasses the state-of-the-art method by 8.4% on the PRO metric and by 7.8% on the AP metric. Figure 5 The sample segmentation in the image demonstrates the qualitative results of some samples. CRDN achieves accurate anomaly segmentation, even better than the results from the label. Figure 5 (Second column).

[0140] The results of image-level anomaly detection are shown in Table 2 below. CRDN achieved a peak performance of 96.5% in AUROC. It achieved the best results in product 02 due to its more accurate detection of small anomalies.

[0141]

[0142] Table 1

[0143]

[0144] Table 2

[0145] Tables 3 and 4 present the quantitative pixel-level anomaly localization results on the MVTecAD dataset. CRDN outperforms all current surface anomaly detection methods, achieving top performance across all three metrics: AUROC, PRO, and AP. It surpasses state-of-the-art methods by 9.3% in AP. It achieves better AP scores in 11 out of 15 categories, comparable to the highest levels in other categories. Notably, the method of this invention achieves the highest scores in all texture categories. Qualitative comparisons with state-of-the-art methods DRAEM, RDistillation, and PatchCore are shown below. Figure 6 As shown, while representation-based methods such as RDistillation and PatchCore can coarsely locate the main parts of anomaly regions, inaccurate transition regions with low anomaly scores exist outside the edges of the anomaly regions. There are also many erroneous responses in anomaly-free regions. The DRAEM method predicts many erroneous anomaly regions in classes with structural anomalies, such as carpets, toothbrushes, and zippers. CRDN is able to predict accurate masks of structural anomalies because the network learns to detect deformations and distortions due to the structural anomalies of SAGs. Due to its strong reconstruction and discrimination capabilities, it predicts more accurate masks than DRAEM. CRDN achieves a significant improvement in anomaly segmentation accuracy.

[0146] The second row of Table 4 shows the quantitative results of CRDN compared to state-of-the-art methods for image-level anomaly classification tasks. CRDN achieved a result comparable to the previous best-performing method with an AUROC of 98.6%.

[0147]

[0148] Table 3

[0149]

[0150] Table 4

[0151] Furthermore, an example of the present invention also discloses a surface anomaly detection device based on a cascaded reconstruction-discrimination structure, the device comprising: a first reconstruction module, a second reconstruction module, a first discrimination module, a second discrimination module, and a result generation module.

[0152] The first reconstruction module is used to reconstruct the image to be detected to obtain the first reconstructed image.

[0153] The second reconstruction module is used to reconstruct the first reconstructed image to obtain the second reconstructed image;

[0154] The first discrimination module is used to concatenate the image to be detected with the first reconstructed image from the channel dimension to obtain a first anomaly score image;

[0155] The second discrimination module is used to concatenate the image to be detected and the second reconstructed image along the channel dimension to obtain a second anomaly score image.

[0156] The result generation module is used to obtain the apparent anomaly detection result of the image to be detected by averaging the first anomaly score map and the second anomaly score map pixel by pixel.

[0157] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A surface anomaly detection method based on a cascade-based reconstruction-discriminative structure, characterized in that, The method comprises: reconstructing the to-be-detected image to obtain a first reconstructed image; reconstructing the first reconstructed image to obtain a second reconstructed image; splicing the to-be-detected image, the first reconstructed image and the second reconstructed image from a channel dimension to obtain a first anomaly score map and a second anomaly score map; obtaining an anomaly detection result of the to-be-detected image by pixel-wise average processing of the first anomaly score map and the second anomaly score map; wherein the reconstructing the to-be-detected image to obtain a first reconstructed image comprises: constructing a first reconstruction subnetwork, wherein the first reconstruction subnetwork adopts an encoder-decoder architecture of U-Net; generating a texture anomaly image of the original image; performing nonlinear transformation on the original image to obtain a squeezed structure image, an enlarged structure image, a rotated structure image and a wave structure image; Randomly generate 3 polygonal and one circular regions to segment the label ; based on the segmentation label segmenting the original image, and applying the extrusion structure image, the magnification structure image, the rotation structure image and the wave structure image in the four segmented regions respectively, and keeping other regions unchanged to obtain a structure anomaly image; constructing a training data set based on the texture anomaly image and the structure anomaly image; training the first reconstruction subnetwork based on the training data set; inputting the to-be-detected image into the trained first reconstruction network to obtain the first reconstructed image; the splicing the to-be-detected image and the first reconstructed image from the channel dimension to obtain the first anomaly score map comprises: A first discriminant subnetwork based on a bidirectional channel attention module is constructed; the first discriminant subnetwork adopts a U-Net structure, which is composed of a down-sampling stage and an up-sampling stage, generates a plurality of feature layers of different sizes, and each layer in the first stage uses a bidirectional channel attention module. ​​​ generating a texture anomaly image and a structure anomaly image to construct a training data set; training the first discrimination subnetwork based on the training data set; inputting the to-be-detected image and the first reconstructed image into the trained first reconstruction network to obtain the first anomaly score map.

2. The method of claim 1, wherein, the nonlinear transformation on the original image to obtain the squeezed structure image comprises: placing the original image in a polar coordinate system, and setting a squeezing center at the origin of the polar coordinate system; Obtaining parameters for adjusting the ratio of extrusion ; For any point of the original image , the angle is kept unchanged and the polar radius after the extrusion is calculated to obtain the polar coordinate of the point after the extrusion ;​ obtaining the squeezed structure image after completing the squeezing change of each point of the original image.

3. The method of claim 1, wherein, the nonlinear transformation on the original image to obtain the enlarged structure image comprises: placing the original image in a polar coordinate system, and setting an enlargement center at the origin of the polar coordinate system; Obtain the parameter for adjusting the amplification ratio. For any point in the original image Maintain angle The radius remains unchanged, and the polar radius after the amplification change is calculated. To obtain that point Magnified polar coordinates ;in, Indicates the radius of the enlarged circular region; obtaining the enlarged structure image after completing the enlargement change of each point of the original image.

4. The method of claim 1, wherein, the nonlinear transformation on the original image to obtain the rotated structure image comprises: placing the original image in a polar coordinate system, and setting a rotation center at the origin of the polar coordinate system; Obtaining parameters regulating the ratio of rotation ; For any point of the original image , the polar radius is kept unchanged and the polar angle is calculated after rotation to get the polar coordinates of the point after rotation ; wherein , R represents the radius of the enlarged circular region ; and , θ represents the polar angle of the point after rotation . obtaining the rotated structure image after completing the rotation change of each point of the original image.

5. The method of claim 1, wherein, the nonlinear transformation on the original image to obtain the wave structure image comprises: placing the image in a two-dimensional rectangular coordinate system; Obtaining amplitudes of trigonometric functions and frequency parameters ; For any point of the original image , keeping the horizontal coordinate unchanged, the changed vertical coordinate is calculated to obtain the changed coordinate of the point .

6. The method of claim 1, wherein, loss ; wherein, represents an original image in the training data set, represents an image output by the first reconstruction sub-network, represents a weight, represents a pixel-level L2 loss, a structure similarity loss based on an image block , is a pixel number in the original image, and is a pixel position in the image.

7. The method of claim 1, wherein, the obtaining an anomaly detection result of the to-be-detected image by pixel-wise average processing of the first anomaly score map and the second anomaly score map comprises: pixel-wise average processing the first anomaly score map and the second anomaly score map to obtain an anomaly score map; applying an average pooling operation to remove noise in the anomaly score map to obtain a smoothed anomaly score map; obtaining a maximum value in the smoothed anomaly score map to obtain an image-level anomaly score; comparing the image-level anomaly score with a threshold to obtain the anomaly detection result of the to-be-detected image.

8. A surface anomaly detection apparatus based on a cascade-based reconstruction-discriminative structure, characterized by, The device comprises: The first reconstruction module is configured to reconstruct the to-be-detected image to obtain a first reconstructed image. The second reconstruction module is configured to reconstruct the first reconstructed image to obtain a second reconstructed image. The first discrimination module is configured to splice the to-be-detected image and the first reconstructed image from a channel dimension to obtain a first anomaly score map. The second discrimination module is configured to splice the to-be-detected image and the second reconstructed image from a channel dimension to obtain a second anomaly score map. The result generation module is configured to obtain an anomaly detection result of the to-be-detected image by performing pixel-by-pixel average processing on the first anomaly score map and the second anomaly score map. The reconstruction of the to-be-detected image to obtain the first reconstructed image comprises: constructing a first reconstruction subnetwork, wherein the first reconstruction subnetwork adopts an encoder-decoder architecture of U-Net; generating a texture anomaly image of the original image; performing nonlinear transformation on the original image to obtain a squeezed structure image, an enlarged structure image, a rotated structure image, and a wave structure image; Randomly generate 3 polygonal and one circular regions to segment the label ; based on the segmentation label segmenting the original image, and applying the extrusion structure image, the magnification structure image, the rotation structure image and the wave structure image in the four segmented regions respectively, and keeping other regions unchanged to obtain a structure anomaly image; constructing a training data set based on the texture anomaly image and the structure anomaly image; training the first reconstruction subnetwork based on the training data set; inputting the to-be-detected image into the trained first reconstruction network to obtain the first reconstructed image; The splicing of the to-be-detected image and the first reconstructed image from the channel dimension to obtain the first anomaly score map comprises: A first discriminant subnetwork based on a bidirectional channel attention module is constructed; the first discriminant subnetwork adopts a U-Net structure, which is composed of a down-sampling stage and an up-sampling stage, generates a plurality of feature layers of different sizes, and each layer in the former stage uses a bidirectional channel attention module. ​​​ generating a texture anomaly image and a structure anomaly image to construct a training data set; training the first discrimination subnetwork based on the training data set; inputting the to-be-detected image and the first reconstructed image into the trained first reconstruction network to obtain the first anomaly score map.

Images