Differential trainable adaptive memory type defect detection and positioning method

Abstract

The invention provides a difference-trainable adaptive memory type defect detection and positioning method, and relates to the technical field of industrial defect detection. The method comprises the following steps: firstly, acquiring normal image data from an existing image data set; selecting and storing memory feature information of the acquired normal image data through a memory information construction module; generating an abnormal sample through data enhancement, and performing feature extraction on the abnormal sample by using an encoder; the feature matching and connecting module is used for matching the sample features extracted by the encoder with the memory feature information, and then connecting with the sample features extracted by the encoder to obtain connecting information; sending the data to a multi-level feature fusion module, and sending the data to a decoder for decoding; obtaining a final prediction segmentation map by using difference calculation and a segmentation network; and finally, comparing the prediction segmentation image with the mask image to obtain loss and training the loss so as to complete anomaly detection and positioning. The method has a good defect positioning effect and can meet the requirements of industrial production.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial defect detection, and in particular to a difference-trainable adaptive memory-type defect detection and positioning method. Background Art

[0002] In today's industrial production, surface defect detection is a growing trend. Surface defect detection involves determining whether an image contains abnormal areas and locating them. However, in reality, abnormal samples are rare and difficult to collect, and there is a lack of effectively utilized supervisory information. Abnormal samples and abnormal patterns often have variable forms and lack stable statistical regularities. This makes it difficult for supervised learning models to capture sufficient statistical information or significant features about abnormal patterns. Therefore, most current anomaly detection methods are unsupervised, trained only on normal samples. This offers significant advantages in practical applications.

[0003] In unsupervised surface defect detection, models do not use any labels. Instead, they rely on features learned from the data itself. Unsupervised surface defect detection methods include those based on image reconstruction, generative models, and deep feature embedding. For example, image reconstruction methods based on autoencoders reconstruct defect-free images from the original image and then identify defects by comparing the original and reconstructed images. Generative models based on generative adversarial networks use two neural networks: a generator and a discriminator. The generator learns to generate realistic samples from an underlying noise distribution, while the discriminator learns to distinguish between generated samples and real samples. Deep feature embedding methods generally reduce the dimensionality of the data while preserving important features, and then detect and localize anomalies. However, deep feature embedding methods have difficulty learning feature representations and poor model interpretability. Generative model methods are complex to train and may suffer from instability during training. Image reconstruction methods may lose or blur some information, resulting in low model efficiency and instability. Therefore, research on industrial defect detection is still in its infancy, and the corresponding basic theoretical and methodological frameworks are still lacking. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention provides a difference-trainable adaptive memory-type defect detection and positioning method to achieve defect detection.

[0005] The technical solution adopted by the present invention is:

[0006] A difference-trainable adaptive memory-based defect detection and location method comprises the following steps:

[0007] Step 1: Obtain normal image data, i.e., defect-free image data, from an existing image dataset;

[0008] The existing image dataset comprises a total of M image datasets of different categories, wherein m are object categories and n are texture categories, where m+n=M; the image datasets of each category are divided into a training set, a test set, and a label dataset, wherein the training set comprises only normal images, and the test set comprises normal images and abnormal images, wherein the abnormal images are defective images; the images in the label dataset are black and white images, and the defect locations of the abnormal images in the test set are marked; the obtaining of normal image data from the existing image dataset refers to obtaining normal image data I from the training set;

[0009] Step 2: The acquired normal image data is used to select and store memory feature information through a memory information construction module;

[0010] Step 2-1: Use the encoder to extract features from the input normal image data;

[0011] The encoder is composed of the first, second, and third layer features output by the pre-trained ResNet18 network, denoted by F1, F2, and F3 respectively;

[0012] Step 2-2, perform k-means clustering on the features extracted by the encoder;

[0013] Step 2-3, obtain the optimal number of cluster center points;

[0014] Specifically, the silhouette coefficient is calculated for the k-means clustering results to obtain the optimal number of cluster centers, and then the cluster center features are obtained and stored. The silhouette coefficient describes the index of the outline clarity of each category after clustering. The calculation formula of the silhouette coefficient is as follows:

[0015]

[0016] Among them, S i is the silhouette coefficient, f j Representative and sample features f i Other sample features in the same class, a (i) is the cohesion, that is, the average distance of the input sample feature to other features within the same category, b (i) is the separation degree, that is, the average distance between the input sample features and the features in other classes, and N is the number of samples in the class;

[0017] Step 2-4: Obtain hard sample features and store memory feature information;

[0018] The hard sample feature is the sample feature farthest from the cluster center feature in each class selected in steps 2-3 calculated by Euclidean distance;

[0019] The memory feature information includes hard sample features and cluster center features;

[0020] Step 3: Perform data enhancement on the acquired normal image data I to obtain a synthetic abnormal image I a ;

[0021] Step 3-1: Randomly generate two-dimensional Perlin noise and perform binarization on the randomly generated two-dimensional Perlin noise to obtain the initial mask image M p ;

[0022] Step 3-2: In the initial mask image M p Randomly crop two rectangular blocks of random size to obtain the final mask image Mask;

[0023] Step 3-3, the mask image Mask and the randomly selected texture image or structure image I n Multiply to get I n1 ;

[0024] The texture image is from the FMD texture dataset, and the structural image is from the normal image data I, which is obtained by randomly adjusting the normal image data I, wherein the random adjustment includes one or more of brightness, contrast, sharpness, color, and rotation;

[0025] Step 3-4: Invert the mask image Mask and multiply it with the normal image data I to obtain I1;

[0026] Step 3-5: Introduce the transparency factor δ to diversify the defects, thereby obtaining the enhanced synthetic abnormal image I a , the specific formula is:

[0027] I a =δI n1 +I1

[0028] Step 4: The synthetic abnormal image I obtained after data enhancement a The feature is sent to the encoder for feature extraction, and the L2 distance between the extracted feature and the stored memory feature information is calculated. The feature with the smallest distance is the best matching feature. The best matching feature is then connected in series with the feature extracted by the encoder to obtain three levels of connection information CI1, CI2 and CI3;

[0029] Step 5: Perform multi-scale feature fusion on the three-level connection information CI1, CI2 and CI3;

[0030] Step 5-1: The connection information CI3 undergoes two 3×3 convolutions to reduce its channel number to 256, and the fused feature CI3′ is obtained;

[0031] Step 5-2, the connection information CI2 is subjected to two 3×3 convolutions to make its channel number 128, and we get CI3′ is then upsampled and passed through a 3×3 convolution to make its size and number of channels the same as Align and add to obtain the fusion feature CI2′;

[0032] Step 5-3, the connection information CI1 is subjected to two 3×3 convolutions to make its number of channels 64, and we get Then Upsample and pass 3×3 convolution to make its size and number of channels the same Align and add to obtain the fusion feature CI1′;

[0033] Step 6: Send to the decoder for decoding;

[0034] Step 6-1: Perform a 3×3 convolution on CI3′ to obtain F3′, which is the first layer feature of the decoder;

[0035] Step 6-2: Upsample F3′ and perform a 3×3 convolution to align the size with CI2′. Then add the two and perform a 3×3 convolution to obtain F2′, which is the second layer feature of the decoder.

[0036] Step 6-3: Upsample F2′ and perform a 3×3 convolution to align the size with CI1′. Then add the two and perform a 3×3 convolution to obtain F1′, which is the third layer feature of the decoder.

[0037] Step 7: Calculate the difference and feed it into the segmentation network to predict the final segmentation map;

[0038] Step 7-1: Subtract F3′ from the input feature F3, then perform 3×3 convolution and upsampling to obtain segmentation. Figure 1 ;

[0039] Step 7-2, subtract F2′ from the input feature F2, and then add it to the segmentation Figure 1 Multiply, then perform 3×3 convolution and upsampling to get the segmentation Figure 2 ;

[0040] Step 7-3, subtract F1′ from the input feature F1, and then add it to the segmentation Figure 2 The preliminary segmentation map is obtained by multiplication and is input into the segmentation network to obtain the final predicted segmentation map.

[0041] The segmentation network includes 5 batch modules, namely the first batch module to the fifth batch module, 2 LeakyRELU modules, namely the first LeakyRELU module and the second LeakyRELU module, and 2 upsampling modules, namely the first upsampling module and the second upsampling module;

[0042] The preliminary segmentation map passes through the first batch module and the second batch module in sequence, and is added with the output of the third batch module. The added result then passes through the first LeakyRELU module, the first upsampling module, and the fourth batch module. The output of the fourth batch module is added to the output of the first upsampling module, and the added result is then sent to the second LeakyRELU module, the second upsampling module, and the fifth batch module to output the predicted segmentation map;

[0043] The first to fourth batch modules are composed of 3×3 convolution, batch normalization layer and LeakyRELU activation function, and the fifth batch module is composed of 3×3 convolution, batch normalization layer and Softplus activation function;

[0044] Step 8: Calculate the loss and perform network training;

[0045] The calculation loss is specifically as follows: calculating the L1 loss and Focal loss between the predicted segmentation map and the obtained mask image Mask, and summing them up to form the total loss function. When the total loss is minimized, defect detection and positioning are achieved;

[0046] The total loss function formula is as follows:

[0047] L 总 =λ1L1+λ f L f

[0048] Among them, L 总 is the total loss function, L1 is the L1 loss, L f is the Focal loss, λ1 and λ f is the corresponding weight.

[0049] The beneficial effects of adopting the above technical solution are:

[0050] The present invention provides a differentially trainable adaptive memory-type defect detection and positioning method. The present invention provides a differentially trainable adaptive memory-type defect detection and positioning method. (1) A new enhancement method is proposed to generate defect samples outside the original normal samples, thereby increasing the diversity of samples, making up for the current problem of the scarcity of abnormal samples, and improving the efficiency of the model. (2) A new memory module construction method is proposed. Unlike the traditional memory module construction method, it not only solves the problem that the traditional method lacks representativeness in storing sample features and cannot cover the data distribution of the entire data set or consumes a large amount of memory resources, but also enhances the reasoning and cognitive ability of the model. (3) A trainable difference calculation method is proposed, which integrates multi-level information, so that feature expression is enhanced and information loss is reduced; unlike the traditional image-level difference calculation, we perform difference calculation at the feature level and send it to the segmentation network for training, which can accurately predict the segmented image and improve the model's ability to distinguish and locate defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flow chart of a difference-trainable adaptive memory-type defect detection and location method provided by an embodiment of the present invention;

[0052] Figure 2 A diagram of a segmented network structure provided by an embodiment of the present invention;

[0053] Figure 3 This is a diagram showing the effect of using the method of the present invention to perform defect detection on data in 15 types of data sets in MVTec AD, provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0055] A difference trainable adaptive memory type defect detection and location method, this embodiment takes the MVTec AD dataset as an example, Figure 1 As shown, the following steps are included:

[0056] Step 1: Obtain normal image data, i.e., defect-free image data, from an existing image dataset;

[0057] This embodiment uses the MVTec AD dataset, which was proposed at the 2019 CVPR conference and is specifically used to evaluate and test the performance of unsupervised image abnormal pattern detection and segmentation. It contains 15 image datasets of different categories, 10 of which are object categories and 5 are texture categories, with a total of 5354 high-resolution images. Each category dataset is divided into a training set, a test set, and a labeled dataset. The training set contains only normal images, while the test set contains both normal images and real abnormal images. Abnormal images are defect images. The labeled dataset contains black and white images of abnormal images in the test set, with the defect locations marked. Obtaining normal image data from the MVTec AD dataset refers to normal image data I obtained from the training set.

[0058] The MVTec AD dataset includes 10 object categories: Bottle, Cable, Capsule, Hazelnut, Screw, Toothbrush, Pill, Zipper, Transistor, and Metal Nut; and 5 texture categories: Carpet, Grid, Leather, Tile, and Wood.

[0059] Step 2: The acquired normal image data is used to select and store memory feature information through a memory information construction module;

[0060] Step 2-1: Use the encoder to extract features from the input normal image data;

[0061] The encoder is composed of the first, second, and third layer features output by the pre-trained ResNet18 network, denoted by F1, F2, and F3 respectively. In this embodiment, the sample feature sizes extracted in each layer are 64×64×64, 128×32×32, and 256×16×16 respectively.

[0062] Step 2-2, perform k-means clustering on the features extracted by the encoder;

[0063] In this embodiment, the value range of k is 20 to 200;

[0064] Step 2-3, obtain the optimal number of cluster center points;

[0065] Specifically, the silhouette coefficient is calculated for the results of k-means clustering to evaluate the effectiveness of k-means clustering, thereby obtaining the optimal number of cluster centers, and then obtaining and storing the characteristics of the cluster centers; the silhouette coefficient describes the index of the clarity of the outline of each category after clustering, and the clustering effect is evaluated by judging its size. The calculation formula of the silhouette coefficient is as follows:

[0066]

[0067] Among them, S i is the silhouette coefficient, f j Representative and sample features f i Other sample features in the same class, a (i) is the cohesion, that is, the average distance of the input sample feature to other features within the same category, b (i) is the separation degree, that is, the average distance between the input sample features and the features in other classes, and N is the number of samples in the class;

[0068] Step 2-4: Obtain hard sample features and store memory feature information;

[0069] The hard sample feature is the sample feature farthest from the cluster center feature in each class selected in steps 2-3 calculated by Euclidean distance;

[0070] The memory feature information includes hard sample features and cluster center features;

[0071] Step 3: Perform data enhancement on the acquired normal image data I to obtain a synthetic abnormal image I a ;

[0072] Step 3-1: Randomly generate two-dimensional Perlin noise and perform binarization on the randomly generated two-dimensional Perlin noise to obtain the initial mask image M p ;

[0073] Step 3-2: In the initial mask image M p Randomly crop two rectangular blocks of random size to obtain the final mask image Mask;

[0074] In this embodiment, the height and width of the cropped rectangular block range from 30 to 50;

[0075] Step 3-3, the mask image Mask and the randomly selected texture image or structure image I n Multiply to get I n1 ;

[0076] The texture image is from the FMD texture dataset, and the structural image is from the normal image data I, which is obtained by randomly adjusting the normal image data I, wherein the random adjustment includes one or more of brightness, contrast, sharpness, color, and rotation;

[0077] Step 3-4: Invert the mask image Mask and multiply it with the normal image data I to obtain I1;

[0078] Step 3-5: Introduce the transparency factor δ to diversify the defects, thereby obtaining the enhanced synthetic abnormal image I a , the specific formula is:

[0079] I a =δI n1 +I1

[0080] In this embodiment, the transparency factor δ introduced is a random number between 0.15 and 1;

[0081] Step 4: The synthetic abnormal image I obtained after data enhancement a The feature is sent to the encoder for feature extraction, and the L2 distance between the extracted feature and the stored memory feature information is calculated. The feature with the smallest distance is the best matching feature. The best matching feature is then connected in series with the feature extracted by the encoder to obtain three levels of connection information CI1, CI2 and CI3;

[0082] In this embodiment, the sample feature sizes of the obtained three levels of connection information CI1, CI2, and CI3 are 128×64×64, 256×32×32, and 512×16×16, respectively;

[0083] Step 5: Perform multi-scale feature fusion on the three-level connection information CI1, CI2 and CI3;

[0084] Step 5-1: The connection information CI3 undergoes two 3×3 convolutions to reduce its channel number to 256, and the fused feature CI3′ is obtained;

[0085] Step 5-2, the connection information CI2 is subjected to two 3×3 convolutions to make its channel number 128, and we get CI3′ is then upsampled and passed through a 3×3 convolution to make its size and number of channels the same as Align and add to obtain the fusion feature CI2′;

[0086] Step 5-3, the connection information CI1 is subjected to two 3×3 convolutions to make its number of channels 64, and we get Then Upsample and pass 3×3 convolution to make its size and number of channels the same Align and add to obtain the fusion feature CI1′;

[0087] In this embodiment, the sample feature sizes of CI1′, CI2′, and CI3′ are 64×64×64, 128×32×32, and 256×16×16, respectively;

[0088] Step 6: Send to the decoder for decoding;

[0089] Step 6-1: Perform a 3×3 convolution on CI3′ to obtain F3′, which is the first layer feature of the decoder;

[0090] Step 6-2: Upsample F3′ and perform a 3×3 convolution to align the size with CI2′. Then add the two and perform a 3×3 convolution to obtain F2′, which is the second layer feature of the decoder.

[0091] Step 6-3: Upsample F2′ and perform a 3×3 convolution to align the size with CI1′. Then add the two and perform a 3×3 convolution to obtain F1′, which is the third layer feature of the decoder.

[0092] Step 7: Calculate the difference and feed it into the segmentation network to predict the final segmentation map;

[0093] Step 7-1: Subtract F3′ from the input feature F3, then perform 3×3 convolution and upsampling to obtain segmentation. Figure 1 ;

[0094] Step 7-2, subtract F2′ from the input feature F2, and then add it to the segmentation Figure 1 Multiply, then perform 3×3 convolution and upsampling to get the segmentation Figure 2 ;

[0095] Step 7-3, subtract F1′ from the input feature F1, and then add it to the segmentation Figure 2 The preliminary segmentation map is obtained by multiplication and is input into the segmentation network to obtain the final predicted segmentation map.

[0096] The segmentation network structure is as follows Figure 2 As shown, it includes 5 batch modules, namely the first batch module to the fifth batch module, 2 LeakyRELU modules, namely the first LeakyRELU module and the second LeakyRELU module, and 2 upsampling modules, namely the first upsampling module and the second upsampling module;

[0097] The preliminary segmentation map passes through the first batch module and the second batch module in sequence, and is added with the output of the third batch module. The added result then passes through the first LeakyRELU module, the first upsampling module, and the fourth batch module. The output of the fourth batch module is added to the output of the first upsampling module, and the added result is then sent to the second LeakyRELU module, the second upsampling module, and the fifth batch module to output the predicted segmentation map;

[0098] The first to fourth batch modules are composed of 3×3 convolution, batch normalization layer and LeakyRELU activation function, and the fifth batch module is composed of 3×3 convolution, batch normalization layer and Softplus activation function;

[0099] Step 8: Calculate the loss and perform network training;

[0100] The calculation loss is specifically as follows: calculating the L1 loss and Focal loss between the predicted segmentation map and the obtained mask image Mask, and summing them up to form the total loss function. When the total loss is minimized, defect detection and positioning are achieved;

[0101] The total loss function formula is as follows:

[0102] L 总 =λ1L1+λ f L f

[0103] Among them, L 总 is the total loss function, L1 is the L1 loss, L f is the Focal loss, λ1 and λ f is the corresponding weight.

[0104] In this embodiment, λ1 and λ f The values ​​of are 0.6 and 0.4 respectively;

[0105] In this example, the method of the present invention is compared with existing methods on the MVTec AD dataset. Tables 1 and 2 show the image-level and pixel-level AUC-ROC scores of different methods, and Table 3 shows the pixel-level PRO scores of different methods.

[0106] Table 1. Image-level AUC-ROC anomaly detection performance comparison on the MVTec AD dataset

[0107]

[0108]

[0109] Table 2. Comparison of pixel-level AUC-ROC anomaly detection performance on the MVTec AD dataset

[0110]

[0111]

[0112] Table 3. Comparison of pixel-level AUC-PRO anomaly detection performance on the MVTec AD dataset

[0113]

[0114] As can be seen from Table 1, in 7 of the 15 categories, the image-level AUC-ROC score of this method is the highest, reaching 100%, and the average image-level AUC-ROC of 15 categories is 97.8%, which is significantly better than other methods. As can be seen from Tables 2 and 3, compared with other methods, the pixel-level AUC-ROC of this method is the highest in 4 categories, and the PRO score is mostly superior. In addition, the total average of the two methods is also the highest, which confirms its accuracy in anomaly positioning. The defect detection effect of this method is as follows Figure 3 As shown in the figure, from left to right, the categories are: carpet, mesh, leather, wood, tile, bottle, cable, capsule, hazelnut, metal nut, pill, screw, toothbrush, transistor, and zipper. The first row shows the abnormal sample, the second row shows the detection results of this method, and the third row shows the labels corresponding to the abnormal image. It can be seen that this method can effectively detect defects of all sizes, whether large or small, line or dot, semantic or logical errors, fully demonstrating the effectiveness of detection.

[0115] The above description is merely a preferred embodiment of the present disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A difference-trainable adaptive memory-based defect detection and location method, characterized in that: The following steps are involved: Step 1: Obtain normal image data, i.e., defect-free image data, from an existing image dataset; The existing image dataset contains a total of M image datasets of different categories, where m are object categories and n are texture categories, and m+n=M; Step 2: The acquired normal image data is used to select and store memory feature information through a memory information construction module; Step 3: Perform data enhancement on the acquired normal image data I to obtain a synthetic abnormal image I a ; Step 4: The synthetic abnormal image I obtained after data enhancement a The feature is sent to the encoder for feature extraction, and the L2 distance between the extracted feature and the stored memory feature information is calculated. The feature with the smallest distance is the best matching feature. The best matching feature is then connected in series with the feature extracted by the encoder to obtain three levels of connection information CI1, CI2 and CI3; Step 5: Perform multi-scale feature fusion on the three-level connection information CI1, CI2 and CI3; Step 6: Send to the decoder for decoding; Step 7: Calculate the difference and feed it into the segmentation network to predict the final segmentation map; Step 8: Calculate the loss and perform network training to achieve defect detection and location.

2. The difference-trainable adaptive memory-based defect detection and location method according to claim 1, characterized in that: The step 1 is specifically as follows: each category of image data sets are divided into a training set, a test set and a label data set, wherein the training set contains only normal images, and the test set contains normal images and abnormal images, and the abnormal images are defective images; the images in the label data set are black and white images, and the defect positions of the abnormal images in the test set are marked; the acquisition of normal image data from the existing image data set refers to acquiring normal image data I from the training set.

3. The difference trainable adaptive memory type defect detection and location method according to claim 1, characterized in that: The step 2 comprises the following steps: Step 2-1: Use the encoder to extract features from the input normal image data; The encoder is composed of the first, second, and third layer features output by the pre-trained ResNet18 network, denoted by F1, F2, and F3 respectively; Step 2-2, perform k-means clustering on the features extracted by the encoder; Step 2-3, obtain the optimal number of cluster center points; Specifically, the silhouette coefficient is calculated for the k-means clustering results to obtain the optimal number of cluster centers, and then the cluster center features are obtained and stored. The silhouette coefficient describes the index of the outline clarity of each category after clustering. The calculation formula of the silhouette coefficient is as follows: Among them, S i is the silhouette coefficient, f j Representative and sample features f i Other sample features in the same class, a (i) is the cohesion, that is, the average distance of the input sample feature to other features within the same category, b (i) is the separation degree, that is, the average distance between the input sample features and the features in other classes, and N is the number of samples in the class; Step 2-4: Obtain hard sample features and store memory feature information; The hard sample feature is the sample feature farthest from the cluster center feature in each class selected in steps 2-3 calculated by Euclidean distance; The memory feature information includes hard sample features and cluster center features.

4. The difference trainable adaptive memory type defect detection and location method according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3-1: Randomly generate two-dimensional Perlin noise and perform binarization on the randomly generated two-dimensional Perlin noise to obtain the initial mask image M p ; Step 3-2: In the initial mask image M p Randomly crop two rectangular blocks of random size to obtain the final mask image Mask; Step 3-3, the mask image Mask and the randomly selected texture image or structure image I n Multiply to get I n1 ; The texture image is from the FMD texture dataset, and the structural image is from the normal image data I, which is obtained by randomly adjusting the normal image data I, wherein the random adjustment includes one or more of brightness, contrast, sharpness, color, and rotation; Step 3-4: Invert the mask image Mask and multiply it with the normal image data I to obtain I1; Step 3-5: Introduce the transparency factor δ to diversify the defects, thereby obtaining the enhanced synthetic abnormal image I a , the specific formula is: I a =δI n1 +I1。 5. The difference trainable adaptive memory type defect detection and location method according to claim 1, characterized in that: The step 5 comprises the following steps: Step 5-1: The connection information CI3 undergoes two 3×3 convolutions to reduce its channel number to 256, and the fused feature CI3′ is obtained; Step 5-2, the connection information CI2 is subjected to two 3×3 convolutions to make its channel number 128, and we get CI3′ is then upsampled and passed through a 3×3 convolution to make its size and number of channels the same as Align and add to obtain the fusion feature CI2′; Step 5-3, the connection information CI1 is subjected to two 3×3 convolutions to make its number of channels 64, and we get Then Upsample and pass 3×3 convolution to make its size and number of channels the same Align and add to obtain the fusion feature CI1′.

6. The difference trainable adaptive memory type defect detection and location method according to claim 1, characterized in that: The step 6 comprises the following steps: Step 6-1: Perform a 3×3 convolution on CI3′ to obtain F3′, which is the first layer feature of the decoder; Step 6-2: Upsample F3′ and perform a 3×3 convolution to align the size with CI2′. Then add the two and perform a 3×3 convolution to obtain F2′, which is the second layer feature of the decoder. Step 6-3: Upsample F2′ and perform a 3×3 convolution to align the size with CI1′. Then add the two and perform a 3×3 convolution to obtain F1′, which is the third layer feature of the decoder.

7. The difference trainable adaptive memory type defect detection and location method according to claim 1, characterized in that: The step 7 comprises the following steps: Step 7-1: Subtract F3′ from the input feature F3, then perform 3×3 convolution and upsampling to obtain segmentation map 1; Step 7-2: Subtract F2′ from the input feature F2, multiply it by segmentation map 1, and then perform 3×3 convolution and upsampling to obtain segmentation map 2; Step 7-3: Subtract F1′ from the input feature F1, then multiply it by the segmentation map 2 to obtain a preliminary segmentation map, and input it into the segmentation network to obtain the final predicted segmentation map; The segmentation network includes 5 batch modules, namely the first batch module to the fifth batch module, 2 LeakyRELU modules, namely the first LeakyRELU module and the second LeakyRELU module, and 2 upsampling modules, namely the first upsampling module and the second upsampling module; The preliminary segmentation map passes through the first batch module and the second batch module in sequence, and is added with the output of the third batch module. The added result then passes through the first LeakyRELU module, the first upsampling module, and the fourth batch module. The output of the fourth batch module is added to the output of the first upsampling module, and the added result is then sent to the second LeakyRELU module, the second upsampling module, and the fifth batch module to output the predicted segmentation map; The first to fourth batch modules are composed of 3×3 convolution, batch normalization layer and LeakyRELU activation function, and the fifth batch module is composed of 3×3 convolution, batch normalization layer and Softplus activation function.

8. The difference trainable adaptive memory type defect detection and location method according to claim 1, characterized in that: The step 8 is specifically as follows: the calculation of loss is specifically as follows: calculating the L1 loss and Focal loss between the predicted segmentation map and the obtained mask image Mask, and summing them up to form the total loss function. When the total loss is minimized, defect detection and positioning are achieved; The total loss function formula is as follows: L 总 =λ1L1+λ f L f Among them, L 总 is the total loss function, L1 is the L1 loss, L f is the Focal loss, λ1 and λ f is the corresponding weight.

Images