A metal surface defect detection method for sample scarcity
By employing ResNet50 multilayer feature extraction and fusion, along with adaptive core feature selection, the problem of insufficient detection accuracy in metal surface defect detection is solved, enabling accurate identification and robust detection of minute defects.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUAIYIN INSTITUTE OF TECHNOLOGY
- Filing Date
- 2026-04-29
- Publication Date
- 2026-07-24
Smart Images
Figure CN122453759A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and industrial intelligent inspection technology, specifically to a method for detecting defects on metal surfaces where samples are scarce. Background Technology
[0002] In the field of industrial metal product quality inspection, the accurate identification and location of metal surface defects is of great significance for ensuring product quality and improving production efficiency. Among them, automated defect detection methods based on depth features are increasingly being applied to industrial vision inspection tasks to replace traditional manual inspection methods, thereby improving inspection efficiency and stability.
[0003] In scenarios involving few-shot anomaly detection, current mainstream methods typically utilize convolutional neural networks to extract features from normal sample images. Networks such as ResNet18 are used as the backbone for feature extraction, acquiring multi-scale feature information of the image through different layers of network structure. The extracted features are then used to construct a normal sample feature library or feature distribution model. During the detection phase, the distance or reconstruction error between the features of the image to be detected and the normal features is calculated to achieve the identification and localization of anomaly regions.
[0004] However, existing methods still have certain shortcomings in industrial metal surface defect detection. On the one hand, some methods mainly rely on single-level or mid-to-high-level semantic features for anomaly detection, paying insufficient attention to fine-grained structural changes and local texture anomalies in defect areas. When the number of training samples is limited, it is easy for minute defects to be difficult to accurately characterize, thus affecting the defect localization accuracy. On the other hand, in anomaly detection methods that only use normal samples for training, if all extracted features are directly modeled uniformly, unstable features caused by factors such as metal surface reflection, high-frequency noise, or uneven texture are easily included in the normal feature distribution description, resulting in feature redundancy and feature distribution shift. This leads to large reconstruction errors or false detections in the detection stage, reducing the accuracy and localization precision of the anomaly detection results. Summary of the Invention
[0005] To address the aforementioned technical issues, this technical solution provides a method for detecting metal surface defects with scarce samples. By using ResNet50 multi-layer feature extraction and fusion, combined with adaptive core feature selection and prototype-guided constraint reconstruction detection mechanisms, it achieves effective modeling of normal feature distribution, thereby reducing noise feature interference and improving the accuracy of metal surface defect detection and defect localization precision under conditions of few samples. This meets the application requirements of few-sample defect detection in industrial scenarios and effectively solves the technical problems.
[0006] This invention is achieved through the following technical solution:
[0007] A method for detecting surface defects in metals with scarce samples, applicable to surface defect detection scenarios in industrial metal products, includes the following steps:
[0008] Step 1: Acquire images of the metal surface, divide them into a normal sample image set and an image to be detected, and input each image into a pre-trained ResNet50 residual convolutional neural network for feature extraction. Extract multi-scale feature maps from the Layer 1, Layer 2, and Layer 3 residual layers of the network, respectively, and denoted as L1, L2, and L3.
[0009] Step 2: Input the feature maps L1, L2, and L3 obtained in Step 1 into Unit1, Unit2, and Unit3 respectively for local feature enhancement to obtain enhanced feature maps M1, M2, and M3. Then, concatenate the feature maps M1, M2, and M3 to obtain the fused feature maps MF1 and MF2.
[0010] Step 3: Combine the training and feature maps obtained in Step 2 Input the IAC image adaptive core feature construction module, and obtain the image-specific core feature set K and the final robust global prototype P through feature space partitioning, core feature selection based on image distribution and distance constraint processing;
[0011] Step 4: Develop the fused feature map obtained in Step 2. Input the LRPS local reconstruction and anomaly scoring module, combine the image-specific core feature set K obtained in step 3 with the final robust global prototype P, and perform prototype-guided weighted linear reconstruction on the query features to obtain the image sub-region anomaly score set S.
[0012] Step 5: Input the set of abnormal scores S of the image sub-regions obtained in Step 4 into the defect localization and visualization module. Through spatial mapping, interpolation upsampling and Gaussian filtering, an abnormal heat map H is generated to realize the localization and visualization of metal surface defects.
[0013] Furthermore, the pre-trained ResNet50 residual convolutional neural network mentioned in step 1 is a well-known ResNet50 residual convolutional neural network pre-trained based on the ImageNet dataset; the output features of the last residual unit in Layer 1, Layer 2 and Layer 3 of this network are selected as the multi-scale feature maps L1, L2 and L3 of the corresponding layers.
[0014] Furthermore, the specific operation method of step 2 includes the following steps:
[0015] S21: Perform local average pooling and local max pooling operations on each of the feature maps Li (i=1,2,3) received by Unit1, Unit2, and Unit3 respectively, and fuse the two pooling results to obtain the enhanced feature map Mi of the corresponding layer;
[0016] S22: Perform spatial size alignment on the enhanced feature maps M1, M2, and M3 to make their spatial resolution consistent;
[0017] S23: The spatially aligned feature maps M1, M2, and M3 are concatenated and fused along the channel dimension to obtain a fused feature map MF, wherein the fused feature map MF includes the training fused feature map corresponding to the normal sample image. The fused feature map corresponding to the image to be detected .
[0018] Furthermore, Unit1, Unit2, and Unit3 mentioned in step 2 are three feature enhancement processing units with the same structure, which are respectively applied to feature maps L1, L2, and L3 to perform local feature enhancement operations. The local feature enhancement operations include performing 3×3 average pooling and 3×3 max pooling operations on the first layer feature maps with a kernel size of 3×3, a stride of 1, and padding of 1, and then performing element-wise summation and feature fusion on the pooling results of the two operations.
[0019] Furthermore, the feature fusion process is as follows: First, spatial scale alignment is performed on the enhanced first feature map M1 and the third feature map M3 to ensure their spatial resolution is consistent with that of the second enhanced feature map M2. This spatial scale alignment is achieved through interpolation; in one embodiment, the interpolation method is bilinear interpolation. Subsequently, the spatially aligned M1, M2, and M3 are concatenated along the channel dimension to construct a fused feature map MF. During the training phase, the fused feature map is denoted as... This is used to construct a specific core feature set K for the image and a global prototype P; during the detection phase, the fused feature map is denoted as... It is used as the input for query features and subsequent feature reconstruction module.
[0020] Furthermore, step 3 involves fusing the feature maps. After inputting into the IAC image adaptive core feature construction module, the specific operation method of the IAC module is as follows:
[0021] S31: Convert the feature map corresponding to the training image obtained in step 2. Divide the data according to spatial dimensions and process the feature maps according to a pre-defined spatial scale. The image is divided into multiple sub-regions. In one implementation, this is based on feature maps. The spatial size adaptively determines the scale of the sub-region, so that the number of image sub-regions in a single image is a preset value N. Feature vectorization processing is performed on each image sub-region to obtain the image sub-region feature vector set E.
[0022] S32: The image sub-region feature vector set E executes an adaptive core feature selection method based on image distribution awareness. It adaptively selects representative core feature vectors from each training image. The number of selected core feature vectors accounts for 5% to 20% of the total number of corresponding image sub-regions to form the core feature subset K corresponding to the training image. When the number of image sub-regions is lower than a preset threshold, the weighted value of the normalized feature variance and L2 norm of each feature vector is calculated as the selection weight, and the core feature selection is completed according to the selection weight.
[0023] S33: Calculate the image-level mean vector for each core feature subset and average them to obtain the initial global mean vector. ; Calculate the sum of each eigenvector in the eigenvector set E and the initial global mean vector. The distance is used to remove outliers with the largest distance from the target feature vector. The mean of the remaining feature vectors is then calculated to obtain the global prototype P.
[0024] Furthermore, step 4 involves fusing the feature maps. After inputting the LRPS local reconstruction and anomaly scoring module, the specific operation method of the LRPS module is as follows:
[0025] S41: The fused feature map obtained in step 2 is used for detection. The image is divided into sub-regions with the same size as those in step S31 according to the spatial dimension. The features of each sub-region are vectorized to obtain the query feature vector set Q.
[0026] S42: Perform L2 normalization on the query feature vector Q, the image-specific core feature set K, and the global prototype P constructed based on distance constraints, respectively;
[0027] S43: Introducing a prototype-guided weighted linear reconstruction mechanism, using the core feature subset K and the global prototype P to collaboratively constrain the query feature vector Q, thereby reconstructing the feature set. Based on the number of query feature vectors With the number of core features The proportional relationship adaptively determines the weight parameter λ;
[0028] S44: Calculate the query feature vector Q and the corresponding reconstructed feature vector. The L2 distance is used as the image sub-region anomaly score; all image sub-region anomaly scores are sorted and the top α proportion of high-scoring regions are selected. The anomaly scores are then weighted by increasing square weights and weighted to obtain the image sub-region anomaly score set S.
[0029] Furthermore, the specific processing steps for locating and visualizing metal surface defects described in step 5 are as follows:
[0030] S51: Map the set of abnormal scores S of image sub-regions to an abnormal response matrix with the same spatial size as the fused feature map MF2 to be detected, according to the spatial division correspondence in step S41.
[0031] S52: Perform bilinear interpolation upsampling on the anomaly response matrix to make its spatial resolution consistent with the input image to be detected, thereby obtaining an initial anomaly heatmap. ;
[0032] S53: Initial anomaly thermal map Gaussian filtering is performed to smooth the surface, resulting in the final anomaly heatmap H. Based on the anomaly heatmap H, the location and visualization of metal surface defects are completed.
[0033] Beneficial effects
[0034] The present invention proposes a method for detecting defects on metal surfaces where samples are scarce. Compared with existing technologies, it has the following advantages:
[0035] (1) This invention uses a pre-trained ResNet50 residual convolutional neural network as the feature extraction backbone, selects the output features of the last residual unit of the network Layer1, Layer2 and Layer3 for fusion, and jointly models the local texture details of the metal workpiece surface and the global structural information of the workpiece. The local texture details include fine-grained information such as the inherent processing texture of the metal and the micron-level surface morphology, while the global structural information includes consistent information such as the overall outline of the workpiece and the global distribution of regular textures. It can accurately depict the subtle differences between the normal area and the defect area of the metal surface, and thus accurately identify the fine-grained structural changes and local abnormal features in the metal surface image, significantly improving the model's ability to identify small defects and weak contrast defects.
[0036] (2) This invention performs adaptive core feature screening based on the image-level feature distribution of normal samples on metal surfaces, constructs a specific core feature set of images, and generates a robust global prototype by combining a distance-constrained outlier removal strategy, thereby achieving effective compression of the normal feature distribution and accurate removal of abnormal interference features. The adaptive core feature screening can adapt to the texture distribution characteristics of different metal workpieces under a small number of samples, and the distance-constrained outlier removal strategy can filter out non-essential outlier features. It can effectively suppress unstable feature interference introduced by factors such as metal surface reflection, high-frequency noise in the environment, and uneven texture, thereby making the normal feature distribution more concentrated and compact, and significantly improving the stability and reliability of feature modeling in a small number of sample scenarios.
[0037] (3) Based on a pre-generated set of specific core features of the image and a robust global prototype, the present invention performs prototype-guided adaptive weighted linear reconstruction of the fusion features of the metal surface image to be detected, and calculates the abnormal response based on the fitting error between the query features and the corresponding reconstructed features, thereby completing the abnormal scoring of the image sub-region and the image level; wherein the adaptive weighted reconstruction can dynamically adjust the constraint weight according to the scale ratio of the features to be detected and the core features, and achieve accurate fitting of the features to be detected based on the normal metal surface features, which can effectively amplify the feature differences between the normal texture area and the defect abnormal area, thereby significantly enhancing the ability to distinguish abnormal areas, avoiding the abnormal features being mistakenly included in the normal feature distribution in the case of few samples, thereby improving the accuracy of metal surface defect detection and the robustness in complex industrial scenarios. Attached Figure Description
[0038] Figure 1 This is a schematic diagram of the overall process of the present invention.
[0039] Figure 2 This is a schematic diagram of the overall architecture of the detection model in this invention.
[0040] Figure 3 This is a partial effect diagram of the invention in detecting surface defects in electronic commutators. Detailed Implementation
[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. The described embodiments are merely some embodiments of the present invention, and not all embodiments. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the design concept of the present invention should fall within the protection scope of the present invention.
[0042] Example 1:
[0043] like Figure 1As shown, a method for detecting surface defects in metals with scarce samples is described. This embodiment uses the detection of surface defects in an electronic commutator as an example. First, the surface image of the electronic commutator to be detected is acquired as a grayscale image. The detection method includes the following steps:
[0044] Step 1: Acquire images of the metal surface, divide them into a normal sample image set and an image to be detected, and sequentially input each image into a pre-trained ResNet50 residual convolutional neural network for feature extraction. Multi-scale feature maps are extracted from the Layer 1, Layer 2, and Layer 3 residual layers of this network, denoted as L1, L2, and L3 respectively; specifically:
[0045] The original input image is first scaled to a fixed size of 368×368, and then tensor quantization and center cropping are performed sequentially, followed by normalization. The image after these preprocessing steps is used as the network input. The preprocessed image is first input to an initial feature extraction module consisting of 7×7 convolutional layers and max pooling layers. Through convolution operations with a large receptive field and downsampling operations, the input image is initially feature-encoded to obtain a primary feature representation with a spatial resolution of 92×92.
[0046] Subsequently, the primary feature representation is input into the ResNet50 feature extraction backbone and sequentially passed through its Layer 1, Layer 2, and Layer 3 residual layer structures. In each residual layer, the output feature map of the last residual block of the corresponding layer is selected as the feature representation of that layer. Specifically, the output of the last residual block of Layer 1 is used as the first layer feature map L1, with 256 channels and a spatial resolution of 92×92; the output of the last residual block of Layer 2 is used as the second layer feature map L2, with 512 channels and a spatial resolution of 46×46; and the output of the last residual block of Layer 3 is used as the third layer feature map L3, with 1024 channels and a spatial resolution of 23×23.
[0047] The pre-trained ResNet50 residual convolutional neural network is a well-known ResNet50 residual convolutional neural network pre-trained based on the ImageNet dataset. The output features of the last residual unit in Layer 1, Layer 2 and Layer 3 of the network are selected as the multi-scale feature maps L1, L2 and L3 of the corresponding layers.
[0048] Step 2: Input the feature maps L1, L2, and L3 obtained in Step 1 into Unit1, Unit2, and Unit3 respectively for local feature enhancement to obtain enhanced feature maps M1, M2, and M3. Then, concatenate the feature maps M1, M2, and M3 to obtain the fused feature maps MF1 and MF2.
[0049] The Unit1 operation involves performing a 3×3 average pooling operation with a kernel size of 3×3, a stride of 1, and a padding of 1 on the first layer feature map, and then performing a 3×3 max pooling operation on the first layer feature map. The pooling results are then summed and fused element-wise to obtain the enhanced first feature map M1. Since the average pooling and max pooling operations both use the same stride and padding method, the spatial size of the mapped feature map M1 is consistent with that of the first layer feature map L1.
[0050] The Unit2 operation performs a 3×3 average pooling operation with a kernel size of 3×3, a stride of 1, and a padding of 1 on the second layer feature map, and a 3×3 max pooling operation. The pooling results are then summed and fused element-wise to obtain the enhanced second feature map M2. The number of channels in the second layer feature map M2 is the same as that in the second layer feature map L2, and its spatial size is consistent with that of L2.
[0051] The Unit3 operation performs a 3×3 average pooling operation with a kernel size of 3×3, a stride of 1, and a padding of 1 on the third layer feature map, and performs a 3×3 max pooling operation on the third layer feature map. The pooling results are then summed and fused element-wise to obtain the enhanced third feature map M3. The spatial dimensions of the third layer mapped feature map M3 are consistent with those of the third layer feature map L3.
[0052] In the feature fusion stage, the enhanced first feature map M1 and the third feature map M3 are first subjected to spatial scale alignment to ensure their spatial resolution is consistent with that of the second enhanced feature map M2. This spatial scale alignment is achieved through interpolation; in one implementation, bilinear interpolation is used. Subsequently, the spatially aligned M1, M2, and M3 are concatenated along the channel dimension to construct a fused feature map MF. During the training phase, this fused feature map is denoted as... This is used to construct an image-specific core feature set K and a global prototype P. During the detection phase, the fused feature map is denoted as... It is used as the input for query features and subsequent feature reconstruction module.
[0053] In one implementation, when the input image size is 368×368, the fused feature map... and The spatial resolution of all layers is 46×46; when the number of feature channels in the three layers are C1, C2, and C3 respectively, the number of channels in the fused feature map is C1+C2+C3. In one specific embodiment, the number of channels in the fused feature map is 1792.
[0054] Step 3: Convert the feature maps corresponding to the training images obtained in Step 2. The feature map is divided according to spatial dimensions and processed according to a pre-defined spatial scale. The image is divided into multiple sub-regions. In one implementation, this is based on feature maps. The spatial dimensions are adaptively determined to define the scale of sub-regions, ensuring that the number of image sub-regions in a single image is a preset value N. Feature vectorization is performed on each image sub-region to obtain a set of image sub-region feature vectors E. A core feature adaptive selection method based on image distribution awareness is then applied to the image sub-region feature vector set E, adaptively selecting representative core feature vectors from each training image. The number of selected core feature vectors accounts for 5% to 20% of the total number of corresponding image sub-regions, forming the core feature subset K corresponding to that training image.
[0055] In the core feature adaptive selection method based on image distribution awareness, when the number of usable image sub-regions in a single training image is lower than a pre-set threshold, a weight bias mechanism based on feature statistical distribution is employed. This weight bias mechanism includes: calculating the feature variance of the feature vector corresponding to each image sub-region along the channel dimension, and normalizing the feature variance; then, weighting and fusing the normalized feature variance with the L2 norm value of the feature vector of the corresponding image sub-region, thereby increasing the priority of image sub-regions with large local feature variances as core features, thus enhancing the representativeness and robustness of specific core feature sets under limited sample conditions.
[0056] After obtaining the image-specific core feature set corresponding to all training images, the core features are aggregated and the mean-pruning strategy is used to remove the 10% of feature points that are furthest from the preliminary mean. The preliminary mean is the value obtained by first calculating the feature mean vector of the image-specific core feature set corresponding to each training image, then averaging the feature mean vectors of all training images, and finally calculating the mean of the remaining feature points to obtain the final global prototype P.
[0057] Step 4: Combine the fused feature maps corresponding to the image to be detected obtained in Step 2. Expanding spatially, a set of query feature vectors Q corresponding to each image sub-region is obtained. Before feature reconstruction, L2 normalization is first performed on the query feature Q, the image-specific core feature set K, and the global prototype P to eliminate the influence of scale differences between different feature sources on subsequent feature reconstruction and distance calculation. Subsequently, the query feature Q is reconstructed by combining the image core feature subset K generated by the IAC network and the global prototype P through a weighted linear combination, resulting in the corresponding reconstructed feature representation. .
[0058] In the feature reconstruction process, an adaptive weight parameter λ is introduced to adjust the relative constraint strength between specific core features of the image and the global prototype during feature reconstruction. The adaptive weight parameter λ is determined based on the number of image sub-regions in the image to be detected. The number of features of a specific core feature set K of an image The proportional relationship between them is adaptively adjusted to balance the reconstruction contributions of local and global features under different image sub-region densities. In one implementation, the weight parameter λ is composed of basic weight coefficients. Determined together with the proportional adjustment factor, and satisfying the following:
[0059] ;
[0060] in The base weight coefficient is 1, which is a preset upper limit value. It is used to prevent reconstruction bias problems in scenarios with few samples or high-density patches, thereby avoiding excessive dominance of local or global features in the reconstruction process and achieving balanced modeling of local abnormal patterns and global background distribution.
[0061] After feature reconstruction is completed, the distance metric between the original feature vector of each image sub-region and its corresponding reconstructed feature vector is calculated. The distance metric is the L2 norm, and the distance value is defined as the anomaly score of the corresponding image sub-region.
[0062] Subsequently, the anomaly scores of all image sub-regions are sorted in descending order, and high-scoring image sub-regions with anomaly scores within the top α proportion range are selected. An increasing squared weighting strategy is then applied to the anomaly scores of the selected high-scoring image sub-regions according to their sorting order.
[0063] In one implementation, assuming the number of selected high-resolution image sub-regions is M, an equally spaced weight sequence from 1 to 2 is constructed, and the weights are squared to obtain the weighting coefficients:
[0064] ;
[0065] in In the interval The values are uniformly distributed across the image. Then, a weighted average of the anomaly scores in the sub-regions of the high-resolution image is calculated to obtain the image-level anomaly score S.
[0066] Step 5: Map the set of anomaly scores S of the image sub-regions back to their original image spatial locations according to their spatial correspondence in the feature map, forming an anomaly response matrix with the same spatial structure as the original image. Then, use bilinear interpolation to upsample the anomaly response matrix, scaling it to the same size as the original image to obtain a preliminary anomaly heatmap. .
[0067] In one implementation, the bilinear interpolation is used to maintain spatial continuity and avoid blocky artifacts produced by nearest-neighbor interpolation, thereby improving the smoothness and positioning accuracy of the anomaly region boundaries. Subsequently, the preliminary anomaly heatmap H0 is smoothed using Gaussian filtering to suppress isolated noise responses and enhance the coherence of local anomaly regions, resulting in the final anomaly heatmap H.
[0068] In one implementation, the standard deviation parameter σ of the Gaussian filter is determined proportionally to the input image size, and a preset lower limit is set to ensure the stability of the smoothing intensity under different resolution conditions. Finally, the final anomaly heatmap H is weighted and fused with the original image for display, achieving visual annotation and precise location of defect areas.
[0069] like Figure 2 As shown, the detection model mainly consists of a feature extraction and fusion module, an IAC image adaptive core feature construction module, an LRPS local reconstruction and anomaly scoring module, and a defect localization and visualization module. The specific contents of each module are as follows:
[0070] In the feature extraction and fusion module, the training image and the image to be detected are preprocessed uniformly and then input in parallel into a ResNet50 network pre-trained based on ImageNet. The images are sequentially passed through the Layer 1, Layer 2, and Layer 3 residual layers of the network, and multi-scale feature maps output from the final residual blocks of each layer are extracted, denoted as L1, L2, and L3 respectively. L1, L2, and L3 are then input into feature enhancement units Unit1, Unit2, and Unit3 with consistent structures to perform local feature enhancement, outputting enhanced feature maps M1, M2, and M3 respectively. After spatial scale alignment of the enhanced feature maps, they are concatenated and fused along the channel dimension, ultimately generating two types of fused feature maps: the training fused feature map corresponding to the training image. The fused feature map corresponding to the image to be detected .
[0071] In the IAC image adaptive core feature construction module, the input training fusion feature map is used. The image is divided into multiple sub-regions according to spatial dimensions and its features are vectorized to obtain a feature vector set E. Core feature adaptive filtering is performed on the feature vector set E to select 5%–20% of the feature vectors to form an image-specific core feature set K. Subsequently, the sum of each feature vector in the feature vector set E and the initial global mean vector is calculated. The distance is used to remove outliers with the largest distance from the target feature vector. The mean of the remaining feature vectors is then calculated to obtain the global prototype P.
[0072] In the LRPS local reconstruction and anomaly scoring module, the input feature extraction and fusion module outputs the fused feature map to be detected. Simultaneously, the image-specific core feature set K and the global prototype P output by the IAC module are introduced as normal feature constraints. First, the... The image is divided into sub-regions, and the features of each sub-region are vectorized to obtain a set of query feature vectors Q. Then, the core feature subset K and the global prototype P are used to jointly constrain the query feature vectors Q. Through prototype-guided weighted linear reconstruction, the reconstructed feature set is obtained. This refers to the reconstructed sample in the image. The reconstructed features are then compared with the query features to be detected, and the query feature vector Q and its corresponding reconstructed feature vector are calculated. The L2 distance is used as the image sub-region anomaly score; all image sub-region anomaly scores are sorted and the high-scoring regions with the top α proportion are selected. The anomaly scores are then weighted by increasing square weights and weighted averaged to finally obtain the image sub-region anomaly score set S.
[0073] In the defect localization and visualization module, the anomaly score set S output by the LRPS module is input to complete the defect localization and visualization. First, the anomaly score set S of the image sub-regions is mapped to a fused feature map with the target region based on spatial correspondence. An anomaly response matrix with consistent spatial dimensions is obtained; bilinear interpolation upsampling is then performed on the anomaly response matrix to make its spatial resolution consistent with the original input image to be detected, thus obtaining an initial anomaly heatmap. For the initial anomalous thermal map Gaussian filtering is applied to smooth the surface, resulting in the final anomaly heatmap H. The anomaly heatmap H is then used to locate and visualize the defects on the metal surface.
[0074] To verify the technical effectiveness of the present invention, the inventors conducted a comparative experiment on the publicly available dataset KolektorSDD.
[0075] The experiments of this invention were conducted under the following conditions: the hardware environment was a computing platform equipped with an NVIDIA RTX 3090 GPU and 24GB of video memory; the software environment was based on Python 3.10.8 and used PyTorch deep learning framework version 2.9.1.
[0076] In the experiments, the KolektorSDD dataset was used for validation. To simulate a low-sample scenario, 2, 4, 6, and 8 images were randomly selected from the normal samples as the training set, and the remaining samples were used as the test set for defect detection. The random seed was set to 42. The input images were uniformly adjusted to a resolution of 368×368. During model training, the batch size was set to 1, and the number of training epochs was set to 1.
[0077] To evaluate the detection performance of this method, two metrics, image-level AUROC and pixel-level AUROC, were used for comprehensive evaluation. Image-level AUROC assesses the model's ability to distinguish between image defects by comparing the consistency between the overall anomaly score and the ground truth label. Pixel-level AUROC assesses the model's ability to locate anomaly regions by comparing the consistency between the predicted anomaly heatmap and the ground truth pixel-level annotations. A higher image-level AUROC indicates a stronger ability to distinguish between normal and anomalous images; a higher pixel-level AUROC indicates more accurate localization of anomaly regions.
[0078] To verify the effectiveness and superiority of the method of this invention, three typical anomaly detection methods—PatchCore, PaDiM, and FastRecon—were selected as comparison models, and experiments were conducted on the KolektorSDD dataset. To ensure fairness in the comparison, each method was trained and tested under the same data partition and uniform hyperparameter settings. In a few-sample setting, [the following methods were selected]. We trained the method on normal samples and recorded the image-level and pixel-level AUROC metrics for each method under different settings. The experimental results are shown in Table 1.
[0079] Table 1
[0080]
[0081] Experimental Analysis:
[0082] A comparison of the evaluation metrics in Table 1 shows that the method of this invention achieves optimal results in both image-level and pixel-level AUROC metrics. Compared with typical anomaly detection methods such as PatchCore, PaDiM, and FastRecon, the method of this invention shows varying degrees of improvement in both metrics. In summary, the experimental results in Table 1 fully verify the effectiveness and superiority of the method of this invention.
[0083] Figure 3 This paper demonstrates the defect detection performance of the proposed method on a dataset. The left side shows the anomaly heatmap generated by the model, and the right side shows the overlay of the anomaly heatmap and the original image. In the heatmap, red areas represent defect areas, yellow areas represent transitional areas with moderate anomaly responses, and blue areas represent normal background. The figure shows that the proposed method can accurately locate defect areas against complex backgrounds. Anomaly responses are mainly concentrated at the actual defect locations, while responses are lower in normal areas, indicating that the model has good defect localization and background suppression capabilities. Furthermore, it can still produce relatively continuous and obvious anomaly responses for defect areas with blurred boundaries or low contrast, demonstrating that the method has high sensitivity to minor defects and weak anomalies. In summary, Figure 3 The visualization results verify the effectiveness of the method of the present invention in terms of defect localization accuracy and detection stability.
[0084] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.
Claims
1. A method for detecting surface defects in metals with scarce samples, characterized in that, Including the following steps: Step 1: Acquire images of the metal surface, divide them into a normal sample image set and an image to be detected, and input each image into a pre-trained ResNet50 residual convolutional neural network for feature extraction. Extract multi-scale feature maps from the Layer 1, Layer 2, and Layer 3 residual layers of the network, respectively, and denoted as L1, L2, and L3. Step 2: Input the feature maps L1, L2, and L3 obtained in Step 1 into Unit1, Unit2, and Unit3 respectively for local feature enhancement to obtain enhanced feature maps M1, M2, and M3. Then, concatenate the feature maps M1, M2, and M3 to obtain the fused feature maps MF1 and MF2. Step 3: Combine the training and feature maps obtained in Step 2 Input the IAC image adaptive core feature construction module, and obtain the image-specific core feature set K and the final robust global prototype P through feature space partitioning, core feature selection based on image distribution and distance constraint processing; Step 4: Develop the fused feature map obtained in Step 2. Input the LRPS local reconstruction and anomaly scoring module, combine the image-specific core feature set K obtained in step 3 with the final robust global prototype P, and perform prototype-guided weighted linear reconstruction on the query features to obtain the image sub-region anomaly score set S. Step 5: Input the set of abnormal scores S of the image sub-regions obtained in Step 4 into the defect localization and visualization module. Through spatial mapping, interpolation upsampling and Gaussian filtering, an abnormal heat map H is generated to realize the localization and visualization of metal surface defects.
2. The method for detecting metal surface defects in cases of scarce samples according to claim 1, characterized in that: The pre-trained ResNet50 residual convolutional neural network mentioned in step 1 is a well-known ResNet50 residual convolutional neural network pre-trained based on the ImageNet dataset; the output features of the last residual unit in Layer 1, Layer 2 and Layer 3 of this network are selected as the multi-scale feature maps L1, L2 and L3 of the corresponding layers.
3. The method for detecting metal surface defects with scarce samples according to claim 1, characterized in that: The specific operation method of step 2 includes the following steps: S21: Perform local average pooling and local max pooling operations on each of the feature maps Li (i=1,2,3) received by Unit1, Unit2, and Unit3 respectively, and fuse the two pooling results to obtain the enhanced feature map Mi of the corresponding layer; S22: Perform spatial size alignment on the enhanced feature maps M1, M2, and M3 to make their spatial resolution consistent; S23: The spatially aligned feature maps M1, M2, and M3 are concatenated and fused along the channel dimension to obtain a fused feature map MF, wherein the fused feature map MF includes the training fused feature map corresponding to the normal sample image. The fused feature map corresponding to the image to be detected .
4. A method for detecting metal surface defects with scarce samples according to claim 1 or 3, characterized in that: In step 2, Unit1, Unit2, and Unit3 are three feature enhancement processing units with the same structure, which are respectively applied to feature maps L1, L2, and L3 to perform local feature enhancement operations. The local feature enhancement operations include performing 3×3 average pooling and 3×3 max pooling operations on the first layer feature maps with a kernel size of 3×3, a stride of 1, and padding of 1, and then performing element-wise summation and feature fusion on the pooling results of the two operations.
5. The method for detecting metal surface defects in cases of scarce samples according to claim 4, characterized in that: The feature fusion process is as follows: First, spatial scale alignment is performed on the enhanced first feature map M1 and the third feature map M3 to make their spatial resolution consistent with that of the second enhanced feature map M2; then, the spatially aligned M1, M2 and M3 are stitched together in the channel dimension to construct the fused feature map MF. During the training phase, the fused feature map is denoted as... , used to construct the image-specific core feature set K and the global prototype P; During the detection phase, the fused feature map is denoted as... It is used as the input for query features and subsequent feature reconstruction module.
6. The method for detecting surface defects in metals with scarce samples according to claim 1, characterized in that: The fusion of feature maps described in step 3 After inputting into the IAC image adaptive core feature construction module, the specific operation method of the IAC module is as follows: S31: Convert the feature map corresponding to the training image obtained in step 2. Divide the data according to spatial dimensions and process the feature maps according to a pre-defined spatial scale. The image is divided into multiple sub-regions, and feature vectorization is performed on each sub-region to obtain a set of feature vectors for the sub-regions, E. S32: The image sub-region feature vector set E executes an adaptive core feature selection method based on image distribution awareness. It adaptively selects representative core feature vectors from each training image. The number of selected core feature vectors accounts for 5% to 20% of the total number of corresponding image sub-regions to form the core feature subset K corresponding to the training image. When the number of image sub-regions is lower than a preset threshold, the weighted value of the normalized feature variance and L2 norm of each feature vector is calculated as the selection weight, and the core feature selection is completed according to the selection weight. S33: Calculate the image-level mean vector for each core feature subset and average them to obtain the initial global mean vector. ; Calculate the sum of each eigenvector in the eigenvector set E and the initial global mean vector. The distance is used to remove outliers with the largest distance from the target feature vector. The mean of the remaining feature vectors is then calculated to obtain the global prototype P.
7. The method for detecting metal surface defects with scarce samples according to claim 1, characterized in that: The fusion of feature maps described in step 4 After inputting the LRPS local reconstruction and anomaly scoring module, the specific operation method of the LRPS module is as follows: S41: The fused feature map obtained in step 2 is used for detection. The image is divided into sub-regions with the same size as those in step S31 according to the spatial dimension. The features of each sub-region are vectorized to obtain the query feature vector set Q. S42: Perform L2 normalization on the query feature vector Q, the image-specific core feature set K, and the global prototype P constructed based on distance constraints, respectively; S43: Introducing a prototype-guided weighted linear reconstruction mechanism, using the core feature subset K and the global prototype P to collaboratively constrain the query feature vector Q, thereby reconstructing the feature set. Based on the number of query feature vectors With the number of core features The proportional relationship adaptively determines the weight parameter λ; S44: Calculate the query feature vector Q and the corresponding reconstructed feature vector. The L2 distance is used as the image sub-region anomaly score; all image sub-region anomaly scores are sorted and the high-scoring regions with the top α proportion are selected. The anomaly scores are then weighted by increasing square weights and weighted to obtain the image sub-region anomaly score set S.
8. The method for detecting metal surface defects with scarce samples according to claim 1, characterized in that: The specific processing steps for locating and visualizing metal surface defects described in step 5 are as follows: S51: Map the set of abnormal scores S of image sub-regions to an abnormal response matrix with the same spatial size as the fused feature map MF2 to be detected, according to the spatial division correspondence in step S41. S52: Perform bilinear interpolation upsampling on the anomaly response matrix to make its spatial resolution consistent with the input image to be detected, thereby obtaining an initial anomaly heatmap. ; S53: Initial anomaly thermal map Gaussian filtering is performed to smooth the surface, resulting in the final anomaly heatmap H. Based on the anomaly heatmap H, the location and visualization of metal surface defects are completed.