Metal surface defect detection method, device and equipment and storage medium

Abstract

The invention discloses a metal surface defect detection method and device, equipment and a storage medium. The method comprises the following steps: constructing a data set containing normal metal surface images; constructing a feature extraction network based on a deep residual network of multi-scale feature cycle fusion, and inputting the data set into the feature extraction network to obtain a multi-scale feature library of normal samples; inputting a to-be-detected metal surface image into the feature extraction network to obtain multi-scale features of the to-be-detected image; and calculating a multi-scale fusion score of the multi-scale features of the to-be-detected image and the multi-scale feature library, and when the multi-scale fusion score is greater than or equal to a preset threshold value, determining that the to-be-detected metal surface has defects. According to the unsupervised metal surface defect detection method provided by the invention, manual labeling is not needed, the deep residual network based on multi-scale feature cycle fusion is used for image feature extraction, image features are output from different scales, a multi-scale fusion scoring mechanism is constructed, and the accuracy and robustness of detection are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method, apparatus, device, and storage medium for detecting defects on metal surfaces. Background Technology

[0002] In the field of industrial inspection, determining the presence or absence of defects on metal surfaces is a crucial step in ensuring product quality and improving production efficiency. Traditional metal surface defect detection mainly relies on manual visual inspection, which suffers from low efficiency, high subjectivity, and susceptibility to missed defects due to fatigue, making it difficult to meet the demands of modern manufacturing for high-precision and high-efficiency inspection.

[0003] With the development of computer vision technology, deep learning-based defect detection methods have gradually become mainstream. Currently, supervised defect detection methods are mainly used. These methods involve manually labeling defect locations and categories to construct a training dataset containing defect samples, allowing the deep learning network to learn the visual features of defects. However, supervised methods require a large number of manually labeled defect samples, resulting in high labeling costs and long processing times. In industrial production scenarios where defect samples are scarce, defect morphologies are diverse, and new defect types are constantly emerging, supervised methods struggle to obtain sufficient training data, leading to insufficient model generalization ability. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for detecting defects on metal surfaces, in order to at least solve the technical problems of high annotation costs and insufficient model generalization ability in related technologies for defect detection methods.

[0005] According to one aspect of the embodiments of this application, a method for detecting defects on a metal surface is provided, comprising: Construct a dataset containing images of normal metal surfaces; A feature extraction network is constructed based on a deep residual network with multi-scale feature cyclic fusion. The dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples. The image of the metal surface to be detected is input into the feature extraction network to obtain multi-scale features of the image to be detected; The multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library is calculated. When the multi-scale fusion score is greater than or equal to a preset threshold, it is determined that there is a defect on the metal surface to be detected.

[0006] In one implementation, the dataset is input into a feature extraction network to obtain a multi-scale feature library of normal samples, including: The images in the dataset are input into multiple deep residual modules of the feature extraction network to obtain feature layers at multiple scales. Multi-scale feature cyclic fusion is performed on feature layers of multiple scales to obtain fused features and full-scale fused features; Feature enhancement is performed on feature layers at multiple scales based on fusion features and full-scale fusion features to obtain multi-scale features of the image. A multi-scale feature library is then obtained based on the multi-scale features of multiple images.

[0007] In one implementation, images from the dataset are input into multiple deep residual modules of a feature extraction network to obtain feature layers at multiple scales, including: The images in the dataset are input into the first deep residual module of the feature extraction network to obtain the first feature layer, which is used to represent the edge and texture information of the image. The first feature layer is input into the second deep residual module of the feature extraction network to obtain the second feature layer, which is used to represent the structural information of the image. The second feature layer is input into the third deep residual module of the feature extraction network to obtain the third feature layer, which is used to represent the semantic information of the image.

[0008] In one implementation, multi-scale feature cyclic fusion is performed on feature layers of multiple scales to obtain fused features and full-scale fused features, including: Upsampling is performed on the second and third feature layers; The first feature layer and the upsampled second feature layer are weighted and fused to obtain the first fused feature; The first fused feature is weighted and fused with the upsampled third feature layer to obtain the second fused feature; The second fusion feature is weighted and fused with the first feature layer to obtain the full-scale fusion feature.

[0009] In one implementation, feature enhancement is performed on feature layers at multiple scales based on fusion features and full-scale fusion features to obtain multi-scale features of the image, including: The full-scale fused features are weighted and fused with the first feature layer to obtain the fused first-scale features; After downsampling the first fusion feature, it is weighted and fused with the second feature layer to obtain the fused second scale feature; After downsampling the second fusion feature, it is weighted and fused with the third feature layer to obtain the fused third-scale feature; Based on the fused first-scale features, fused second-scale features, and fused third-scale features, multi-scale features of the image are obtained.

[0010] In one implementation, calculating a multi-scale fusion score between the multi-scale features of the image to be detected and a multi-scale feature library includes: The first anomaly score of the image to be detected and the multi-scale feature library are calculated based on the weighted Euclidean distance at the first scale feature layer. The second anomaly score in the second-scale feature layer is calculated based on the weighted Manhattan distance and the multi-scale feature library; The third anomaly score of the image to be detected and the multi-scale feature library are calculated based on the weighted cosine distance at the third-scale feature layer. A multi-scale fusion score is obtained by weighted summation of the first, second, and third anomaly scores.

[0011] In one implementation, before calculating the first anomaly score of the image to be detected and the multi-scale feature library based on weighted Euclidean distance in the first scale feature layer, the method further includes: The feature maps of each scale extracted from the image to be detected, as well as the feature maps of the corresponding scale in the multi-scale feature library, are divided into multiple local blocks of fixed size. For each local patch in the image to be detected, calculate its cosine similarity with all corresponding local patches in the multi-scale feature library, and determine the matching local patch based on the cosine similarity. Calculate the structural similarity index between each pair of matching local blocks; Based on the structural similarity index of all matching block pairs at each scale, the weights corresponding to each scale are generated through statistical aggregation.

[0012] According to another aspect of the embodiments of this application, a device for detecting metal surface defects is provided, comprising: The dataset building module is used to construct datasets containing images of normal metal surfaces; The feature extraction network construction module is used to build a feature extraction network based on a deep residual network with multi-scale feature cyclic fusion. The dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples. The image feature extraction module is used to input the image of the metal surface to be detected into the feature extraction network to obtain multi-scale features of the image to be detected; The defect detection module is used to calculate the multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library. When the multi-scale fusion score is greater than or equal to a preset threshold, it is determined that there is a defect on the metal surface to be detected.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the metal surface defect detection method of the embodiments of this application through the computer program.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the metal surface defect detection method of the embodiments of this application when running.

[0015] The technical solutions provided in this application embodiment may include the following beneficial effects: This application provides a method for detecting defects on metal surfaces. Based on a deep residual network using multi-scale feature cyclic fusion, this method serves as the feature extraction network. A dataset containing only normal samples is input into the feature extraction network to obtain a multi-scale feature library of normal samples. The image of the metal surface to be detected is then input into the feature extraction network to obtain multi-scale features of the image. Multi-scale fusion scoring is performed between the multi-scale features of the image and the multi-scale feature library to further determine whether defects exist on the metal surface. This method requires no manual annotation, only a training dataset containing normal samples. It uses a deep residual network based on multi-scale feature cyclic fusion for image feature extraction, obtaining features containing multi-scale fusion information, and outputs image features from different scales. This constructs a multi-scale fusion scoring mechanism, significantly improving the accuracy and robustness of the detection. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a method for detecting metal surface defects according to an embodiment of this application; Figure 2 This is a schematic diagram of a multi-scale feature extraction method according to an embodiment of this application; Figure 3 This is a schematic diagram of a deep residual network structure according to an embodiment of this application; Figure 4 This is a schematic diagram of a multi-scale feature cyclic fusion process according to an embodiment of this application; Figure 5 This is a schematic diagram of a multi-scale feature derivation process according to an embodiment of this application; Figure 6 This is a schematic diagram of a metal surface defect detection device according to an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] This application aims to provide a method for determining the presence or absence of defects on metal processing surfaces based on unsupervised learning. It uses a deep residual network based on multi-scale feature cyclic fusion as the image feature extraction module, outputting image features from different scales to obtain a multi-scale feature library. During training, only normal samples are required, without the involvement of abnormal samples. During detection, the same deep residual network based on multi-scale feature cyclic fusion is used to extract features from the image to be detected. The features of the image to be detected are compared with the features of normal samples to construct a multi-scale fusion scoring mechanism. Different distance calculation methods are used for anomaly scoring of features at different scales. For example, the first feature layer uses weighted Euclidean distance calculation, and a structural similarity attention mechanism is introduced to dynamically adjust the weights. The second feature layer uses weighted Manhattan distance calculation, and the third feature layer, considering the semantic information it contains, uses weighted cosine distance calculation. Finally, the presence or absence of defects is determined based on the comprehensive score.

[0020] The method for detecting metal surface defects according to embodiments of this application will be described in detail below with reference to the accompanying drawings. Figure 1 As shown, the method mainly includes the following steps: S101 constructs a dataset containing images of normal metal surfaces.

[0021] In the data preparation phase, a training dataset containing only images of normal metal surfaces is constructed. Various data augmentation operations, such as rotation, scaling, affine transformation, brightness adjustment, and contrast adjustment, are applied to these normal samples to expand the data scale and increase its diversity. This enhances the robustness of the feature extraction network to normal samples under different scales, angles, lighting conditions, and appearance variations, thus building a more complete and robust multi-scale feature library of normal samples for subsequent unsupervised learning.

[0022] In this embodiment of the application, there is no need to obtain abnormal samples or perform manual annotation when constructing the training dataset, which greatly reduces the annotation cost.

[0023] S102 is based on a deep residual network with multi-scale feature cyclic fusion. A feature extraction network is constructed, and the dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples.

[0024] In one embodiment of this application, a feature extraction network is constructed using a deep residual network that performs multi-scale feature cyclic fusion. The feature extraction network sequentially performs multi-scale feature generation, feature cyclic fusion, and multi-scale feature output to achieve the construction of a cross-scale feature library for metal surfaces.

[0025] In one implementation, the dataset is input into a feature extraction network to obtain a multi-scale feature library of normal samples, including: inputting images from the dataset into multiple deep residual modules of the feature extraction network to obtain feature layers at multiple scales; performing multi-scale feature cyclic fusion on the feature layers at multiple scales to obtain fused features and full-scale fused features; performing feature enhancement on the feature layers at multiple scales based on the fused features and full-scale fused features to obtain the multi-scale features of the image; and obtaining a multi-scale feature library based on the multi-scale features of multiple images.

[0026] First, the images in the dataset are input into multiple deep residual modules of the feature extraction network to obtain feature layers at multiple scales. Specifically, the images in the dataset are input into the first deep residual module of the feature extraction network to obtain the first feature layer, which represents the edge and texture information of the image; the first feature layer is input into the second deep residual module of the feature extraction network to obtain the second feature layer, which represents the structural information of the image; the second feature layer is input into the third deep residual module of the feature extraction network to obtain the third feature layer, which represents the semantic information of the image.

[0027] like Figure 2As shown, taking the original image as input, it first passes through a deep residual network 1 to obtain the first feature layer F1. The features of this layer focus on information such as edges and textures in the original image. The first feature layer is then further processed by a deep residual network 2 to obtain the second feature layer F2, which focuses on the spatial relationships between pixels. The second feature layer F2 is then further processed by a deep residual network 3 to obtain the third feature layer F3, which focuses on the semantic information in the image.

[0028] The first depth residual module, the second depth residual module, and the third depth residual module have the same structure, as shown below. Figure 3 As shown, the network consists of alternately stacked residual modules (Res modules) and densely connected modules (Dense modules). The input features sequentially pass through Res module 1, Dense module 1, Res module n, and Dense module n. After serialization processing through multiple layers of Res and Dense modules, the output features are finally obtained. The number of modules in the network can be adjusted according to the specific detection content.

[0029] Furthermore, to facilitate subsequent feature fusion, feature alignment is required. The second feature layer F2 and the third feature layer F3 are upsampled to match their dimensions with those of the first feature layer.

[0030] Furthermore, multi-scale feature cyclic fusion is performed on feature layers at multiple scales to obtain fused features and full-scale fused features. This includes weighted fusion of the first feature layer and the upsampled second feature layer to obtain the first fused feature; weighted fusion of the first fused feature and the upsampled third feature layer to obtain the second fused feature; and weighted fusion of the second fused feature and the first feature layer to obtain the full-scale fused feature.

[0031] like Figure 4 As shown, the first round of iterative fusion fuses the first feature layer F1 with the upsampled second feature layer F2_align, which integrates the image's detail and structural information. The initial fusion feature F_fusion_0 = F1 is set, and F1 and F2_align are passed to fusion module 1 for processing. Based on the detail feedback feature F_feedback_0 and the structural feature F2_align, the variance of each feature across each channel is calculated to assess the importance of different channels, generating corresponding channel attention weights w1 and w2. Subsequently, the obtained weights are used to perform a channel-level weighted summation of F_feedback_0 and F2_align to obtain the first fusion feature F_fusion_1, thus achieving adaptive aggregation of detail and structural information at the channel dimension. The weighted fusion formula is as follows: w1×F_feedback_0 + w2×F2_align → F_fusion_1.

[0032] The second iteration fuses the first fusion feature F_fusion_1 with the upsampled third feature layer F3_align, combining structural and semantic information. The feedback feature is then updated so that F_feedback_1 = F_fusion_1, and F_fusion_1 and F3_align are passed to fusion module 2 for further processing.

[0033] In the second round of feature fusion, firstly, based on the structural feedback feature F_feedback_1 and the semantic feature F3_align, the spatial attention weights w3 and w4, representing the importance of different spatial locations, are calculated by analyzing their feature response distribution in the spatial dimension. Then, the obtained weights are used to perform a weighted summation of F_feedback_1 and F3_align in terms of spatial location to obtain the second fusion feature F_fusion_2, thereby achieving adaptive fusion of structural and semantic information in the spatial dimension. The weighted fusion formula is as follows: w3×F_feedback_1 + w4×F3_align → F_fusion_2.

[0034] In the third round of feature fusion, the second fusion feature F_fusion_2 is fused with the first feature layer to achieve the fusion of semantic information and detailed information. The feedback feature is updated so that F_feedback_2 = F_fusion_2. F_fusion_2 and F1 are then passed into the fusion module 3 for processing.

[0035] In the third round of feature fusion, firstly, based on the semantic feedback feature F_feedback_2 and the detail feature F1, the Pearson correlation coefficient between the two in the channel dimension is calculated to evaluate the statistical correlation between different channels, and weights w5 and w6 representing the channel correlation strength are generated accordingly. Then, the obtained weights are used to perform channel-level weighted fusion of F_feedback_2 and F1 to obtain the full-scale fusion feature F_fusion, thereby achieving adaptive fusion of semantic and detail information guided by correlation. The weighted fusion formula is: w5×F_feedback_2 + w6×F1 → F_fusion.

[0036] Furthermore, feature enhancement is performed on feature layers at multiple scales based on fusion features and full-scale fusion features to obtain multi-scale features of the image, including: weighted fusion of full-scale fusion features with the first feature layer to obtain fused first-scale features; downsampling the first fusion features and weighted fusion with the second feature layer to obtain fused second-scale features; downsampling the second fusion features and weighted fusion with the third feature layer to obtain fused third-scale features; and obtaining multi-scale features of the image based on the fused first-scale features, fused second-scale features, and fused third-scale features.

[0037] like Figure 5 As shown, firstly, a 1×1 convolution operation is performed on the full-scale fused feature F_fusion and the original first feature layer F1 to unify their channel dimensions. Then, the two features are weighted and fused using preset fusion weights α1 and α2, for example, α1=0.6 and α2=0.4: α1 × F_fusion + α2 × F1.

[0038] The first-scale feature F1_fused after fusion is obtained. Thus, while preserving the original detailed information, global semantic information after multi-scale cyclic fusion is incorporated, achieving further optimization of feature representation.

[0039] In the process of feature enhancement of the second feature layer, the first fused feature F_fusion_1 is first downsampled to obtain F_fusion^1, so that its spatial size matches that of the original second feature layer F2. Then, a 1×1 convolution operation is performed on the downsampled feature to unify its channel dimension with that of F2. Finally, the two features are weighted and fused using preset fusion weights α3 and α4, for example, α3=0.5 and α4=0.5: α3 × F_fusion^1 + α4 × F2.

[0040] The fused second-scale feature F2_fused is obtained, thereby integrating the details and structural interaction features generated by the preceding multi-scale cycles while retaining the original structural information, thus achieving unified optimization of feature representation.

[0041] In the process of feature enhancement of the third feature layer, the second fusion feature F_fusion_2 is first downsampled to obtain F_fusion^2, so that its spatial size matches that of the original third feature layer F3. Then, a 1×1 convolution operation is performed on the downsampled feature to unify its channel dimension with that of F3. Finally, the two features are weighted and fused using preset fusion weights α5 and α6, for example, α5=0.7 and α6=0.3: α5 × F_fusion^2 + α6 × F3.

[0042] The fused third-scale feature F3_fused is obtained, thereby integrating the semantic and structural interaction features generated by the preceding multi-scale cycle while retaining the original semantic information, thus achieving unified optimization of feature representation.

[0043] Finally, a set of feature maps at three scales {F1_fused, F2_fused, F3_fused} is obtained. By processing multiple normal samples, a multi-scale feature library is constructed.

[0044] The deep residual network based on multi-scale feature cyclic fusion constructed in this application achieves progressive interaction and iterative optimization between features of different scales by introducing a cyclic fusion mechanism, significantly enhancing the representational ability and robustness of the feature extraction network. The network first extracts features of different scales, such as edge texture, structural relationships, and semantic information, through a multi-level residual structure. Then, using a cyclic feedback mechanism, it sequentially performs three rounds of cross-scale fusion on detail, structural, and semantic features through channel attention, spatial attention, and relevance attention mechanisms. Each fusion dynamically adjusts the fusion weights based on the output of the previous round. Structural information is introduced at the detail layer to enhance spatial consistency, semantic cues are injected at the structural layer to enhance discriminative power, and detail information is preserved at the semantic layer to improve localization accuracy.

[0045] S103 inputs the image of the metal surface to be detected into the feature extraction network to obtain multi-scale features of the image to be detected.

[0046] In one implementation, an image of the metal surface to be inspected is acquired. During the inspection phase, the surface of the metal workpiece to be inspected is directly photographed or captured using an industrial camera or image acquisition device to obtain the original image to be inspected. This image serves as the input to the entire inspection process and is fed into a feature extraction network for processing.

[0047] The feature extraction network employs the multi-scale cyclic fusion feature extraction method in step S102. The image to be detected is input into multiple depth residual modules of the feature extraction network to obtain feature layers at multiple scales. Multi-scale feature cyclic fusion is performed on these feature layers to obtain fused features and full-scale fused features. Based on the fused features and full-scale fused features, feature enhancement is performed on the feature layers at multiple scales to obtain the multi-scale features of the image. The specific extraction process is not described again here. Finally, the multi-scale features F1_test, F2_test, and F3_test of the image to be detected are obtained.

[0048] S104 calculates the multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library. When the multi-scale fusion score is greater than or equal to a preset threshold, it is determined that there is a defect on the metal surface to be detected.

[0049] This application adopts a multi-scale fusion scoring mechanism. Features at different scales are scored using different distance calculation methods. The first feature layer uses weighted Euclidean distance, the second feature layer uses weighted Manhattan distance to balance computational efficiency and the correlation between feature layers. The third feature layer, considering that it contains semantic information, uses weighted cosine distance. All three feature layers introduce a structural similarity attention mechanism to dynamically adjust the weights. Finally, the existence of defects is judged based on the comprehensive score.

[0050] In one implementation, a first anomaly score is calculated between the image to be detected and the multi-scale feature library at the first scale feature layer based on weighted Euclidean distance. All features are taken from F1_fused, and the Euclidean distance between F1_test and F1_fused is calculated.

[0051] in, This represents the weights of the first feature layer, where C represents the total number of channel dimensions, and c represents the c-th channel. All M channels are considered. The mean value was used as the first anomaly score:

[0052] The second anomaly score at the second scale feature layer is calculated based on the weighted Manhattan distance between the image to be detected and the multi-scale feature library. All features are extracted from F2_fused, and the Manhattan distance between F2_test and F2_fused is calculated.

[0053] in, This represents the weights of the second feature layer, taking all M values. The mean of the score was used as the second anomaly score.

[0054] The third anomaly score at the third scale feature layer is calculated based on the weighted cosine distance between the image to be detected and the multi-scale feature library; all features are taken from F3_fused, and the cosine distance between F3_test and F3_fused is calculated.

[0055] in, The weights of the third feature layer are represented by all M values. The mean of the scores was used as the third anomaly score.

[0056] Furthermore, a weighted summation is performed based on the first anomaly score, the second anomaly score, and the third anomaly score to obtain a multi-scale fusion score. .

[0057]

[0058] In one implementation, the weights of the first feature layer, the second feature layer, and the third feature layer are calculated based on structural similarity attention. For each feature layer at each scale, dynamic weights are generated through four steps: feature segmentation, structural similarity calculation, content similarity calculation, and weight normalization.

[0059] Specifically, the feature maps extracted from the image to be detected at each scale, as well as the feature maps at the corresponding scales in the multi-scale feature library, are divided into multiple local blocks of fixed size. For each local block of the image to be detected, the cosine similarity between it and all local blocks at the corresponding scales in the multi-scale feature library is calculated, and the matching local blocks are determined based on the cosine similarity. The structural similarity index between each pair of matching local blocks is calculated. Based on the structural similarity index of all matching block pairs at each scale, the weights corresponding to each scale are generated by statistical aggregation.

[0060] Let's take the first-scale feature layer (F1_test and F1_fused) as an example for illustration: First, the F1_test feature map is uniformly divided according to a fixed spatial size s×s to obtain n non-overlapping feature blocks. These feature blocks form a set P_test = {P1, P2, ..., Pn}.

[0061] Furthermore, based on cosine similarity, select the n most similar features from F1_fused, divide them into blocks of a fixed size (s×s), and then take the average to obtain the set P_fused = {P1_fused, P2_fused, ...,Pn_fused}.

[0062] Furthermore, the SSIM value for each corresponding block is calculated to measure structural consistency:

[0063] in, Represents the feature blocks of the image to be detected The local average value, Represents image feature blocks in a multi-scale feature library The local average value, and They represent and standard deviation C1 and C2 represent the covariance, and are tiny positive constants introduced to prevent the denominator from being zero.

[0064] Furthermore, the SSIM (Structural Similarity Index) between the i-th block of the feature map to be detected and the i-th block matched in the multi-scale feature library is summed. The SSIM values ​​of all n block pairs are accumulated, and the sum is divided by the total number of blocks n to obtain the attention weights of the first-scale feature layer.

[0065] Determined using the same method and The weighted distance for each scale is calculated based on the determined weights of each feature layer, and a multi-scale fusion score is calculated. After obtaining the multi-scale fusion score, it is compared with a preset threshold. If the multi-scale fusion score is greater than or equal to the preset threshold, it is determined that there is a defect on the metal surface to be detected; if the multi-scale fusion score is less than the preset threshold, it is determined that there is no defect on the metal surface to be detected. The preset threshold value can be set according to actual conditions, and this embodiment does not impose specific limitations.

[0066] The solution proposed in this application is a structural similarity attention mechanism. It uses structural similarity evaluation as an adjustment factor for attention weights. When calculating the attention distribution, it considers not only the similarity between features, but also the structural similarity between image patches. Compared with single content attention, the introduction of structural similarity evaluation enables the model to better maintain the integrity of the image structure and performs better in terms of edge and texture preservation.

[0067] A multi-scale feature anomaly evaluation method is introduced. The first feature layer uses weighted Euclidean distance, the second feature layer uses weighted Manhattan distance, and the third feature layer, considering its semantic information, uses weighted cosine distance. All three feature layers incorporate a structural similarity attention mechanism to dynamically adjust the weights. Finally, a comprehensive score is used to determine whether a defect exists. This avoids the limitations of a single evaluation method and improves the robustness of the defect determination method.

[0068] According to another aspect of the embodiments of this application, a metal surface defect detection apparatus for implementing the above-described method for detecting metal surface defects is also provided. For example... Figure 6 As shown, the device includes: Dataset construction module 601 is used to construct a dataset containing images of normal metal surfaces; The feature extraction network construction module 602 is used to construct a feature extraction network based on a deep residual network with multi-scale feature cyclic fusion. The dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples. The image feature extraction module 603 is used to input the image of the metal surface to be detected into the feature extraction network to obtain multi-scale features of the image to be detected; The defect detection module 604 is used to calculate the multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library. When the multi-scale fusion score is greater than or equal to a preset threshold, it is determined that there is a defect on the metal surface to be detected.

[0069] It should be noted that the metal surface defect detection device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the metal surface defect detection method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the metal surface defect detection device and the metal surface defect detection method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0070] According to another aspect of the present application, an electronic device corresponding to the metal surface defect detection method provided in the foregoing embodiments is also provided to perform the metal surface defect detection method described above.

[0071] Please refer to Figure 7 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 7 As shown, the electronic device includes: a processor 700, a memory 701, a bus 702, and a communication interface 703. The processor 700, the communication interface 703, and the memory 701 are connected via the bus 702. The memory 701 stores a computer program that can run on the processor 700. When the processor 700 runs the computer program, it executes the metal surface defect detection method provided in any of the foregoing embodiments of this application.

[0072] The memory 701 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 703 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0073] Bus 702 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 701 is used to store programs. After receiving execution instructions, processor 700 executes the programs. The metal surface defect detection method disclosed in any of the aforementioned embodiments of this application can be applied to processor 700, or implemented by processor 700.

[0074] The processor 700 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 700 or by instructions in software form. The processor 700 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 701. Processor 700 reads the information in memory 701 and, in conjunction with its hardware, completes the steps of the above method.

[0075] The electronic device provided in this application embodiment and the metal surface defect detection method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0076] According to another aspect of the present application, a computer-readable storage medium corresponding to the metal surface defect detection method provided in the foregoing embodiments is also provided, wherein a computer program (i.e., a program product) is stored thereon, and when the computer program is run by a processor, it executes the metal surface defect detection method provided in any of the foregoing embodiments.

[0077] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0078] The computer-readable storage medium provided in the above embodiments of this application and the method for detecting metal surface defects provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0079] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for detecting defects on a metal surface, characterized in that, include: Construct a dataset containing images of normal metal surfaces; A feature extraction network is constructed based on a deep residual network with multi-scale feature cyclic fusion. The dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples. The image of the metal surface to be detected is input into the feature extraction network to obtain multi-scale features of the image to be detected; Calculate the multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library. When the multi-scale fusion score is greater than or equal to a preset threshold, it is determined that there is a defect on the metal surface to be detected.

2. The method according to claim 1, characterized in that, The dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples, including: The images in the dataset are input into multiple deep residual modules of the feature extraction network to obtain feature layers at multiple scales. Multi-scale feature cyclic fusion is performed on feature layers of multiple scales to obtain fused features and full-scale fused features; Based on the fusion features and the full-scale fusion features, feature enhancement is performed on feature layers at multiple scales to obtain the multi-scale features of the image. Based on the multi-scale features of multiple images, the multi-scale feature library is obtained.

3. The method according to claim 2, characterized in that, The images in the dataset are input into multiple deep residual modules of the feature extraction network to obtain feature layers at multiple scales, including: The images in the dataset are input into the first deep residual module of the feature extraction network to obtain the first feature layer, which is used to represent the edge and texture information of the image. The first feature layer is input into the second deep residual module of the feature extraction network to obtain the second feature layer, which is used to represent the structural information of the image. The second feature layer is input into the third deep residual module of the feature extraction network to obtain the third feature layer, which is used to represent the semantic information of the image.

4. The method according to claim 3, characterized in that, Multi-scale feature cyclic fusion is performed on feature layers at multiple scales to obtain fused features and full-scale fused features, including: Upsampling is performed on the second and third feature layers; The first feature layer and the upsampled second feature layer are weighted and fused to obtain the first fused feature; The first fused feature is weighted and fused with the upsampled third feature layer to obtain the second fused feature; The second fusion feature is weighted and fused with the first feature layer to obtain the full-scale fusion feature.

5. The method according to claim 4, characterized in that, Based on the fusion features and the full-scale fusion features, feature enhancement is performed on feature layers at multiple scales to obtain the multi-scale features of the image, including: The full-scale fusion feature is weighted and fused with the first feature layer to obtain the fused first-scale feature; After downsampling the first fusion feature, it is weighted and fused with the second feature layer to obtain the fused second scale feature; After downsampling the second fusion feature, it is weighted and fused with the third feature layer to obtain the fused third scale feature; Based on the fused first-scale features, fused second-scale features, and fused third-scale features, multi-scale features of the image are obtained.

6. The method according to claim 1, characterized in that, Calculating the multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library includes: The first anomaly score of the image to be detected and the multi-scale feature library are calculated based on the weighted Euclidean distance at the first scale feature layer. The second anomaly score in the second-scale feature layer is calculated based on the weighted Manhattan distance and the multi-scale feature library; The third anomaly score of the image to be detected and the multi-scale feature library are calculated based on the weighted cosine distance at the third-scale feature layer. The multi-scale fusion score is obtained by weighted summation of the first anomaly score, the second anomaly score, and the third anomaly score.

7. The method according to claim 6, characterized in that, Before calculating the first anomaly score of the first scale feature layer based on the weighted Euclidean distance between the image to be detected and the multi-scale feature library, the following steps are also included: The feature maps of each scale extracted from the image to be detected, as well as the feature maps of the corresponding scale in the multi-scale feature library, are divided into multiple local blocks of fixed size. For each local patch in the image to be detected, calculate its cosine similarity with all corresponding local patches in the multi-scale feature library, and determine the matching local patch based on the cosine similarity. Calculate the structural similarity index between each pair of matching local blocks; Based on the structural similarity index of all matching block pairs at each scale, the weights corresponding to each scale are generated through statistical aggregation.

8. A device for detecting defects on metal surfaces, characterized in that, include: The dataset building module is used to construct datasets containing images of normal metal surfaces; The feature extraction network construction module is used to construct a feature extraction network based on a deep residual network with multi-scale feature cyclic fusion. The dataset is input into the feature extraction network to obtain a multi-scale feature library of normal samples. The image feature extraction module is used to input the image of the metal surface to be detected into the feature extraction network to obtain multi-scale features of the image to be detected; The defect detection module is used to calculate the multi-scale fusion score of the multi-scale features of the image to be detected and the multi-scale feature library. When the multi-scale fusion score is greater than or equal to a preset threshold, it is determined that there is a defect on the metal surface to be detected.

9. An electronic device, characterized in that, It includes a processor and a memory storing program instructions, the processor being configured to perform the method for detecting metal surface defects as described in any one of claims 1 to 7 when executing the program instructions.

10. A computer storage medium, characterized in that, It stores computer-readable instructions that are executed by a processor to implement the method for detecting metal surface defects as described in any one of claims 1 to 7.

Images