Lightweight metal surface defect detection method and system

Abstract

The invention relates to the technical field of metal defect detection, and provides a lightweight metal surface defect detection method and system, and the method comprises the steps: inputting a to-be-detected metal surface image into a trained defect detection model, so as to achieve the detection of metal surface defects; the defect detection model is a lightweight RT-DETR model, a feature fusion module in the lightweight RT-DETR model is a spatial semantic information enhancement module, and the spatial semantic information enhancement module is used for carrying out spatial attention processing on input low-level features, splicing and fusing the processed low-level features and input high-level features, and outputting the processed low-level features and the input high-level features; the method comprises the following steps of: obtaining a fused feature, adding the fused feature and a low-level feature, dividing the added feature into three paths, processing the feature by respectively utilizing a 1 * 1 convolution kernel, cavity convolution and average pooling, fusing processing results, performing depth separable convolution processing on the fused results, performing residual connection on the processed feature and the feature before processing, and obtaining a fused feature; the result is the output of the spatial semantic information enhancement module.

Description

Technical Field

[0001] This invention relates to the field of metal defect detection technology, and mainly to a lightweight method and system for detecting metal surface defects. Background Technology

[0002] In the industrial sector, defect detection of manufactured metal surfaces is essential to ensure product quality. Historically, this was largely done manually, a labor-intensive method with a high false positive rate. With the advancements in machine learning, researchers have proposed automated learning-based methods for metal surface defect detection. These methods utilize automated models to achieve high-precision detection of metal surface defects while significantly reducing labor costs; however, they are highly dependent on data. In academia, genetic algorithms are also used to automatically learn morphological parameters for metal surface defect detection. This method avoids the subjectivity and inconsistencies of manual parameter tuning in traditional methods, but its ability to detect complex texture defects or minute defects remains relatively weak. Therefore, while machine learning theoretically holds immense potential in metal surface defect detection, existing machine learning methods still have some limitations in practical industrial applications.

[0003] With the continuous development of deep learning, neural networks are being used more and more widely. For example, convolutional neural networks and related algorithms have been widely applied in the field of metal surface defect detection. For instance, researchers have proposed a bearing surface defect detection method based on a hierarchical attention mechanism. This method significantly improves the detection performance of multi-scale defects by combining features at three levels: texture, semantics, and instance. However, this method has high computational complexity and only applies to bearing surface defects; its performance on other metal surfaces has not been verified, resulting in low detection accuracy. Furthermore, researchers have proposed a PCB defect detection method using multi-scale feature enhancement and an attention mechanism. This method suppresses redundant features through NAM modules, enhancing the saliency of the target region. However, this method does not consider robustness testing in real-world industrial scenarios such as noise and illumination, and it suffers from high computational complexity. Summary of the Invention

[0004] The purpose of this invention is to provide a lightweight method and system for detecting metal surface defects, so as to solve the problems of low accuracy and large computational load in existing metal surface defect detection methods.

[0005] To solve the above-mentioned technical problems, this invention provides a lightweight method for detecting defects on metal surfaces, comprising the following steps: 1) Acquire images of the metal surface to be inspected; 2) The image of the metal surface to be detected is input into the trained defect detection model to detect defects on the metal surface. The defect detection model is a lightweight RT-DETR model. The feature fusion module in the lightweight RT-DETR model is a spatial semantic information enhancement module. The spatial semantic information enhancement module is used to perform spatial attention processing on the input low-level features and concatenate and fuse the processed low-level features with the input high-level features to obtain the fused features. The fused features are added to the low-level features and divided into three paths. The first path uses a 1×1 convolution kernel to obtain the first feature, the second path uses dilated convolution to obtain the second feature, and the third path uses average pooling to obtain the third feature. The first feature and the second feature are concatenated and then fused with the third feature. The fused result is subjected to depthwise separable convolution processing. The result after depthwise separable convolution processing is residually connected with the result without depthwise separable convolution processing. The result is the output of the spatial semantic information enhancement module.

[0006] Furthermore, the dilated convolution includes at least two convolution kernels of the same size, each with a different dilation rate. The features in the second path are processed using different convolution kernels, and the splicing result of the processing results of each convolution kernel is the second feature.

[0007] Furthermore, the encoder in the lightweight RT-DETR model employs an adaptive sparse self-attention-based encoder. This encoder feeds the input features into two attention branches. The first attention branch processes the product of the key vector and the query vector using sparse self-attention, while the second attention branch processes the product of the key vector and the query vector using dense self-attention. The results of the first and second attention branches are added together and multiplied by the value vector for feature space mapping. The result of the feature space mapping is added to the input features of the encoder based on adaptive sparse self-attention. The added result is then normalized by adding it to the input features of the encoder again. The resulting result is input into a feedforward neural network for further processing. The result of the feedforward network processing is then normalized by adding it to the result before inputting it into the feedforward neural network to obtain the output of the adaptive sparse self-attention encoder.

[0008] Furthermore, the first attention branch employs ReLU. 2 Activation function on key vector K and query vector Q The product is processed.

[0009] Furthermore, the second attention branch uses the Softmax activation function on the key vector. K and query vector Q The product is processed.

[0010] Furthermore, the backbone network of the lightweight RT-DETR model employs a lightweight feature extraction module. The features input to this lightweight feature extraction module are divided into two paths. One path undergoes convolution processing to obtain the convolution result, and the other path undergoes convolution processing and is then processed using a dual PCBlock module to obtain the dual PCBlock module processing result. The dual PCBlock module processing result is concatenated with the first and second convolution results and then divided into two paths. One path undergoes convolution processing to obtain the convolution result, and the other path undergoes convolution processing and is then processed using an MCBlock module to obtain the MCBlock processing result. The convolution processing result and the MCBlock processing result are then concatenated and subjected to convolution processing. The processing result is the output of the backbone network.

[0011] Furthermore, the dual PCBlock module includes two PCBlock modules. The convolutional features are input and processed by the first PCBlock module to obtain the first PCBlock processed features. The second PCBlock module processes the features processed by the first PCBlock module to obtain the second PCBlock processed features. The concatenation result of the first PCBlock processed features and the second PCBlock processed features is the dual PCBlock module processing result.

[0012] Furthermore, the MCBlock module includes N convolutional kernels of the same size. The input features are processed by the N convolutional kernels of the same size to obtain features of the corresponding size. The obtained features are concatenated with the unconvolutional data, and the concatenation result is the output of the MCBlock module.

[0013] Furthermore, N is 3.

[0014] The present invention also provides a lightweight metal surface defect detection system, including a processor for running a computer program to implement a lightweight metal surface defect detection method, comprising: 1) Acquire images of the metal surface to be inspected; 2) The image of the metal surface to be detected is input into the trained defect detection model to detect defects on the metal surface. The defect detection model is a lightweight RT-DETR model. The feature fusion module in the lightweight RT-DETR model is a spatial semantic information enhancement module. The spatial semantic information enhancement module is used to perform spatial attention processing on the input low-level features and concatenate and fuse the processed low-level features with the input high-level features to obtain the fused features. The fused features are added to the low-level features and divided into three paths. The first path uses a 1×1 convolution kernel to obtain the first feature, the second path uses dilated convolution to obtain the second feature, and the third path uses average pooling to obtain the third feature. The first feature and the second feature are concatenated and then fused with the third feature. The fused result is subjected to depthwise separable convolution processing. The result after depthwise separable convolution processing is residually connected with the result without depthwise separable convolution processing. The result is the output of the spatial semantic information enhancement module.

[0015] Furthermore, the dilated convolution includes at least two convolution kernels of the same size, each with a different dilation rate. The features in the second path are processed using different convolution kernels, and the splicing result of the processing results of each convolution kernel is the second feature.

[0016] Furthermore, the encoder in the lightweight RT-DETR model employs an adaptive sparse self-attention-based encoder. This encoder feeds the input features into two attention branches. The first attention branch processes the product of the key vector and the query vector using sparse self-attention, while the second attention branch processes the product of the key vector and the query vector using dense self-attention. The results of the first and second attention branches are added together and multiplied by the value vector for feature space mapping. The result of the feature space mapping is added to the input features of the encoder based on adaptive sparse self-attention. The added result is then normalized by adding it to the input features of the encoder again. The resulting result is input into a feedforward neural network for further processing. The result of the feedforward network processing is then normalized by adding it to the result before inputting it into the feedforward neural network to obtain the output of the adaptive sparse self-attention encoder.

[0017] Furthermore, the first attention branch employs ReLU. 2 Activation function on key vector K and query vector Q The product is processed.

[0018] Furthermore, the second attention branch uses the Softmax activation function on the key vector. K and query vector Q The product is processed.

[0019] Furthermore, the backbone network of the lightweight RT-DETR model employs a lightweight feature extraction module. The features input to this lightweight feature extraction module are divided into two paths. One path undergoes convolution processing to obtain the convolution result, and the other path undergoes convolution processing and is then processed using a dual PCBlock module to obtain the dual PCBlock module processing result. The dual PCBlock module processing result is concatenated with the first and second convolution results and then divided into two paths. One path undergoes convolution processing to obtain the convolution result, and the other path undergoes convolution processing and is then processed using an MCBlock module to obtain the MCBlock processing result. The convolution processing result and the MCBlock processing result are then concatenated and subjected to convolution processing. The processing result is the output of the backbone network.

[0020] Furthermore, the dual PCBlock module includes two PCBlock modules. The convolutional features are input and processed by the first PCBlock module to obtain the first PCBlock processed features. The second PCBlock module processes the features processed by the first PCBlock module to obtain the second PCBlock processed features. The concatenation result of the first PCBlock processed features and the second PCBlock processed features is the dual PCBlock module processing result.

[0021] Furthermore, the MCBlock module includes N convolutional kernels of the same size. The input features are processed by the N convolutional kernels of the same size to obtain features of the corresponding size. The obtained features are concatenated with the unconvolutional data, and the concatenation result is the output of the MCBlock module.

[0022] Furthermore, N is 3.

[0023] As an improved invention, this invention incorporates a spatial semantic information enhancement module as a feature fusion module in a lightweight RT-DETR model. It fuses low-level feature maps processed by spatial attention with high-level feature maps to achieve spatial location fusion. The fused features are then added to the low-level features. Subsequently, 1×1 convolutional kernels, dilated convolutions, and global average pooling are used to process the fusion result, expanding the receptive field and enhancing the expressive power of the feature maps. Finally, depthwise separable convolutions are used to process the fusion result, improving the model's sensitivity to defect features and its localization accuracy. The spatial semantic information enhancement module reduces the model's parameter count and computational cost through grouped convolutions, channel compression, and depthwise separable convolutions. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the Lightweight Feature Extraction Module (LFEM). Figure 3 This is a schematic diagram of the ASSE structure of an encoder based on adaptive sparse self-attention; Figure 4 This is a schematic diagram of the spatial semantic information enhancement module (SEIE). Figure 5 This is a comparison chart of the detection effects before and after the improvement of metal surface defects. Detailed Implementation

[0025] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0026] This invention uses a spatial semantic information enhancement module as a feature fusion module in a lightweight RT-DETR model. After spatial attention enhancement, the low-level feature map is fused with the high-level feature map to achieve the fusion of spatial location information. The fused feature is then added to the low-level feature, and the result is divided and processed using 1×1 convolution kernel, dilated convolution, and global average pooling, respectively. The three processing results are then fused and subjected to depthwise separable convolution to improve the model's sensitivity to defect features and its localization accuracy.

[0027] Implementation of a lightweight method for detecting defects on metal surfaces Traditional metal surface defect detection models typically employ numerous convolutions for feature extraction, resulting in a large number of overall model parameters and reduced detection speed. To address this issue, this invention proposes a metal surface defect detection method, such as... Figure 1 As shown, firstly, images of metal surface defects are acquired, and then a lightweight RT-DETR model is used to process the metal surface defect images to determine whether defects exist on the metal surface.

[0028] To improve the accuracy of metal surface defect detection and avoid the loss of some metal surface defects due to their small size, this invention proposes a Spatial Semantic Information Enhancement (SEIE) module for feature fusion to replace the Cross-Scale Feature Fusion Module (CCFM) in the RT-DETR model. The structure of this module is as follows: Figure 4As shown, the SEIE module fuses the spatial location information of the low-level feature map into the high-level feature map, and performs feature fusion through convolutional kernels with different receptive fields to enhance the expressive power of the features. Specifically, the low-level features input to the spatial semantic information enhancement module obtain spatial information weights after spatial attention processing. These spatial information weights are then processed with the high-level features using a Hadamard product operation to fuse spatial location information into the high-level features. The processed result is then added back to the low-level features to obtain the fused features. Subsequently, the fused features are divided into three paths. The first path uses a single convolutional kernel to process the input features to obtain the first feature. As an example, the size of a single convolutional kernel is 1×1. The second path uses dilated convolution to process the input features to obtain the second feature. This dilated convolution includes at least two convolutional kernels of the same size with different dilation rates. Different convolutional kernels are used to process the features in the second path, and the concatenation result of the processing results of each convolutional kernel is the second feature. As an example, the dilated convolution includes three 3×3 convolutional kernels with dilation rates of 1, 3, and 5, respectively. The third path uses average pooling to process the input features to obtain the third feature. After obtaining the first, second, and third features, the first and second features are concatenated and then fused with the third feature through a Hadamard product operation. The fused result is then subjected to depthwise separable convolution. The result of the depthwise separable convolution is then residually concatenated with the result without depthwise separable convolution, and the result is the output of the spatial semantic information enhancement module. The improved spatial semantic information enhancement module not only demonstrates excellent accuracy in the metal surface defect task, but also reduces the number of model parameters and computational cost through channel compression and depthwise separable convolution operations. Although there is a slight decrease in speed, it meets the requirements of real-time detection in industrial fields, achieving a balance between accuracy and speed.

[0029] To further reduce the computational and parameter requirements for metal surface defect detection and enhance the feature extraction capability of the network, this invention replaces the backbone network ResNet18 of the RT-DETR model with the lightweight feature extraction module LFEM.

[0030] like Figure 2As shown, the backbone network of the lightweight RT-DETR model is a lightweight feature extraction module (LFEM). After entering the LFEM, features are split into two paths: one path is processed by a 1×1 convolutional kernel to obtain the convolution result, and the other path, after being processed by a 1×1 convolutional kernel, is processed by a dual PCBlock module to obtain the dual PCBlock module result. The dual PCBlock module result is concatenated with the results of the first and second 1×1 convolutional kernels, and then split into two paths by a channel segmentation module. One path is processed by a 1×1 convolutional kernel to obtain the convolution result, and the other path, after being processed by a 1×1 convolutional kernel, is processed by an MCBlock module to obtain the MCBlock result. The concatenated convolution result and the MCBlock result are then concatenated, and the result is the output of the backbone network.

[0031] The dual PCBlock module comprises two PCBlock modules. The convolution result after processing with a 1×1 convolution kernel is first input into the first PCBlock module for processing, yielding the first PCBlock processed features. Subsequently, the first PCBlock processed features are processed using the second PCBlock module, resulting in the second PCBlock processed features. The concatenation of the first and second PCBlock processed features is the result of the dual PCBlock module processing. The PCBlock module contains two partial convolution modules. The MCBlock module includes N convolution kernels of the same size. The input features are processed by a 1×1 convolution kernel and then by each of the N convolution kernels of the same size to obtain features of the corresponding size. These features are then concatenated with the unconvolutioned data, and the concatenation result is the output of the MCBlock module. The MCBlock module divides the features into multiple groups for computation by segmenting the number of channels, effectively reducing the computational load and parameter count per group and improving computational speed. Simultaneously, the MCBlock module utilizes a hierarchical connection method to expand the receptive field, enhancing the network's feature extraction capabilities.

[0032] Taking a four-way convolutional processing branch as an example, let the four branches xi∈{x1,x2,x3,x4}, where each branch xi has the same number of channels, but the number of channels in xi is reduced to one-quarter of the original. Assume that all convolutional kernels in the MCBlock module are 3×3 convolutions, and N is 3, meaning there are three 3×3 convolutions in total. When data is input into the MCBlock module, the features in branch x1 are not processed by convolution and retain their original features, with the output denoted as y1. The features in branch x2 are processed by a 3×3 convolution to obtain the output y2 of branch x2. The features in branch x3 and the output y2 of branch x2 are processed together by a 3×3 convolution to obtain the output y3 of branch x3. The features in branch x4 and the output y3 of branch x3 are processed together by a 3×3 convolution to obtain the output y4 of branch x4. The above results yi∈{y1,y2,y3,y4} are concatenated using Concat, and the number of channels in the concatenated result is adjusted using a 1×1 convolution to obtain the feature extraction result.

[0033] In metal surface defect detection, there are situations where metal surface defects are highly similar to the background, making them difficult to distinguish. Traditional methods often use self-attention mechanisms to calculate the relationships between all basic processing unit tokens, thereby capturing global dependencies and achieving the capture of metal surface defects. However, this method introduces noise interference from irrelevant regions. To address this issue, this invention replaces the encoder AIFI (Attention-based Intra-scale Feature Interaction) in the RT-DETR model with an adaptive sparse self-attention based encoder (ASSE). The structure of the ASSE encoder is as follows: Figure 3 As shown. When the features are input into the ASSE encoder, they first undergo normalization and feature space mapping to obtain the query vector. Q Key vector K Sum value vector V The key vector K and query vector Q The matrix multiplication process yields the result of the multiplication, which is then input into the first and second attention branches of the encoder for further processing. The first attention branch utilizes Sparse Self-Attention (SSA) to process the product of the key vector and query vector, effectively reducing interference from irrelevant regions. The second attention branch utilizes Dense Self-Attention (DSA) to process the product of the key vector and query vector. K and query vector QThe product of the two attention branches is processed to ensure information integrity. The results of the first and second attention branches are then concatenated and combined with the value vector. V After matrix multiplication, feature space mapping is performed. The result of feature space mapping is added to the encoder input features. At this point, the output of the adaptive sparse attention module (ASSA) is obtained. The result of the adaptive sparse attention module (ASSA) is added to the encoder input and normalized before being input to the feedforward neural network for processing. The processing result is added to the data before the feedforward neural network processing and normalized to obtain the output of the adaptive sparse self-attention encoder (ASSE).

[0034] Specifically, key vector K and query vector Q The product is then processed using ReLU after entering the first attention branch. 2 Instead of the Softmax activation function, a sparse attention function is used to filter out token matches with low relevance, effectively reducing interference from irrelevant regions. (Key vector) K and query vector Q After the product enters the second attention branch, the second attention branch first processes the key vector using the Softmax activation function. K and query vector Q The product of the two attention branches preserves the relationships between all tokens, ensuring information integrity. Finally, the results from the two attention branches are weighted and fused.

[0035] By using an adaptive sparse self-attention encoder to process data using both sparse and dense self-attention, the detection capability of the model can be effectively enhanced.

[0036] As another embodiment, the lightweight RT-DETR model can simply replace the feature fusion module CCFM with the spatial semantic information enhancement module SEIE, or it can replace the backbone of the RT-DETR model with the lightweight feature extraction module LFEM and replace the encoder AIFI of the RT-DETR model with the encoder ASSE while replacing the feature fusion module CCFM with the spatial semantic information enhancement module SEIE.

[0037] To verify that the lightweight RT-DETR model proposed in this invention can still achieve accurate metal surface defect detection while reducing the number of parameters and computational load, the lightweight RT-DETR model proposed in this embodiment is used to process the metal defect dataset.

[0038] This invention uses two datasets. The first is the Northeastern University Steel Surface Defect Detection Dataset (NEU-DET), which includes six types of defects: rolled-in scale (Rs), patches (Pa), cracks (Cr), pitted surface (Ps), inclusions (In), and scratches (Sc). This dataset contains 1800 images, each with a resolution of 200×200. The second dataset is GC10-DET, a publicly available dataset containing 2294 images of steel plate surface defects with a resolution of 2048×1000. It includes ten surface defect types: punching (Pu), weld (Wl), crescent pattern (Cg), water stain (Ws), oil stain (Os), thread-like pattern (Ss), inclusions (In), roll mark (Rp), crease (Cr), and edge waviness (Wf).

[0039] To verify the effectiveness of the module proposed in this invention, ablation experiments were conducted on the NEU-DET dataset, as shown in Table 1.

[0040] Table 1 Ablation Experiment

[0041] Table 1 shows that in the traditional RT-DETR model, the mean mAP accuracy is 74.0%, the FPS speed is 119.04, the GFLOPs computation is 57.0, and the number of params is 19.87M. When the LFEM, ASSE, and SEIE modules are added to the traditional RT-DETR model respectively for improvement, the mean mAP accuracy of the improved RT-DETR model is improved in all aspects. Furthermore, when all three modules are added to the traditional RT-DETR model simultaneously, the mean mAP accuracy reaches 76.8%, the GFLOPs computation is reduced to 32.6, a reduction of 42.8% compared to the original; and the number of params is only 9.18M, ​​a reduction of 53.7% compared to the original. Ablation experiments demonstrate the effectiveness of the modules proposed in this invention. Figure 5 The image shows the detection performance of the traditional RT-DETR model and the lightweight RT-DETR model on the NEU-DET dataset. It can be seen that the lightweight RT-DETR model makes up for the missed detections and false detections of the traditional RT-DETR model.

[0042] In addition, to test the superiority of the lightweight RT-DETR model proposed in this invention, the lightweight RT-DETR model was compared with other state-of-the-art models, including RT-DETR, YOLOv7, YOLOv8m, YOLOv9m, YOLOv10m, YOLOv11m and YOLOv12m. The comparison results are shown in Table 2.

[0043] Table 2 Comparison with other algorithms on the NEU-DET dataset

[0044] Among them, YOLOv8m, YOLOv9m, and YOLOv12m have higher detection accuracy for Cr (38.4%), Pa (96.9%), Ps (85.0%), and Rs (77.5%), but these models are slower and have relatively large parameter counts (Params) and computational GFLOPs (GFLOPs). In contrast, the lightweight RT-DETR model proposed in this invention achieves the highest detection accuracy for In and Sc, and performs excellently in terms of average accuracy (mAP) at 76.8%. Furthermore, the lightweight RT-DETR model also demonstrates superior lightweight design, with parameter counts (Params) and computational GFLOPs (GFLOPs) of 9.18 and 32.6, respectively, which are optimal compared to other algorithms. Although the FPS (Functions Per Second) is slightly lower than the original RT-DETR, it still meets the requirements of industrial real-time detection and outperforms most algorithms. These results demonstrate the superior performance of the lightweight RT-DETR model proposed in this invention in terms of both accuracy and lightweight design.

[0045] In addition, the GC10-DET dataset was used to verify the generalization ability of the model, and the results are shown in Table 3.

[0046] Table 3 Comparison with other algorithms on the GC10-DET dataset

[0047] The performance parameter comparison in Table 3 illustrates that the lightweight RT-DETR model proposed in this invention exhibits excellent generalization ability. Compared to other models, the proposed model achieves a mean accuracy of 68.6% mAP, an FPS of 105.26, a GFLOPs computation of 32.6, and 9.18M params. When processing the GC10-DET dataset, the lightweight RT-DETR model proposed in this invention significantly outperforms other models in terms of mAP accuracy, GFLOPs computation, and params. Although the FPS speed is slightly lower than the original RT-DETR, it still meets the requirements for real-time industrial detection.

[0048] In summary, these results demonstrate that the lightweight RT-DETR model proposed in this invention is particularly suitable for metal surface defect detection tasks.

[0049] Implementation of a lightweight metal surface defect detection system This invention proposes an implementation method for a metal surface defect detection system, including a processor for running a computer program to implement the metal surface defect detection method described above. This method employs a lightweight RT-DETR model, which replaces the feature fusion module CCFM with a spatial semantic information enhancement module SEIE, replaces the backbone of the RT-DETR model with a lightweight feature extraction module LFEM, and replaces the encoder AIFI with an encoder ASSE. The SEIE module is used to fuse spatial location information from low-level feature maps into high-level feature maps, and performs feature fusion using convolutional kernels with different receptive fields to enhance the expressive power of the features. Specifically, the low-level features input to the spatial semantic information enhancement module undergo spatial attention processing to obtain spatial information weights. These weights are then processed with the high-level features using a Hadamard product operation to fuse spatial location information into the high-level features. The processed result is then added back to the low-level features to obtain the fused features. Subsequently, the fused features are divided into three paths. The first path processes the input features using a single convolutional kernel to obtain the first feature. In one embodiment, the size of the single convolutional kernel is 1×1. The second path processes the input features using dilated convolution to obtain the second feature. This dilated convolution includes at least two convolutional kernels of the same size with different dilation rates. Different convolutional kernels are used to process the features in the second path, and the concatenation of the processing results is the second feature. In one embodiment, the dilated convolution includes three 3×3 convolutional kernels with dilation rates of 1, 3, and 5, respectively. The third path processes the input features using average pooling to obtain the third feature. After obtaining the first, second, and third features, the first and second features are concatenated and then fused with the third feature using a Hadamard product operation. The fused result is then subjected to depthwise separable convolution. The result of the depthwise separable convolution is residually connected to the result without depthwise separable convolution, and the result is the output of the spatial semantic information enhancement module. The improved spatial semantic information enhancement module not only demonstrates excellent accuracy in metal surface defect tasks, but also reduces the number of model parameters and computational cost through channel compression and depthwise separable convolution operations. Although there is a slight decrease in speed, it meets the requirements of real-time detection in industrial fields and achieves a balance between accuracy and speed.

[0050] This invention replaces the ResNet18 backbone network of the RT-DETR model with a lightweight feature extraction module (LFEM). For example... Figure 2As shown, the backbone network of the lightweight RT-DETR model is a lightweight feature extraction module (LFEM). After entering the LFEM, features are split into two paths: one path is processed by a 1×1 convolutional kernel to obtain the convolution result, and the other path, after being processed by a 1×1 convolutional kernel, is processed by a dual PCBlock module to obtain the dual PCBlock module result. The dual PCBlock module result is concatenated with the results of the first and second 1×1 convolutional kernels, and then split into two paths by a channel segmentation module. One path is processed by a 1×1 convolutional kernel to obtain the convolution result, and the other path, after being processed by a 1×1 convolutional kernel, is processed by an MCBlock module to obtain the MCBlock result. The concatenated convolution result and the MCBlock result are then concatenated, and the result is the output of the backbone network.

[0051] The dual PCBlock module comprises two PCBlock modules. The convolution result after processing with a 1×1 convolution kernel is first input into the first PCBlock module for processing, yielding the first PCBlock processed features. Subsequently, the first PCBlock processed features are processed using the second PCBlock module, resulting in the second PCBlock processed features. The concatenation of the first and second PCBlock processed features is the result of the dual PCBlock module processing. The PCBlock module contains two partial convolution modules. The MCBlock module includes N convolution kernels of the same size. The input features are processed by a 1×1 convolution kernel and then by each of the N convolution kernels of the same size to obtain features of the corresponding size. These features are then concatenated with the unconvolutioned data, and the concatenation result is the output of the MCBlock module. The MCBlock module divides the features into multiple groups for computation by segmenting the number of channels, effectively reducing the computational load and parameter count per group and improving computational speed. Simultaneously, the MCBlock module utilizes a hierarchical connection method to expand the receptive field, enhancing the network's feature extraction capabilities.

[0052] This invention replaces the encoder AIFI (Attention-based Intra-scale Feature Interaction) in the RT-DETR model with an adaptive sparse self-attention based encoder ASSE. The structure of the ASSE encoder is as follows: Figure 3 As shown. When the features are input into the ASSE encoder, they first undergo normalization and feature space mapping to obtain the query vector. Q Key vector K Sum value vectorV The key vector K and query vector Q The matrix multiplication process yields the result of the multiplication, which is then input into the first and second attention branches of the encoder for further processing. The first attention branch utilizes Sparse Self-Attention (SSA) to process the product of the key vector and query vector, effectively reducing interference from irrelevant regions. The second attention branch utilizes Dense Self-Attention (DSA) to process the product of the key vector and query vector. K and query vector Q The product of the two attention branches is processed to ensure information integrity. The results of the first and second attention branches are then concatenated and combined with the value vector. V After matrix multiplication, feature space mapping is performed. The result of feature space mapping is added to the encoder input features. At this point, the output of the adaptive sparse attention module (ASSA) is obtained. The result of the adaptive sparse attention module (ASSA) is added to the encoder input and normalized before being input to the feedforward neural network for processing. The processing result is added to the data before the feedforward neural network processing and normalized to obtain the output of the adaptive sparse self-attention encoder (ASSE).

[0053] Specifically, key vector K and query vector Q The product is then processed using ReLU after entering the first attention branch. 2 Instead of the Softmax activation function, a sparse attention function is used to filter out token matches with low relevance, effectively reducing interference from irrelevant regions. (Key vector) K and query vector Q After the product enters the second attention branch, the second attention branch first processes the key vector using the Softmax activation function. K and query vector Q The product of the two attention branches preserves the relationships between all tokens, ensuring information integrity. Finally, the results from the two attention branches are weighted and fused.

[0054] By using an adaptive sparse self-attention encoder to process data using both sparse and dense self-attention, the detection capability of the model can be effectively enhanced.

Claims

1. A lightweight method for detecting defects on metal surfaces, characterized in that the steps include... include: 1) Acquire images of the metal surface to be inspected; 2) Input the image of the metal surface to be detected into the trained defect detection model to detect defects on the metal surface; The defect detection model is a lightweight RT-DETR model. The feature fusion module in the lightweight RT-DETR model is a spatial semantic information enhancement module. The spatial semantic information enhancement module is used to perform spatial attention processing on the input low-level features, and then concatenate and fuse the processed low-level features with the input high-level features to obtain the fused features. The fused features are then added to the low-level features, and the added features are divided into three paths. The first path uses a 1×1 convolution kernel to obtain the first feature, the second path uses dilated convolution to obtain the second feature, and the third path uses average pooling to obtain the third feature. The first feature and the second feature are concatenated and then fused with the third feature. The fused result is then subjected to depthwise separable convolution processing. The result of depthwise separable convolution processing is then residually connected with the result without depthwise separable convolution processing. The result is the output of the spatial semantic information enhancement module.

2. The lightweight metal surface defect detection method according to claim 1, characterized in that, The dilated convolution includes at least two convolution kernels of the same size, each with a different dilation rate. The features in the second path are processed using different convolution kernels, and the splicing result of the processing results of each convolution kernel is the second feature.

3. The lightweight metal surface defect detection method according to claim 1, characterized in that, The encoder in the lightweight RT-DETR model employs an adaptive sparse self-attention-based encoder. This encoder feeds the input features into two attention branches. The first attention branch processes the product of the key vector and query vector using sparse self-attention, while the second attention branch processes the product using dense self-attention. The results of the first and second attention branches are added together and multiplied by the value vector for feature space mapping. The result of the feature space mapping is then added to the input features of the adaptive sparse self-attention-based encoder. This sum is then normalized by adding the sum to the encoder input features again. The resulting result is then input into a feedforward neural network for further processing. The feedforward network processing result is then normalized by adding the result to the previous result before inputting it into the feedforward neural network to obtain the output of the adaptive sparse self-attention encoder.

4. The lightweight metal surface defect detection method according to claim 3, characterized in that, The first attention branch uses ReLU. 2 Activation function on key vector K and query vector Q The product is processed.

5. The lightweight metal surface defect detection method according to claim 3, characterized in that, The second attention branch uses the Softmax activation function on the key vector. K and query vector Q The product is processed.

6. The lightweight metal surface defect detection method according to claim 1, characterized in that, The backbone network of the lightweight RT-DETR model adopts a lightweight feature extraction module. The features input to the lightweight feature extraction module are divided into two paths. One path is processed by convolution to obtain the convolution result, and the other path is processed by convolution and then processed by a dual PCBlock module to obtain the dual PCBlock module processing result. The processing results of the dual PCBlock module are concatenated with the first and second convolution results, and then divided into two paths. One path is convolved to obtain the convolution result, and the other path is convolved and processed by the MCBlock module to obtain the MCBlock result. The convolution result and the MCBlock result are concatenated and then convolved again. The result is the output of the backbone network.

7. The lightweight metal surface defect detection method according to claim 6, characterized in that, The dual PCBlock module includes two PCBlock modules. The convolutional features are input and processed by the first PCBlock module to obtain the first PCBlock processed features. The second PCBlock module processes the features processed by the first PCBlock module to obtain the second PCBlock processed features. The concatenation result of the first PCBlock processed features and the second PCBlock processed features is the dual PCBlock module processing result.

8. The lightweight metal surface defect detection method according to claim 6, characterized in that, The MCBlock module includes N convolutional kernels of the same size. The input features are processed by the N convolutional kernels of the same size to obtain features of the corresponding size. The obtained features are concatenated with the unconvolutional data, and the concatenation result is the output of the MCBlock module.

9. The lightweight metal surface defect detection method according to claim 8, characterized in that, The value of N is 3.

10. A lightweight metal surface defect detection system, characterized in that, It includes a processor for running a computer program to implement the lightweight metal surface defect detection method as described in any one of claims 1-9 above.

Images