Target detection method for detecting metal surface defects
By improving the YOLOv8 model to StarNet network and bidirectional feature pyramid network, the problems of low efficiency and poor accuracy of traditional detection methods are solved, achieving efficient and accurate metal surface defect detection, adapting to complex textures and tiny defects, and meeting the needs of industrial production.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING XINGHANG MECHANICAL ELECTRICAL EQUIP CO LTD
- Filing Date
- 2025-12-30
- Publication Date
- 2026-05-08
AI Technical Summary
Existing traditional detection methods are inefficient and have poor accuracy, making it difficult to meet the high-precision and high-efficiency requirements of modern industrial production for metal surface defect detection. Deep learning-based models have insufficient detection accuracy when faced with complex textures and tiny defects on metal surfaces, and their adaptability is limited, making them unable to cope with the diverse needs of different lighting conditions and metal materials.
An improved YOLOv8 model is adopted, replacing the backbone network with a StarNet network and replacing the feature pyramid network with a bidirectional feature pyramid network. By combining multi-scale feature extraction and feature fusion, the detection accuracy and efficiency are improved through the multi-level feature extraction units of the StarNet network and the feature weighting processing of the bidirectional feature pyramid network.
It improves the accuracy and efficiency of metal surface defect detection, better adapts to complex textures and tiny defects, reduces computational complexity and inference time, and meets the real-time detection needs in industrial production.
Smart Images
Figure CN121998912A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of metal surface defect detection technology, and in particular to a target detection method for inspecting metal surface defects. Background Technology
[0002] In the field of metal processing and manufacturing, metal surface defect detection is a crucial step in ensuring product quality. Traditional detection methods mainly rely on manual visual inspection or automated inspection systems based on simple image processing techniques. Manual inspection is inefficient and highly subjective, easily leading to missed or false detections due to factors such as fatigue. While automated inspection systems based on traditional image processing techniques improve detection efficiency, their ability to identify complex defects is limited, and they struggle to adapt to different lighting conditions and variations in metal surface materials. In recent years, with the development of deep learning technology, defect detection methods based on convolutional neural networks have gradually emerged, such as the YOLO series of object detection models. These models perform well in terms of real-time performance and accuracy, but when directly applied to metal surface defect detection, they still suffer from insufficient accuracy in detecting minute defects and poor adaptability to complex textures.
[0003] In implementing the embodiments of the present invention, the prior art has at least the following problems or defects: traditional detection methods are inefficient and have poor accuracy, making it difficult to meet the high-precision and high-efficiency requirements of modern industrial production for metal surface defect detection; existing deep learning-based detection models have problems such as insufficient detection accuracy and limited model adaptability when facing complex textures and tiny defects on metal surfaces, and cannot effectively cope with the diverse needs of different lighting conditions and metal materials. Summary of the Invention
[0004] Based on the above analysis, the present invention aims to provide a target detection method for inspecting metal surface defects, in order to solve the problems of insufficient detection accuracy and efficiency of existing traditional metal surface defect detection methods.
[0005] This invention provides a target detection method for inspecting defects on metal surfaces, comprising the following steps:
[0006] Acquire image data of the metal surface to be inspected;
[0007] The metal surface image data is input into the metal surface defect detection model, which is an improvement based on the YOLOv8 model. The improvement includes replacing the backbone network of the YOLOv8 model with the StarNet network and replacing the feature pyramid network of the YOLOv8 model with a bidirectional feature pyramid network.
[0008] The metal surface defect detection model outputs the category, location, and confidence level of the metal surface defects.
[0009] Furthermore, the metal surface defect detection model includes a StarNet network, a bidirectional feature pyramid network, and a detection head connected in sequence;
[0010] The StarNet network receives the metal surface image data and performs multi-scale feature extraction to obtain a multi-scale feature map, which is then sent to the bidirectional feature pyramid network.
[0011] The bidirectional feature pyramid network performs feature fusion and enhancement on the multi-scale feature map to obtain an enhanced multi-scale feature map, and then performs feature weighting processing on the enhanced multi-scale feature map to obtain weighted multi-scale features, which are then sent to the detection head.
[0012] The detection head identifies and processes the weighted multi-scale features, and outputs the type, location, and confidence level of the metal surface defects.
[0013] Furthermore, the StarNet network includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, and a fourth feature extraction unit connected in series.
[0014] The first feature extraction unit receives the metal surface image data and outputs a first-scale feature map to the second feature extraction unit;
[0015] The second feature extraction unit receives the first-scale feature map and outputs the second-scale feature map to the third feature extraction unit and the bidirectional feature pyramid network;
[0016] The third feature extraction unit receives the second-scale feature map and outputs the third-scale feature map to the fourth feature extraction unit and the bidirectional feature pyramid network.
[0017] The fourth feature extraction unit receives the third-scale feature map and outputs the fourth-scale feature map to the bidirectional feature pyramid network.
[0018] Furthermore, the first feature extraction unit includes a StarNet module. The StarNet module of the first feature extraction unit receives the metal surface image data, processes the metal surface image data through the StarNet module, and outputs the first scale feature map.
[0019] The second feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the second feature extraction unit receives the first-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the second feature extraction unit processes the downsampled feature map and outputs the second-scale feature map.
[0020] The third feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the third feature extraction unit receives the second-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the third feature extraction unit processes the downsampled feature map and outputs the third-scale feature map.
[0021] The fourth feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the fourth feature extraction unit receives the third-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the fourth feature extraction unit processes the downsampled feature map and outputs the fourth-scale feature map.
[0022] Furthermore, the StarNet module includes a first depthwise separable convolutional layer, a first fully connected layer, a ReLU6 activation function layer, a second fully connected layer, a batch normalization layer, and a second depthwise separable convolutional layer connected in sequence.
[0023] The first depthwise separable convolutional layer receives the input feature map and performs feature extraction, and outputs the first feature map to the first fully connected layer;
[0024] The first fully connected layer receives the first feature map and performs nonlinear combination of the channels of the input feature map through a multinomial kernel function to realize channel dimension transformation, and outputs the second feature map to the ReLU6 activation function layer;
[0025] The ReLU6 activation function layer performs non-linear activation on the second feature map and outputs the third feature map to the second fully connected layer;
[0026] The second fully connected layer receives the third feature map and restores the channel dimension, then outputs the fourth feature map to the batch normalization layer;
[0027] The batch normalization layer normalizes the fourth feature map and outputs the fifth feature map to the second depth separable convolutional layer.
[0028] The second depthwise separable convolutional layer receives the fifth feature map and performs depth feature extraction, outputting the final output feature map of the StarNet module.
[0029] Furthermore, the computational expression for the polynomial kernel function is:
[0030]
[0031] Where, x j For the j-th channel of the feature map input to the first fully connected layer, w ij b is a learnable weight parameter. ij Here, p is the learnable bias parameter, p is the polynomial order, and C is the polynomial order. in y is the number of channels in the input feature map. i This refers to the i-th channel of the feature map output by the first fully connected layer.
[0032] Furthermore, the bidirectional feature pyramid network includes an upsampling fusion module, a feature refinement module, a downsampling fusion module, a first multi-scale channel attention module, a second multi-scale channel attention module, and a third multi-scale channel attention module;
[0033] The upsampling fusion module receives the multi-scale feature map output by the StarNet network, fuses the high-level features with the low-level features through an upsampling operation, generates the first set of multi-scale feature maps, and outputs them to the feature refinement module.
[0034] The feature refinement module performs convolution enhancement on the first set of multi-scale feature maps and outputs the second set of multi-scale feature maps to the downsampling fusion module.
[0035] The downsampling fusion module performs a downsampling operation on the second group of multi-scale feature maps, and then fuses the enhanced low-level features with the high-level features again to generate P3 feature map, P4 feature map and P5 feature map. The P3 feature map is output to the first multi-scale channel attention module, the P4 feature map is output to the second multi-scale channel attention module, and the P5 feature map is output to the third multi-scale channel attention module.
[0036] The first multi-scale channel attention module performs feature weighting on the P3 feature map, the second multi-scale channel attention module performs feature weighting on the P4 feature map, and the third multi-scale channel attention module performs feature weighting on the P5 feature map, and then sends the weighted features to the detection head.
[0037] Furthermore, the feature refinement module includes a series of depthwise separable convolutional layers, batch normalization layers, and Swish activation function layers;
[0038] The depthwise separable convolutional layer receives the first set of multi-scale feature maps and performs feature extraction, then outputs the extracted features to the batch normalization layer.
[0039] The batch normalization layer receives the extracted features from the depth-separable convolutional layer and performs normalization processing to obtain the normalized features.
[0040] The Swish activation function layer performs nonlinear activation on the features normalized by the batch normalization layer, and outputs the second set of multi-scale feature maps.
[0041] Furthermore, the first multi-scale channel attention module, the second multi-scale channel attention module, and the third multi-scale channel attention module all include a dual-path feature transformation unit and an adaptive fusion unit;
[0042] The dual-path feature transformation unit is used to receive the input feature map and generate preliminary channel attention weights, which are then output to the adaptive fusion unit.
[0043] The adaptive fusion unit is used to receive preliminary channel attention weights from different feature maps and perform weighted fusion to generate the final multi-scale fused attention weights.
[0044] Furthermore, the dual-path feature transformation unit processes the input feature map using the following steps:
[0045] Global average pooling is performed on the input feature map X to obtain the channel statistics vector;
[0046] The channel statistical vector z is input into the first fully connected sub-path and the lightweight convolutional sub-path respectively for feature transformation. The first fully connected sub-path includes a fully connected layer, and the lightweight convolutional sub-path includes a 1×1 convolutional layer.
[0047] The outputs of the first fully connected sub-path and the lightweight convolutional sub-path are fused and then processed by the Sigmoid activation function to generate preliminary channel attention weights.
[0048] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0049] 1. By replacing the backbone network of the YOLOv8 model with the StarNet network, this invention can more effectively extract multi-scale features from metal surface images. The multi-level feature extraction units of the StarNet network, combined with depthwise separable convolution and multinomial kernel functions, not only improve the efficiency of feature extraction but also enhance the model's sensitivity to complex textures and minute defects, thereby improving the accuracy of metal surface defect detection and solving the problem of insufficient ability of traditional detection methods to identify complex defects.
[0050] 2. A bidirectional feature pyramid network is introduced to achieve bidirectional feature fusion and enhancement. This network structure effectively combines high-level semantic information with low-level detailed information through upsampling and downsampling operations, and performs weighted processing on features through a multi-scale channel attention module, enabling the model to better capture features at different scales when detecting defects on metal surfaces.
[0051] 3. The optimized detection model structure and feature processing mechanism of this invention improve overall detection efficiency. While ensuring detection accuracy, it reduces the computational complexity and inference time of the model, enabling it to better adapt to the real-time detection needs in industrial production. This efficient and accurate detection capability provides the metal processing and manufacturing industry with a more reliable automated quality control method, solving the problems of low detection efficiency and insufficient accuracy in existing technologies, and promoting the advancement of metal surface defect detection technology.
[0052] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description
[0053] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.
[0054] Figure 1 This is a schematic flowchart of a target detection method for inspecting defects on a metal surface according to an embodiment of the present invention;
[0055] Figure 2 This is a schematic diagram of the StarNet module structure provided in an embodiment of the present invention;
[0056] Figure 3 This is a schematic diagram of a bidirectional feature pyramid network structure provided in an embodiment of the present invention;
[0057] Figure 4 This is a schematic diagram of a multi-scale channel attention module structure provided in an embodiment of the present invention. Detailed Implementation
[0058] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.
[0059] A specific embodiment of the present invention discloses a target detection method for inspecting defects on metal surfaces, such as... Figure 1 As shown, it includes the following steps:
[0060] S1. Acquire image data of the metal surface to be inspected. Metal surface image data refers to digital images reflecting the surface condition of metal products (such as steel plates, aluminum, and copper pipes) acquired by an image acquisition system on an industrial production line. Metal surface image data includes various textures, colors, and potential defect information of the metal surface. In practice, industrial area scan cameras such as the Basler ace series or Hikvision MV-CH series can be used, along with bar light sources or dome light sources, to eliminate the influence of strong reflections from the metal surface. The acquired raw images are typically in RGB three-channel format with a resolution of 2448×2048 pixels.
[0061] To adapt to the model input, preprocessing is required. The image is uniformly scaled to 640 pixels × 640 pixels using bilinear interpolation. The R, G, and B values of each pixel are divided by 255 and the mean values [0.485, 0.456, 0.406] obtained from large datasets such as ImageNet are subtracted. This converts the image data into a tensor format that the model can process and reduces interference caused by lighting differences.
[0062] S2. Input the metal surface image data into the metal surface defect detection model, which is an improvement based on the YOLOv8 model. The improvement includes replacing the backbone network of the YOLOv8 model with the StarNet network and replacing the feature pyramid network of the YOLOv8 model with a bidirectional feature pyramid network.
[0063] The YOLOv8 model is a target detection model with fast detection speed. In order to meet the needs of metal surface detection, the backbone network of the YOLOv8 model built using the C2f module is replaced with a lightweight StarNet network, and the feature pyramid network (FPN / PANet) in the original model is replaced with an enhanced bidirectional feature pyramid network. The aim is to reduce the computational cost of the model and improve the detection capability of small defects on complex metal surfaces.
[0064] Specifically, the metal surface defect detection model includes a StarNet network, a bidirectional feature pyramid network, and a detection head connected in sequence;
[0065] The StarNet network receives the metal surface image data and performs multi-scale feature extraction to obtain a multi-scale feature map, which is then sent to the bidirectional feature pyramid network.
[0066] The StarNet network comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, and a fourth feature extraction unit connected in series. These four feature extraction units are connected in series to progressively extract and refine image features.
[0067] The first feature extraction unit receives the metal surface image data and outputs a first-scale feature map to the second feature extraction unit. The first feature extraction unit receives the original metal surface image data and outputs a first-scale feature map through its internal StarNet module. This scale feature map contains preliminary feature information of the image and is subsequently passed to the second feature extraction unit.
[0068] The second feature extraction unit receives the first-scale feature map and outputs a second-scale feature map to the third feature extraction unit and the bidirectional feature pyramid network. The second feature extraction unit further processes these feature maps, reduces the spatial dimension of the feature maps through a downsampling layer, and extracts deeper features through the StarNet module, outputting a second-scale feature map.
[0069] The third feature extraction unit receives the second-scale feature map and outputs a third-scale feature map to the fourth feature extraction unit and the bidirectional feature pyramid network; the fourth feature extraction unit receives the third-scale feature map and outputs a fourth-scale feature map to the bidirectional feature pyramid network. Similarly, the third and fourth feature extraction units continue to downsample and extract features from the feature maps, outputting third-scale and fourth-scale feature maps respectively, which can capture features at different levels of detail on the metal surface.
[0070] Furthermore, the first feature extraction unit includes a StarNet module. The StarNet module of the first feature extraction unit receives the metal surface image data, processes the metal surface image data, and outputs a first-scale feature map. The StarNet module of the first feature extraction unit performs preliminary feature extraction on the input metal surface image data. The first-scale feature map refers to the feature map obtained after preliminary processing, and its resolution and number of channels are preset according to the model design requirements. For example, if the resolution of the input image is 1024×1024 pixels, after processing by the StarNet module, the resolution of the output first-scale feature map is reduced to 512×512 pixels, and the number of channels increases to 64.
[0071] The second feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer receives the first-scale feature map and downsamples it. The downsampled feature map is then input to the StarNet module of the unit. The StarNet module processes the downsampled feature map and outputs the second-scale feature map. The downsampling layer in the second feature extraction unit further reduces the resolution of the feature map, for example, through max pooling or convolutional layers. The stride is typically set to 2 to halve the resolution of the feature map. Subsequently, the downsampled feature map is input to the StarNet module of the unit for further feature extraction, outputting the second-scale feature map. In practice, the downsampling layer can be implemented using a standard convolutional layer with a stride of 2 and a 3×3 kernel.
[0072] The third feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the third feature extraction unit receives the second-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the third feature extraction unit processes the downsampled feature map and outputs the third-scale feature map.
[0073] The fourth feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the fourth feature extraction unit receives the third-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the fourth feature extraction unit processes the downsampled feature map and outputs the fourth-scale feature map.
[0074] Similarly, the third and fourth feature extraction units continue to downsample and extract features from the feature map, gradually reducing the resolution and increasing the number of channels to extract deeper feature information.
[0075] Furthermore, such as Figure 2 As shown, the StarNet modules of the first feature extraction unit to the fourth feature extraction unit all include a first depthwise separable convolutional layer, a first fully connected layer, a ReLU6 activation function layer, a second fully connected layer, a batch normalization layer, and a second depthwise separable convolutional layer connected in sequence.
[0076] The first depthwise separable convolutional layer receives the input feature map and performs feature extraction, outputting a first feature map to the first fully connected layer. The function of this first depthwise separable convolutional layer is to perform preliminary feature extraction on the input feature map by separating the standard convolution operation into channel-wise convolution and pointwise convolution, reducing computational cost while maintaining the effectiveness of feature extraction. In implementation, the resolution and number of channels of the input feature map are first determined; for example, if the resolution of the input feature map is 512×512 pixels and the number of channels is 64, the first depthwise separable convolutional layer can be set to a 3×3 convolutional kernel with a stride of 1, and an output channel count of 64.
[0077] The first fully connected layer receives the first feature map and performs a non-linear combination of the channels of the input feature map using a polynomial kernel function to achieve channel dimension transformation, outputting the second feature map to the ReLU6 activation function layer. The first fully connected layer also receives the output of the first depthwise separable convolutional layer and performs a non-linear combination of the channels of the input feature map using a polynomial kernel function to achieve channel dimension transformation and enhance feature expressiveness. The order of the polynomial kernel function can be adjusted according to the feature complexity, for example, set to 2 or 3. The calculation expression of the polynomial kernel function is:
[0078]
[0079] Where, x j For the j-th channel of the feature map input to the first fully connected layer, w ij b is a learnable weight parameter. ij is a learnable bias parameter used to adjust the contribution of each channel; p is the polynomial order, which determines the complexity of the nonlinear combination; C in y is the number of channels in the input feature map. i This refers to the i-th channel of the feature map output by the first fully connected layer.
[0080] During implementation, the number of channels C of the input feature map is first determined. in For example, for a 64-channel input feature map, C in =64. The polynomial order p can be chosen based on experimental results, usually set to 2 or 3 to balance nonlinear expressiveness and computational complexity. During model training, the weight parameters w ij and bias parameter b ij Updates are performed via backpropagation to minimize the model's loss function. Each input channel x... j With the corresponding weight w ij and bias b ij After multiplication, the expression is transformed using a polynomial to ultimately generate the output channel y. i .
[0081] The ReLU6 activation function layer performs non-linear activation on the second feature map and outputs the third feature map to the second fully connected layer. The ReLU6 activation function layer introduces non-linear activation into the feature map and uses ReLU6(min(max(0,x),6)) to limit the maximum value, which enhances the robustness of the model when deployed with low precision and further improves the model's expressive power and adaptability.
[0082] The second fully connected layer receives the third feature map and restores its channel dimensions, then outputs the fourth feature map to the batch normalization layer. The second fully connected layer is responsible for restoring the channel dimensions of the ReLU6-activated feature map. The second fully connected layer restores the channel dimensions of the ReLU6-activated feature map, ensuring that the number of output channels is the same as the number of input channels, thus maintaining the dimensionality consistency of the feature map.
[0083] The batch normalization layer normalizes the fourth feature map and outputs the fifth feature map to the second depthwise separable convolutional layer; the batch normalization layer normalizes the feature map, stabilizes the training process, and accelerates the model convergence speed.
[0084] The second depthwise separable convolutional layer receives the fifth feature map and performs depth feature extraction, outputting the final output feature map of the StarNet module. The second depthwise separable convolutional layer extracts depth features from the normalized feature map and outputs the final feature map. The second depthwise separable convolutional layer can be set to a 3×3 kernel with a stride of 1 and 64 output channels to further extract depth features.
[0085] The bidirectional feature pyramid network performs feature fusion and enhancement on the multi-scale feature map to obtain an enhanced multi-scale feature map, and then performs feature weighting processing on the enhanced multi-scale feature map to obtain weighted multi-scale features, which are then sent to the detection head. The bidirectional feature pyramid network fuses and enhances the multi-scale feature map, and through upsampling and downsampling operations, effectively combines features at different levels to generate a more accurate feature representation.
[0086] Furthermore, such as Figure 3 As shown, the bidirectional feature pyramid network includes an upsampling fusion module, a feature refinement module, a downsampling fusion module, a first multi-scale channel attention module, a second multi-scale channel attention module, and a third multi-scale channel attention module;
[0087] The upsampling fusion module receives the multi-scale feature map output by the StarNet network, and fuses the high-level features with the low-level features through upsampling to generate the first set of multi-scale feature maps, which are then output to the feature refinement module. Upsampling refers to increasing the spatial resolution of the feature map through interpolation or other methods in order to better preserve detailed information.
[0088] The feature refinement module performs convolutional enhancement on the first set of multi-scale feature maps and outputs the second set of multi-scale feature maps to the downsampling fusion module; the feature refinement module performs convolutional enhancement on the first set of multi-scale feature maps to further improve the quality of the features.
[0089] The feature refinement module includes a series of depthwise separable convolutional layers, batch normalization layers, and Swish activation function layers;
[0090] The 3×3 depthwise separable convolutional layer receives the first set of multi-scale feature maps and extracts features, outputting the extracted features to the batch normalization layer; it then performs convolution operations on the input first set of multi-scale feature maps. Depthwise separable convolution decomposes standard convolution into two steps: depthwise convolution and pointwise convolution, which significantly reduces the number of parameters and computational cost. The kernel size of depthwise separable convolution is 3×3, used to extract local spatial features.
[0091] The batch normalization layer receives the extracted features from the output of the 3×3 depth separable convolutional layer and normalizes them to obtain normalized features. The batch normalization layer standardizes the feature map output from the previous layer to keep its data distribution stable with a mean of 0 and a variance of 1, which can effectively accelerate the convergence speed of the model training process and solve the problem of internal covariate shift.
[0092] The Swish activation function layer performs non-linear activation on the features normalized by the batch normalization layer, outputting the second set of multi-scale feature maps. The Swish activation function layer performs non-linear activation on the normalized features, ultimately outputting the second set of multi-scale feature maps. The Swish activation function is shown in the following equation;
[0093] f(x) = x*sigmoid(x);
[0094]
[0095] Where x is an element in the feature tensor output by the batch normalization layer, and e is the natural constant.
[0096] Compared to the traditional ReLU function, the Swish activation function is smoother near zero, which can improve the model's expressive power and training stability in deep networks.
[0097] The downsampling fusion module performs a downsampling operation on the second set of multi-scale feature maps, and then fuses the enhanced low-level features with the high-level features again to generate P3, P4, and P5 feature maps. The P3 feature map is output to the first multi-scale channel attention module, the P4 feature map is output to the second multi-scale channel attention module, and the P5 feature map is output to the third multi-scale channel attention module. The P3, P4, and P5 feature maps are respectively input to their respective multi-scale channel attention modules for feature weighting processing.
[0098] The first multi-scale channel attention module performs feature weighting on the P3 feature map, the second multi-scale channel attention module performs feature weighting on the P4 feature map, and the third multi-scale channel attention module performs feature weighting on the P5 feature map. The weighted P3, P4, and P5 feature maps are then sent to the detection head.
[0099] Furthermore, the first, second, and third multi-scale channel attention modules have the same structure, all including a dual-path feature transformation unit and an adaptive fusion unit, such as... Figure 4 As shown;
[0100] The dual-path feature transformation unit is used to receive the input feature map and generate preliminary channel attention weights, which are then output to the adaptive fusion unit.
[0101] Furthermore, the dual-path feature transformation unit receives the input feature map and generates preliminary channel attention weights, including:
[0102] Global average pooling is performed on the input feature map X to obtain the channel statistics vector. The dual-path feature transformation unit receives the input feature map from a certain level of the bidirectional feature pyramid network, such as P3, P4, or P5, denoted as X∈R. H×W×C Where H is the feature map height, W is the feature map width, C is the number of channels, R represents the real number field, each channel represents the response map of a feature filter, and the c-th element z of the channel statistics vector z is... c The calculation formula is:
[0103]
[0104] Where i and j are spatial location indices.
[0105] The channel statistical vector z is input into the first fully connected sub-path and the lightweight convolutional sub-path for feature transformation. The first fully connected sub-path includes a fully connected layer, and the lightweight convolutional sub-path includes a 1×1 convolutional layer. The outputs of the first fully connected sub-path and the lightweight convolutional sub-path are fused and then processed by the Sigmoid activation function to generate preliminary channel attention weights.
[0106] The outputs of the two paths are fused and activated by Sigmoid to form preliminary attention weights. One path is non-linearly mapped through a fully connected layer, while the other path captures local channel relationships through a lightweight 1×1 convolutional layer, as shown in the following equation.
[0107]
[0108] Where σ is the Sigmoid function, ⊙ is element-wise multiplication, * is the convolution operation, T represents the matrix transpose operation, W1 is the weight matrix of the first fully connected sub-path, W2 is the kernel weight of the lightweight convolution sub-path, z is the channel statistics vector, and b is the bias vector.
[0109] The adaptive fusion unit is used to receive preliminary channel attention weights from different feature maps and perform weighted fusion to generate the final multi-scale fused attention weights.
[0110] The adaptive fusion unit receives the initial outputs from the aforementioned three dual-path feature transformation units, namely the initial channel attention weights M1, M2, and M3 corresponding to three different scales, P3, P4, and P5. It then performs adaptive weighted fusion of the attention information at these three scales, first learning an adjustable saliency score S for each scale k. k And its fusion weight α is calculated using the Softmax function. k The saliency score is an adjustable parameter automatically learned through backpropagation during model training. It is randomly set during model initialization and uses a dataset of metal surface defect images and their annotations as input. The parameter is continuously updated by optimizing the model's loss function, and the final output is the optimized parameter value. The fusion weight α... k As shown in the following formula;
[0111]
[0112] The initial attention weights are weighted and summed based on the fused attention weights to generate a unified multi-scale fused attention weight, as shown in the following formula;
[0113]
[0114] The detection head processes the weighted multi-scale features and outputs the category, location, and confidence level of the metal surface defect. The detection head can use coupling heads such as those employed in YOLOv3 / v4 / v5, or decoupling heads such as those used in YOLOX, YOLOv6 / v7 / v8. The detection head receives the feature map from the adaptive fusion unit, which has undergone multi-scale attention weighting, and outputs the category, location, and confidence level of the metal surface defect.
[0115] S3. Output the category, location, and confidence level of the metal surface defect through the metal surface defect detection model.
[0116] Category refers to the specific type of defect, such as cracks, indentations, roller marks, etc., corresponding to the defect classification in production quality records. Location is represented by a bounding box, including the center coordinates, width, and height of the box, used to locate the defect area in the image. Confidence represents how confident the model is about the detection result, ranging from 0 to 1. A threshold such as 0.5 is typically set to filter out low-confidence detection results.
[0117] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.
[0118] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A target detection method for inspecting defects on metal surfaces, characterized in that, Includes the following steps: Acquire image data of the metal surface to be inspected; The metal surface image data is input into the metal surface defect detection model, which is an improvement based on the YOLOv8 model. The improvement includes replacing the backbone network of the YOLOv8 model with the StarNet network and replacing the feature pyramid network of the YOLOv8 model with a bidirectional feature pyramid network. The metal surface defect detection model outputs the category, location, and confidence level of the metal surface defects.
2. The method according to claim 1, characterized in that, The metal surface defect detection model includes a StarNet network, a bidirectional feature pyramid network, and a detection head connected in sequence. The StarNet network receives the metal surface image data and performs multi-scale feature extraction to obtain a multi-scale feature map, which is then sent to the bidirectional feature pyramid network. The bidirectional feature pyramid network performs feature fusion and enhancement on the multi-scale feature map to obtain an enhanced multi-scale feature map, and then performs feature weighting processing on the enhanced multi-scale feature map to obtain weighted multi-scale features, which are then sent to the detection head. The detection head identifies and processes the weighted multi-scale features, and outputs the type, location, and confidence level of the metal surface defects.
3. The method according to claim 2, characterized in that, The StarNet network includes a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, and a fourth feature extraction unit connected in series. The first feature extraction unit receives the metal surface image data and outputs a first-scale feature map to the second feature extraction unit; The second feature extraction unit receives the first-scale feature map and outputs the second-scale feature map to the third feature extraction unit and the bidirectional feature pyramid network; The third feature extraction unit receives the second-scale feature map and outputs the third-scale feature map to the fourth feature extraction unit and the bidirectional feature pyramid network. The fourth feature extraction unit receives the third-scale feature map and outputs the fourth-scale feature map to the bidirectional feature pyramid network.
4. The method according to claim 3, characterized in that, The first feature extraction unit includes a StarNet module. The StarNet module of the first feature extraction unit receives the metal surface image data, processes the metal surface image data through the StarNet module, and outputs the first scale feature map. The second feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the second feature extraction unit receives the first-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the second feature extraction unit processes the downsampled feature map and outputs the second-scale feature map. The third feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the third feature extraction unit receives the second-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the third feature extraction unit processes the downsampled feature map and outputs the third-scale feature map. The fourth feature extraction unit includes a downsampling layer and a StarNet module. The downsampling layer of the fourth feature extraction unit receives the third-scale feature map and downsamples it. The downsampled feature map is then input into the StarNet module of the unit. The StarNet module of the fourth feature extraction unit processes the downsampled feature map and outputs the fourth-scale feature map.
5. The method according to claim 4, characterized in that, Each StarNet module includes a first depthwise separable convolutional layer, a first fully connected layer, a ReLU6 activation function layer, a second fully connected layer, a batch normalization layer, and a second depthwise separable convolutional layer connected in sequence. The first depthwise separable convolutional layer receives the input feature map and performs feature extraction, and outputs the first feature map to the first fully connected layer; The first fully connected layer receives the first feature map and performs nonlinear combination of the channels of the input feature map through a multinomial kernel function to realize channel dimension transformation, and outputs the second feature map to the ReLU6 activation function layer; The ReLU6 activation function layer performs non-linear activation on the second feature map and outputs the third feature map to the second fully connected layer; The second fully connected layer receives the third feature map and restores the channel dimension, then outputs the fourth feature map to the batch normalization layer; The batch normalization layer normalizes the fourth feature map and outputs the fifth feature map to the second depth separable convolutional layer. The second depthwise separable convolutional layer receives the fifth feature map and performs depth feature extraction, outputting the final output feature map of the StarNet module.
6. The method according to claim 5, characterized in that, The expression for calculating the polynomial kernel function is: Where, x j For the j-th channel of the feature map input to the first fully connected layer, w ij b is a learnable weight parameter. ij Here, p is the learnable bias parameter, p is the polynomial order, and C is the polynomial order. in y is the number of channels in the input feature map. i This refers to the i-th channel of the feature map output by the first fully connected layer.
7. The method according to claim 2, characterized in that, The bidirectional feature pyramid network includes an upsampling fusion module, a feature refinement module, a downsampling fusion module, a first multi-scale channel attention module, a second multi-scale channel attention module, and a third multi-scale channel attention module; The upsampling fusion module receives the multi-scale feature map output by the StarNet network, fuses the high-level features with the low-level features through an upsampling operation, generates the first set of multi-scale feature maps, and outputs them to the feature refinement module. The feature refinement module performs convolution enhancement on the first set of multi-scale feature maps and outputs the second set of multi-scale feature maps to the downsampling fusion module. The downsampling fusion module performs a downsampling operation on the second group of multi-scale feature maps, and then fuses the enhanced low-level features with the high-level features again to generate P3 feature map, P4 feature map and P5 feature map. The P3 feature map is output to the first multi-scale channel attention module, the P4 feature map is output to the second multi-scale channel attention module, and the P5 feature map is output to the third multi-scale channel attention module. The first multi-scale channel attention module performs feature weighting on the P3 feature map, the second multi-scale channel attention module performs feature weighting on the P4 feature map, and the third multi-scale channel attention module performs feature weighting on the P5 feature map. The weighted P3, P4, and P5 feature maps are then sent to the detection head.
8. The method according to claim 7, characterized in that, The feature refinement module includes a series of depthwise separable convolutional layers, a batch normalization layer, and a Swish activation function layer. The depthwise separable convolutional layer receives the first set of multi-scale feature maps and performs feature extraction, then outputs the extracted features to the batch normalization layer. The batch normalization layer receives the extracted features from the depth-separable convolutional layer and performs normalization processing to obtain the normalized features. The Swish activation function layer performs nonlinear activation on the features normalized by the batch normalization layer, and outputs the second set of multi-scale feature maps.
9. The method according to claim 7, characterized in that, The first multi-scale channel attention module, the second multi-scale channel attention module, and the third multi-scale channel attention module all include a dual-path feature transformation unit and an adaptive fusion unit; The dual-path feature transformation unit is used to receive the input feature map and generate preliminary channel attention weights, which are then output to the adaptive fusion unit. The adaptive fusion unit is used to receive preliminary channel attention weights from different feature maps and perform weighted fusion to generate the final multi-scale fused attention weights.
10. The method according to claim 9, characterized in that, The dual-path feature transformation unit receives the input feature map and generates preliminary channel attention weights, including: Global average pooling is performed on the input feature map X to obtain the channel statistics vector; The channel statistical vectors are respectively input into the first fully connected sub-path and the lightweight convolutional sub-path for feature transformation. The first fully connected sub-path includes a fully connected layer, and the lightweight convolutional sub-path includes a 1×1 convolutional layer. The outputs of the first fully connected sub-path and the lightweight convolutional sub-path are fused and then processed by the Sigmoid activation function to generate preliminary channel attention weights.