Aluminum surface defect detection method with enhanced multi-scale feature extraction and target detection

Abstract

The present invention discloses an aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection, including: constructing a data set annotated with aluminum surface defects; constructing an improved aluminum surface defect detection model; placing the data set annotated with aluminum surface defects into the aluminum surface defect detection model for iterative training, and obtaining an optimized model after convergence; obtaining an aluminum surface image, and inputting it into the optimized model, detecting the category and position of the aluminum surface defects, and obtaining a recognition result. Further extracting more abundant feature information in the feature map, thereby enhancing the ability of the model to extract features, accurately identifying defect features and categories, preventing the loss of tiny defect features in the feature map caused by the deepening of the backbone network hierarchical operation, and reducing the interference of background information, solving the problem that the IoU metric is sensitive to the position offset of small objects and the CIoU is inactivated in some cases, and realizing high-speed, accurate and robust aluminum surface defect detection.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and computer vision technology, and in particular to an aluminum surface defect detection method that enhances multi-scale feature extraction and target detection. Background Art

[0002] Aluminum profiles are regarded as a vital industrial raw material due to their high strength, light weight, good plasticity, good corrosion resistance and recyclability. They are increasingly widely used in many fields such as manufacturing machinery, automobiles, light industry, ships, aerospace, etc. In addition, the low cost and machinability of aluminum profiles make them the preferred substitute for expensive industrial raw materials such as copper, which has kept the consumption of industrial aluminum growing rapidly. In recent years, with the continuous progress and development of the industrial manufacturing industry, higher requirements have been put forward for the overall quality of aluminum. However. In the actual industrial production process, factors such as poor quality of raw materials, poor production environment, uneven production equipment, and human errors often lead to the appearance of aluminum surface defects. These defects include orange peel, bottom leakage, bubbles, etc. In order to prevent such defective products from being supplied to other industries and causing unnecessary economic losses and casualties, the detection of aluminum surface defects has become a top priority.

[0003] The development of surface defect detection technology is inseparable from the research of domestic and foreign scholars in recent decades. The current defect detection methods can be roughly divided into three categories: manual visual detection methods, mechanical equipment detection methods, and machine vision detection methods. Manual visual detection requires workers to work for a long time and with high intensity. The long-term high-load state of workers will greatly increase the missed detection rate and false detection rate. In addition, this method also requires a certain degree of training for workers, and the cost of manual learning and training is also huge. Mechanical equipment detection methods mainly use various signal acquisition devices to collect specific light, electricity, and magnetic signals to detect surface defects. Although these methods have solved the problem of surface defect detection to a certain extent. However, the detection methods based on mechanical equipment have problems such as high detection cost, high learning cost, and limited detection accuracy, which are far from meeting the requirements for large-scale use in industrial production. For the first two methods, machine vision, as an emerging non-contact, non-destructive detection technology with relatively high detection efficiency and detection accuracy, has been gradually applied to diversified product quality detection. Generally speaking, traditional defect detection methods based on machine vision are mainly composed of specially designed manual features and classifiers. Although the traditional machine vision defect detection method has basically realized the automatic detection of surface defects, its carefully designed features are extremely sensitive to environmental factors such as light source intensity and different backgrounds, which makes it difficult to find the best artificial features to describe different defects, resulting in poor robustness and generalization of this type of defect detection method, and even requires "one scene, one method". Therefore, the possibility of its popularization in industrial production is very small. Deep learning methods make up for this shortcoming. The emergence of convolutional neural networks (CNNs) solves the problem of artificial feature extraction. CNNs can automatically capture deep semantic features. CNN-based methods have stronger robustness and generalization capabilities than traditional methods. Convolutional neural networks have become a very important method in the industry. There are two types of target detection algorithms based on deep learning: two-stage and single-stage. The two-stage network is divided into two steps: generating proposed regions and image classification, and has high detection accuracy. Commonly used two-stage target detection algorithms include R-CNN, Fast R-CNN, Faster R-CNN, etc. The single-stage model directly performs classification and regression. Therefore, the detection speed of the single-stage algorithm is fast, but the accuracy is low, especially for overlapping targets and small targets. Commonly used single-stage target detection algorithms include SSD, YOLO, etc.

[0004] Defect detection has a very wide range of applications. However, the metal aluminum defect detection task still has certain limitations. Directly using the existing target detection model cannot well complete the task of detecting certain types of defects. In fact, metal aluminum surface defect detection will face three major challenges: 1) There are many types of defects and some defects vary greatly. Due to the influence of the production process and the surrounding environment, the length, size, shape, direction and depth of various aluminum surface defects are different, especially there are relatively concentrated small targets, and the intra-class differences of some defects are also very large. 2) Some defect features are not obvious. Due to factors such as light and background color, the foreground where the defect is located is difficult to distinguish from the background, and there are problems such as high similarity between some defect classes. 3) The trade-off between detection accuracy and detection speed. Both detection speed and detection accuracy are crucial parts in industrial applications. It is necessary to improve the detection accuracy of the improved model as much as possible while meeting the premise of real-time detection. Therefore, there is an urgent need for an aluminum surface defect detection method with enhanced multi-scale feature extraction and small target detection capabilities to quickly enhance multi-scale feature extraction and target detection to solve the above problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent. To this end, the purpose of the present invention is to propose an aluminum surface defect detection method that enhances multi-scale feature extraction and target detection, further extracts richer feature information (especially local spatial information of neighboring pixels) in the feature map, thereby enhancing the model's ability to extract features, accurately identifying defect features and categories, preventing the loss of tiny defect features in the feature map caused by the deepening of the backbone network hierarchical operation, and reducing the interference of background information, solving the problem that the IoU metric is sensitive to the position offset of small objects and the CIoU is inactivated in some cases, and realizing high-speed, accurate and robust aluminum surface defect detection.

[0006] To achieve the above object, an embodiment of the present invention proposes an aluminum surface defect detection method that enhances multi-scale feature extraction and target detection, comprising:

[0007] Construct a dataset with annotated aluminum surface defects;

[0008] Build an improved aluminum surface defect detection model;

[0009] The data set marked with aluminum surface defects is put into the aluminum surface defect detection model for iterative training, and the optimized model is obtained after convergence;

[0010] The aluminum surface image is obtained and input into the optimization model to detect the type and location of aluminum surface defects and obtain the recognition result.

[0011] According to some embodiments of the present invention, constructing a data set annotated with aluminum surface defects includes:

[0012] Get an initial dataset of aluminum materials;

[0013] The aluminum image size in the initial data set is adjusted to 640*640, and the initial data set is expanded based on the data enhancement method to obtain an enhanced data set;

[0014] The Labelimg tool is used to annotate defects in the enhanced data set, generate corresponding data labels for the data obtained based on different data enhancement methods, and divide all the obtained data sets into training sets and validation sets according to an 8:2 ratio; the data enhancement methods include at least one of horizontal flipping, vertical flipping and random translation, contrast adjustment, cropping, and scale transformation.

[0015] According to some embodiments of the present invention, the initial data set has a total of ten types of defects: bruises, scratches, non-conductivity, coating cracks, bottom leakage, depressions, convex powder, pits, orange peel, and dirty spots.

[0016] According to some embodiments of the present invention, an improved aluminum surface defect detection model is constructed, including:

[0017] Build the YOLOv8n detection model;

[0018] Replace all C2f modules in the YOLOv8n detection model with mAtt-C2f modules;

[0019] Add the SPPF-LSAtt module with adaptive enhanced feature fusion capability to the backbone part of the network;

[0020] The bounding box regression CIoU loss function of the YOLOv8n detection model is replaced with the NWD-MPDIoU loss function to obtain an improved aluminum surface defect detection model.

[0021] According to some embodiments of the present invention, the mAtt-C2f module replaces all Bottleneck modules in the original C2f module with a Bottleneck_MLCA ​​module; based on the original Bottleneck module, the Bottleneck_MLCA ​​module embeds the local mixed attention mechanism MLCA into two convolutional layers, so that it has stronger feature extraction and feature fusion capabilities.

[0022] According to some embodiments of the present invention, the SPPF-LSAtt module decomposes the large-core convolution operation, and decomposes the decomposed 2D convolution into cascaded 1D kernels again; the SPPF-LSAtt module is added to the last layer of the backbone part of the network.

[0023] According to some embodiments of the present invention, the further step further includes: adjusting the relative contribution of NWD and MPDIoU in the total loss in the NWD-MPDIoU loss function, and setting the loss function expression to:

[0024] Loss total_box =(1-λ)L NWD +λL MPDIoU

[0025] Among them, L NWD is the NWD loss function; L MPDIoU is the MPDIoU loss function; λ is the adjustment factor, λ∈[0,1]; when λ is 1, it means that only the MPDIoU loss function is used as the bounding box loss function; when λ is 0, it means that only the NWD loss function is used as the bounding box loss function; λ is 0.5, indicating that the contribution of the NWD loss function and the MPDIoU loss function to the total loss is equal.

[0026] According to some embodiments of the present invention, a data set annotated with aluminum surface defects is put into an aluminum surface defect detection model for iterative training, and an optimized model is obtained after convergence, including:

[0027] The data set labeled with aluminum surface defects is put into the aluminum surface defect detection model for iterative training. After multiple trainings, the best model weight of this training is retained after the loss value stops decreasing. Through transfer learning, the best weight of each training is used for several more iterations, and finally the detection model with the highest accuracy is selected as the optimization model.

[0028] According to some embodiments of the present invention, before using the Labelimg tool to annotate defects on the enhanced dataset, the method further includes:

[0029] Get the RGB color values ​​of all pixels in each image in the enhanced dataset. The RGB color values ​​include R channel values, G channel values, and B channel values.

[0030] According to the RGB color values ​​of all pixels, the average value of the R channel, the average value of the G channel, and the average value of the B channel of all pixels in the image are calculated, and the color feature coefficient of each image is calculated:

[0031]

[0032] Wherein, T is the color characteristic coefficient of the image; N is the number of pixels included in the image; It is the average value of all pixels in the image in the R channel; is the average value of the pixels in the G color channel included in the image; is the average value of the pixels in the image in the B color channel; Ri Take the value of the R channel of the image including the i-th pixel; G i Take the value of the G channel of the image including the i-th pixel; B i is the value of the B channel of the image including the i-th pixel;

[0033] Eliminate images whose color feature coefficients are greater than a preset color feature threshold in the enhanced data set to obtain a first corrected data set;

[0034] Get the pixel gradients in the vertical direction and the pixel gradients in the horizontal direction of the image in the first corrected data set, and calculate the pixel feature coefficients of each image:

[0035]

[0036] Among them, S is the pixel characteristic coefficient of the image; D1 is the pixel gradient of the image in the vertical direction; D2 is the pixel gradient of the image in the horizontal direction; M θ is the number of pixels with an angle of θ in the image; T 1(x) is the pixel gradient of the image at x in the vertical direction; T 2(x) is the pixel gradient of the image at x in the horizontal direction; || ||0 is the L0 norm, which is used to count the number of non-zero vectors;

[0037] Eliminate images with pixel feature coefficients less than or equal to a preset pixel feature threshold in the first corrected data set to obtain a second corrected data set, which is also an enhanced data set after data screening;

[0038] Defects are annotated on the enhanced dataset after data screening based on the Labelimg tool.

[0039] The present invention proposes an aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection. An improved YOLOv8n model is adopted. In order to further extract richer feature information (especially local spatial information of neighboring pixels) in the feature map, thereby enhancing the model's ability to extract features, a mAtt-C2f module is introduced into the network to replace all C2f modules; in order to prevent the loss of tiny defect features in the feature map caused by the deepening of the backbone network hierarchical operation and reduce the interference of background information, an SPPF-LSAtt module is added to the backbone part of the network; finally, in order to better solve the problem that the IoU metric is sensitive to the position offset of small objects and is inactivated in some cases of CIoU, a new loss function, NWD-MPDIoU, is used to replace the CIoU loss function of the original model, and the model with the highest accuracy is selected from the converged model as the final model, thereby realizing high-speed, accurate and robust aluminum surface defect detection.

[0040] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0041] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 is a flow chart of an aluminum surface defect detection method for enhancing multi-scale feature extraction and target detection according to an embodiment of the present invention;

[0044] Figure 2 is a block diagram of a mAtt-C2f module according to an embodiment of the present invention;

[0045] Figure 3 is a block diagram of a SPPF-LSAtt module according to an embodiment of the present invention;

[0046] Figure 4 is a structural diagram of an improved YOLOv8n model according to an embodiment of the present invention;

[0047] Figure 5 is a dataset image according to an embodiment of the present invention;

[0048] Figure 6 is a radar comparison diagram of defect categories before and after model improvement according to an embodiment of the present invention;

[0049] Figure 7 is a comparison chart of radar accuracy under COCO indicators (large, medium, and small) before and after model improvement according to an embodiment of the present invention;

[0050] Figure 8 is a bar chart comparing AP values ​​of an improved model and a mainstream model on different defect categories according to an embodiment of the present invention;

[0051] Fig. 9 It is a PR curve chart based on the test results of the existing original model;

[0052] Fig.10 is a PR curve diagram of the detection result of the improved model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0054] According to the attached Figure 1-10 A method for detecting aluminum surface defects by enhancing multi-scale feature extraction and target detection proposed in the present invention is described.

[0055] like Figure 1 As shown, the embodiment of the present invention proposes an aluminum surface defect detection method for enhancing multi-scale feature extraction and target detection, including steps S1-S4:

[0056] S1. Build a dataset with annotated aluminum surface defects;

[0057] S2. Construct an improved aluminum surface defect detection model;

[0058] S3, putting the data set marked with aluminum surface defects into the aluminum surface defect detection model for iterative training, and obtaining the optimized model after convergence;

[0059] S4. Obtain the surface image of the aluminum material and input it into the optimization model to detect the type and location of the surface defects of the aluminum material and obtain the recognition result.

[0060] The beneficial effects of the above technical solution are: further extracting richer feature information in the feature map (especially the local spatial information of neighboring pixels), thereby enhancing the model's ability to extract features, accurately identifying defect features and categories, preventing the loss of tiny defect features in the feature map caused by the deepening of the backbone network hierarchical operation, and reducing the interference of background information, solving the problem that the IoU metric is sensitive to the position offset of small objects and the CIoU is inactivated in some cases, and realizing high-speed, accurate and robust aluminum surface defect detection. Based on the optimized model, the aluminum surface image is recognized, the category and position of the aluminum surface defects are detected, and the recognition result is obtained.

[0061] According to some embodiments of the present invention, constructing a data set annotated with aluminum surface defects includes:

[0062] Get an initial dataset of aluminum materials;

[0063] The aluminum image size in the initial data set is adjusted to 640*640, and the initial data set is expanded based on the data enhancement method to obtain the enhanced data set;

[0064] The Labelimg tool is used to annotate defects in the enhanced data set, generate corresponding data labels for the data obtained based on different data enhancement methods, and divide all the obtained data sets into training sets and validation sets according to an 8:2 ratio; the data enhancement methods include at least one of horizontal flipping, vertical flipping and random translation, contrast adjustment, cropping, and scale transformation.

[0065] The working principle of the above technical solution is: obtain the initial data set of aluminum materials. For example, the mainstream public data set Tianchi aluminum profile surface defect data set (APDDD) is used. The APDDD data set is an open source data set for the target detection competition in the Guangdong Industrial Big Data Innovation Competition. It includes bruises (Bp), scratches (Sc), non-conductivity (Nc), coating cracks (Cc), bottom leakage (Bl), depressions (De), convex powder (Cp), pits (Pi), orange peel (Op) and dirty spots (Ds). The 10 defect categories are as follows: Figure 5 As shown, they correspond to (a), (b), (c), (d), (e), (f), (g), (h), and (i) respectively. In image (i), there are two types of defects: orange peel (Op) and dirty spots (Ds); there are 2250 images in total, and the image resolution is 2560*1960.

[0066] In this embodiment, the initial data set is expanded based on the data enhancement method to achieve the expansion of the data set size. Different data enhancement methods are applied to increase the diversity and quantity of the data set. These enhancement methods are designed to simulate different conditions and changes in the real world to improve the generalization ability of the model. Select a suitable annotation tool: In this example, LabelImg is selected as the annotation tool. LabelImg is an open source image annotation tool that supports rectangular box annotation and classification label addition. Prepare the annotation environment: Make sure that the computer has installed the LabelImg tool and related dependent libraries (such as Python and OpenCV). Create a folder structure for storing annotation data, including original image folders, enhanced image folders, and corresponding label folders. Annotate the enhanced data set: Open the LabelImg tool and load the enhanced image folder through the "Open Dir" option. For each enhanced image, use the "Create RectBox" tool to draw a rectangular box around the target object. Make sure that the size and position of the box can accurately represent the position and size of the target object. In the pop-up dialog box, add the corresponding category label for the rectangular box. These labels should be consistent with the labels in the original data set to ensure that the model can correctly identify different defect types during training. After completing the annotation, click the "Save" button to save the annotation results. LabelImg will automatically save the annotation information as a label file in XML format and place it in the specified label folder. Ensure the quality and accuracy of the annotation data: During the annotation process, you need to regularly check and correct the annotation results that are incorrect, inaccurate, or inconsistent. This can be achieved by adding a quality inspection process or using other annotation tools for cross-validation. Export and organize the annotation data: If needed, the label file in XML format can be converted to other formats (such as the format required by YOLOv8) for use in model training. Organize the annotation data to ensure that the association between the image file and the corresponding label file is correct. When annotating, you need to pay attention to the correspondence between the original image and the enhanced image. Make sure that a corresponding label file is created for each enhanced image, and the file name and path are consistent with the original image. For different data enhancement methods, you may need to adjust the annotation strategy to adapt to different image transformations. For example, when rotating an image, you need to ensure that the direction and position of the rectangular box are consistent with the target object after rotation. Use the LabelImg tool to annotate defects on the enhanced dataset and generate corresponding data labels for the data obtained based on different data enhancement methods. This helps improve the performance and accuracy of the model in object detection tasks.

[0067] The improved aluminum surface defect detection model is trained based on the training set, and the trained model is verified based on the verification set. If the verification fails, training needs to be continued.

[0068] The beneficial effects of the above technical solution are: obtaining an accurate data set with labeled aluminum surface defects, and making good data preparation before model training.

[0069] According to some embodiments of the present invention, the initial data set has a total of ten types of defects: bruises, scratches, non-conductivity, coating cracks, bottom leakage, depressions, convex powder, pits, orange peel, and dirty spots.

[0070] According to some embodiments of the present invention, an improved aluminum surface defect detection model is constructed, including:

[0071] Build the YOLOv8n detection model;

[0072] Replace all C2f modules in the YOLOv8n detection model with mAtt-C2f modules;

[0073] Add the SPPF-LSAtt module with adaptive enhanced feature fusion capability to the backbone part of the network;

[0074] The bounding box regression CIoU loss function of the YOLOv8n detection model is replaced with the NWD-MPDIoU loss function to obtain an improved aluminum surface defect detection model.

[0075] The working principle of the above technical solution: The improved aluminum surface defect detection model is obtained by improving the existing YOLOv8n detection model. Therefore, we first need to build a YOLOv8n detection model. The feature extraction ability of the original C2f module in the YOLOv8n detection model is poor. All C2f modules in the entire model structure are replaced with mAtt-C2f modules. Under the premise of ensuring a certain inference speed, the defect feature extraction and fusion capabilities in the network are further enhanced. Figure 2As shown in Figure 1, the mAtt-C2f module replaces all the Bottleneck modules in the original C2f module with the Bottleneck_MLCA ​​module; the Bottleneck_MLCA ​​module in box A embeds the local mixed attention mechanism MLCA into two convolutional layers based on the original Bottleneck module, making it have stronger feature extraction and feature fusion capabilities. Box B further demonstrates the implementation principle of the MLCA attention mechanism. The input feature vector of MLCA is subjected to two pooling steps, and the input is converted into a vector of 1*C*ks*ks so that local spatial information can be extracted through the first local average pooling (LAP). Based on the initial stage, two branches are used to convert the input into a one-dimensional vector. The first branch contains global information, and the second branch contains local spatial information. After the one-dimensional convolution, the original resolution of the two vectors is restored by unaverage pooling (UNAP), and then information fusion is performed to achieve the purpose of mixed attention. The MLCA attention mechanism is different from the previous mainstream attention mechanisms, such as CBAM, SE and CA. The previous attention mechanisms cannot meet the requirements of simultaneously including channel information, spatial information, local channel information and local spatiality. The MLCA attention mechanism can not only meet this requirement well, but also has fewer parameters and smaller computational complexity, so the computational cost of the model can be ignored. Finally, Box C describes the characteristics of LAP (local average pooling), GAP (global average pooling) and UNAP (unaverage pooling).

[0076] Figure 6 The dark polygons represent the training results before the model was improved, and the light polygons represent the training results after the model was improved. Each vertex of the polygon corresponds to a defect category. The multiple gray deformations in the figure represent contour lines, and the values ​​gradually increase from the inside to the outside. Figure 6 As can be seen from the two sub-figures, the AP50 values ​​of different categories of targets have been improved to varying degrees after the model is improved. In sub-figure (a), the increase in bruises and scratches is more significant. Figure 9-10 It can be seen that the increase is 9% and 12.8% respectively compared with the model before improvement. In sub-image (b), the growth of indentation and crease categories is also significant, increasing by 20.8% and 18% respectively compared with the model before improvement. In addition, compared with other defects, the size of dirty spot defects and foreign body defects in the image is smaller, and the detection accuracy of the two defects has increased to varying degrees after improvement (11.3% and 6.8%), which shows that the improved model improves the effectiveness of small target defect detection. Figure 7The figure shows a comparison of the detection results of large, medium and small defect scales of different data sets under the COCO evaluation index before and after the improvement. From the two sub-figures, it can be seen that the detection accuracy of the improved model on different defect scales of the two data sets has been improved to a certain extent, among which the improvement in the detection of small defect scales is particularly obvious, which better supports the above-mentioned viewpoints.

[0077] like Figure 8 As shown in the figure, according to the detection results of the unoptimized model, the detection effect of dirty points (Ds) is the least ideal among all target types. The AP value is the lowest; the relative area of ​​this type of target is small, and there are certain differences within the class; in order to improve the detection effect of the model for this type of situation, the SPFF-LSAtt module is added to the last layer of the Backbone part of the network to strengthen the processing and extraction of effective feature information from defects of many different scales; the SPPF-LSAtt module with adaptive enhanced feature fusion capability is added to the Backbone part of the network, so that the network can ignore the interference of irrelevant background information, avoid excessive redundancy of feature fusion, enhance the spatial perception ability of the network, and thus more accurately distinguish the texture feature information of large defects and small defects.

[0078] like Figure 3 As shown in the figure, it is the structure diagram of the SPPF-LSAtt module. The SPPF-LSAtt module decomposes the large-core convolution operation and decomposes the decomposed 2D convolution into cascaded 1D kernels. The SPPF-LSAtt module is added to the last layer of the backbone part of the network. It captures long-range dependencies and adaptability by decomposing the large-core convolution operation, and decomposes the decomposed 2D convolution into cascaded 1D kernels, thereby further reducing the model's computational complexity and memory requirements. In order to further improve the model's ability to extract key features of aluminum profile defects, as shown in the figure, Figure 3 As shown in the left part, we introduce the Large Separable Kernel Attention (LSKA) mechanism in the SPPF module of the YOLOv8n backbone network. LSKA is an innovative large separable kernel attention module that captures long-range dependencies and adaptability by decomposing large kernel convolution operations, and decomposes the decomposed 2D convolution into cascaded 1D kernels to further reduce the model's computational complexity and memory requirements. The implementation process is as follows: Figure 3 As shown on the right, a k×k convolution is first decomposed into a (2d-1)×(2d-1) depth convolution, a k / d×k / d depth dilated convolution, and a 1×1 convolution. Then, the 2D depth convolution kernel and the depth dilated convolution kernel are further decomposed into a 1D horizontal (transverse) convolution kernel and a vertical (longitudinal) convolution kernel. Finally, the decomposed convolution kernels are cascaded in sequence.

[0079] In order to better solve the problem of IoU metric being sensitive to small target position offset during network training, enhance the accuracy of small target detection, and alleviate the problem of CIoU inactivation in some cases to a certain extent, so that the final regression result is more accurate, the bounding box regression CIoU loss function of the YOLOv8n detection model is replaced with the NWD-MPDIoU loss function, and the relative contribution of NWD and IoU similarity metrics in the total loss is controlled by parameters.

[0080] The beneficial effect of the above technical solution is that it is easy to obtain an improved aluminum surface defect detection model.

[0081] like Figure 4 As shown in FIG. 1 , it is a structural diagram of the improved aluminum surface defect detection model. The specific improvement of the YOLOv8n algorithm structure in the present invention is Figure 4 The dark-boxed mAtt-C2f module replaces the C2f structure in the original structure of YOLOv8n, further enhancing the feature extraction capability of YOLOv8n; the dark-boxed SPPF-LSAtt module replaces the SPPF module in the original structure, avoiding excessive redundancy of feature fusion in the network; the introduced dark-boxed DyHead module is used to enrich the expression capability of the detection head; the bounding box regression loss function is replaced by the NWD-MPDIoU loss function to improve the efficiency of the model in detecting small defect targets.

[0082] According to some embodiments of the present invention, the further step further includes: adjusting the relative contribution of NWD and MPDIoU in the total loss in the NWD-MPDIoU loss function, and setting the loss function expression to:

[0083] Loss total_box =(1-λ)L NWD +λL MPDIoU

[0084] Among them, L NWD is the NWD loss function; L MPDIoU is the MPDIoU loss function; λ is the adjustment factor, λ∈[0,1]; when λ is 1, it means that only the MPDIoU loss function is used as the bounding box loss function; when λ is 0, it means that only the NWD loss function is used as the bounding box loss function; λ is 0.5, indicating that the contribution of the NWD loss function and the MPDIoU loss function to the total loss is equal.

[0085] According to some embodiments of the present invention, a data set marked with aluminum surface defects is put into an aluminum surface defect detection model for iterative training, and an optimized model is obtained after convergence, including:

[0086] The data set labeled with aluminum surface defects is put into the aluminum surface defect detection model for iterative training. After multiple trainings, the best model weight of this training is retained after the loss value stops decreasing. Through transfer learning, the best weight of each training is used for several more iterations, and finally the detection model with the highest accuracy is selected as the optimization model.

[0087] In a specific embodiment, the Pytorch deep learning framework is used to train and test the improved aluminum surface defect detection model. The experimental environment adopts the following configuration: AMD EPYC7543CPU, RTXA5000GPU, 30GB memory, SGD optimizer is used to optimize the model, and larger batch sizes are beneficial because they can improve the detection performance of the model. Therefore, this paper adopts a batch size of 16, a training round of 400, and the image size is set to 640x640. During the training process, random flipping, adjusting contrast, cropping, scale transformation and other data enhancement methods are used to improve the robustness of the model. In order to analyze whether the proposed improvements are effective, a combination experiment is conducted on each improvement strategy to control the variables.

[0088] The recall rate R (Recall), precision P (Precision), mean average precision value mAP (meanAveragePrecision) and the number of images processed per second FPS (Frames Per Second) are used as evaluation indicators; the intersection over union (IoU) threshold is selected as 0.5 during calculation, and IoU>0.5 means that the target is successfully detected; the recall rate formula is:

[0089]

[0090] Among them, when all hyperparameters are the same, the experimental results of comparing the improved network model with other existing mainstream models are shown in the following two tables:

[0091] Table 1 Comparative experiments with mainstream models on the APDDD dataset

[0092]

[0093] Table 2 Comparison of AP50 values ​​of various defect categories on the APDDD dataset with mainstream models

[0094]

[0095] Table 1 shows the experimental results of the optimization model proposed in the present invention and other eight excellent models on the APDDD validation dataset. From Table 1, we can see that for mAP50, the optimization model has the highest 88.7%, while YOLOv8s is the second highest 82.5%. Compared with YOLOv8s, the optimization model has improved by 6.2%. For mAP50:95, the optimization model shows the best detection accuracy, reaching 69.3%, followed by the original model, with a value of 60.7%. The improved model has increased by 8.6% over the previous model. Compared with the typical representative of the single stage (SSD model) and the typical representative of the two stages (Faster R-CNN model), the improved model of the present invention has a huge improvement in mAP50, mAP50:95, or FPS, especially compared with the Faster R-CNN model, mAP50 has increased by 18.8%, and the more stringent mAP50:95 index has increased by 30.8%, and FPS has increased by 53.1. This is enough to show that the performance improvement of the optimized model on the APDDD validation dataset has a great advantage. In terms of inference speed, the network of the optimized model maintains a relatively small difference with the original network and can still meet the standards of real-time detection in real industrial scenarios.

[0096] Table 2 shows the comparison of the AP50 experimental results of the optimized model and the other eight models for each defect category on the APDDD validation dataset. There are a total of 11 rows of data in the table, the first row represents the average AP50 value of all defect categories, and the remaining 10 rows represent the AP50 value of each defect category. Based on the data in Table 2, we created Figure 8 to visualize the comparison results between models.

[0097] according to Figure 8 From the data in each row of Table 2, we can see that the optimized model has significant improvements in all 10 defect categories, with the most significant improvements in dirty spot defects and scratch defects, with AP50 values ​​of 53.0% and 87.9% respectively. Compared with YOLOv8s, which has the second highest mAP50 score, they are 11.6% and 13.6% higher respectively. Figure 5 It can be observed that the dirty spots are all relatively small defects, and there are also a large number of small target defects in the scratch category. The defects in this category are large in length and size and exist in the same image at the same time. Combining the above analysis, it is verified again that the optimization model has advantages in detecting small object targets, and can also handle the situation where the length and shape of the same type of defects in the same image are quite different.

[0098] According to some embodiments of the present invention, before using the Labelimg tool to annotate defects on the enhanced dataset, the method further includes:

[0099] Get the RGB color values ​​of all pixels in each image in the enhanced dataset. The RGB color values ​​include R channel values, G channel values, and B channel values.

[0100] According to the RGB color values ​​of all pixels, the average value of the R channel, the average value of the G channel, and the average value of the B channel of all pixels in the image are calculated, and the color feature coefficient of each image is calculated:

[0101]

[0102] Wherein, T is the color characteristic coefficient of the image; N is the number of pixels included in the image; It is the average value of all pixels in the image in the R channel; is the average value of the pixels in the G color channel included in the image; is the average value of the pixels in the image in the B color channel; R i Take the value of the R channel of the image including the i-th pixel; G i Take the value of the G channel of the image including the i-th pixel; B i is the value of the B channel of the image including the i-th pixel;

[0103] Eliminate images whose color feature coefficients are greater than a preset color feature threshold in the enhanced data set to obtain a first corrected data set;

[0104] Get the pixel gradients in the vertical direction and the pixel gradients in the horizontal direction of the image in the first corrected data set, and calculate the pixel feature coefficients of each image:

[0105]

[0106] Among them, S is the pixel characteristic coefficient of the image; D1 is the pixel gradient of the image in the vertical direction; D2 is the pixel gradient of the image in the horizontal direction; M θ is the number of pixels with an angle of θ in the image; T 1(x) is the pixel gradient of the image at x in the vertical direction; T 2(x) is the pixel gradient of the image at x in the horizontal direction; || ||0 is the L0 norm, which is used to count the number of non-zero vectors;

[0107] Eliminate images with pixel feature coefficients less than or equal to a preset pixel feature threshold in the first corrected data set to obtain a second corrected data set, which is also an enhanced data set after data screening;

[0108] Defects are annotated on the enhanced dataset after data screening based on the Labelimg tool.

[0109] The working principle and beneficial effects of the above technical solution: Before using the Labelimg tool to annotate defects in the enhanced data set, the enhanced data set is screened to remove some abnormal data. First, the RGB color values ​​of all pixels in each image in the enhanced data set are obtained, and the RGB color values ​​include R channel values, G channel values, and B channel values; according to the RGB color values ​​of all pixels, the average values ​​of all pixels in the image in the R channel, the average values ​​of the G channel, and the average values ​​of the B channel are calculated, and the color feature coefficient of each image is calculated. The color deviation degree of the image is represented based on the color feature coefficient of each image. Images with color feature coefficients greater than the preset color threshold are eliminated in the enhanced data set to obtain the first corrected data set, and images with large color deviations are removed as the first data screening. Secondly, the pixel gradients in the vertical direction and the horizontal direction of the images in the first corrected data set are obtained, the pixel feature coefficients of each image are calculated, and a second screening is performed based on the pixel feature coefficients. Images with pixel feature coefficients less than or equal to a preset pixel feature threshold are eliminated in the first corrected data set. The pixel feature coefficient can accurately reflect the detail information of the image. The richer the pixel feature coefficient represents, the more detailed information can be obtained through two screenings. This avoids the impact of abnormal data on subsequent data use and improves the efficiency and accuracy of model training.

[0110] The present invention proposes an aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection. An improved YOLOv8n model is adopted. In order to further extract richer feature information (especially local spatial information of neighboring pixels) in the feature map, thereby enhancing the model's ability to extract features, a mAtt-C2f module is introduced into the network to replace all C2f modules; in order to prevent the loss of tiny defect features in the feature map caused by the deepening of the backbone network hierarchical operation and reduce the interference of background information, an SPPF-LSAtt module is added to the backbone part of the network; finally, in order to better solve the problem that the IoU metric is sensitive to the position offset of small objects and is inactivated in some cases of CIoU, a new loss function, NWD-MPDIoU, is used to replace the CIoU loss function of the original model, and the model with the highest accuracy is selected from the converged model as the final model, thereby realizing high-speed, accurate and robust aluminum surface defect detection.

[0111] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting aluminum surface defects by enhancing multi-scale feature extraction and target detection, characterized in that: include: Construct a dataset with annotated aluminum surface defects; Build an improved aluminum surface defect detection model; The data set marked with aluminum surface defects is put into the aluminum surface defect detection model for iterative training, and the optimized model is obtained after convergence; Obtain the aluminum surface image and input it into the optimization model to detect the type and location of aluminum surface defects and obtain the recognition result; Construct a dataset with annotated aluminum surface defects, including: Get an initial dataset of aluminum materials; The aluminum image size in the initial data set is adjusted to 640*640, and the initial data set is expanded based on the data enhancement method to obtain an enhanced data set; Use the Label img tool to annotate defects on the enhanced data set, generate corresponding data labels for the data obtained based on different data enhancement methods, and divide all the obtained data sets into training sets and validation sets according to an 8:2 ratio; the data enhancement methods include at least one of horizontal flipping, vertical flipping and random translation, contrast adjustment, cropping, and scale transformation; Build an improved aluminum surface defect detection model, including: Build the YOLOv8n detection model; Replace all C2f modules in the YOLOv8n detection model with mAtt-C2f modules; Add the SPPF-LSAtt module with adaptive enhanced feature fusion capability to the backbone part of the network; The bounding box regression CIoU loss function of the YOLOv8n detection model is replaced with the NWD-MPDIoU loss function to obtain an improved aluminum surface defect detection model; The dataset with labeled aluminum surface defects is put into the aluminum surface defect detection model for iterative training. After convergence, the optimized model is obtained, including: The data set labeled with aluminum surface defects is put into the aluminum surface defect detection model for iterative training. After multiple trainings, the best model weight of this training is retained after the loss value stops decreasing. Through transfer learning, the best weight of each training is used for several more iterations, and finally the detection model with the highest accuracy is selected as the optimization model.

2. The aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection according to claim 1, characterized in that: The initial data set contains ten types of defects: bruises, scratches, non-conductivity, coating cracks, bottom leakage, dents, convex powder, pits, orange peel and dirty spots.

3. The aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection according to claim 1, characterized in that: The mAtt-C2f module replaces all the Bottleneck modules in the original C2f module with the Bottleneck_MLCA ​​module. Based on the original Bottleneck module, the Bottleneck_MLCA ​​module embeds the local mixed attention mechanism MLCA into two convolutional layers, giving it stronger feature extraction and feature fusion capabilities.

4. The aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection according to claim 1, characterized in that: The SPPF-LSAtt module decomposes the large-core convolution operation and decomposes the decomposed 2D convolution into cascaded 1D kernels again; the SPPF-LSAtt module is added to the last layer of the backbone part of the network.

5. The aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection according to claim 1, characterized in that: Also includes: Adjust the relative contribution of NWD and MPDIoU in the total loss in the NWD-MPDIoU loss function and set the loss function expression as: Loss total_box =(1-λ)L NWD +λL MPDIoU Among them, L NWD is the NWD loss function; L MPDIoU is the MPDIoU loss function; λ is the adjustment factor, λ∈[0,1]; when λ is 1, it means that only the MPDIoU loss function is used as the bounding box loss function; when λ is 0, it means that only the NWD loss function is used as the bounding box loss function; λ is 0.5, indicating that the contribution of the NWD loss function and the MPDIoU loss function to the total loss is equal.

6. The aluminum surface defect detection method for enhanced multi-scale feature extraction and target detection according to claim 1, characterized in that: Before using the Labelimg tool to annotate defects on the enhanced dataset, the following steps are also included: Get the RGB color values ​​of all pixels in each image in the enhanced dataset. The RGB color values ​​include R channel values, G channel values, and B channel values. According to the RGB color values ​​of all pixels, the average value of the R channel, the average value of the G channel, and the average value of the B channel of all pixels in the image are calculated, and the color feature coefficient of each image is calculated: Wherein, T is the color characteristic coefficient of the image; N is the number of pixels included in the image; It is the average value of all pixels in the image in the R channel; is the average value of the pixels in the G color channel included in the image; is the average value of the pixels in the image in the B color channel; R i Take the value of the R channel of the image including the i-th pixel; G i Take the value of the G channel of the image including the i-th pixel; B i is the value of the B channel of the image including the i-th pixel; Eliminate images whose color feature coefficients are greater than a preset color feature threshold in the enhanced data set to obtain a first corrected data set; Get the pixel gradients in the vertical direction and the pixel gradients in the horizontal direction of the image in the first corrected data set, and calculate the pixel feature coefficients of each image: Among them, S is the pixel characteristic coefficient of the image; D1 is the pixel gradient of the image in the vertical direction; D2 is the pixel gradient of the image in the horizontal direction; M θ is the number of pixels with an angle of θ in the image; T 1(x) is the pixel gradient of the image at x in the vertical direction; T 2(x) is the pixel gradient of the image at x in the horizontal direction; || ||0 is the L0 norm, which is used to count the number of non-zero vectors; Eliminate images with pixel feature coefficients less than or equal to a preset pixel feature threshold in the first corrected data set to obtain a second corrected data set, which is also an enhanced data set after data screening; Based on the Label img tool, defects are annotated on the enhanced data set after data screening.

Images