Aluminum profile defect detection and identification method based on YOLOv7-ESC

Abstract

The aluminum profile defect detection and recognition method based on YOLOv7-ESC comprises the following steps: S1, obtaining a public data set, adopting an aluminum profile surface flaw recognition data set of Tianchi, and performing data preprocessing; S2, improving the YOLOv7 model, replacing the 3*3Conv convolution in the ELAN module of the backbone network with PCnov convolution; S3, constructing a residual pyramid pooling structure, inputting the features extracted by the backbone network into the residual pyramid pooling module SPPCSPC-F; S4, adding a CBAM attention mechanism between the backbone network and the feature fusion network; S5, replacing the CIoU loss function with the WIoU loss function, and inputting the features after feature fusion into the detection head part for classification prediction; S6, training the YOLO network model, and reserving the optimal weight; S7, testing by using the trained optimal weight, evaluating the detection result, and finally realizing the automatic and intelligent recognition of the aluminum profile surface defects. The method can improve the aluminum profile surface defect detection precision and reduce the small target defect omission rate.

Description

Technical Field

[0001] This invention relates to the field of aluminum profile surface defect detection, and particularly to a method for aluminum profile defect detection and identification based on YOLOv7-ESC. Background Technology

[0002] Aluminum profiles are the most widely used non-ferrous metal material in industrial applications, extensively used in aerospace, automotive, machinery manufacturing, and construction. They are favored for their lightweight, corrosion resistance, and good plasticity. The production of aluminum profiles in factories involves multiple processes, each playing a crucial role in the quality and performance of the profiles. However, surface defects are unavoidable in these processes. Surface defects affect the appearance, quality, and performance of aluminum profiles, sometimes rendering them unsellable. Therefore, rapid and accurate detection of aluminum profile defects is essential, requiring pre-shipment defect inspection to avoid adverse effects on the product's aesthetics and performance. Traditional manual inspection methods rely on operator subjective judgment, leading to problems such as misjudgment, missed detections, low accuracy, and low efficiency. Later, the emergence of machine vision-based image processing algorithms improved these problems to some extent. However, applying machine vision to the diverse surface defects of aluminum profiles presents challenges such as complex feature extraction, poor generalization, and insufficient performance in detecting small-target defects.

[0003] With the improvement of computer performance and the advent of the big data era, deep learning algorithms have become a hot topic, achieving breakthroughs in fields such as natural language processing, image classification, object detection, and data mining. They have also compensated for the shortcomings of traditional detection methods in detecting surface defects in aluminum profiles. Deep learning algorithms automatically extract high-level features from input images through neural networks and perform classification and detection operations, eliminating the tedious manual feature extraction process and improving detection efficiency.

[0004] Currently, aluminum profile defect detection methods are mainly divided into two-stage detection algorithms, represented by Faster R-CNN, and single-stage detection algorithms, represented by the YOLO series. Two-stage detection methods have a deeper network structure and higher detection accuracy, but their detection speed is slow, making them difficult to meet user needs in real-time target detection. Single-stage detection methods, due to their simpler network structure, improve detection speed while maintaining accuracy, enabling real-time detection. However, aluminum profiles contain defects such as dirt spots and scratches, which occupy a small pixel area in the image and have inconspicuous features compared to other types of defects, resulting in poor defect detection performance. The accuracy of the YOLO model in detecting these small surface defects still needs improvement. Aluminum profile defect detection algorithms also have other problems, such as large model size, need for improved detection speed, insufficient feature extraction capabilities leading to missed detection of weak, elongated defects, and small defects. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a defect detection and identification method for aluminum profiles based on YOLOv7-ESC, which can improve detection accuracy and speed, reduce missed detections of small target defects, and reduce the number of model parameters; it also possesses good robustness and generalization ability.

[0006] The technical solution adopted in this invention is as follows:

[0007] The method for detecting and identifying defects in aluminum profiles based on YOLOv7-ESC includes the following steps:

[0008] Step 1: First, obtain the publicly available aluminum profile surface defect dataset from Alibaba Cloud Tianchi. Then, perform data preprocessing on the aluminum profile surface images and labels to achieve noise reduction and image enhancement, resulting in a surface defect map of aluminum profiles with prominent defects. Finally, establish the YOLOv7-ESC network framework.

[0009] Step 2: Improve the ELAN module by replacing the Conv convolutional module in the backbone network's ELAN module with a PCnov convolutional module to extract surface defects in aluminum profiles. Compared to ordinary convolutional modules, the PCnov convolutional module uses a subset of features from the input and output channels for computation, reducing redundant computation and memory access without losing channel features, thus extracting defect features more effectively. This reduces network computation, parameters, and complexity, making the model more lightweight.

[0010] Step 3: Reconstruct the SPPCSPC module and build the residual pyramid pooling structure SPPCSPC-F. The features extracted from the backbone network are input into the SPPCSPC-F residual pyramid pooling structure.

[0011] Step 4: Add a CBAM attention mechanism module between the backbone network and the feature fusion network;

[0012] Step 5: Replace the CIoU regression loss function with the WIoU loss function, and input the fused features from the feature fusion network into the detection head for classification prediction;

[0013] Step 6: Train the YOLOv7-ESC network model. First, divide the aluminum profile surface defect dataset into training, validation, and test sets in a 4:1:1 ratio, and use Mosaic for further data augmentation during training. Then, input the dataset into the YOLOv7-ESC network model for pre-training, and select the pre-trained weights with the smallest training loss for transfer learning. Finally, combine the aluminum profile surface defect dataset and the pre-trained weights for further training to obtain the optimal weights.

[0014] Step 7: Use the trained optimal weights to perform tests and evaluate the detection results, ultimately achieving automated and intelligent identification of surface defects in aluminum profiles.

[0015] In step 1, the images of the produced aluminum profiles are first collected and organized to obtain a dataset of surface defects of the aluminum profiles. The defects include 10 categories and a total of 1885 color images, including dents, non-conductivity, scratches, orange peel, exposed bottom, dents, pits, powdering, coating cracks, and dirt spots.

[0016] In step 1, data enhancement and preprocessing are performed on the aluminum profile surface image. Defect areas are marked using Labelme software, and a JSON file is generated to save the annotation information. Then, the aluminum profile surface defect image undergoes data preprocessing to achieve noise reduction and image enhancement, resulting in a surface defect image of the aluminum profile with prominent defects. Specifically, this includes:

[0017] Traditional image processing is used to process the surface image of aluminum profiles. First, the enhanced surface defect image of the aluminum profile is converted between RGB color space and HSI color space. The components are extracted as needed and grayscale transformation is performed. Then, the noise of the image is analyzed and a local median filtering algorithm is proposed and applied to process the noise, resulting in a surface defect image of the aluminum profile with prominent defects.

[0018] In step 1, the YOLOv7-ESC network framework includes: a backbone network, a feature fusion network in the neck region, and a detection head. The structure diagram is shown below. Figure 4 As shown, the backbone network extracts surface defect features from the input image of the aluminum profile. First, the input feature image is 640×640×3, and features are refined using four layers of CBL convolutional blocks. Then, the ELAN-P module reduces the number of parameters and downsamples to reduce the number of feature channels. Next, a feature map of size 20×20×1024 is input to the neck feature fusion network. The neck feature fusion network uses top-down and bottom-up connections to perform feature fusion processing on multi-scale feature layers. Finally, the feature maps of different scales are fed into the detection head for loss calculation, defect localization, and defect prediction.

[0019] In step 2, replacing the 3*3 Conv convolutional module in the ELAN module with a PConv convolutional module reduces the number of module parameters. The ELAN-P module consists of short paths and long paths. The short path is a 1*1 CBS convolution, and the long path consists of a 1*1 CBS convolution and four consecutive PConv convolutions. The PConv convolutional module is characterized by using ordinary convolution to extract a portion of the features from the input and output channels for calculation, which can reduce redundant calculations without losing the number of channel features, thus accelerating model inference.

[0020] The specific principle is as follows:

[0021] Let the size of the input feature map X be c×h×w, where h, w, and c represent the height, width, and number of channels of the input feature map X, respectively. Let the input feature I∈R. c×h×w The output is O∈R c×h×w The standard convolution kernel size is w∈R c×k×k , where k represents the kernel size. Since the PConv convolution module only needs to extract the feature subset cp, the kernel size is reduced to w∈R. cp×k×k The formula for the computational cost ratio between PConv convolution and Conv convolution is as follows:

[0022]

[0023] In the above formula, FLOPS PConv Indicates PConv computational cost; FLOPS Conv The expression represents the computational cost of PConv; h represents the height of the input feature map; w represents the width of the input feature map; c p represents the number of channels extracted by PConv; c represents the number of channels in the input feature map.

[0024] In step 2, within the PConv convolution module, to avoid high redundancy caused by different channels of the feature map, convolution operations are performed only on a subset of the input channels, while the remaining input channels remain unchanged. The input cp channels are used as representatives of the entire feature map for computation. At this point, the computational cost of PConv is h×w×k. 2 ×c p 2 If cp is set to 1 / 4 of the input channel c, the computational cost is 1 / 16 of that of a regular convolution, and the memory access is 1 / 4 of that of a regular convolution.

[0025] In step 3, a residual pyramid pooling structure SPPCSPC-F is constructed, and the features extracted by the backbone network are input into the constructed residual pyramid pooling structure. The residual pyramid pooling structure SPPCSPC-F includes an upper-layer multi-scale feature extraction channel and a lower-layer small target feature extraction channel. Its structure diagram is shown below. Figure 2 As shown, the upper-layer multi-scale feature extraction channel captures feature information at different scales through different pooling kernels to obtain richer feature target information. A 1×1 convolutional layer is placed before and after the pooling layer to adjust the channel. The lower-layer small target feature extraction channel introduces three layers of residual convolutional blocks. The CF module of three consecutive linear residual convolutional blocks refines the extracted features and extracts more small target features.

[0026] The CF module consists of convolution and the FReLU activation function. The FReLU activation function can improve the spatial sensitivity of multi-scale defect information in aluminum profiles. Tensors are added through the add operation, increasing the amount of information without changing the number of feature channels, and 1×1 convolution feature fusion is performed.

[0027] In step 3, the features extracted by the backbone network are input into the residual pyramid pooling structure SPPC-F. First, the input features are divided into two parts. One part undergoes residual convolution processing to refine the extracted features. The other part is processed using an SPP structure, with four pooling layers of different sizes outputting four different receptive fields, facilitating the differentiation between large and small targets. Finally, these two parts are merged. This enhances the network's multi-scale feature learning ability, capturing more small target defect features and solving the problem of missed detection of small target defects in aluminum profile surface defects.

[0028] In step 4, a CBAM attention mechanism module is added after the residual pyramid pooling structure SPPCSPC-F. The introduction of channel attention and spatial attention mechanisms into the network improves the model's perception ability. Without significantly increasing the model parameters and scale, the model performance is rapidly expanded and feature refinement is adaptively improved, reducing the loss of positional information, gaining more attention to small target information, and increasing the accuracy of small target detection.

[0029] Step 4 includes the following steps:

[0030] S4.1. The input feature map is processed through parallel MaxPool and AvgPool layers, resulting in two 1×1 channel weight matrices. This transforms the feature map dimension from c×h×w to c×1×1. The MLP module compresses the number of channels to 1 / r of the original number, then expands it to the original number of channels using the ReLU activation function. This is added to the original feature map, and the channel attention output is obtained through the sigmoid activation function. This output is then multiplied by the original image to generate an intermediate feature map (where c represents the number of channels in the feature map, and r represents the channel number adjustment parameter). The weights W0 and W1 of the MLP module are shared across the input. The channel attention mechanism is expressed as follows:

[0031] M c (F)=σ(MLP(AvgPool(Fm))+MLP(MaxPool(Fm))

[0032] In the formula, M c (F) represents the channel attention mechanism; σ represents the sigmoid activation function; W0 and W1 represent the weight coefficients, W0∈R c / r×c W1∈R c×c / r AvgPool(Fm) represents the feature map processed by the average pooling layer; MaxPool(Fm) represents the feature map processed by the max pooling layer; MLP(AvgPool(Fm)) represents the feature map processed by the average pooling layer and the MLP module; MLP(MaxPool(Fm)) represents the feature map processed by the max pooling layer and the MLP module.

[0033] S4.2. Input the feature map in the spatial dimension, and apply average pooling and max pooling operations to the feature map F output by the channel attention module. The new feature map is denoted as F. S avg and F S max F S avg F represents the feature map that has undergone channel-dimensional average pooling. S max The first feature map represents the channel-dimensional max pooling operation, with a size of 1×h×w. Then, the new feature maps obtained from the two pooling operations are concatenated. Finally, a convolution operation is applied to the feature map to generate a spatial attention feature map of size R×h×w.

[0034] The spatial attention mechanism is expressed as follows:

[0035] M s (F)=σ(f 7×7 ([AvgPool(Fm);MaxPool(Fm)]))

[0036] In the formula, M s (F) represents the spatial attention mechanism, f 7×7 This represents a convolution process with a kernel size of 7×7.

[0037] In step 5, the regression loss function CIoU is replaced with the WIoU loss function, and the features fused by the feature fusion network are fed into the detection head for classification prediction.

[0038] WIoU is a bounding box loss function used to evaluate the quality of the bounding box. It features dynamic focusing and better measures the similarity between the predicted and ground truth bounding boxes, thus improving object detection performance. The WIoU loss function formula is shown below:

[0039]

[0040] In the formula, α is the hyperparameter; β is the outlier; θ represents the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box; r represents the gradient gain; w c h c L represents the width and height of the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box, respectively; WIOU Indicates WIoU loss, L IOU This represents the Cross-Union Loss (CUI). x represents the x-coordinate of the predicted bounding box center point, and y represents the y-coordinate of the predicted bounding box center point. t The x-coordinate of the center point of the target box is represented by y. t This represents the ordinate of the center point of the target bounding box.

[0041] Training inevitably involves low-quality samples. Factors such as aspect ratio and distance further increase the penalty for low-quality samples, reducing the model's generalization ability. When the overlap between the target box and the anchor box is high, the penalty for geometric factors should be reduced. Minimizing intervention in training can enhance the model's generalization ability. A dynamic, non-monotonic focused gradient gain allocation strategy is adopted to reduce the competitive advantage of high-quality anchor boxes and mitigate the adverse gradient effects caused by low-quality samples, resulting in higher model accuracy. This strategy focuses more on high-quality anchor boxes in the initial training phase, assigning smaller weights to low-quality anchor boxes to reduce harmful gradients.

[0042] Step 6 includes the following steps:

[0043] S6.1. Randomly divide the aluminum profile surface defect dataset into training, validation, and test sets in a ratio of 4:1:1. Randomly generate a reference point in the mosaic canvas with coordinates (x, y), where x represents the x-coordinate and y represents the y-coordinate. The reference point is randomly generated. The coordinates of the top right corner vertex of the mosaic canvas are (0, 0).

[0044] S6.2 As shown in Figure 6(a), based on the generated mosaic canvas, the original image is placed at the upper left, upper right, lower left, and lower right positions of the reference point of the mosaic canvas according to a fixed size; the size of the mosaic canvas is generally fixed and is twice the size of the original image, and the image inside the canvas is the image after mosaic data enhancement.

[0045] S6.3. Based on the size transformation method of each image, map the relationship to the image label;

[0046] S6.4 As shown in Figure 6(b), the mosaic canvas is stitched together based on the randomly generated reference point's horizontal and vertical coordinates. Four original images are randomly selected and edge-processed on the mosaic canvas. For example, when the original image in the upper right corner extends beyond the upper right edge of the canvas, the portion of the image extending beyond the canvas needs to be processed, retaining the image area within the canvas.

[0047] S6.5. The prepared dataset is input into the YOLOv7-ESC network model for pre-training. The pre-trained weights with the minimum training loss are selected for transfer learning. The aluminum profile surface defect dataset and the pre-trained weights are combined for training to obtain the optimal weights.

[0048] In step 7, the detection results are qualitatively evaluated, and the evaluation indicators include mean precision, recall, number of model parameters, computational cost, number of parameters, and number of frames.

[0049] The present invention provides a method for detecting and identifying defects in aluminum profiles based on YOLOv7-ESC, with the following technical advantages:

[0050] 1) In step 1 of this invention, the data preprocessing of the aluminum profile surface image and labels is an indispensable part of deep learning, which can improve the quality of data and the performance of the model. By preprocessing the training data, the accuracy of model prediction can be enhanced.

[0051] Image labeling is a crucial step in image preprocessing. In the aluminum profile image dataset, accurate labels provide the model with a clear learning objective, helping it to identify and classify features in the images and improve prediction accuracy. Defect areas are labeled using the Labelme image labeling software to ensure consistency and accuracy, reduce bias and errors in the data, improve the model's generalization ability, reduce the risk of overfitting, and thus enhance the model's learning performance. Next, the collected aluminum profile surface defect images are converted between RGB and HSI color spaces. This conversion not only helps improve the accuracy and robustness of defect detection but also simplifies the image processing, providing stronger support for surface defect analysis.

[0052] 2) In step 2 of this invention, in an industrial context, the aluminum profile surface defect detection model needs to emphasize practicality and often needs to be deployable in embedded devices for detection. Therefore, the detection model needs to meet accuracy requirements while being lightweight. The ELAN-P module uses a partially convolutional PConv convolutional module, which reduces the overall model's computational load, parameter count, and complexity, speeds up inference, and makes the model more lightweight, which is beneficial for its deployment on embedded devices.

[0053] Compared to the standard CNV convolution module, the PConv convolution module utilizes a subset of features from both the input and output channels for computation, preserving channel feature count while reducing redundant computation and memory access, thus extracting defect features more effectively. PConv offers lower computational cost compared to conventional convolution. Furthermore, it better utilizes computing power and demonstrates superior performance in simultaneously extracting spatial features. Therefore, employing the PConv convolution module not only maintains a lightweight model but also enhances feature map generation capabilities, providing significant advantages for the training and inference of aluminum profile surface defect detection models.

[0054] 3) In step 3 of this invention, there are certain similarities between different defects in aluminum profile defect detection, the background information is complex, and the defect scales vary greatly. The SPPC module is used to fuse feature maps of different scales, build associations between targets of different scales, and capture features of different scales. However, when processing small targets, it may still be unable to effectively extract sufficient contextual information, resulting in a decrease in overall detection performance.

[0055] Therefore, constructing a residual pyramid pooling structure SPPCSPC-F enhancement module focuses on small target defect information, better captures multi-scale defect information, and improves feature extraction capabilities. The residual structure in the SPPCSPC-F enhancement module can better propagate gradients during training, improve model performance, learn multi-scale features, and enhance the module's feature extraction capabilities.

[0056] 4) In step 4 of this invention, the background information in aluminum profile detection is complex, making it difficult to detect surface defects. To improve the model's ability to detect surface defects while suppressing redundant information, a CBAM attention mechanism module is added between the backbone network and the feature fusion network.

[0057] The CBAM attention mechanism module is an attention module for feedforward convolutional neural networks. It combines cross-channel information and spatial information to extract information features, allowing the model to adaptively learn the importance of channels and spatial locations. This helps the model focus on important features and suppress unimportant features, improving the model's detection capabilities without incurring excessive computational or parameter requirements.

[0058] 5) In step 5 of this invention, the loss function is used to measure the degree of difference between the predicted label and the ground truth label. A good loss function should weaken the penalty for geometric factors when the target box and the ground truth box overlap, thereby enabling the model to achieve better generalization ability. A suitable loss function can make the model converge faster, thereby improving the accuracy and stability of object detection.

[0059] YOLOv7's original coordinate loss function, CloU, uses complex calculations to consider not only the overlap between the predicted and ground truth bounding boxes and the distance between their center points, but also the aspect ratio. While it can accurately determine the similarity between predicted and ground truth bounding boxes, the extensive calculations consume significant computational resources and increase training time. Furthermore, when measuring small defect detection boxes on aluminum profile surfaces, CloU is highly sensitive to the positional deviations of tiny targets, resulting in poor small target recognition capabilities. Therefore, a dynamic non-monotonic focusing mechanism, namely the WIoU loss function, is adopted to replace the original CIoU loss function in YOLOv7.

[0060] 6) In step 6 of this invention, the Mosaic technique generates a new synthetic image by stitching together multiple images. This method can effectively increase the diversity of the dataset, help the model learn richer features, and thus improve the model's generalization ability. Through Mosaic enhancement, the generated synthetic image can provide more information in a single training session, thereby reducing the number of training rounds required, shortening training time, and improving training efficiency.

[0061] Pre-training allows the model to learn rich features on an aluminum profile surface defect dataset. This significantly reduces training time for subsequent training tasks because the initial weights of the network are already reasonably set, avoiding the time wasted training from scratch and improving model learning efficiency. Simultaneously, model pre-training effectively reduces the risk of overfitting when training on small datasets, improving model accuracy and robustness.

[0062] 4) The YOLOv7-ESC network model proposed in this invention reduces the model size, number of parameters, and computational cost by 16.7%, 16.9%, and 23.4% respectively compared to the original YOLOv7 model. The model's detection accuracy is improved, while the false negative rate is also reduced. The FPS is increased by 5.01, and the increased detection speed leads to better detection efficiency.

[0063] 5) This invention proposes a YOLOv7-ESC network model for the detection of 10 surface defects in aluminum profiles, which can meet the real-time performance requirements of industrial inspection. Attached Figure Description

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0065] Figure 1 The flowchart shows a method for detecting and identifying surface defects in aluminum profiles based on YOLOv7-ESC.

[0066] Figure 2 This is a structural diagram of the SPPC-F module;

[0067] Figure 3 This is a structural diagram of the ELAN-P module;

[0068] Figure 4 This is a structural diagram of the YOLOv7-ESC aluminum profile surface defect detection model.

[0069] Figure 5 The neural network recognition effect diagram obtained in the example is shown below;

[0070] Figure 6(a) shows the principle of Mosaic data augmentation;

[0071] Figure 6(b) is a schematic diagram of splicing the mosaic canvas. Detailed Implementation

[0072] like Figure 1 As shown, the surface defect detection and identification process for aluminum profiles based on YOLOv7-ESC includes the following steps:

[0073] S1. Obtain the publicly available dataset, then preprocess the aluminum profile surface images and labels to establish the YOLOv7-ESC network framework.

[0074] First, the bare aluminum profiles produced were collected and organized to obtain a dataset of surface defects. The defects included 10 categories: dents, non-conductivity, scratches, orange peel, exposed substrate, dents, pits, powdering, coating cracks, and dirt spots. A total of 1885 color images were collected. Then, traditional image processing was performed on the aluminum profile surface images to make it easier for the YOLOv7-ESC network to extract defect information.

[0075] Traditional image processing is used to process aluminum profile surface images. First, the collected aluminum profile surface defect images are converted between RGB and HSI color spaces. Then, the components are extracted as needed and grayscale transformation is performed. Next, noise is analyzed in the image, and a local median filtering algorithm is proposed and applied to process the noise, resulting in a surface defect image of the aluminum profile with prominent defects.

[0076] RGB color space:

[0077] The RGB color space is based on a cube constructed using Cartesian coordinates. The three primary colors, R, G, and B, correspond to the three coordinate axes, with gray levels assigned along each axis. In this coordinate model, the origin represents black, and the vertices of the cube on the coordinate plane correspond to cyan, magenta, and yellow, respectively. White is the point farthest from black, i.e., the vertex furthest from the origin. By normalizing all colors, meaning the values ​​of R, G, and B take values ​​in the range [0,1], various colors can be defined using vectors distributed around the origin. It is evident that different colors reside within the unit cube or on its surface.

[0078] HSI color space:

[0079] To separate regions of interest on a single plane based on color segmentation of aluminum profile surface images, a component of the HSI color space is used as a template image. The I component, representing brightness, is independent of the color information of the aluminum profile surface, while the H component (hue) and S component (saturation) are mathematically related to how human vision perceives color. Therefore, the HSI color space can be used as a color feature of the human visual system for research on image processing algorithms.

[0080] Grayscale transformation:

[0081] Grayscale transformation is an important method for spatial domain image enhancement. When applied to aluminum profile surface images, it can effectively improve the image quality, enhance the contrast, highlight more details in the target area and the inherent features of the aluminum profile surface image, and can also remove unwanted features from the image.

[0082] Median filtering can be expressed by the following formula:

[0083] g(x,y)=med{f(xw,yh),(w,h∈S)}

[0084] In the above formula, S is a two-dimensional template, i.e., a convolution kernel, with commonly used sizes of 3×3 and 5×5; f(xw,yh) represents the pixel grayscale value of the template window; f(x,y) and g(x,y) are the original image and the processed image, respectively.

[0085] like Figure 4 As shown, the YOLOv7-ESC network framework structure includes: a backbone network, a neck fusion network, and a detection head. The backbone network extracts the surface features of the aluminum profile in the image. The neck fusion network uses top-down and bottom-up connection methods to fuse multi-scale feature layers. Finally, the data is fed into the detection head for loss calculation and defect prediction.

[0086] S2. Replace the 3*3Conv module in the network ELAN module with the PCnov module to reduce redundant information extraction in the lightweight network.

[0087] The feature extraction module ELAN-P, as shown below Figure 3 As shown, it contains four consecutive PConv feature extractions. Compared to the Conv module, the Pconv module is characterized by using Conv convolutions to extract a portion of the features from the input and output channels for computation. This reduces redundant computation and memory access without losing channel feature count, and more effectively extracts defect features. The network computational cost, parameter count, and complexity are reduced, making the model more lightweight.

[0088] S3. Surface defects in aluminum profiles include ten categories: dents, non-conductivity, scratches, orange peel, exposed substrate, dents, pits, powdering, coating cracks, and dirt spots. Aluminum profile defects vary greatly in scale, and small-target defects are easily missed due to low detection accuracy. A residual pyramid pooling SPPCSPC-F module is constructed. The feature map downsampled from the backbone network is fed into the SPPCSPC-F module for feature extraction, extracting more multi-scale defect information.

[0089] The spatial pyramid pooling SPPCSPC-F module, as shown in... Figure 2 As shown, the input features are first processed into two channels. One channel undergoes continuous residual convolution, with the `add` operation maintaining the number of channels to obtain more defect information and refine the extracted features. The other channel is processed using an SPP (Single-Pyramid Pooling) structure, with four pooling layers of different sizes outputting four different receptive fields to facilitate the differentiation between large and small targets. Finally, the two channels are merged together using a `concat` fully connected operation. The reconstructed pyramid pooling module enhances the network's multi-scale feature learning ability, captures more defect information, improves the model's focus on small target information, and solves the problem of missed detection of small target defects on aluminum profile surfaces.

[0090] S4. A CBAM attention mechanism module is added to the neck network. FPN is a bottom-up multi-scale fusion network that fuses high-level features to low-level features, increasing the expressive power of low-level features and improving performance. Targets of different scales can be assigned to different layers for prediction, achieving a divide-and-conquer approach. CBAM is channel-space attention, which automatically learns the importance of each feature channel and feature space, then assigns a weight value to each feature channel and space based on this importance, allowing the neural network to focus on specific areas. This solves the problem of losing small target defect features on the aluminum profile surface due to different scale fusion. The CBAM module combines channel attention and spatial attention mechanisms into the network, rapidly expanding model performance without significantly increasing model parameters and scale.

[0091] Specifically, the following steps are included:

[0092] S4.1. The input feature map is processed through parallel MaxPool and AvgPool layers, resulting in two 1×1 channel weight matrices. This transforms the feature map dimension from c×h×w to c×1×1. The MLP module compresses the number of channels to 1 / r of the original number, then expands it to the original number of channels using the ReLU activation function. This is added to the original feature map, and the channel attention output is obtained through the sigmoid activation function. This output is then multiplied by the original image to generate an intermediate feature map (where c represents the number of channels in the feature map, and r represents the channel number adjustment parameter). The weights W0 and W1 of the MLP module are shared across the input. The channel attention mechanism is expressed as follows:

[0093] M c (F)=σ(MLP(AvgPool(Fm))+MLP(MaxPool(Fm)))

[0094] In the formula, M c (F) represents the channel attention mechanism, σ represents the sigmoid activation function; W0 and W1 represent the weight coefficients, W0∈R c / r×c, W1∈R c×c / r .

[0095] S4.2. Input the feature map in the spatial dimension, and apply average pooling and max pooling operations to the feature map F output by the channel attention module. The new feature map is F. S avg and F S max The two operations are then concatenated, and finally, a spatial attention feature map of size R×h×w is generated by applying the convolution operation feature map.

[0096] The spatial attention mechanism is expressed as follows:

[0097] M s (F)=σ(f 7×7 ([AvgPool(Fm);MaxPool(Fm)]))

[0098] In the formula, M s (F) represents the spatial attention mechanism; f 7×7 This represents a convolution process with a kernel size of 7×7.

[0099] S5. Replace the regression loss function CIoU with the WIoU loss function, and input the fused features into the detection head for classification prediction.

[0100] During training, low-quality samples are unavoidable. Geometric metrics such as distance and aspect ratio can exacerbate the penalty for low-quality instances, reducing the model's generalization performance. The model should reduce the gradient penalty of geometric metrics when the overlap between anchor boxes and target boxes is high, thereby reducing the gradient penalty for low-quality samples and focusing the target on ordinary samples to improve the model's generalization ability.

[0101] In the regression loss calculation, the original network used CIoU loss, which did not consider that geometric metrics such as distance and aspect ratio would exacerbate the penalty for low-quality instances, thus reducing the model's generalization performance. To address this issue, a new loss function, WIoU, is introduced to replace the original network's CIoU loss function. By weighting the region between the predicted and ground truth boxes, the gradient penalty of geometric metrics is weakened when the target and ground truth boxes overlap significantly, reducing the gradient penalty for low-quality samples. This makes the target more focused on the target, resulting in better model generalization. Assume (x, y) is located at position (x...y) in the target box. t y t The formula for WIOU is as follows.

[0102]

[0103] In the formula, θ represents the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box, and w c h c The width and height of the minimum bounding rectangle of the predicted bounding box and the ground truth bounding box.

[0104] S6. Training the YOLOv7-ESC network model: Divide the aluminum profile surface defect dataset into training, validation, and test sets in a 4:1:1 ratio and input them into the YOLOv7-ESC network model for pre-training. Train for a total of 200 rounds and select the pre-trained weights with the smallest training loss for transfer learning. Combine the aluminum profile surface defect dataset and the pre-trained weights for training and use Mosaic for data augmentation to obtain the optimal weights.

[0105] Specifically, the following steps are included:

[0106] S6.1. Randomly divide the aluminum profile surface defect dataset into training, validation, and test sets in a ratio of 4:1:1. Randomly generate a reference point in the mosaic canvas with coordinates (x, y), where x represents the x-coordinate and y represents the y-coordinate. The reference point is randomly generated. The coordinates of the top right corner vertex of the mosaic canvas are (0, 0).

[0107] S6.2 As shown in Figure 6(a), based on the generated mosaic canvas, the original image is placed at the upper left, upper right, lower left, and lower right positions of the mosaic canvas reference point according to a fixed size; the size of the mosaic canvas is generally fixed, which is twice the size of the original image, and the image inside the canvas is the image after mosaic data augmentation;

[0108] S6.3. Based on the size transformation method of each image, map the relationship to the image label;

[0109] S6.4, as shown in Figure 6(b). The mosaic canvas is stitched together based on randomly generated reference points' horizontal and vertical coordinates. Four original images are randomly selected and edge-processed on the mosaic canvas. For example, when the original image in the upper right corner extends beyond the upper right edge of the canvas, the portion of the image extending beyond the canvas needs to be processed, retaining the image area within the canvas. Mosaic data augmentation stitches together the four images. Each image contains defects with corresponding coordinates. Stitching the four images together yields a new image, along with the coordinates of the defects within that image. This new image is then fed into the neural network for learning.

[0110] S6.5. The prepared dataset is input into the YOLOv7-ESC network model for pre-training. The pre-trained weights with the minimum training loss are selected for transfer learning. The aluminum profile surface defect dataset and the pre-trained weights are combined for training to obtain the optimal weights.

[0111] S7. The trained optimal weights are used for testing, and the detection results are evaluated to ultimately achieve automated and intelligent identification of surface defect categories of aluminum profiles.

[0112] When evaluating detection results, the evaluation metrics include mean average precision (mAP), recall (R), model size, computational cost, number of parameters, and frames per second (FPS).

[0113] The mean precision is the average precision (AP) across all classes, representing the AP for each class. iLet be the area enclosed by the curves representing the precision (P) and corresponding recall (R) of the i-th class in the range of 0-1.

[0114] The formulas for calculating mAP, recall, and frames per second (FPS) are shown below.

[0115]

[0116] Among them, AP i FN represents the area enclosed by the curves representing the precision (P) and recall (R) of the i-th category in the range of 0-1. N represents the total number of categories, TP is the number of samples correctly classified as positive, and FN is the number of samples incorrectly classified as negative. S represents the time taken to detect one image, in seconds.

[0117] Table 1 Comparison of detection results of different models

[0118]

[0119] As shown in Table 1, the YOLOv7-ESC model proposed in this invention significantly improves performance compared to the YOLOv3 model, with an 8.7% increase in mAP@0.5, a 49.5% reduction in model size, a 48.2% reduction in parameters, and a substantial increase in FPS. Compared to the YOLOv5 model, mAP@0.5 increases by 4.4%, the model size decreases by 32.8%, and PFS is significantly improved. Compared to the original YOLOv7 model, the YOLOv7-ESC model can still guarantee real-time detection requirements in terms of FPS, with a 5.01 increase. However, this method improves mAP by 6.8%, and reduces model size, number of parameters, and computational cost by 16.7%, 16.9%, and 23.4%, respectively. Therefore, the model proposed in this invention better meets the needs of industrial real-time detection.

[0120] The surface defect detection results of aluminum profiles using the YOLOv7-ESC model are as follows: Figure 5 As shown, the algorithm detects ten types of defects, including dents, non-conductivity, scratches, orange peel texture, exposed substrate, dents, pits, powder buildup, coating cracks, and dirt spots. The YOLOv7-ESC network demonstrates higher confidence in defect detection and higher accuracy, as well as higher accuracy in detecting other defects. In summary, the algorithm of this invention is better able to detect small, often overlooked defect areas, achieving better detection results in industrial production defect detection.

Claims

1. A method for detecting and identifying defects in aluminum profiles based on YOLOv7-ESC, characterized in that... Includes the following steps: Step 1: Obtain the aluminum profile surface defect dataset, then preprocess the aluminum profile surface images and labels to obtain surface defect maps of aluminum profiles with prominent defects, and finally, establish the YOLOv7-ESC network framework. Step 2: Improve the ELAN module by replacing the Conv convolutional module in the backbone network ELAN module with the PCnov convolutional module to extract surface defects of aluminum profiles. Step 3: Reconstruct the SPPCSPC module and build the residual pyramid pooling structure SPPCSPC-F. The features extracted from the backbone network are input into the SPPCSPC-F residual pyramid pooling structure. Step 4: Add a CBAM attention mechanism module between the backbone network and the feature fusion network; Step 5: Replace the CIoU regression loss function with the WIoU loss function, and input the fused features from the feature fusion network into the detection head for classification prediction; Step 6: Train the YOLOv7-ESC network model; First, divide the aluminum profile surface defect dataset into training, validation, and test sets, and use Mosaic for further data augmentation during training; then input it into the YOLOv7-ESC network model for pre-training, and select the pre-trained weights with the smallest training loss for transfer learning; finally, combine the aluminum profile surface defect dataset and the pre-trained weights for training to obtain the optimal weights; Step 7: Use the trained optimal weights to perform tests and evaluate the detection results, ultimately achieving automated and intelligent identification of surface defects in aluminum profiles; In step 1, the YOLOv7-ESC network framework includes a backbone network, a neck feature fusion network, and a detection head. The backbone network extracts surface defect features of the aluminum profile from the input image. First, the input feature image is 640×640×3, and the extracted features are refined through four layers of CBL convolutional blocks. Then, the ELAN-P module reduces the number of parameters and downsamples to reduce the number of feature channels. Next, a feature map of size 20×20×1024 is input to the neck feature fusion network. The neck feature fusion network uses top-down and bottom-up connection methods to perform feature fusion processing on multi-scale feature layers. Finally, the output feature maps of different scales are fed into the detection head for loss calculation, defect localization, and defect prediction. In step 3, the residual pyramid pooling structure SPPCSPC-F includes an upper-layer multi-scale feature extraction channel and a lower-layer small target feature extraction channel. The upper-layer multi-scale feature extraction channel captures feature information at different scales through different pooling kernels to obtain richer feature target information. There is a 1×1 convolution adjustment channel before and after the pooling layer. The lower-layer small target feature extraction channel introduces three layers of residual convolution blocks. The CF module of three consecutive linear residual convolution blocks refines the extracted features and extracts more small target features. Step 4 includes the following steps: S4.1: The input feature map is processed through parallel MaxPool and AvgPool layers, resulting in two 1×1 channel weight matrices. This transforms the feature map dimension from c×h×w to c×1×1. The MLP module then compresses the number of channels to 1 / r of the original number, where r represents the gradient gain. The ReLU activation function expands the feature map to the original number of channels, which is then added to the original feature map. The sigmoid activation function is used to obtain the channel attention output, which is multiplied by the original image to generate an intermediate feature map. The weights W0 and W1 of the MLP module are shared across the input. The channel attention mechanism is expressed as follows: ； In the formula, This represents the channel attention mechanism; σ represents the sigmoid activation function; W0 and W1 represent the weight coefficients, W0∈R c / r×c W1∈R c×c / r ; Feature map representing the average pooling layer processing; This represents the feature map after processing by the max pooling layer; This represents the feature map processed by the average pooling layer and the MLP module. This represents the feature map processed by the max pooling layer and the MLP module. S4.2: Input the feature map in the spatial dimension, apply average pooling and max pooling operations to the feature map F output by the channel attention module, and denote the new feature map as F. S avg and F S max F S avg F represents the feature map that has undergone channel-dimensional average pooling. S max The feature map is represented by the channel dimension max pooling operation, with a size of 1×h×w; then the new feature maps obtained from the two pooling operations are concatenated; finally, the convolution operation is applied to the feature map to generate a spatial attention feature map of size R×h×w. The spatial attention mechanism is expressed as follows: ； In the formula, The spatial attention mechanism represented by f 7×7 This represents a convolution process with a kernel size of 7×7.

2. The aluminum profile defect detection and identification method based on YOLOv7-ESC according to claim 1, characterized in that: In step 1, the images of the produced aluminum profiles are first collected and organized to obtain a dataset of surface defects of the aluminum profiles. The defects include 10 categories: dents, non-conductivity, scratches, orange peel, exposed bottom, dents, pits, powdering, coating cracks, and dirt spots.

3. The aluminum profile defect detection and identification method based on YOLOv7-ESC according to claim 2, characterized in that: In step 1, data enhancement and data preprocessing are performed on the surface image of the aluminum profile. The defect area is marked using Labelme software, and a JSON file is generated to save the annotation information. Then, the surface defect image of the aluminum profile is preprocessed to achieve the effects of noise reduction and image enhancement, resulting in a surface defect image of the aluminum profile with prominent defects.

4. The aluminum profile defect detection and identification method based on YOLOv7-ESC according to claim 1, characterized in that: In step 2, replacing the 3*3 Conv convolutional module in the ELAN module with a PConv convolutional module can reduce the number of module parameters. The ELAN-P module consists of a short path and a long path. The short path is a 1*1 CBS convolution, and the long path consists of a 1*1 CBS convolution and four consecutive PConv convolutions.

5. The aluminum profile defect detection and identification method based on YOLOv7-ESC according to claim 4, characterized in that: Let the size of the input feature map X be c×h×w, where h, w, and c represent the height, width, and number of channels of the input feature map X, respectively; Input features I∈R c ×h×w The output is O∈R c×h×w The standard convolution kernel size is w'∈R c×k×k k represents the kernel size; since the PConv convolution module only needs to extract the feature from the cp channel, the kernel size is reduced to w'∈R. cp×k×k The formula for the computational cost ratio between PConv convolution and Conv convolution is as follows: ； In the above formula, Indicates the computational cost of PConv; Indicates the computational cost of Conv; Indicates the height of the input image features; The width of the input feature map; This indicates the number of channels extracted by PConv; This indicates the number of channels in the input feature map.

6. The aluminum profile defect detection and identification method based on YOLOv7-ESC according to claim 1, characterized in that: In step 5, the regression loss function CIoU is replaced with the WIoU loss function, and the features fused by the feature fusion network are fed into the detection head for classification prediction. The formula for the WIoU loss function is shown below: ； ； In the formula, For hyperparameters; Outlier degree; The Euclidean distance between the center points of the predicted bounding box and the center points of the ground truth bounding box is represented by r; the gradient gain is represented by w. c h c These are the width and height of the minimum bounding rectangles of the predicted bounding box and the ground truth bounding box, respectively; Indicates WIoU loss, Indicates the crossover and union ratio loss; This represents the x-coordinate of the center point of the prediction box. This represents the ordinate of the center point of the prediction box. This represents the x-coordinate of the center point of the target box. This represents the ordinate of the center point of the target bounding box.

7. The aluminum profile defect detection and identification method based on YOLOv7-ESC according to claim 1, characterized in that: Step 6 includes the following steps: S6.1: Randomly divide the aluminum profile surface defect dataset into training, validation, and test sets in a ratio of 4:1:1; randomly generate a reference point in the mosaic canvas with coordinates ( , ), representing the x-coordinate of the reference point, and , representing the y-coordinate of the reference point; the reference point is randomly generated; the coordinates of the upper right vertex of the mosaic canvas are (0, 0). S6.2: Based on the generated mosaic canvas, the original image is placed at the upper left, upper right, lower left, and lower right positions of the mosaic canvas reference point according to a fixed size; the size of the mosaic canvas is generally fixed, which is twice the size of the original image, and the image inside the canvas is the image after mosaic data augmentation; S6.3: Based on the size transformation method of each image, map the relationship to the image label; S6.4: The mosaic canvas is stitched together based on the randomly generated reference point's horizontal and vertical coordinates. Four original images are randomly selected and their edges are processed on the mosaic canvas. When the original image in the upper right corner exceeds the upper right corner edge of the canvas, the portion of the image that exceeds the canvas needs to be processed, while retaining the image area within the canvas. S6.5: Input the prepared dataset into the YOLOv7-ESC network model for pre-training, and take the pre-trained weights with the smallest training loss for transfer learning; combine the aluminum profile surface defect dataset and the pre-trained weights for training to obtain the optimal weights.

Images