A high-reflective metal surface defect detection method based on improved YOLOX

Abstract

The application discloses a high-reflective metal surface defect detection method based on an improved YOLOX, constructs an image dataset of metal surface defects, inputs the dataset into an image feature enhancement module for pretreatment, and makes corresponding labels for each type of defect; an improved YOLOX high-reflective metal surface defect detection model composed of a backbone feature extraction network CSPDarknet-53, a BT-FPN bidirectional feature fusion network and a classification prediction network YOLO Head is constructed, loss of feature information is reduced, and feature fusion performance is enhanced; a pre-training model is acquired by using transfer learning, and a weight w is obtained by training a metal defect dataset in combination with a Mosaic and Mixup data enhancement method; and the weight w is input into the improved YOLOX network for classification prediction. Compared with the prior art, the application improves the detection effect and detection speed of high-reflective metal defects, and has good real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of image processing and detection technology, specifically relating to a method for detecting defects on highly reflective metal surfaces based on an improved YOLOX. Background Technology

[0002] Aluminum boasts high recyclability and versatility, making it virtually irreplaceable in the chemical, construction, and automotive industries. However, as demand increases, the surface quality of aluminum profiles becomes increasingly crucial. Extruded aluminum profiles are frequently used in automotive parts, train frames, building doors and windows, and the supporting structures of skyscrapers. During production, surface defects such as scratches and blistering often occur due to tool wear and variations in process conditions. Proper identification of these surface defects is essential to prevent the delivery of defective materials to customers. Based on the severity and location of each defect, corresponding profiles can be appropriately prioritized to classify the defects.

[0003] Currently, the main method for inspecting the surface quality of aluminum profiles is still manual sampling and visual inspection. However, manual inspection is not only inefficient but also relies heavily on worker experience for accuracy, thus falling far short of the needs of my country's vigorous efforts to optimize and expand aluminum profile production capacity. At present, the detection of surface defects in aluminum profile equipment mainly relies on the human eye. In contrast, machine vision systems are widely used for quality inspection in industrial production sites for high-texture materials such as cork and textiles. For example, Gonzalez Adrados and Pereira applied image analysis and discriminant analysis techniques to classify certain defects in cork with an accuracy of over 90%. Lopes and Pereira used automated visual inspection technology to determine the quality of cork boards. Furthermore, Jordanov used an automated visual inspection system to classify four different types of cork tiles. First, they extracted features, then applied a feedforward neural network for classification, achieving an accuracy of up to 95%. Therefore, using machine vision for defect detection in the aluminum profile field is essential. Machine vision-based surface defect detection for aluminum profiles is a non-contact detection method. This method can quickly and effectively extract information from the surface of aluminum profiles, enabling the detection of surface defects.

[0004] Image analysis technology is not widespread in the metal processing industry because extruded aluminum is highly reflective, which leads to lighting problems in the images, making classification tasks more difficult. However, the high cost and time-intensive nature of quality control through the human eye are forcing the metal processing industry to move towards automated defect detection, providing a strong impetus for the development of automated defect detection for aluminum profiles. Summary of the Invention

[0005] Purpose of the invention: In view of the problems pointed out in the background art, the present invention provides a method for detecting defects on highly reflective metal surfaces based on the improved YOLO X, which is used to solve the technical problems of low accuracy and slow detection speed when detecting various defects in highly reflective metals.

[0006] Technical solution: This invention discloses a method for detecting defects on highly reflective metal surfaces based on improved YOLOX, comprising the following steps:

[0007] S1: Construct an image dataset of metal surface defects, input the dataset into the image feature enhancement module for preprocessing, and create corresponding labels for each type of defect;

[0008] S2: Construct an improved YOLOX high-reflectivity metal surface defect detection model consisting of a backbone feature extraction network CSPDarknet-53, a BT-FPN bidirectional feature fusion network, and a classification prediction network YOLO Head, to reduce the loss of feature information and enhance feature fusion performance;

[0009] S3: Use transfer learning to obtain a pre-trained model, and combine Mosaic and Mixup data augmentation methods to train the metal defect dataset to obtain weights. ;

[0010] S4: Weight The data is input into the improved YOLOX high-reflectivity metal surface defect detection model for classification and prediction.

[0011] Furthermore, the implementation process of step S1 is as follows:

[0012] S1.1: Obtain defect data samples required for high-reflectivity metal surface defect detection from the publicly available GC10-DET dataset, including four types of defect data: creases, oil spots, indentations, and welds;

[0013] S1.2: Preprocess the data samples by converting them into the standard VOC data format. Then, use the Labelme annotation tool to annotate the metal defects in the image data. The labels for metal defect types include creases, oil spots, indentations, and welds.

[0014] S1.3: Gaussian filtering and gamma correction image enhancement processing are performed sequentially on the image dataset that has already been annotated with defects. The image dataset is then denoised using Gaussian filtering. A weighted average is performed on each metal defect image. The value of each pixel is obtained by weighted averaging of its own value and the values ​​of other pixels in its neighborhood. The formula for calculating the Gaussian weight of each pixel is as follows:

[0015]

[0016] in , These are the coordinates of each pixel, and σ is... The variance;

[0017] The image dataset after Gaussian filtering is then subjected to gamma correction to adjust the grayscale values ​​and enhance contrast. This is achieved by performing a γ product operation on each pixel in the image. The gamma correction formula is as follows:

[0018]

[0019] Where s is the gray value of the output pixel, r is the gray value of the input pixel, and c and γ are constants. The gray values ​​of the image are adjusted by setting the values ​​of c and γ to achieve the optimal value.

[0020] S1.4: Divide the metal defect dataset after feature enhancement into training, validation and test sets in a ratio of 8:1:1.

[0021] Furthermore, the implementation process of step S2 is as follows:

[0022] S2.1: Construct the backbone feature extraction network CSPDarknet-53 using Focus, CBS, Resblock body and SPP modules, and establish a connection between the backbone feature extraction network CSPDarknet-53 and the BT-FPN bidirectional feature fusion network.

[0023] S2.2: Construct the CBS module using ordinary convolution Conv, normalized BN and SiLU activation function, split the residual block into two parts, the backbone and the large residual edge shortconv, to construct the CSP structure, and construct the Resblockbody structure using CBS and CSP structures. The SPP module consists of CBS and four pooling kernels of sizes 1×1, 5×5, 9×9 and 13×13.

[0024] S2.3: A BT-FPN bidirectional feature fusion network that emphasizes texture information is used. This network adds a bottom-up information flow and adds shortcut connections in the P4 and P5 layers.

[0025] S2.4: In the BT-FPN network structure, the three input layers P3, P4, and P5 correspond to the outputs of the Dark3, Dark4, and Dark5 backbones of the three Dark modules, including the CSP2_X layer, CBS block, up-sampling, and connection function. The CSP2_X layer uses stacked CBS modules instead of the remaining small part of the stack. The three related output formulas are as follows:

[0026] (1)

[0027] (2)

[0028] (3)

[0029] in, Indicates the first Image output, This represents the convolution operation. Indicates the first Layer Each node's weight Indicates the image number 1 Layer With input from each node, the feature extraction network has three different scales of output: small target, medium target, and large target. Because the bottom-up information flow is enhanced, the texture information acquisition capabilities of P4 and P5 are strengthened.

[0030] S2.5: Three preliminary feature layers of different sizes are extracted from the Resblock body structure of the backbone feature extraction network CSPDarknet-53 and input into the bidirectional feature fusion network BT-FPN for feature fusion. The three enhanced feature layers after feature fusion are then input into the YOLO Head for classification prediction.

[0031] Furthermore, step S3 is implemented as follows:

[0032] S3.1: Using transfer learning, the CSPDarknet-53 model, which has already been trained on the ImageNet dataset, is used as a pre-trained backbone network model. The training set is then input into the improved YOLOX high-reflectivity metal surface defect detection model for training, which is conducted for 100 rounds.

[0033] S3.2: In the first stage, the backbone of the improved YOLOX high-reflectivity metal surface defect detection model is frozen and trained iteratively for 50 epochs using Mosaic and Mixup data augmentation methods. In the second stage, the backbone of the improved YOLOX high-reflectivity metal surface defect detection model is unfrozen and trained iteratively for another 50 epochs to obtain the training weights. .

[0034] Furthermore, step S4 is implemented as follows:

[0035] S4.1: Input the metal surface defect dataset into the improved YOLOX high-reflectivity metal surface defect detection model and train it 100 times to obtain 100 sets of weights. Take the weight with the minimum loss value. Input into the improved YOLOX high-reflectivity metal surface defect detection model;

[0036] S4.2: Input the dataset of metal surface defects to be tested into the improved YOLOX high reflectivity metal surface defect detection model with optimal training weights, and use test set images to verify the defect detection effect of the improved YOLOX high reflectivity metal surface defect detection model.

[0037] Beneficial Effects: Compared with the prior art, the beneficial effects of this invention are as follows: This invention increases the diversity of sample data by performing data augmentation on the high reflectivity metal defect image dataset, adds a BT-FPN bidirectional feature fusion network module to reduce the loss of feature information and enhance the feature fusion performance, and connects the CSP_X layer with two parallel CBS modules and X residual units in series to enhance the feature fusion and learning ability of the backbone network to extract deep features, thereby classifying and locating metal defects, achieving the effect of multi-scale target detection, improving the accuracy and detection speed of high reflectivity metal defect detection, and having good real-time performance. Attached Figure Description

[0038] Figure 1 This is a flowchart of the present invention;

[0039] Figure 2 This is a diagram illustrating the types of highly reflective metal defects in an embodiment of the present invention.

[0040] Figure 3 This is a flowchart illustrating the structure of the improved YOLOX algorithm in an embodiment of the present invention.

[0041] Figure 4 This is a diagram of the backbone network structure of the improved YOLOX algorithm in an embodiment of the present invention;

[0042] Figure 5 This is a detailed diagram of the BT-FPN feature extraction network structure in an embodiment of the present invention;

[0043] Figure 6 This is a diagram of a simple BT-FPN feature extraction structure in an embodiment of the present invention. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings.

[0045] like Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for detecting defects on highly reflective metal surfaces based on improved YOLOX, which mainly includes the following steps:

[0046] Step 1: Construct an image dataset of metal surface defects, input the dataset into the image feature enhancement module for preprocessing, and create corresponding labels for each type of defect.

[0047] The dataset in this embodiment is sourced from GC10-DET metal defect image data, such as metal defect images. Figure 2 As shown, the image data consists of images of different types of defects caused by certain factors during production or transportation. The collected metal defect image data is classified into four categories: creases, oil spots, indentations, and welds. The defects and their respective quantities are shown in Table 1.

[0048] Table 1. Types and Quantities of Defects

[0049]

[0050] The data samples are preprocessed by converting them into the standard VOC data format. Then, the metal defects in the image data are labeled using the Labelme annotation tool. The labels for metal defect types include creases, oil spots, indentations, and welds. Data augmentation is performed on the sample data by rotating and cropping to expand the sample dataset.

[0051] The image dataset with labeled defects is sequentially subjected to Gaussian filtering and gamma correction image enhancement processing. First, the image dataset is denoised using Gaussian filtering. Then, a weighted average is performed on each metal defect image. The value of each pixel is obtained by weighted averaging of its own value and the values ​​of other pixels in its neighborhood. The formula for calculating the Gaussian weight of each pixel is as follows:

[0052]

[0053] in, , These are the coordinates of each pixel, and σ is... The variance;

[0054] Next, the image dataset after Gaussian filtering will undergo gamma correction to adjust the grayscale values ​​and enhance contrast. This involves performing a γ product operation on each pixel in the image. The gamma correction formula is as follows:

[0055]

[0056] Where s is the gray value of the output pixel, r is the gray value of the input pixel, and c and γ are constants. The gray values ​​of the image are adjusted by setting the values ​​of c and γ to achieve the optimal value.

[0057] Finally, the metal defect dataset after feature enhancement was divided into training set, validation set and test set in a ratio of 8:1:1.

[0058] Step 2: Construct an improved YOLOX high-reflectivity metal surface defect detection model consisting of a backbone feature extraction network CSPDarknet-53, a BT-FPN bidirectional feature fusion network, and a classification prediction network YOLO Head, to reduce the loss of feature information and enhance feature fusion performance.

[0059] like Figure 3 As shown, an improved YOLOX network structure is constructed. An improved YOLOX algorithm for detecting high-reflectivity metal surface defects is built, consisting of a backbone feature extraction network CSPDarknet-53, a bidirectional feature fusion network BT-FPN, and a classification prediction network YOLO Head. The backbone feature extraction network CSPDarknet-53 is constructed using Focus, CBS, Resblock body, and SPP modules. A connection is established between the backbone feature extraction network CSPDarknet-53 and the bidirectional feature fusion network. Three preliminary feature layers of different sizes are extracted from the Resblock body structure of the backbone feature extraction network CSPDarknet-53 and input into the BT-FPN bidirectional feature fusion network for feature fusion. The three enhanced feature layers after feature fusion are then input into the YOLO Head for classification prediction. The specific network structure includes:

[0060] The CBS module is constructed using ordinary convolution (Conv), normalized batch normalization (BN), and SiLU activation function. The residual block is split into a backbone and a shortconv part with large residual edges to construct the CSP structure. The Resblock body structure is constructed using the CBS and CSP structures. The SPP module consists of a CBS and four pooling kernels of sizes 1×1, 5×5, 9×9, and 13×13. The CSPDarknet-53 network structure is shown in Table 2.

[0061] Table 2 CSPDarknet-53 Network Structure

[0062]

[0063] The Focus structure obtains the pixel value channels of each image, regenerates each channel image into four new channel images, and then superimposes them. At this time, the width and height information of the image is concentrated into the channel information. Compared with the original image with 3 channels, the image becomes 12 channels, which allows the backbone network to retain more complete sampling information. The CSP layer structure is mainly composed of two parallel CBS modules and X residual units connected in series in the CSP_X layer. It can be divided into two parallel branches. One branch contains a CBL module followed by X residual units, and the other branch contains a CBL module. Such residual unit modules introduce a fast branch for the superimposed network, reducing the gradient vanishing and gradient difference return problems in deep networks.

[0064] Constructing a BT-FPN bidirectional feature fusion network, such as Figure 5 The diagram illustrates a BT-FPN bidirectional feature fusion network that prioritizes texture information. This network incorporates a bottom-up information flow and adds shortcut connections in layers P4 and P5. In the BT-FPN network structure, the three input layers (P3, P4, P5) correspond to the outputs of the three Dark modules (Dark3, Dark4, Dark5) backbone network, which include the CSP2_X layer, CBS blocks, up-sampling, and connection functions. Here, the CSP2_X layer is replaced by stacked CBS modules instead of the remaining small stack. The three relevant output formulas are as follows:

[0065] (1)

[0066] (2)

[0067] (3)

[0068] in, Indicates the first Image output, This represents the convolution operation. Indicates the first Layer Each node's weight Indicates the image number 1 Layer With input from each node, the feature extraction network has a total of three outputs at different scales: small target, medium target, and large target. This is because the bottom-up information flow is enhanced, thus… The ability to acquire texture information has been enhanced, such as Figure 6 As shown;

[0069] The YOLO Head structure feeds the fused feature maps into the YOLOX anchorless decoupled detection head. The YOLOX anchorless decoupled detection head obtains the target classification feature map, the target box position regression feature map, and the target box confidence regression map. The target classification feature map can be used to obtain the corresponding metal defect type information, and the target box position regression feature map can be used to obtain the corresponding position information.

[0070] like Figure 4 As shown, the metal defect image size is set to 640×640×3, and input into the Focus module for initial feature extraction. At this point, the channels are expanded to 12, and the image size becomes 320×320×12. Next, the metal defect image is processed through a CBS structure, reducing the size to 160×160×64. Further feature extraction is performed using a five-stage Resblock structure, with an SPP structure added to the last Resblock structure. The feature layer sizes for the five convolutions are 160×160×64, 80×80×128, 40×40×256, 20×20×512, and 10×10×1024, respectively. Four feature layers are derived from the backbone feature extraction network, with sizes of 80×80×128, 40×40×256, 20×20×512, and 10×10×1024. 1024, resulting in 4 output features. The features of the three feature layers P3, P4, and P5 are taken as the input of the BT-FPN bidirectional feature fusion network. The fused feature maps are then fed into the YOLOX anchorless decoupled detection head for classification and prediction.

[0071] Step 3: Use transfer learning to obtain a pre-trained model, and combine Mosaic and Mixup data augmentation methods to train the metal defect dataset to obtain weights. .

[0072] The CSPDarknet-53 model, already trained on the ImageNet dataset, was used as a pre-trained backbone network model through transfer learning. The image-enhanced training set of metal defects was input into the improved YOLOX high-reflectivity metal surface defect detection model for training, which lasted for 100 epochs. In the first stage, the backbone of the improved YOLOX high-reflectivity metal surface defect detection model was frozen and iterated 50 times using Mosaic and Mixup data augmentation methods. In the second stage, the backbone of the improved YOLOX high-reflectivity metal surface defect detection model was unfrozen, and iterated for another 50 epochs to obtain the training weights. .

[0073] Step 4: Weight The data is input into the improved YOLOX high-reflectivity metal surface defect detection model for classification and prediction.

[0074] After training the improved YOLOX high-reflectivity metal surface defect detection model 100 times using a dataset of metal surface defects, 100 sets of weights were obtained. Take the weight with the minimum loss value. The test dataset of metal surface defects was input into the improved YOLOX high reflectivity metal surface defect detection model. The defect detection effect of the improved YOLOX high reflectivity metal surface defect detection model was verified by using test set images.

Claims

1. A method for detecting defects on highly reflective metallic surfaces based on improved YOLOX, characterized in that, Includes the following steps: S1: Construct an image dataset of metal surface defects, input the dataset into the image feature enhancement module for preprocessing, and create corresponding labels for each type of defect; S2: Construct an improved YOLOX high-reflectivity metal surface defect detection model consisting of a backbone feature extraction network CSPDarknet-53, a BT-FPN bidirectional feature fusion network, and a classification prediction network YOLO Head, to reduce the loss of feature information and enhance feature fusion performance; S3: Use transfer learning to obtain a pre-trained model, and combine Mosaic and Mixup data augmentation methods to train the metal defect dataset to obtain weights w; S4: Input the weight w into the improved YOLOX high-reflectivity metal surface defect detection model for classification and prediction; The implementation process of step S2 is as follows: S2.1: Construct the backbone feature extraction network CSPDarknet-53 using Focus, CBS, Resblockbody and SPP modules, and establish a connection between the backbone feature extraction network CSPDarknet-53 and the BT-FPN bidirectional feature fusion network; S2.2: Construct the CBS module using ordinary convolution Conv, normalized BN and SiLU activation function, split the residual block into two parts, the backbone and the large residual edge shortconv, to construct the CSP structure, and construct the Resblockbody structure using CBS and CSP structures. The SPP module consists of CBS and four pooling kernels of sizes 1×1, 5×5, 9×9 and 13×13. S2.3: A BT-FPN bidirectional feature fusion network that emphasizes texture information is used. This network adds a bottom-up information flow and adds shortcut connections in the P4 and P5 layers. S2.4: In the BT-FPN network structure, the three input layers P3, P4, and P5 correspond to the outputs of the Dark3, Dark4, and Dark5 backbones of the three Dark modules, including the CSP2_X layer, CBS block, up-sampling, and connection function. The CSP2_X layer uses stacked CBS modules instead of the remaining small part of the stack. The three related output formulas are as follows: Among them, Pk out Represents the output of the k-th image, Conv represents the convolution operation, and w kj This represents the weight of the j-th node in the k-th layer. This represents the input of the nth node in the kth layer of the image. The feature extraction network has three outputs at different scales: small target, medium target, and large target. Because the bottom-up information flow is enhanced, the texture information acquisition capabilities of P4 and P5 are strengthened. S2.5: Three preliminary feature layers of different sizes are extracted from the Resblockbody structure of the backbone feature extraction network CSPDarknet-53 and input into the bidirectional feature fusion network BT-FPN for feature fusion. The three enhanced feature layers after feature fusion are then input into the YOLO Head for classification prediction.

2. The method for detecting high-reflectivity metallic surface defects based on improved YOLOX according to claim 1, characterized in that, The implementation process of step S1 is as follows: S1.1: Obtain defect data samples required for high-reflectivity metal surface defect detection from the publicly available GC10-DET dataset, including four types of defect data: creases, oil spots, indentations, and welds; S1.2: Preprocess the data samples by converting them into the standard VOC data format. Then, use the Labelme annotation tool to annotate the metal defects in the image data. The labels for metal defect types include creases, oil spots, indentations, and welds. S1.3: Gaussian filtering and gamma correction image enhancement processing are performed sequentially on the image dataset that has already been annotated with defects. The image dataset is then denoised using Gaussian filtering. A weighted average is performed on each metal defect image. The value of each pixel is obtained by weighted averaging of its own value and the values ​​of other pixels in its neighborhood. The formula for calculating the Gaussian weight of each pixel is as follows: Where x and y are the coordinates of each pixel, and σ is the variance of x; The image dataset after Gaussian filtering is then subjected to gamma correction to adjust the grayscale values ​​and enhance contrast. This is achieved by performing a γ product operation on each pixel in the image. The gamma correction formula is as follows: s=cr γ Where s is the gray value of the output pixel, r is the gray value of the input pixel, and c and γ are constants. The gray values ​​of the image are adjusted by setting the values ​​of c and γ to achieve the optimal result. S1.4: Divide the metal defect dataset after feature enhancement into training, validation and test sets in a ratio of 8:1:

1.

3. The method for detecting high-reflectivity metallic surface defects based on improved YOLOX according to claim 1, characterized in that, The implementation process of step S3 is as follows: S3.1: Using transfer learning, the CSPDarknet-53 model, which has already been trained on the ImageNet dataset, is used as a pre-trained backbone network model. The training set is then input into the improved YOLOX high-reflectivity metal surface defect detection model for training, which is conducted for 100 rounds. S3.2: In the first stage, the backbone of the improved YOLOX high-reflectivity metal surface defect detection model is frozen and trained iteratively for 50 rounds using Mosaic and Mixup data augmentation methods. In the second stage, the backbone of the improved YOLOX high-reflectivity metal surface defect detection model is unfrozen and trained iteratively for another 50 rounds to obtain the training weights w.

4. The method for detecting high-reflectivity metallic surface defects based on improved YOLOX according to claim 1, characterized in that, The implementation process of step S4 is as follows: S4.1: Input the metal surface defect dataset into the improved YOLOX high reflectivity metal surface defect detection model and train it 100 times to obtain 100 sets of weights w. Take the weight w with the smallest loss value and input it into the improved YOLOX high reflectivity metal surface defect detection model. S4.2: Input the dataset of metal surface defects to be tested into the improved YOLOX high reflectivity metal surface defect detection model with optimal training weights, and use test set images to verify the defect detection effect of the improved YOLOX high reflectivity metal surface defect detection model.

Images