Industrial defect detection method based on improved YOLOX model

Abstract

The application relates to the technical field of industrial defect detection, and discloses an industrial defect detection method based on an improved YOLOX model, which comprises the following steps: taking pictures of industrial products of a pipeline through an industrial depth camera, pretreating high-resolution industrial pictures into standard pictures after standard size cutting, inputting each pixel of each standard picture into an industrial defect detection network based on the improved YOLOX model after normalization, finally obtaining detection frame coordinates, confidence and categories of the pictures, carrying out post-processing on the obtained picture information, drawing the detection frame coordinates, confidence and category information to the original picture, and outputting the processed picture after drawing. The application adds an attention mechanism and self-adaptive feature fusion in a feature extraction network, so that the network can better focus on target objects. Finally, the prediction accuracy is improved without excessively losing speed during network prediction, and the speed and the accuracy are balanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial defect detection, and particularly relates to an industrial defect detection method based on an improved YOLOX model. BACKGROUND

[0002] In the field of industrial manufacturing, ensuring the quality of industrial products is an important task of industrial manufacturing. For industrial products, a small defect can sometimes harm the overall effect. For example, a breakpoint in a printed circuit board can affect the stable conduction of the device signal; metal cracks can affect the appearance and strength of the product. Generally, general industrial product quality detection is performed by human operation, which brings many problems, such as the need for a large amount of training for pre-detection personnel, increasing personnel training costs; as the on-duty detection time of personnel increases, the mis-detection rate caused by their own reasons increases. With the development of computer vision, using automated quality inspection has become a solution for the industry. Visual inspection of industrial defects can reduce costs and improve efficiency.

[0003] In the prior art, Kou et al. published "Development of a YOLO-V3-based model for detecting defects on steel strip surface", which discloses a feature selection mechanism based on YOLOv3 using anchor-free boxes to select ideal feature scales for steel defect detection. Although the above detector has achieved relatively good speed, it has not achieved high accuracy, and cannot achieve a good balance between speed and accuracy, making it difficult to meet actual industrial needs. Existing detectors often can only detect specific defect targets, and cannot detect small defects commonly found in industrial scenes, and cannot detect all types of defects, and the overall generalization ability of the model is insufficient. SUMMARY

[0004] The present application provides an industrial defect detection method based on an improved YOLOX model to solve the above problems.

[0005] The bottleneck structure in the backbone network is reduced, the activation function and normalization operation are activated, the attention mechanism and adaptive feature fusion are added in the feature extraction network, so that the network can better focus on the target object. Finally, the prediction accuracy is improved without too much loss of speed during network prediction, achieving the balance between speed and accuracy. Experiments show that the method proposed in the application takes the industry standard evaluation methods mAP@0.5:0.95 and mAP@0.5 as the benchmark, and the average accuracy of the NRSD data set reaches 61.06% and 85.00% respectively, the average accuracy of the PCB data set reaches 51.58% and 91.09% respectively, and the average accuracy of the NEU-DET data set reaches 49.08% and 80.84% respectively. A large number of comparative and ablation experiments verify the effectiveness and generalization ability of the model in the industrial defect detection scene.

[0006] The technical solution of the application solves the technical problems:

[0007] The industrial defect detection method based on the improved YOLOX model comprises the following steps: taking a photo of an industrial product on a production line through an industrial depth camera, preprocessing the high-resolution industrial picture into a standard picture after standard size cutting, and then normalizing each pixel of each standard picture as input information into the industrial defect detection network based on the improved YOLOX model.

[0008] The industrial defect detection network based on the improved YOLOX model comprises an attention-based backbone network, an attention-based multi-scale feature layer, an adaptive feature extraction and fusion network, and a decoupling head network.

[0009] The attention-based backbone network comprises adding an ECA channel attention mechanism at the output position of the last three layers of the backbone network of the original YOLOX network.

[0010] The attention-based multi-scale feature layer and the adaptive feature extraction and fusion network comprise adding an ECA channel attention mechanism at the output position of the Fpn layer-2 of the PANet feature extraction network and the output positions of the three pan layers based on the feature extraction and fusion network of the original YOLOX network, and taking the different scale feature maps of 20x20, 40x40 and 80x80 output by the attention module of the PANet feature extraction network as input, performing weighted fusion operation, and taking the calculated results as output in the scale of 20x20, 40x40 and 80x80, respectively, and entering the respective decoupling head network.

[0011] In the attention-based backbone network, the input industrial image undergoes five convolution operations with halved size and doubled channels to perform preliminary feature extraction and fusion. The third, fourth, and fifth convolutions of the backbone network's industrial image features are then output. In the attention-based multi-scale feature layer and adaptive feature extraction and fusion network, the image feature information output from the three layers of the backbone network is used as input. This input is then superimposed and fused with the feature extraction from the multi-scale feature layer using both bottom-up and top-down structures, completing deep adaptive feature extraction and fusion of the image features and outputting it. In the decoupled head network, the final image feature information undergoes matrix calculation to obtain the bounding box coordinates, confidence score, and category. The network then performs post-processing on the acquired image information, plotting the bounding box coordinates, confidence score, and category information onto the original image, and outputting the processed image.

[0012] Furthermore, the weighted fusion operation is performed by an adaptive spatial feature fusion module, and the operation method is as follows:

[0013] These represent the feature information from the three attention mechanisms eca1, eca2, and eca3 of the PANet feature extraction layer, respectively.

[0014] Combine the above feature information with weight parameters Multiply the feature maps, adjust them to the same size, and then add them together to obtain the fusion layer. The final output of this layer is given by the following formula:

[0015]

[0016]

[0017]

[0018]

[0019] In formula (1) Let be the weight parameters shared by α, β, and γ in the channel at position (i,j), respectively. In formulas (2) and (3)... The weight parameters are defined by the softmax function and are summed to 1 and belong to the range [0,1]. Formula (4) is the weight parameter. The calculation method, in which Through convolution in X eca1→level X eca2→level X eca3→level The calculated θ is the set of weight parameters α, β, and γ. is the weight parameter name of the calculation collectively.

[0020] Further, the SiLU activation function after the first 1x1 convolution of the bottleneck structure in the original YOLOX network is deleted to retain a normalization function, and the normalization function after the second 3x3 convolution of the bottleneck structure in the original YOLOX network is deleted to retain a SiLU activation function.

[0021] A computer readable medium having stored thereon a computer program for performing the method as described above.

[0022] Advantages of the application:

[0023] 1. The feature extraction network part is optimized and improved, attention mechanism and adaptive feature fusion are added at a reasonable position, so that the feature extraction network has strong object focusing.

[0024] 2. The adaptive spatial feature fusion solves the inconsistency problem between multi-scale features, so that the network can directly learn how to spatially filter features at other levels, so as to only retain useful information for combination. Figure 6 As shown in the figure, the feature extraction layer retains three different scale feature maps output from the attention module, the adaptive spatial feature fusion mechanism performs weighted summation on the three different scale feature map information with scales of 20x20, 40x40 and 80x80 respectively, calculates the corresponding weight, enriches the multi-scale information, and achieves a spatial attention effect. The adaptive spatial feature fusion module is introduced at the end position of the PANet feature extraction network, which is simple for the network, plug and play, and the additional calculation cost is suitable. No additional inference overhead is introduced, and the experimental results are obviously improved.

[0025] 3. The SiLU activation function after the first 1x1 convolution of the bottleneck structure in the original YOLOX network and the normalization function after the second 3x3 convolution are deleted, so that the network calculation amount is reduced, and the convergence speed of the model is accelerated.

[0026] 4. Through comparison experiments with other mainstream models, it is proved that the effect of the model of the application in detecting industrial defects is remarkable, which embodies the effectiveness of the whole model. Through extensive comparison and ablation experiments, it is proved that the detection effect of the improved model is improved in VOC dataset and public industrial dataset, which verifies the effectiveness and generalization ability of each module. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a schematic diagram of the original YOLOX network structure;

[0028] Figure 2Structure diagram of the network model of the present application;

[0029] Figure 3 Structure diagram of the PANet of the original YOLOX network;

[0030] Figure 4 Structure diagram of the PANet of the present application;

[0031] Figure 5 Structure diagram of the ECA channel attention mechanism;

[0032] Figure 6 Structure diagram of the spatial feature fusion module of the present application;

[0033] Figure 7 Improvement comparison diagram of bottleneck structure design, wherein Figure 7 a is a bottleneck structure diagram of the original YOLOX network, Figure 7 b is a bottleneck structure diagram of the present application;

[0034] Figure 8 NRSD detection result diagram, wherein Figure 8 a is a data set real label, Figure 8 b is a prediction label of the original YOLOX network, Figure 8 c is a prediction label of the model of the present application;

[0035] Figure 9 PCB detection result diagram, wherein Figure 9 a is a data set real label, Figure 9 b is a prediction label of the original YOLOX network, Figure 9 c is a prediction label of the model of the present application;

[0036] Figure 10 NEU-DET detection result diagram, wherein Figure 10 a is a data set real label, Figure 10 b is a prediction label of the original YOLOX network, Figure 10 c is a prediction label of the model of the present application. DETAILED DESCRIPTION

[0037] For purposes of clarity, the present application will be described with reference to specific examples thereof. Each example is provided to further explain the present application and should not be considered as limiting. The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For purposes of simplicity of the present disclosure, specific examples are described in terms of specific materials, procedures, conditions and the like. Such examples are provided for purposes of illustration only and should not be considered as limiting the present application in any way. In addition, the present application can be implemented in different embodiments and of different combinations of the disclosed examples. The disclosure is not limited to the specific details set forth herein.

[0038] Explanation of terms: mAP means mean average precision, which is a standard for evaluating the accuracy of a model in the field of object detection;

[0039] mAP@0.5:0.95: 0.5 represents the starting intersection over union threshold, and 0.95 is the end intersection over union threshold, that is, the average value of the accuracy of all classes is calculated in the range of 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9 and 0.95, and then the average value of the accuracy of the entire interval is calculated;

[0040] mAP@0.5: the average value of the accuracy of all classes is calculated at the intersection over union threshold of 0.5;

[0041] FPS: frames per second transmission rate;

[0042] ASFF: adaptive spatial feature fusion.

[0043] The operation of the industrial defect detection method based on the improved YOLOX model is as follows: an industrial product on a production line is photographed by an industrial depth camera, the photographed high-resolution industrial picture is preprocessed into a standard picture after being cut to 640*640 size, and then each pixel of each standard picture is normalized and input into the industrial defect detection network based on the improved YOLOX model as input information; in the attention-based backbone network part, the input industrial picture is subjected to five times of convolution operation of size reduction by half and channel expansion by two, to complete preliminary feature extraction and fusion of the industrial picture, and finally the third, fourth and fifth industrial picture feature information of the backbone network part is output; in the attention-based multi-scale feature layer and adaptive feature extraction and fusion network, the picture feature information of the three layers of the backbone network is input into the input part, and the input part is superimposed and fused with the feature extraction of the multi-scale feature layer from bottom to top and from top to bottom structure, to complete deep adaptive feature extraction and fusion of the picture feature information and output; in the decoupling head network, the final feature information of the picture is subjected to matrix calculation here, and finally the detection box coordinates, confidence and class of the picture are obtained, the network is subjected to post-processing for the obtained picture information, the detection box coordinates, confidence and class information are drawn to the original picture, and the processed picture after drawing is output.

[0044] In the prior art, YOLOX has six different models, namely YOLOX-nano, YOLOX-tiny, YOLOX-s, YOLOX-m, YOLOX-l and YOLOX-x. Two papers "YOLOv4: Optimal Speed and Accuracy of Object Detection" and "Spatial pyramid pooling in deep convolutional networks for visual recognition" disclose that YOLOX uses CSP-Darknet and SPP as the backbone network structure, and "Path aggregation network for instance segmentation" discloses that PANet is used as the Neck part network, and the Head part adopts a decoupling structure different from the previous YOLO series.

[0045] The reference target detector selected in the application is YOLOX, which achieves a good balance between speed and accuracy in the overall structure and each substructure, and has better convergence during training.

[0046] As Figure 1As shown, the original network structure of YOLOX is divided into a backbone network, a feature extraction and fusion network, and a decoupling head network.

[0047] As shown in the prior art, Figure 3 As shown, the PANet structure of the original YOLOX network only contains basic convolution, upsampling, merging function and CSP layer.

[0048] As shown in the prior art, Figure 4 As shown, the ECA channel attention mechanism is added at the output positions of the last three layers of the backbone network of the prior art YOLOX network and the output positions of all CSP layers of the PANet network.

[0049] Embodiment 1

[0050] As shown in the prior art, Figure 2 As shown, the industrial defect detection network based on the improved YOLOX model is based on the YOLOX network, including an attention-based backbone network, an attention-based multi-scale feature layer, and a self-adaptive feature extraction and fusion network, and a decoupling head network.

[0051] The attention-based backbone network includes adding an ECA channel attention mechanism at the output positions of the last three layers of the backbone network of the original YOLOX network.

[0052] The attention-based multi-scale feature layer and the self-adaptive feature extraction and fusion network include adding an ECA channel attention mechanism at the output positions of the Fpn layer-2 of the PANet feature extraction network and the output positions of the three pan layers based on the feature extraction and fusion network of the original YOLOX network, and taking the feature maps of different scales of 20x20, 40x40 and 80x80 output by the attention module of the PANet feature extraction network as input, performing weighted fusion operation, and taking the calculated results as output in the scales of 20x20, 40x40 and 80x80, respectively, and entering the respective decoupling head network.

[0053] As shown in the prior art, Figure 2As shown, in order to better focus on industrial defects, ECA channel attention mechanisms are added to the output positions of the last three layers of the backbone network of the prior art YOLOX network, the output position of the Fpn layer-2, and the output positions of the three pan layers, respectively. By adding the ECA channel attention mechanism, the output positions after part of the feature extraction and fusion are changed, and the fusion between various structures of the model is improved. Using the ECA channel attention mechanism will not increase too many parameters for the model of the application, and it will assign a weighting coefficient to the relevance of different feature maps, thereby strengthening the effect of key features. The application adds an adaptive spatial feature fusion after the PANet feature extraction network, which performs weighted summation on the three scale feature information outputs of the last three layers of the feature extraction network, thereby improving the invariance of the feature scale.

[0054] As shown in Figure 5 , the ECA channel attention mechanism is a simple, effective, and plug-and-play attention mechanism, which is a lightweight attention module. In the use process, dimension reduction measures are avoided, local cross-channel interaction is efficiently realized by using one-dimensional convolution, the dependency between channels is extracted, additional network parameters are avoided, and the key features transmitted from the previous layer are strengthened. In order to enable the network to learn the required features each time, the application adds an ECA attention module in the improved model. The CSP layer can enhance the ability of the entire network to learn features, and after feature extraction, the ECA attention mechanism is transmitted. ECA first performs average pooling operation on the transmitted feature map, then uses one-dimensional convolution operation with a convolution kernel of 3 and passes through a Sigmoid activation function to obtain the weight of each channel. Finally, a basic convolution is used as the transition of network learning, and the result is output to the subsequent basic convolution block or is output separately. By adding the ECA attention module, the overall model can better locate and strengthen the region of interest.

[0055] The weighted fusion operation is completed through the adaptive spatial feature fusion module, and the operation method is as follows:

[0056] respectively represent the feature information of the three-layer attention mechanisms eca1, eca2, and eca3 of the PANet, and the above feature information is multiplied by the weight parameter , and after adjusting to the same size of feature map, the result is added to obtain the fusion layer , which is the final output of the layer, and the formula is as follows:

[0057]

[0058]

[0059]

[0060]

[0061] in formula (1) are scalars shared in the channel at position (i,j) respectively for a, b, g, and are parameters defined by a softmax function and are 1 and range belongs to [0, 1], and formula (4) is a weight parameter is calculated by the method, wherein is calculated by convolution in X eca1→level , X eca2→level , X eca3→level , and θ is a set of weight parameters a, b, g, is a general term of the weight parameter name calculated.

[0062] The feature maps of different scales of 20x20, 40x40 and 80x80 output by the attention module are input, and the results calculated after weighted fusion operation by the adaptive spatial feature fusion module are output in the scale of 20x20, 40x40 and 80x80, and respectively enter the respective decoupling head layer.

[0063] The reason for the adaptive spatial feature fusion is to solve the inconsistency problem between multi-scale features, so that the network can directly learn how to spatially filter features at other levels, so as to only retain useful information for combination. As shown in Figure 6 the feature extraction layer of the present application retains three different scale feature maps output from the attention module, and the adaptive spatial feature fusion mechanism performs weighted summation on the feature map information of three different scale feature maps with scales of 20x20, 40x40 and 80x80 respectively, calculates the corresponding weight, enriches the multi-scale information, and achieves the effect of spatial attention. The adaptive spatial feature fusion module is introduced at the end position of the PANet in the present application, which is simple for the network, plug and play, and the additional calculation cost is suitable, without introducing additional inference overhead, and the experimental results are obviously improved.

[0064] Example two

[0065] The difference from example one is that the structure of the bottleneck structure in the original YOLOX network is that a normalization function and a SiLU activation function are used after the first 1x1 convolution, and a normalization function and a SiLU activation function are used after the second 3x3 convolution. This embodiment deletes the SiLU activation function after the first 1x1 convolution of the bottleneck structure in the original YOLOX network and retains a normalization function, and deletes the normalization function after the second 3x3 convolution of the bottleneck structure in the original YOLOX network and retains a SiLU activation function, as shown in Figure 7 .

[0066] Example three

[0067] A computer readable medium for implementing the method of examples 1 and 2.

[0068] The present application uses three public industrial datasets to evaluate the model of the present application, and reports the mAP@0.5, mAP@0.5:0.95 and FPS of each dataset in the detection experiment.

[0069] The following public datasets are used in this experiment: NRSD dataset, PCB detection dataset and NEU-DET detection dataset, which are disclosed in "MCnet: multiple context information segmentation network of No-service rail surface defects", "A PCB dataset for defects detection and classification", "A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects". Among them, the NRSD dataset has a total of 4101 pictures of 1 class, and the picture size is in the range of 600 to 800 pixels. In the experiment, it is divided into 2971 training sets and 1130 validation sets; the PCB detection dataset has a total of 693 pictures of 6 classes, and the picture size is all above 1000 pixels. In the experiment, it is divided into 554 training sets and 139 validation sets; the NEU-DET detection dataset has 1800 pictures of 6 classes, and the picture size is 200x200 pixels. In the experiment, it is divided into 1620 training sets and 180 validation sets.

[0070] Early industrial data sets have problems such as less category, less obvious features, low resolution and poor recognition accuracy, and therefore cannot meet the requirements of current detection tasks. In order to detect complex or multiple defect conditions, it is not limited to detecting the target box of the defect. The present application additionally labels the required label file for detection on the basis of the existing public NRSD data set.

[0071] Data processing:

[0072] Data augmentation is a key link in data processing. Through the diversity processing of picture data style, the richness of data categories can be ensured, and the size of the data amount can be expanded. In the beginning of training, the present application performs basic enhancement operations such as random cutting, scaling and flipping on the picture, and also performs Mosaic and MixUp processing on the picture to make the data set style more diverse, wherein the Mosaic and MixUp processing are disclosed in “YOLOv4: Optimal Speed and Accuracy of Object Detection” and “mixup: Beyond empirical risk minimization”, respectively.

[0073] Experimental setup:

[0074] The experimental environment of the present application is Ubuntu 18.04, Python 3.8, Pytorch 1.8 and CUDA11, and all model training and testing are performed using the same NVIDIA RTX 3060 with 12GB of video memory.

[0075] Before training, in order to retain as many model learning features as possible for better results after training, the training settings of the present application are as follows: the input image size is 416x416 or 640x640; the model training period is 200 times, and the first 5 times use warmup; the optimizer is selected as the stochastic gradient descent method; the learning rate is lr x batch size / 64, and the initial learning rate is 0.01 and the cosine annealing method is used. In order to prevent memory overflow during training, the batch size is set to 8 each time, and the network does not use pre-trained weights. When training the NRSD detection data set, the experiment uses Mosaic and Mixup data augmentation throughout; when training the PCB and NEU-DET detection data sets, the experiment uses Mosaic and Mixup data augmentation for the last 25 training periods.

[0076] After training, in order to verify the effects of the comparative experiment and the ablation experiment after training, the comparative experiment is compared with YOLOv3-tiny, YOLOv5-s and YOLOX-tiny and YOLOX-s, and the ablation experiment is compared with independent modules to ensure the rigor of the experiment.

[0077] Evaluation indicators of the model:

[0078] For the model evaluation indicators, precision, recall and mAP are commonly used.

[0079] The precision, recall and mAP formulas are as follows:

[0080]

[0081]

[0082]

[0083] Wherein, precision represents accuracy; recall represents recall rate. TP represents a positive sample predicted as a positive class by the model; FP represents a negative sample predicted as a positive class by the model; FN represents a positive sample predicted as a negative class by the model. The precision is the accuracy of the model in predicting the positive sample; the recall rate is the proportion of the correctly identified industrial defects in all pictures of the type. The mAP is the average of the areas of the precision and recall in the respective ranges.

[0084] Experimental results and analysis:

[0085] In the research of the present application, one comparative experiment and four ablation experiments are designed, and mAP@0.5:0.95, mAP@0.5 and FPS are used as evaluation standards. The detector takes the original YOLOX as the baseline. In the target detection experiment, the tiny and s models of YOLOX are used as the basic model, and compared with the YOLOv3-tiny and YOLOv5-s models. In the comparative experiment, (a) represents the real label of the picture; (b) represents the predicted label of the original YOLOX network; (c) represents the predicted label of the model of the present application. In the ablation experiment, A represents the ECA attention mechanism; B represents the adaptive spatial feature fusion; and C represents the improved bottleneck structure.

[0086] Comparative experiment results and analysis

[0087] As shown in Table 1, the model of the application has better effect in the selected data set. The score in the NRSD data set is 61.06, higher than the highest score 57.74 of the baseline model, higher than the highest score 52.10 of YOLOv5-s, and far higher than the highest score 46.29 of YOLOv3-tiny. In the PCB data set, the score of the model of the application is 51.58, higher than the highest score 49.72 of the baseline model, higher than the highest score 45.19 of YOLOv5-s, and higher than the highest score 42.48 of YOLOv3-tiny; in the NEU-DET data set, the score of the model of the application is 49.08, higher than the highest score 47.61 of the baseline model, and far higher than the highest scores of YOLOv5-s and YOLOv3-tiny.

[0088] Table 1 detection comparison experiment results on industrial defect data set

[0089]

[0090] The model of the application in the NRSD data set in different brightness scene result pictures is shown in Figure 8 . As can be seen from Figure 8 b, the first, second and third pictures show that the original YOLOX network model has incomplete detection in the first and second pictures, and multiple detection in the third picture. As shown in Figure 8 c, Figure 8 the first and third pictures of Figure 8 c, the model of the application detects accurately; in the second picture of c, the model of the application has multiple detection boxes, but the model of the application detects completely in the case of using accurate detection boxes than the original YOLOX network model, and the overall detection effect of the model of the application in different brightness scenes is appropriate, which achieves the expected detection.

[0091] Figure 9 As shown in Figure 9 , when checking the experimental results on the PCB data set, it is found that both the model of the application and the original YOLOX network model have missing detection. As shown in Figure 9 c, the first and third pictures of the model of the application detect accurately, and as shown in c, the second picture of the model of the application also has missing detection, but the overall number of missing detection is less than that of the original YOLOX network.

[0092] Figure 10 The NEU-DET data set experimental results are shown in Figure 10 , when checking the predicted results, it is found that, as shown in Figure 10As shown in the fourth image of b, the original YOLOX network exhibits missed detections in the fourth image when performing multi-type predictions on a single image. For example... Figure 10 As shown in image c, the model of this invention performs accurately in the second and third images. While the model of this invention does not exhibit missed detections, over-detections, or incomplete detections in the first and fourth images, it does show minor errors. Overall, the improved model of this invention performs quite well.

[0093] To verify the effectiveness of each module of the model in this invention, ablation experiments were first conducted on the classic VOC dataset. As shown in Table 2, the experiment used YOLOX-tiny as the baseline model, with the input size set to 416×416. Other training settings remained consistent with the baseline settings. The model's mAP and FPS results were measured by adding modules one by one. Attention mechanisms were added to YOLOX-tiny at different positions in FPN and PAN. It was found that adding the attention mechanism to the entire feature extraction network improved the performance by 0.11% compared to adding it only to FPN. Adding an ASFF network after the feature extraction network improved the performance by 0.81% compared to the original network. Adding both the attention mechanism and the ASFF network simultaneously improved the performance by 1.56% compared to the baseline network. Therefore, the model with the attention mechanism and ASFF network added to the overall feature extraction network was selected as the experimental model for this invention.

[0094] Table 2 Ablation Experiment Results of VOC Public Dataset

[0095] Network name mAP@0.5:0.95 (%) mAP@0.5 (%) FPS Original YOLOX network 35.85 59.49 340 +A(FPN) 36.62 60.45 320 +A(FPN+PAN) 36.73 60.10 328 +B 36.66 60.84 288 +A(FPN)+B 37.10 60.93 289 +A(FPN+PAN)+B 37.41 61.06 298

[0096] The modules selected in this invention were validated through ablation experiments on the NRSD, PCB, and NEU-DET datasets. YOLOX-s was used as the baseline model, with the input size uniformly set to 640×640. Effective experimental methods were extracted from the YOLOX-tiny test on the VOC public dataset and added to the baseline model of this invention.

[0097] Table 3 is the mAP and FPS result of adding modules one by one. From Table 3, the influence of each part of the model on the performance of all indicators can be seen. The experiment improves 0.98% than the baseline model after using ECA attention mechanism alone in the NRSD data set; improves 1.40% than the baseline model after using adaptive spatial feature fusion alone; improves 0.52% than the baseline model after using the modified bottleneck structure alone. Adding all modules to the baseline model improves 2.79% than the baseline model. It can be seen that the results of the modules added in the application on the NRSD single defect type data set improve more in comparison with the baseline, although the FPS is in a small loss range, but the speed and accuracy of the model of the application reaches a good balance.

[0098] Table 3 is the mAP and FPS result of adding modules one by one. From Table 3, the influence of each part of the model on the performance of all indicators can be seen. The experiment improves 0.98% than the baseline model after using ECA attention mechanism alone in the NRSD data set; improves 1.40% than the baseline model after using adaptive spatial feature fusion alone; improves 0.52% than the baseline model after using the modified bottleneck structure alone. Adding all modules to the baseline model improves 2.79% than the baseline model. It can be seen that the results of the modules added in the application on the NRSD single defect type data set improve more in comparison with the baseline, although the FPS is in a small loss range, but the speed and accuracy of the model of the application reaches a good balance.

[0099] Network name NRSD mAP@0.5:0.95 (%) NRSD mAP@0.5 (%) FPS Original YOLOX network 58.27 81.89 144 +A 59.25 83.92 142 +B 59.67 83.55 124 +C 58.79 83.29 151 +A+B+C 61.06 85.00 125

[0100] The ablation experiment results of the PCB data set are shown in Table 4, taking the original YOLOX network as the baseline model, the mAP and FPS results of adding modules one by one. After using ECA attention mechanism alone, it improves 0.72% than the baseline model; after using adaptive spatial feature fusion alone, it improves 1.55% than the baseline model; after using the modified bottleneck structure alone, it improves 1.30% than the baseline model. Adding all modules to the baseline model improves 1.86% than the baseline model. It can be seen that the results of the modules added in the application on the PCB multi-defect type high-resolution data set have obvious improvement on the baseline, and the FPS is also within a reasonable loss range.

[0101] Table 4 is the ablation experiment result of the PCB data set.

[0102]

[0103]

[0104] The ablation experiment results of the NEU-DET data set are shown in Table 5, taking the original YOLOX network as the baseline model, the mAP and FPS results of adding modules one by one. After using ECA attention mechanism alone, it improves 0.87% than the baseline model; after using adaptive spatial feature fusion alone, it improves 0.57% than the baseline model; after using the modified bottleneck structure alone, it improves 0.92% than the baseline model. Adding all modules to the baseline model improves 1.47% than the baseline model.

[0105] Table 5 is the ablation experiment result of the NEU-DET data set.

[0106]

[0107] In order to meet the needs of defect detection, the present application performs various data augmentations on a small number of industrial datasets to ensure data diversity, proposes an improved industrial defect detector based on the YOLOX model in YOLox: Exceeding yolo series in 2021 published by Z. Ge, S. Liu, F. Wang, Z. Li and J. Sun, adds attention mechanism and adaptive spatial feature fusion to the feature extraction network, reduces the activation function and normalization operation for the bottleneck structure in the backbone network, meets the needs of industrial defect recognition and positioning in multiple complex scenes, and improves the accuracy of the detector. The method detects mAP@0.5:0.95 in the NRSD dataset as 61.06%, which is improved by 2.79% compared with the original YOLOX network and 8.96% compared with YOLOv5-s in detection; detects mAP@0.5:0.95 in the PCB dataset as 51.58%, which is improved by 1.86% compared with the original YOLOX network and 6.39% compared with YOLOv5-s in detection; detects mAP@0.5:0.95 in the NEU-DET dataset as 49.08%, which is improved by 1.47% compared with the original YOLOX network and 11.61% compared with YOLOv5-s in detection.

[0108] Although the specific embodiments of the application have been described above with reference to the accompanying drawings, the description is not a limitation on the scope of protection of the application. Various modifications or variations made by those skilled in the art without creative labor on the basis of the technical solutions of the present application are still within the scope of protection of the present application.

Claims

1. An industrial defect detection method based on an improved YOLOX model, characterized in that, The industrial product is photographed through an industrial depth camera pipeline, high-resolution industrial pictures are preprocessed into standard pictures after being cut into standard sizes, and each pixel of each standard picture is normalized as input information into an industrial defect detection network based on an improved YOLOX model; The industrial defect detection network based on the improved YOLOX model comprises an attention-based backbone network, an attention-based multi-scale feature layer, a self-adaptive feature extraction and fusion network and a decoupling head network. The attention-based backbone network comprises adding an ECA channel attention mechanism at the output position of the last three layers of the backbone network of the original YOLOX network. The attention-based multi-scale feature layer and the self-adaptive feature extraction and fusion network comprise adding an ECA channel attention mechanism at the output position of the second Fpn layer of the top-down three Fpn layers of the PANet feature extraction network and the output positions of the three pan layers based on the feature extraction and fusion network of the original YOLOX network, and taking the feature maps of different scales of 20x20, 40x40 and 80x80 output by the attention module of the PANet feature extraction network as inputs, performing weighted fusion operation, and taking the calculated results as outputs in the scales of 20x20, 40x40 and 80x80, and respectively entering the respective decoupling head network. In the attention-based backbone network part, the input industrial pictures are subjected to five times of size reduction and twice convolution operation of channel expansion, to complete preliminary feature extraction and fusion of the industrial pictures, and finally the third, fourth and fifth industrial picture feature information of the backbone network part is taken as output; in the attention-based multi-scale feature layer and the self-adaptive feature extraction and fusion network, the three-layer output industrial picture feature information of the backbone network is taken as the input part, the input part is superimposed and fused with the three top-down Fpn layers and the three bottom-up Pan layers of the multi-scale feature layer, to complete deep adaptive feature extraction and fusion of the picture feature information and output; in the decoupling head network, the final feature information of the picture is subjected to matrix calculation, to finally obtain the detection frame coordinates, confidence and category of the picture; the obtained picture information is post-processed, the detection frame coordinates, confidence and category information are drawn to the original picture, and the processed picture after drawing is output.

2. The industrial defect detection method based on the improved YOLOX model according to claim 1, wherein, The SiLU activation function after the first 1x1 convolution of the bottleneck structure in the original YOLOX network is retained with a normalization function, and the normalization function after the second 3x3 convolution of the bottleneck structure in the original YOLOX network is retained with a SiLU activation function.

3. The industrial defect detection method based on the improved YOLOX model according to claim 1 or 2, wherein, The weighted fusion operation is completed through the adaptive spatial feature fusion module, and the operation method is: , , respectively represent the feature information of the three-layer attention mechanisms eca1, eca2, eca3 from the PANet feature extraction layer. The above feature information is multiplied by the weight parameters , , , added after being adjusted to the same size of feature map, to obtain a fusion layer , The formula is: In formula (1) , , respectively , , are weight parameters shared in the channel at position , , , are weight parameters defined by the softmax function, and are 1 and the range belongs to [0, 1], formula (4) is the calculation method of weight parameters , , , wherein , , are calculated by convolution , , , is a set of weight parameters , , , is a general term of the name of the calculated weight parameter , , .

4. A computer readable medium characterized by The computer program stored thereon is used to execute the method of any one of claims 1-3. The computer program stored thereon is used to execute the method of any one of claims 1-3.

Images