A small target industrial defect detection method and system based on deep learning
By improving the YOLOv8 network model, combining the improved downsampling algorithm and Wasserstein distance optimization task, aligning the learning mechanism and loss function, the problem of low detection accuracy of small objects is solved, and industrial defect detection with high accuracy and robustness is achieved.
Patent Information
- Application Number
- CN202410997931.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-07-24
AI Technical Summary
The existing industrial defect detection methods have low accuracy for small target detection, especially the YOLO series algorithms have problems such as insufficient feature extraction capabilities and inaccurate detection in small target industrial defect detection.
The improved YOLOv8 network model is adopted, combined with the improved downsampling algorithm and the normalized Wasserstein distance optimization task alignment learning mechanism and loss function, and the feature extraction and bounding box alignment metrics are optimized to improve detection accuracy.
Through improved feature extraction and loss functions, the accuracy and robustness of small-target industrial defect detection is significantly improved, information loss is reduced, and assessment accuracy and impartiality are ensured in special circumstances.
Smart Images

Figure CN118552523B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of industrial defect detection, and in particular to a small target industrial defect detection method and system based on deep learning. Background Art
[0002] Industrial defect detection is a key link in factory production. It is directly related to product quality and production efficiency in the manufacturing industry. Defect detection can ensure that products meet quality standards before leaving the factory and reduce the return rate and maintenance costs caused by quality problems. In early industrial production, defect detection methods mainly include manual detection, simple mechanical assisted detection, and the use of some basic electronic or optical instruments.
[0003] There are two main methods for industrial defect detection at present: one is based on traditional image processing, and the other is a machine learning defect detection method. Traditional image processing methods are mainly image pattern matching, but in industrial detection applications, traditional template matching methods are sensitive to image translation and illumination, and can only detect larger defects. When using machine learning methods for defect detection, they can be divided into traditional machine learning methods and deep learning methods. Compared with the former, the latter has the ability to automatically extract features and process complex patterns, so it has been more widely used. Among them, deep learning single-stage algorithms such as the YOLO series have shown advantages such as real-time, accuracy, easy integration, and high adaptability in the field of industrial defect detection, making it an ideal choice for solving the needs of high-speed and accurate defect detection.
[0004] Although the YOLO series of deep learning single-stage algorithms have achieved good results in industrial defect detection, the problem of low accuracy in detecting small-target industrial defects still needs to be improved. Because the features of small targets are usually weak, although CNN performs well in tasks such as image classification, it is not capable of extracting features from small targets. In addition, when the size of the small target in the image does not match the size of the detection frame, it will lead to inaccurate detection. For the above reasons, there is still a need for research on YOLO in the field of small-target industrial defect detection. Summary of the invention
[0005] In view of the above existing problems, the present invention is proposed.
[0006] Therefore, the present invention provides a small target industrial defect detection method and system based on deep learning to solve the problem of low detection accuracy of small target industrial defects in existing industrial defect detection methods.
[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0008] In a first aspect, the present invention provides a small target industrial defect detection method based on deep learning, comprising:
[0009] Collect industrial small target defect sample data and label the sample data;
[0010] Divide the labeled sample data into training set, validation set and test set;
[0011] The training set is input into the improved YOLOv8 network model for iterative training. When the improved YOLOv8 network model converges iteratively, the optimal model is obtained to achieve accurate detection of small target industrial defects. The improved YOLOv8 network model includes an information integration part that optimizes feature extraction using an improved downsampling algorithm and a task alignment learning mechanism and a loss function that are optimized through a normalized Wasserstein distance.
[0012] As a preferred solution of the small target industrial defect detection method based on deep learning described in the present invention, wherein:
[0013] The improved downsampling algorithm comprises the following steps:
[0014] Perform average pooling on the feature map with a pooling kernel size of 2×2, a stride of 1, and no padding;
[0015] The new feature map is evenly divided into two halves along the channel and input into the upper and lower branches for downsampling respectively;
[0016] The upper branch captures local features through 3×3 dimensional maximum pooling and adjusts feature dimensions in combination with a 1×1 convolutional layer;
[0017] The lower tributary uses two-dimensional wavelet transform to decompose the characteristic graph to obtain different frequency components;
[0018] The feature maps output by the upper and lower tributaries are concatenated along the channel dimension.
[0019] As a preferred solution of the small target industrial defect detection method based on deep learning described in the present invention, wherein:
[0020] The normalized Wasserstein distance comprises the following steps:
[0021] Assume that the bounding box information is N = (cx, cy, w, h), its center coordinates are (cx, cy), width is w, and height is h. Convert the information of each bounding box into a two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and covariance matrix.
[0022] For a given bounding box position x, calculate the Gaussian distribution probability density;
[0023] Select the Gaussian distribution of two bounding boxes Na and Nb, and use the normalized Wasserstein distance NWD to calculate the first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b )
[0024] After normalization, we get an index to measure the similarity between two targets.
[0025] As a preferred solution of the small target industrial defect detection method based on deep learning described in the present invention, wherein:
[0026] The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and the covariance matrix as:
[0027]
[0028] Among them, w 2 represents the square of the bounding box width, h 2 Represents the square of the bounding box height.
[0029] The calculated Gaussian distribution probability density is expressed as:
[0030]
[0031] Where exp(·) represents the natural exponential function, T represents the transpose operation of the vector, and ∑ -1 represents the inverse matrix of the covariance matrix, |∑| 1 / 2 Represents the square root of the determinant of the covariance matrix.
[0032] The first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b ) is expressed as:
[0033]
[0034] As a preferred solution of the small target industrial defect detection method based on deep learning described in the present invention, wherein:
[0035] The described task alignment learning mechanism is optimized by normalized Wasserstein distance, comprising the following steps:
[0036] Use the normalized Wasserstein distance to calculate the similarity between the predicted label and the true label to obtain the normalized Wasserstein distance value;
[0037] Based on the normalized Wasserstein distance value, all predicted labels are traversed, and the predicted labels whose center points fall within any true label are selected as the preliminary candidate set;
[0038] The candidate sets obtained from the initial screening are further screened, and the top 10 with the highest anchor point alignment metric values are selected as further candidate sets according to the anchor point alignment metric ranking;
[0039] Based on the further candidate set, the predicted label with the largest normalized Wasserstein distance value to the true label in the candidate set is retained as the final positive sample.
[0040] As a preferred solution of the small target industrial defect detection method based on deep learning described in the present invention, wherein:
[0041] The loss function is optimized by the normalized Wasserstein distance, including combining the complete intersection-over-union loss and the normalized Wasserstein distance in a weighted manner as a new complete intersection-over-union loss function, and using the new complete intersection-over-union loss function with the distribution focus loss and the binary cross entropy loss to weight the final loss function.
[0042] As a preferred solution of the small target industrial defect detection method based on deep learning described in the present invention, wherein:
[0043] The complete intersection-over-union loss function is expressed as:
[0044]
[0045] Among them, IoU is the intersection over union ratio of the predicted box and the real box of the label, ρ(b,b gt ) is the Euclidean distance between the center point of the positive sample and the true label, c is the diagonal length of the smallest envelope that can contain the positive sample and the true label, v is the difference in the aspect ratio between the positive sample and the true label, and α is the weight coefficient;
[0046] The new complete intersection-over-union loss function is expressed as:
[0047] CIonew =αCIoU+(1-α)NWD
[0048] Among them, α represents the weight ratio of CIoU, 0≤α≤1;
[0049] The distribution focus loss function is expressed as:
[0050] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))
[0051] Among them, y i and i+1 Represents two positive samples at adjacent positions of the true label y, S i and S i+1 is the probability prediction value of two positive samples at adjacent positions;
[0052] The binary cross entropy loss function is expressed as:
[0053]
[0054] Where c is the number of categories, p(c) is the true label, is the class probability of the positive sample;
[0055] The final loss function is expressed as:
[0056] Loss=Loss cls +Loss reg =0.5BCE+(7.5CIoU new +1.5DFL).
[0057] In a second aspect, the present invention provides a small target industrial defect detection system based on deep learning, comprising:
[0058] The labeling module is used to collect industrial small target defect sample data and label the sample data;
[0059] A partitioning module is used to divide the labeled sample data into a training set, a validation set, and a test set;
[0060] An improvement module is used to input the training set into an improved YOLOv8 network model for iterative training, and when the improved YOLOv8 network model converges iteratively, an optimal model is obtained to achieve accurate detection of small target industrial defects. The improved YOLOv8 network model includes an information integration part that optimizes feature extraction using an improved downsampling algorithm and a task alignment learning mechanism and a loss function that are optimized by a normalized Wasserstein distance;
[0061] The improved downsampling algorithm comprises the following steps:
[0062] The pooling module is used to perform average pooling on the feature map with a pooling kernel size of 2×2, a stride of 1, and no padding;
[0063] The downsampling module is used to evenly split the new feature map into two halves along the channel and input them into the upper and lower branches for downsampling respectively;
[0064] A feature dimension adjustment module, used for the upper tributary to capture local features through 3×3 dimensional maximum pooling, and to adjust the feature dimension in combination with a 1×1 convolution layer;
[0065] A decomposition module, used for applying two-dimensional wavelet transform to decompose the characteristic graph into different frequency components in the lower tributary;
[0066] A splicing module is used to splice the feature maps output by the upper and lower tributaries along the channel dimension;
[0067] The normalized Wasserstein distance comprises the following steps:
[0068] A setting module is used to set the bounding box information to N = (cx, cy, w, h), with the center coordinates (cx, cy), the width to w, and the height to h, and convert the information of each bounding box into a two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and the covariance matrix;
[0069] A calculation module, for calculating the Gaussian distribution probability density for a given bounding box position x;
[0070] Normalization module, used to select the Gaussian distribution of two bounding boxes Na and Nb, and use the normalized Wasserstein distance NWD to calculate the first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b)
[0071] The indicator module is used to obtain an indicator for measuring the similarity between two targets after normalization.
[0072] In a third aspect, the present invention provides a computing device, comprising:
[0073] Memory, used to store programs;
[0074] A processor is used to execute the computer executable instructions, which, when executed by the processor, implement the steps of the small target industrial defect detection method based on deep learning.
[0075] In a fourth aspect, the present invention provides a computer-readable storage medium, comprising: when the program is executed by a processor, the steps of implementing the small target industrial defect detection method based on deep learning are implemented.
[0076] Beneficial effects of the present invention: The present invention combines the dual-branch downsampling module of maximum pooling and Haar wavelet transform, which not only retains local details but also emphasizes the global structure, thus reducing information loss. In addition, the new evaluation method NWD introduces scale invariance to ensure the accuracy and fairness of the evaluation, has a smooth response to position deviation, and can effectively measure the similarity of bounding boxes in special cases, so that it can show high accuracy and robustness in the detection of small target industrial defects. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:
[0078] Figure 1 A basic flow chart of a small target industrial defect detection method based on deep learning provided by one embodiment of the present invention;
[0079] Figure 2 A complete flow chart of a small target industrial defect detection method based on deep learning provided by one embodiment of the present invention;
[0080] Figure 3 A flow chart of a loss function calculation method for a small target industrial defect detection method based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0081] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0082] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0083] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0084] Example 1
[0085] Reference Figure 1-Figure 3 , is an embodiment of the present invention, and provides a small target industrial defect detection method based on deep learning, such as Figure 1 , Figure 2 As shown, the following steps are included:
[0086] S1: Collect industrial small target defect sample data and label the sample data;
[0087] S2: Divide the labeled sample data into training set, validation set and test set;
[0088] S3: Inputting the training set into the improved YOLOv8 network model for iterative training, and obtaining the optimal model when the improved YOLOv8 network model converges iteratively, thereby realizing accurate detection of small target industrial defects. The improved YOLOv8 network model includes an information integration part for optimizing feature extraction using an improved downsampling algorithm and a task alignment learning mechanism and a loss function for optimizing the normalized Wasserstein distance.
[0089] Furthermore, the improved YOLOv8 network includes: optimizing the information integration part of feature extraction and improving the loss function calculation method. The information integration part of feature extraction optimization is used to improve the problem of feature map with excessive detail information and insufficient global structure information, aiming to enrich the receptive field while retaining more details. The improved loss function calculation method is used to address the problem of insufficient scale sensitivity and poor effect in handling non-overlapping bounding boxes when the IoU metric method is used to detect small targets.
[0090] Furthermore, the information integration part of feature extraction is optimized to enhance information perception. Specifically, the feature extraction link in the YOLOv8 model training process uses FPN and PAN structures, and replaces the downsampling module at the connection of FPN and PAN structures. A new downsampling module is used to optimize the information integration part of feature extraction, that is, a dual-branch downsampling module that combines maximum pooling and Haar wavelet transform. This module first performs 2×2 average pooling with a step size of 1 and no padding on the feature map, aiming to reduce noise, maintain features and reduce the amount of calculation. Then the output is evenly divided into two halves along the channel, and input into the upper and lower tributaries for downsampling, aiming to reduce the size of the feature map to half of the original size while increasing the diversity of sampling and effectively retaining information. Finally, the outputs of the two tributaries are spliced along the channel dimension.
[0091] Furthermore, the upper stream consists of maximum pooling and convolution, where the pooling kernel size of the maximum pooling is 3×3, the stride is 2, the padding is 1, and the convolution is 1×1. The maximum pooling layer extracts the most significant features in the local area of the feature map, such as edges and textures, which can reduce the computational complexity while maintaining the key information of the features. The 1×1 convolution layer processes the feature map to adjust the channel dimension.
[0092] Furthermore, the lower stream includes two parts: a lossless feature encoding block and a feature representation learning block.
[0093] Furthermore, the lossless feature coding block is responsible for feature conversion and reducing the size of feature maps. This part uses the two-dimensional Haar wavelet transform as its core technology, where the basis function of the Haar wavelet includes a wavelet basis function φ(x) for generating details and a scale function ψ(x) for generating approximations. Based on the above one-dimensional function, the two-dimensional Haar wavelet basis function ψ(x, y) and the scale function φ(x, y) are obtained:
[0094]
[0095] If the original signal is represented as f(x,y), then after the Haar wavelet transform, four components (A, H, V, D) are output:
[0096] A=f(x,y)·φ(x,y)
[0097] H=f(x,y)·ψ(x,y)
[0098] V=f(x,y)·ψ(x,y) T
[0099]
[0100] A represents the overall trend or low-frequency part of the signal; H, V, and D represent the high-frequency details of the signal in the horizontal, vertical, and diagonal directions, respectively. This is equivalent to quadrupling the number of channels after feature conversion, but reducing the size of the feature map of each channel to half of the original size.
[0101] Furthermore, the feature representation learning block consists of a 1×1 convolution, a batch normalization layer BN, and an activation layer ReLU, which are used to adjust the number of channels and enhance feature expression.
[0102] Furthermore, if Figure 3 As shown in the figure, the loss function calculation method is improved by introducing a new measurement method NWD of normalized Wasserstein distance to improve the loss function calculation method of YOLOv8.
[0103] Furthermore, the calculation method of NWD is as follows: the bounding box is modeled as a 2D Gaussian distribution, and the normalized Wasserstein distance is introduced to calculate the similarity between two targets. Assuming that the bounding box information is, its center coordinates are (cx, cy), width is w, and height is h, it can be modeled as a two-dimensional Gaussian distribution. The mean vector μ and covariance matrix Σ of the Gaussian distribution are expressed as:
[0104]
[0105] Among them, w 2 represents the square of the bounding box width, h 2 Represents the square of the bounding box height.
[0106] The specific two-dimensional Gaussian distribution is calculated as follows, where x = (x, y):
[0107]
[0108] Where exp(·) represents the natural exponential function, T represents the transpose operation of the vector, and ∑ -1 represents the inverse matrix of the covariance matrix, |∑| 1 / 2 Represents the square root of the determinant of the covariance matrix.
[0109] For the first bounding box N a =(cx a ,cy a ,wa ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b ) and Second-order distance Wasserstein distance can be simplified to:
[0110]
[0111] The normalized exponent is used to obtain a new metric to measure the similarity between two targets, where the constant c is set to 12.8:
[0112]
[0113] Compared with IoU, the new evaluation method NWD introduces scale invariance to ensure the accuracy and fairness of the evaluation, has a smooth response to position deviation, and can effectively measure the similarity of bounding boxes in special cases, enabling it to show high accuracy and robustness in the detection of small target industrial defects.
[0114] Furthermore, the task alignment learning mechanism (TAL) filters positive samples for the predicted labels output by the network prediction. Positive samples refer to those samples that are correctly identified and labeled as containing the target object. TAL has four main steps: preprocessing, initial screening, fine screening, and removal of duplicate assignments. In the preprocessing stage, the NWD method is used to calculate the similarity between the predicted label and the true label to obtain the NWD value, and then based on the NWD value, the anchor alignment metric align_metric is calculated for each predicted label. The calculation method is as follows:
[0115] align_metric=s α ×u β
[0116] Among them, α and β are weight hyperparameters, which are 0.5 and 6.0 respectively, s is the category score of the predicted label, and u is the NWD value of the predicted label and the true label.
[0117] The initial screening stage will traverse all predicted labels and select the predicted labels whose center points fall within any true label as the preliminary candidate set.
[0118] In the fine screening stage, the candidate sets obtained from the initial screening are further screened, and the top 10 with the highest align_metric values are selected as further candidate sets based on the align_metric ranking.
[0119] The last step is the removal. In order to avoid duplicate allocation, only the predicted label with the largest NWD value with the true label in the candidate set is retained as the final positive sample.
[0120] Furthermore, the loss function consists of classification loss and regression loss, both of which are calculated using the true labels and the positive samples obtained based on TAL. The classification loss consists of binary cross entropy loss (BCE), while the regression function consists of complete intersection over union loss (CIoU) and distribution focus loss (DFL).
[0121] BCE loss calculates the difference in class probabilities, and the formula is as follows:
[0122]
[0123] Where C is the number of categories, p(c) is the true label, and for category c, if p(c) is 1, it means it is the target category, and if it is 0, it means it is not the target category. is the class probability of the positive sample.
[0124] Furthermore, the formula of CIoU is expressed as:
[0125]
[0126] Among them, IoU is the intersection over union ratio of the predicted box and the real box of the label, ρ(b,b gt ) is the Euclidean distance between the center point of the positive sample and the true label, c is the diagonal length of the smallest envelope that can contain the positive sample and the true label, v measures the difference in the aspect ratio of the positive sample and the true label, and α is the weight coefficient.
[0127] Furthermore, CIoU and NWD are combined in a weighted manner as a new CIoU loss function to optimize the overall loss function. The weighted formula is as follows, where α represents the weight of CIoU, and α = 0.2:
[0128] CIo new =αCIoU+(1-α)NWD
[0129] Furthermore, DFL discretizes the continuous target position into a set of fixed equally spaced discrete positions, which are represented as integer indices in the output space of the model. The DFL loss formula is expressed as:
[0130] DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))
[0131] Among them, y i and i+1 Represents two positive samples at adjacent positions of the true label y. The specific value is calculated as y i = the distance from the center to a certain edge / the current downsampling multiple, S i and S i+1 is the probability prediction value of two positive samples at adjacent positions.
[0132] Use optimized CIoU new The final loss function is weighted with DFL and BCE:
[0133] Loss=Loss cls +Loss reg =0.5BCE+(7.5CIoU new +1.5DFL)
[0134] This embodiment also provides a small target industrial defect detection system based on deep learning, including:
[0135] The labeling module is used to collect industrial small target defect sample data and label the sample data;
[0136] A partitioning module is used to divide the labeled sample data into a training set, a validation set, and a test set;
[0137] An improvement module is used to input the training set into an improved YOLOv8 network model for iterative training, and when the improved YOLOv8 network model converges iteratively, an optimal model is obtained to achieve accurate detection of small target industrial defects. The improved YOLOv8 network model includes an information integration part that optimizes feature extraction using an improved downsampling algorithm and a task alignment learning mechanism and a loss function that are optimized by a normalized Wasserstein distance;
[0138] The improved downsampling algorithm comprises the following steps:
[0139] The pooling module is used to perform average pooling on the feature map with a pooling kernel size of 2×2, a stride of 1, and no padding;
[0140] The downsampling module is used to evenly split the new feature map into two halves along the channel and input them into the upper and lower branches for downsampling respectively;
[0141] A feature dimension adjustment module, used for the upper tributary to capture local features through 3×3 dimensional maximum pooling, and to adjust the feature dimension in combination with a 1×1 convolution layer;
[0142] A decomposition module, used for applying two-dimensional wavelet transform to decompose the characteristic graph into different frequency components in the lower tributary;
[0143] A splicing module is used to splice the feature maps output by the upper and lower tributaries along the channel dimension;
[0144] The normalized Wasserstein distance comprises the following steps:
[0145] A setting module is used to set the bounding box information to N = (cx, cy, w, h), with the center coordinates (cx, cy), the width to w, and the height to h, and convert the information of each bounding box into a two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and the covariance matrix;
[0146] A calculation module, for calculating the Gaussian distribution probability density for a given bounding box position x;
[0147] Normalization module, used to select the Gaussian distribution of two bounding boxes Na and Nb, and use the normalized Wasserstein distance NWD to calculate the first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b )
[0148] The indicator module is used to obtain an indicator for measuring the similarity between two targets after normalization.
[0149] Furthermore, it also includes:
[0150] Memory, used to store programs;
[0151] A processor is used to load the program to execute the small target industrial defect detection method based on deep learning.
[0152] This embodiment also provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the small target industrial defect detection method based on deep learning.
[0153] The storage medium proposed in this embodiment and the small target industrial defect detection method based on deep learning proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.
[0154] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ReadOnly, Memory, ROM), random access memory (RandomAccess Memory, RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform the methods of various embodiments of the present invention.
[0155] Example 2
[0156] Referring to Tables 1 and 2, an embodiment of the present invention provides a small target industrial defect detection method based on deep learning. In order to verify its beneficial effects, scientific demonstration is carried out through specific implementation methods and implementation effects.
[0157] The present embodiment is as follows:
[0158] Experimental environment:
[0159] Linux Debian 12LTS operating system, Python 3.11, Pytorch 2.2, Torchvision 0.17, Intel(R) Xeon(R) CPU E5-2630 v3@2.40GHz, and Tesla T4 GPU with 16GB video memory. CUDA version 12.0, CUDNN version 11.7; model training parameters include: initial learning rate 0.01, momentum set to 0.937, weight decay set to 0.0005, training threshold 0.2, all images normalized to 640×640, iterations 200, and batch size 16.
[0160] Taking the public HRIPCB defect dataset as an example, the dataset has been annotated, so the annotation step is omitted in the following steps.
[0161] The specific steps are as follows:
[0162] S1. Expand the data set. Expand the original data set from 693 to 7930 and divide it into training set, validation set and test set according to the ratio of 7:1:2.
[0163] S2. Input the training set into the improved YOLOv8 network for iterative training.
[0164] S3. Optimize the information integration part of feature extraction to enhance information perception.
[0165] Specifically, the feature extraction link in the YOLOv8 model training process uses FPN and PAN structures, replaces the downsampling module at the connection of FPN and PAN structures, and uses a new downsampling module to optimize the information integration part of feature extraction, that is, a dual-branch downsampling module that combines maximum pooling and Haar wavelet transform. This module first performs 2×2 average pooling with a step size of 1 and no padding on the feature map, aiming to reduce noise, maintain features and reduce the amount of calculation. Then the output is evenly divided into two halves along the channel and input into the upper and lower tributaries for downsampling, aiming to reduce the size of the feature map to half of the original size while increasing the diversity of sampling and effectively retaining information. Finally, the outputs of the two tributaries are spliced along the channel dimension.
[0166] S4. The improved loss function calculation method introduces a new measurement method NWD based on normalized Wasserstein distance to improve the loss function calculation method of YOLOv8.
[0167] S5. Put the data set into the improved yolov8 network structure for training, and the model is iteratively converged to obtain the best model.
[0168] The above steps were verified using the PCB dataset and the YOLOv8 network model. This study performed additional noise enhancement on the original test set to obtain a new test set, and then conducted a detection comparison experiment on the new test set. It can be observed that the improved model has achieved improvements in reducing missed detections and false detections, showing stronger robustness. The model was subjected to an ablation experiment, and the results are shown in Tables 1 and 2.
[0169] Table 1 Ablation experiment data table
[0170]
[0171] Table 2 Comparison of accuracy of each category before and after improvement
[0172]
[0173] The experimental results show that the performance of the improved model on the four evaluation indicators has improved by 2.3%, 2.4%, 2.0% and 7.6%, and the improvement of the accuracy of each category is balanced. This result proves the effectiveness of the model improvement measures in improving detection accuracy and robustness.
[0174] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A small target industrial defect detection method based on deep learning, It is characterized in that include: Collect industrial small target defect sample data and label the sample data; Divide the labeled sample data into training set, validation set and test set; Inputting the training set into the improved YOLOv8 network model for iterative training, obtaining an optimal model when the improved YOLOv8 network model converges iteratively, and realizing accurate detection of small target industrial defects, wherein the improved YOLOv8 network model includes optimizing the information integration part of feature extraction by using an improved downsampling algorithm and optimizing the task alignment learning mechanism and loss function by using a normalized Wasserstein distance; The improved downsampling algorithm comprises the following steps: Perform average pooling on the feature map with a pooling kernel size of 2×2, a stride of 1, and no padding; The new feature map is evenly divided into two halves along the channel and input into the upper and lower branches for downsampling respectively; The upper branch captures local features through 3×3 dimensional maximum pooling and adjusts feature dimensions in combination with a 1×1 convolutional layer; The lower tributary uses two-dimensional wavelet transform to decompose the characteristic graph to obtain different frequency components; Concatenate the feature maps output by the upper and lower tributaries along the channel dimension; The normalized Wasserstein distance comprises the following steps: Assume that the bounding box information is N = (cx, cy, w, h), its center coordinates are (cx, cy), width is w, and height is h. Convert the information of each bounding box into a two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and covariance matrix. For a given bounding box position x, calculate the Gaussian distribution probability density; Select the Gaussian distribution of two bounding boxes Na and Nb, and use the normalized Wasserstein distance NWD to calculate the first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b )'s Gaussian distribution distance After normalization, we get an index to measure the similarity between two targets.
2. The small target industrial defect detection method based on deep learning as claimed in claim 1, Features: The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and the covariance matrix as: Among them, w 2 represents the square of the bounding box width, h 2 Represents the square of the bounding box height; The calculated Gaussian distribution probability density is expressed as: Where exp(·) represents the natural exponential function, T represents the transpose operation of the vector, and ∑ -1 represents the inverse matrix of the covariance matrix, |∑| 1 / 2 represents the square root of the determinant of the covariance matrix; The first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b ) is expressed as:
3. The small target industrial defect detection method based on deep learning as claimed in claim 2, Features: The described task alignment learning mechanism is optimized by normalized Wasserstein distance, comprising the following steps: Use the normalized Wasserstein distance to calculate the similarity between the predicted label and the true label to obtain the normalized Wasserstein distance value; Based on the normalized Wasserstein distance value, all predicted labels are traversed, and the predicted labels whose center points fall within any true label are selected as the preliminary candidate set; The candidate sets obtained from the initial screening are further screened, and the top 10 with the highest anchor point alignment metric values are selected as further candidate sets according to the anchor point alignment metric ranking; Based on the further candidate set, the predicted label with the largest normalized Wasserstein distance value to the true label in the candidate set is retained as the final positive sample.
4. The small target industrial defect detection method based on deep learning as claimed in claim 3, Features: The loss function is optimized by the normalized Wasserstein distance, including combining the complete intersection-over-union loss and the normalized Wasserstein distance in a weighted manner as a new complete intersection-over-union loss function, and using the new complete intersection-over-union loss function with the distribution focus loss and the binary cross entropy loss to weight the final loss function.
5. The small target industrial defect detection method based on deep learning as claimed in claim 4, Features: The complete intersection-over-union loss function is expressed as: Among them, IoU is the intersection over union ratio of the predicted box and the real box of the label, ρ(b,b gt ) is the Euclidean distance between the center point of the positive sample and the true label, c is the diagonal length of the smallest envelope that can contain the positive sample and the true label, v is the difference in the aspect ratio between the positive sample and the true label, and α is the weight coefficient; The new complete intersection-over-union loss function is expressed as: CIoU new =αCIoU+(1-α)NWD Among them, α represents the weight ratio of CIoU, 0≤α≤1; The distribution focus loss function is expressed as: DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(y-y i )log(S i+1 )) Among them, y i and i+1 Represents two positive samples at adjacent positions of the true label y, S i and S i+1 is the probability prediction value of two positive samples at adjacent positions; The binary cross entropy loss function is expressed as: Where c is the number of categories, p(c) is the true label, is the class probability of the positive sample; The final loss function is expressed as: Loss=Loss cls +Loss reg =0.5BCE+(7.5CIoU new +1.5DFL)。 6. A system based on the small target industrial defect detection method based on deep learning as claimed in claim 1, Features: The labeling module is used to collect industrial small target defect sample data and label the sample data; A partitioning module is used to divide the labeled sample data into a training set, a validation set, and a test set; An improvement module is used to input the training set into an improved YOLOv8 network model for iterative training, and when the improved YOLOv8 network model converges iteratively, an optimal model is obtained to achieve accurate detection of small target industrial defects. The improved YOLOv8 network model includes an information integration part that optimizes feature extraction using an improved downsampling algorithm and a task alignment learning mechanism and a loss function that are optimized by a normalized Wasserstein distance; The improved downsampling algorithm comprises the following steps: The pooling module is used to perform average pooling on the feature map with a pooling kernel size of 2×2, a stride of 1, and no padding; The downsampling module is used to evenly split the new feature map into two halves along the channel and input them into the upper and lower branches for downsampling respectively; A feature dimension adjustment module, used for the upper tributary to capture local features through 3×3 dimensional maximum pooling, and to adjust the feature dimension in combination with a 1×1 convolution layer; A decomposition module, used for applying two-dimensional wavelet transform to decompose the characteristic graph into different frequency components in the lower tributary; A splicing module is used to splice the feature maps output by the upper and lower tributaries along the channel dimension; The normalized Wasserstein distance comprises the following steps: A setting module is used to set the bounding box information to N = (cx, cy, w, h), with the center coordinates (cx, cy), the width to w, and the height to h, and convert the information of each bounding box into a two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution corresponding to the bounding box is defined by the mean vector and the covariance matrix; A calculation module, for calculating the Gaussian distribution probability density for a given bounding box position x; Normalization module, used to select the Gaussian distribution of two bounding boxes Na and Nb, and use the normalized Wasserstein distance NWD to calculate the first bounding box N a =(cx a ,cy a ,w a ,h a ) and the second bounding box N b =(cx b ,cy b ,w b ,h b )'s Gaussian distribution distance The indicator module is used to obtain an indicator for measuring the similarity between two targets after normalization.
7. An electronic device, It is characterized in that include: Memory, used to store programs; A processor, used to load the program to execute the steps of the small target industrial defect detection method based on deep learning as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing a program, It is characterized in that When the program is executed by a processor, the steps of the small target industrial defect detection method based on deep learning as described in any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Steel wire surface anomaly detection method, device and equipment and storage medium
CN117437200A
