A glass defect detection method based on improved YOLOv5

Abstract

The application discloses a glass defect detection method based on an improved YOLOv5, steps: S1: a high dynamic range industrial camera is used to collect multispectral image data of a glass product production line, after data processing, a glass defect detection data set is constructed, and the data set is divided; S2: a dimension decoupling feature extraction module DDFM is introduced based on the YOLOv5 model, a corner point block attention module CoPA is added, and the DDFM and the CoPA are fused to replace the original C3 module, and a DDC-YOLO model is obtained; S3: the constructed DDC-YOLO model is trained and verified by using the data set, and the DDC-YOLO model is used for real-time detection of transparent material defects. Through the reconstruction of the network architecture of YOLOv5 and the particularity of glass defect detection, the perception ability of the model to low-contrast features, multi-scale defects and small targets is strengthened, and the detection accuracy and real-time requirement are balanced.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and specifically to a glass defect detection method based on an improved YOLOv5. Background Technology

[0002] Glass, as an important industrial material, is widely used in construction, automobiles, electronic displays, and home furnishings. Due to its transparency, weather resistance, and aesthetic appeal, glass products require extremely high quality. However, during the glass production process, various defects inevitably occur on the glass surface or inside, such as bubbles, inclusions, impurities, scratches, cracks, and optical distortions, influenced by factors such as raw materials, production processes, and the environment. These defects not only affect the appearance and performance of the glass but may also reduce its mechanical strength and safety performance, leading to product defects or even safety accidents. Therefore, achieving efficient and accurate defect detection during glass production is crucial.

[0003] Traditional glass defect inspection mainly relies on manual visual inspection or simple optical instruments. Manual inspection is usually carried out by experienced workers using strong light or light sources at specific angles. However, this method is inefficient, labor-intensive, and susceptible to subjective factors, making it difficult to meet the high-speed and high-precision requirements of modern production lines. While optical or laser-based instrument inspection (such as transmission, scattering, and interferometry) has improved inspection efficiency to some extent, it still has limitations in detecting minute defects, complex backgrounds, or high-transmittance glass, and the equipment is also expensive and lacks adaptability.

[0004] In recent years, with the rapid development of computer vision and deep learning technologies, machine learning-based defect detection methods have gradually become a research hotspot. In particular, object detection algorithms have shown great potential in the field of industrial defect detection. Among them, the YOLO (You Only Look Once) series of algorithms has attracted much attention due to its speed and high accuracy. YOLOv5, as a classic version of the YOLO family, performs excellently in general object detection tasks, but it still faces many challenges in the specific scenario of glass defect detection.

[0005] 1) Diversity and complexity of glass defects: Glass defects vary greatly in shape, size and distribution, such as point bubbles, linear scratches, and planar optical deformation. The traditional YOLOv5 design may be difficult to adapt to the detection needs of multi-scale defects.

[0006] 2) High light transmittance and low contrast: The high light transmittance of glass results in low contrast between defects and the background. In particular, minute defects may only appear as slight grayscale changes in imaging, which places higher demands on the feature extraction capabilities of the algorithm.

[0007] 3) Difficulty in detecting small targets: Many glass defects (such as microcracks or microbubbles) have a very small pixel ratio. The YOLOv5 network structure is prone to losing feature information of small targets during downsampling, resulting in missed detections or false detections.

[0008] 4) Industrial real-time requirements: Glass production lines typically require high-speed continuous inspection, and ordinary inspection algorithms may be unable to meet real-time requirements while ensuring accuracy.

[0009] To address the above problems, this invention proposes a glass defect detection method based on an improved YOLOv5. Summary of the Invention

[0010] The purpose of this invention is to provide a glass defect target detection method based on an improved YOLOv5, which can effectively improve the detection accuracy and speed of glass defects.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: This glass defect detection method based on the improved YOLOv5 includes the following steps:

[0012] S1: High dynamic range industrial cameras are used to acquire multispectral image data of glass product production lines. After data processing, a glass defect detection dataset is constructed; then the dataset is divided.

[0013] S2: Based on the YOLOv5 model, the dimension decoupling feature extraction module DDFM is introduced, the corner block attention module CoPA is added, and the dimension decoupling feature extraction module DDFM and the corner block attention module CoPA are fused to replace the original C3 module to improve the three-stage network architecture, resulting in the DDC-YOLO model.

[0014] S3: The DDC-YOLO model was trained and validated using a glass defect dataset, and then the DDC-YOLO model was used to detect defects in transparent materials in real time.

[0015] By adopting the above technical solution, the YOLOv5 network architecture is reconstructed, and combined with the special characteristics of glass defect detection, the model's ability to perceive low-contrast features, multi-scale defects, and small targets is enhanced, while balancing detection accuracy and real-time requirements. The introduced Dimension Decoupling Feature Extraction (DDFM) module decouples dimensions progressively from three-dimensional to two-dimensional to one-dimensional. Each branch gradually shifts the focus of feature extraction to a specified dimension, reducing information loss caused by dimensional changes. Finally, the output feature maps of the three branches are fused as the output, which belongs to the feature extraction module. This invention addresses the high light transmittance of glass, the diversity of defects, and the complexity of industrial scenarios by introducing a multi-angle feature extraction and fusion mechanism and a lightweight attention module. It constructs a highly efficient detection framework adapted to the characteristics of glass defects, enabling high-precision, low-latency defect identification in high-speed production lines, meeting the stringent requirements of industrial-grade glass product quality inspection.

[0016] Preferably, in step S1, image enhancement processing is performed using a dynamic thresholding and reflection suppression algorithm to construct a glass defect detection dataset containing bubbles, scratches, and impurities; the specific steps are as follows:

[0017] S11 Data Acquisition: Industrial cameras are deployed on the glass production line to cover key processes including cutting, grinding, and tempering, acquiring high-resolution images (≥4K) and short-time videos. The data includes common defects (bubbles, scratches, cracks, impurities) and covers different lighting conditions (strong light / weak light / reflective light), glass thickness (2-12mm), and surface condition (transparent / frosted / coated). Simultaneously, the production line status is recorded, such as normal production, equipment restart after maintenance, or material batch change, ensuring data diversity.

[0018] S12 Data Preprocessing: Blurred or invalid images are removed, and the retained qualified data are enhanced using dynamic thresholding and reflection suppression algorithms. First, polarization decomposition is used to eliminate reflections on the glass surface: diffuse reflection components are calculated and specular reflections are weakened to reduce ambient light interference and improve the contrast of defect areas. Then, the defect boundary threshold (mean + 0.5 standard deviation) is dynamically calculated, and morphological closing operations are fused to enhance defect continuity. Finally, the segmentation results are weighted and fused with the original image (enhancement factor α = 0.3) to enhance defect contrast and make small defects more prominent. Then, the enhanced dataset is labeled, and the defect locations are marked with rectangular boxes. Defects are classified into four categories: bubbles, scratches, cracks, and impurities. A small number of difficult samples (such as small scratches or semi-transparent cracks) are retained to improve the robustness of the model.

[0019] S13 Dataset Construction: A stratified sampling strategy was adopted to ensure a balanced distribution of different glass types (flat / curved) and defect sizes (<1mm / 1-5mm / >5mm) to obtain the dataset. The dataset was then divided into training, validation, and test sets in a 7:2:1 ratio. The test set must include data for all production line equipment models to verify generalization. To facilitate iterative optimization, data subsets were established according to production quarters, and basic metadata (such as ambient temperature and humidity) was recorded.

[0020] Preferably, the improvement of the three-stage network architecture in step S2 to construct the DDC-YOLO model specifically includes:

[0021] S21: Introduce the Dimensionally Decoupled Feature Extraction Module (DDFM) into the feature extraction network to perform independent feature modeling for height, width, and channel dimensions;

[0022] S22: Add the corner point segmentation attention module CoPA to the head network to generate a spatial weight map based on the statistical comparison of the four corner regions;

[0023] S23: The dual-correlation collaborative module DC-Block is constructed by fusing the dimensionally decoupled feature extraction module DDFM and the corner block attention module CoPA, replacing the original C3 module and reconstructing the feature pyramid network.

[0024] The above technical solution introduces a Dimension-Decomposed Feature Module (DDFM), which strengthens the cross-dimensional correlation of glass texture by independently decoupling the feature representations of height, width, and channel dimensions. Instead of replacing traditional convolutions within the model with depthwise separable convolutions, the module uses three variants of convolution—depthwise convolution, strip convolution, and point convolution—as components of the DDFM module. Unlike the existing MCA module, which also employs a three-branch architecture but is an attention mechanism, the DDFM module in this solution processes feature maps differently in terms of both method and purpose. MCA directly uses pooling to compress dimensions from three dimensions to one dimension, then calculates weights using convolution and applies them to the initial input. A Corner-referenced Patch Attention (CoPA) module is designed to generate spatial attention weights based on statistical comparisons of the four corner regions, accurately enhancing the feature response of defects. The YOLO framework is reconstructed by fusing DDFM and CoPA, replacing the original C3 module with a fusion module to achieve decoupling enhancement and dynamic fusion of multi-scale defect features. This method enables real-time and efficient detection of defects on glass surfaces, improving detection accuracy and speed.

[0025] Preferably, the specific steps of the Dimension Decoupling Feature Extraction Module (DDFM) in step S21 to achieve dimensional decoupling are as follows:

[0026] S211: The input feature map is rotated sequentially along the height, width, and channel axes to obtain three sets of feature maps;

[0027] S212: Use depthwise convolution to extract features from the three sets of feature maps obtained in step S211, and extract three two-dimensional correlation features respectively, thus obtaining three sets of feature maps with two-dimensional correlation features.

[0028] S213: Then combine the feature maps obtained in step S213 in pairs and splice them together according to their intersection feature dimensions;

[0029] S214: Finally, use strip convolution and point convolution to compress the features of the corresponding dimension, remove redundant features, and obtain three sets of compressed feature maps.

[0030] S215: Then, superimpose the three sets of compressed feature maps, take their average value, and output the result.

[0031] Preferably, the specific steps for generating spatial attention weights by the corner point block attention module CoPA in step S22 are as follows:

[0032] S221 Gridded Feature Analysis: The input feature map is divided into n_win×n_win regular grid blocks, and the mean μ and standard deviation σ of each grid block are calculated.

[0033] S222 Corner Reference Difference Calculation: Select the top-left, top-right, bottom-left, and bottom-right corner blocks as reference points. Scan each grid block and calculate the statistical difference between the current grid block and the four corner grid blocks respectively; the formula is:

[0034] Δμ=Σ|μ_current-μ_corner_i|;

[0035] Δσ=Σ|σ_current-σ_corner_i|(i=1~4);

[0036] Where μ_current represents the mean of the current network block scanned when scanning each network block, σ_current is the standard deviation of the current network block; μ_corner_i represents the mean of the corner block, σ_corner_i represents the standard deviation of the corner block, and i is the corner index; Δμ represents the difference between the mean of the grid block and the corner block; Δσ represents the difference between the standard deviation of the grid block and the corner block.

[0037] S223 Dynamic Weight Generation: The difference values ​​are mapped to weight coefficients through the Sigmoid activation function to obtain the block-level weights W, as shown in the formula:

[0038] W=Sigmoid(Δμ)+Sigmoid(Δσ)-0.5;

[0039] S224 Spatial Attention Enhancement: The block-level weights W are then reconstructed into pixel-level spatial weight maps, and multiplied point by point with the original feature map, that is, the weight map is added to the input feature map.

[0040] Preferably, in step S23, the dual-related collaborative module DC-Block includes a dual-branch collaborative structure, namely a first branch and a second branch; wherein,

[0041] The first branch performs dimensional decoupling feature extraction through the Dimension Decoupling Feature Extraction Module (DDFM), focusing on capturing the cross-dimensional correlation characteristics of the glass texture;

[0042] The second branch generates a spatial attention mask through the corner block attention module CoPA to enhance the feature response of edge cracks and minor defects;

[0043] The outputs of the first and second branches are cross-fused through channels, and then feature reshaping is performed through 1×1 convolution to ensure that the output dimension is consistent with the input.

[0044] Preferably, the specific steps of step S3 are as follows:

[0045] S31: The DDC-YOLO model is trained using the training set, validated using the validation set, and tested using the test set to assess its detection capabilities.

[0046] S32: Deploy the trained DDC-YOLO model to the production line for detection and output the detection results in real time.

[0047] Preferably, the specific steps of step S31 are as follows:

[0048] S311 Training Model: The training set is input into DDC-YOLO, which is an improvement on YOLO, for training, and goes through a warm-up phase, a main training phase, and a fine-tuning phase.

[0049] In the warm-up phase, a linear learning rate is used (the learning rate lr is increased from 1e-5 to 1e-2), some layers of the backbone network are frozen, and only the detection head module is trained; then the CIoU loss function is used to initialize the weights to avoid early gradient instability.

[0050] During the main training phase (50-450 epochs), all network layers are unfrozen, and cosine annealing learning rate scheduling is enabled (base_lr = 1e-2, min_lr = 1e-4). A dynamic positive sample allocation strategy is introduced, which dynamically adjusts the ground truth allocation threshold based on the matching degree between the predicted bounding box and the ground truth bounding box.

[0051] During the fine-tuning phase (450-500 epochs), cross-GPU synchronous batch normalization (SyncBN) is enabled, and exponential moving average (EMA) is used to update model parameters.

[0052] S312 Validation Model: Configure the loss function and use the validation set to validate the trained DDC-YOLO model to evaluate its performance;

[0053] The S313 test model sets the input image to its original size, padding edges if the size is less than a multiple of 32. Then, an improved Non-Maximum Suppression (NMS) algorithm is used, with a class-independent IoU threshold (0.7) and a confidence threshold (initial threshold of 0.001, adaptively adjusted with target density) for filtering, reducing redundant bounding boxes in target detection and improving detection accuracy.

[0054] Preferably, the loss function configured in step S312 includes three parts: localization loss, classification loss, and confidence loss.

[0055] The localization loss, or bounding box regression loss, is quantified using SIoU and incorporates angle and distance cost terms to optimize the bounding box regression direction; the formula is:

[0056] L box =1-IoU+(Δ+Ω) / 2;

[0057] Among them, L box For bounding box regression loss, IoU represents the intersection-union ratio of the predicted box and the ground truth box, Δ represents the corner cost term, and Ω represents the distance cost term;

[0058] The category classification loss is an improved Focal Loss function, which applies a modulation coefficient of γ to the hard examples; this is also an improved cross-entropy function, which reduces the weight of easily classified samples by adjusting the factor to solve the imbalance between positive and negative samples; the formula is:

[0059] L cls =-α t *(1-p t ) γ *log(p t );

[0060] Among them, L cls For category classification loss, αt p represents the class balance factor. t This represents the model's predicted probability of the true class, where γ is the modulation coefficient; preferably, γ = 2.0.

[0061] The confidence loss is achieved by adding a dynamic weighting factor to alleviate the imbalance between positive and negative samples in dense target scenarios; the formula is:

[0062]

[0063] Among them, L obj λ represents the confidence loss. pos λ represents the dynamic weight of positive samples. neg This represents the dynamic weight of negative samples, and y is the true label (1 represents the target, 0 represents the background). The confidence score for the model's prediction;

[0064] Total loss function L total For weighted summation, the formula is:

[0065] L total =0.05L box +0.5L cls +L obj .

[0066] Preferably, the specific steps for detection using the trained model in step S32 are as follows:

[0067] S321 Image Preprocessing: Adjusting brightness and contrast to reduce glare interference; automatically identifying glass edges and cropping the effective detection area;

[0068] S322 Defect Detection: First, a two-stage detection method is adopted, which is: first, the entire glass is quickly scanned, and then the suspicious areas are carefully inspected; automatically identifying four types of defects: bubbles, cracks, scratches, and impurities;

[0069] S323 Result Processing: Merge adjacent defect boxes of the same type, filter out minor noise (<5 pixels) and normal texture areas; record the defect location, type and size;

[0070] S324 system linkage: Real-time output of test results to display device, automatic marking of unqualified products and triggering of sorting device.

[0071] The entire process takes less than 200ms per piece, meeting the real-time inspection requirements of the production line. The inspection parameters can be flexibly adjusted according to different glass types (such as flat glass and curved glass).

[0072] Compared with the prior art, the present invention has the following characteristics and beneficial effects:

[0073] 1) The algorithm of this invention designs a dimension decoupling feature extraction module DDFM, which decouples the three-dimensional features of height, width and channel and extracts them separately. Then, it integrates the relevant features between each pair of dimensions and finally combines the features of each dimension to perform feature fusion. It gradually transitions from extracting local refined features to fusing overall features, which optimizes the feature extraction capability of the network and speeds up the convergence speed of the model.

[0074] 2) The algorithm of this invention involves a corner point block attention module (CoPA), which simulates the normal feature distribution using the mean and standard deviation, and calculates the inter-block correlation accordingly to locate the distribution of abnormal regions and assign corresponding weights, thereby enhancing the feature map and improving the model's detection accuracy. Simultaneously, this module achieves parameterless construction of the attention module, enhancing model performance without increasing the number of parameters. Attached Figure Description

[0075] Figure 1 The flowchart shows the glass defect detection method based on the improved YOLOv5 of the present invention.

[0076] Figure 2 This is a model diagram of DDC-YOLO in the glass defect detection method based on the improved YOLOv5 of the present invention;

[0077] Figure 3 This is a diagram of the DDFM module in the glass defect detection method based on the improved YOLOv5 of the present invention;

[0078] Figure 4 This is a block diagram of the CoPA module of the glass defect detection method based on the improved YOLOv5 of the present invention. Detailed Implementation

[0079] To illustrate the technical solutions disclosed in this invention in detail, further description is provided below with reference to the accompanying drawings.

[0080] Example: Figure 1 As shown, this glass defect detection method based on the improved YOLOv5 includes the following steps:

[0081] S1: High dynamic range industrial cameras are used to acquire multispectral image data of glass product production lines. After data processing, a glass defect detection dataset is constructed; then the dataset is divided.

[0082] In step S1, image enhancement processing is performed using dynamic thresholding and reflection suppression algorithms to construct a glass defect detection dataset containing bubbles, scratches, and impurities. The specific steps are as follows:

[0083] S11 Data Acquisition: Industrial cameras are deployed on the glass production line to cover key processes including cutting, grinding, and tempering, acquiring high-resolution images (≥4K) and short-time videos. The data includes common defects (bubbles, scratches, cracks, impurities) and covers different lighting conditions (strong light / weak light / reflective light), glass thickness (2-12mm), and surface condition (transparent / frosted / coated). Simultaneously, the production line status is recorded, such as normal production, equipment restart after maintenance, or material batch change, ensuring data diversity.

[0084] S12 Data Preprocessing: Blurred or invalid images are removed, and the retained qualified data are enhanced using dynamic thresholding and reflection suppression algorithms. First, polarization decomposition is used to eliminate reflections on the glass surface: diffuse reflection components are calculated and specular reflections are weakened to reduce ambient light interference and improve the contrast of defect areas. Then, the defect boundary threshold (mean + 0.5 standard deviation) is dynamically calculated, and morphological closing operations are fused to enhance defect continuity. Finally, the segmentation results are weighted and fused with the original image (enhancement factor α = 0.3) to enhance defect contrast and make small defects more prominent. Then, the enhanced dataset is labeled, and the defect locations are marked with rectangular boxes. Defects are classified into four categories: bubbles, scratches, cracks, and impurities. A small number of difficult samples (such as small scratches or semi-transparent cracks) are retained to improve the robustness of the model.

[0085] S13 Dataset Construction: A stratified sampling strategy was adopted to ensure a balanced distribution of different glass types (flat / curved) and defect sizes (<1mm / 1-5mm / >5mm) to obtain the dataset. The dataset was then divided into training, validation, and test sets in a 7:2:1 ratio. The test set must include data for all production line equipment models to verify generalization. To facilitate iterative optimization, data subsets were created according to production quarters, and basic metadata (such as environmental temperature and humidity) was recorded.

[0086] S2: Based on the YOLOv5 model, the Dimension Decoupling Feature Extraction Module (DDFM) is introduced, and the Corner Block Attention Module (CoPA) is added. The DDFM and CoPA are then fused to replace the original C3 module, resulting in a three-stage network architecture improvement, thus obtaining the DDC-YOLO model. The DDFM module decouples dimensions gradually from three-dimensional to two-dimensional to one-dimensional. Each branch gradually shifts the focus of feature extraction to the specified dimension, reducing information loss caused by dimensional changes. Finally, the output feature maps of the three branches are fused as the output, which belongs to the feature extraction module.

[0087] This technical solution does not use depthwise separable convolutions to replace traditional convolutions within the model. Instead, the three variants of convolution—depthwise convolution, strip convolution, and point convolution—are used as components of the DDFM module. Unlike existing MCA modules, which also employ a three-branch architecture, our DDFM module handles feature maps differently in terms of both method and purpose. MCA directly uses pooling to compress dimensions from three dimensions to one dimension, then calculates weights using convolution and applies them to the initial input, and it is an attention mechanism.

[0088] like Figure 2 As shown, the improvement of the three-stage network architecture in step S2 to construct the DDC-YOLO model specifically includes:

[0089] S21: Introduce the Dimension Decoupled Feature Extraction Module (DDFM) into the feature extraction network to perform independent feature modeling for height, width, and channel dimensions; by independently decoupling the feature representation of height, width, and channel dimensions, the cross-dimensional correlation of glass texture is enhanced.

[0090] like Figure 3 As shown, the specific steps of the Dimension Decoupling Feature Extraction Module (DDFM) in step S21 to achieve dimensional decoupling are as follows:

[0091] S211 3D Axis Rotation and Recombination: The input feature map is rotated sequentially along the height, width, and channel axes to obtain three sets of feature maps; specifically:

[0092] Input feature map X_in∈R CxHxW Perform rotation operations sequentially along the dimension axes:

[0093] X_hw=X_in∈R CxHxW ;

[0094] X_hc=Rotate_H(X_in)∈R WxHxC (Width and channel dimensions interchanged);

[0095] X_cw=Rotate_W(X_in)∈R HxCxW (Channel and height dimension permutation);

[0096] Where X_in represents the input feature map, X_hw represents the input feature map that retains the original dimension axes; X_hc represents the feature map after X_in is rotated and reconstructed along the height axis; X_cw represents the feature map after X_in is rotated and reconstructed along the width axis; Rotate_H represents rotation along the height axis, and Rotate_W represents rotation along the width axis;

[0097] ∈R a1xa2xa3This indicates that the dimensions of the feature map's channels, height, and width are a1, a2, and a3, respectively.

[0098] S212 Two-Dimensional Correlation Modeling: The three feature maps obtained in step S211 are subjected to depthwise convolution for feature extraction, yielding three two-dimensional correlation features, thus resulting in three sets of feature maps with two-dimensional correlation features. Specifically, depthwise convolution is performed on the rotated three sets of feature maps while restoring the dimensional order:

[0099] HW association feature: F_hw=DWConv3×3(X_hw)∈R CxHxW (Capture height-width relationship);

[0100] HC association feature: F_hc=Rotate_H(DWConv3×3(X_hc))∈R CxHxW (Capture height-channel correlation);

[0101] CW association feature: F_cw=Rotate_W(DWConv3×3(X_cw))∈R CxHxW (Capture channel-width correlation);

[0102] Where DWConv3×3 represents a depthwise convolution with a kernel size of 3×3; F_hw represents the output feature map that captures height-width related features; F_hc represents the output feature map that captures height-channel related features; and F_cw represents the output feature map that captures channel-width related features.

[0103] S213 Cross-feature fusion: The feature maps obtained in step S213 are then combined pairwise, and their intersection feature dimensions are concatenated. Specifically:

[0104] H-dimensional concatenation: F_h = Concat(F_hw, F_hc) ∈ R Cx2HxW ;

[0105] W-dimensional concatenation: F_w = Concat(F_hw, F_cw) ∈ R CxHx2W ;

[0106] C-dimensional concatenation: F_c = Concat(F_hc, F_cw) ∈ R 2CxHxW ;

[0107] Wherein, Concat means concatenating two feature maps along a specified dimension; F_h represents the output feature map after concatenating F_hw and F_hc along the height dimension, F_w represents the output feature map after concatenating F_hw and F_cw along the width dimension, and F_c represents the output feature map after concatenating F_hc and F_cw along the channel dimension;

[0108] S214 Feature Compression and Aggregation: Finally, strip convolution and point convolution are used to compress the features in the corresponding dimensions, remove redundant features, and obtain three sets of compressed feature maps, as follows:

[0109] Out1 = Conv 3x1 (F_h)∈R CxHxW ;

[0110] Out2 = Conv 1x3 (F_w)∈R CxHxW ;

[0111] Out3 = Conv 1x1 (F_c)∈R CxHxW ;

[0112] Among them, Conv 3x1 This represents a strip convolution with a kernel size of 3x1, Conv 1x3 This represents a strip convolution with a kernel size of 1x3, Conv 1x1 This represents a point convolution with a kernel size of 1x1; Out i The output after performing corresponding convolution processing on the feature map (i = 1 to 3);

[0113] S215 Feature Fusion Output: The three compressed feature maps are then superimposed and their average is taken for output. Specifically, the three feature maps are weighted and fused with the input feature map through residual connections for output.

[0114] X_out=(W1*Out1⊕W2*Out2⊕W3*Out3⊕W4*X_in);

[0115] Among them, W i X_out represents the learnable weights corresponding to each branch (i = 1 to 4); X_out represents the final output feature map of the module.

[0116] The DDFM module breaks through the spatial constraints of conventional convolution by rotating along three-dimensional axes, explicitly modeling the cross-dimensional relationship between height, width, and channels. At the same time, the combination of point convolution and strip convolution achieves a balanced optimization of cross-dimensional feature interaction and redundant information filtering.

[0117] S22: The Corner Point Block Attention Module (CoPA) is added to the head network to generate a spatial weight map based on statistical comparison of the four corner regions; the Corner Point Block Attention Module (CoPA) is designed, such as... Figure 4 As shown;

[0118] The specific steps for generating spatial attention weights by the corner point block attention module CoPA in step S22 are as follows:

[0119] S221 Gridded Feature Analysis: The input feature map is divided into n_win×n_win regular grid blocks, and the mean μ and standard deviation σ of each grid block are calculated; the formula is:

[0120] Mean: μ = AvgPool(X_in)∈R 1×n_win×n_win ;

[0121] Standard deviation: σ = StdPool(X_in) ∈ R 1×n_win×n_win ;

[0122] Where X_in represents the input feature map of the module; AvgPool represents the mean value calculated within each grid block after the feature map is divided into blocks; and StdPool represents the standard deviation calculated within each grid block after the feature map is divided into blocks.

[0123] S222 Corner Reference Difference Calculation: Select the top-left, top-right, bottom-left, and bottom-right corner blocks as reference points. Scan each grid block and calculate the statistical difference between the current grid block and the four corner grid blocks respectively; the formula is:

[0124] Δμ=Σ|μ_current-μ_corner_i|;

[0125] Δσ=Σ|σ_current-σ_corner_i|(i=1~4);

[0126] Where μ_current represents the mean of the current network block scanned when scanning each network block, σ_current is the standard deviation of the current network block; μ_corner_i represents the mean of the corner block, σ_corner_i represents the standard deviation of the corner block, and i is the corner index; Δμ represents the difference between the mean of the grid block and the corner block; Δσ represents the difference between the standard deviation of the grid block and the corner block.

[0127] S223 Dynamic Weight Generation: The difference values ​​are mapped to weight coefficients through the Sigmoid activation function to obtain the block-level weights W, as shown in the formula:

[0128] W=Sigmoid(Δμ)+Sigmoid(Δσ)-0.5;

[0129] S224 Spatial Attention Enhancement: The block-level weights W are then reconstructed into pixel-level spatial weight maps, and multiplied point-by-point with the original feature map, that is, the weight maps are added to the input feature map; specifically:

[0130] The block-level weights W are extended to pixel-level weight maps W_map∈R using bilinear interpolation. 1×H×W And it is weighted and added to the input feature map, as shown in the formula:

[0131] X_out=X_in⊙W_map∈R C×H×W ;

[0132] Where W_map represents the expanded pixel-level weight map; X_out represents the final output feature map of the module after weighting the input feature map;

[0133] The CoPA module quantifies the feature distribution by statistically combining the mean shift and standard deviation fluctuation of the corner benchmark blocks. By comparing the differences in feature distribution, it enhances the sensitivity to outlier regions without introducing additional parameters.

[0134] S23: The dual-correlation collaborative module DC-Block is constructed by fusing the dimensionally decoupled feature extraction module DDFM and the corner block attention module CoPA, replacing the original C3 module and reconstructing the feature pyramid network;

[0135] In step S23, the dual-related collaborative module DC-Block includes a dual-branch collaborative structure, namely a first branch and a second branch; wherein,

[0136] The first branch performs dimensional decoupling feature extraction through the Dimension Decoupling Feature Extraction Module (DDFM), focusing on capturing the cross-dimensional correlation characteristics of the glass texture;

[0137] The second branch generates a spatial attention mask through the corner block attention module CoPA to enhance the feature response of edge cracks and minor defects;

[0138] The outputs of the first and second branches are fused through channel cross-fusion, and then feature reshaping is performed through 1×1 convolution to make the output dimension consistent with the input.

[0139] S3: The DDC-YOLO model was trained and validated using a glass defect dataset, and then the DDC-YOLO model was used to detect defects in transparent materials in real time.

[0140] The specific steps of step S3 are as follows:

[0141] S31: The DDC-YOLO model is trained using the training set, validated using the validation set, and tested using the test set to assess its detection capabilities.

[0142] First, the training environment is configured: During the model training phase, a distributed training environment is built using a high-performance computing cluster, and a multi-GPU server (such as NVIDIA A6000×3) is configured. Mixed precision training (AMP) is implemented based on the PyTorch framework. CUDA acceleration is enabled during training, and a uniform random seed (seed=626) is set to ensure the reproducibility of experiments. Asynchronous loading (num_workers=8) and pin_memory technology are used for data processing to optimize the data transfer efficiency between GPU memory and system memory.

[0143] Next, the training data is loaded and augmented: The input data is processed through an online augmentation pipeline, including:

[0144] Geometric transformations: random horizontal flip (p = 0.5), ±15° rotation, and scaling by 0.8-1.2 scales;

[0145] Color perturbation: HSV color space adjustment (hue ±0.1, saturation ±0.7, brightness ±0.4);

[0146] Mosaic enhancement: Four images are stitched together to generate complex background samples, improving the robustness of small target detection;

[0147] The specific steps of step S31 are as follows:

[0148] S311 Training Model: The training set is input into DDC-YOLO, which is an improvement on YOLO, for training, and goes through a warm-up phase, a main training phase, and a fine-tuning phase.

[0149] In the warm-up phase, a linear learning rate is used (the learning rate lr is increased from 1e-5 to 1e-2), some layers of the backbone network are frozen, and only the detection head module is trained; then the CIoU loss function is used to initialize the weights to avoid early gradient instability.

[0150] During the main training phase (50-450 epochs), all network layers are unfrozen, and cosine annealing learning rate scheduling is enabled (base_lr = 1e-2, min_lr = 1e-4). A dynamic positive sample allocation strategy is introduced, which dynamically adjusts the ground truth allocation threshold based on the matching degree between the predicted bounding box and the ground truth bounding box.

[0151] During the fine-tuning phase (450-500 epochs), cross-GPU synchronous batch normalization (SyncBN) is enabled, and exponential moving average (EMA) is used to update model parameters.

[0152] S312 Validation Model: Configure the loss function and use the validation set to validate the trained DDC-YOLO model to evaluate its performance;

[0153] S313 test model: The input image is set to its original size, and edge padding is performed if it is less than a multiple of 32. Then, an improved non-maximum suppression algorithm (NMS) is used to filter the image by setting a class-independent IoU threshold (set to 0.7 here) and a confidence threshold (initial threshold of 0.001, which is adaptively adjusted according to the target density) to reduce redundant boxes in target detection and improve detection accuracy.

[0154] The loss function configured in step S312 includes three parts: localization loss, classification loss, and confidence loss.

[0155] The localization loss, or bounding box regression loss, is quantified using SIoU and incorporates angle and distance cost terms to optimize the bounding box regression direction; the formula is:

[0156] L box =1-IoU+(Δ+Ω) / 2;

[0157] Among them, L box For bounding box regression loss, IoU represents the intersection-union ratio of the predicted box and the ground truth box, Δ represents the corner cost term, and Ω represents the distance cost term;

[0158] The category classification loss is an improved Focal Loss function, applying a modulation coefficient of γ = 2.0 to the hard examples; this is also an improved cross-entropy, which reduces the weight of easily classified samples by adjusting the factor to solve the imbalance between positive and negative samples. The formula is:

[0159] L cls =-α t *(1-p t ) γ *log(p t )

[0160] Among them, L cls For category classification loss, α t p represents the class balance factor. t This represents the model's predicted probability of the true class, where γ is the modulation coefficient; in this embodiment, γ = 2.0.

[0161] The confidence loss is achieved by adding a dynamic weighting factor to alleviate the imbalance between positive and negative samples in dense target scenarios; the formula is:

[0162]

[0163] Among them, L obj λ represents the confidence loss. pos λ represents the dynamic weight of positive samples. negrepresents the dynamic weight of negative samples, y is the true label, 1 represents the target and 0 represents the background; The confidence score for the model's prediction;

[0164] Total loss function L total For weighted summation, the formula is:

[0165] L total =0.05L box +0.5L cls +L obj ;

[0166] S32: Deploy the trained DDC-YOLO model to the production line for detection and output the detection results in real time;

[0167] The specific steps for detection using the trained model in step S32 are as follows:

[0168] S321 Image Preprocessing: Adjusting brightness and contrast to reduce glare interference; automatically identifying glass edges and cropping the effective detection area;

[0169] S322 Defect Detection: First, a two-stage detection method is adopted, which is: first, the entire glass is quickly scanned, and then the suspicious areas are carefully inspected; automatically identifying four types of defects: bubbles, cracks, scratches, and impurities;

[0170] S323 Result Processing: Merge adjacent defect boxes of the same type, filter out tiny noise and normal texture areas smaller than 5 pixels; record the defect location, type, and size;

[0171] S324 system linkage: Real-time output of test results to display device, automatic marking of unqualified products and triggering of sorting device.

[0172] The entire inspection process takes less than 200ms per piece, meeting the real-time inspection requirements of the production line. The inspection parameters can be flexibly adjusted according to different glass types (such as flat glass and curved glass).

[0173] To verify the detection performance of the proposed DDC-YOLO model on the glass dataset, it was compared with various existing improved YOLO models. Table 1 below shows the performance comparison of each method on this glass dataset.

[0174] Table 1 compares the performance of various methods on this glass dataset.

[0175]

[0176] The test results in Table 1 show that the DDC-YOLO model, which improves the three-stage network architecture by introducing the Dimension Decoupling Feature Extraction (DDFM) module, adding the Corner Point Block Attention (CoPA) module, and fusing the DDFM and CoPA modules to replace the original C3 module, achieves a fitness rate of 48.6% for detecting minor defects such as scratches, pores, and cracks. This is a 3.2% improvement compared to the previous model. Furthermore, with only 5.1M parameters and 8.8G of computation, the training speed is significantly accelerated, and the training cost for industrial defect detection is reduced.

[0177] The above embodiments are merely specific implementations of the present invention and are not intended to limit the invention. Any modifications, substitutions, or improvements made by those skilled in the art using conventional technical means without departing from the core ideas of the present invention are within the scope of protection of the present invention.

Claims

1. A glass defect detection method based on an improved YOLOv5, characterized in that, Includes the following steps: S1: High dynamic range industrial cameras are used to acquire multispectral image data of glass product production lines. After data processing, a glass defect detection dataset is constructed; then the dataset is divided. S2: Based on the YOLOv5 model, the dimension decoupling feature extraction module DDFM is introduced, the corner block attention module CoPA is added, and the dimension decoupling feature extraction module DDFM and the corner block attention module CoPA are fused to replace the original C3 module to improve the three-stage network architecture, resulting in the DDC-YOLO model. S3: The constructed DDC-YOLO model is trained and validated using a glass defect dataset, and then the DDC-YOLO model is used to detect defects in transparent materials in real time; The improvements to the three-stage network architecture in step S2, specifically in constructing the DDC-YOLO model, include: S21: Introduce the Dimensionally Decoupled Feature Extraction Module (DDFM) into the feature extraction network to perform independent feature modeling for height, width, and channel dimensions; S22: Add the corner point segmentation attention module CoPA to the head network to generate a spatial weight map based on the statistical comparison of the four corner regions; S23: The dual-correlation collaborative module DC-Block is constructed by fusing the dimensionally decoupled feature extraction module DDFM and the corner block attention module CoPA, replacing the original C3 module and reconstructing the feature pyramid network; The specific steps of the dimensional decoupling feature extraction module DDFM in step S21 to achieve dimensional decoupling are as follows: S211: The input feature map is rotated sequentially along the height, width, and channel axes to obtain three sets of feature maps; S212: Use depthwise convolution to extract features from the three sets of feature maps obtained in step S211, and extract three two-dimensional correlation features respectively, thus obtaining three sets of feature maps with two-dimensional correlation features. S213: Then combine the feature maps obtained in step S212 in pairs and splice them together according to their intersection feature dimensions; S214: Finally, use strip convolution and point convolution to compress the features of the corresponding dimension, remove redundant features, and obtain three sets of compressed feature maps. S215: Then, superimpose the three sets of compressed feature maps, take their average value, and output the result. The specific steps for generating spatial attention weights by the corner point block attention module CoPA in step S22 are as follows: S221 Gridded Feature Analysis: The input feature map is divided into n_win×n_win regular grid blocks, and the mean μ and standard deviation σ of each grid block are calculated. S222 Corner Reference Difference Calculation: Select the top-left, top-right, bottom-left, and bottom-right corner blocks as reference points. Scan each grid block and calculate the statistical difference between the current grid block and the four corner grid blocks respectively; the formula is: Δμ = Σ|μ_current - μ_corner_i|; Δσ = Σ|σ_current - σ_corner_i|, i=1~4; Where μ_current represents the mean of the current network block scanned when scanning each network block, σ_current is the standard deviation of the current network block; μ_corner_i represents the mean of the corner block, σ_corner_i represents the standard deviation of the corner block, and i is the corner index; Δμ represents the difference between the mean of the grid block and the corner block; Δσ represents the difference between the standard deviation of the grid block and the corner block. S223 Dynamic Weight Generation: The difference values ​​are mapped to weight coefficients through the Sigmoid activation function to obtain the block-level weights W, as shown in the formula: W = Sigmoid(Δμ) + Sigmoid(Δσ) - 0.5; S224 Spatial Attention Enhancement: The block-level weights W are then reconstructed into pixel-level spatial weight maps, and multiplied point by point with the original feature map, that is, the weight map is added to the input feature map.

2. The glass defect detection method based on the improved YOLOv5 according to claim 1, characterized in that, In step S1, image enhancement processing is performed using dynamic thresholding and reflection suppression algorithms to construct a glass defect detection dataset containing bubbles, scratches, and impurities. The specific steps are as follows: S11 Data Acquisition: Deploy industrial cameras on the glass production line to cover key processes including cutting, grinding, and tempering, acquiring high-resolution images and videos of ≥4K resolution. The data includes defects and covers different lighting conditions, glass thickness, and surface conditions; it also records the production line status to ensure data diversity. S12 Data Preprocessing: Blurry or invalid images are removed, and the remaining qualified data are enhanced using dynamic thresholding and reflection suppression algorithms. Then, the enhanced dataset is labeled, with rectangular boxes used to mark the defect locations and the defects are classified into four categories: bubbles, scratches, cracks, and impurities. S13 Dataset Construction: A stratified sampling strategy was adopted to ensure a balanced distribution of different glass types and defect sizes, resulting in a dataset. The dataset was then divided proportionally into training, validation, and test sets.

3. The glass defect detection method based on the improved YOLOv5 according to claim 2, characterized in that, In step S23, the dual-related collaborative module DC-Block includes a dual-branch collaborative structure, namely a first branch and a second branch; wherein, The first branch performs dimensional decoupling feature extraction through the Dimension Decoupling Feature Extraction Module (DDFM) to capture the cross-dimensional correlation characteristics of the glass texture; The second branch generates a spatial attention mask through the corner block attention module CoPA to enhance the feature response of edge cracks and minor defects; The outputs of the first and second branches are cross-fused through channels and then remodeled through convolution to ensure that the output dimension is consistent with the input dimension.

4. The glass defect detection method based on the improved YOLOv5 according to claim 3, characterized in that, The specific steps of step S3 are as follows: S31: The DDC-YOLO model is trained using the training set, validated using the validation set, and tested using the test set to assess its detection capabilities. S32: Deploy the trained DDC-YOLO model to the production line for detection and output the detection results in real time.

5. The glass defect detection method based on the improved YOLOv5 according to claim 4, characterized in that, The specific steps of step S31 are as follows: S311 Training Model: The training set is input into DDC-YOLO, which is an improvement on YOLO, for training, and goes through a warm-up phase, a main training phase, and a fine-tuning phase. In the warm-up phase, a linear learning rate is used to warm up the network, freezing some layers of the backbone network and training only the detection head module. Then, the weights are initialized using the CIoU loss function. In the main training phase, all network layers are unfrozen, and cosine annealing learning rate scheduling is enabled with base_lr=1e-2 and min_lr=1e-4. A dynamic positive sample allocation strategy is introduced to dynamically adjust the ground truth allocation threshold based on the matching degree between the predicted bounding box and the ground truth bounding box. During the fine-tuning phase, cross-GPU synchronous batch normalization is enabled, and exponential moving average (EMA) is used to update model parameters. S312 Validation Model: Configure the loss function and use the validation set to validate the trained DDC-YOLO model to evaluate its performance; S313 test model: The input image is set to its original size, and edge padding is performed if it is less than a multiple of 32; then the improved non-maximum suppression algorithm NMS is used to filter by setting class-independent IoU threshold and confidence threshold to reduce redundant boxes in object detection.

6. The glass defect detection method based on the improved YOLOv5 according to claim 5, characterized in that, The loss function configured in step S312 includes three parts: localization loss, classification loss, and confidence loss. The localization loss, i.e., the bounding box regression loss, is quantified using SIoU, and incorporates angle and distance cost terms to optimize the bounding box regression direction. The formula is as follows: L box = 1 - IoU + (Δ+Ω) / 2; Among them, L box For bounding box regression loss, IoU represents the intersection-union ratio of the predicted box and the ground truth box, Δ represents the corner cost term, and Ω represents the distance cost term; The category classification loss is an improved Focal Loss function, with a modulation coefficient of γ applied to the hard examples; the formula is: L cls =-a t * (1-p t ) γ * log(p t ); Among them, L cls For category classification loss, α t p represents the class balance factor. t This represents the model's predicted probability of the true class, where γ is the modulation coefficient. The confidence loss is calculated by adding a dynamic weighting factor, and the formula is as follows: L obj =-λ pos * y * log(ŷ) - λ neg * (1 - y) * log(1 - ŷ); Among them, L obj λ represents the confidence loss. pos λ represents the dynamic weight of positive samples. neg ŷ represents the dynamic weight of negative samples, y is the true label, where 1 represents the target and 0 represents the background; ŷ is the confidence score of the model prediction. Total loss function L total For weighted summation, the formula is: L total =0.05L box +0.5L cls +L obj 。 7. The glass defect detection method based on the improved YOLOv5 according to claim 4, characterized in that, The specific steps for detection using the trained model in step S32 are as follows: S321 Image Preprocessing: Adjusting brightness and contrast; Automatically identifying glass edges and cropping the effective detection area; S322 Defect Detection: First, a two-stage detection method is adopted, which is: first, the entire glass is quickly scanned, and then the suspicious areas are carefully inspected; automatically identifying four types of defects: bubbles, cracks, scratches, and impurities; S323 Result Processing: Merge adjacent defect boxes of the same type, and filter out tiny noise and normal texture areas of less than 5 pixels; Record the location, type, and size of the defect; S324 system linkage: Real-time output of test results to display device, automatic marking of unqualified products and triggering of sorting device.

Images