High-reflection high-transmission material surface flaw detection method based on improved YOLOv11

Abstract

The application discloses a high-reflection high-transparency material surface flaw detection method based on improved YOLOv11, which comprises the following steps: S1, collecting mobile phone screen surface defect images, preprocessing and labeling, and generating a training data set containing three types of defects, i.e., scratches, edge collapse and cracks; S2, constructing a flaw detection model; S3, dividing the input image into four independent detection regions, using a boundary box overlap area weight distribution strategy for cross-region defects, and using the flaw detection model to count the image by region; S4, constructing a surface defect detection network meeting the real-time demand of an industrial production line, and outputting the detection result and processing frame rate; S5, performing pixel-level mask segmentation on the scratch defect, and extracting the contour coordinates and geometric features; and S6, fitting the actual damage size of the scratch defect on the surface of the high-reflection high-transparency material according to the segmentation result. The application has a faster detection speed under the condition of ensuring the accuracy, and provides a reliable solution for high-reflection high-transparency material surface flaw detection.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence machine vision inspection technology, and specifically relates to a method for detecting surface defects of high reflectivity and high transmittance materials based on the improved YOLOv11. Background Technology

[0002] High-reflectivity and high-transmittance materials are key materials in high-end manufacturing, widely used in the production of precision components such as mobile phone screens, automotive glass, and optical lenses. However, their surfaces are susceptible to mechanical friction or environmental stress during processing, cleaning, and transportation, resulting in microscopic defects such as scratches, chipping, and cracks. These defects not only affect the product's appearance but also reduce optical performance and even cause structural failure. For example, minute scratches on a mobile phone screen can lead to decreased touch sensitivity, while chipping defects in automotive glass can cause stress concentration, reducing impact resistance. Therefore, achieving high-precision and high-efficiency surface defect detection during the production process is crucial.

[0003] In recent years, deep learning-based machine vision technology has made significant progress in surface defect detection. By extracting defect features using convolutional neural networks (CNNs) and training models with large-scale defect datasets, existing techniques can now automatically classify most defects. To further improve detection robustness, researchers have introduced multimodal data fusion methods, such as overlaying infrared images with visible light images to enhance defect contrast, or using thermal imaging technology to capture differences in material stress distribution.

[0004] While such methods can improve the recognition rate in complex backgrounds, they still have limitations: insufficient accuracy in detecting minute defects, high false positive rate for dense defects, and lack of defect quantification and traceability. This cannot meet the inspection rules for various defects in the touch screen inspection result report as specified in the current industry standard GB / T36259—2018 "High-aluminosilicate Glass for Touch Screen Covers".

[0005] Therefore, this invention proposes a method for detecting surface defects in high-reflectivity and high-transparency materials based on the improved YOLOv11. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention aims to propose a surface defect detection method for high-reflectivity and high-transparency materials based on an improved YOLOv11. This method enhances the identification capability of minute defects through a dynamic deformable convolution module, achieves accurate defect tracing by combining a cross-regional area weight allocation strategy, and constructs a pixel-physical size mapping model to make the scratch length detection error ≤ ±0.05mm, while meeting the real-time detection speed requirement (≥200FPS), providing an efficient and reliable solution for the quality inspection of high-reflectivity material surfaces.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] The first aspect of this invention provides a method for detecting surface defects in high-reflectivity and high-transmittance materials based on an improved YOLOv11, comprising the following steps:

[0009] S1. Collect images of surface defects on mobile phone screens in industrial production lines to construct an original surface defect image dataset; preprocess and label the original surface defect image dataset to generate a training dataset containing three types of defects: scratches, chipping, and cracks.

[0010] S2. Build and train an improved YOLOv11 network model to construct a defect detection model;

[0011] S3. Divide the input image into four independent detection regions on an average basis. For cross-region defects, adopt the bounding box overlap area weighting strategy and use the defect detection model to perform partition counting on the image.

[0012] S4. By adjusting the lightweight network structure and optimizing the hardware acceleration, a surface defect detection network that meets the real-time requirements of industrial production lines is constructed, and the detection results and processing frame rate are output.

[0013] S5. Perform pixel-level mask segmentation on scratch defects and extract contour coordinates and geometric features;

[0014] S6. Fit the actual damage size of the scratch defects on the surface of the high-reflectivity and high-transparency material based on the segmentation results.

[0015] Preferably, in step S1, the steps of acquiring original mobile phone screen surface defect images, constructing a surface defect dataset, preprocessing and labeling the original surface defect image dataset, and generating a training dataset containing three types of defects: scratches, chipping, and cracks, specifically include:

[0016] S11. Fix the relative spatial parameters of the moving camera and the mobile phone screen under test in a dark room environment. The camera is vertically installed on the top of the platform, and the mobile phone screen is horizontally placed on a black background board and fixed to the bottom of the platform. The equipment is wrapped with black light-blocking cloth to eliminate the interference of ambient stray light. Multiple strip light sources are evenly arranged at 45° above the screen to provide directional oblique lighting. Multi-angle images of surface defects on the mobile phone screen are collected to construct the original surface defect image dataset.

[0017] S12. Preprocess the original surface defect image dataset, including format unification, size normalization, random rotation, flipping, and brightness adjustment operations, to generate an enhanced defect sample library.

[0018] S13. Label the bounding boxes of the defect targets in the augmented dataset, and define three types of defect labels: scratches, chipped edges, and cracks. Generate a label file containing the three types of defects and conforming to the PASCAL VOC format, which will serve as the training dataset.

[0019] Preferably, in step S2, the construction of the improved YOLOv11 network model includes the following steps:

[0020] S21. The DySnakeConv dynamic deformable convolutional module is used to replace the C3K2 standard convolutional layer in the original Bottleneck structure of the YOLOv11 network. By introducing illumination invariance preprocessing to suppress high reflectivity interference, a stable input is provided for subsequent feature extraction. The dynamic deformable convolutional kernel relies on the adaptive deformation mechanism to accurately fit the shape of the defect edge, enhancing the ability to capture the edge and texture features of small defects.

[0021] S22. In the C2PSA feature output of layer 11 of YOLOv11 network, the ContextAggregation module is integrated to construct a multi-scale feature mechanism that coordinates local and global features. This mechanism simultaneously addresses the issues of feature loss in small defects and insufficient contextual information in large defects. Furthermore, it integrates shallow detail information with deep semantic features through a cross-level feature pyramid aggregation mechanism to suppress interference from complex background noise.

[0022] S221. Addressing the issue that existing YOLOv11 designs do not adequately consider the contextual differences in defects of different scales, this invention introduces a local-global contextual branching mechanism. For small defects <5×5 pixels, local window attention (setting the window spatial radius r = 5) is used to capture fine-grained features; for large defects >20×20 pixels, a global multi-head attention mechanism is used to model long-range dependencies. Branch fusion is adaptively controlled through a dynamic weight mapping function, defined as:

[0023] α=σ(MLP(||x||2))

[0024] Where x is the feature vector of the branch output, ||x||2 is used to calculate its L2 norm to measure the significance of the feature; the multilayer perceptron (MLP) performs a nonlinear transformation on the norm result; the sigmoid function σ compresses the output to the (0,1) interval and generates the weight α that controls the local-global branch fusion ratio;

[0025] In particular, addressing the pain points of highly reflective and high-transmittance material surfaces being prone to reflective interference and densely overlapping defects, this module relies on local-global branching to accurately capture fine-grained and long-range features, and dynamic weighting to adaptively control the fusion of complex features. It can effectively distinguish overlapping defect boundaries and suppress reflective noise, demonstrating unique advantages in accurate identification and anti-interference in the detection of densely overlapping defects.

[0026] S23. The standard convolutional layer in the Neck part is replaced with a depthwise separable convolution (DSConv) module, which reduces computational complexity while maintaining detection accuracy and optimizes the model inference speed.

[0027] Preferably, in step S2, training the improved YOLOv11 network model includes the following steps:

[0028] S24. The surface defect dataset is divided into training set, validation set and test set in a 7:2:1 ratio using a random stratified sampling method to ensure a balanced distribution of each defect category.

[0029] S25. Initialize the model parameters of the YOLOv11 network pre-trained on the COCO dataset, freeze the first 50 training epochs of the backbone network, and set the initial learning rate to 1×10. -4 The decay rate was 0.96; the backbone network was unfrozen after 50 cycles, and the learning rate was adjusted to 1×10. -5 And maintaining a decay rate of 0.96, the Adam optimizer was used for iterative training up to 150 epochs;

[0030] S26. The improved YOLOv11 network model trained is validated using a validation set. The mean precision, recall, and false positive rate are calculated using the validation set. The optimal weights are then selected and saved to the test set for final performance evaluation.

[0031] Preferably, in step S3, the steps of dividing the input image into four independent detection regions on an average basis, using a bounding box overlap area weighting strategy for cross-region defects, and using a defect detection model to perform partition counting on the image include:

[0032] S31. Divide the input image into four independent detection regions: upper left, lower left, upper right, and lower right. For defect targets distributed across regions, determine their assigned region based on the maximum weight allocation strategy of the overlap area between the bounding box and each region. The calculation formula is as follows:

[0033] A k =Area(B∩R) k )

[0034] Among them, A k Represents the bounding box B and the partition R k The overlapping area, B = (x min y min x max y max R represents the bounding box coordinate parameters. k ∈{TL, TR, BL, BR} represents the four partitions: top left, bottom left, top right, and bottom right. R * The partition indicating the final attribution of the defect;

[0035] S32. Using root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) 2The method of residual analysis was used to compare the statistical differences between defect detection results and actual measurement data to verify the accuracy of zonal counting.

[0036] Preferably, in step S4, the step of constructing a surface defect detection network that meets the real-time requirements of industrial production lines through lightweight network structure adjustment and hardware acceleration optimization includes the following steps:

[0037] Based on the improved YOLOv11 defect detection model, network layer fusion and INT8 quantization are performed using the TensorRT framework to reduce the computational complexity of the model. Depthwise separable convolutions are used to replace the standard convolutional layers in the Neck part, and channel pruning techniques are combined to remove redundant parameters, thereby reducing the number of model parameters and constructing a surface defect detection network that meets the real-time requirements of industrial production lines.

[0038] Preferably, in step S5, the pixel-level mask segmentation of the scratch defect includes the following steps:

[0039] S51. Based on labeled scratch defect samples, multi-scale segmentation training data is generated through random rotation, affine transformation and brightness perturbation.

[0040] S52. Use Labelme software to outline defects in mobile phone screen glass, generate a JSON format mask file, and then convert it into a YOLO format TXT file according to the COCO dataset specification. Each record contains the defect category number, normalized center coordinates, and bounding box size.

[0041] S53. Call the batch inference interface of the defect detection model to perform detection on the segmented image and output the structured detection results. Save each record in the text file containing the coordinates of the defect center point (x, y). C ,y C The bounding box dimensions (w, h) and confidence level p, whose numerical ranges satisfy x C y C ∈[0,1], w, h∈(0,1], p≥0.5.

[0042] Preferably, in step S6, the specific steps of fitting the actual damage size of the scratch defect on the surface of the high-reflectivity and high-transparency material based on the segmentation result include:

[0043] S61. Based on the contour point set output by the segmentation mask, calculate the total scratch length using the Euclidean distance accumulation algorithm between adjacent points. The calculation formula is as follows:

[0044]

[0045] Where L is the scratch length based on pixel coordinates, n is the total number of contour points, k represents the index variable of the contour point set, and x k yk x represents the pixel coordinates of the k-th contour point extracted from the mask. k+1 y k+1 Represents the pixel coordinates of the (k+1)th contour point adjacent to point k;

[0046] S62. An evaluation is conducted by comparing the difference between the detected defect range and the actual measured data. The evaluation methods include root mean square error, mean absolute error, coefficient of determination, and residual analysis. The calibration parameters are mapped to the actual physical dimensions to generate a defect quantification report that conforms to the GB / T36259-2018 standard.

[0047] The present invention also provides a computer device, including a memory and a processor; the memory is used to store a computer program and the execution logic of the above-mentioned method for detecting surface defects of high reflectivity and high transmittance materials based on the improved YOLOv11.

[0048] The present invention has the following beneficial effects:

[0049] (1) This invention provides a method for detecting surface defects of high reflectivity and high transmittance materials based on YOLOv11. The YOLOv11 network framework is used as the basic framework, and the DySnakeConv module is introduced to replace the standard convolution operation in C3K2. The high reflectivity interference is suppressed by introducing illumination invariance preprocessing, which provides stable input for subsequent feature extraction. The dynamic deformable convolution kernel relies on the adaptive deformation mechanism to accurately fit the defect edge shape, which effectively enhances the model's ability to capture defect edge and texture features. The ContextAggregation module is also added in an appropriate position in the YOLOv11 network framework. Its cross-level feature aggregation mechanism is used to strengthen the modeling of global context information. Local-global context branches are introduced. For small defects with a size <5×5 pixels, local window attention (setting the window space radius r=5) is used to capture fine-grained features. For large defects with a size >20×20 pixels, a global multi-head attention mechanism is used to model long-range dependencies, which improves the recognition robustness under complex background interference. This invention offers a fast detection speed while ensuring accuracy, making it suitable for detecting images containing numerous defects and providing a reliable solution for detecting surface defects in high-reflectivity and high-transparency materials.

[0050] (2) This invention also proposes a defect statistics method based on image region division. By establishing a precise defect location and tracing mechanism, it achieves accurate diagnosis of production equipment faults. Specifically, this method first divides each image to be detected into four detection regions: upper left, lower left, upper right, and lower right. For defect targets distributed across regions, the principle of maximizing the overlap area of ​​the bounding box is used for attribution determination. That is, by calculating the overlap area between the defect bounding box and each detection region, the defect is classified into the region with the largest overlap area. This geometric feature-based allocation strategy effectively solves the problem of attribution determination in cross-region defect statistics using traditional methods. Furthermore, by continuously monitoring the dynamic changes in defect counts in each region, when a specific region has a significantly higher number of defects than the threshold within a continuous detection cycle, the system can automatically generate equipment fault warnings, providing accurate spatial location references for production line maintenance. To verify the reliability of the method, the manually labeled real value counts are compared and analyzed with the model detection values. By calculating quantitative indicators such as mean absolute error and coefficient of determination, the detection performance of the system is evaluated from multiple dimensions to ensure that the detection results meet the requirements of industrial-grade quality control. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating a method for detecting surface defects in high-reflectivity and high-transmittance materials based on an improved YOLOv11 according to the present invention.

[0053] Figure 2 To improve the network structure of YOLOv11;

[0054] Figure 3 Schematic diagrams of different types of defects;

[0055] Figure 4 The flowchart shows the convolution operation of DySnakeConv.

[0056] Figure 5 ContextAggregation aggregation flowchart;

[0057] Figure 6 This is a diagram showing the detection results of the improved YOLOv11 model for surface defects in Example 1.

[0058] Figure 7 This is a graph showing the results of the counting analysis after partition counting. Detailed Implementation

[0059] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0060] Example 1

[0061] Reference Figure 1-7 This invention provides a method for detecting surface defects in high-reflectivity and high-transmittance materials based on an improved YOLOv11, comprising the following steps:

[0062] S1. Collect images of surface defects on mobile phone screens in industrial production lines to construct an original surface defect image dataset; preprocess and label the original surface defect image dataset to generate a training dataset containing three types of defects: scratches, chipping, and cracks.

[0063] Specifically, it includes the following steps:

[0064] S11. A high-precision mobile industrial camera with a resolution of 1920×1080 pixels was used to acquire images of surface defects on the mobile phone screen in a standardized acquisition environment. The relative spatial parameters between the mobile camera and the screen under test were fixed in a darkroom environment. The camera was vertically mounted on the top of the platform, and the mobile phone screen was horizontally placed on a black background board and fixed to the bottom of the platform. The equipment was wrapped with black light-blocking cloth to eliminate ambient stray light interference. Multiple strip light sources were evenly distributed at a 45° angle above and around the screen for directional oblique lighting. Images of surface defects on the mobile phone screen were acquired from multiple angles, resulting in 452 original images. To address the imaging characteristics of highly reflective and highly transparent materials, a black light-absorbing background board was used to eliminate ambient light interference, and multi-angle strip light sources were configured for directional supplementary lighting, effectively suppressing surface reflection and edge exposure, ensuring that defect features were clearly visible.

[0065] S12. Perform preprocessing such as data cleaning and data augmentation on the original surface defect image dataset, including format unification, size normalization, random rotation, flipping, and brightness adjustment operations, to generate an enhanced defect sample library.

[0066] S13. Use Labelimg software to annotate the bounding boxes of defects in the enhanced defect dataset, dividing it into training, validation, and test sets in a 7:2:1 ratio, and defining three defect labels: Crack, Scratch, and Chipping. Figure 3As shown, a labeling file containing three types of defects—scratches, chipping, and cracks—and conforming to the PASCALVOC format is generated.

[0067] S2. Improve upon YOLOv11 by building an improved YOLOv11 network model, training and optimizing parameters, and constructing a defect detection model.

[0068] Specifically, the YOLOv11 network structure is improved. The YOLOv11 network structure mainly consists of three parts: Backbone, Neck, and Head.

[0069] The backbone is responsible for feature extraction and employs a series of convolutional and deconvolutional layers, combined with residual connections and bottleneck structures, to reduce network size and improve performance. This part uses C3K2 modules as basic building blocks.

[0070] The Neck section is responsible for multi-scale feature fusion, which enhances feature representation capabilities by integrating feature maps from different stages of the Backbone.

[0071] Specifically, the Neck section of YOLOv11 includes the following components:

[0072] SPPF (Spatial Pyramid Pooling Fast) module: For pooling operations at different scales, it stitches together feature maps of different scales to improve the detection capability of targets of different sizes.

[0073] C2PSA module (convolutional module with parallel spatial attention): enhances the expressive power of feature maps through a parallel spatial attention mechanism.

[0074] The Head section is responsible for the final object detection and classification tasks, and includes a detection head and a classification head:

[0075] Detection head: Contains a series of convolutional and deconvolutional layers to generate detection results.

[0076] Classification Head: Global Average Pooling is used to classify each feature map, reducing the dimensionality of the feature maps and outputting the probability distribution of each class.

[0077] To address the issues of small target detection and similar object confusion on mobile phone screens, this invention introduces the DySnakeConv module, the ContextAggregation module, and the lightweight DSConv module based on YOLOv11 to construct a network model, wherein...

[0078] The DySnakeConv module suppresses high reflectivity interference by introducing illumination invariance preprocessing, providing stable input for subsequent feature extraction. The dynamically deformable convolutional kernel, relying on an adaptive deformation mechanism, precisely fits the shape of defect edges, enhancing the ability to capture minute defect edges and texture features. Furthermore, by introducing learnable offsets to dynamically adjust the shape of the convolutional kernel, it better adapts to different features in the image, solving the problem of difficulty in distinguishing similar objects and confused categories.

[0079] The ContextAggregation module integrates contextual information through multi-scale feature fusion and feature recalibration, enhancing the discriminative power of target features. Please refer to [link / reference]. Figure 5 .

[0080] The DSConv module: It achieves lightweighting of the model through depthwise separable convolutions, thereby improving the inference speed of the model.

[0081] Among them, the DySnakeConv dynamic deformable convolutional module is used to replace the C3K2 standard convolutional layer in the original Bottleneck structure of the YOLOv11 network. By introducing illumination invariance preprocessing to suppress high reflectivity interference, it provides stable input for subsequent feature extraction. The dynamic deformable convolutional kernel accurately fits the shape of the defect edge by relying on the adaptive deformation mechanism, which enhances the ability to capture the edge and texture features of small defects.

[0082] In the YOLOv11 network, the ContextAggregation module is integrated into the C2PSA feature output of the 11th layer to build a multi-scale feature mechanism that coordinates local and global features. This simultaneously solves the problems of feature loss in small defects and insufficient contextual information in large defects. Furthermore, the shallow detail information and deep semantic features are fused through a cross-level feature pyramid aggregation mechanism to suppress interference from complex background noise.

[0083] To address the issue that existing YOLOv11 designs do not adequately consider the contextual differences in defects of different scales, this invention introduces a local-global contextual branching mechanism. For small defects <5×5 pixels, local window attention (with a window radius r=5) is used to capture fine-grained features. For large defects >20×20 pixels, a global multi-head attention mechanism is used to model long-range dependencies. Branch fusion is adaptively controlled through a dynamic weight mapping function, defined as:

[0084] α=σ(MLP(||x||2))

[0085] Where x is the feature vector of the branch output, ||x||2 is used to calculate its L2 norm to measure the significance of the feature; the multilayer perceptron (MLP) performs a nonlinear transformation on the norm result; the sigmoid function σ compresses the output to the (0,1) interval and generates the weight α that controls the local-global branch fusion ratio.

[0086] In particular, addressing the pain points of highly reflective and high-transmittance material surfaces being prone to reflective interference and densely overlapping defects, this module relies on local-global branching to accurately capture fine-grained and long-range features, and dynamic weighting to adaptively control the fusion of complex features. It can effectively distinguish overlapping defect boundaries and suppress reflective noise, demonstrating unique advantages in accurate identification and anti-interference in the detection of densely overlapping defects.

[0087] Specifically, the pre-trained network weights are used as initial weights, and the training is performed using the training dataset with the transfer learning method. The pre-trained weights are iterated for 100 epochs with a learning rate of 0.01. The input image size is 640×640. The improved YOLOv11 network model after training is validated using a validation set.

[0088] S3. Divide the input image into four independent detection regions on an average basis. For cross-region defects, adopt the bounding box overlap area weighting strategy and use the defect detection model to perform partition counting on the image.

[0089] Specifically, it includes the following steps:

[0090] S31. Divide the input image into four independent detection regions: upper left, lower left, upper right, and lower right. For defect targets distributed across regions, determine their assigned region based on the maximum weight allocation strategy of the overlap area between the bounding box and each region. The calculation formula is as follows:

[0091] A k =Area(B∩R) k )

[0092]

[0093] Among them, A k Represents the bounding box B and the partition R k The overlapping area, B = (x min y min x max y max R represents the bounding box coordinate parameters. k ∈{TL, TR, BL, BR} represents the four partitions: top left, bottom left, top right, and bottom right. R * The partition indicating the final attribution of the defect;

[0094] S32. Using root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) 2 The method of residual analysis was used to compare the statistical differences between defect detection results and actual measurement data to verify the accuracy of zonal counting.

[0095] S4. By adjusting the lightweight network structure and optimizing hardware acceleration, a surface defect detection network that meets the real-time requirements of industrial production lines is constructed.

[0096] Specifically, based on the defect detection model, network layer fusion and INT8 quantization are performed using the TensorRT framework to reduce the computational complexity of the model; depthwise separable convolutions are used to replace the standard convolutional layers in the Neck part, and channel pruning techniques are combined to remove redundant parameters, thereby reducing the number of model parameters and constructing a surface defect detection network that meets the real-time requirements of industrial production lines.

[0097] S5. Perform pixel-level mask segmentation on scratch defects and extract contour coordinates and geometric features;

[0098] Specifically, it includes the following steps:

[0099] S51. Based on labeled scratch defect samples, multi-scale segmentation training data is generated through random rotation, affine transformation and brightness perturbation.

[0100] S52. Construct a scratch defect dataset; Use Labelme software to outline defects in mobile phone screen glass, generate a JSON format mask file, and then convert it into a YOLO format TXT file according to the COCO dataset specification. Each record contains the defect category number, normalized center coordinates, and bounding box size.

[0101] S53. Use the batch detection function in the defect detection model to detect all segmented images, and save the detection results of the segmented images to a text file. The text file records the center point coordinates of the detection box of the defect in the segmented image, the length and width of the detection box, and the confidence level.

[0102] S6. Fit the actual damage size of the scratch defects on the surface of the high-reflectivity and high-transparency material based on the segmentation results.

[0103] Specifically, it includes the following steps:

[0104] S61. Based on the contour point set output by the segmentation mask, calculate the total scratch length using the Euclidean distance accumulation algorithm between adjacent points. The calculation formula is as follows:

[0105]

[0106] Where L is the scratch length based on pixel coordinates, n is the total number of contour points, k represents the index variable of the contour point set, and x k yk x represents the pixel coordinates of the k-th contour point extracted from the mask. k+1 y k+1 This represents the pixel coordinates of the (k+1)th contour point adjacent to point k.

[0107] S62. An evaluation is conducted by comparing the difference between the data detected in the defect range and the data actually measured. The evaluation methods include root mean square error, mean absolute error, coefficient of determination, and residual analysis. The calibration parameters are combined with the actual physical dimensions to generate a defect quantification report that conforms to the GB / T36259-2018 standard.

[0108] verify:

[0109] To verify the effectiveness of this invention in detecting surface defects in high-reflectivity and high-transparency materials, the obtained weight file was used to test surface defects in the test set. Furthermore, to evaluate the superiority and effectiveness of the proposed detection algorithm compared to current popular target detection models, this invention selected YOLOv5n, YOLOv8n, YOLOv11n, SSD, and Faster R-CNN algorithms and conducted comparative experiments under the same configuration and dataset conditions.

[0110] To further verify the effectiveness of the improved modules, this invention uses the original YOLOv11n model as a baseline and selects the DySnakeConv, ContextAggregation, and DSConv modules. Ablation experiments were conducted using different combinations of these improved modules. This invention uses two datasets for experimental verification: a standard dataset of mobile phone screen surface defects released by the Peking University Intelligent Robotics Open Laboratory, named D1; and a dataset of actual defects in mobile phone screens from industrial production lines, named D2, collected independently by this invention. Through these two datasets with different data distribution characteristics, combined with cross-dataset generalization experiments, the performance of the algorithm was comprehensively evaluated. The experimental results are shown in Table 1.

[0111] Table 1. Comparison of the improved models of this invention with other improvements to YOLOv11n.

[0112]

[0113] The results in Table 1 show that, under conditions of high dataset complexity, the improved network significantly outperforms other methods in all detection metrics, demonstrating its superior performance in detecting surface defects on mobile phone screens.

[0114] In summary, this invention is suitable for the efficient and accurate detection of surface defect features in highly reflective and transparent materials. It can perform zoned detection of defects to trace the source of defective instruments and can also fit the actual damage size of scratch defects on the surface of highly reflective and transparent materials. Furthermore, it should be noted that the application of deep learning in industrial inspection is not limited to mobile phone screen defect detection; it can also be applied to the surface defect detection of other industrial products, such as metal parts and plastic products. Therefore, this invention can also be applied to the surface defect detection of other industrial products.

[0115] This invention is not limited to the specific embodiments described above. Any modifications made by those skilled in the art based on the above concept without creative effort are within the scope of protection of this invention.

Claims

1. A method for detecting surface defects in high-reflectivity and high-transmittance materials based on an improved YOLOv11, characterized in that, Includes the following steps: S1. Collect images of surface defects on mobile phone screens in industrial production lines to construct an original surface defect image dataset; preprocess and label the original surface defect image dataset to generate a training dataset containing three types of defects: scratches, chipping, and cracks. S2. Build an improved YOLOv11 network model and train and optimize the parameters to construct a defect detection model; S3. Divide the input image into four independent detection regions on an average basis. For cross-region defects, adopt the bounding box overlap area weight allocation strategy and use the defect detection model to perform partition counting on the image. S4. By adjusting the lightweight network structure and optimizing the hardware acceleration, a surface defect detection network that meets the real-time requirements of industrial production lines is constructed, and the detection results and processing frame rate are output. S5. Perform pixel-level mask segmentation on scratch defects and extract contour coordinates and geometric features; S6. Fit the actual damage size of the scratch defects on the surface of the high-reflectivity and high-transparency material based on the segmentation results; In step S2, the construction of the improved YOLOv11 network model involves introducing the DySnakeConv module, the ContextAggregation module, and the DSConv lightweight module on top of YOLOv11. The specific steps include: S21. The C3K2 standard convolutional layer in the original Bottleneck structure of the YOLOv11 network is replaced with the DySnakeConv dynamic deformable convolutional module. High reflectivity interference is suppressed by introducing an illumination invariance preprocessing mechanism. S22. In the YOLOv11 network, a ContextAggregation module is integrated into the C2PSA feature output of layer 11. This ContextAggregation module introduces a local-global dual-branch context structure: for small defects <5×5 pixels, local window attention is used to capture fine-grained features; for large defects >20×20 pixels, a global multi-head attention mechanism is used to model long-range dependencies. Branch fusion is adaptively adjusted through a dynamic weight mapping function, defined as follows: Where x is the feature vector of the branch output. Calculate its L2 norm to measure feature significance; perform a nonlinear transformation on the norm result using a multilayer perceptron (MLP); Sigmoid function. The output is compressed to the (0,1) interval to generate weights that control the fusion ratio of local and global branches. ; S23. Replace the standard convolutional layer in the Neck section with a depth-separable convolutional module; In step S3, the input image is divided into four independent detection regions on an average basis. A bounding box overlap area weighting strategy is used for cross-region defects. The defect detection model is used to perform partition counting on the image. The specific steps include: S31. Divide the input image into four independent detection regions: upper left, lower left, upper right, and lower right. For defect targets distributed across regions, determine their assigned region based on the maximum weight allocation strategy of the overlap area between the bounding box and each region. The calculation formula is as follows: in, Represents bounding box B and partition The overlapping area, These are the bounding box coordinate parameters. This indicates four partitions: top left, bottom left, top right, and bottom right. The partition indicating the final attribution of the defect; S32. Using root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R²) 2 The residual analysis method was used to compare the statistical differences between the defect detection results and the actual measurement data to verify the accuracy of the zonal counting. In step S5, the pixel-level mask segmentation of the scratch defect includes the following steps: S51. Based on labeled scratch defect samples, multi-scale segmentation training data is generated through random rotation, affine transformation and brightness perturbation. S52. Use Labelme software to outline defects in mobile phone screen glass, generate a JSON format mask file, and then convert it into a YOLO format TXT file according to the COCO dataset specification. Each record contains the defect category number, normalized center coordinates, and bounding box size. S53. Call the batch inference interface of the defect detection model to perform detection on the segmented image and output the structured detection results. Save each record in the text file containing the coordinates of the defect center point. The bounding box dimensions (w, h) and confidence level p, whose numerical ranges satisfy the following: , ∈[0,1], w, h∈(0,1], p≥0.5; In step S6, the specific steps for fitting the actual damage size of the scratch defects on the surface of the high-reflectivity and high-transparency material based on the segmentation results include: S61. Based on the contour point set output by the segmentation mask, calculate the total scratch length using the Euclidean distance accumulation algorithm between adjacent points. The calculation formula is as follows: in, The scratch length is based on pixel coordinates. The total number of contour points. The index variable represents the set of contour points. , Indicates the first digit extracted from the mask. The pixel coordinates of each contour point , Indicates and The adjacent point The pixel coordinates of each contour point; S62. An evaluation is conducted by comparing the difference between the detected defect range and the actual measured data. The evaluation methods include root mean square error, mean absolute error, coefficient of determination, and residual analysis. The calibration parameters are mapped to the actual physical dimensions to generate a defect quantification report that conforms to the GB / T36259-2018 standard.

2. The method for detecting surface defects of high-reflectivity and high-transmittance materials based on the improved YOLOv11 according to claim 1, characterized in that, In step S1, the process of collecting surface defect images of mobile phone screens from an industrial production line, constructing an original surface defect image dataset, preprocessing and labeling the original surface defect image dataset, and generating a training dataset containing three types of defects: scratches, chipping, and cracks, includes the following steps: S11. Fix the relative spatial parameters of the moving camera and the mobile phone screen under test in a dark room environment. The camera is vertically installed on the top of the platform, and the mobile phone screen is horizontally placed on a black background board and fixed at the bottom of the platform. The equipment is wrapped with black light-blocking cloth to eliminate the interference of ambient stray light. Multiple strip light sources are evenly arranged at 45° above the screen to provide directional oblique lighting. Multi-angle images of surface defects on the mobile phone screen are collected to construct the original surface defect image dataset. S12. Preprocess the original surface defect image dataset, including format unification, size normalization, random rotation, flipping, and brightness adjustment operations, to generate an enhanced defect sample library. S13. Label the bounding boxes of the defect targets in the enhanced defect dataset, and define three types of defect labels: scratches, chipping, and cracks. Generate a label file containing the three types of defects and conforming to the PASCAL VOC format.

3. The method for detecting surface defects of high-reflectivity and high-transmittance materials based on the improved YOLOv11 according to claim 1, characterized in that, In step S2, the training optimization parameters include the following steps: S24. The surface defect dataset is divided into training set, validation set and test set in a 7:2:1 ratio using a random stratified sampling method to ensure a balanced distribution of each defect category. S25. Initialize the model parameters of the YOLOv11 network pre-trained on the COCO dataset, freeze the first 50 training epochs of the backbone network, and set the initial learning rate to 1×10. -4 The decay rate was 0.96; the backbone network was unfrozen after 50 cycles, and the learning rate was adjusted to 1×10. -5 And maintaining a decay rate of 0.96, the Adam optimizer was used for iterative training up to 150 epochs; S26. The improved YOLOv11 network model trained is validated using a validation set. The mean precision, recall, and false positive rate are calculated using the validation set. The optimal weights are selected and saved to the test set for final performance evaluation.

4. The method for detecting surface defects of high-reflectivity and high-transmittance materials based on the improved YOLOv11 according to claim 1, characterized in that, In step S4, the construction of a surface defect detection network that meets the real-time requirements of industrial production lines through lightweight network structure adjustment and hardware acceleration optimization includes the following steps: Based on the defect detection model, network layer fusion and INT8 quantization are performed using the TensorRT framework to reduce the computational complexity of the model. Depthwise separable convolutions are used to replace the standard convolutional layers in the Neck part, and channel pruning techniques are combined to remove redundant parameters, thereby reducing the number of model parameters and constructing a surface defect detection network that meets the real-time requirements of industrial production lines.

Images