A mobile phone screen defect detection method and system

Abstract

The application discloses a mobile phone screen defect detection method and system, relates to the technical field of industrial mobile phone screen defect detection, acquires a mobile phone screen defect data set, inputs the improved YOLOv8n model for training, and obtains a mobile phone screen defect detection model; the mobile phone screen image to be identified is input into the mobile phone screen defect detection model, and a defect detection result is obtained. On the basis of improving YOLOv8n, the mobile phone screen defect is detected, the detection accuracy is significantly improved, the size of the model is reduced, and the computing power is reduced to the maximum extent while maintaining high precision of the model.

Description

Technical Field

[0001] This invention relates to the field of mobile phone screen defect detection technology, and more specifically to a method and system for mobile phone screen defect detection. Background Technology

[0002] With the widespread use of smartphones around the world and the rapid development of mobile communication technology and smartphones themselves, smartphones have become an indispensable part of people's daily lives. As the primary medium for human-computer interaction, the integrity of the phone screen directly affects the user experience and the operational quality of the device. However, due to various factors during the manufacturing process, phone screens are prone to defects such as oil stains, scratches, and spots. These defects not only affect the user experience but also reduce the reliability and lifespan of the product. Therefore, inspecting the quality of phone screens is particularly necessary.

[0003] Currently, the main methods for detecting defects in mobile phone screens include manual inspection and machine vision inspection. While traditional manual visual inspection methods can provide highly accurate results and can be flexibly adjusted and processed according to specific situations to adapt to different types of mobile phone screen defects, this method requires a significant investment of manpower and suffers from drawbacks such as low work efficiency, susceptibility to subjective factors, and high time and effort consumption. In contrast, machine vision inspection methods can identify and analyze defects in real time, quickly, and accurately, and present the inspection results in the form of images or data, making it easier for production managers to understand the screen quality status and promptly identify and resolve problems. Driven by computer vision and deep learning technologies, achieving automated defect detection has become a hot topic. This automated inspection method greatly improves the efficiency and accuracy of defect detection, providing strong support for ensuring and improving the quality of mobile phone screens.

[0004] While existing technologies employ different methods to significantly improve detection accuracy, they fail to meet the demands of industrial defect detection due to the lack of lightweight design in current models.

[0005] Therefore, how to further lightweight the detection algorithm to facilitate industrial deployment and improve detection speed and accuracy are problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for detecting defects in mobile phone screens to solve the problems in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] On the one hand, a method for detecting defects in mobile phone screens includes the following specific steps:

[0009] Obtain a dataset of mobile phone screen defects;

[0010] An improved YOLOv8n model is constructed by adding a tiny target detection head to the original three detection heads of the YOLOv8n model. The shallow features of the backbone network are input, and the deep features are upsampled and then fused with the shallow features.

[0011] The mobile phone screen defect dataset is input into the improved YOLOv8n model for training to obtain the mobile phone screen defect detection model.

[0012] The mobile phone screen image to be identified is input into the mobile phone screen defect detection model to obtain the defect detection result of the mobile phone screen image to be identified.

[0013] Preferably, in the above-mentioned method for detecting defects in a mobile phone screen, the construction of the improved YOLOv8n model further includes: using partial convolutional modules to replace all C2f modules in the backbone network; the partial convolutional modules apply traditional convolutional modules to only a portion of the input channels, while the remaining channels remain unchanged; the partial convolutional modules treat the first or last consecutive channel as a representative of the overall access feature map; and pointwise convolutional modules are also introduced.

[0014] Preferably, in the above-mentioned method for detecting defects in a mobile phone screen, the construction of the improved YOLOv8n model further includes: replacing all C2f modules of the feature extraction network with GhostNetV2; GhostNetV2 uses decoupled fully connected attention to enhance the output features of the Ghost module, thereby capturing the long-term dependencies between different spatial pixels.

[0015] Preferably, in the above-mentioned mobile phone screen defect detection method, GhostNetV2 first runs the decoupled fully connected attention branch in parallel with the first Ghost module to enhance the extended features, and then sends the enhanced features to the second Ghost module to generate output features; the input features are then processed... The input X is passed to the Ghost module for generating output features and the decoupled fully connected module for generating attention map A, and a 1×1 convolution is used to transform the module's input X into the DFC's input Z; the final output... It is the product of the outputs of the two branches, as shown in the following formula:

[0016] O = Sigmoid(A) ⊙ V(X).

[0017] Preferably, in the above-mentioned mobile phone screen defect detection method, the SeaAttention module of SeaFormer is introduced into the tiny target detection head to perform weighted fusion of shallow feature maps.

[0018] Preferably, in the above-mentioned mobile phone screen defect detection method, the SeaAttention module performs global semantic extraction by squeezing axial attention and supplements local details by using a convolution-based detail enhancement kernel, and fuses the enhanced features with the features given by squeezing axial attention.

[0019] Preferably, in the above-mentioned method for detecting defects in a mobile phone screen, the specific steps for obtaining the features given by the extrusion axial direction are as follows:

[0020] Given features q, k, v are linear projections of X, i.e., q = W q X, k = W k X, v = W v X, where W q , These are learnable weights; firstly, horizontal and vertical compression are achieved by averaging the query feature maps in the horizontal and vertical directions, respectively, as shown in the following formula:

[0021]

[0022] Where z →(·) This represents the dimension of the permutation of the given tensor z. Let q represent a vector in which all elements are equal to 1; the squash operation on q is also repeated on k and v, thus ultimately resulting in q. (h) ,k (h) , q (v) ,k (v) ,

[0023] For squeeze axial attention, each location in the feature map is processed; firstly, through location embedding... Come for q (h) ,k (h) Incorporating location information and embedding learnable parameters Linear interpolation is obtained; similarly, Applicable to q (v) ,k (v) Therefore, position-aware compression axial attention is represented as:

[0024]

[0025] Preferably, in the above-mentioned method for detecting defects in a mobile phone screen, the specific steps for obtaining the enhanced features are as follows:

[0026] Another linear projection representation is obtained from the feature map X. and And concatenate along the channel dimension, passing to a block consisting of 3×3 depthwise convolutions and batch normalization; aggregate auxiliary local details from q, k, v using 3×3 convolutions; use activation functions and batch normalized linear projections to (2C) qk +C v The 1)-dimensional features are compressed to C-dimensionality, and detail enhancement weights are generated.

[0027] On the other hand, a mobile phone screen defect detection system includes:

[0028] The dataset acquisition module is used to acquire mobile phone screen defect datasets.

[0029] An improved model module is constructed, which builds an improved YOLOv8n model. Based on the original three detection heads of the YOLOv8n model, an ultra-small target detection head for small target detection is added. The shallow features of the backbone network are input, and the deep features are upsampled and then fused with the shallow features.

[0030] The model training module is used to input the mobile phone screen defect dataset into the improved YOLOv8n model for training, so as to obtain a mobile phone screen defect detection model.

[0031] The mobile phone screen defect detection module is used to input the mobile phone screen image to be identified into the mobile phone screen defect detection model to obtain the defect detection result of the mobile phone screen image to be identified.

[0032] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method and system for detecting defects in mobile phone screens. It improves YOLOv8n by introducing PConv into the backbone network of YOLOv8n, making full use of the terminal's computing resources, reducing floating-point operations and memory access requirements, making the entire network architecture more lightweight. At the same time, GhostNetV2 is introduced into the feature extraction network, which improves the model's expressive power while reducing the number of model parameters, achieving a better trade-off between accuracy and speed. Furthermore, the SeaAttention attention module is introduced into the newly added tiny object detection head, which adds local details after global semantic extraction, resulting in better performance in model feature extraction and model detection.

[0033] This invention, based on the improvement of YOLOv8n, performs defect detection on mobile phone screens, which not only improves the speed and accuracy of detection but also reduces computing power. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the method steps of the present invention;

[0036] Figure 2(a) is a schematic diagram of the structure of the traditional YOLOv8 model;

[0037] Figure 2(b) is a schematic diagram of the structure of the improved YOLOv8 model (PSG-YOLO) of this invention;

[0038] Figure 3 This is a schematic diagram of the structure of the minimum target detection branch of the present invention;

[0039] Figure 4(a) is a schematic diagram of the structure of PConv of the present invention;

[0040] Figure 4(b) is a schematic diagram of the PConv+PWConv structure of the present invention;

[0041] Figure 5 This is a schematic diagram of the structure of GhostNetV2 of the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of the SeaAttention method of the present invention;

[0043] Figure 7 This is an example image of the mobile phone screen defect dataset in Example 2;

[0044] Figure 8 The image shows an example of the detection results from the PSG-YOLO of this invention.

[0045] Figure 9(a) is a visualization heatmap of the minimal target detection head during model training in Example 2;

[0046] Figure 9(b) is a visualization heatmap of the SeaAttention module added to the minimal target detection head during model training in Example 2. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] What needs to be understood is:

[0049] YOLO has quickly gained popularity among AI enthusiasts due to its speed and accuracy. As the latest version of YOLO, YOLOv8 is a cutting-edge, state-of-the-art (SOTA) model. Building on the success of previous YOLO versions, it introduces new features and improvements, further enhancing performance and flexibility. One of YOLOv8's key features is its scalability; it not only supports all previous YOLO versions but also allows for easy conversion and performance comparison between different versions. The reasons for choosing YOLOv8 as the object detection algorithm in this invention are as follows: (1) YOLOv8 adopts a new convolutional model C2f, which greatly reduces model parameters and computational load, and improves running efficiency; (2) PConv is added to the feature extraction network of YOLOv8, which simultaneously reduces redundant computation and memory access, and achieves efficient extraction of spatial domain features, thereby accelerating the speed of model training and inference; (3) The GhostNetV2 lightweight module is added to the feature extraction network of YOLOv8, which enables the model to capture long-distance dependencies between pixels at different spatial locations, better understand the semantic information of the image, and improve the model's performance in complex scenes; (4) By adding a branch for the detection of tiny objects and incorporating the SeaAttention module, the model's attention to the target is more focused and sensitive, achieving the best trade-off between accuracy and inference latency. Due to its cutting-edge features and excellent performance, YOLOv8 has become an important reference model in the fields of object detection and image segmentation, bringing new possibilities to the development of image processing technology, and is suitable for the task of mobile phone screen defect detection in this invention.

[0050] This invention introduces a new branch specifically for detecting very small defects, specifically for detecting tiny targets. Based on YOLOv8n, deeper features are upsampled an additional time and then fused with shallower features to form this additional branch. The receptive field of this branch is [amount missing] times that of YOLOv8n's small target detection branch. It is more sensitive to extremely small targets and can accurately detect very small target defects.

[0051] In the backbone network, PConv replaces all C2f modules. As a novel partial convolution, PConv reduces computational cost and memory access to some extent. This module cleverly utilizes redundant information in the feature map, applying traditional convolution to only a small portion of the input channels, while leaving the remaining channels unchanged. For continuous or regular memory access, PConv treats the first or last consecutive channel as representative of the overall accessed feature map. It also incorporates pointwise convolution to ensure effective extraction of spatial features.

[0052] In the feature extraction network, all C2f modules are replaced with GhostNetV2. This module employs Decoupled Fully Connected (DFC) attention to enhance the output features of the Ghost modules, thereby capturing long-term dependencies between pixels in different spatial locations and further improving the expressive power of the lightweight model. GhostNetV2 first runs the DFC attention branch in parallel with the first Ghost module to enhance the expanded features, and then sends the enhanced features to the second Ghost module to generate the output features. With the same input, the Ghost modules and DFC attention are two parallel branches, extracting information from two different perspectives. The result is the element-wise product of two features from the Ghost modules and the attention value from the DFC attention module. The computation of each attention value involves a large range of image patches so that the output features can contain information from these patches.

[0053] In the minimal object detection branch, the SeaAttention module is introduced. It performs global semantic extraction through concise axial squeezing attention and supplements local details with an efficient convolution-based detail enhancement kernel. After global semantic extraction, local details are added, resulting in better performance for model feature extraction and model detection.

[0054] Example 1:

[0055] This embodiment discloses a method for detecting defects in mobile phone screens. (See also...) Figure 1 This includes the following steps:

[0056] Step 1: Obtain the mobile phone screen defect dataset;

[0057] Step 2: Construct an improved YOLOv8n model. On the basis of the original three detection heads of the YOLOv8n model, add a tiny target detection head for tiny target detection. Input the shallow features of the backbone network, upsample the deep features, and then fuse them with the shallow features.

[0058] Step 3: Input the mobile phone screen defect dataset into the improved YOLOv8n model for training to obtain the mobile phone screen defect detection model;

[0059] The improved YOLOv8n model is based on the traditional YOLOv8n model (see Figure 2(a)) and the following four improvements are made to obtain the PSG-YOLO model (see Figure 2(b)):

[0060] (1) Add a branch for detecting extremely small targets, such as Figure 3 As shown. The receptive field of this branch is the same as that of the small object detection branch in YOLOv8. The input mainly comes from the shallow features of the backbone network. Shallow features are often sensitive to small-scale objects or details in the image and can capture some subtle changes and details in the image, making them very suitable for the detection of small targets.

[0061] (2) PConv is used in the backbone network of the traditional YOLOv8n model to extract spatial features more efficiently by reducing redundant computations and memory accesses. See Figure 4(a). PConv cleverly utilizes redundant information in the feature maps, applying traditional convolution to only a small portion of the input channels, while leaving the remaining channels unchanged. For continuous or regular memory accesses, PConv treats the first or last consecutive channel as representative of the overall accessed feature map. To ensure generality, PConv defaults to having the same number of channels in the input and output feature maps.

[0062] Referring to Figure 4(b), pointwise convolution is further fused into PConv, which improves the expressive power of the feature map.

[0063] (3) GhostNetV2 is used in the backbone network of the traditional YOLOv8n model. See [link / reference] Figure 5 GhostNetV2 employs decoupled fully connected attention to enhance the output features of the Ghost module, thereby capturing long-term dependencies between pixels in different spatial locations and further improving the expressive power of the lightweight model. This involves converting input features... The input X is passed to the Ghost module for generating output features and the decoupled fully connected attention module for generating attention map A. A 1×1 convolution is then used to transform the module's input X into the DFC's input Z. The module's final output is... It is the product of the outputs of the two branches:

[0064] O = Sigmoid(A) ⊙ V(X),

[0065] (4) The SeaAttention module performs global semantic extraction through concise squeeze axial attention and supplements local details through an efficient convolution-based detail enhancement kernel.

[0066] See Figure 6 The SeaAttention module includes:

[0067] 1) Given features q, k, v are linear projections of X, i.e., q = W q X, k = W k X, v = W v X, where W q , These are learnable weights. First, horizontal and vertical compression are achieved by averaging the query feature maps in the horizontal and vertical directions, respectively:

[0068]

[0069] Where z →(·) This represents the dimension of the permutation of the given tensor z. This represents a vector in which all elements are equal to 1.

[0070] 2) For squeeze axial attention, each location in the feature map is processed. First, through location embedding... Come for q (h) ,k (h) Incorporate location information:

[0071]

[0072] 3) Apply an auxiliary convolution-based kernel to enhance spatial details and obtain another linear projection representation from the feature map X. and The concatenation is performed along the channel dimension and passed to a block consisting of 3×3 depthwise convolutions and batch normalization. Using 3×3 convolutions, auxiliary local details can be aggregated from q, k, v. Linear projection using activation functions and batch normalization is applied to (2C) qk +C v The 1D features are compressed to CD, and detail enhancement weights are generated. Finally, the enhanced features are fused with the features given by the squeeze axial attention.

[0073] In this embodiment, YOLOv8 is an open-source, single-stage object detection algorithm developed by Ultralytics.

[0074] Step 4: Input the mobile phone screen image to be identified into the mobile phone screen defect detection model to obtain the defect detection result of the mobile phone screen image to be identified.

[0075] Example 2

[0076] This embodiment demonstrates the beneficial effects of the method of the present invention by performing target detection on the mobile phone screen image to be identified through specific experiments.

[0077] Experimental Platform: This experiment was conducted on a Windows operating system, using PyCharm Community as the integrated development environment (IDE) and implementing the model framework in Python. The main hardware configuration was: Windows 10 64-bit operating system, Intel(R) Core(TM) i5-10400F@2.90GHz CPU, Nvidia GeForce RTX 2080 Ti (22GB) GPU, and 32GB of RAM. The deep learning development environment consisted of: PyCharm 2023.5.17, Python 3.8, CUDA 12.1, cuDNN 7.6.5, and PyTorch 2.0.1.

[0078] Dataset: The dataset selected is the open mobile phone screen defect dataset released by the Peking University Development Lab. This dataset has three defect types: oil stains (Oil), scratches (Scr), and spots (Sta). Copy-paste data augmentation was performed on this dataset. Figure 7 Example images selected from the aforementioned mobile phone screen defect dataset.

[0079] Experimental setup: YOLOv8n was used as the baseline model. The batch size was set to 16, the number of epochs to traverse the entire training dataset was 300, the learning rate was 0.01, the input image size was 640, the number of workers used during data loading was 8, and SGD was used as the optimizer.

[0080] Experimental Results: This experiment compared the performance of the YOLOv8n algorithm after incorporating this improvement, and also compared it with the results of YOLOv5, YOLOv6, and YOLOv7tiny. Four metrics were considered: parameter count, GFLOPs, mAP@.5, and mAP@.95, comparing model performance and complexity. The comparison revealed that the proposed PSG-YOLO model improves detection accuracy with only a slight increase in parameters and GFLOPs (see Table 1 for details). The final detection results are shown below. Figure 8 .

[0081] Table 1 Comparison of PSG-YOLO with various detector versions

[0082]

[0083] This embodiment also implemented each of the four steps on the YOLOv8n detector, and the experimental results are shown in Table 2. For a detailed model structure heatmap, please refer to... Figures 9(a)-9(b)As can be clearly seen, after the above improvements were added, the final detection head can accurately focus on extremely small targets.

[0084] Table 2 PSG-YOLO Ablation Experiment

[0085]

[0086] Another embodiment of the present invention discloses a mobile phone screen defect detection system, comprising:

[0087] The dataset acquisition module is used to acquire mobile phone screen defect datasets.

[0088] An improved model module is constructed, which builds an improved YOLOv8n model. Based on the original three detection heads of the YOLOv8n model, an ultra-small target detection head for small target detection is added. The shallow features of the backbone network are input, and the deep features are upsampled and then fused with the shallow features.

[0089] The model training module is used to input the mobile phone screen defect dataset into the improved YOLOv8n model for training, so as to obtain a mobile phone screen defect detection model.

[0090] The mobile phone screen defect detection module is used to input the mobile phone screen image to be identified into the mobile phone screen defect detection model to obtain the defect detection result of the mobile phone screen image to be identified.

[0091] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0092] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting defects in mobile phone screens, characterized in that, The specific steps are as follows: Obtain a dataset of mobile phone screen defects; An improved YOLOv8n model is constructed by adding a tiny target detection head to the original three detection heads of the YOLOv8n model. The shallow features of the backbone network are input, and the deep features are upsampled and then fused with the shallow features. The mobile phone screen defect dataset is input into the improved YOLOv8n model for training to obtain the mobile phone screen defect detection model. The mobile phone screen image to be identified is input into the mobile phone screen defect detection model to obtain the defect detection result of the mobile phone screen image to be identified.

2. The method for detecting defects in a mobile phone screen according to claim 1, characterized in that, The improved YOLOv8n model also includes: replacing all C2f modules with partial convolutional modules in the backbone network; the partial convolutional modules apply traditional convolutional modules to only a portion of the input channels, while the remaining channels remain unchanged; the partial convolutional modules treat the first or last consecutive channel as a representation of the overall access feature map; and pointwise convolutional modules are also introduced.

3. The method for detecting defects in a mobile phone screen according to claim 1, characterized in that, The improved YOLOv8n model also includes: replacing all C2f modules of the feature extraction network with GhostNetV2; GhostNetV2 uses decoupled fully connected attention to enhance the output features of the Ghost module, thereby capturing long-term dependencies between different spatial pixels.

4. The method for detecting defects in a mobile phone screen according to claim 3, characterized in that, The GhostNetV2 module first runs the decoupled fully connected attention branch in parallel with the first Ghost module to enhance the extended features, and then sends the enhanced features to the second Ghost module to generate output features; the input features are then processed by the second Ghost module. The input X is passed to the Ghost module for generating output features and the decoupled fully connected module for generating attention map A, and a 1×1 convolution is used to transform the module's input X into the DFC's input Z; the final output... It is the product of the outputs of the two branches, as shown in the following formula: O = Sigmoid(A) ⊙ V(X).

5. The method for detecting defects in a mobile phone screen according to claim 1, characterized in that, The SeaAttention module from SeaFormer is introduced into the tiny object detection head to perform weighted fusion of shallow feature maps.

6. The method for detecting defects in a mobile phone screen according to claim 5, characterized in that, The SeaAttention module performs global semantic extraction by squeezing axial attention and supplements local details by using a convolution-based detail enhancement kernel, fusing the enhanced features with the features given by squeezing axial attention.

7. A method for detecting defects in a mobile phone screen according to claim 6, characterized in that, The specific steps to obtain the extrusion axial direction feature are as follows: Given features q, k, v are linear projections of X, i.e., q = W q X, k = W k X, v = W v X, where W q , These are learnable weights; firstly, horizontal and vertical compression are achieved by averaging the query feature maps in the horizontal and vertical directions, respectively, as shown in the following formula: Where z →(·) This represents the dimension of the permutation of the given tensor z. Let q represent a vector in which all elements are equal to 1; the squash operation on q is also repeated on k and v, thus ultimately resulting in q. (h) ,k (h) , q (v) ,k (v) , For squeeze axial attention, each location in the feature map is processed; firstly, through location embedding... Come for q (h) ,k (h) Incorporating location information and embedding learnable parameters Linear interpolation is obtained; similarly, Applicable to q (v) ,k (v) Therefore, position-aware compression axial attention is represented as:

8. The method for detecting defects in a mobile phone screen according to claim 7, characterized in that, The specific steps to obtain the enhanced features are as follows: Another linear projection representation is obtained from the feature map X. and And concatenate along the channel dimension, passing to a block consisting of 3×3 depthwise convolutions and batch normalization; aggregate auxiliary local details from q, k, v using 3×3 convolutions; use activation functions and batch normalized linear projections to (2C) qk +C v The 1)-dimensional features are compressed to C-dimensionality, and detail enhancement weights are generated.

9. A mobile phone screen defect detection system, characterized in that, include: The dataset acquisition module is used to acquire mobile phone screen defect datasets. An improved model module is constructed, which builds an improved YOLOv8n model. Based on the original three detection heads of the YOLOv8n model, an ultra-small target detection head for small target detection is added. The shallow features of the backbone network are input, and the deep features are upsampled and then fused with the shallow features. The model training module is used to input the mobile phone screen defect dataset into the improved YOLOv8n model for training, so as to obtain a mobile phone screen defect detection model. The mobile phone screen defect detection module is used to input the mobile phone screen image to be identified into the mobile phone screen defect detection model to obtain the defect detection result of the mobile phone screen image to be identified.

Images