Edge defect detection method, device and equipment
The integration of traditional edge detection and deep learning algorithms addresses the challenge of complex edge detection in mobile phone cover plates, improving accuracy and yield through combined defect classification.
Patent Information
- Application Number
- CN202210946591.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-09
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2042-08-09
AI Technical Summary
Existing edge detection methods in industrial automation machine vision struggle with accuracy due to diverse product edges, 2.5D/3D curved surfaces, and background interference, particularly in backlight fields, leading to insufficient detection of complex edge defects like those found in mobile phone cover plates.
A method combining traditional edge detection techniques with deep learning algorithms, utilizing image preprocessing, ROI extraction, and Squeezenet model classification to enhance edge defect detection accuracy.
Improves edge defect detection accuracy by integrating traditional methods with deep learning, effectively identifying and classifying defects in mobile phone cover plates, enhancing production yield.
Smart Images
Figure CN115393290B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of industrial automation machine vision detection technology, and in particular to an edge defect detection method, device, computer equipment and computer-readable storage medium. Background Art
[0002] With the rapid development of the manufacturing industry, people's requirements for product quality are increasing, and the surface quality of products has an increasingly important impact on the commercial value of products. Surface defect detection has become an important part of the product quality control process, which can effectively improve product quality and production efficiency. Surface defect detection is widely used in cloth defect detection, workpiece surface quality detection, aerospace field, etc. Traditional algorithms can work well for regular defects and occasions with relatively simple scenes, but they are no longer applicable for occasions with unclear features, diverse shapes, and chaotic scenes. In recent years, recognition algorithms based on machine learning have become more and more mature, and many companies have begun to try to apply deep learning algorithms to industrial occasions.
[0003] Machine vision surface defect detection technology based on optical image sensing has gradually replaced manual visual inspection methods and become an important means of surface defect detection. This method has the advantages of automation, non-contact, fast speed, high precision, and good stability. The emergence of this technology has greatly improved the efficiency of production operations, avoided the accuracy of detection results affected by operating conditions and subjective judgment, and achieved better and more accurate surface defect detection, and faster identification of product surface defects. Product surface defect detection is a type of machine vision technology, which uses computer vision to simulate the function of human vision, collect and process images from specific objects, calculate, and finally perform actual detection, control and application. Product surface defect detection is an important part of machine vision detection, and the accuracy of its detection will directly affect the final quality of the product.
[0004] In the field of industrial inspection, edge detection is an important part of detecting product defects. However, due to the diversity of product edges, 2.5D curved surfaces or 3D curved surfaces, or R angles, U-shaped grooves, S-shaped angles, etc. after laser cutting and grinding, the difficulty of edge detection is increased. In addition, most edge detections are performed under backlight fields, generally using traditional digital image processing technology. However, under backlight fields, only products with optically easy imaging such as edge chipping can be processed, and more process morphological defects cannot be detected. In the relevant technical solutions for edge detection, the accuracy of edge detection is not high due to background interference or insufficient algorithm accuracy. For example, there is a clear lack of accuracy in edge defect detection for mobile phone cover plates. Summary of the invention
[0005] Based on this, it is necessary to propose an edge defect detection method, device, computer equipment and computer-readable storage medium to address the above problems.
[0006] In the first part of the present invention, a method for detecting edge defects is provided, the method comprising:
[0007] Acquire a target image of the cover plate to be inspected, wherein the target image is an image of the cover plate to be inspected;
[0008] Based on a first defect detection method, defect detection is performed on the target image to obtain a first edge defect in the target image;
[0009] Based on a second defect detection method, defect classification is performed on the first edge defect in the target image to determine a second edge defect in the target image; wherein the second defect detection method is a defect detection method based on deep learning;
[0010] The first edge defect and the second edge defect are screened to determine a defect detection result of the cover plate to be detected.
[0011] Optionally, the method further includes:
[0012] The target image is preprocessed based on a preset image filtering algorithm; wherein the preset image filtering algorithm includes one or more combinations of mean filtering, median filtering, Markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, adaptive filtering, and Hilbert filtering.
[0013] Optionally, the target image is a grayscale image; after the step of obtaining the target image of the cover to be inspected, it also includes: obtaining an ROI area in the target image, the ROI area including an ROI image area in the target image corresponding to a vertical edge, a horizontal edge and an R-angle edge of the cover to be inspected.
[0014] Optionally, the step of performing defect detection on the target image based on the first defect detection method to obtain a first edge defect in the target image also includes: performing a first defect detection on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be detected to obtain a first edge defect in the target image, wherein the first edge defect includes an edge defect in the vertical direction and / or the horizontal direction; performing a first defect detection on the R-corner edge of the cover plate to be detected to obtain a first edge defect in the target image, wherein the first edge defect includes an edge defect on the R-corner.
[0015] Optionally, the step of performing a first defect detection on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be inspected to obtain a first edge defect in the target image also includes: binarizing the target image according to a preset first grayscale threshold to obtain a binarized image corresponding to the target image; dilating the binarized image corresponding to the target image to obtain a corresponding first mask image; performing edge detection on the target image to obtain a first defect image in the target image; performing 8-neighborhood contour tracking on the target image to determine at least one inflection point, and performing a fitting operation on the at least one inflection point; performing defect detection based on the fitting result to obtain a second defect image; and determining the first defect feature in the vertical direction and / or horizontal direction based on the first defect image, the second defect image and the first mask image.
[0016] Optionally, the step of performing a first defect missing part on the R-angle edge of the cover plate to be inspected to obtain a first edge defect in the target image further includes: performing a first nonlinear gradient processing on the target image to obtain a first nonlinear gradient map; wherein the first nonlinear gradient processing is a nonlinear gradient processing with a kernel of 15*15; performing a binarization processing on the first nonlinear gradient map according to a preset first gradient threshold to obtain a binarized image corresponding to the first nonlinear gradient map; performing a defect merging processing according to a preset first pixel radius based on the binarized image corresponding to the nonlinear gradient map, and determining a second mask image according to the image after defect merging; performing a second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second nonlinear gradient processing is a nonlinear gradient processing with a kernel of 3*3; performing a binarization processing on the second nonlinear gradient map according to a preset second gradient threshold to obtain a binarized image corresponding to the second nonlinear gradient map; performing a contour tracking processing and a hole detection processing on the binarized image corresponding to the second nonlinear gradient map, and performing a defect merging processing based on a preset second pixel radius to obtain a third defect image; and determining the first edge defect on the R angle based on the third defect image and the second mask image.
[0017] Optionally, the deep learning model is obtained by training small clip images containing defects of training samples; the deep learning model is a Squeezenet model, the Squeezenet model includes a Squeeze layer and an Expand layer, the Squeeze layer includes a convolution layer with a 1*1 convolution kernel, and the Expand layer includes a convolution layer with a 1*1 convolution kernel and a convolution layer with a 3*3 convolution kernel; the step of classifying the first edge defect in the target image based on the second defect detection method and determining the second edge defect in the target image also includes: inputting the first edge defect into the Squeeze layer, and then inputting the output result of the Squeeze layer into the Expand layer, and using the output result of the Expand layer as the second edge defect.
[0018] Optionally, the step of screening the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be inspected further includes: screening the first edge defect and the second edge defect based on preset feature screening expressions corresponding to different classification defects, and determining the defect detection result under each classification defect.
[0019] In the second part of the present invention, an edge defect detection device is provided, the device comprising:
[0020] An image acquisition module, used to acquire a target image of the cover plate to be detected, wherein the target image is an image of the cover plate to be detected;
[0021] A first defect detection module, configured to perform defect detection on the target image based on a first defect detection method, and obtain a first edge defect in the target image;
[0022] A second defect detection module, configured to classify the first edge defect in the target image based on a second defect detection method, and determine a second edge defect in the target image; wherein the second defect detection method is a defect detection method based on deep learning;
[0023] The defect fusion screening module is used to screen the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be detected.
[0024] In the third part of the present invention, a computer device is provided, which includes a processor and a memory, wherein the memory is used to store a computer program; the processor is used to execute the steps of the edge defect detection method described in the first aspect according to the computer program.
[0025] In a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is used to execute the steps of the edge defect detection method as described in the first aspect above.
[0026] The embodiments of the present invention have the following beneficial effects:
[0027] After adopting the above-mentioned edge defect detection method, device, computer equipment and computer-readable storage medium, when it is necessary to detect the edge features of the mobile phone glass cover plate, first obtain the corresponding target image, then based on the first defect detection method, perform defect detection on the target image to obtain the first edge defect in the target image; based on the second defect detection method, classify the first edge defect in the target image and determine the second edge defect in the target image; wherein the second defect detection method is a defect detection method based on deep learning; the first edge defect and the second edge defect are screened to determine the defect detection result of the cover plate to be detected. In other words, through the edge defect detection method, device, computer equipment and computer-readable storage medium, preliminary defect classification is performed based on the traditional first defect detection method, and the defects are further classified based on the second defect detection method based on deep learning, combining the advantages of traditional algorithms and deep learning algorithms, improving the accuracy of edge detection, and improving the yield rate of mobile phone glass cover plate generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0029] in:
[0030] Figure 1 is a schematic flow chart of an edge defect detection method in an embodiment;
[0031] Figure 2 (A)-(F) are schematic diagrams of performing first defect detection on a target image to obtain first edge defects in the vertical / horizontal direction in an embodiment;
[0032] Figure 3 (A)-(F) are schematic diagrams of performing a second defect detection on a target image to obtain an image of a first edge defect on an R corner in one embodiment;
[0033] Figure 4A schematic diagram of a process of acquiring a first edge feature in a vertical / horizontal direction based on a first defect detection method in an embodiment;
[0034] Figure 5 It is a schematic diagram of a process of obtaining a second edge feature on an R corner based on a second defect detection method in one embodiment;
[0035] Figure 6 A schematic diagram of the structure of a Squeezenet model in one embodiment;
[0036] Figure 7 A schematic diagram of the calculation process of the Squeezenet model in one embodiment;
[0037] Figure 8 A schematic diagram of a defect screening process based on a feature screening expression in one embodiment;
[0038] Fig. 9 is a schematic structural diagram of an edge defect detection device in one embodiment;
[0039] Fig.10 The figure is a schematic diagram of the structure of a computer device for running the above-mentioned edge defect detection method in one embodiment. DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] In an embodiment of the present invention, in order to detect edge defects of a mobile phone cover, an edge defect detection method is proposed by acquiring an image of the cover to be detected and then detecting edge defects based on the image, so as to detect whether there are defects on the edge of the mobile phone cover.
[0042] For details, see Figure 1 , a schematic flow chart of an edge defect detection method provided by an embodiment of the present invention is given, wherein the edge defect detection method includes: Figure 1 Steps S101-S104 shown:
[0043] Step S101: acquiring a target image of a cover plate to be inspected, wherein the target image is an image of the cover plate to be inspected.
[0044] In this embodiment, the cover plate to be inspected is a mobile phone glass cover plate, which is a cover plate in the production line of mobile phone glass cover plates. Here, it is necessary to capture images of the cover plate to be inspected for defect detection. Among them, an image acquisition device (such as a CCD device) can be set on the production line to capture images of the cover plate to be inspected for defect detection. Among them, the image captured here is an image captured in a dark field lighting mode of the cover plate to be inspected, and the target image is a grayscale image.
[0045] Furthermore, in this embodiment, for the cover plate to be inspected, it is necessary to perform edge defect detection on the edges in the vertical direction, the horizontal direction, and the R corner (rounded corner) respectively. In order to perform defect detection by partition, it is necessary to extract the ROI area in the vertical direction, the horizontal direction, and the R corner respectively, and then perform defect detection based on the ROI area respectively. That is to say, for the acquired target image, the ROI area in the target image is obtained from the directions of the vertical edge, the horizontal edge, and the R corner edge respectively, and the ROI area includes the ROI image area in the target image corresponding to the vertical edge, the horizontal edge, and the R corner edge of the cover plate to be inspected.
[0046] Furthermore, in order to improve the accuracy of subsequent image processing, in this step, it is also necessary to pre-process the target image based on a preset image filtering algorithm to remove useless information in the image while preventing the image from losing useful detail information. The preset image filtering algorithm includes one or more combinations of mean filtering, median filtering, Markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, adaptive filtering, and Hilbert filtering, which are not specifically limited in this embodiment.
[0047] Step S102: Based on a first defect detection method, defect detection is performed on the target image to obtain a first edge defect in the target image.
[0048] It should be noted that, in this embodiment, two defect detection methods are required to detect edge defects of the cover plate to be detected, including a first defect detection method and a second defect detection method. The first defect detection method is a traditional defect detection method, and the second defect detection method is a defect detection method based on deep learning. That is to say, in this embodiment, defect detection is performed by using a traditional defect detection method and a defect detection method based on deep learning respectively to improve the accuracy of edge defect detection.
[0049] Specifically, in this step, the first defect detection method is a traditional edge defect detection method, which completes the detection of edge defects in each ROI area in the target image to obtain the first edge defect in the target image.
[0050] Edge information is a very representative image feature. The purpose of edge detection is to find a set of pixels in an image with drastic changes in brightness, which often appear as contours. If the edges in an image can be accurately measured and located, then it means that the actual object can be located and measured, including the area of the object, the diameter of the object, the shape of the object, etc. In image acquisition, the following four situations will form an edge when they appear in the image. 1) Depth discontinuity; 2) Surface direction discontinuity; 3) Different object materials (this will result in different light reflection coefficients); 4) Different lighting in the scene.
[0051] In a specific embodiment, a first defect detection is performed on the target image, and the defect features in the obtained target image include but are not limited to position information, defect area, circumference, length and width, roundness, ellipse flatness, aspect ratio, duty cycle, and one or more of other edge defect features such as other contour features and grayscale features.
[0052] The above-mentioned multiple edge defect characteristics are described below respectively.
[0053] Position information: represents the coordinate (x, y) distribution information of the defect on the glass image, which helps to distinguish whether the defect is inside the glass surface or on the edge of the glass;
[0054] Defect area: represents the effective pixel area of the defect (s = Pi*R*R or s=minimum enclosing rectangle w*h);
[0055] Length and width: represent the length and width of the minimum enclosing rectangle;
[0056] Roundness: Let p be the center point of the region (particle), p_i be all the pixels on the contour, and F be the contour area (here refers to the number of contour pixels, not the area of the area enclosed by the contour);
[0057] Ellipse flatness: represents the ratio of the major and minor axes of the defect circumscribed ellipse;
[0058] Aspect ratio: represents the ratio of the length to the width of the minimum circumscribed rectangle of the defect;
[0059] Duty cycle: represents the ratio of the effective pixel area of the defect to the area of the minimum enclosing rectangle.
[0060] In the process of detecting the first edge defect, it is necessary to perform defect detection from the vertical direction, horizontal direction and R angle respectively. Because the defect characteristics of the R angle in the vertical direction and horizontal direction are different, different defect detection methods need to be adopted to improve the accuracy of edge defect detection.
[0061] Specifically, the above-mentioned step of performing defect detection on the target image based on the first defect detection method to obtain a first edge defect in the target image also includes: performing a first defect detection on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be detected to obtain a first edge defect in the target image, and the first edge defect includes edge defects in the vertical direction and / or the horizontal direction; performing a first defect detection on the R-corner edge of the cover plate to be detected to obtain a first edge defect in the target image, and the first edge defect includes an edge defect on the R-corner.
[0062] The following describes how to detect edge defects in the vertical and horizontal directions, as well as edge defects at the R angle.
[0063] Specifically, in the vertical and / or horizontal direction, during the detection of the first edge defect, the target image is binarized according to a preset first grayscale threshold to obtain a binarized image corresponding to the target image; the binarized image corresponding to the target image is dilated to obtain a corresponding first mask image; edge detection is performed on the target image to obtain a first defect image in the target image; 8-neighborhood contour tracking is performed on the target image to determine at least one inflection point, and a fitting operation is performed on the at least one inflection point; defect detection is performed based on the fitting result to obtain a second defect image; based on the first defect image, the second defect image and the first mask image, the first defect feature in the vertical and / or horizontal direction is determined.
[0064] Assume that the width of the target image is W, the height is H, and the target image is a set of pixel points I (i, j). For details, see Figure 2 (A) shown.
[0065] The target image is binarized using a preset first grayscale threshold T1 to obtain a binarized image corresponding to the target image.
[0066] Dilation is performed on the binary image corresponding to the target image. The binary image corresponding to the target image is regarded as a defect map, and then the defects in the binary image are merged with a preset dilation radius (for example, 10 pixels). Then, the area corresponding to the defect with the largest contour is selected from the merged defects to determine the corresponding area mask Mask (first mask image). For details, please refer to Figure 2 (B) shown.
[0067] Perform edge detection on the target image to obtain the first defect image corresponding to the target image. The Prewitt edge detection operator can be used to process the target image. First, horizontal filtering is performed, and then vertical filtering is performed to generate the first defect image Prewitt_Map. For details, please refer to Figure 2 (C) In the Prewitt edge detection operator, the grayscale values of the eight pixels in the surrounding neighborhood are used to estimate the center gradient. The corresponding gradient calculation formula is as follows:
[0068]
[0069] Among them, the convolution kernel of the Prewitt edge detection operator is:
[0070]
[0071] Perform 8-neighborhood contour tracking on the target image and find the Open_Line graph. For details, see Figure 2 As shown in (D), the disconnected Open Line is subjected to inflection point detection (Ramer-Douglas–Peucker algorithm, also known as the Ramer-Douglas-Peucker algorithm, iterative adaptive point algorithm), and at least one inflection point is obtained recursively. Then, the contour lines corresponding to the inflection points are sorted, and the baseline Line_0 is calculated by segment fitting. The points on the contour line corresponding to at least one inflection point are compared with the baseline, and defects are pre-selected based on the preset threshold T2 to generate the second defect image Fit_Map. For details, please refer to Figure 2 (E) As shown. Use Prewitt_Map and Fit_Map to perform an OR operation, and then perform an AND operation with the first mask image, and filter out small defect noise to obtain the defect Result_Map (first defect feature). For details, please refer to Figure 2 (F) shown.
[0072] For further information, see Figure 4 , a schematic diagram of the process of obtaining the first edge feature in the vertical / horizontal direction based on the above-mentioned first defect detection method is given.
[0073] Specifically, in the defect edge detection at the R angle, in the detection process of the first edge defect, the target image is subjected to a first nonlinear gradient processing to obtain a first nonlinear gradient map; wherein the first nonlinear gradient processing is a nonlinear gradient processing with a kernel of 15*15; the first nonlinear gradient map is subjected to a binarization processing according to a preset first gradient threshold to obtain a binarized image corresponding to the first nonlinear gradient map; based on the binarized image corresponding to the nonlinear gradient map, a defect merging processing is performed according to a preset first pixel radius, and a second mask image is determined according to the image after defect merging; the target image is subjected to a second nonlinear gradient processing to obtain a second nonlinear gradient map; wherein the second nonlinear gradient processing is a nonlinear gradient processing with a kernel of 3*3; the second nonlinear gradient map is subjected to a binarization processing according to a preset second gradient threshold to obtain a binarized image corresponding to the second nonlinear gradient map; the binarized image corresponding to the second nonlinear gradient map is subjected to a contour tracking processing and a hole detection processing, and a defect merging processing is performed based on a preset second pixel radius to obtain a third defect image; based on the third defect image and the second mask image, the first edge defect at the R angle is determined.
[0074] Assume that the width of the target image is W, the height is H, and the target image is a set of pixel points I (i, j). For details, see Figure 3 (A) shown.
[0075] Perform the first nonlinear gradient processing of the target image with a 15*15 kernel to obtain the first nonlinear gradient map Map_Grad_0. For details, see Figure 3 (B) is shown. Among them, through the method
[0076]
[0077] The gradient value at each coordinate point is calculated to determine the corresponding gradient map, which is the first nonlinear gradient map Map_Grad_0, where gu and gv represent the gradient values of the coordinate point in the x and y directions, respectively.
[0078] The first nonlinear gradient map Map_Grad_0 is binarized using a preset first gradient threshold T1 to obtain a binarized image corresponding to the first nonlinear gradient map.
[0079] The binary image corresponding to the first nonlinear gradient map is regarded as a defect Map1, and a defect merging operation is performed with a preset first pixel radius (for example, 10 pixels). Then, based on the defects after the merging operation, the defect with the largest contour is selected to determine the second mask image Mask. For details, please refer to Figure 3 (C) shown.
[0080] Perform the second nonlinear gradient processing of the 3*3 kernel on the target image to obtain the second nonlinear gradient map Map_Grad_1. For details, see Figure 3 (D) shown.
[0081] The second nonlinear gradient map Map_Grad_1 is binarized using a preset second gradient threshold T2 to obtain a binarized image corresponding to the second nonlinear gradient map.
[0082] Perform contour tracing processing on the binary image corresponding to the second nonlinear gradient map to obtain a contour map Contour_Map; see Figure 3 (E) is shown. Then, the hole detection process is performed, and the run-length code defect is merged based on the preset second pixel radius (for example, the radius is 1) to obtain the third defect image Map2. Map2 is used to perform an AND operation with the second mask image Mask, and then large defects are filtered to generate the first edge defect Result_Map, which is the first edge defect on the R corner. For details, please refer to Figure 3 (F) shown.
[0083] For further information, see Figure 5 , a schematic diagram of the process of obtaining the second edge feature on the R corner based on the second defect detection method is given.
[0084] Step S103: Based on the second defect detection method, the first edge defect in the target image is classified to determine the second edge defect in the target image;
[0085] Among them, the second defect detection method is a defect detection method based on deep learning.
[0086] For some defects, the effect of using traditional methods to detect them is not good under the interference of complex backgrounds. Therefore, after acquiring the image, for some specific defects, it is necessary to further use the deep learning model to train certain defects separately, so that certain types of specific defects can be detected and classified. In a specific embodiment, the commonly used deep learning networks are ThunderNet, PeleeNet, Yolo series, etc. This is not limited in this embodiment.
[0087] Specifically, after calculating the first edge defect, it is necessary to further classify the defect features corresponding to the first edge defect. Here, we choose to classify the defect features. Because the original image data dimension is too large, we select the image of the clip (256*256 or 64*64) pixel size of the extracted defect for model training. We use a lightweight network to reduce the amount of calculation for model training. Commonly used networks include Yolo-tiny, MobileNet, SqueezeNet, ShuffleNet, etc.
[0088] Here we take the Squeezenet model as an example. Specifically, unlike the traditional convolution method, the core of SqueezeNet lies in the Fire module. The Fire module consists of two layers, namely the Squeeze layer and the Expand layer. Figure 6 As shown in the figure, the Squeeze layer is a convolutional layer with a 1*1 convolutional kernel, and the Expand layer is composed of convolutional layers with 1*1 and 3*3 convolutional kernels.
[0089] In the process of further classifying the first edge defect through the above-mentioned Squeezenet model to obtain the corresponding second edge defect, the first edge defect is input into the Squeeze layer, and then the output result of the Squeeze layer is input into the Expand layer, and the output result of the Expand layer is used as the second edge defect.
[0090] For details, please see Figure 7 As shown, in the Squeeze layer, the first edge defect of W*H*M is input into the convolution layer of the 1*1 convolution kernel in the Squeeze layer to obtain the feature map of W*H*s1, and then, the feature map of W*H*s1 is respectively input into the convolution layers of the 1*1 and 3*3 convolution kernels in the Expand layer to obtain the feature maps corresponding to H*M*e1 and H*M*e3 respectively, and the contact operation is performed to obtain the feature map of H*M*(e1+e3).
[0091] In the training log of the Squeezenet model above, the accuracy quickly reaches 1 and the loss value is very small and close to 0.
[0092] That is to say, in this step, after the edge defects are preliminarily classified using the traditional defect detection method in step S102, deep learning classification is required based on the defect clip thumbnails, and the SqueezeNet model is used to further classify the edge defects to improve the accuracy of edge defect classification detection.
[0093] Step S104: screening the first edge defects and the second edge defects to determine the defect detection result of the cover plate to be detected.
[0094] The first edge defect is obtained based on the traditional first defect detection method, and the second edge defect is obtained based on the second defect detection method based on deep learning. In this embodiment, it is necessary to combine the results of the first defect detection method and the second defect detection method as a whole to perform comprehensive defect screening to obtain the final defect detection result. Specifically, for the preset multiple classification defects, it is necessary to screen the first edge defect and the second edge defect based on their corresponding preset feature screening expressions to determine the defect detection results for each classification defect. Among them, each type of classification defect can be further classified more carefully and accurately, and the screening speed is also extremely fast through expressions, which can fully meet the rapid needs of actual production. In a specific embodiment, please refer to Figure 8 , a schematic diagram of the above-mentioned defect screening process based on feature screening expression is given.
[0095] For further information, see Fig. 9 , a schematic diagram of the structure of an edge defect detection device is given, in which, Fig. 9 As shown, the above-mentioned edge defect detection device includes:
[0096] An image acquisition module 101 is used to acquire a target image of a cover plate to be inspected, wherein the target image is an image of the cover plate to be inspected;
[0097] A first defect detection module 102, configured to perform defect detection on the target image based on a first defect detection method to obtain a first edge defect in the target image;
[0098] A second defect detection module 103 is used to classify the first edge defect in the target image based on a second defect detection method, and determine the second edge defect in the target image; wherein the second defect detection method is a defect detection method based on deep learning;
[0099] The defect fusion screening module 104 is used to screen the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be detected.
[0100] In an optional embodiment, the image acquisition module 101 is also used to pre-process the target image based on a preset image filtering algorithm; wherein the preset image filtering algorithm includes one or more combinations of mean filtering, median filtering, Markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, adaptive filtering, and Hilbert filtering.
[0101] In an optional embodiment, the target image is a grayscale image; the image acquisition module 101 is also used to acquire the ROI area in the target image, and the ROI area includes the ROI image area in the target image corresponding to the vertical edge, horizontal edge and R angle edge of the cover plate to be detected.
[0102] In an optional embodiment, the first defect detection module 102 is also used to perform a first defect detection on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be inspected to obtain a first edge defect in the target image, and the first edge defect includes edge defects in the vertical direction and / or the horizontal direction; perform a first defect detection on the R corner edge of the cover plate to be inspected to obtain a first edge defect in the target image, and the first edge defect includes an edge defect on the R corner.
[0103] In an optional embodiment, the first defect detection module 102 is also used to binarize the target image according to a preset first grayscale threshold to obtain a binarized image corresponding to the target image; dilate the binarized image corresponding to the target image to obtain a corresponding first mask image; perform edge detection on the target image to obtain a first defect image in the target image; perform 8-neighborhood contour tracking on the target image to determine at least one inflection point, and perform a fitting operation on the at least one inflection point; perform defect detection based on the fitting result to obtain a second defect image; and determine the first defect feature in the vertical direction and / or horizontal direction based on the first defect image, the second defect image and the first mask image.
[0104] In an optional embodiment, the first defect detection module 102 is further used to perform a first nonlinear gradient processing on the target image to obtain a first nonlinear gradient map; wherein the first nonlinear gradient processing is a nonlinear gradient processing with a kernel of 15*15; binarize the first nonlinear gradient map according to a preset first gradient threshold to obtain a binarized image corresponding to the first nonlinear gradient map; perform defect merging processing according to a preset first pixel radius based on the binarized image corresponding to the nonlinear gradient map, and determine a second mask image according to the image after defect merging; perform a second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second nonlinear gradient processing is a nonlinear gradient processing with a kernel of 3*3; binarize the second nonlinear gradient map according to a preset second gradient threshold to obtain a binarized image corresponding to the second nonlinear gradient map; perform contour tracking processing and hole detection processing on the binarized image corresponding to the second nonlinear gradient map, and perform defect merging processing based on a preset second pixel radius to obtain a third defect image; determine the first edge defect on the R angle based on the third defect image and the second mask image.
[0105] In an optional embodiment, the deep learning model is obtained by training a clip thumbnail containing defects of a training sample; the deep learning model is a Squeezenet model, the Squeezenet model includes a Squeeze layer and an Expand layer, the Squeeze layer includes a convolution layer with a 1*1 convolution kernel, the Expand layer includes a convolution layer with a 1*1 convolution kernel and a convolution layer with a 3*3 convolution kernel; the second defect detection module 103 is also used to input the first edge defect into the Squeeze layer, and then input the output result of the Squeeze layer into the Expand layer, and use the output result of the Expand layer as the second edge defect.
[0106] In an optional embodiment, the defect fusion screening module 104 is further used to screen the first edge defect and the second edge defect based on preset feature screening expressions corresponding to different classification defects, and determine the defect detection result under each classification defect.
[0107] Fig.10 FIG. 1 shows an internal structure diagram of a computer device for implementing the above-mentioned edge defect detection method in an embodiment. The computer device may be a terminal or a server. Fig.10As shown, the computer device includes a processor, a memory and a network interface connected via a system bus. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can implement the above method. Those skilled in the art can understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0108] After adopting the above-mentioned edge defect detection method, device, computer equipment and computer-readable storage medium, when it is necessary to detect the edge features of the mobile phone glass cover plate, first obtain the corresponding target image, then based on the first defect detection method, perform defect detection on the target image to obtain the first edge defect in the target image; based on the second defect detection method, classify the first edge defect in the target image and determine the second edge defect in the target image; wherein the second defect detection method is a defect detection method based on deep learning; the first edge defect and the second edge defect are screened to determine the defect detection result of the cover plate to be detected. In other words, through the edge defect detection method, device, computer equipment and computer-readable storage medium, preliminary defect classification is performed based on the traditional first defect detection method, and the defects are further classified based on the second defect detection method based on deep learning, combining the advantages of traditional algorithms and deep learning algorithms, improving the accuracy of edge detection, and improving the yield rate of mobile phone glass cover plate generation.
[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0111] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A method for detecting edge defects, characterized in that: The method comprises: Acquire a target image of the cover plate to be detected, wherein the target image is an image of the cover plate to be detected, and the target image is a grayscale image; Acquire a ROI region in the target image, wherein the ROI region includes a ROI image region in the target image corresponding to a vertical edge, a horizontal edge, and an R-angle edge of the cover plate to be detected, and update the ROI image region to the target image; Based on the first defect detection method, defect detection is performed on the target image to obtain a first edge defect in the target image; wherein the first defect detection is performed on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be detected to obtain the first edge defect in the target image, and the first edge defect includes an edge defect in the vertical direction and / or the horizontal direction; the first defect detection is performed on the R corner edge of the cover plate to be detected to obtain the first edge defect in the target image, and the first edge defect includes an edge defect on the R corner; Based on a second defect detection method, defect classification is performed on the first edge defect in the target image to determine a second edge defect in the target image; wherein the second defect detection method is a defect detection method based on a deep learning model; Screening the first edge defect and the second edge defect to determine a defect detection result of the cover plate to be detected; Among them, the step of performing a first defect detection on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be detected to obtain the first edge defect in the target image includes: binarizing the target image according to a preset first grayscale threshold to obtain a binarized image corresponding to the target image; dilating the binarized image corresponding to the target image to obtain a corresponding first mask image; performing edge detection on the target image to obtain a first defect image in the target image; performing 8-neighborhood contour tracking on the target image to determine at least one inflection point, and performing a fitting operation on the at least one inflection point; performing defect detection based on the fitting result to obtain a second defect image; and determining the first defect feature in the vertical direction and / or horizontal direction based on the first defect image, the second defect image and the first mask image.
2. The edge defect detection method according to claim 1, characterized in that: The method further comprises: The target image is preprocessed based on a preset image filtering algorithm; wherein the preset image filtering algorithm includes one or more combinations of mean filtering, median filtering, Markov random field filtering, simulated annealing filtering, dynamic threshold segmentation, linear filtering, mathematical morphology filtering, gradient processing, adaptive filtering, and Hilbert filtering.
3. The edge defect detection method according to claim 1, characterized in that: The step of performing a first defect detection on the R-angle edge of the cover plate to be detected to obtain a first edge defect in the target image also includes: Performing a first nonlinear gradient processing on the target image to obtain a first nonlinear gradient map; wherein the first nonlinear gradient processing is a nonlinear gradient processing with a kernel of 15*15; performing a binarization processing on the first nonlinear gradient map according to a preset first gradient threshold to obtain a binarized image corresponding to the first nonlinear gradient map; performing a defect merging processing according to a preset first pixel radius based on the binarized image corresponding to the nonlinear gradient map, and determining a second mask image according to the image after defect merging; Perform a second nonlinear gradient processing on the target image to obtain a second nonlinear gradient map; wherein the second nonlinear gradient processing is a nonlinear gradient processing with a kernel of 3*3; binarize the second nonlinear gradient map according to a preset second gradient threshold to obtain a binarized image corresponding to the second nonlinear gradient map; perform contour tracking processing and hole detection processing on the binarized image corresponding to the second nonlinear gradient map, and perform defect merging processing based on a preset second pixel radius to obtain a third defect image; determine the first edge defect on the R corner based on the third defect image and the second mask image.
4. The edge defect detection method according to claim 1, characterized in that: The deep learning model is obtained by training the clip images containing defects of the training samples; The deep learning model is a Squeezenet model, which includes a Squeeze layer and an Expand layer, the Squeeze layer includes a convolution layer with a 1*1 convolution kernel, and the Expand layer includes a convolution layer with a 1*1 convolution kernel and a convolution layer with a 3*3 convolution kernel; The step of classifying the first edge defect in the target image based on the second defect detection method and determining the second edge defect in the target image further includes: The first edge defect is input into the Squeeze layer, and then the output result of the Squeeze layer is input into the Expand layer, and the output result of the Expand layer is used as the second edge defect.
5. The edge defect detection method according to claim 1, characterized in that: The step of screening the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be detected further includes: Based on preset feature screening expressions corresponding to different classification defects, the first edge defects and the second edge defects are screened to determine the defect detection result for each classification defect.
6. An edge defect detection device, characterized in that: The device comprises: An image acquisition module is used to acquire a target image of the cover plate to be detected, wherein the target image is an image of the cover plate to be detected; the target image is a grayscale image; an ROI area in the target image is acquired, wherein the ROI area includes an ROI image area in the target image corresponding to a vertical edge, a horizontal edge, and an R angle edge of the cover plate to be detected, and the ROI image area is updated to the target image; A first defect detection module is used to perform defect detection on the target image based on a first defect detection method to obtain a first edge defect in the target image; wherein the first defect detection is performed on the ROI image area of the vertical edge and the horizontal edge of the cover plate to be detected to obtain the first edge defect in the target image, and the first edge defect includes an edge defect in the vertical direction and / or the horizontal direction; the first defect detection is performed on the R corner edge of the cover plate to be detected to obtain the first edge defect in the target image, and the first edge defect includes an edge defect on the R corner; A second defect detection module, configured to classify the first edge defect in the target image based on a second defect detection method, and determine a second edge defect in the target image; wherein the second defect detection method is a defect detection method based on deep learning; A defect fusion screening module, used for screening the first edge defect and the second edge defect to determine the defect detection result of the cover plate to be detected; Among them, the first defect detection module is also used to binarize the target image according to a preset first grayscale threshold to obtain a binarized image corresponding to the target image; dilate the binarized image corresponding to the target image to obtain a corresponding first mask image; perform edge detection on the target image to obtain a first defect image in the target image; perform 8-neighborhood contour tracking on the target image to determine at least one inflection point, and perform a fitting operation on the at least one inflection point; perform defect detection based on the fitting result to obtain a second defect image; and determine the first defect feature in the vertical direction and / or horizontal direction based on the first defect image, the second defect image and the first mask image.
7. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein the memory has executable codes, and when the executable codes are run on the processor, the edge defect detection method according to any one of claims 1 to 5 is implemented.