A method for detecting defects in optical wafers

Through image acquisition and computer vision technology, the defects of the optical pass sheet are automatically detected, which solves the problems of low detection efficiency and missed detection in the existing technology, and realizes high-precision optical pass sheet defect detection and improves product quality control.

CN118570213BActive Publication Date: 2024-11-05YANTAI UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411060113.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-05
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

In the prior art, the detection efficiency of optical pass sheets is low, and it is difficult to detect small defects in manual inspection, which is prone to missed detection and error detection, which affects product quality.

Method used

The image acquisition device is used to obtain the original image of the optical pass sheet. Through the steps of contour detection, Blob center coordinate comparison, polygon shrinking algorithm and integral binaryization, the defects of the optical pass sheet are automatically detected, including contour vertex coordinate calculation and mask area. Handling and defect marking.

Benefits of technology

It realizes high-precision optical pass defect detection, improves detection efficiency, reduces false detection and missed detection, and enhances product quality control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118570213B_ABST
    Figure CN118570213B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for detecting defects of a light pass sheet, comprising the following steps: step 1: converting an original image into a grayscale image for preprocessing to obtain a binary image; step 2: obtaining the vertex coordinates of the light pass sheet in the binary image, namely, the contour detection coordinates, as well as the size of the light pass sheet, the inclination angle of the light pass sheet, and the center coordinates of the light pass sheet through contour detection; step 3: using Blob detection to obtain the center coordinates of the light pass sheet in the grayscale image; step 4: performing vertex indentation on the coordinates after the contour detection coordinates and the Blob detection coordinates are fused to remove the influence of the burrs of the light pass sheet in the binary image on the final detection result and to prepare a mask map; step 5: performing defect detection on the circumscribed rectangular area of ​​the binary image, and then marking the defects in the original image. The present invention compares the Blob center coordinates with the contour center coordinates, combines the Blob center coordinates with the contour information, supplements the contour vertex coordinates, prevents missed detection, and obtains more accurate coordinates.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a detection method, in particular to a method for detecting defects in optical pass sheets. Background Art

[0002] Optical film, also known as optical thin film, has a complex and precise production process and plays an important role in many high-tech applications, such as lasers, solar cells, and fiber-optic communications. During the production and transportation of optical film, it is easy to cause surface scratches, edge collapse, pitting and other small defects that are difficult to find. These defects may cause product performance degradation or even make the product unusable.

[0003] Since the optical sheet module contains multiple optical sheet units, the length and width of each optical sheet unit are between 1.4-2.5 mm. For such tiny surface defects, high-power eyepieces are often used for observation during manual inspection, and the position of the optical sheet is manually adjusted in small increments to observe the defects of each optical sheet unit. This inspection method not only has low inspection efficiency and large workload, but also causes visual damage to inspectors during long working hours. In addition, the inspection levels of inspectors are uneven, which is prone to missed inspections and wrong inspections, which has a great impact on product quality.

[0004] In summary, it is necessary to introduce an automated defect detection method for optical wafers with high-precision detection effects, which can effectively identify various defects, replace manual detection and quickly execute it on the production line to improve detection efficiency. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the purpose of the present invention is to provide a method for detecting defects in optical pass sheets.

[0006] The object of the present invention is achieved by the following technical solution: A method for detecting defects in a light-transmitting sheet comprises the following steps:

[0007] Step 1: Obtain the original image with the optical pass film through an image acquisition device, and copy the original image into a grayscale image;

[0008] Step 2: Preprocess the grayscale image, obtain the coordinates of the contour vertices through contour detection and angle cosine value screening, and calculate the contour center coordinates, contour width, contour height and contour inclination angle;

[0009] Step 3: Use Blob detection to obtain the center coordinates of the Blob in the grayscale image, compare the center coordinates of the Blob with the center coordinates of the contour, combine the center coordinates of the Blob with the contour information, supplement the coordinates of the contour vertices, obtain the supplementary coordinates, and use the indentation algorithm on the supplementary coordinates to obtain the indentation coordinates;

[0010] Step 4: Create a black image with the same resolution as the grayscale image, use the polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the circumscribed rectangular area of ​​the indented coordinates, i.e., the black image mask area, perform integral binarization and preprocessing on the grayscale image mask area with the same coordinates as the black image mask area, and generate a mask edge map;

[0011] Step 5: Perform defect detection on the mask edge image to obtain defect information, and then mark the defects in the original image.

[0012] Preferably, obtaining the original image: obtaining the original image with the optical pass film by an image acquisition device with a high-definition camera, and copying the original image into a grayscale image;

[0013] Obtain contour information: Use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image and make the grayscale image smooth in preparation for feature extraction; Use the Sobel operator to calculate the total gradient of the grayscale image and generate the total gradient map of the grayscale image, so that the grayscale image only displays the pixel intensity change rate, and then use the parameter threshold method to binarize the total gradient map of the grayscale image, and divide the total gradient map of the grayscale image into foreground and background, that is, obtain the parameter contour map; then use the OpenCV adaptive threshold method to binarize the parameter contour map, divide the parameter contour map into edge areas and non-edge areas, so that the contour in the parameter contour map becomes a single pixel display, that is, obtain the edge contour map; calculate the area of ​​the connected domain detected in the edge contour map and compare it with the first set area, and delete the area in the edge contour map where the connected domain area is smaller than the first set area; Use the Sobel operator to calculate the total gradient of the edge contour map and binarize it by the parameter threshold method to obtain the parameter edge contour Figure; use the edge search algorithm to obtain the edge of the connected area in the parameter edge contour map, perform polygonal approximation on the edge of the connected area, and obtain the contour of the connected area; remove the connected area contours except the convex quadrilateral contour in the connected area contour, calculate the cosine value of the angle between the vertex coordinates of the convex quadrilateral contour, select the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour and compare it with the second set value, and if it is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle, and the convex quadrilateral contour with a shape close to a rectangle is screened out, and the circumscribed inclined rectangle algorithm is used to calculate the circumscribed inclined rectangle of the convex quadrilateral contour, that is, the contour vertex coordinates; calculate the contour center coordinates through the contour vertex coordinates, compare the contour vertex coordinates with the contour center coordinates, and rearrange them clockwise; perform inverse trigonometric function calculation on the angle between the first two points sorted by the contour vertex coordinates and the X-coordinate axis to obtain the contour inclination angle, and calculate the contour width and contour height;

[0014] Obtain the Blob center coordinates and supplement the contour vertex coordinates: the Blob detector sets the detection style so that the Blob detector only detects quadrilateral style patches; the Blob detector sets the detection area so that the Blob detector screens out patches outside the third set area range; the configured Blob detector is used to directly detect the grayscale image to obtain the Blob center coordinates; the distance dis between the Blob center coordinates and the contour center coordinates is calculated using the Euclidean distance formula. When dis is less than or equal to the fourth set value, the Blob center coordinates and the contour center coordinates are the center points of the same optical pass sheet and no supplement is required; when dis is greater than the fourth set value, the Blob center coordinates are considered inconsistent with the contour center coordinates. At this time, the optical pass sheet obtained by the Blob detection needs to be supplemented to the optical pass sheet obtained by the contour detection to obtain supplementary coordinates; in order to prevent the optical pass sheet at the edge of the image from being misdetected, the supplementary coordinates that exceed the edge of the grayscale image are removed; and the supplementary coordinates within the edge of the grayscale image are polygonally indented to obtain the indented coordinates;

[0015] Obtain the mask area and process it: create a black image with the same resolution as the grayscale image, draw a simulated light pass film on the black image with the indented coordinates as vertices, and fill the indented coordinate area of ​​the black image with white; use the polygonal circumscribed rectangle algorithm for the indented coordinates to obtain the four vertices of the circumscribed rectangle of the indented coordinates, and the circumscribed rectangular area surrounded by the four vertices is the black image mask area, traverse the pixels in the black image mask area, and if the ratio of the sum of black pixels in the black image mask area to the total sum of pixels is less than the fifth set value, the black image mask area is judged to be qualified; for the grayscale image mask area with the same coordinates as the black image mask area, use the calculated integral map and square integral map to calculate the pixel sum, square pixel sum, pixel mean and pixel variance of all positions, obtain information of multiple pixels in one calculation, and use the accumulated pixel values ​​for calculation to complete the integral binarization to obtain Mask integral binary image; using a connected domain algorithm to obtain the area of ​​each connected domain on the mask integral binary image, and the connected domain area smaller than the first set area is a minimal connected domain and the minimal connected domain is removed, that is, impurities are removed; setting the pixels in the mask integral binary image except the indented coordinate area to 0, that is, removing interference factors; using the Sobel operator to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays the pixel intensity change rate; using the parameter threshold method to binarize the mask gradient image, the mask gradient image is divided into the foreground and the background, that is, the mask parameter contour image is obtained; using the adaptive threshold method to binarize the mask parameter contour image, the mask parameter contour image is divided into the edge area and the non-edge area, and the mask edge contour image is obtained; using the Sobel operator to calculate the total gradient of the mask edge contour image again and binarize it by the parameter threshold method to obtain the mask edge image;

[0016] Obtain defect information and mark it: Use the connected domain algorithm to obtain the internal connected domain information of the mask edge map. If the connected domain information exceeds the corresponding set parameters, it is determined that there is a defect in the mask edge map, and the coordinates of the defective mask edge map, that is, the defect coordinates, are recorded, and marked in the original map according to the defect coordinates.

[0017] Preferably, step 2 further comprises the following steps:

[0018] According to the approximation algorithm, polygonal approximation is performed on the edge of the connected domain to obtain the connected domain contour, and the connected domain contour except the convex quadrilateral contour is removed from the connected domain contour, and the coordinates of the four vertices surrounding the convex quadrilateral contour are calculated, that is, the coordinates of the four contour vertices of the convex quadrilateral are obtained;

[0019] The cosine values ​​of the angles between the vertices of the convex quadrilateral outline are calculated, and the cosine values ​​of the angles between the two vectors formed by three adjacent points are calculated and the maximum value is selected to determine whether the shape of the convex quadrilateral is close to a rectangle. The specific formula is as follows:

[0020]

[0021] In the formula For a convex quadrilateral i The vertex coordinates, For a convex quadrilateral i Vertex x coordinate, For a convex quadrilateral i Vertex y coordinate; For a convex quadrilateral i +1 vertex coordinates, For a convex quadrilateral i +1 vertex x coordinate, For a convex quadrilateral i +1 vertex y coordinate; , For a convex quadrilateral i +2 vertex coordinates, For a convex quadrilateral i +2 vertices x coordinate, For a convex quadrilateral i +2 vertices y Coordinates; Vectors For point Pointing Point vector, vector For point Pointing Point The vector of For vector and vector The angle between two vectors, is the maximum cosine value calculated, point With point for the same point;

[0022] like If the value of is less than the second set value, it is considered that the shape of the convex quadrilateral is close to a rectangle, and the contour of the convex quadrilateral is the contour of the light pass sheet, which meets the requirements.

[0023] Preferably, the contour inclination angle is calculated according to the contour vertex coordinates, specifically as follows:

[0024]

[0025] In the formula, is the profile inclination angle, is the number of contours, For the i The coordinates of the first point of the contour, For the i The first point of the contour x coordinate, For the i The first point of the contour y coordinate; For the i The coordinates of the second point of the contour, For the i The second point of the contour x coordinate, For the i The second point of the contour y coordinate;

[0026] The contour width and height are calculated according to the contour vertex coordinates using the Euclidean distance formula, as shown below:

[0027]

[0028]

[0029] In the formula, is the outline width, is the profile height, is the number of contours, For the i The coordinates of the first point of the light channel contour, For the i The first point of the contour x coordinate, For the i The first point of the contour y coordinate; For the i The coordinates of the second point of the light channel contour, For the i The second point of the contour x coordinate, For the i The second point of the contour y coordinate; For the i The coordinates of the third point of the light channel contour, For the i The third point of the contour x coordinate, For the i The third point of the contour y coordinate.

[0030] Preferably, obtaining the coordinates of four contour vertices in step 2 includes the following steps:

[0031] The cosine value of the angle is calculated using three adjacent points of the detected vertices of the convex quadrilateral outline, and the maximum cosine value of the three or four corners of the convex quadrilateral outline is compared with the second set value. If the maximum cosine value of the convex quadrilateral outline is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle; the inclination angle of each convex quadrilateral outline is calculated by using the inverse trigonometric function of the upper left corner vertex and the upper right corner vertex of each convex quadrilateral outline, the sum of the inclination angles of all convex quadrilateral outlines is calculated, and then the average value is calculated.

[0032] Preferably, in step 3, the light flux sheet obtained by Blob detection is added to the light flux sheet obtained by contour detection, specifically: the center coordinates of the Blob are used as the center of the rectangle, and the contour inclination angle is used as the center of the rectangle. The tilt angle of the rectangle and the width of the outline is the rectangle width, and the outline height Draw a Blob rectangle for the rectangle height and get the coordinates of the Blob rectangle vertices;

[0033] Sort the Blob rectangle vertex coordinates clockwise, and add the sorted Blob rectangle vertex coordinates to the contour vertex coordinate points to obtain the supplementary coordinates;

[0034] The supplementary coordinates within the edge of the grayscale image are indented to obtain the indented coordinates, as shown in the following formula:

[0035]

[0036] in

[0037]

[0038]

[0039]

[0040] In the formula are supplementary coordinates, is the indentation coordinate, L is the inward distance, , is a vector consisting of a supplementary coordinate and two adjacent supplementary coordinates. for The modulus of a vector, for The magnitude of a vector, is the angle between the two sides of the supplementary coordinates, Representation vector exist x The direction component, Representation vector exist y The direction of the component, Representation vector exist x The direction of the component, Representation vector exist y The direction component.

[0041] Preferably, the polygonal circumscribed rectangle algorithm is used to obtain the four vertices of the circumscribed rectangle of the indented coordinates of the black image, as shown in the following formula:

[0042]

[0043] In the formula left is the left edge of the bounding rectangle, right is the right edge of the bounding rectangle, top The upper boundary of the bounding rectangle bottom is the lower boundary of the circumscribed rectangle, The smallest of all coordinates in the indented coordinate area x Coordinate values, The smallest of all coordinates in the indented coordinate area y Coordinate values, The largest of all coordinates in the indented coordinate area x Coordinate values, The largest of all coordinates in the indented coordinate area y Coordinate value;

[0044] The black image mask area is a black image mask area without a tilt angle, pixels in the black image mask area are traversed, the traversed black pixels are accumulated, and the ratio of the sum of black pixels in the black image mask area to the sum of total pixels is obtained. If the ratio is less than a fifth set value, the mask area is judged to be qualified;

[0045] The grayscale image mask area image processing uses an integral threshold algorithm to perform Gaussian blur processing on the grayscale image mask area, sets the Gaussian blur standard deviation value sigma, and uses the boundary extension algorithm to copy the grayscale image mask area boundary;

[0046] Calculate the pixel sum of the integral kernel in the integral image: use the anti-border mode to expand the grayscale mask area boundary, sum represents the integral area of ​​the image after boundary extension, sqsum Represents the integral square area of ​​the image after boundary expansion, and obtains the cumulative sum of the pixels in the area enclosed by the four vertices of the integral area and the origin of the background image coordinates, as shown in the following formula:

[0047]

[0048]

[0049] In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integration area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower right corner of the integration area and the coordinate origin. i The forward direction of the X direction of the integration area i The sum of pixels, j The Y direction forward of the integration area j The sum of pixels, ker is the integration kernel size, is the pixel sum;

[0050] Similarly, calculate the cumulative sum of pixels in the area enclosed by the four vertices of the integral square area and the origin of the background coordinate system, as follows:

[0051]

[0052]

[0053] In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral square area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower right corner of the integral square area and the coordinate origin. i The forward direction of the X direction of the integral square area i The sum of pixels, j The Y direction forward of the integral square area j The sum of pixels, ker is the integration kernel size, is the sum of square pixels;

[0054] By integrating the pixels in the area and obtaining the pixel mean , as follows:

[0055]

[0056] And get the pixel variance var , as follows:

[0057]

[0058] Calculate the integral binarization threshold th , as follows:

[0059]

[0060] The grayscale image mask area is binarized pixel by pixel through the integral binarization threshold to obtain the mask integral binary image, as shown in the following formula:

[0061]

[0062] in, is the grayscale mask area The pixel value of the coordinate point, is the integral binarization threshold calculated for the corresponding pixel value, is the pixel value after binarization;

[0063] The removing of the extremely small connected domain whose area is smaller than the first set area is to set the pixels in the extremely small connected domain whose area is smaller than the first set area to 0.

[0064] Preferably, the resolution of the black image is the same as that of the grayscale image, that is, the coordinates of the grayscale image mask area and the black image mask area are the same, and an integral binarization threshold calculation is performed on the grayscale image mask area with no tilt angle having the same coordinates as the black image mask area with no tilt angle. When calculating the pixel variance, a value greater than 0 is taken. If the pixel variance is less than 0, the pixel variance is taken as 0. During binarization, the calculated integral binarization threshold is used to binarize the grayscale image mask area pixel by pixel.

[0065] Preferably, performing defect detection on the mask edge image and then marking the defects in the original image comprises the following steps:

[0066] 5.1. Using a connected domain algorithm to obtain the connected domain information inside the mask edge map, the connected domain information inside the mask edge map includes the area, number, width and height of the connected domain inside the mask edge map;

[0067] 5.2. Point defect detection: compare the area of ​​the connected domain obtained by the mask edge map detection with the area calculated by the sixth set diameter. When the area of ​​the connected domain is larger than the area calculated by the sixth set diameter, it is considered that there is a point defect in the mask edge map;

[0068] 5.3. Pockmark defect detection: compare the number of connected domains detected by the mask edge map with the set number. If the number of connected domains is greater than the seventh set number of pockmarks, it is considered that the mask edge map has a pockmark defect;

[0069] 5.4. Slender defect detection, step 1: whether the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, wherein the eighth set value is greater than the ninth set value, if the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, it is a slender connected domain, step 2: according to the comparison between the maximum edge value of the connected domain and the tenth set value, when the maximum edge value of the connected domain is greater than the tenth set value, it is considered that the mask edge map has a slender defect; the larger value of the width and height of the connected domain is the maximum edge value of the connected domain;

[0070] 5.5. After detection, store the center coordinates of all mask edge images that are judged to be defects, and mark the corresponding coordinate points in the original image.

[0071] Preferably, the kernel for defining Gaussian blur is 1, 3, 5 or 7, and the standard deviation value of Gaussian blur is 0.5-15. More preferably, the kernel for defining Gaussian blur is 3, and the standard deviation value of Gaussian blur is 15.

[0072] Compared with the prior art, the present invention has the following advantages and effects:

[0073] 1. The present invention uses the calculated angle cosine value to screen the vertex coordinates of the convex quadrilateral contour to prevent false detection; uses the Blob center coordinates to compare with the contour center coordinates, and combines the Blob center coordinates with the contour information to supplement the contour vertex coordinates to prevent missed detection, and the acquired coordinates are more accurate.

[0074] 2. The present invention uses a polygon shrinking algorithm to obtain shrinking coordinates and remove the influence of the burrs of the optical pass sheet on defect detection.

[0075] 3. The present invention uses a polygon circumscribed rectangle algorithm to obtain the circumscribed rectangle of the indented coordinates, namely the mask area, and subsequently only processes the mask area, thereby reducing the calculation range of the integral threshold algorithm and improving the detection efficiency.

[0076] 4. The present invention expands the inward coordinates with tilt angles into a mask area without tilt angles through a polygon circumscribed rectangle algorithm, which can more simply perform integral threshold calculation on the grayscale mask area without tilt angles, thereby accelerating the calculation speed.

[0077] 5. The grayscale image mask area integral threshold binarization processing module of the present invention uses an integral image and a square integral image to pre-calculate the grayscale image mask area image, which accelerates the calculation of local statistical information, improves performance, avoids repeated calculations, corrects variance, avoids numerical instability, and enhances accuracy.

[0078] 6. The present invention sets pixels outside the indented coordinate region of the mask integral binary image to 0, thereby eliminating the influence of factors outside the indented coordinate region of the grayscale image on image processing, and making image defects more accurately judged.

[0079] 7. When judging defects, the present invention uses multiple factors such as defect area, defect quantity, and defect aspect ratio to make a joint judgment, which can more accurately judge multiple defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 It is a schematic diagram of the process of the optical pass sheet defect detection method of the present invention;

[0081] Figure 2 The original image tested by the present invention;

[0082] Figure 3 The total gradient map of the grayscale image calculated by the Sobel operator in the present invention;

[0083] Figure 4 A contour vertex graph obtained by contour detection of the present invention;

[0084] Figure 5 The Blob center coordinate map obtained by the Blob detection of the present invention;

[0085] Figure 6Blob rectangle vertex coordinate diagram for comparison and supplementation of the present invention;

[0086] Figure 7 Supplementary coordinate diagram obtained for the present invention;

[0087] Figure 8 It is a schematic diagram of supplementary coordinates and indented coordinates of the present invention;

[0088] Fig. 9 It is a black image mask area map obtained by the polygon circumscribed rectangle algorithm of the present invention;

[0089] Fig.10 A grayscale image mask region map obtained by the present invention;

[0090] Fig.11 A process diagram for obtaining a mask edge map according to the present invention;

[0091] Fig.12 Clear the connected domain schematic diagram for the present invention;

[0092] Fig.13 The schematic diagram of calculating the cosine value of the angle according to the present invention;

[0093] Fig.14 The schematic diagram of reordering the contour vertex coordinates of the present invention;

[0094] Fig.15 Calculating a contour information graph for the contour vertex coordinates of the present invention;

[0095] Fig.16 The schematic diagram of the inner contraction coordinates is calculated for the present invention;

[0096] Fig.17 Calculate the circumscribed rectangle schematic diagram for the present invention;

[0097] Fig.18 This is a schematic diagram of the principle of calculating the pixel sum within the integral kernel of the present invention. DETAILED DESCRIPTION

[0098] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0099] The present invention aims to solve the problem that there are few weak feature detection technologies for optical wafers and the detection accuracy of weak feature defects of optical wafers in the prior art is not high. The present invention provides a method for detecting defects of optical wafers, which can effectively identify tiny defects on the surface of optical wafers (such as slight scratches, punctures, pitting, etc.), replace manual detection to improve detection efficiency, be quickly executed on the production line, and provide highly accurate detection results.

[0100] A method for detecting defects in a light-transmitting sheet comprises the following steps:

[0101] Step 1: Obtain an original image with multiple optical pass sheets through an image acquisition device, and copy the original image into a grayscale image;

[0102] Step 2: Preprocess the grayscale image, obtain the coordinates of the contour vertices through contour detection and angle cosine value screening, and calculate the contour center coordinates, contour width, contour height, and contour inclination angle;

[0103] Step 3: Use Blob detection to obtain the center coordinates of the Blob in the grayscale image, compare the center coordinates of the Blob with the center coordinates of the contour, combine the center coordinates of the Blob with the contour information, supplement the coordinates of the contour vertices, obtain the supplementary coordinates, and use the indentation algorithm on the supplementary coordinates to obtain the indentation coordinates;

[0104] Step 4: Create a black image with the same resolution as the grayscale image, use the polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the circumscribed rectangular area of ​​the indented coordinates, i.e., the black image mask area, perform integral binarization and preprocessing on the grayscale image mask area with the same coordinates as the black image mask area, and generate a mask edge map;

[0105] Step 5: Perform defect detection on the mask edge image to obtain defect information, and then mark the defects in the original image.

[0106] Specifically, obtaining the original image: obtaining the original image with multiple optical pass sheets through an image acquisition device with a high-definition camera, and copying the original image into a grayscale image;

[0107] Get contour information: Use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image and make it smooth in preparation for feature extraction; use the Sobel operator to calculate the total gradient of the grayscale image and generate the total gradient map of the grayscale image, so that the grayscale image only displays the pixel intensity change rate, and then use the parameter threshold method (the algorithm provided in OpenCV) to binarize the total gradient map of the grayscale image, and divide the total gradient map of the grayscale image into foreground and background, that is, obtain the parameter contour map; then use the OpenCV adaptive threshold method to binarize the parameter contour map, and divide the parameter contour map into edge area and non-edge area. The contour in the parameter contour map is changed to a single pixel display, that is, the edge contour map is obtained; the area of ​​the connected domain detected in the edge contour map is calculated and compared with the first set area, and the area in the edge contour map with an area smaller than the first set area is deleted, specifically, all pixels in the area smaller than the first set area are set to black; the total gradient of the edge contour map is calculated using the Sobel operator and binarized using the parameter threshold method to obtain the parameter edge contour map (i.e.: grayscale image preprocessing); the edge search algorithm (the algorithm provided by Opencv) is used to obtain the edge of the connected area in the parameter edge contour map, and the connected area is The edge of the domain is polygonally approximated to obtain the contour of the connected area (that is, contour detection obtains the coordinates of the vertices of the convex quadrilateral contour); the contours of the connected area other than the convex quadrilateral contour are removed from the contour of the connected area, the cosine values ​​of the angles are calculated by the vertex coordinates of the convex quadrilateral contour, and the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour is selected to determine whether the shape of the convex quadrilateral is close to a rectangle. Specifically, the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour is selected and compared with a second set value. If it is less than the second set value, it is determined that the shape of the convex quadrilateral is close to a rectangle, and the convex quadrilaterals with a shape close to a rectangle are screened out. The convex quadrilateral contour is obtained by using the circumscribed tilted rectangle algorithm (the circumscribed tilted rectangle algorithm is an algorithm built into Opencv), that is, the circumscribed tilted rectangle of the convex quadrilateral contour is calculated, that is, the cosine value of the angle is filtered to obtain the contour vertex coordinates; the contour center coordinates are calculated by the contour vertex coordinates, the contour vertex coordinates are compared with the contour center coordinates, and rearranged in a clockwise direction; the contour tilt angle is obtained by performing inverse trigonometric calculations based on the angle between the first two points of the contour vertex coordinates and the X-coordinate axis, and the contour width and contour height are calculated (that is, the contour center coordinates, contour width, contour height, and contour tilt angle are obtained);

[0108] Get the Blob center coordinates and supplement the contour vertex coordinates: the Blob detector sets the detection style (i.e., sets the Blob detector) so that the Blob detector only detects quadrilateral style patches; the Blob detector sets the detection area so that the Blob detector screens out patches outside the third set area range; the configured Blob detector is used to directly detect the grayscale image to obtain the Blob center coordinates; the distance dis between the Blob center coordinates and the contour center coordinates is calculated using the Euclidean distance formula. When dis is less than or equal to the fourth set value, the Blob center coordinates If the center point of the light pass sheet is the same as the center coordinate of the contour, no supplement is required; when dis is greater than the fourth set value, it is considered that the center coordinate of the Blob is inconsistent with the center coordinate of the contour. At this time, the light pass sheet obtained by the Blob detection needs to be supplemented to the light pass sheet obtained by the contour detection to obtain supplementary coordinates (i.e., the center coordinates of the Blob detection are compared with the center coordinates of the contour detection to obtain supplementary coordinates); to prevent the light pass sheet at the edge of the image from being misdetected, the supplementary coordinates that exceed the edge of the grayscale image are removed; and the supplementary coordinates within the edge of the grayscale image are polygonally indented to obtain the indented coordinates (i.e., the supplementary coordinates are indented to obtain the indented coordinates);

[0109] Obtain the mask area and process it: create a black image with the same resolution as the grayscale image, draw a simulated light pass film on the black image with the indented coordinates as vertices, and fill the indented coordinate area of ​​the black image with white; use the polygonal circumscribed rectangle algorithm for the indented coordinates to obtain the four vertices of the circumscribed rectangle of the indented coordinates, and the circumscribed rectangular area surrounded by the four vertices is the black image mask area (that is, obtain the mask area of ​​the black image according to the circumscribed rectangle of the indented coordinates), traverse the pixels in the black image mask area, and obtain the ratio of the black pixel sum to the total pixel sum of the black image mask area. If the ratio of the black pixel sum to the total pixel sum is less than the fifth set value, the black image mask area is judged to be qualified; for the grayscale image mask area with the same coordinates as the black image mask area, use the pre-calculated integral map and square integral map to calculate the pixel sum, square pixel sum, pixel mean and pixel variance at any position, obtain information of multiple pixels in one calculation, and use the accumulated pixel values ​​for calculation to complete the integral binarization to obtain the mask. Integral binary image (i.e., grayscale image mask area integral binarization); use the connected domain algorithm (Opencv's own algorithm) to obtain the area of ​​each connected domain on the mask integral binary image, and remove the extremely small features (i.e., extremely small connected domains) whose connected domain area is smaller than the first set area, i.e., remove impurities; set the pixels in the mask integral binary image except the indented coordinate area to 0, i.e., remove interference factors; use the Sobel operator to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays the pixel intensity change rate; use the parameter threshold method to binarize the mask gradient image, divide the mask gradient image into foreground and background, and obtain the mask parameter contour image; use the adaptive threshold method to binarize the mask parameter contour image, divide the mask parameter contour image into edge area and non-edge area, and obtain the mask edge contour image; use the Sobel operator to calculate the total gradient of the mask edge contour image again and binarize it by the parameter threshold method to obtain the mask edge image (i.e., mask integral binary image preprocessing);

[0110] Obtain defect information and mark it: Use the connected domain algorithm to obtain the internal connected domain information of the mask edge map. The internal connected domain information of the mask edge map includes the area, number, width and height of the internal connected domain of the mask edge map. The connected domain area, number, aspect ratio and maximum edge value of the connected domain are compared with the corresponding set parameters. If the connected domain information exceeds the corresponding set parameters, it is determined that there is a defect in the mask edge map, and the coordinates of the defective mask edge map, i.e., the defect coordinates, are recorded, and marked in the original map according to the defect coordinates.

[0111] Specific: such as Figure 1 As shown, the optical pass sheet defect detection method comprises the following steps:

[0112] 1. Get the original image:

[0113] The image is acquired by an image acquisition device with a high-resolution camera. Figure 2 The original image with multiple light pass sheets is shown, and the original image is copied as a grayscale image;

[0114] 2. Get contour information:

[0115] Use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image so that the grayscale image becomes smoother and easier to extract features; the kernel of the Gaussian blur can be defined as 3 and the standard deviation as 15. Use the Sobel operator to calculate the total gradient of the grayscale image and generate the following Figure 3 The total gradient map of the grayscale image shown in the figure makes the grayscale image only display the pixel intensity change rate, and the total gradient map of the grayscale image is binarized using the parameter threshold method, and the total gradient map of the grayscale image is divided into the foreground and the background, that is, the parameter contour map is obtained; then the OpenCV adaptive threshold method is used to binarize the parameter contour map, and the parameter contour map is divided into the edge area and the non-edge area, so that the contour in the parameter contour map becomes a single pixel display, and the edge contour map is obtained.

[0116] The purpose of contour detection is to obtain the coordinates of the vertices of the optical pass sheet, so the non-optical pass sheet connected area is cleaned up to reduce the amount of subsequent calculations; by calculating the area of ​​the connected domain detected in the edge contour map and comparing it with the first set area, the area in the edge contour map that is smaller than the first set area is deleted. The specific method is to set all pixels in the area in the edge contour map that is smaller than the first set area to black (pixel value is set to 0) to achieve the effect of cleaning impurities; among them, if Fig.12 , the impurity region can be obtained by the connected domain algorithm to obtain the impurity connected domain attribute information, the impurity connected domain attribute information includes the upper left corner of the impurity connected domain x Coordinates are initial pixels x coordinate , the upper left corner of the impurity connected domain y Coordinates are initial pixels y coordinate , impurity connected domain width and impurity connected domain height , we can get the lower right corner of the impurity connected area x Coordinates are ending pixels x coordinate , the lower right corner of the impurity connected region y Coordinates are ending pixels y coordinate ,in:

[0117]

[0118]

[0119] Pixels within the impurity connected domain are cleared pixel by pixel, and all connected domains whose area is smaller than the first set area are traversed. Preferably, the first set area may be 5, 6, 7, 8 or 9 pixels, but may be set according to actual needs. The traversal method is as follows:

[0120]

[0121] In the formula, the matrix The edge contour map i The initial pixels of the connected domain, matrix The edge contour map i The width and height of the connected domain, matrix The edge contour map i The connected component ends pixel.

[0122] The cleaned adaptive edge contour map is once again calculated using the Sobel operator for gradient calculation and binarized using the parameter threshold method to obtain a parameter edge contour map; the edge of the connected domain in the parameter edge contour map is obtained using the edge search algorithm, and the edge of the connected domain is polygonally approximated using the approximation algorithm to obtain the connected domain contour, and the connected domain contour except the convex quadrilateral contour in the connected domain contour is removed, and the vertex coordinates surrounding the convex quadrilateral contour are calculated, such as Fig.13 By calculating the cosine value of the angle between the two vectors formed by the three adjacent points of the convex quadrilateral contour vertex and selecting the maximum value, it is determined whether the coordinates of the obtained convex quadrilateral contour vertex meet the requirements and whether the shape of the convex quadrilateral is close to a rectangle. The following formula is the formula for calculating the cosine value through three adjacent points:

[0123]

[0124] In the formula For a convex quadrilateral i Vertex coordinates, For a convex quadrilateral i The x coordinates of the vertices, For a convex quadrilateral i Vertex y coordinate; For a convex quadrilateral i +1 vertex coordinates, For a convex quadrilateral i +1 vertex x coordinate, For a convex quadrilateral i +1 vertex y coordinate; , For a convex quadrilateral i +2 vertex coordinates, For a convex quadrilateral i +2 vertices x coordinate, For a convex quadrilateral i +2 vertices y Coordinates; Vectors For point Pointing Point vector, vector For point Pointing Point The vector of For vector and vector The angle between two vectors, is the maximum cosine value calculated, point With point for the same point;

[0125] like The value of is less than the second set value (the second set value can be 0.3, 0.4 or 0.5, etc., which can be selected according to the implementation situation), then it is considered that the convex quadrilateral is close to a rectangle, and the contour of the convex quadrilateral is the contour of the optical pass sheet, which meets the requirements; screen out the following Figure 4 The vertex coordinates of the convex quadrilateral outline that meet the requirements.

[0126] The circumscribed inclined rectangle algorithm is used to calculate the circumscribed inclined rectangle of the convex quadrilateral contour, that is, the coordinates of the contour vertices. Since the contour vertex coordinates obtained by the circumscribed inclined rectangle algorithm are arranged counterclockwise, in order to facilitate subsequent calculations, the contour center coordinates are first calculated through the contour vertex coordinates, as shown in the following formula:

[0127]

[0128] In the formula Indicates The center coordinates of the contour of the light pass film, Indicates The coordinates of the first contour vertex of the light pass sheet, For the The first contour vertex of the light pass sheet x coordinate, For the The y coordinate of the first contour vertex of the light pass sheet, Indicates The coordinates of the third contour vertex of the light pass sheet, For the The x coordinate of the third contour vertex of the light pass sheet, For the The y coordinate of the third contour vertex of the light channel.

[0129] Then compare the coordinate values ​​of the contour vertex coordinates relative to the contour center coordinates, such as Fig.14 Rearrange clockwise as follows:

[0130]

[0131] The right side of the equation Indicates i The coordinates of the contour center, Indicates i The center of the contour x coordinate, Indicates i The center of the contour y coordinate, and All indicate i The original outline j The coordinates of the contour vertices, Indicates i The original outline j of contour vertices x coordinate, Indicates i The original outline j The y coordinates of the contour vertices; the left side of the equation Represents the coordinates of the first contour vertex after reordering. Represents the coordinates of the second contour vertex after reordering. Represents the coordinates of the third contour vertex after reordering. Represents the coordinates of the 4th contour vertex after reordering.

[0132] like Fig.15 The tilt angle is calculated based on the coordinates of the vertices of the light pass sheet contour, and the contour tilt angle is obtained by accumulating the tilt angles calculated from all contour vertex coordinates and averaging them. theta , the following formula is used to calculate the contour inclination angle:

[0133]

[0134] In the formula, is the profile inclination angle, is the number of contours, For the i The coordinates of the first point of the contour, For the i The first point of the contour x coordinate, For the i The first point of the contour y coordinate; For the i The coordinates of the second point of the contour, For the i The second point of the contour x coordinate, For the i The second point of the contour y coordinate;

[0135] like Fig.15 The contour width and contour height are calculated using the Euclidean distance formula:

[0136]

[0137]

[0138] In the formula, is the outline width, is the profile height, is the number of contours, For the i The coordinates of the first point of the light channel contour, For the i The first point of the contour x coordinate, For the i The first point of the contour y coordinate; For the i The coordinates of the second point of the light channel contour, For the i The second point of the contour x coordinate, For the i The second point of the contour y coordinate; For the i The coordinates of the third point of the light channel contour, For the i The third point of the contour x coordinate, For the i The third point of the contour y coordinate;

[0139] 3. Get the coordinates of the Blob center and add the coordinates of the contour vertices:

[0140] The Blob detector sets the detection pattern so that the Blob detector detects only quadrilateral pattern patches; the Blob detector sets the detection area so that the Blob detector screens out patches that are not within the third set area (the light pass sheet in the image can also be regarded as a patch), that is, screens out patches outside the third set area; the configured Blob detector is used to detect the grayscale image to obtain the following Figure 5 The Blob center coordinates;

[0141] Calculate the distance between the center coordinates of the contour and the center coordinates of the Blob using the Euclidean distance formula , as follows:

[0142]

[0143] in For the The coordinates of the contour center, For the The center of the contour x coordinate, For the The center of the contour y coordinate, For the Blob center coordinates, For the Blob Center x coordinate, For the The y coordinate of the center of the Blob;

[0144] when dis When it is less than or equal to the fourth setting value, the Blob center coordinate and the contour center coordinate are the center point of the same optical pass sheet, and no supplement is required. Preferably, the fourth setting value can be 17, 18, 20, 21 or 22 pixels, but it can be set according to actual needs; when dis When the value is greater than the fourth setting value, it is considered that the Blob center coordinates are inconsistent with the contour center coordinates. At this time, the light pass obtained by Blob detection needs to be supplemented to the light pass obtained by contour detection. Specifically, the Blob center coordinates are used as the center of the rectangle, and the contour inclination angle is used as the center of the rectangle. The tilt angle of the rectangle and the width of the outline is the rectangle width, and the outline height Draw a Blob rectangle for the rectangle height, get the Blob rectangle vertex coordinates and remove the Blob rectangle vertex coordinates that exceed the edge of the grayscale image; Figure 6 The Blob rectangle vertex coordinates are sorted clockwise, and the obtained Blob rectangle vertex coordinates are added to the contour vertex coordinate points to obtain supplementary coordinates; in order to prevent the optical pass sheet at the edge of the image from being misdetected, the supplementary coordinates are screened and the supplementary coordinates beyond the edge of the grayscale image are removed;

[0145] like Figure 7 As shown, the supplementary coordinates within the edge range of the grayscale image are used with a shrinking algorithm to remove the influence of the burrs on the final detection effect of the optical pass sheet (such as Fig.16 As shown in the figure), obtain the indented coordinates after indentation (as shown in the figure). Figure 8 Calculate the indentation coordinates The formula is:

[0146]

[0147] in

[0148]

[0149]

[0150]

[0151] In the formula are supplementary coordinates, is the indentation coordinate, L The retraction distance is preferably 3, 4, 5 or 6 pixels, but it can be set according to the actual situation. , is a vector consisting of a supplementary coordinate and two adjacent supplementary coordinates. for The modulus of a vector, for The magnitude of a vector, is the angle between the two sides of the supplementary coordinates, Representation vector exist x The direction component, Representation vector exist y The direction of the component, Representation vector exist x The direction of the component, Representation vector exist y The direction component.

[0152] 4. Get the mask area and process it:

[0153] In order to avoid the detection area with tilt angle and the non-optical pass area affecting the detection efficiency, it is necessary to obtain the grayscale image mask area without tilt angle, create a black image with the same resolution as the grayscale image, draw the simulated optical pass on the black image with the indented coordinates as the vertex, and fill the indented coordinate area of ​​the black image with white, that is, fill the simulated optical pass with white (pixel value is 255), such as Fig. 9 As shown, the polygonal circumscribed rectangle algorithm is used to obtain the four vertices of the circumscribed rectangle of the indented coordinates of the black image. The circumscribed rectangle is a rectangle without an inclination angle, as shown in the following formula:

[0154] like Fig.17As shown, the bounding rectangle is obtained by finding the maximum or minimum coordinate value in each direction of the quadrilateral. 0, 1, 2, and 3 are the four vertices of the indented coordinates. The formula for obtaining the boundary is:

[0155]

[0156] In the formula left is the left edge of the bounding rectangle, right is the right edge of the bounding rectangle, top The upper boundary of the bounding rectangle bottom is the lower boundary of the circumscribed rectangle, The smallest of all coordinates in the indented coordinate area x Coordinate values, The smallest of all coordinates in the indented coordinate area y Coordinate values, The largest of all coordinates in the indented coordinate area x Coordinate values, The largest of all coordinates in the indented coordinate area y Coordinate value;

[0157] The bounding rectangle is Fig. 9 The black image mask area shown is a non-tilt angle area. The pixels in the black image mask area are traversed, and the traversed black pixels are accumulated. If the ratio of the sum of the traversed black pixels to the sum of the total pixels in the black image mask area is less than the fifth set value, the mask area is judged to be qualified. Preferably, the fifth set value is 0.1, 0.2, 0.3, 0.4 or 0.5, etc., but can be set according to actual needs;

[0158] When preprocessing grayscale images, the light source will affect image areas at different angles. Images in the middle direct range will show higher brightness, while images in the edge oblique range will show lower brightness. Due to the uneven brightness, rough binarization processing can easily lead to misjudgment of image defects. Therefore, it is necessary to divide the image into different local areas, and the illumination angle of each local area is almost the same to minimize the uneven brightness problem.

[0159] like Fig.10 As shown, the non-mask area of ​​the grayscale image is the non-optical pass area. The grayscale image mask area is intercepted for detection to avoid the non-optical pass area affecting the detection efficiency. The existing integral threshold algorithm is used for the grayscale image mask area. First, the grayscale image mask area is Gaussian blurred, and the Gaussian blur standard deviation value sigma is set. The boundary extension algorithm (the algorithm provided by Opencv) is used to copy the grayscale image mask area boundary;

[0160] Calculate the pixel sum of the integral kernel in the integral image: Fig.18As shown, the pixel sum of the integral kernel in the integral image is calculated by the relative coordinate origin position of the four vertices in the grayscale mask area; the anti-boundary mode of OpenCV is used to expand the image area boundary. sum , sqsum Respectively represent the integral area (i.e., integral map) of the image after boundary extension and the integral square area (i.e., square integral map) of the image after boundary extension, and obtain the cumulative sum of the pixels in the area enclosed by the four vertices of the integral area and the origin of the background image coordinates, as shown in the following formula:

[0161]

[0162]

[0163] In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral area and the origin of the coordinates. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integration area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower right corner of the integration area and the coordinate origin. i The forward direction of the X direction of the integration area i The sum of pixels, j The Y direction forward of the integration area j The sum of pixels, ker is the integration kernel size, is the pixel sum;

[0164] Similarly, calculate the cumulative sum of pixels in the area enclosed by the four vertices of the integral square area and the origin of the background coordinate system, as follows:

[0165]

[0166]

[0167] In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral square area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower right corner of the integral square area and the coordinate origin. i The forward direction of the X direction of the integral square area i The sum of pixels, j The Y direction forward of the integral square area j The sum of pixels, ker is the integration kernel size, is the sum of square pixels;

[0168] By integrating the pixels in the area and obtaining the pixel mean , as follows:

[0169]

[0170] And get the pixel variance var , as follows:

[0171]

[0172] Calculate the integral binarization threshold as follows:

[0173]

[0174] The grayscale image mask area is binarized pixel by pixel through the integral binarization threshold to obtain the mask integral binary image, as shown in the following formula:

[0175]

[0176] in is the grayscale mask area The pixel value of the coordinate point, is the integral binarization threshold calculated for the corresponding pixel value, is the pixel value after binarization;

[0177] When performing integral binarization, the pixel sum, square pixel sum, pixel mean and pixel variance within the integral kernel are calculated. The integral map and integral square map are used to obtain information of multiple pixels in one calculation, speed up the calculation of local statistical information, improve performance, avoid repeated calculations, reduce calculation complexity, and use accumulated pixel values ​​to enhance accuracy.

[0178] Use the connected domain algorithm to calculate the area of ​​the connected domain in the mask integral binary graph, such as Fig.11As shown in the figure, the extremely small features (i.e., the extremely small connected domain) in the mask integral binary image that do not affect the quality of the optical pass film (i.e., the area of ​​the connected domain is less than the first set area) are removed (i.e., impurities are removed), and the pixels in the mask integral binary image except the indented coordinate area are set to 0 (i.e., interference factors are removed) to avoid the burrs of the optical pass film and the non-optical pass film area in the mask area affecting the detection; the Sobel operator is used to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays the pixel intensity change rate; the parameter threshold is used for the mask gradient image The mask gradient map is binarized by the parameter threshold method to divide the mask gradient map into the foreground and the background, that is, the mask parameter contour map is obtained; the mask gradient map is binarized by the parameter threshold method to divide the mask gradient map into the foreground and the background, that is, the mask parameter contour map is obtained; the mask parameter contour map is binarized by the adaptive threshold method to divide the mask parameter contour map into the edge area and the non-edge area, and the mask edge contour map is obtained; the total gradient of the mask edge contour map is calculated again by the Sobel operator and binarized by the parameter threshold method (i.e., preprocessing) to obtain the mask edge map;

[0179] 5. Obtain defect information and mark it:

[0180] Using a connected domain algorithm to obtain the connected domain information inside the mask edge map, the connected domain information inside the mask edge map includes the area, number, width and height of the connected domain inside the mask edge map, and judging the defects of the mask edge map (i.e. judging the surface defects of the optical pass sheet according to the set conditions) by comparing the connected domain information with the set conditions (i.e. setting parameters);

[0181] Among them, larger area defects such as point damage are obtained by comparing the connected domain area obtained by mask edge map detection with the area calculated by the sixth set diameter:

[0182]

[0183] In the formula, Indicates i Is there any point damage in the mask edge image? =1 indicates that there is a point injury. =0 means there is no point damage; For the i The first j The area of ​​a connected domain, The sixth set diameter is preferably 5, 6, 7, 8 or 9 pixels, but can be set according to actual needs. n For the i The number of connected domains in the mask edge graph;

[0184] Among them, the defects with more spots such as pits are detected by comparing the number of connected domains obtained by the mask edge map detection with the seventh set number, and the number of connected domains belonging to all defects is counted:

[0185] In the formula, Indicates i There is no pitting defect in the mask edge image. Indicates i The mask edge image has pitting defects. N Set the number of pits for the seventh, n For the i The number of connected domains in the mask edge graph, preferably, the seventh set number of pits can be 2, 3, 4 or 5 pits;

[0186] Among them, the detection of relatively long and thin defects such as scratches is determined by the ratio of the connected domain width to the connected domain height and the maximum edge value of the connected domain. The first step is to determine whether it is a long and thin connected domain, based on whether the ratio of the connected domain width to the height is greater than the eighth set value or less than the ninth set value, wherein the eighth set value is greater than the ninth set value. If the ratio of the mask edge map connected domain width to the height is greater than the eighth set value or less than the ninth set value, it is a long and thin connected domain, as shown in the following formula:

[0187]

[0188] In the formula, For the i The mask edge map j The width of the connected domain, No. i The mask edge map j The height of the connected domain, n For the i The number of connected domains in the mask edge graph, is the eighth setting value, is the ninth setting value, , indicating that the connected domain does not belong to the elongated region. When , it indicates that the connected domain belongs to a slender area. Preferably, the ninth setting value is 2 / 17, 1 / 16, 1 / 15, 2 / 15, 1 / 14 or 2 / 13, and the eighth setting value is 12, 13, 14, 15, 16 or 17, but it can be set according to actual needs;

[0189] Step 2: Determine whether the maximum edge value of the connected domain is greater than the tenth set value (the larger value of the width and height of the slender defect or scratch is the maximum edge value of the connected domain):

[0190]

[0191] In the formula is the maximum edge value of the connected domain, is the tenth setting value, When , it means that the connected domain does not belong to the scratch defect. When , it indicates that the connected domain belongs to a scratch defect. Preferably, the tenth setting value is 13, 14, 15, 16 or 17 pixels.

[0192] After counting all the mask edge images that are defects (spots, pits, scratches, etc.), record the center coordinates of the mask edge image, and mark the corresponding coordinate points in the original image to complete the optical film defect detection.

[0193] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on these embodiments, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field can still combine, add, delete or make other adjustments to the features in the various embodiments of the present invention according to the circumstances without conflict, without making creative work, so as to obtain different other technical solutions that do not deviate from the concept of the present invention in essence, and these technical solutions also belong to the scope of protection of the present invention.

Claims

1. A method for detecting defects in optical wafers, It is characterized in that The following steps are involved: Step 1: Obtain the original image with the optical pass film through an image acquisition device, and copy the original image into a grayscale image; Step 2: Preprocess the grayscale image, obtain the coordinates of the contour vertices through contour detection and angle cosine value screening, and calculate the coordinates of the contour center, contour width, contour height and contour inclination angle; specifically: use Gaussian blur processing on the grayscale image to reduce the noise of the grayscale image and make the grayscale image smooth, in preparation for feature extraction; use the Sobel operator to calculate the total gradient of the grayscale image and generate the total gradient map of the grayscale image, so that the grayscale image only displays the pixel intensity change rate, and then use the parameter threshold method to binarize the total gradient map of the grayscale image, and divide the total gradient map of the grayscale image into foreground and background, that is, obtain the parameter contour map; then use the OpenCV adaptive threshold method to binarize the parameter contour map, divide the parameter contour map into edge area and non-edge area, so that the contour in the parameter contour map becomes a single pixel display, that is, obtain the edge contour map; calculate the area of ​​the connected domain detected in the edge contour map and compare it with the first set area, and delete the area in the edge contour map where the connected domain area is smaller than the first set area; use the Sobel operator to calculate Calculate the total gradient of the edge contour map and perform binarization through the parameter threshold method to obtain the parameter edge contour map; use the edge search algorithm to obtain the edge of the connected area in the parameter edge contour map, perform polygonal approximation on the edge of the connected area, and obtain the connected area contour; remove the connected area contours except the convex quadrilateral contour in the connected area contour, calculate the cosine value of the angle between the vertex coordinates of the convex quadrilateral contour, select the maximum value of the cosine value of the angle formed by the vertex coordinates of the convex quadrilateral contour and compare it with the second set value, and if it is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle, screen out the convex quadrilateral contour with a shape close to a rectangle, and use the circumscribed inclined rectangle algorithm to calculate the vertices of the circumscribed inclined rectangle of the convex quadrilateral contour, that is, the coordinates of the contour vertices; calculate the contour center coordinates through the contour vertex coordinates, compare the contour vertex coordinates with the contour center coordinates, and rearrange them clockwise; perform inverse trigonometric function calculation on the angle between the first two points sorted by the contour vertex coordinates and the X-coordinate axis to obtain the contour inclination angle, and calculate the contour width and contour height; Step 3: Use Blob detection to obtain the center coordinates of the Blob in the grayscale image, compare the center coordinates of the Blob with the center coordinates of the contour, combine the center coordinates of the Blob with the contour information, supplement the coordinates of the contour vertices, obtain the supplementary coordinates, and use the shrinkage algorithm on the supplementary coordinates to obtain the shrinkage coordinates; specifically: the Blob detector sets the detection style so that the Blob detector only detects quadrilateral style patches; the Blob detector sets the detection area so that the Blob detector screens out patches outside the third set area range; Use the configured Blob detector to directly detect the grayscale image and obtain the Blob center coordinates; calculate the distance between the Blob center coordinates and the contour center coordinates using the Euclidean distance formula dis ,when dis When it is less than or equal to the fourth setting value, the Blob center coordinate and the contour center coordinate are the center point of the same optical pass sheet, and no supplement is required; when dis When it is greater than the fourth set value, it is considered that the Blob center coordinates are inconsistent with the contour center coordinates. At this time, the light pass sheet obtained by the Blob detection needs to be supplemented to the light pass sheet obtained by the contour detection to obtain supplementary coordinates; to prevent the light pass sheet at the edge of the image from being misdetected, the supplementary coordinates beyond the edge of the grayscale image are removed; and the supplementary coordinates within the edge of the grayscale image are polygonally shrunk using the shrinkage algorithm to obtain the shrunk coordinates; Step 4: Create a black image with the same resolution as the grayscale image, use the polygonal circumscribed rectangle algorithm for the indented coordinates of the black image to obtain the circumscribed rectangular area of ​​the indented coordinates, i.e., the black image mask area, perform integral binarization and preprocessing on the grayscale image mask area with the same coordinates as the black image mask area, and generate a mask edge map; Specifically, a black image with the same resolution as the grayscale image is created, a simulated light pass film is drawn on the black image with the indented coordinates as vertices, and the indented coordinate area of ​​the black image is filled with white; a polygonal circumscribed rectangle algorithm is used for the indented coordinates to obtain four vertices of the circumscribed rectangle of the indented coordinates, and the circumscribed rectangular area surrounded by the four vertices is the black image mask area, and the pixels in the black image mask area are traversed. If the ratio of the sum of black pixels in the black image mask area to the sum of total pixels is less than the fifth set value, the black image mask area is judged to be qualified; For the grayscale mask area with the same coordinates as the black image mask area, the calculated integral image and square integral image are used to calculate the pixel sum, square pixel sum, pixel mean and pixel variance of all positions, obtain the information of multiple pixels in one calculation, and use the accumulated pixel values ​​for calculation to complete the integral binarization to obtain the mask integral binary image; the connected domain algorithm is used to obtain the area of ​​each connected domain for the mask integral binary image, and the connected domain area smaller than the first set area is the minimum connected domain and the minimum connected domain is removed, that is, impurities are removed; the pixels in the mask integral binary image except the indented coordinate area are set to 0, that is, interference factors are removed; the Sobel operator is used to obtain the total gradient of the mask integral binary image and generate a mask gradient image, so that the mask integral binary image only displays the pixel intensity change rate; The mask gradient map is binarized using the parameter threshold method, and the mask gradient map is divided into the foreground and the background, that is, the mask parameter contour map is obtained; the mask parameter contour map is binarized using the adaptive threshold method, and the mask parameter contour map is divided into the edge area and the non-edge area, and the mask edge contour map is obtained; the total gradient of the mask edge contour map is calculated again using the Sobel operator and binarized using the parameter threshold method to obtain the mask edge map; Step 5: Perform defect detection on the mask edge map to obtain defect information, and then mark the defects in the original image; specifically: use the connected domain algorithm on the mask edge map to obtain the internal connected domain information of the mask edge map. If the connected domain information exceeds the corresponding set parameters, it is determined that there is a defect in the mask edge map, and the coordinates of the defective mask edge map, that is, the defect coordinates, are recorded, and marked in the original image according to the defect coordinates.

2. The optical wafer defect detection method according to claim 1, It is characterized in that Step 2 also includes the following steps: According to the approximation algorithm, polygonal approximation is performed on the edge of the connected domain to obtain the connected domain contour, and the connected domain contour except the convex quadrilateral contour is removed from the connected domain contour, and the coordinates of the four vertices surrounding the convex quadrilateral contour are calculated, that is, the coordinates of the four contour vertices of the convex quadrilateral are obtained; The cosine values ​​of the angles between the vertices of the convex quadrilateral outline are calculated, and the cosine values ​​of the angles between the two vectors formed by three adjacent points are calculated and the maximum value is selected to determine whether the shape of the convex quadrilateral is close to a rectangle. The specific formula is as follows: In the formula For a convex quadrilateral i The vertex coordinates, For a convex quadrilateral i Vertex x coordinate, For a convex quadrilateral i Vertex y coordinate; For a convex quadrilateral i +1 vertex coordinates, For a convex quadrilateral i+ 1 vertex x coordinate, For a convex quadrilateral i+ 1 vertex y coordinate; , For a convex quadrilateral i +2 vertex coordinates, For a convex quadrilateral i+ 2 vertices x coordinate, For a convex quadrilateral i+ 2 vertices y Coordinates; Vectors For point Pointing Point vector, vector For point Pointing Point The vector of For vector and vector The angle between two vectors, is the maximum cosine value calculated, point With point for the same point; like If the value of is less than the second set value, it is considered that the shape of the convex quadrilateral is close to a rectangle, and the contour of the convex quadrilateral is the contour of the light pass sheet, which meets the requirements.

3. The optical fiber defect detection method according to claim 2, It is characterized in that The contour inclination angle is calculated according to the contour vertex coordinates, as follows: In the formula, is the profile inclination angle, is the number of contours, For the i The coordinates of the first point of the contour, For the i The first point of the contour x coordinate, For the i The first point of the contour y coordinate; For the i The coordinates of the second point of the contour, For the i The second point of the contour x coordinate, For the i The second point of the contour y coordinate; The contour width and height are calculated according to the contour vertex coordinates using the Euclidean distance formula, as shown below: In the formula, is the outline width, is the profile height, is the number of contours, For the i The coordinates of the first point of the light channel contour, For the i The first point of the contour x coordinate, For the i The first point of the contour y coordinate; For the i The coordinates of the second point of the light channel contour, For the i The second point of the contour x coordinate, For the i The second point of the contour y coordinate; For the i The coordinates of the third point of the light channel contour, For the i The third point of the contour x coordinate, For the i The third point of the contour y coordinate.

4. The optical wafer defect detection method according to claim 2, It is characterized in that The steps of obtaining the coordinates of the four contour vertices in step 2 include: The cosine value of the angle is calculated using three adjacent points of the detected vertices of the convex quadrilateral outline, and the maximum cosine value of the three or four corners of the convex quadrilateral outline is compared with the second set value. If the maximum cosine value of the convex quadrilateral outline is less than the second set value, it is judged that the shape of the convex quadrilateral is close to a rectangle; the inclination angle of each convex quadrilateral outline is calculated by using the inverse trigonometric function of the upper left corner vertex and the upper right corner vertex of each convex quadrilateral outline, the sum of the inclination angles of all convex quadrilateral outlines is calculated, and then the average value is calculated.

5. The optical wafer defect detection method according to claim 1, It is characterized in that In step 3, the light pass obtained by Blob detection is added to the light pass obtained by contour detection. Specifically, the center coordinates of the Blob are used as the center of the rectangle, and the contour tilt angle is used as the center of the rectangle. is the rectangular tilt angle, and the outline width is the rectangle width, and the outline height Draw a Blob rectangle for the rectangle height and get the coordinates of the Blob rectangle vertices; Sort the Blob rectangle vertex coordinates clockwise, and add the sorted Blob rectangle vertex coordinates to the contour vertex coordinate points to obtain the supplementary coordinates; The supplementary coordinates within the edge of the grayscale image are indented to obtain the indented coordinates, as shown in the following formula: in In the formula are supplementary coordinates, is the indentation coordinate, L is the inward distance, , is a vector consisting of a supplementary coordinate and two adjacent supplementary coordinates. for The modulus of a vector, for The magnitude of a vector, is the angle between the two sides of the supplementary coordinates, Representation vector exist x The direction component, Representation vector exist y The direction of the component, Representation vector exist x The direction of the component, Representation vector exist y Direction component.

6. The optical wafer defect detection method according to claim 1, Features: The polygonal circumscribed rectangle algorithm is used to obtain the four vertices of the circumscribed rectangle of the indented coordinates of the black image, as shown below: In the formula left is the left edge of the bounding rectangle, right is the right edge of the bounding rectangle, top The upper boundary of the bounding rectangle bottom is the lower boundary of the circumscribed rectangle, The smallest of all coordinates in the indented coordinate area x Coordinate values, The smallest of all coordinates in the indented coordinate area y Coordinate values, The largest of all coordinates in the indented coordinate area x Coordinate values, The largest of all coordinates in the indented coordinate area y Coordinate value; The black image mask area is a black image mask area without an inclination angle, pixels in the black image mask area are traversed, the traversed black pixels are accumulated, and a ratio of the sum of black pixels in the black image mask area to the sum of total pixels is obtained. If the ratio is less than a fifth set value, the mask area is judged to be qualified; The grayscale image mask area image processing uses an integral threshold algorithm to perform Gaussian blur processing on the grayscale image mask area, sets the Gaussian blur standard deviation value sigma, and uses the boundary extension algorithm to copy the grayscale image mask area boundary; Calculate the pixel sum of the integral kernel in the integral image: use the anti-border mode to expand the grayscale mask area boundary, sum represents the integral area of ​​the image after boundary extension, sqsum Represents the integral square area of ​​the image after boundary expansion, and obtains the cumulative sum of the pixels in the area enclosed by the four vertices of the integral area and the origin of the background image coordinates, as shown in the following formula: In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral area and the origin of the coordinate system. 、 and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral area and the coordinate origin. 、 and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral area and the coordinate origin. 、 and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower right corner of the integration area and the origin of the coordinates. i The forward direction of the X direction of the integration area i The sum of pixels, j The Y direction forward of the integration area j The sum of pixels, ker is the integration kernel size, is the pixel sum; Similarly, calculate the cumulative sum of pixels in the area enclosed by the four vertices of the integral square area and the origin of the background coordinate system, as follows: In the formula, and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper left corner of the integral square area and the coordinate origin. and Both represent the sum of the pixels in the area enclosed by the coordinates of the upper right corner of the integral square area and the origin of the coordinate system. and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower left corner of the integral square area and the coordinate origin. 、 and Both represent the sum of the pixels in the area enclosed by the coordinates of the lower right corner of the integral square area and the coordinate origin. i The forward direction of the X direction of the integral square area i The sum of pixels, j The Y direction forward of the integral square area j The sum of pixels, ker is the integration kernel size, is the sum of square pixels; By integrating the pixels in the area and obtaining the pixel mean , as follows: And get the pixel variance var , as follows: Calculate the integral binarization threshold th , as follows: The grayscale image mask area is binarized pixel by pixel through the integral binarization threshold to obtain the mask integral binary image, as shown in the following formula: in: is the grayscale mask area The pixel value of the coordinate point, is the integral binarization threshold calculated for the corresponding pixel value, is the pixel value after binarization; The removing of the extremely small connected domain whose area is smaller than the first set area is to set the pixels in the extremely small connected domain whose area is smaller than the first set area to 0.

7. The optical wafer defect detection method according to claim 6, It is characterized in that The resolution of the black image is the same as that of the grayscale image, that is, the coordinates of the grayscale image mask area and the black image mask area are the same. An integral binarization threshold calculation is performed on the grayscale image mask area with no tilt angle having the same coordinates as the black image mask area with no tilt angle. When calculating the pixel variance, a value greater than 0 is taken. If the pixel variance is less than 0, the pixel variance is taken as 0. During binarization, the calculated integral binarization threshold is used to binarize the grayscale image mask area pixel by pixel.

8. The optical wafer defect detection method according to claim 1, Features: Defect detection on the mask edge image and subsequent marking of defects in the original image include the following steps: 5.

1. Using a connected domain algorithm to obtain the connected domain information inside the mask edge map, the connected domain information inside the mask edge map includes the area, number, width and height of the connected domain inside the mask edge map; 5.

2. Point defect detection: compare the area of ​​the connected domain obtained by the mask edge map detection with the area calculated by the sixth set diameter. When the area of ​​the connected domain is larger than the area calculated by the sixth set diameter, it is considered that there is a point defect in the mask edge map; 5.

3. Pockmark defect detection: compare the number of connected domains detected by the mask edge map with the set number. If the number of connected domains is greater than the seventh set number of pockmarks, it is considered that the mask edge map has a pockmark defect; 5.

4. Slender defect detection, step 1: whether the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, wherein the eighth set value is greater than the ninth set value, if the ratio of the width to the height of the connected domain of the mask edge map is greater than the eighth set value or less than the ninth set value, it is a slender connected domain, step 2: according to the comparison between the maximum edge value of the connected domain and the tenth set value, when the maximum edge value of the connected domain is greater than the tenth set value, it is considered that the mask edge map has a slender defect; the larger value of the width and height of the connected domain is the maximum edge value of the connected domain; 5.

5. After detection, store the center coordinates of all mask edge images that are judged to be defects, and mark the corresponding coordinate points in the original image.

Citation Information

Patent Citations

  • Edge defect detection method and system for glass panel

    CN105405142A

  • Intelligent terminal panel visual fault detection system

    CN110070524A

  • Method and device for determining operation boundary of unmanned aerial vehicle

    CN111221934A

  • Metal closed water pump impeller appearance defect detection method

    CN113298776A

  • Textile spinning cake oil stain defect detection method based on graph centroid tracking algorithm

    CN114494226A